curl_cffi.cr

Crystal HTTP client that impersonates real browser TLS fingerprints, built on lexiforest/curl-impersonate. A Crystal port of the idea behind Python's curl_cffi.

Why

Sites behind Cloudflare, Akamai, and similar services fingerprint the TLS handshake and block ordinary HTTP clients regardless of the User-Agent they send. This client performs the handshake exactly like a real browser, so requests present a genuine Chrome, Firefox, or Safari fingerprint. Use it when a plain HTTP client gets blocked but a browser does not.

Installation

Add the shard to your shard.yml:

dependencies:
  curl_cffi:
    github: veelenga/curl_cffi.cr

This shard links against libcurl-impersonate, which is not bundled. The Homebrew formula installs only the command-line binary, so fetch the library build with the included script:

make lib

This downloads the prebuilt libcurl-impersonate for your platform from the lexiforest/curl-impersonate releases into vendor/. Point the toolchain at it when building and running:

export LIBRARY_PATH="$PWD/vendor:$LIBRARY_PATH"
export DYLD_FALLBACK_LIBRARY_PATH="$PWD/vendor:$DYLD_FALLBACK_LIBRARY_PATH" # macOS
export LD_LIBRARY_PATH="$PWD/vendor:$LD_LIBRARY_PATH"                       # Linux

To install it system-wide instead, copy the contents of vendor/ into a standard library directory such as /usr/local/lib.

Usage

Simple request with the default impersonation target:

require "curl_cffi"

response = CurlCffi.get("https://example.com")
response.status_code # => 200
response.body
response.json

Sessions persist cookies across requests:

session = CurlCffi::Session.new(impersonate: "chrome136")
session.get("https://example.com/login")
session.get("https://example.com/account")
session.cookies
session.close

Post JSON or form data:

CurlCffi.post("https://example.com/api", json: {"name" => "crystal"})
CurlCffi.post("https://example.com/form", form: {"user" => "me"})

Error handling:

begin
  CurlCffi.get("https://example.com", timeout: 5.seconds)
rescue CurlCffi::Timeout
  puts "request timed out"
end

Disable impersonation:

session = CurlCffi::Session.new(impersonate: nil)
session.get("https://example.com")

Impersonation targets

Pass an impersonate target name to choose which browser fingerprint to present. The default is chrome136. Supported targets track upstream and include recent Chrome, Edge, Safari, Firefox, and Tor builds (for example chrome142, safari184, firefox147, tor145). The full list for your installed version is CurlCffi::Browser::TARGETS.

Testing

crystal spec

Specs run against a local HTTP server, so no network access is required. The linker and runtime need libcurl-impersonate on the library path. The Makefile wires this up for you:

make lib   # once, downloads the library into vendor/
make test  # sets the library path and runs crystal spec

License

MIT. See LICENSE.