curl.cr

high level curl library for crystal

curl.cr Build Status

High level curl library for Crystal. This is a handy wrapper for blocknotes's curl-crystal

Installation

  1. Add the dependency to your shard.yml:
dependencies:
  curl:
    github: maiha/curl.cr
    version: 0.4.7
  1. Run shards install

Usage

libcurl provides two interfaces those are Easy and Multi. Easy Interface is standard api that provides synchronous access, and well used by many applications.

Easy Interface

  • Curl::Easy#get returns Curl::Easy::Response.
  • Curl::Easy::Response#response returns HTTP::Client::Response.
require "curl"

curl = Curl::Easy.new("http://examples.com")
res = curl.get
res.code         # => 200
res.content_type # => "text/html; charset=UTF-8"
res.headers      # => HTTP::Headers{"content-type" => "text/html", ...
res.body         # => "<html>..."
curl.to_s        # => "200 http://examples.com"

curl = Curl::Easy.new
curl.content_type = "application/json"
res = curl.post("http://examples.com", body: "[1,2,3]")

See src/curl/easy.cr for all variables.

  var uri      : URI
  var logger   : Logger
  var response : Response
  var info     : Info
  var body     : String

  # behavior
  var dump_header = false # Pass headers to the data stream
  var verbose     = false # Set verbose mode
  
  var timeout         : Time::Span
  var connect_timeout : Time::Span

  var compress : Bool     = false
  var encoding : Encoding = Encoding::ALL
  var decoding : Bool     = true

Curl::Easy::Info

Curl::Easy::Response#info returns Curl::Easy::Info.

info = curl.get.info
info.namelookup_time # => 0.00439
info.to_s            # => "[10.0KB](232KB/s, 0.0s)"
puts res.info.times_overview

(0.004s) |--NAMELOOKUP
(0.018s) |--|--CONNECT
(0.000s) |--|--|--APPCONNECT
(0.018s) |--|--|--|--PRETRANSFER
(0.032s) |--|--|--|--|--STARTTRANSFER
(0.032s) |--|--|--|--|--|--TOTAL
(0.000s) |--|--|--|--|--|--REDIRECT

Write file

By specifying the output file path with the output method, you can write it directly to a file instead of in-memory.

curl = Curl::Easy.new("http://examples.com")
curl.output = "index.html"

Compress

Compression is enabled by default, but you can use three variables to control fine behavior. For example, it is possible to acquire route compressed gzip data as it is by requesting encoding by gzip compression and skipping decoding, as follows.

curl.compress = true
curl.encoding = Curl::Easy::Encoding::GZIP
curl.decoding = false

Multi Interface

Execution of requests by Easy is blocked at the libcurl level, so even if executed within spawn, execution of crystal is blocked there.

Multi Interface provides asynchronous access. It enables multiple simultaneous transfers in the same thread without making forks.

multi = Curl::Multi.new
multi << Curl::Easy.new("https://example.com")
multi << Curl::Easy.new("https://github.com")
multi.run(timeout: 10.seconds)

multi.map(&.code)       # => [200, 200]
multi.human_code_counts # => {"200" => 2}
multi.summary           # => "2 Requests (0.5s) {\"200\" => 2}"

Roadmap

  • Core
    • Auth
      • #basic_auth
      • #digest_auth
    • Callback
      • before_execute
      • after_execute
    • Compress
      • #compress=(v : Bool) # control 'Accept-Encoding' header
      • #encoding=(v : Easy::Encoding) # specify the value of encoding
      • #decoding=(v : Bool) # control automatic decompression
    • Info
      • times stats
    • Logging
      • #logger
    • Request
      • #content_type=(v : String)
      • #headers
    • Response
      • #headers
      • #code
      • #body
      • #io
      • #success?
    • Timeout
      • connect_timeout
      • timeout
    • URI
      • #uri=(v)
      • #port
    • Verbose
      • #verbose
  • Easy Interface : https://curl.haxx.se/libcurl/c/
    • #get
    • #post
    • #put
  • Multi Interface : https://curl.haxx.se/libcurl/c/libcurl-multi.html
    • #<<(easy : Easy)
    • #run(timeout)
    • #requests : Array(Easy)
    • #responses : Array(Multi::Response)

Sample

samples/crurl.cr is an example using libcurl to make a curl compatible application in Crystal.

$ make build

Development

curl is a very multifunctional and large library, and the functionality implemented in this library is very small. If there is a missing function, please implement it. PR is greatly appreciated.

$ make spec

Here, libcurl.* is needed. For ubuntu, it can be installed by following packages.

  • libcurl4-openssl-dev
  • libidn2-0-dev

Constants

All constants in libcurl are mapped to Curl::XXX. For example, CURLOPT_VERBOSE is declared as enum. So it can be used as follows.

Curl::CURLOPT_VERBOSE.value # => 41

See doc/const.md for further details.

Symbols

All symbols are extracted to LibCurlSymbols from https://github.com/curl/curl/blob/master/docs/libcurl/symbols-in-versions.

LibCurlSymbols.each do |symbol|
  symbol # => LibCurlSymbols::CurlSymbol(@name="CURLAUTH_ANY", @introduced="7.10.6", @deprecated="", @removed="")

Automatically generated files

$ make gen

Docker

Rebuild docker containers if you modified docker/*.

$ make rebuild-docker

Static link

Minimum libcurl.a will be prepared as follows.

$ make libcurl.a
...
[libcurl.a]
  curl version:     7.64.0-DEV
  Protocols:        HTTP HTTPS
   LIBS:            -lidn2 -lssl -lcrypto -lssl -lcrypto -lz

Then you can compile your apps by adding --link-flags "-static $PWD/libcurl.a".

$ crystal build app.cr --link-flags "-static $PWD/libcurl.a"

If your environment does not have a dependent library it will result in an error. In that case, you can easily create it by using the container that created libcurl.a as follows.

$ docker-compose run --rm static crystal build app.cr --link-flags "-static /v/libcurl.a"

Contributing

  1. Fork it (https://github.com/maiha/curl.cr/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

  • maiha - creator and maintainer for this repository.
  • blocknotes - author of LibCurl which is a function of lower layer
Repository

curl.cr

Owner
Statistic
  • 6
  • 0
  • 3
  • 4
  • 3
  • about 4 years ago
  • February 10, 2019
License

MIT License

Links
Synced at

Thu, 02 May 2024 20:38:19 GMT

Languages
Crystal 97.34% Makefile 1.52% Shell 0.86% Dockerfile 0.28%