http-easy:   a high-level HTTP client
1 Guide
1.1 Making Requests
1.2 Streaming Responses
1.3 Authenticating Requests
1.4 Sending Data
1.5 Cookie Storage
2 Reference
get
post
delete
head
options
patch
put
2.1 Sessions
method/  c
headers/  c
form-data/  c
query-params/  c
current-session
session?
make-session
session-close!
session-request
2.2 Responses
status-code/  c
response?
response
response-status-line
response-http-version
response-status-code
response-status-message
response-headers
response-output
response-headers-ref
response-headers-ref*
response-history
response-body
response-json
response-xexpr
response-xml
read-response
read-response-json
read-response-xexpr
read-response-xml
response-drain!
response-close!
2.3 Connection Pooling
limit/  c
pool-config?
make-pool-config
2.4 Authentication
auth-procedure/  c
basic-auth
bearer-auth
2.5 Payload Procedures
payload-procedure/  c
form-payload
json-payload
gzip-payload
pure-payload
part?
field-part
file-part
multipart-payload
2.6 Timeouts
timeout/  c
timeout-config?
make-timeout-config
2.7 Errors
exn:  fail:  http-easy?
exn:  fail:  http-easy:  timeout?
exn:  fail:  http-easy:  timeout-kind
2.8 User Agents
current-user-agent
3 Changelog
3.1 HEAD
3.2 v0.1.1 – 2020/  08/  18
3.2.1 Changed
3.2.2 Fixed
3.3 v0.1.0 – 2020/  06/  13
7.8

http-easy: a high-level HTTP client

Bogdan Popa <bogdan@defn.io>

This library wraps net/http-client to provide a simple interface for day-to-day use. It automatically handles:

The following features are currently planned:

The following features may be supported in the future:

The API is currently in flux so be aware that it may change before the final release.

1 Guide

1.1 Making Requests

Getting started is as easy as requiring the net/http-easy module:

> (require net/http-easy)

And using one of the built-in requesters to perform a request:

> (define res
    (get "https://example.com"))

The result is a response? value that you can inspect:

> (response-status-code res)

200

> (response-status-message res)

#"OK"

> (response-headers-ref res 'date)

#"Sat, 13 Jun 2020 16:29:47 GMT"

> (subbytes (response-body res) 0 30)

#"<!doctype html>\n<html>\n<head>\n"

Connections to remote servers are automatically pooled so closing the response returns its underlying connection to the pool:

> (response-close! res)

If you forget to manually close a response, its underlying connection will get returned to the pool when the response gets garbage-collected. Unless you explicitly use Streaming Responses, you don’t have to worry about this much.

1.2 Streaming Responses

Response bodies can be streamed by passing #t as the #:stream? argument to any of the requesters:

> (define res
    (get "https://example.com" #:stream? #t))

The input port representing the response body can be accessed using response-output:

> (input-port? (response-output res))

#t

> (read-string 5 (response-output res))

"<!doc"

> (read-string 5 (response-output res))

"type "

1.3 Authenticating Requests

The library provides an auth procedure for HTTP basic auth:

> (response-status-line
   (get "https://httpbin.org/basic-auth/Aladdin/OpenSesame"))

#"HTTP/1.1 401 Unauthorized"

> (response-json
   (get "https://httpbin.org/basic-auth/Aladdin/OpenSesame"
        #:auth (basic-auth "Aladdin" "OpenSesame")))

'#hasheq((authenticated . #t) (user . "Aladdin"))

And for bearer auth:

> (response-json
   (get "https://httpbin.org/bearer"
        #:auth (bearer-auth "secret-api-key")))

'#hasheq((authenticated . #t) (token . "secret-api-key"))

The above example is equivalent to:

> (response-json
   (get "https://httpbin.org/bearer"
        #:auth (lambda (uri headers params)
                 (values (hash-set headers 'authorization "Bearer secret-api-key") params))))

'#hasheq((authenticated . #t) (token . "secret-api-key"))

1.4 Sending Data

You can supply a list of pairs to be sent as an application/x-www-form-urlencoded payload:

> (define res
   (response-json
    (post "https://httpbin.org/post"
          #:form '((a . "hello")
                   (b . "there")))))
> (hash-ref res 'form)

'#hasheq((a . "hello") (b . "there"))

Alternatively, you can supply the #:json keyword argument to send an application/json payload:

> (define res
   (response-json
    (post "https://httpbin.org/anything"
          #:json (hasheq 'a "hello"
                         'b "there"))))
> (hash-ref res 'json)

'#hasheq((a . "hello") (b . "there"))

To send data using arbitrary formats, you can use the #:data keyword argument:

> (define res
   (response-json
    (post "https://httpbin.org/anything"
          #:data #"hello")))
> (hash-ref res 'data)

"hello"

To gzip the payload, use the gzip-payload combinator:

> (define res
   (response-json
    (post "https://httpbin.org/anything"
          #:data (gzip-payload (pure-payload #"hello")))))
> (hash-ref res 'data)

"data:application/octet-stream;base64,H4sIAP/+5F4AA8tIzcnJBwCGphA2BQAAAA=="

> (define res
   (response-json
    (post "https://httpbin.org/anything"
          #:data (gzip-payload (json-payload (hasheq 'hello "world"))))))
> (hash-ref res 'data)

"data:application/octet-stream;base64,H4sIAP/+5F4AA6tWykjNyclXslIqzy/KSVGqBQDRQQnYEQAAAA=="

1.5 Cookie Storage

To store cookies between requests pass a cookie-jar<%> into your session?:

> (require net/cookies
           net/url
           racket/class)
>
> (define jar (new list-cookie-jar%))
> (define session-with-cookies
    (make-session #:cookie-jar jar))
>
> (parameterize ([current-session session-with-cookies])
    (get "https://httpbin.org/cookies/set/hello/world")
    (response-json (get "https://httpbin.org/cookies")))

'#hasheq((cookies . #hasheq((hello . "world"))))

>
> (for ([c (in-list (send jar cookies-matching (string->url "https://httpbin.org")))])
    (printf "~a: ~a" (ua-cookie-name c) (ua-cookie-value c)))

hello: world

2 Reference

procedure

(get uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(post uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(delete uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(head uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(options uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(patch uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)

procedure

(put uri    
  [#:close? close?    
  #:stream? stream?    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : query-params/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)
Requesters for each of the standard HTTP methods. See session-request for a description of the individual arguments.

2.1 Sessions

value

method/c

 : (or/c 'delete 'head 'get 'options 'patch 'post 'put symbol?)

value

headers/c : (hash/c symbol? (or/c bytes? string?))

value

form-data/c : (listof (cons/c symbol? (or/c false/c string?)))

value

query-params/c

 : (listof (cons/c symbol? (or/c false/c string?)))

parameter

(current-session)  session?

(current-session session)  void?
  session : session?
 = (make-session)
Holds the current session that is used by the delete, head, get, options, patch, post and put requesters.

procedure

(session? v)  boolean?

  v : any/c
Returns #t when v is a session value.

procedure

(make-session [#:pool-config pool-config    
  #:ssl-context ssl-context    
  #:cookie-jar cookie-jar])  session?
  pool-config : pool-config? = (make-pool-config)
  ssl-context : ssl-client-context?
   = (ssl-secure-client-context)
  cookie-jar : (or/c false/c (is-a?/c cookie-jar<%>)) = #f
Produces a session? value with pool-config as its connection pool configuration. Each requested scheme, host and port pair has its own connection pool.

The ssl-context argument controls how HTTPS connections are handled. The default implementation verifies TLS certificates, verifies hostnames and avoids using weak ciphers. To use a custom certificate chain or private key, you can use ssl-make-client-context.

The cookie-jar argument specifies the cookie jar to use to store cookies between requests made against a session. The default is to discard all cookies. See list-cookie-jar%.

procedure

(session-close! s)  void?

  s : session?
Closes s and all of its associated connections and responses.

procedure

(session-request s    
  uri    
  [#:close? close?    
  #:stream? stream?    
  #:method method    
  #:headers headers    
  #:params params    
  #:auth auth    
  #:data data    
  #:form form    
  #:json json    
  #:timeouts timeouts    
  #:max-attempts max-attempts    
  #:max-redirects max-redirects    
  #:user-agent user-agent])  response?
  s : session?
  uri : (or/c bytes? string? url?)
  close? : boolean? = #f
  stream? : boolean? = #f
  method : method/c = 'get
  headers : headers/c = (hasheq)
  params : query-params/c = null
  auth : (or/c false/c auth-procedure/c) = #f
  data : (or/c false/c bytes? string? input-port? payload-procedure/c)
   = #f
  form : form-data/c = unsupplied
  json : jsexpr? = unsupplied
  timeouts : timeout-config? = (make-timeout-config)
  max-attempts : exact-positive-integer? = 3
  max-redirects : exact-nonnegative-integer? = 16
  user-agent : (or/c bytes? string?) = (current-user-agent)
Requests uri using s’s connection pool and associated settings (SSL context, proxy, cookie jar, etc.).

Response values returned by this function must be closed before their underlying connection is returned to the pool. If the close? argument is #t, this is done automatically. Ditto if the responses are garbage-collected.

If the close? argument is #t, then the response’s output port is drained and the connection is closed.

If the stream? argument is #f (the default), then the response’s output port is drained and the resulting byte string is stored on the response value. The drained data is accessible using the response-body function. If the argument is #t, then the response body is streamed and the data is accessible via the response-output function. This argument has no effect if the close? is #t.

The method argument specifies the HTTP request method to use.

Query parameters may be specified directly on the uri argument or via the params argument. If query parameters are specified via both arguments, then the list of params is appended to those already in the uri.

The auth argument allows authentication headers and query params to be added to the request. When following redirects, the auth procedure is applied to subsequent requests only if the target URL has the same origin as the original request. Two URLs are considered to have the same origin if their scheme, hostname and port are the same.

The data argument can be used to send arbitrary request data to the remote end. A number of payload procedures are available for producing data in standard formats:

> (define res
    (post #:data (json-payload (hasheq 'hello "world"))
          "https://httpbin.org/post"))
> (hash-ref (response-json res) 'data)

"{\"hello\":\"world\"}"

The form argument is a shorthand for passing a form-payload as the data argument.

The json argument is a shorthand for passing a json-payload as the data argument.

The data, form and json arguments are mutually-exclusive. Supplying more than one at a time causes a contract error to be raised.

The timeouts argument controls how long various aspects of the request cycle will be waited on. When a timeout is exceeded, an exn:fail:http-easy:timeout? error is raised. When redirects are followed, the timeouts are per request.

The max-attempts argument controls how many times connection errors are retried. This meant to handle connection resets and the like and isn’t a general retry mechanism.

The max-redirects argument controls how many redirects are followed by the request. Redirect cycles are not detected. To disable redirect following, set this argument to 0. The Authorization header is stripped from redirect requests if the target URL does not have the same origin as the original request.

2.2 Responses

value

status-code/c : (integer-in 100 999)

The contract for HTTP status codes.

procedure

(response? v)  boolean?

  v : any/c
Returns #t when v is a response.

syntax

(response clause ...)

 
clause = #:status-line e
  | #:status-code e
  | #:status-message e
  | #:http-version e
  | #:history e
  | #:headers heads maybe-rest
  | #:body e
  | #:json e
     
heads = ([header-id e] ...)
     
maybe-rest = 
  | e
A match expander for response? values.

> (require racket/match)
>
> (match (get "https://example.com")
   [(response
     #:status-code 200
     #:headers ([content-type (and (regexp #"text/html") the-content-type)]))
    the-content-type])

#"text/html; charset=UTF-8"

procedure

(response-status-line r)  bytes?

  r : response?

procedure

(response-http-version r)  bytes?

  r : response?

procedure

(response-status-code r)  status-code/c

  r : response?

procedure

(response-status-message r)  bytes?

  r : response?

procedure

(response-headers r)  (listof bytes?)

  r : response?

procedure

(response-output r)  input-port?

  r : response?
Accessors for the raw data available on a response.

procedure

(response-headers-ref r h)  (or/c false/c bytes?)

  r : response?
  h : symbol?
Looks up the first response header whose name is h. Header names are normalized to lower case.

procedure

(response-headers-ref* r h)  (listof bytes?)

  r : response?
  h : symbol?
Looks up all the response headers whose names are h. As in response-headers-ref, the names are all normalized to lower case.

procedure

(response-history r)  (listof response?)

  r : response?
When redirects are followed, the trail of redirected responses is preserved in each individual response. The responses are sorted reverse chronologically.

procedure

(response-body r)  bytes?

  r : response?
Drains r’s output port and returns the result as a byte string.

procedure

(response-json r)  (or/c eof-object? jsexpr?)

  r : response?
Drains r’s output port, parses the data as JSON and returns it. An exception is raised if the data is not valid JSON.

procedure

(response-xexpr r)  xexpr?

  r : response?
Drains r’s output port, parses the data as an xexpr? and returns it. An exception is raised if the data is not valid XML.

procedure

(response-xml r)  document?

  r : response?
Drains r’s output port, parses the data as an XML document? and returns it. An exception is raised if the data is not valid XML.

procedure

(read-response r)  any/c

  r : response?
Equivalent to (read (response-output r)).

procedure

(read-response-json r)  (or/c eof-object? jsexpr?)

  r : response?
Equivalent to (read-json (response-output r)).

procedure

(read-response-xexpr r)  xexpr?

  r : response?

procedure

(read-response-xml r)  document?

  r : response?
Equivalent to (read-xml/document (response-output r)).

procedure

(response-drain! r)  void?

  r : response?
Drains r’s output port.

procedure

(response-close! r)  void?

  r : response?
Closes r and returns its underlying connection to the pool.

2.3 Connection Pooling

The contract for limit values.

procedure

(pool-config? v)  boolean?

  v : any/c
Returns #t when v is a pool config.

procedure

(make-pool-config [#:max-size max-size    
  #:idle-timeout idle-timeout])  pool-config?
  max-size : limit/c = 128
  idle-timeout : timeout/c = 600
Produce a pool config values that can be passed to make-session.

The max-size argument controls the maximum number of connections in a pool. Once a pool reaches this size, leasing a connection blocks until one is available or until the lease timeout is reached.

The idle-timeout argument controls the amount of time idle connections are kept open for.

2.4 Authentication

The contract for auth procedures. An auth procedure takes the current request url, headers and query params and returns a new set of headers and query params augmented with authentication information.

procedure

(basic-auth username password)  auth-procedure/c

  username : (or/c bytes? string?)
  password : (or/c bytes? string?)
Generates an auth procedure that authenticates requests using HTTP basic auth.

procedure

(bearer-auth token)  auth-procedure/c

  token : (or/c bytes? string?)
Generates an auth procedure that authenticates requests using the given bearer token.

2.5 Payload Procedures

Payload procedures produce data and associated headers to be sent to a remote server.

The contract for payload procedures. A payload procedure takes the current set of request headers and returns new request headers and a value to be used as the request body.

procedure

(form-payload v)  payload-procedure/c

  v : form-data/c
Produces a payload procedure that encodes v as form data using the application/x-www-form-urlencoded content type.

procedure

(json-payload v)  payload-procedure/c

  v : jsexpr?
Produces a payload procedure that encodes v as JSON data.

Produces a payload procedure that gzips the output of p.

Produces a payload procedure that uses v as the request body.

procedure

(part? v)  boolean?

  v : any/c
Returns #t when v is a multipart-payload part.

procedure

(field-part name value [content-type])  part?

  name : (or/c bytes? string?)
  value : (or/c bytes? string?)
  content-type : (or/c bytes? string?) = #"text/plain"
Produces a part? for use with multipart-payload that encapsulates a form field.

procedure

(file-part name inp [filename content-type])  part?

  name : (or/c bytes? string?)
  inp : input-port?
  filename : (or/c bytes? string?) = (~a (object-name inp))
  content-type : (or/c bytes? string?)
   = #"application/octet-stream"
Produces a part? for use with multipart-payload that encapsulates a file.

procedure

(multipart-payload f    
  ...    
  [#:boundary boundary])  payload-procedure/c
  f : part?
  boundary : (or/c bytes? string?) = unsupplied
Produces a multipart/form-data payload.

> (define resp
    (post
     #:data (multipart-payload
             (field-part "a" "hello")
             (file-part "f" (open-input-string "hello world!")))
     "https://httpbin.org/anything"))
> (hash-ref (response-json resp) 'form)

'#hasheq((a . "hello"))

> (hash-ref (response-json resp) 'files)

'#hasheq((f . "hello world!"))

2.6 Timeouts

The contract for timeout values. All timeout values represent seconds.

procedure

(timeout-config? v)  boolean?

  v : any/c
Returns #t when v is a timeout config value.

procedure

(make-timeout-config [#:lease lease    
  #:connect connect    
  #:request request])  timeout-config?
  lease : timeout/c = 5
  connect : timeout/c = 5
  request : timeout/c = 30
Produces a timeout config value that can be passed to session-request.

The lease argument controls the maximum amount of time leasing a connection from the connection pool can take.

The connect argument controls how long each connection can take to connect to the remote end.

The request argument controls how long to wait on a request before its response headers are returned.

2.7 Errors

procedure

(exn:fail:http-easy? v)  boolean?

  v : any/c

procedure

(exn:fail:http-easy:timeout? v)  boolean?

  v : any/c

procedure

(exn:fail:http-easy:timeout-kind e)

  (or/c 'lease 'connect 'request)
  e : exn:fail:http-easy:timeout?

2.8 User Agents

parameter

(current-user-agent)  (or/c bytes? string?)

(current-user-agent user-agent)  void?
  user-agent : (or/c bytes? string?)
Holds the value of the User-Agent header that is sent with every request.

3 Changelog

3.1 HEAD

3.2 v0.1.1 – 2020/08/18

3.2.1 Changed
3.2.2 Fixed

3.3 v0.1.0 – 2020/06/13

Initial release.