1 Basic Disposable API and Concepts

9.1

Racket

1 Basic Disposable API and Concepts🔗ℹ

A disposable is a producer of values that allocates external resources when producing a value and provides a way to deallocate those resources when a produced value is no longer needed. A value produced by a disposable is called a disposable value.

Disposables are closely related to custodians, but differ in what sort of resources they manage and how those resources are reclaimed. Broadly speaking, resources requiring management can be divided into two categories:

System resources — resources acquired from a program’s execution environment such as memory, file descriptors, operating system threads, handles to other local processes, etc.
External resources — resources managed by arbitrary external systems that a program communicates with, such as test data in a database, network connections with graceful termination protocols, machines used to execute distributed data processing, temporary access credentials acquired from a remote key management service, etc.

These two kinds of resources have very different implications for programs that use them. Due to the distributed nature of external resources and the inherent unreliability of network communication, it is impossible to guarantee that a given external resource is released. However, system resources are very rarely impossible to release. Custodians are designed to mangage system resources and assume reliable reclamation is possible, placing several restrictions on how programs can use custodians as a result:

It must be possible to forcibly reclaim system resources during program termination as occurs when custodian-shutdown-all is called. This prevents reclaiming resources by communicating over a network because it’s impossible to guarantee successful distributed communication.
Placing a user-defined resource under the management of a custodian requires using the ffi/unsafe/custodian module. This is an unsafe API because custodian shutdown actions are executed in atomic mode, resulting in possible deadlocks if shutdown actions perform asynchronous IO.
Custodian shutdown callbacks normally should not strongly reference the values they’re meant to clean up, as these shutdown callbacks frequently double as finalizers that run when the garbage collector determines the value to finalize is no longer reachable. This defeats certain composition patterns for shutdown callbacks, particularly composition that uses closures to reference both composed callbacks and managed values.

These restrictions make managing external resources with custodians inappropriate. Instead, an external resource should be produced by a disposable resulting in a disposable value that can be safely managed in a distributed system.

All of the bindings documented in this section are provided by disposable with the exception of acquire!, which is provided by disposable/unsafe.

1.1 Data Definition

1.2 Consuming Disposable Values

1.3 Unsafe Allocation of Disposables

1.4 Monadic Composition of Disposables

1.5 Asynchronous Cleanup

1.6 Disposables with Associated Custodians

1.7 Memoization and Disposables

1.1 Data Definition🔗ℹ

Concretely, a disposable is implemented as a thunk that when called allocates a new disposable value and returns that value alongside another thunk that deallocates the external resources associated with that particular disposable value.

procedure
(disposable alloc dealloc) → disposable?
alloc : (-> any/c)
dealloc : (-> any/c void?)

Returns a disposable that allocates a disposable value by calling alloc and deallocates external resources associated with that value by calling dealloc on the allocated value. Both procedures are called with breaks disabled. For a more flexible but less convenient interface, see make-disposable.

procedure
(make-disposable proc) → disposable?
proc : (-> (values any/c (-> void?)))

Returns a disposable that is implemented with proc. The given proc should return two values: a newly allocated value for use by consumers of the disposable, and a thunk that can be used to deallocate any external resources created during allocation of the value. Both proc and the disposal thunk it returns are called with breaks disabled. For the common case where deallocation can be implemented with a function that takes the allocated value as input, prefer the disposable procedure.

procedure
(disposable? v) → boolean?
v : any/c

Returns #t if v is a disposable, returns #f otherwise.

procedure
(disposable/c c) → contract?
c : contract?

Returns a contract for a disposable that allocates values matching c.

1.2 Consuming Disposable Values🔗ℹ

Disposable values can be acquired in a variety of ways, ranging from unsafe low level access with acquire! to automated per-thread allocation with acquire-virtual.

syntax
(with-disposable ([id disp-expr] ...) body ...+)

disp-expr : disposable?

Allocates a disposable value with each disp-expr and binds each value to its corresponding id within the scope of the body expressions. The values are deallocated upon exiting the body expressions, even if exiting occurs with an exception or a continuation jump. Additionally, the body expressions are called with a continuation barrier to prevent jumping back into the expressions after deallocation. Deallocation of the id values occurs concurrently; deallocation of one value does not block on deallocation of any other values.

Examples:

> (with-disposable ([x example-disposable]
                    [y example-disposable])
    (- x y))
Allocated 86
Allocated 99
Deallocated 86
Deallocated 99
13
> (with-disposable ([n example-disposable])
    (error "uh oh!"))
Allocated 89
Deallocated 89
uh oh!

procedure
(call/disposable disp proc) → any
disp : disposable?
proc : (-> any/c any)

Allocates a disposable value with disp, calls proc with the value as an argument, deallocates the value, then returns the result of calling proc. This is essentially a non-macro form of with-disposable. Like with-disposable, the value is deallocated if control leaves proc due to an exception or a continuation jump, and a continuation barrier prevents jumping back into proc. To use with multiple disposables that are deallocated concurrently, use disposable-apply with list to transform multiple disposables into a single disposable that allocates a list of values.

Examples:

> (call/disposable example-disposable (λ (n) (* n n)))
Allocated 62
Deallocated 62
3844
> (call/disposable example-disposable (λ (_) (error "uh oh!")))
Allocated 18
Deallocated 18
uh oh!

procedure
(acquire disp [#:dispose-evt evt]) → any/c
disp : disposable?
evt : evt? = (thread-dead-evt (current-thread))

Allocates and returns a value with disp and launches a background thread that deallocates the value when evt is ready for synchronization. Using the default for evt causes the value to be deallocated when the calling thread dies. This is for when a disposable value is inherently tied to the lifteime of the thread using it, such as a connection used while handling a web server request in a servlet model where a new thread is spawned for each request. Other uses include alarm-evt to return a value that is deallocated after a timeout, or using a subprocess? value to deallocate after a subprocess terminates.

Example:

> (sync
   (thread
    (λ ()
      (define n (acquire example-disposable))
      (printf "Acquired ~v\n" n))))
Allocated 63
Acquired 63
Deallocated 63
#<thread>

procedure
(acquire-global disp [#:plumber plumber]) → any/c
disp : disposable?
plumber : plumber? = (current-plumber)

Allocates and returns a value with disp and attaches a flush callback to plumber that deallocates the value. This is intended for when a disposable value is used throughout the lifetime of a program, such as a global database connection pool. Globally allocated values can be safely provided by modules for use in other modules just like normal values, allowing the modular definition of global program resources. Note that plumbers expect that after all flush callbacks return the program may exit and kill any remaining threads, so a disposable that is acquired globally should not deallocate values in any asynchronous manner (such as in a background thread). In particular, do not use acquire-global with disposable/async-dealloc.

Examples:

> (define plumb (make-plumber))
> (define n (acquire-global example-disposable #:plumber plumb))
Allocated 50
> (add1 n)
51
> (plumber-flush-all plumb)
Deallocated 50

procedure
(acquire-virtual disp) → (-> any/c)
disp : disposable?

Returns a thunk that returns a virtual instance of disp, meaning an instance is allocated per-thread the first time a thread calls the thunk and returned in subsequent calls by the same thread. The returned thunk maintains a weak mapping of threads to allocated instances of disp, with instances deallocated whenever their associated threads die (in the same manner as acquire). This may be expensive in high-concurrency scenarios with short lived threads. To use with high concurrency, consider combining with disposable-pool to reuse instances between threads. In particular, see Global Pools with Virtual Leases for a common and convenient access pattern.

Examples:

> (define virtual-example (acquire-virtual example-disposable))
> (define (spawn)
(thread (λ () (printf "Acquired ~v\n" (virtual-example)))))
> (sync (spawn) (spawn) (spawn))
Allocated 53
Acquired 53
Allocated 6
Acquired 6
Allocated 96
Acquired 96
Deallocated 96
Deallocated 6
Deallocated 53
#<thread>

1.3 Unsafe Allocation of Disposables🔗ℹ

(require disposable/unsafe)

package: disposable

The disposable/unsafe module provides a single export, acquire!, which provides an unsafe building block upon which safe allocation abstractions can be built.

procedure
(acquire! disp) →
any/c (-> void?)
disp : disposable?

Returns a newly-allocated value with disp as well as a thunk that deallocates the value when called. This is unsafe, as the caller is responsible for ensuring that the deallocation thunk is called. Both the acquire! procedure and the disposal thunk it returns should be called with breaks disabled.

Examples:

> (define-values (n dispose!) (acquire! example-disposable))
Allocated 99
> (printf "Acquired ~v unsafely\n" n)
Acquired 99 unsafely
> (dispose!)
Deallocated 99

1.4 Monadic Composition of Disposables🔗ℹ

procedure
(disposable-pure v) → disposable?
v : any/c

Returns a pure disposable. The disposable always returns v when asked to allocate a value and takes no action upon deallocation. Useful with disposable-apply for combining a mix of disposable and plain values.

Example:

> (with-disposable ([n (disposable-pure 42)])
n)
42

procedure
(disposable-apply f disp ...) → disposable?
f : procedure?
disp : disposable?

Returns a disposable value that allocates a value from each disp, calls f with the allocated values, then returns the result of calling f as the allocated value. Deallocation of the value is performed by deallocating each of the source values produced. Deallocation occurs concurrently.

Examples:

> (struct posn (x y) #:transparent)
> (define disposable-posn
(disposable-apply posn example-disposable example-disposable))
> (with-disposable ([p disposable-posn])
(printf "Acquired ~v\n" p))
Allocated 42
Allocated 90
Acquired (posn 90 42)
Deallocated 42
Deallocated 90

procedure
(disposable-chain disp f) → disposable?
disp : disposable?
f : (-> any/c disposable?)

Returns a disposable that allocates a value from disp, calls f with the allocated value producing a new disposable, then returns a newly allocated value with the disposable returned by f. Deallocation of the value is performed by first deallocating using the disposable returned by f, then deallocating the source value produced by disp after finishing deallocating with the disposable produced by f.

Examples:

> (define (add-example x)
(disposable-apply (λ (y) (+ x y)) example-disposable))
> (define sum-disp (disposable-chain example-disposable add-example))
> (with-disposable ([v sum-disp])
(printf "Acquired ~v\n" v))
Allocated 54
Allocated 42
Acquired 96
Deallocated 42
Deallocated 54

1.5 Asynchronous Cleanup🔗ℹ

procedure
(disposable/async-dealloc disp) → disposable?
disp : disposable?

Returns a disposable that is like disp, but deallocation happens asynchronously in a background thread spawned when deallocation would normally occur. This is intended for disposables where immediately releasing the resource is not required. Note that this is not recommended for use with acquire-global, as it’s important that the plumber used by acquire-global does not finish flushing until all resources have been deallocated.

Examples:

> (with-disposable ([n (disposable/async-dealloc example-disposable)])
(printf "Acquired ~v\n" n))
Allocated 41
Acquired 41
Deallocated 41
> (sleep 0.1)

1.6 Disposables with Associated Custodians🔗ℹ

procedure
(disposable/custodian disp cust) → disposable?
disp : disposable?
cust : custodian?

Returns a disposable that is like disp, but allocation and deallocation occur with cust as the current custodian. Normally, disposables only manage external resources whereas custodians only manage system resources — use of disposable/custodian is reserved for when allocation and deallocation creates both system resources and external resources. This frequently occurs when allocation of a disposable involves creating ports to communicate with other parties, which must be closed by a custodian. In these circumstances disposable/custodian can be used to control which custodian manages all system resources created by a particular disposable.

Added in version 0.4 of package disposable.

1.7 Memoization and Disposables🔗ℹ

procedure
(disposable/memoize f
[ #:make-dict make-dict])
→ (disposable/c procedure?)
f : (unconstrained-domain-> disposable?)
make-dict : (-> (and/c dict? dict-mutable?)) = make-hash

Returns a disposable that produces a memoized version of f. Whereas f would normally return a disposable for any given call, the same call to the memoized f instead allocates and returns a value using the disposable that f returns. Furthermore, repeated calls to the memoized f with the same arguments reuse the disposable allocated for the first call. All disposables allocated by the memoized f are deallocated when the disposable returned by disposable/memoize deallocates the memoized f.

Examples:

> (define (color+ex-disp c)
    (disposable-apply list (disposable-pure c) example-disposable))
> (with-disposable ([color+ex (disposable/memoize color+ex-disp)])
    (displayln (color+ex "red"))
    (displayln (color+ex "blue"))
    (displayln (color+ex "red")))
Allocated 62
(red 62)
Allocated 51
(blue 51)
(red 62)
Deallocated 51
Deallocated 62

Reuse of values by the memoized f is achieved by mapping arguments (both positional and keyword) to allocated values in a mutable dictionary. Allocation of the memoized f creates that mutable dictionary using make-dict, which by default produces a mutable hash table that holds keys and values strongly.

Added in version 0.5 of package disposable.

1	Basic Disposable API and Concepts
2	Reusing Disposables with Pools
3	Transient Values
4	Batteries-Included Disposables

1.1	Data Definition
1.2	Consuming Disposable Values
1.3	Unsafe Allocation of Disposables
1.4	Monadic Composition of Disposables
1.5	Asynchronous Cleanup
1.6	Disposables with Associated Custodians
1.7	Memoization and Disposables