1.2 Concepts🔗ℹ

This library provides an array data structure that can run on CUDA devices. There are three key types of objects:

• buffer: one RMM allocation in GPU memory, with its device, byte size and phantom bytes. It owns the memory, and its finalizer frees it.

• array: a device vector or matrix, a typed and shaped view of a buffer (dtype, shape, strides, offset).

• resources: a raft::handle_t on one GPU, owning a CUDA stream (an ordered queue of GPU work) and the cuBLAS, cuSOLVER and cuSPARSE handles. Every array and every operation runs on resources, by default the thread’s.

How do these differ from Racket arrays or vectors?

• The memory is on the GPU, managed by RMM, not in the Racket heap. Moving data in or out is a copy.

• Work is asynchronous. Operations queue on the resources’ stream and return; the program waits when a value comes back to Racket or on resources-sync!.

• Dtype and layout are fixed. The dtype is 'float32, 'float64, 'int32 or 'int64; the layout is row-major, each row contiguous, or column-major, each column contiguous. Nothing converts silently: a wrong layout raises, and contiguous copies.

• Strides count elements. A 2-by-3 matrix has strides '(3 1) row-major and '(1 2) column-major.

• The collector cannot see device memory. A buffer’s phantom bytes make the device memory it holds bring collections.

An array’s memory, from RMM’s pool through the Racket collector and back.

You never free an array. The finalizer is the default and is correct on its own: each buffer is released once, on its own device and stream, from whatever OS thread runs it. It has no timeline, though: it runs at some collection after the last use. The with- forms give one. with-device-resources releases resources when its body returns, raises, escapes or yields, and with-array-views holds arrays for exactly one native call. The finalizer stays the backstop.