2.5.1 Checking native calls🔗ℹ

The call and both reads run in one call-as-atomic, because the C side keeps its last error per OS thread, which the place’s Racket threads share. The native call therefore holds up the place’s other threads until it returns: a call that can run long is a job instead (define-job).

A failure raises with the library’s message. The handle here comes from the thread’s default resources, which the thread holds; a binding uses with-array-views’s #:resources instead (The handle):

> (define ints (list*->device-matrix '((1 2) (3 4) (5 6)) #:dtype 'int32))

> (define centers (device-matrix 2 2))

> (define handle (resources->handle-pointer (current-device-resources)))

> (with-array-views ([x ints] [c centers])     (check 'kmeans-fit            (lambda () (kc-fit handle x #f c 0 300 0.0001 1 0.0 42))))

kmeans-fit: X: expected float32 or float64, got int32

> (with-handlers ([exn:fail:raft? exn:fail:raft-kind])     (with-array-views ([x ints] [c centers])       (check 'kmeans-fit (lambda () (kc-fit handle x #f c 0 300 0.0001 1 0.0 42)))))

'logic

With #:exn, a package raises its own subtype of exn:fail:raft:

> (struct exn:fail:canary exn:fail:raft ())

> (define check/canary     (status-checker kc-last-error kc-last-error-kind #:exn exn:fail:canary))

> (define X (device-matrix 300 2))

> (define truth (device-vector 300 #:dtype 'int32))

> (with-array-views ([x X] [y truth])     (check/canary 'blobs (lambda () (kc-make-blobs handle x y 3 0.5 1 -10.0 10.0 7))))

> (define fitted (device-matrix 3 2))

> (define-values (inertia n-iter)     (with-array-views ([x X] [c fitted])       (check/canary 'kmeans-fit (lambda () (kc-fit handle x #f c 0 300 0.0001 1 0.0 42)))))

> (list (> inertia 0) (>= n-iter 1))

'(#t #t)

> (with-handlers ([exn:fail:canary? (lambda (e) (list 'canary (exn-message e)))])     (with-array-views ([x X] [c (device-matrix 3 5)])       (check/canary 'kmeans-fit (lambda () (kc-fit handle x #f c 0 300 0.0001 1 0.0 42)))))

'(canary "kmeans-fit: centroids: expected 2 columns, got 5")

2.5.2 Long calls🔗ℹ

A native call that can run long, such as a fit, runs as a job: on a worker thread its resources object owns, while the calling thread waits and the place’s other threads run. Jobs on one resources object run one at a time, in the order they were made, as work on one stream does; jobs on different resources objects run at the same time. Each thread’s current-device-resources are its own, and so are their workers.

A job has two parts. On the C++ side, the entry point is written as usual (The C interface), plus a one-line job entry that raftrkt::run_job fills in:

C++

extern "C" KC_API void kc_fit_job(void* args, size_t args_len, rr_job_outcome* outcome) {

raftrkt::run_job<&kc_fit>(args, args_len, outcome, kc::last_error_slot());

}

On the Racket side, define-job declares the entry point’s parameters in order.

A failure of the entry point raises (exn message marks kind) as status-checker does, with who and the job’s own message and kind; exn is exn:fail:raft unless #:exn says otherwise. Released resources or arrays raise exn:fail:raft of kind 'logic before the job is made, as does a call in atomic mode, where the calling thread cannot wait. A view argument that is neither a device array nor #f, or a deposit argument that is not a deposit, raises exn:fail:raft of kind 'logic naming who, also before the job is made. A number its type refuses raises exn:fail:contract naming who.

The job holds what it uses, the handle, each array’s memory and each deposit, until it returns, even if the calling thread is killed or the arrays, deposits and resources become unreachable. A deposit that cannot be bound raises exn:fail:raft of kind 'logic before the job runs. A job that cannot be submitted raises exn:fail:raft, of kind 'generic when the system refuses a file descriptor or a thread, and 'out-of-memory when memory runs out. Either closes the deposits bound before it. Each view is bound once the work queued on its array’s stream is done. The calling thread waits with breaks disabled: a break that arrives meanwhile is raised when the job returns, and its outputs are dropped. When the process exits, a running job is waited for as current-exit-wait says.

The declaration must match the C entry point. run_job refuses an argument block whose size differs from what the entry point’s parameters take ('logic), but it cannot tell apart types of the same size.

A failure raises with the job’s own message:

> (kc-fit-job 'kmeans-fit (current-device-resources) ints #f centers 0 300 0.0001 1 0.0 42)

kmeans-fit: X: expected float32 or float64, got int32

A view that is not an array raises before the job is made:

> (kc-fit-job 'kmeans-fit (current-device-resources) 'not-an-array #f centers 0 300 0.0001 1 0.0 42)

kmeans-fit: expected a device array or #f for a view, given

'not-an-array

Other threads run while a job does:

> (define big (device-matrix 1000000 32))

> (kc-make-blobs-job 'blobs (current-device-resources) big (device-vector 1000000 #:dtype 'int32)                      64 1.0 1 -10.0 10.0 1)

> (define (ticks-during thunk)     (define ticks (box 0))     (define ticker       (thread (lambda ()                 (let loop ()                   (sleep 0.005)                   (set-box! ticks (add1 (unbox ticks)))                   (loop)))))     (thunk)     (kill-thread ticker)     (unbox ticks))

> (<= 10 (ticks-during           (lambda ()             (kc-fit-job 'kmeans-fit (current-device-resources) big #f (device-matrix 64 32)                         1 100 1e-12 1 0.0 1))))

#t

A declaration of the wrong size is refused:

> (define-job kc-fit-job/short canary "kc_fit_job" (handle view view view _int32))

> (kc-fit-job/short 'kmeans-fit (current-device-resources) X #f fitted 0)

kmeans-fit: the argument block is 472 bytes; the job expects

520

A deposit parameter takes outputs sized by the call (deposit):

> (define labels (device-vector 300 #:dtype 'int32))

> (kc-predict-job 'kmeans-predict (current-device-resources) fitted X #f 1 labels)

154.17282104492188

> (define-job kc-cluster-members-job canary "kc_cluster_members_job" (handle view view _int32 deposit))

> (define found (deposit))

> (kc-cluster-members-job 'cluster-members (current-device-resources) X labels 0 found)

> (deposit->list found)

'(#<device-vector int64[100] cuda:0 [  1   4   7 ... 292 295 298]> #<device-matrix float32[100×2] row-major cuda:0  [[  9.078729  -8.686529]   [  8.251212  -8.188368]   [  8.923475   -9.37299]   ...   [  8.185297 -7.1188946]   [  8.874211  -8.659465]   [  9.315469  -8.734181]]>)

> (kc-cluster-members-job 'cluster-members (current-device-resources) X labels 0 found)

cluster-members: the deposit is closed

> (kc-cluster-members-job 'cluster-members (current-device-resources) X labels 0 'not-a-deposit)

cluster-members: expected a deposit for a deposit parameter,

given 'not-a-deposit

2.5.3 The ABI tag🔗ℹ

The canary’s load check returns nothing when the headers match:

> (check 'kmeans-canary (lambda () (kc-check-abi (raft-abi-pointer))))

A mismatch is refused, naming the first field that differs. Here a copy of the tag is edited to look like RAFT 26.10 (the third int32), then like a handle of another size (the seventh int64 slot):

> (define (forged-tag type index value)     (define tag (make-bytes 56))     (memcpy tag (raft-abi-pointer) 56)     (ptr-set! tag type index value)     tag)

> (check 'kmeans-canary (lambda () (kc-check-abi (forged-tag _int32 2 10))))

kmeans-canary: RAFT: built against 26.08.00, but libraftrkt

has 26.10.00; build both against the same rapids package set

> (define bigger-handle (add1 (raft-abi-handle-size (raft-abi))))

> (check 'kmeans-canary (lambda () (kc-check-abi (forged-tag _int64 6 bigger-handle))))

kmeans-canary: raft::handle_t size: built against 32, but

libraftrkt has 33; build both against the same rapids

package set

2.5.4 The handle🔗ℹ

The pointer is borrowed and does not hold resources: once they are unreachable, their finalizer frees the handle. A binding therefore takes the handle with with-array-views’s #:resources; this procedure suits code that holds the resources some other way, as (current-device-resources) does for its thread.

Each resources object has its own handle, and so its own stream:

> (define other (device-resources))

> (ptr-equal? handle (resources->handle-pointer other))

#f

A binding allocates its outputs on the resources it runs on, so they share one stream; #:resources binds the same pointer:

> (define Y (device-matrix 6 2 #:resources other))

> (define classes (device-vector 6 #:dtype 'int32 #:resources other))

> (with-array-views #:resources [on-other other] ([y Y] [k classes])     (check 'blobs (lambda () (kc-make-blobs on-other y k 2 0.1 0 -1.0 1.0 3)))     (ptr-equal? on-other (resources->handle-pointer other)))

#t

> (sort (remove-duplicates (device-vector->list classes)) <)

'(0 1)

> (define finished (with-device-resources ([r (device-resources)]) r))

> (resources->handle-pointer finished)

resources->handle-pointer: the device resources on device 0

were released

2.5.5 Array views🔗ℹ

The arrays and the resources stay managed by the garbage collector: the form frees neither, and their finalizers are the default and correct on their own. What the form gives a native call is a clear timeline for what it borrows: from the start of the body until it exits, the views are valid and the arrays and the resources are held reachable; at exit the views are cleared and the form’s hold ends, and the arrays return to ordinary reclamation. To release the resources themselves at a known point, scope them with with-device-resources around the form. A job binds its own views; this form serves short calls through status-checker.

Every exit (a return, a raise, an escape, a generator’s yield) clears each view: its data becomes NULL and its device and memory -1, so the C side refuses a view kept past the form. The handle pointer is not cleared, so it must not escape the body. Control that jumps back into the body raises exn:fail:raft of kind 'logic.

Before binding a view, the form waits, by polling, for the work queued on the array’s stream. It does not wait after the body: a native call that returns with work still queued must run on the arrays’ own resources or synchronise before it returns.

A view holds the element type code (RR_DTYPE_FLOAT32 is 0), the rank, the shape and the strides at the offsets of The C interface:

> (define F (contiguous X #:layout 'col-major))

> (with-array-views ([f F])     (list (ptr-ref f _int32 'abs 8)           (ptr-ref f _int32 'abs 20)           (list (ptr-ref f _int64 'abs 24) (ptr-ref f _int64 'abs 32))           (list (ptr-ref f _int64 'abs 88) (ptr-ref f _int64 'abs 96))))

'(0 2 (300 2) (1 300))

The C side checks each view against what the call needs, and a cleared view is refused:

> (with-array-views ([x F] [c (device-matrix 3 2)])     (check 'kmeans-fit (lambda () (kc-fit handle x #f c 0 300 0.0001 1 0.0 42))))

kmeans-fit: X: expected row-major, got col-major 300x2

> (define stale (with-array-views ([x X]) x))

> (with-array-views ([c fitted] [y labels])     (check 'kmeans-predict (lambda () (kc-predict handle c stale #f 1 y))))

kmeans-predict: X: not a view bound to device memory

2.5.6 Outputs sized by the call🔗ℹ

Some native calls allocate outputs whose size only the call knows, such as HDBSCAN’s condensed tree. A deposit is a record the call fills with that memory and with the owner that frees it; raft then turns each region into an array.

A deposit can be converted once it is filled, it is filled only once, and it belongs to the resources it was made on:

> (define pending (deposit))

> (with-deposits ([out pending]) (deposit->list pending))

deposit->list: the deposit is still open

> (deposit->list pending)

'()

> (with-deposits ([out members]) out)

with-deposits: the deposit is closed

> (define elsewhere (deposit #:resources other))

> (with-deposits ([out elsewhere])     (with-array-views #:resources [handle (current-device-resources)] ([x X] [y labels])       (check 'cluster-members (lambda () (kc-cluster-members handle x y 0 out)))))

cluster-members: out: on resources other than the call's

2.5.7 The C interface🔗ℹ

The C side is the public C headers of libraftrkt (core.h, array.h, memory.h, gathered by c_api.h), the ABI header and two header-only C++ helpers, shipped with a CMake package (find_package(raftrkt) defines raftrkt::headers, which links raft::raft). A downstream library compiles against them and the same RAPIDS 26.08 libraries and headers as libraftrkt, and does not link libraftrkt.

The handle. int rr_resources_handle(rr_resources* resources, void** out) sets out to NULL, then to the raft::handle_t*. Like every entry point, it answers RR_OK (0) or RR_ERROR (1), with the reason in rr_last_error() and rr_last_error_kind().

Views. An rr_view is 152 bytes:

int rr_buffer_view(const rr_buffer* buffer, uint64_t offset, rr_view* view) is the one way a view gets its pointer: the caller fills dtype, rank, shape and strides, and the call checks them against the buffer and fills data, device and memory, or on refusal NULL, -1 and -1. data is valid only while the buffer lives, and only for work ordered on the buffer’s stream.

raftrkt/abi.h. The rr_abi_tag struct (56 bytes: twelve int32_t fields and the int64_t handle size), RR_ABI_VERSION (1), and, for C and C++, int rr_abi_compare(const rr_abi_tag* built, const rr_abi_tag* loaded, char* why, size_t why_size), which answers 0 when the tags agree and otherwise 1, writing the first difference into why. For C++, raftrkt::compiled_abi() is the tag of the headers being compiled (libraftrkt builds its own the same way), and raftrkt::require_abi(loaded) throws raftrkt::logic_error when they differ:

C++

extern "C" int kc_check_abi(const rr_abi_tag* loaded) {

return raftrkt::translate_exceptions(kc::last_error_slot(),

[&] { raftrkt::require_abi(loaded); });

}

raftrkt/error.hpp. The error convention, so a downstream library’s errors look like libraftrkt’s:

• raftrkt::error_slot: a fixed 4 KiB message buffer and a kind. The library owns one per OS thread (thread_local) and exports the two readers status-checker calls.

• int raftrkt::translate_exceptions(error_slot& slot, Fn&& fn): clears the slot, runs fn, and answers RR_OK; if fn throws, records the message (truncated at a UTF-8 character boundary, without allocating) and its kind, and answers RR_ERROR.

• raftrkt::classify: the kind of an exception. rmm::out_of_memory, std::bad_alloc, CUDA errors with status cudaErrorMemoryAllocation and RAFT’s cuBLAS, cuSOLVER and cuSPARSE errors with status *_STATUS_ALLOC_FAILED are 'out-of-memory; other CUDA, RMM and RAFT CUDA, cuBLAS, cuSOLVER and cuSPARSE errors are 'cuda; std::logic_error and raft::logic_error are 'logic; anything else is 'generic.

• raftrkt::logic_error, raftrkt::cuda_error, raftrkt::cuda_check(status, call) and raftrkt::require(pointer, name), which throws "name is NULL".

• raftrkt::device_scope: makes a device current for its scope, throwing if cudaSetDevice fails, and restores the previous one.

• raftrkt::sync_guard: synchronises a stream on every exit from its scope. finish() synchronises and throws on failure; on a throw, the destructor synchronises and swallows a second error, so the first is reported.

raftrkt/view.hpp. Views to typed pointers and RAFT views. Each takes the argument’s name for its messages and throws raftrkt::logic_error:

• T* raftrkt::matrix_data<T, Index>(const rr_view* view, const char* name, raftrkt::layout l, int64_t rows = any_extent, int64_t cols = any_extent): refuses a NULL or unbound view, a rank other than 2, an element type other than T’s (const T reads the same type), a layout other than l (an axis of extent 1 fits either), extents other than those asked for, and extents or an element count that do not fit Index.

• T* raftrkt::vector_data<T, Index>(const rr_view* view, const char* name, int64_t n = any_extent): the same for a contiguous rank-1 view.

• raftrkt::matrix_view<T, Layout, Index> and raftrkt::vector_view<T, Index>: the same checks, answering a raft::device_matrix_view or raft::device_vector_view.

• raftrkt::has_layout(view, l), raftrkt::dtype_code<T> and raftrkt::dtype_name(code).

• raftrkt::handle_of(void* handle): the raft::handle_t a pointer from with-array-views or resources->handle-pointer carries, refusing NULL; raftrkt::device_of(handle): its device.

• raftrkt::require_on_device(device, {{"X", x}, ...}): refuses a bound view on another device, naming it.

C++

const T* data = raftrkt::matrix_data<const T, int>(x, "X", raftrkt::layout::row_major);

T* centers = raftrkt::matrix_data<T, int>(centroids, "centroids", raftrkt::layout::row_major,

raftrkt::any_extent, d);

auto m = raftrkt::matrix_view<const float, raft::col_major, int64_t>(a, "A");

raftrkt/deposit.h. An rr_deposit is 1256 bytes. raft fills the first three fields when it makes the deposit; the native call fills the rest:

Once the record is filled, it owns the owner, and libraftrkt calls release(owner) exactly once. release runs on any OS thread, with the deposit’s device current and its raft::handle_t still alive, and may run concurrently with the library’s other calls; it must not throw or call Racket. Every region lies in memory the owner frees, on the handle’s stream, and its contents are complete when the call returns.

raftrkt/deposit.hpp. Header-only, like view.hpp:

• rr_view raftrkt::region(rmm::device_uvector<T>& v), region(T* data, int64_t n) and region(T* data, int64_t rows, int64_t cols, raftrkt::layout l): a vector or a matrix with canonical strides, for T one of the four element types.

• void raftrkt::deposit(rr_deposit* into, const char* name, const raft::handle_t& handle, std::unique_ptr<Owner>&& owner, std::initializer_list<rr_view> regions): refuses, in order, a NULL record, a record that is not open ("out: being filled" while another filler holds it, "out: already filled", "out: closed"), a record on other resources than handle’s, no owner, more than 8 regions, a region of another rank or element type, with a negative extent, a byte size that overflows, another layout, no data or misaligned data, overlapping regions, and memory that is not on the record’s device. A refusal throws raftrkt::logic_error and leaves the owner to the caller’s unique_ptr. After the NULL check it claims an open record by moving it to FILLING; a record that cannot be claimed is refused with nothing touched, so a second filler cannot disturb the first. A later refusal touches nothing but state: the record is open again, or closed if it was closed meanwhile, and its other fields are as they were. Otherwise it fills the record, moves it to FILLED and takes the owner; a record closed during the fill is refused with "out: closed", as above.

C++

auto owner = std::make_unique<members<T>>(handle, m, d);

// The regions read owner before deposit takes it: std::move is only a cast.

raftrkt::deposit(out, "out", handle, std::move(owner),

{raftrkt::region(owner->indices),

raftrkt::region(owner->rows.data(), m, d, raftrkt::layout::row_major)});

raftrkt/job.h. An rr_job_outcome is 4104 bytes: int32_t status at 0, int32_t kind at 4 and char message[RR_JOB_MESSAGE_BYTES] (4096) at 8. A job entry is an rr_job_fn, void (*)(void* args, size_t args_len, rr_job_outcome* outcome): it runs on a raft worker thread, after the previous job on the same resources returned, and must not throw or call Racket.

raftrkt/job.hpp. Header-only. raftrkt::run_job<&f>(args, args_len, outcome, slot) unpacks the block define-job builds, calls f, writes its outputs back, and records its status and, on failure, the kind and message from slot, the library’s error slot, read on the worker right after f returns. f returns int and takes only void* (the handle), const rr_view*, rr_deposit*, numbers, and pointers to non-const numbers (its outputs); anything else does not compile (raftrkt::job_parameter). raftrkt::job_args_size<&f> is the block’s size.

What is frozen. rr_view, RR_MAX_RANK and the dtype and memory codes; rr_buffer_view; rr_resources_handle; rr_abi_tag, rr_abi and RR_ABI_VERSION; the status codes, error kinds and rr_last_error convention; the three headers above; the four exports of this module; and the rules a downstream entry point keeps: run inside translate_exceptions, select the handle’s device, refuse arrays on another device, read arrays only through view.hpp, and synchronise the handle’s stream on every exit. A change to any of them is a new ABI version, with the canary changed in the same pull request. Additions at version 1 (#22, #32): rr_deposit, RR_MAX_REGIONS, the RR_DEPOSIT_* states, rr_release_fn, deposit.hpp, deposit, deposit?, with-deposits, deposit->list, rr_job_outcome, RR_JOB_MESSAGE_BYTES, rr_job_fn, job.hpp, define-job, and the rule that a call that can run long is a job; a downstream detects them by requiring raft version 0.2, or with __has_include.