Racket

onnxruntime: CPU inference of ONNX models in Racket🔗ℹ

This package provides Racket FFI bindings to the ONNX Runtime C API for CPU inference of ONNX models. It is the runtime leg of the “train in Python once, export .onnx, infer anywhere” workflow: train with PyTorch/TensorFlow/scikit-learn, export the model, then load and run it from Racket with a small, dependency-light API.

Version 0.1 scope. float32 tensors only, CPU execution provider only, models loaded from a file path or from bytes. Sequence, map, and string tensors are out of scope; sessions cannot be closed explicitly (native memory is reclaimed when the Racket process exits).

1 Installation🔗ℹ

Racket package:
raco pkg install onnxruntime

Native library: the loader must find libonnxruntime (.dylib on macOS, .so on Linux).

On macOS with Homebrew:

brew install onnxruntime

The dylibs land in /opt/homebrew/opt/onnxruntime/lib/ (Apple silicon), which the loader probes automatically as a fallback. You can also export DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/opt/onnxruntime/lib to make the system dynamic linker find it directly.

On Linux, download and extract an official release tarball, e.g.

ORT_VERSION=1.19.2

curl -fLO "https://github.com/microsoft/onnxruntime/releases/download/v${ORT_VERSION}/onnxruntime-linux-x64-${ORT_VERSION}.tgz"

tar -xzf onnxruntime-linux-x64-${ORT_VERSION}.tgz

export LD_LIBRARY_PATH="$PWD/onnxruntime-linux-x64-${ORT_VERSION}/lib:$LD_LIBRARY_PATH"

Explicit override: set ONNXRUNTIME_LIB to the full path of the library file (or a directory containing it) and it is tried first.

2 API version negotiation🔗ℹ

The ORT C API is a versioned function table: OrtGetApiBase() yields a GetApi(version) accessor, and each API version’s function table is a fixed, append-only ordering of function pointers. At load time this package asks for the newest table it knows (version 31, from the onnxruntime 1.31.0 header its index was generated from) and walks down to version 10 until the runtime answers with a non-null table. Consequently:

against onnxruntime >= 1.31 the negotiated version is 31;
older runtimes work too, down to API version 10 (onnxruntime 1.10.x), and (onnx-api-version) reports what was negotiated;
while negotiating downward on an older runtime, ORT prints one warning line per unavailable version to stderr – that is expected.

3 Quick start🔗ℹ

Given a model add.onnx computing Z = X + [[1,2],[3,4]] over a float32 [2,2] input X:

(require onnxruntime racket/flonum)

(define s (open-onnx-session "add.onnx"))
(session-input-names s)
(session-output-names s)
(session-input-shape s "X")

(define x (make-ort-tensor '(2 2) (flvector 1.0 1.0 1.0 1.0)))
(define results (onnx-run s (list (cons "X" x))))
(results-ref results "Z")

procedure
(results-ref results name) → ort-tensor?
results : (listof (cons/c string? ort-tensor?))
name : string?

Convenience: look up the tensor bound to name in an onnx-run result. Raises exn:fail if absent.

4 Reference🔗ℹ

procedure
(onnx-version) → string?

Returns the ONNX Runtime library version string, e.g. "1.31.0".

procedure
(onnx-api-version) → exact-nonnegative-integer?

Returns the OrtApi table version negotiated at load time (at most 31; see API version negotiation for what that means on older runtimes).

struct
(struct ort-tensor (shape data))
shape : (listof exact-nonnegative-integer?)
data : flvector?

A float32 tensor: shape in row-major order and data as a flat flvector of exactly (product shape) elements.

procedure
(make-ort-tensor shape data) → ort-tensor?
shape : (listof exact-nonneg-integer?)
data : flvector?

Constructor that validates the shape/data size agreement.

procedure
(open-onnx-session src
[ #:optimization-level level]) → onnx-session?
src : (or/c path-string? bytes?)
level : (or/c 'disable 'basic 'extended 'all) = 'all

Opens a session from a model file path or from raw .onnx bytes. level maps to ORT’s graph optimization levels (ORT_DISABLE_ALL=0, ORT_ENABLE_BASIC=1, ORT_ENABLE_EXTENDED=2, ORT_ENABLE_ALL=99).

procedure
(onnx-session? v) → boolean?
v : any/c

Predicate for the opaque session handle.

procedure
(session-input-names s) → (listof string?)
s : onnx-session?

Model input names in declaration order.

procedure
(session-output-names s) → (listof string?)
s : onnx-session?

Model output names in declaration order.

procedure
(session-input-shape s name)
→ (listof (or/c exact-positive-integer? 'dynamic))
s : onnx-session?
name : string?

Model metadata shape for the named input. Symbolic dimensions come back as 'dynamic.

procedure
(session-output-shape s name)
→ (listof (or/c exact-positive-integer? 'dynamic))
s : onnx-session?
name : string?

Model metadata shape for the named output. Symbolic dimensions come back as 'dynamic.

procedure
(onnx-run s inputs) → (listof (cons/c string? ort-tensor?))
s : onnx-session?
inputs : (listof (cons/c string? ort-tensor?))

Runs the model. inputs must cover every model input (pass all of them, even unchanged ones); duplicate or unknown names raise exn:fail. Results come back in model output order. All native scratch memory is released before the call returns.

5 Documented limits (v0.1)🔗ℹ

float32 tensors only (other element types raise exn:fail);
CPU execution provider only;
all model inputs must be passed on every run;
no explicit session close: native session memory lives until process exit;

6 License🔗ℹ

MIT. ONNX Runtime itself is separately licensed by Microsoft under the MIT license.

1	Installation
2	API version negotiation
3	Quick start
4	Reference
5	Documented limits (v0.1)
6	License