3 The embedding boundary
(current-batch-embed) -> (-> (listof string?) (listof flvector?)) |
(current-embed) -> (-> string? flvector?) |
(embed-texts texts) -> (listof flvector?) |
(make-cached-batch-embedder batch-fn) -> (-> (listof string?) (listof flvector?)) |
The batch function is the primitive: texts in, one vector per text, same order. Out of the box both parameters raise exn:fail:sonar — an unconfigured embedder must fail loudly, not silently degrade. embed-texts deduplicates and preserves order. make-cached-batch-embedder wraps any batch function with an exact-string cache, so re-reading a corpus does not re-embed known text.
Installing the Switchboard adapter:
(require sonar/switchboard) (current-batch-embed switchboard-batch-embed) (current-embed-model "openai/text-embedding-3-small") (current-embed-dimensions 768)
Change the model only before building or loading a store: saved stores hold vectors from the old model, and search signals a dimension mismatch.