2 Chunking
(chunk-text text #:size size #:overlap overlap) -> (listof string?) |
(current-chunk-size) -> exact-positive-integer? ; default 600 |
(current-chunk-overlap) -> exact-nonnegative-integer? ; default 80 |
Chunks break just after the last sentence terminator within a bounded look-back from the hard cut, otherwise hard-cut. Two invariants hold for any input: every returned chunk is non-empty, and the scan always advances. Overlap must be smaller than size.