tensorfold serve MODEL --recall keeps a record of the token ids the server has read and written, and drafts from it.
When a reply starts to repeat text the server has seen before, the lanes verify the earlier continuation in a few rounds instead of drafting it token by token.
A regenerated or repeated request copies its earlier reply, an agent copies code and tool calls it wrote in an earlier session, and boilerplate copies itself.
The target model verifies every copied token, so replies are the same tokens with Recall on, off or read-only. Recall is off unless a flag turns it on.
Recall is not the prompt cache and not --learn.
Those keep the model's state after a prompt so a later request can skip reading it again.
Recall keeps only token ids, and it changes how fast a reply is written, not how a prompt is read.
| Flag | Meaning |
|---|---|
--recall |
Keep the record and draft from it. |
--recall-dir DIR |
The record's own directory. Without it the record lives in ~/.cache/tensorfold/recall, in a directory named by the tokenizer. Implies --recall. |
--recall-gib GIB |
The record's disk, 1/16 GiB (64 MiB) by default; the oldest ids go first past it. Implies --recall. |
--recall-read-only |
Draft from the record and add nothing to it. Implies --recall. |
GET /v1/recall reports the record: its state, size on disk and in memory, how many copies it offered and how many of their tokens landed.
DELETE /v1/recall empties it.
Both need the server's API key when it has one.
The record holds token ids only, never text, in chunk files of up to 1,048,576 ids (4 MiB) under one directory, made with mode 0700, each file 0600.
The model's tokenizer turns those ids back into the prompts and replies they came from, so treat the directory as you would those conversations.
Beside them a short text file, identity, names the tokenizer that wrote the record: a digest of its vocabulary, merges and added tokens, its vocabulary size, and the model the server first served.
An empty file, lock, is held by the server that writes the record.
Models whose tokenizers have the same identity share one record, and a server whose tokenizer differs refuses to start on that directory and says which tokenizer wrote it.
Each request adds the part of its prompt that no earlier request read, from where the prompt cache resumed it, and its reply when the reply ends.
Runs of 128 or more ids the record already holds are not added again, so a request sent twice adds nothing the second time.
Past --recall-gib the oldest chunk file is deleted.
While the disk has under 4 GiB free the server stops adding to the record, and serving goes on.
A second server that opens a directory another server is writing reads the record as it was and adds nothing.
Nothing in the record leaves the machine: no request, reply or route sends it anywhere.
DELETE /v1/recall deletes every chunk file and empties the record in memory, with its landing rates and counters.
The directory, identity and lock stay; they hold no ids.
When a chunk file cannot be deleted the reply is a 500 that names it, and the server adds nothing to the record until a later DELETE deletes every chunk file.
A server started with --recall-read-only, or reading a record another server writes, refuses DELETE with a 409.
Deleting the directory while no server uses it removes everything.
In memory each chunk takes 8 MiB from its first id, so even a short record takes 8 MiB, plus a key table fixed by --recall-gib (8 MiB at the default 64 MiB).
Recall drafts in the lane engine's copy window and its grafted lanes, for streams that share rounds and streams alone.
No served engine turns grafted lanes on yet, so today the record reaches replies through the copy window.
On CUDA a lone Nemotron stream's own driver asks the record when its context has nothing to copy.
Flash Next's replay engine copies from its own history on the GPU and reads no record, so a Flash Next server refuses --recall before it loads.
A lone Nemotron stream on a Mac and a lone Qwen3.8-27B stream on a Mac with tensor units run their own drivers, which do not read the record yet.
A populated record makes any benchmark that repeats its prompts faster than a fresh server would be.
Run release gates and speed comparisons with Recall off, or with --recall-read-only on an empty record, and say which.
The server's startup line and GET /v1/recall state the record's state and size.
A copy is offered when the record's continuation of the stream's last 16 tokens or more is expected to land more tokens than the stream's own drafts have lately. The expected tokens come from how the record's earlier copies landed, kept apart for matches of 16 to 31 tokens, 32 to 63, and 64 or more. A copy that misses costs its round the drafts it took the place of. A record whose copies keep missing stops offering them, apart from a probe that comes rarer while probes miss, and a stream whose copy landed nothing leaves the record alone for four rounds. Grafted lanes ask the record each round while its grafts land, and every eighth round otherwise. The stream's own context is asked first, and wins ties.