[Bizuayeu]

What this changes

The CUDA host wired Nemotron's prompt-cache copies, rules and budget in by name. They now come from the family's
snap_* declarations through cuda_snaps.zig: a family without snap_save gets no prompt cache, snap_ready sets
aside its state buffers, and snap_grid makes the store keep states only on the family's chunk starts (for a family
whose rows depend on where a prompt's chunks begin). A family that declares warm_quiet gets 1.0.9's warm turns
(.warm = .waits). Nemotron declares warm_quiet and no grid and keeps 1.0.9's rules: any row, the head one
token on, warm turns, the same budget.

Three more hooks for a family split over hosts: weightShare (the weights this rank holds, for the memory check),
explains_open (the family's own text for an open error) and the learner only for a family with slide. The
GLM-5.3-Flash family (#548) uses all of them.

Receipt

• Environment: TensorFold 1.0.9 (c36b536d) plus this change; one GB10 (DGX Spark, sm_121, CUDA 13.3).
• Exactness: Nemotron's open, prompt cache and --slide are unchanged (the same calls and rules, now through the
  family's declarations).
• Decode and prompt speed: not applicable.
• Tests run:
  • zig build test-cuda-host: 430 passed, 0 failed; the new "a family with snap_grid keeps on its chunk starts, any
    other at any row as Nemotron does" needs this change.
  • zig build test: 620 passed, 5 skipped, 0 failed. zig fmt --check: clean. lean_check: 538, as main.
• Not run, and why: a served Nemotron run on CUDA (no Nemotron checkpoint on these machines); Metal (CUDA only). weightShare, explains_open and the slide guard are not covered by a host test: they are branches of openWith, which needs a GPU and a real checkpoint to reach. The GLM-5.3-Flash family's runs at TP=2 and TP=3 (Zig CUDA ports: shared layers below lanes.Backend, and one place to coordinate them #548) go through all three.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.
• No precision traded for speed. If the bits change, the description says which sums change and why.
• Prompt processing is no slower than the last release.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[Bizuayeu]

Rebased onto 1.0.9. Its Nemotron store gained warm turns (.warm = .waits, warm_turns), so cuda_snaps.cacheRules now sets .waits for a family that declares warm_quiet (Nemotron does) and warm_turns follows the same declaration; Nemotron's rules, budget and warm turns are 1.0.9's, and a family without warm_quiet keeps none. The test covers the three cases. Host tests 430/430, zig build test 620 passed, 5 skipped.

[ashhart]

Thank you, @Bizuayeu. Taking a CUDA family's prompt cache, rules and budget from its own snap_* declarations, instead of wiring Nemotron's in by name, is what lets a second family bring its cache with it.

It ships in TensorFold 1.0.12 under your name (39307003b). One line of the rebase carried an older text back, so our 4a0a31f31 restores 1.0.9's startup wording ("prompts in passes of up to {d} rows") and wraps the new lines. Your head is merged after them with no change to the tree (bbab0f6d8), so this pull request shows as merged.

On a GB10 with Nemotron 3.5 Lightning, MLX 4-bit and NVFP4: multi-turn chats (three turns, then a regenerated, an edited and a deleted turn, thinking on and off) resumed with the same cached tokens as the build before, step for step, and their replies equal both that build's and a cache-off server's, 12 of 12 on each format. Drafted equals "draft": false in all 17 release cells with the same token SHAs, and 2, 4 and 8 streams equal solo.

weightShare, explains_open and snap_grid get their served check with GLM-5.3-Flash on CUDA (#804).

Posted by Ash's AI Assistant Synthena