[jrb-ai]

What this changes

Interleaved template string accumulations retain their own growing buffers, avoiding repeated copies of the full prefix. Saved and branched strings retain their bytes and request provenance. Related to #1633; this remains a draft because the reported 66 GiB production failure is not reproduced.

The final implementation uses a one-based u32 handle into the render context's builder registry. Str remains 40 bytes and Value 48 bytes, matching the base on the tested 64-bit targets. Reuse requires both text and provenance to identify the builder's complete current value; older snapshots start a new builder. Earlier backing arrays remain valid for retained aliases. These checks are necessary for byte and provenance correctness, including mutation while evaluating the right operand.

Receipt

Base f350cd3ea253a042e205746928271f9d8f71a82d; current candidate implementation 8d00b490f5ddf45721546e9b9816a083a46e5fa2. Later commits add qualification helpers and receipts. Historical receipts retain their original revision labels.

Template memory and rendering

The current allocation controls cover 306 processes across three revisions and three optimization modes: 297 completed and matched Jinja bytes, nine baseline processes (three shapes in three modes) hit the fixed 240 MiB combined compiled-template and render-scratch cap, and all 102 current-candidate cases completed. Independent verification checked 176,407,221 rendered bytes and 72 provenance comparisons. Scratch is peak live allocator memory, not RSS or GPU memory.

Exact-template shape, ReleaseSafe	Base scratch bytes	Current scratch bytes	Result
2,001-message filler	145,623,620	145,623,620	Same peak as base
64 cycles with 8 KiB serialized object arguments	123,597,630	16,491,748	−86.66%
128 cycles with large object arguments	Fixed cap exceeded	41,822,772	Candidate completes

Remaining shape-dependent costs are disclosed in that receipt: short large-catalog scratch is 3.34% above base, 128 small-object cycles 1.05% above base, and the 128-large-JSON-string case uses 38,094,060 bytes versus 34,618,268 for the earlier pointer implementation (+10.04%; original base hits the cap). This does not remove all quadratic list operations.

Fixed host timings use six balanced three-way permutations, 72 processes and 216 timed renders. These are descriptive medians, not an equivalence test or model throughput:

Shape	Base render ms	Current render ms	Change
Nine-message control	0.0535835	0.0538125	+0.43%
2,001-message filler	16.525146	15.9432495	−3.52%
Filler with 157 synthetic schemas	16.437979	15.6985415	−4.50%
64 large-object cycles	10.0649585	2.1371875	−78.77%

Current CUDA build and model qualification

Full CUDA receipt and reproducible raw data. Environment: one DGX Spark GB10 (sm_121), Ubuntu 24.04.5, driver 580.178.04, Zig 0.17.0, nvcc 13.3.73 in nvcr.io/nvidia/pytorch:26.07-py3, checkpoint nvidia/Qwen3.8-Flash-Next-FP4-Local at b9ce18a6887e18c089fdea8509a33e63429a3300. Context 8,192, parallel two, bf16 KV, prompt cache zero, top-k 20/top-p 0.95, 9 GiB reserve and 6 GiB guard.

The candidate built successfully through 218/218 CUDA steps and a separate 112/112-step build embedding all 103 retained baseline GPU assets. The baseline install was reused unchanged. Four independently compiled CUDA fatbins differed, so measured binaries use identical baseline assets to isolate the host change. Source, executable, helper, image and checkpoint metadata identities are pinned.

Exactness: 18 complete 256-token drafted/plain responses form nine exact pairs across three fixtures, greedy seed 1234 and sampled seeds 1234/1235. Six generation/history tokenizations also match. All 480 decode responses match across repetition, solo/concurrent and builds; every candidate response matches the corresponding plain P0 output. The corrected experiment covers 498 responses and 127,488 token IDs, with zero mismatches and zero cached tokens.

Decode speed

Six paired fresh-server blocks (AB, BA, BA, AB, AB, BA), 256-token replies, two repetitions per cell. Each interval uses six paired log ratios of server-level medians (95% t, df=5), not individual requests. All twelve intervals are wholly within the prespecified ±2% band. No interim intervals, replacement pairs or extra runs.

Throughput columns are arithmetic means of six per-run medians. The paired geometric change and confidence interval come unchanged from the frozen comparator and determine the gate.

Fixture	Temperature	Streams	Base mean tok/s	Candidate mean tok/s	Paired geometric change	95% paired interval	Gate
34-message conversation	0	1	61.16	61.02	-0.233%	[-0.559%, +0.095%]	equivalent
34-message conversation	0	2	85.04	84.94	-0.111%	[-0.584%, +0.364%]	equivalent
34-message conversation	1	1	54.68	54.61	-0.131%	[-0.376%, +0.115%]	equivalent
34-message conversation	1	2	54.78	54.79	+0.012%	[-0.203%, +0.228%]	equivalent
Prose	0	1	56.17	56.00	-0.309%	[-1.105%, +0.494%]	equivalent
Prose	0	2	102.77	103.14	+0.360%	[-0.589%, +1.317%]	equivalent
Prose	1	1	47.72	47.76	+0.073%	[-0.520%, +0.669%]	equivalent
Prose	1	2	62.90	62.87	-0.050%	[-0.567%, +0.470%]	equivalent
Code as chat	0	1	63.23	63.00	-0.358%	[-1.175%, +0.466%]	equivalent
Code as chat	0	2	111.08	110.87	-0.181%	[-0.821%, +0.464%]	equivalent
Code as chat	1	1	62.34	62.25	-0.142%	[-0.659%, +0.377%]	equivalent
Code as chat	1	2	81.51	81.40	-0.141%	[-0.713%, +0.434%]	equivalent

TTFT

Measured cells show arithmetic means of six per-run median request TTFTs. The last row separately averages six first-cold excluded warmups per side. TTFT has no equivalence gate.

Fixture	Temperature	Streams	Base mean ms	Candidate mean ms	Change ms	Change %
34-message conversation	0	1	511.793	511.248	-0.545	-0.11%
34-message conversation	0	2	797.017	794.532	-2.485	-0.31%
34-message conversation	1	1	514.206	510.457	-3.749	-0.73%
34-message conversation	1	2	801.953	799.136	-2.817	-0.35%
Prose	0	1	118.997	121.030	+2.033	+1.71%
Prose	0	2	180.663	179.668	-0.995	-0.55%
Prose	1	1	122.837	123.505	+0.668	+0.54%
Prose	1	2	181.404	182.307	+0.903	+0.50%
Code as chat	0	1	113.333	111.833	-1.500	-1.32%
Code as chat	0	2	169.230	167.928	-1.301	-0.77%
Code as chat	1	1	112.179	112.602	+0.423	+0.38%
Code as chat	1	2	166.586	168.498	+1.911	+1.15%
First-cold excluded warmup	0	1	572.219	596.913	+24.693	+4.32%

First-cold mean TTFT increased by 24.69 ms (+4.32%). This is descriptive; TTFT equivalence is not established.

Tests run and limits

• Current template suite: 37/37 focused tests in each of Debug, ReleaseSafe and ReleaseFast. Broad host build: 299/299 steps, 1,099 executed tests passed, three Linux-only tests skipped on macOS; other groups cached. Includes test-cuda-host, server CPU, kernel-contract host, golden and fuzz targets.
• Wire goldens: 322/335 equal, 13 known differences, zero unexpected; independent template controls and current model checks are described above. Actual Qwen Jinja template: six cases per revision match Jinja2 3.1.6 bytes; that separate oracle does not check provenance.
• All eight changed Zig files pass zig fmt --check; git diff --check passes. Lean remains 426 base / 426 branch, zero added problems. Existing findings are not called clean.
• An initial new-client parity attempt stopped after one valid drafted response because its checker expected lists but the shared stream client returns tuples. The corrected checker accepts both without weakening content checks, and passes a regression using the actual shared SSE client. The aborted receipt is preserved and excluded: zero completed pairs and zero decode observations. All four corrected GPU windows restored production without guard events or cleanup errors.
• No GPU kernel changed, so new device-contract registration and backend SUMMARY lines are not applicable. CUDA compilation and the checkpoint-specific model gate passed as reported above; a fresh full GPU kernel-contract suite was not run for this host-only change.
• Metal compilation was attempted but the local Xcode installation lacks its Metal compiler component. Metal device/model tests were not run. Cached resume, a long-prompt cold prefill benchmark, comparison with the last release, other checkpoints, and the original patched multi-rank request remain unqualified. Short-request TTFT does not establish long-prompt performance.
• The exact supplied DeepSeek template with 2,001 filler messages does not reproduce the reported allocation failure. The reporter's complete 157 schemas, private tool-heavy history and patched multi-rank build remain unavailable; this PR does not claim to resolve that incident.

Coordination

ATC check was complete with advice yours and PR #1652 as the only linked implementation. Official overlap remains incomplete with pullFilesLimit. A separate paginated audit at 2026-10-30T15:14:52.694814+00:00 covered all 110 open PRs, 4,998 changed-file entries and all 153 branch paths. A second inventory confirmed stable PR heads, bases and local targets. No other PR touched these paths, so no touching PR required a merge-tree check.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Drafted/plain and concurrent/solo pass; resumed-cache equality remains unrun.
• No precision traded for speed. No floating-point sums or model sampling changed; saved-string corruption cases deliberately match Jinja rather than corrupted baseline bytes.
• A new or changed GPU kernel is registered with the kernel contract. Not applicable: no GPU kernel changed.
• Prompt processing is no slower than the last release. Long-prompt/last-release gate remains unrun.
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• Baseline accumulation/snapshot failures are retained in the linked receipts; the current 37 focused tests pass in all three modes. Dependency-free Zig tests run directly; the layout assertions also match the original base.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

Authored by JRB AI in a private capacity. This contribution does not represent the views or positions of any organisation.