[jschmied]

Branch hostfix-prompt-cache, commit 74be69cd3 (on main c0c7bd2).

victimBut asks extended() about every candidate. extended() compares the candidate's tokens with every other entry's (prompt_cache_policy.zig:21-25, called from victimBut at :9). That is O(E²·L) per victim, and eviction loops (reserve, keepBorn, giveBack, recall) pick several. For a 27B the slot count is unbounded (most), so E reaches hundreds. Entries of one conversation share long prefixes, so each compare runs almost the whole length before it mismatches.

Changes, all bookkeeping, with the same choices:

• Each entry keeps ext, the entries that extend it: longer, its tokens as their prefix, and born later. Links are made once when an entry joins the store (keepBorn, recall, rows_read) and dropped when it leaves (remove). tokens and born never change after that, and expired is read when asked, so extended() gives the same answer as the scan. If a link allocation fails, extended() falls back to the scan.
• victimBut compares each entry's tokens with the prompt once and uses the result in both passes. Before, it compared them again for every pair in the oldest scan (before → isActiveRewind twice).
• keepBorn copies the prompt once instead of twice (prompt_cache.zig:482/:486). The state's tokens are a prefix of the kept last, and a later resume replaces last as before.

Test: prompt_cache_scan_test.zig runs random stores at 24 and 200 slots. Prompts share long prefixes from a small vocabulary. Each run does keeps at random positions and births, lookups that resume or fail, and expiries. After every operation it checks victimBut (random skip, prompt, born and active rewind) and extended() against copies of the old scans. Over 1000 evictions. Broken variants fail it: links for equal births, or dropping the rewind tiebreak.

Measured on the previous base (ReleaseFast, host only; one conversation of 220 turns, 8k to 30k tokens, every other turn a fork, 200 slots, 3 runs):

	main	this
victimBut	20-22.5 ms	0.50-0.51 ms
keepBorn (evicting when full)	1.6-1.9 ms	0.27-0.37 ms
lookupRewind	1.2-1.4 ms	0.27 ms

Gates: full zig build test -Dnvcc exits 0. Lean: 0 new. Served on GB10: identical replies (61 of 61 replies match main c0c7bd2 (Qwen3.6-35B-A3B, 8 lanes), drafted==plain 8/8, concurrent==solo 15/15, resumed==fresh).

Overlap: #1291, #1254, #1241 and #1240 touch prompt_cache.zig. #896 touches prompt_cache_policy.zig.

Metal (M-series Mac, zig 0.17.0): zig build test passes (RC=0). A served Qwen3.5-2B smoke run (strict JSON schema solo and two at once, tool_choice required, a three-turn chat with turn 3 repeated, a plain prompt, temperature 0) is byte-identical to main c0c7bd2.

[ashhart]

Landed on main in 8f0885b (changelog in 52f5d95). On an M3 Ultra with Qwen3.8-27B, decode at 1 to 8 streams and chat were unchanged against main. Thank you, @jschmied.