[jschmied]

Branch hostfix-grammar, commit dc84aa99b (on main c0c7bd2).

The HTTP thread compiles a request's grammar so it can refuse a bad one with a 400 (server.zig compiledStructure). It then freed that compile. The lane thread compiled the same text again under Compiler.mutex (lane_stream.zig:61 → lane_host.zig:393 → lanes/grammar.zig:81). That compile is usually an xgrammar cache hit. The cost is the lock: the lane thread, which runs every stream's rounds, waits while another request's schema compiles on its HTTP thread.

Now api.Structure carries the compiled handle (compiled). The lane host builds the matcher from it with Compiler.constrained, without a compile or the lock. The server frees the handle once the engine has sent finished:

• Server.release(preparing, request) replaces chat.release and frees the grammar. It is called from generate's defer (after the drain) and wherever a prepared request is dropped (compact, cortheon, imprint, openai).
• A required tool's grammar that response_format replaces is freed at once.
• A request refused after its compile frees it (an errdefer in checkFeatures and in prepare).

A request without a handle (other engines, or tests) is compiled as before.

Tests:

• lanes/grammar_handoff_test.zig: the handle gives no second compile, and the masks along a token walk match a fresh compile. A second grammar's masks differ, so the comparison is not vacuous.
• lane_host_grammar_test.zig: a request through the lane host with a handle compiles 0 times on the lane thread, and its masked reply matches the compile path's. If the lane host ignores the handle, the test fails (2 compiles, not 1).
• tool_required_test.zig: each compile is freed exactly once, including the replaced grammar and a refused one.
• A stand-in libtfgrammar (lanes/grammar_fake.zig) counts compiles and frees.

Gates: full zig build test -Dnvcc from the repo root exits 0. Lean: 0 new problems. Served on GB10 (Qwen3.6-35B-A3B MLX 4-bit + MTP, xgrammar v0.2.8, 8 lanes):

• Replies are identical to main for temp 0, sampled, drafted, concurrent at 1/2/4/8, resumed, a JSON-schema request and stop strings: 61 of 61 replies match main c0c7bd2 (Qwen3.6-35B-A3B, 8 lanes), drafted==plain 8/8, concurrent==solo 15/15, resumed==fresh.
• Measured: no decode-rate change (8 streams 257 vs 256 tok/s, single runs on the previous base). The lock wait occurs only when grammars compile concurrently, and this test did not time it.

Overlap: #1295, #1290, #1281, #1268, #1254, #1239, #1235 and #1215 also touch server.zig, chat.zig, engine_api.zig or lane_host.zig.

Metal (M-series Mac, zig 0.17.0): zig build test passes (RC=0). A served Qwen3.5-2B smoke run (strict JSON schema solo and two at once, tool_choice required, a three-turn chat with turn 3 repeated, a plain prompt, temperature 0) is byte-identical to main c0c7bd2.

[ashhart]

Landed on main in 6e90531 (changelog in 52f5d95). On an M3 Ultra with Qwen3.8-27B, decode at 1 to 8 streams was unchanged against main. Thank you, @jschmied.