Repository navigation
lanes: a shared round drafts only while drafting measures faster (Model.shared_guard) - #1379
BlivionIaG wants to merge 4 commits into
Conversation
engine.zig is 592 lines and the shared-round guard needs room in it. pipelinedRound and landInflight move unchanged to pipelined.zig as round and land; Engine.familyRound and Result become public for them. No behaviour change.
…el.shared_guard) A drafter whose hold is a pass of its own costs a shared round more than the depth rule's head steps price: the rule treats the rest of a round's time as overhead that plain rounds pay too. Facts.shared_guard (and Model.shared_guard for a backend) lets the engine compare the two modes directly. shared_guard.zig keeps, for each stream count, committed tokens over the round's wall time in plain stretches (no budgets, so no holds) and in drafted stretches. It starts plain, measures plain rounds for 16 rounds and drafted ones for 4, runs the faster mode for 512 rounds and doubles that, up to 64 times, while the same mode wins again; then it measures the other mode again. Drafted rounds must beat plain ones by 5%. A mode's first round still carries the other mode's drafts or holds and is not measured. An entry is allocated once a stream count; later rounds take no allocation. A plain stretch changes only how many drafts a window carries, never what the target draws, so replies stay the same tokens. Without the flag the engine never consults the guard. Tests: the guard's choice and re-measures, ever rarer re-measures of a losing mode, its skipped first rounds, no allocation after a stream count's first round, and replies equal to one-token decode with the guard on and off.
… stream joining one holds no drafts With the shared-round guard on, a plain round still verified the lane core's context and Recall copies, which are drafts too, priced from a prior; and a stream joining other streams held its first drafts even while the guard ran plain rounds at that stream count, a hold pass for each join. The engine now marks a shared round plain when the guard runs that stream count plain, and copyProposal proposes nothing for it, so the guard's plain mode is the drafts-off round. Open.firstDepth gives no drafts to a stream that joins live streams while the guard runs their count plus one plain. Both are behind Model.shared_guard; without it plain_round stays false and firstDepth is unchanged. Tests: a guarded plain round proposes no copies; a stream joining plain-guarded rounds holds no first drafts and, without the flag, holds them as before.
ashhart
left a comment
ashhart
left a comment
There was a problem hiding this comment.
Thanks @BlivionIaG. The shared guard keeps adapting even with `Config.timed = false`, which promises table-only pricing independent of wall clock (`lanes/config.zig`; the guard is not gated in `lanes/engine.zig` planning and observation). Please disable adaptive observation/selection in that mode and test identical schedules under different fake clocks. The guard also deliberately creates plain (undrafted) stretches; the native recipe expects drafting for every admitted stream, so that policy needs a decision before it is on by default. (The invalid-timing crash fix is good.) This stack also needs alternating A/B on current main for every family the shared lane core serves.
…ule does not depend on the clock The guard planned, observed and chose with the wall clock even when timed was false. Round planning, joining a shared round and observation now skip it in that mode. A test runs the same three streams on a fast and a slow fake clock and checks equal holds and replies.
|
Thanks @ashhart. Fixed in On the plain stretches: the guard runs a shared round plain only where drafted rounds measure slower than plain ones, and measures drafting again after a wait. DSpark's own prefix scheduler (Section 3.2.2, Algorithm 1 of arXiv 2607.05147) also admits drafted tokens only while the estimated throughput rises, and it reports that the average verification length decreases with load (Section 5.4). The case for and against this is in #1374, Question 1; I will follow your decision there. Guard head against main, M4 Pro, LFM2 4 streams, 4 alternated rounds: within 0.5% of main, tokens equal. The guard runs only for the LFM2 DSpark drafter, so other families never reach it. |
Branch
series/lanes-guard, head4fa02928a: 4 commits on mainc0c7bd24f(the last one is thetimedfix). Sits on main. Part of #1374.What this changes
A drafter whose hold is a pass of its own may set
Facts.shared_guard(a backend:Model.shared_guard). The enginethen measures, for each stream count, committed tokens a ms of plain shared rounds and of drafted ones, and runs the
faster mode. It starts plain, measures the slower mode again after 512 rounds, and doubles that wait (up to 64 times)
while the same mode keeps winning, so drafting is never switched off for good. A guarded plain round proposes no
copies either, and a stream that joins while the guard runs plain holds no first drafts. Off by default.
The first commit only moves
pipelinedRoundandlandInflightout of engine.zig (592 lines, 545 after the series);no behaviour change.
Nothing on main sets the flag. It lands with the LFM2 DSpark drafter PR. That PR measures the guard on the drafter at
4 and more streams.
Receipts at this head
With
Config.timed = falsethe guard no longer plans, joins a shared round or observes (engine.zigtwice,open.zigonce). A test runs the same three streams on a fast (0.5 ms) and a slow (400 ms) fake clock and checks equal holds and replies, and equal holds to a run without the guard.zig build test: 500 of 500 steps passed;zig fmt --checkclean; lean 433, the same as the base's.Tests that fail before the change:
245 pass, 1 fail, "the guard changes only which shared rounds draft"(
on.holds < off.holds).246 pass, 2 fail, the engine_test copy check and "a stream joiningshared rounds that the guard runs plain holds no first drafts".
A/B on current main
M4 Pro, LFM2.5-24B-A2B 4-bit, no drafter, 4 streams, a different prompt on each, greedy 256 tokens, 4 alternated rounds, a fresh server per cell, a quiet machine (load under 2, no Zig build), every build merged with main
8c36359d5; aggregate tok/s by round:Every build is within 0.5% of main and every reply token-equal to main's. An earlier single-round run showed #1378 at -6% and #1379 at -4%; four rounds showed nothing, so those were session effects.
Not measured
Related PRs
Sits on main. Merges cleanly with #1378 and #1380. the first commit moves code out of
engine.zig, which gives it room for later lane PRs. The DSpark drafter PR (#1391) sits on it.