Repository navigation
lanes: an external drafter's chain chances price verify rows; holds follow the grants and pay for their pass - #1381
Open
BlivionIaG wants to merge 7 commits into
Open
BlivionIaG wants to merge 7 commits into
BlivionIaG wants to merge 7 commits into
Conversation
engine.zig is 592 lines and the shared-round guard needs room in it. pipelinedRound and landInflight move unchanged to pipelined.zig as round and land; Engine.familyRound and Result become public for them. No behaviour change.
…el.shared_guard) A drafter whose hold is a pass of its own costs a shared round more than the depth rule's head steps price: the rule treats the rest of a round's time as overhead that plain rounds pay too. Facts.shared_guard (and Model.shared_guard for a backend) lets the engine compare the two modes directly. shared_guard.zig keeps, for each stream count, committed tokens over the round's wall time in plain stretches (no budgets, so no holds) and in drafted stretches. It starts plain, measures plain rounds for 16 rounds and drafted ones for 4, runs the faster mode for 512 rounds and doubles that, up to 64 times, while the same mode wins again; then it measures the other mode again. Drafted rounds must beat plain ones by 5%. A mode's first round still carries the other mode's drafts or holds and is not measured. An entry is allocated once a stream count; later rounds take no allocation. A plain stretch changes only how many drafts a window carries, never what the target draws, so replies stay the same tokens. Without the flag the engine never consults the guard. Tests: the guard's choice and re-measures, ever rarer re-measures of a losing mode, its skipped first rounds, no allocation after a stream count's first round, and replies equal to one-token decode with the guard on and off.
… stream joining one holds no drafts With the shared-round guard on, a plain round still verified the lane core's context and Recall copies, which are drafts too, priced from a prior; and a stream joining other streams held its first drafts even while the guard ran plain rounds at that stream count, a hold pass for each join. The engine now marks a shared round plain when the guard runs that stream count plain, and copyProposal proposes nothing for it, so the guard's plain mode is the drafts-off round. Open.firstDepth gives no drafts to a stream that joins live streams while the guard runs their count plus one plain. Both are behind Model.shared_guard; without it plain_round stays false and firstDepth is unchanged. Tests: a guarded plain round proposes no copies; a stream joining plain-guarded rounds holds no first drafts and, without the flag, holds them as before.
… holds follow the grants A drafter that gives each held chain draft's chance (Drafter.VTable.chances) may set Facts.chances. Drafted.facts then sets Model.draft_probabilities and Model.granted_depth, so the window allocator prices each stream's verify rows by those chances, as it does a head's node chances. With Facts.confidence_floor it instead verifies the drafts at least that likely (a fixed window when 0). With granted_depth a stream's next hold is as deep as its last grant plus a slack: one more draft when the grant took every held draft, none when it stopped short of them (Stream.grant_short), and after a grant of none a probe of draft_slack drafts at the depth rule's probe interval. Facts.slack sets one slack for all three. A shared round's granted depths are cut so the drafting streams' blocks fit one hold pass of batch_rows. When the shared-round guard starts a drafted stretch, its first round grants every stream one draft, so the stretch measures drafted rounds rather than plain ones left by grants of none. All of it is behind Facts.chances and Model.granted_depth, off by default; windows.allocate records grant_short only for granted depths. Tests: chances price the chain's rows and drafted replies equal one-token decode alone and at 1, 4 and 8 shared streams, with the scheduled verify ragged across streams and verifying fewer rows than a fixed window; the facts reach the lane config; granted depths, their slacks and the one-pass cut; holds as wide as the grants.
…py, and holds kept only while they pay The depth rule asks one head draft for a stream with a copy ahead, and a shared round's granted holds were priced only by their verify rows. A drafter whose hold runs as a pass of its own (Facts.copy_first) pays that pass for the one draft, and the copy round then drops it; Model.copy_first now asks it for none. A drafter may also give its hold pass's timed cost by blocks and depth (Facts.hold_costs, priced by hold_price.zig). With granted depths a shared round then keeps granted holds, most expected drafts a row first, only while each raises the next round's expected tokens a ms over its verify rows and the pass; a probe after a grant of none keeps its hold. A stream's last grant records the drafts it expected to keep (Stream.grant_gain). Both are off by default (copy_first false, hold_costs empty). Tests: with copy_first no hold is asked for a copy round and replies equal one-token decode at 1 and 3 streams; a hold's cost between and past its timed passes; granted holds kept while they pay, none for a dear pass, and a probe's hold kept.
BlivionIaG
marked this pull request as ready for review
October 26, 2026 12:24
ashhart
requested changes
Oct 26, 2026
ashhart
left a comment
ashhart
left a comment
Owner
There was a problem hiding this comment.
Thanks @BlivionIaG. Its own chain/grant/hold pricing is opt-in and fine, but it inherits #1379's `timed = false` and plain-stretch issues. This stack also needs alternating A/B on current main for every family the shared lane core serves.
…ule does not depend on the clock The guard planned, observed and chose with the wall clock even when timed was false. Round planning, joining a shared round and observation now skip it in that mode. A test runs the same three streams on a fast and a slow fake clock and checks equal holds and replies.
Contributor
Author
This was referenced Oct 26, 2026
Owner
|
Thanks @BlivionIaG. The pricing here is the lane work we want: the window allocator taking each stream's verify rows What blocks it is its base. The branch sits on #1379, which we are not taking (see there), and it reaches into the Smaller points:
The A/B in the description ran on main |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Branch
series/lanes-chances, head91b0cf2a5: 2 commits onseries/lanes-guardand a merge of the #1379 fix. Sits on the guard PR. Part of #1374.What this changes
Facts.chances(withDrafter.VTable.chances): the window allocator prices each stream's verify rows by thedrafter's chain chances, as it prices a head's node chances.
Facts.confidence_floorgives a fixed window instead.Model.granted_depth: the next hold is the last grant plus a slack (one when the grant took every held draft,none when it stopped short, a probe after a grant of none).
Facts.slacksets one slack. Granted depths are cutso the drafting streams' blocks fit one hold pass. The guard's drafted stretch starts from a grant of one.
Facts.copy_first: no hold for a stream whose next round is a copy.Facts.hold_costs(hold_price.zig): a shared round keeps granted holds only while they raise the next round'sexpected tokens a ms over their verify rows and the hold pass.
All off by default.
windows.allocaterecords the new grant fields only with granted depths. Nothing on main setsthese facts; it lands with the LFM2 DSpark drafter PR (its confidence head is the chances).
Receipts at this head
Carries the lanes: a shared round drafts only while drafting measures faster (Model.shared_guard) #1379
timedfix (merged in).zig build test: 500 of 500 steps passed; lean 433, the same as the base's.Tests that fail before the change:
249 pass, 6 fail(the five chances tests and theone-pass cut).
256 pass, 3 fail.A/B on current main (other families and builds)
M4 Pro, LFM2.5-24B-A2B 4-bit, no drafter, 4 streams, a different prompt on each, greedy 256 tokens, 4 alternated rounds, a fresh server per cell, a quiet machine (load under 2, no Zig build), every build merged with main
8c36359d5; aggregate tok/s by round:Every build is within 0.5% of main and every reply token-equal to main's. An earlier single-round run showed #1378 at -6% and #1379 at -4%; four rounds showed nothing, so those were session effects.
Not measured
Related PRs
Sits on #1379. Merges cleanly with #1378 and #1380. #1382 sits on this PR.