Skip to content

lanes: a copy window may be priced by its stream's own copies (Model.own_copy_rates) - #1380

Open
BlivionIaG wants to merge 1 commit into
ashhart:mainfrom
BlivionIaG:series/lanes-copy-rates
Open

BlivionIaG wants to merge 1 commit into
ashhart:mainfrom
BlivionIaG:series/lanes-copy-rates

Conversation

@BlivionIaG

@BlivionIaG BlivionIaG commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Branch series/lanes-copy-rates, 1 commit on main 1e718458f: 1e718458f. Sits on main. Part of #1374.

What this changes

A shared round prices every copy window's rows (context and Recall copies) at cfg.copy_rate, a fixed 0.94. With
Model.own_copy_rates a copy window is priced from the stream's own decayed copy history, by source, starting from
four rows of that prior. A stream whose copies keep breaking then stops paying verify rows for them. Off by default:
allocation prices copies at copy_rate as before, and no history is kept.

In the DSpark work this change was not opt-in: it changed every family's shared-round copy pricing. Here it is behind
a family flag. Nothing on main sets it. To keep the DSpark work's behaviour, the LFM2 PR has to set it for LFM2.

Measured with the flag forced on for every family (l5on, M5 Ultra, three rounds each, on the code of 1e718458f):
Qwen3.5-2B and Nemotron-3.5-Lightning give the same tokens as main in every cell; the medians move -0.4% to +2.1%
(Nemotron code4 +2.1%, main spread 1.3%, l5on spread 3.0%). Qwen3.8-27B with DFlash2: tokens equal; its speed is within the build-to-build variation shown in #1382 (a second build of main differs by 1.0%). This is not a claim of benefit for other families; the flag stays off for them.

Receipts at this head

  • Unchanged by the review; mergeable with main. The field it adds is read only when the LFM2 DSpark drafter sets own_copy_rates, so this head was not run on its own.

  • Test that fails before the change (copied_rate_test.zig, allocation pricing copies at copy_rate as main does):
    242 pass, 1 fail (rows[0] <= 1). With the change: 243/243.

  • Host gates (fresh cache):

fmt exit=0 files=6
diff-check exit=0
lean exit=1 4083 files, 433 problems
lanes exit=0 secs=32 Build Summary: 3/3 steps succeeded; 243/243 tests passed
lane-api exit=0 secs=11 Build Summary: 3/3 steps succeeded; 297/297 tests passed
lane-lone exit=0 secs=2 Build Summary: 3/3 steps succeeded; 19/19 tests passed
lane-halt exit=0 secs=2 Build Summary: 3/3 steps succeeded; 18/18 tests passed
lane-interrupted exit=0 secs=3 Build Summary: 3/3 steps succeeded; 18/18 tests passed
root-copy exit=0 secs=2 Build Summary: 5/5 steps succeeded; 31/31 tests passed
server-cpu exit=0 secs=133 Build Summary: 5/5 steps succeeded; 792/795 tests passed (3 skipped)
golden exit=0 secs=83 Build Summary: 5/5 steps succeeded
cuda-host exit=0 secs=60 Build Summary: 258/258 steps succeeded; 2021/2033 tests passed (12 skipped)
check-cuda exit=0 secs=3 Build Summary: 7/7 steps succeeded
consumers exit=0 secs=98 Build Summary: 7/7 steps succeeded
hip-host exit=0 secs=3 Build Summary: 8/8 steps succeeded; 52/52 tests passed
hip-qwen35 exit=0 secs=5 Build Summary: 4/4 steps succeeded; 32/33 tests passed (1 skipped)
hip-check exit=0 secs=4 Build Summary: 5/5 steps succeeded
native exit=0 secs=130 Build Summary: 57/57 steps succeeded
dev-tools exit=0 secs=264 Build Summary: 213/213 steps succeeded

Lean: 433 problems, 433 on main, 0 new.

A/B on current main (other families and builds)

M4 Pro, LFM2.5-24B-A2B 4-bit, no drafter, 4 streams, a different prompt on each, greedy 256 tokens, 4 alternated rounds, a fresh server per cell, a quiet machine (load under 2, no Zig build), every build merged with main 8c36359d5; aggregate tok/s by round:

build round 1 round 2 round 3 round 4
main 193.4 194.1 193.3 193.2
held (#1378) 194.8 192.9 193.5 193.4
guard (#1379) 193.2 193.4 193.5 193.3
chances (#1381) 193.1 193.3 193.4 193.3
lone guard (#1382) 193.3 194.0 193.2 193.0

Every build is within 0.5% of main and every reply token-equal to main's. An earlier single-round run showed #1378 at -6% and #1379 at -4%; four rounds showed nothing, so those were session effects.

Not measured

  • M1, M2 and M3 Macs. Nothing here depends on chip-specific kernels (this PR changes no kernel).
  • CUDA and HIP at run time: only host tests and compile checks. Linux CI is run by the PR checks.

Related PRs

Sits on main. Merges cleanly with #1378 and #1379. The DSpark scheduled PR (#1392) sits on it.

…own_copy_rates)

A shared round prices every copy window's rows at cfg.copy_rate, a fixed prior. A stream whose copies keep
breaking (a different prompt on each stream, say) keeps paying verify rows for them.

With Model.own_copy_rates a copy window is priced from the stream's own decayed copy history, by source
(context or Recall), starting from four rows of copy_rate's prior; each verified copy window teaches its
source's history. Off by default: without the flag allocation prices copies at copy_rate as before and the
history is not kept.

Test (copied_rate_test.zig): four streams' seven-row copy windows in one shared round, through
windows.allocate. With the flag, after its context copies land one row of seven, a stream gets at most one copy
row where the prior gave seven, the other streams keep theirs and its Recall copies keep copy_rate; without it
every copy keeps seven rows and no history is kept.
@ashhart

ashhart commented Oct 26, 2026

Copy link
Copy Markdown
Owner

Thanks @BlivionIaG. This one reviewed clean on its own: default-off, no source defect found in its commit. It lands after a device check showing unchanged tokens and a per-family A/B.

@BlivionIaG

BlivionIaG commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks @ashhart. No code change; the branch is mergeable with main. The device check passes on an M4 Pro (see #1378). This head was not run on its own: the field it adds is read only when the LFM2 DSpark drafter sets own_copy_rates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants