Repository navigation
Conversation
…e_attention.cu, block_attention.cu), checked against core/attention_ref.zig
… merge combines them in order; a slice takes its own one-warp block where four-warp blocks would leave SMs idle (TreeAttention.sms), the same bits
…lves again, only one-warp blocks leave them to the merge (a small fixed slice area), so the scratch stays at one partial a group
…async stages), the same arithmetic in the same order
…w), partials sized to the query heads, one-warp blocks with 3 tile stages where they fit one wave of the SMs, else 2
…ts) before folding them, the same arithmetic in the same order
… gates, DFlash2's conv, Gated DeltaNet inputs, QK norm and RoPE), checked against float64
…ot, correctly rounded on every architecture
…iton's tuned sizes): swiglu 128 and gate 64 threads, gdn_prep a warp a block pair; the same bits at any row count
…, and dense rows for every family, checked against float64
# Conflicts: # zig/src/core/root.zig # zig/tests/cuda/main.zig
…te value there stays out of its sum A block whose output holds a NaN or infinity runs again with the values of the tiles some row cannot see all of taken out of the mma; each row adds back the ones it sees. On finite values the first pass is the kernel as it was, byte for byte.
…te value there stays out of its sum A block whose output holds a NaN or infinity runs again with the values of the tiles some row cannot see all of taken out of the mma; each row adds back the ones it sees. On finite values the first pass is the kernel as it was, byte for byte.
…ax flag, so a NaN head leaves the other heads' keys
…ld zeros, not a copy of the window's first key, so an infinite value there reaches no sum
…, so a later row's infinite value stays out of earlier rows' sums A block whose output holds a NaN or infinity runs again, out of line, with the values of the tiles some row cannot see all of taken out of the mma; each row adds back the ones it sees. On finite values the first pass is the kernel as it was, byte for byte.
…block-FP8 layouts (cuda/fp8.zig, qlinear, fp8g experts)
…rows, norms, sigmoid routing, grouped FP8 experts), one-row step graphs
…) and prompt-cache states at any row
…--parallel streams at open, no-lookahead states
…w edge, full layers, steps, drafted, shared rounds, cuts, kept states
…scale (a dropped routing bias or swapped expert scales pass 0.7 %)
…i1-family # Conflicts: # zig/src/core/contract/registry.zig # zig/tests/cuda/contract_tests.zig
…mily # Conflicts: # zig/src/core/contract/registry.zig # zig/tests/cuda/contract_tests.zig
…sion, the shared operators it runs on, checks, speed
… flag (a head with no key keeps m, exp2 of +0 is exactly 1), the old kernel's speed
…(sliding rings hold a pass and the window, 8,768 keys); cuts of 5,000 and 20,000 rows check against them
There was a problem hiding this comment.
Both admission findings from this historical review are addressed at 179648432b561e986a2ec83951360b49749e114a, verified in a source/CPU follow-up: #991 (review) . No material admission finding remains in that scope. Device/model qualification remains separate.
Original review scope:
Two admission issues remain in the Kolibri/native integration below. I checked the shared registry/options, refusal and snapshot hooks, the configuration checks and the memory-admission call path, with an independent Astra source review.
The CPU probe imports the actual cuda_memory.Pool and admit; its memory sizes are synthetic examples. It demonstrates the admission arithmetic, while the connection to Kolibri's allocations and slot ownership comes from source inspection. The three configuration tests pass in Debug and ReleaseFast. No device allocation, kernel, model-quality or performance validation was performed here. The four current-head CI checks are green.
| errdefer own.s.deinit(); | ||
| const context = positions(o.context); | ||
| const n = streams(o.lanes); | ||
| own.m = try fwd.Model.init(gpa, ctx.d, own.s, &own.w, &own.k, context, n, be.prompt_rows); |
There was a problem hiding this comment.
[P2] Budget the prepaid caches and scratch before allocating them
openWith checks only weight bytes before F.open. Kolibri has no Options.room hook and allocates all requested stream caches plus scratch here, then returns them as free_streams. If those allocations cross TENSORFOLD_CUDA_MEMORY_LIMIT_GB or the reserve, the later after.room(model) just becomes zero; budget.admit(0, stream_bytes, 8, 8, true) still accepts all eight prepaid slots. For example, the CPU probe accepts 70 GiB of weights under a 72 GiB cap, then admits eight slots after synthetic held bytes reach 78 GiB. A budget/size plan is needed before cache and scratch allocation; auto parallelism should allocate only what fits, and a fixed oversized count should fail before consuming the protected memory. The shared admit function itself is behaving according to its prepaid-slot contract.
|
|
||
| /// Streams whose caches the open allocates: the server's --parallel, at most a round's rows. | ||
| fn streams(lanes_asked: u32) usize { | ||
| return std.math.clamp(@as(usize, lanes_asked), 1, be.round_rows); |
There was a problem hiding this comment.
[P2] Refuse a fixed parallel count above the backend's slot capacity
The server CLI accepts --parallel 65, but this clamp creates only 64 slots. If room remains for one stream, generic admission returns 65 (admit(gib, gib, 64, 65, true) in the CPU probe), and the host advertises/adopts 65 lanes. facts.max_streams=64 limits a shared forward, not the host's admitted request count. Once all 64 slots are occupied, backend.opening returns NoFreeSlot for the next admitted request and the host ends it; only LaneMemory is requeued there. Reject the fixed count at open or carry the backend's hard capacity into admission, rather than treating unallocated slots as growable.
…llocating them; refuse a fixed --parallel past the memory or a round's 64 rows; admission adds no lane to a family that allocated its streams at open Model.openBytes counts init's allocations by the same path without allocating; the open reads the server's room, allocates as many streams as asked when they fit, the most that fit under auto, and refuses a fixed count that does not fit or exceeds 64 before any cache is allocated. cuda_memory.growRoom gives a family whose Options carry `lanes` no room to grow into, so the host admits only the streams it holds.
|
@cwschroeder, thank you. Both findings were right. Fixed in native.zig:70, budget before allocating. Kolibri now has the
The planned bytes equal the bytes allocated (12,578,056,556 at 8 streams and 32k). native.zig:48, more than 64. A fixed Tests. Your probe's cases, with synthetic sizes:
Mutation: I put back the old clamp in GB10, real checkpoint:
Exactness is unchanged:
|
cwschroeder
left a comment
cwschroeder
left a comment
There was a problem hiding this comment.
Both admission findings are addressed in 17964843. I rechecked the hook before cache/scratch allocation, fixed-count refusals and the shared host's zero-growth admission for this preallocated family. A fixed 65 refuses before weight loading, and auto cannot add an unowned slot later.
Local Zig 0.17.0 checks: whole-tree formatting passes; test-cuda-host passes 872 tests with 3 skipped (41/41 steps). Focused native tests pass 9/9; shared memory tests pass 6 with one Linux-only skip.
The additional CPU allocation probe and receipts call the actual forward/backend initializers with a fake driver. For 12 synthetic slot/context/cluster combinations, planning makes zero driver calls and its count equals the requested allocation bytes. This covers the accounting path, not device residency or tensor contents.
All four current-head CI checks are green. My two source/admission requests are closed. I did not rerun the author-reported GB10 model/exactness results, so this remains a scoped source/CPU follow-up.
|
Thanks @jschmied for the budget-before-allocation and fixed-slot controls, including the real refusal at the tighter cap. We reviewed 1796484: the source admission findings are addressed, and the text-family increment is suitable for integration. Keep the shared operators, lanes and prefix snapshots, with copy drafting available and EAGLE-3/external head drafting explicitly refused. The unchanged seven served hashes and 80/80 checks do not establish general equality with Python; the disclosed reference differences remain. The final merged source still needs its numerical/resource, copy-draft, cache, cancellation and concurrent/solo device gates. There is no new source correction to send back from this review. Posted by Ash's AI Assistant Synthena |
…ook when a later allocation fails, with a host test that fails each allocation in turn
…ut of the mma and adds it back at its key's float64 weight, so an infinite value gives NaN where float64 does
… value out of the mma and adds it back at its key's float64 weight (attn_odd), so an infinite value gives NaN where float64 does
…ded back before it The replay adds the context's NaN and infinite values back before the block's own tile, whose rescale is exp(m - m') in float32. When a block key holds the row's max and the context's scores sit more than ~103 below it, that factor is 0 and turns an added-back infinity into NaN, where float64 (attention_ref) still weighs it above 0 and gives inf. The replay's block tile now leaves an infinite o as it is (attn_odd::keep); the common path is unchanged. Gate 1 plants the case on stream 1, key head 7: block key 0 all 8, every other key all -8 with +inf at element 23, queries all 2 (inf) and all 8 (float64 weights of 0: NaN).
…values out of the mma, added back at float64's weights A visible infinite value whose weight underflows in float32 (more than ~103 below the max) but not in float64 (745) went through the mma as 0 x inf, or reached a merge at a factor of 0: NaN, where attention_ref gives inf. As in block attention, a block whose partial holds a NaN or infinite value now runs again (ODD, out of line): its V tiles' NaN and infinite values leave the mma (attn_odd::clean_rows), and each pair adds them back after its last tile at the key's float64 weight (attn_odd::add_back), so weight 0 gives NaN. The four-warp combine and the merge fold as before; only a NaN result folds again with the infinities unscaled (attn_odd::fold, keep). tf_tree_shared takes the window's keys (kn) for the path keys' weights. Finite outputs keep their bits. Gate 1 plants key head 1, row 0: committed key p0 - 1 all 8, every other key (committed and its path) all -8 with +inf at element 21, queries all 2 (inf) and all 8 (NaN).
…again with those values out of the sums, added back at float64's weights A visible infinite value whose weight underflows in float32 (more than ~103 below the max) but not in float64 (745) went into a sum as 0 x inf, or reached a fold of partials at a factor of 0: NaN, where attention_ref gives inf. The prompt rows had the replay (462d952); the decode kernels now have it too, out of line: - tf_window_decode_tc: a NaN output (the one wrong class: an infinite output holds a weight above float32's least, so above float64's) runs the block, or the one warp that saw keys, again as ODD. Its V tiles' NaN and infinite values leave the mma (attn_odd::clean_rows), each head adds them back after its last tile at the key's float64 weight (attn_odd::add_back), and the warps' fold keeps them (attn_odd::fold). - tf_window_part: a tile with a NaN or infinite output recomputes those outputs from the finite values, then adds each NaN or infinite value at its key's float64 weight against the row's float64 max over its window (attn_odd::score64 with a run-time head dim). tf_window_merge folds a NaN result again with the infinities unscaled. Finite outputs keep their bits. Gate 1 plants the case in each decode form: a row's key p all 8, its window's other keys all -8 with +inf at element 21, one head querying all 2 (inf), the next all 8 (NaN).
…split.cuh (the file was past 600 lines), comments on one line
…i1-family # Conflicts: # zig/src/core/contract/registry.zig
|
I merged #798's
The Python reference differences are unchanged and still disclosed. The description has these results. |
|
Thank you, @jschmied, for Kolibri-1 and the concurrent/solo and resumed/fresh comparisons. Kolibri moves with its shared CUDA family dependencies to 1.1.1, the next patch release. The reviewed implementation uses the common lanes, prompt cache and corrected memory planning. Shared copy drafting is available; MTP, external drafting and vision remain explicitly refused. It needs the integrated checkpoint's device and resource gates, cache resumes and concurrent copy drafting before release. The supplied comparisons should stay separate from general Python/native equality, which was not established. On the RTX PRO 6000 host, the Kolibri checkpoint verified 40/40 files, while final device exactness, resource, cache and concurrency checks remain owed. Posted by Ash's AI Assistant Synthena |
|
Thanks @jschmied. 1.1.1 ships Kolibri-1's native CUDA text-family port through the common lane scheduler, prompt cache, shared rounds and verified copy drafts. Closing that scoped family contribution; EAGLE-3, a neural drafter and vision remain unsupported. The later d2f6230 merge leaves Kolibri's family files unchanged and brings in #746/#798's shared attention repairs. We'll carry those once into 1.1.2 with your credit and their final-image plus real Kolibri exactness, timing and memory checks. General Python/native token identity is not claimed. Posted by Ash's AI Assistant Synthena |
What this changes
Kolibri-1, a research checkpoint, on native CUDA as a plain family behind the lane host (#481). It is registered in the CUDA family registry as
kolibri1(fp8-block) and serves throughtensorfold-nativewith shared rounds, copy drafts from the prompt's text and the prompt cache. The EAGLE-3 drafter is not in this PR: head drafts and--drafterare refused by name.The family brings no kernels of its own. Every launch is a shared operator: block-FP8 lane projections through
qlinear(cuda/fp8.zig), groupedfp8gexperts with the shared expert stacked last,qk_rows,norms,embed_rows,moe_glue,dense_rows,window_attention(prompt rows of every layer, sliding decode rows) andtree_attention(full layers' decode rows).Stacked on #798 (
cuda-moe-dense-windowat74b749fa, which carries #746's fixes; merged asd2f62303), #746 (cuda-attn-sharedatee81000d, merged; #798 carries its later commits up tob502747c) and #797 (cuda-glue-rowsat31d3baf4). Not on #734: windows are chains, so no tree rounds are needed. Base: main6ee828c2(1.0.12 plus docs). Only the commits below are this PR's.zig/src/families/kolibri1/:3c7a1860);de56c8dc);prefillandprefill_step, shared verify windows of up to 16 rows a stream and 64 a round,keep, keyed draws; prompt-cache states at any row (full layers' keys, each sliding layer's last 512) (0a267b0b);7633646a);17964843): the open reads the room the weights leave (Options.room, as GLM does) and plans the streams before allocating caches or scratch.Model.openBytescountsinit's allocations by the same path, allocating nothing. Auto allocates the most streams that fit, up to 8. A fixed--parallelthat does not fit, or is above 64 (a round's rows), is refused at open, the latter before the weights load.6498344d):kolibri1innative/cuda.zig's registry; a family with anOptions.lanesfield gets--parallel(Kolibri allocates every stream's caches at open);cuda_refusal.zigtakes a family's owndrafter_refusaltext;cuda_snaps.zigtakes a family'ssnap_lookahead(Kolibri's states need no token past their row). In17964843, such a family also gets--parallel's fixedness, andcuda_memory.growRoomgives it no room to grow into: the host admits only the streams it allocated, never a lane without a slot. Each has a host test.0101e1ff,60d5bdd8,7633646a):tf-cuda-test kolibri-reference,kolibri-layers,kolibri-full,kolibri-greedy,kolibri-logitson the checkpoint; no fixtures.docs/recipes/kolibri-1.md(57755519,7633646a).Refusals, each worded: other quantizations,
norm_topk_prob, other layer kinds, a config with a vision tower, shapes the shared kernels do not take,--visionand--precision(no such option for this family),--drafter, head drafts, structured output, a captured kernel set.Receipt
Environment: this branch at
7633646aon main6ee828c2unless a receipt names another commit (the admission receipts and the re-checks under them are at17964843), one GB10 (sm_121), CUDA 13.0, Zig 0.17.0, ReleaseFast. Checkpoint pin:Aleph-Alpha/Kolibri-1ate52eb4627d11516b0c01de49210ab5a4e4061444, 74 GB block FP8; publisher sha256 ofmodel.safetensors.index.jsonc6769a34820c9057b404b2a5cae1ac24fd7b4a9ca08b84eb0a06e7b5bad3005e, first sharda7a98d7a62ea98d5ea087803ce018f51a15e442bc9a621617c68df7582947542. The shards were not re-hashed for this receipt.Plain reference: the Python engine on our fork (
kolibri1/cuda, Triton glue), the one the family was ported from.Greedy through serving,
/v1/completionswith fixed token ids (6 chat prompts rendered by the checkpoint's template, 54 to 71 tokens, and 2,000 tokens of real text), 160 tokens,"draft": false. The Python engine's own greedy loop gives its server's SHAs on all 7.34e0f8a1b2c034e0f8a1b2c034e0f8a1b2c07a8162e41fb8c1657fc2f1b6c1657fc2f1b642e2eebedcb23b35960140503b3596014050f47ad4367586cc206f8c50adcc206f8c50ad8a51bc7eeb5524865c85fd4c24865c85fd4cbbc25f7ae2ca947b0722b165947b0722b16576a017be68d4e619651b8e23e619651b8e23Served replies equal the family's own one-row reference on all 7. Both native columns re-run at
7633646a: the same SHAs, andkolibri-greedy's output (ids and margins) is byte-identical to the 2,048-row build's. Against Python, 1 of 7 is equal over 160 tokens; the other six part at near-ties (top-2 margins 0.003 to 0.137 logits on Python's side, 0.006 to 0.189 on ours).Logits at prompt rows are not byte-equal (measured at
57755519; the 8,192-row passes leave every row's bits as they were, see the cuts below). Last rows of the 7 prompts: top-1 equal 7 of 7, max |difference| 0.11 to 1.50. Teacher-forced over 2,048 rows of real text: top-1 agrees on 92.3 % of rows, KL mean 0.060 (p99 0.63). Why: the Python engine's glue is Triton (approximateexp, its own sum orders); these operators sum in other orders with IEEE division and square root, so outputs move by an ulp or so; Kolibri-1's top-6-of-384 routing turns that into other experts on near-tied rows, and later layers carry it. On cuda: tree attention and DFlash block attention for every family, checked against float64 #746/cuda: SwiGLU, output gates, Gated DeltaNet inputs, QK norm and RoPE for every family, checked against float64 #797/cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798 alone, each glue change by itself gave KL 0.04 to 0.06 against Triton, the same as all of them.Independent check,
kolibri-reference: layer 0 and the head against float64 computed from the checkpoint's tensors (block-FP8 dequantised in float64, rounded where the engine stores bf16): max |logit difference| 0.0045 to 0.0073 of a 6.6 to 7.8 logit scale (0.11 %), top-1 equal, prompt and decode rows, 3 cases.Quality on real text, at
57755519(kolibri-logits+nll.py, 2,047 rows of our notes text, one 2,048-row pass either way): native NLL 3.5737, top-1 35.96 %; Python engine 3.5784, 35.47 %; earlier Triton-free run 3.586, 35.52 %. Mean entropy 2.354 against 2.358: not softer.Exactness, served (
tensorfold-native,--parallel 8,--context 32768):"draft": false: 7 of 7 completions (no copy matched there); every turn of the 36-request agent run below drafted and plain alike (19 of 19 pairs), with copies landing;bench_concurrent.py --serial54 of 54;bench_concurrent.py --alone --serial --levels 1,4,8 --temperatures 0,1.0 --tokens 256 --reps 3, 156 of 156 equal, 0 failed;tools/zig/reuse_served.py record(prompt cache 8 GiB,--min-prompt 0 --turns 5), 36 requests all cached exactly as planned (history marks, shared system cuts at 4,955 / 6,491 / 7,003, a 1,566-token history, a resume after a cancel), thenreplayon--prompt-cache-gib 0andcompare: 35 of 35 replies equal. Resumes start inside passes (8,246, 12,729, 17,210 ...). All 36 replies equal the 2,048-row build's record;kolibri-full): a 5,000- and a 20,000-token prompt's last logits in 1,000-, 1,536- and 2,048-row chunks equal the 8,192-row passes byte for byte, 6 of 6.Exactness through the lane rounds (
kolibri-full, 128 tokens): eager one-row forwards == step graphs == lane rounds == copy-drafted rounds (77 drafts landed) on a short and a long prompt; sampled drafted == plain; four streams in shared rounds each equal alone; states kept at 2,048, 3,001 and 4,096 by one stream and resumed by another equal fresh. Same hashes before and after merging the cuda: tree attention and DFlash block attention for every family, checked against float64 #746/cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798 NaN fixes, after cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798's flag-free tensor-core decode (7d599181, merged as3dd1d1a1), and with 8,192-row passes (7633646a);kolibri-layersandkolibri-referenceprint the same lines too.Decode,
bench_openai.py --tokens 256 --reps 3(one stream, median,--context 65536 --parallel 8): code 52.0 tok/s greedy, 50.4 sampled; chat 50.3 greedy, 50.3 sampled (2,048-row build3dd1d1a1, same server: 51.7 / 49.9 and 49.9 / 49.9, so unchanged). Aggregate,bench_concurrent.py(--context 32768): 4 streams 124.6 (code) / 126.5 (chat), 8 streams 176.2 / 184.2 greedy; sampled 111.2 / 115.1 and 149.1 / 159.5. No before against another engine: a new family.Cold prompt,
bench_cache.py --require-cold --repeats 3,--prompt-cache-gib 0,--context 65536 --parallel 8, real text, cached 0 on every request,ttft_sthe median of the 4 cold requests. Before: the 2,048-row build of3dd1d1a1, same flags, same idle GPU, measured just before:Same token SHA on every repeat and on both builds. The 1.00 / 5.04 / 21.5 s in this PR's earlier receipt were measured at
57755519. Expected before the run: 14 to 16.5 s at 32k (first written against the 21.5 s above, restated against the 17.98 s baseline before building); it is 16.82 s, just outside. Each 2,048-row pass dropped saves about 0.1 s (0.29 s for 3 at 8k, 1.17 s for 12 at 32k), so reading the experts once more costs a tenth of a 1 s pass: the passes were already bound by the expert and attention math, not by reading the weights. A full pass now holds other streams' rounds for about 3.7 s instead of 1 s.Admission, at
17964843, on the checkpoint (--context 32768):--parallel 8: the same load as at7633646a, model 85.23 GiB, 8 streams at 1.29 GiB; planned bytes equal allocated bytes, 12,578,056,556 (checked once with a temporary log line, not committed).TENSORFOLD_CUDA_MEMORY_LIMIT_GB=79.4, auto: 3 streams, model 78.76 GiB, 0.6 GiB left for the prompt cache.--parallel 8: refused at open,--parallel 8 needs 11.71 GiB at a 32768-token window and the CUDA memory budget leaves 5.89 GiB after the weights: 3 streams fit. The7633646abuild admitted 8 there, with 0.0 GiB left.--parallel 65: refused in 2 s, before the load.17964843:kolibri-fullandkolibri-layersprint the same lines as at7633646a; the 7 served greedy SHAs are equal;bench_concurrent.py --alone --serial --levels 1,4 --temperatures 0,1.0 --tokens 256 --reps 3gives 80 of 80 equal at the same speed (4 streams 124.3 / 126.2 greedy).Memory: weights 73.4 GiB; 1.29 GiB a stream at 32k, 1.92 GiB at 64k (2,048-row passes: 0.83 / 1.45): each sliding layer's ring grows from 2,624 to 8,768 keys. Forward scratch about 1.2 GiB (0.3 GiB at 2,048 rows). At 8 streams and 32k the load reports 85.23 GiB (80.61 with 2,048-row passes). Peak device memory 91.8 GiB (8 streams at 32k, 8 GiB prompt cache in use, after the concurrent and agent runs; 87.2 GiB with 2,048-row passes at
57755519), 128 GB unified.Tests:
17964843, on a fresh cache:zig build test86/86 steps, 1,398/1,406 tests (8 skipped); thentest-cuda-hostwithTF_KOLIBRI_DIRon the same cache, 41/41 steps, 678/678 tests.zig fmt --checkis clean,lean_checkis at 517 and every file is at most 589 lines;17964843), with synthetic sizes as in the review's probe:kolibri1/native.zig: a 72 GiB cap over 70 GiB of weights gives auto 2 and refuses fixed 8; the same through the reserve; scratch counted; no room; 65 refused and 64 taken;native/cuda_memory.zig: 64 held plus room for one refuses a fixed 65; auto with 3 held keeps 3; 8 prepaid stay admitted at zero room.Mutation: the old clamp put back and the room passed through
growRoom. Three tests fail (found 8, 64 and 65);7633646a, on a fresh cache:zig build test86/86 steps, 1,271/1,279 tests (8 skipped);test-cuda-hostwithTF_KOLIBRI_DIRset 41/41 steps, 872/872 tests, 13/13 of the family;tf-cuda-test kolibri-reference,kolibri-layers,kolibri-fullpass;zig build kernel-contract(CUDA, GB10):SUMMARY 60 kernels: 60 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems. No family kernels; the shared ones are cuda: tree attention and DFlash block attention for every family, checked against float64 #746/cuda: SwiGLU, output gates, Gated DeltaNet inputs, QK norm and RoPE for every family, checked against float64 #797/cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798's;zig fmt --check build.zig zigclean;lean_check517, as on main; every file at most 382 lines;git diff --checkclean.Mutations at
60d5bdd8, each applied alone, the runner rebuilt, then restored:kolibri-referencekolibri-layerskolibri-layersi % 5 == 0)kolibri-reference,kolibri-layerskolibri-referencekolibri-referencekolibri-referenceThe first run used a 2 % bound and missed the bias and the swapped scales (0.75 to 1.7 % of the scale); the bound is now 0.3 % (
60d5bdd8), clean rows sit at 0.11 %.After merging cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798's
74b749fa(d2f62303: cuda: tree attention and DFlash block attention for every family, checked against float64 #746's and cuda: sliding-window attention, sigmoid top-k routing and combine, dense rows for every family, checked against float64 #798's review fixes, the float64-weight replay in block, tree and window attention), on the checkpoint:kolibri-referenceandkolibri-layersprint the same lines;kolibri-fullgives the same hashes (eager, steps, drafted, sampled, four shared streams each alone, the six cuts byte-equal, kept states at 2,048, 3,001 and 4,096); served (--parallel 8 --context 32768): the 7 greedy SHAs are equal, drafted =="draft": false7 of 7,bench_concurrent.py --alone --serial --levels 1,4 --temperatures 0,1.0 --tokens 256 --reps 380 of 80 at the same speed (4 streams 125.0 / 126.9 greedy),reuse_served.pyrecord with the same 36 SHAs and replay/compare 35 of 35; admission: planned bytes equal allocated bytes at 8 streams, 12,578,056,556 (temporary log line, not committed), auto underTENSORFOLD_CUDA_MEMORY_LIMIT_GB=79.4takes 3 streams, a fixed 8 there is refused with the same message,--parallel 65is refused in 3 s.zig build kernel-contract:SUMMARY 60 kernels: 60 pass gates 1-4, 0 fail, 0 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems.zig build testfrom an empty cache 88/88 steps, 1,399/1,407 tests (8 skipped);test-cuda-hostwithTF_KOLIBRI_DIR43/43 steps;zig fmt --checkclean,lean_check517.Not run, and why:
kernel-contract-limitson multi-SM fatbins: only a GB10 here, and this PR adds no kernels.Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.SUMMARYline is in the receipt. (No new kernels; CUDASUMMARYabove.)