Repository navigation
Zig CUDA: Kolibri-1 (kolibri1), block-FP8 #481
Description
This work is underway; please don't start a second Zig port of this family.
Kolibri-1 (model_type: kolibri1, 50 layers of 2560, 48 query and 4 KV heads of 128, 384 routed experts of width 512 plus a shared one, top 6 by unnormalised sigmoid weights, sliding window 513 with every fifth layer full attention) ships as block FP8 (128 x 128 e4m3, fp32 weight_scale_inv). We serve it on our fork's Python engine on a GB10, with an EAGLE-3 drafter (josch15366/Kolibri-1-EAGLE3-drafter).
The Zig CUDA engine has no FP8 path yet, so the plan is small PRs in this order, each with its receipt:
- Block-FP8 projections: the Python lane matmul's FP8G device code (
nvfp4/qmmf.cu) aszig/kernels/cuda/fp8_lane.cu, a launcher with the Python host's tile, slice and cluster choice, and atf-cuda-test fp8-laneoracle. Model-free. - Block-FP8 experts:
fp8/experts.cuthe same way. Model-free. - The
kolibri1family from the command line, drafts off: tokens equal to the Python engine's. - The family served through
tensorfold-native, as an entry in the CUDA registry. - The drafter behind
Drafted(lanes: an external drafter behind any target backend #455), once 4 and lanes: an external drafter behind any target backend #455 are in.
Branch: jschmied/TensorFold:kolibri1-zig. 1 is up as a pull request with its receipt.
Activity
ashhart commentedon Oct 10, 2026
Thank you, @jschmied. The first piece is in: #482's block-FP8 lane projections landed on main (128 x 128 e4m3 blocks with fp32 scales, equal byte for byte to the Python engine's on a GB10), and #455 put the external drafter into the lane core for the drafting step. #587, your quantized-weight interface, is in our gates now. The Kolibri-1 family comes next on top of it, and this issue stays open as the port's tracker.
ashhart commentedon Oct 17, 2026
Thank you, @jschmied, for splitting Kolibri's block-FP8 operators from the family and drafter work.
The block-FP8 infrastructure is present in 1.0.12 (
zig/src/cuda/fp8.zig:1), butkolibri1is still absent from the CUDA family registry (zig/src/native/cuda.zig:18). This remains the native port tracker, with your work as its starting point.The next slice is the plain family on those shared operators, then serving through the lane host, followed by the EAGLE-3 drafter. Please bring the current native family PR here when it is ready, with the checkpoint pin and plain-reference checks; we will review that before qualifying drafting.
Posted by Ash's AI Assistant Synthena
jschmied commentedon Oct 17, 2026
@ashhart, the plain family is in draft #991: Kolibri-1 (research checkpoint) on the shared operators, serving through the lane host, with drafts from the prompt's text only. The EAGLE-3 drafter is refused by name for now.
The checkpoint is pinned at
e52eb462. Against the plain reference (the Python engine it was ported from), the replies are not byte-identical: 1 of 7 greedy replies is equal over 160 tokens, and the other six part at near-ties where the top-2 margin is 0.003 to 0.137 logits. The shared operators round differently from the Triton glue, and the top-6-of-384 routing amplifies that. The per-prompt table, float64 checks against the checkpoint's own tensors, NLL (3.574 vs the Python engine's 3.578) and the exactness receipts are in the description.The family stacks on #746, #797 and #798.
ashhart commentedon Oct 17, 2026
Thanks @jschmied for #991 and for disclosing the one-of-seven greedy match with the Python reference. Our source review supports carrying the text family on the shared CUDA operators, lanes and prompt cache, with the corrected memory admission.
Shared copy drafts are already verified by the target; EAGLE-3 remains explicitly refused. We should not describe that as general neural-drafter support or general Python/native equality. Keep the layer references and numerical differences visible, then rerun the merged checkpoint's resource, draft/plain, resume/fresh and concurrent/solo gates. EAGLE-3 can follow through the shared drafter contract with its own feature and state checks.
Posted by Ash's AI Assistant Synthena
ashhart commentedon Oct 19, 2026
Thank you @jschmied for Kolibri-1 and the disclosed one-of-seven greedy match with the Python reference. 1.1.1 includes #991's scoped native CUDA text port, using shared operators, lane scheduling, copy drafting and prompt caching. That completes the agreed text-family port. EAGLE-3, a neural drafter and general Python/native token identity remain separate, so this closure does not claim those features.
Posted by Ash's AI Assistant Synthena