Skip to content

kernel contract: Nemotron's pocket kernels (nemotron_ops.cu) - #1629

Merged
ashhart merged 2 commits into
ashhart:mainfrom
BobClawblaw:nemotron-pocket-contract
Oct 30, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
BobClawblaw:nemotron-pocket-contract

Conversation

@BobClawblaw

Copy link
Copy Markdown
Contributor

What this changes

Registers Nemotron's pocket kernels (nemotron_ops.cu, the learned neurons the living-weights learner adds beside the
shared expert) with the kernel contract: relu^2 up into act rows and down into the shared expert's slot, block-row and
tiled, down to fp32 and bf16, against float64. Neurons past the count are NaN so a read of one shows. Two controls:
the up's relu by fmaxf (the bug the previous PR fixes) and the down launched one neuron short.
tf-cuda-test pocket-contract runs the set alone. Stacked on #1628.

Receipt

  • Environment: one GB10 (sm_121), nvcc 13.0 and 13.2.
  • SUMMARY pocket: 6 kernels, 6 pass gates 1-4, 0 fail, 0 known findings; 2 controls: 2 red as planted, 0 missed
  • Gate-5 records for sm_80/86/89/120/121 (pocket_records.zig); nvcc 13.2 gives tf_pocket_in 39 registers on
    sm_120/121, recorded as its override.
  • Full contract on the stack's top: SUMMARY 452 kernels: 452 pass gates 1-4, 0 fail, 6 known findings; 325 controls: 325 red as planted, 0 missed; 321 limit problems (the stack on main d33dcfc; none of the limit problems is in this stack's launches). The stack then rebased onto 3f0a715 without conflicts; the three sets, zig build test and the Transcribe clips were re-run there.

Overlap

atc overlap lists #1445 and #1446 as conflicting, in the shared registration lists (registry.zig, cuda_kernels.zig, kernels.zig, kernel-contract.md, tests/cuda/main.zig); whichever lands second rebases.

BobClawblaw and others added 2 commits October 30, 2026 22:27
…e shared expert's slot, block-row and tiled, fp32 and bf16 out), gates 1-5 with two controls
@ashhart
ashhart force-pushed the nemotron-pocket-contract branch from 8be0103 to 3c53eb9 Compare October 30, 2026 22:28
@ashhart
ashhart merged commit 31ebc29 into ashhart:main Oct 30, 2026
8 of 9 checks passed
@ashhart

ashhart commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Thank you @BobClawblaw. This is in TensorFold 1.2.0, released today: https://github.com/ashhart/TensorFold/releases/tag/v1.2.0

It landed with your commits under your own name, so you're listed as a TensorFold contributor, and it's credited to you in the release notes and the changelog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants