[BobClawblaw]

What this changes

relu^2 keeps a NaN. Three relu^2 sites used fmaxf(v, 0), which turns a NaN sum into 0, so a NaN in the experts or
the pocket vanished instead of reaching the output. They now use v < 0 ? 0 : v:

• experts.cuh: relu^2 in the grouped experts' up epilogue (MLX affine, ggml, NVFP4 relu^2 forms);
• ck_experts.cu: relu^2 up, and the NVFP4 quantizer's block maximum (max_nan, so a block holding a NaN gets a NaN
  e4m3 scale, 0x7F, as the decode path's bf16 rows and separate quantizer already did);
• nemotron_ops.cu: the pocket's relu at its two sites.

Finite inputs give the same bits. First of five stacked PRs for #1549 (Nemotron 3.5 Transcribe).

Receipt

• Environment: one GB10 (sm_121), nvcc 13.0, TensorFold main d33dcfc.
• SASS: only the relu^2 functions change (13 in experts.cuh, 4 in ck_experts.cu, 3 in nemotron_ops.cu), with the
  same register counts; every Qwen and Gemma kernel is identical code.
• Exactness, main against this branch, greedy and seeded-sampled replies plus an image where the model has a tower,
  identical replies on every model:

Model	Identical replies	Decode tok/s	Long-prompt TTFT
Nemotron 3.5 Lightning NVFP4	12/12	112.0 → 112.1
Nemotron 3.5 Lightning MLX-4bit	12/12	134.8 → 135.1
Nemotron 3.5 Bolt 24GB + image	13/13	39.0 → 38.8
Nemotron 3.5 Super VL NVFP4 + image (ck_experts)	13/13	38.4 → 38.5	1.287 → 1.302 s (one sample)

• Prompt speed, cold, 10 prompts x 2 alternated rounds: Lightning NVFP4 3.2k 0.381/0.380 → 0.381/0.381 s, 13k
  1.682/1.683 → 1.683/1.678 s; Lightning MLX 3.2k 0.410/0.410 → 0.409/0.410 s, 13k 1.786/1.782 → 1.792/1.789 s.
• Bolt's decode is 0.5% lower, more than its sample spread; the others are within it.
• Tests: zig build test, tf-cuda-test checkpoint and pocket pass; the full kernel contract is in #.
• Not run: Qwen3.6-35B-A3B, GLM, Flash Next and MiniMax use none of these functions (their SASS is unchanged); the
  pocket has no published checkpoint to A/B, so the pocket contract (next PR) covers it.

Overlap

atc overlap lists #1445 and #1446 as conflicting, in the shared registration lists (registry.zig, cuda_kernels.zig, kernels.zig, kernel-contract.md, tests/cuda/main.zig); whichever lands second rebases.

[ashhart]

Thank you @BobClawblaw. This is in TensorFold 1.2.0, released today: https://github.com/ashhart/TensorFold/releases/tag/v1.2.0

It landed with your commits under your own name, so you're listed as a TensorFold contributor, and it's credited to you in the release notes and the changelog.