jschmied · Oct 27, 2026 · Oct 27, 2026 · Oct 27, 2026 · Oct 27, 2026 · Oct 27, 2026
diff --git a/README.md b/README.md
@@ -173,7 +173,7 @@ Conversions are explicitly marked: their upstream weights need the linked conver
 | GLM-5.3-Flash, FP4-Local | `tensorfold pull nvidia/GLM-5.3-Flash-FP4-Local@76348ddb2be2e35e39ecfc00e75dc07703f4f3d5` | Two GB10s at TP=2, or three at TP=3 over a ring of direct links: @Bizuayeu loaded it as stored and served it at both sizes. 165.3 GiB of weights, 13% less than NVFP4's 190.4 GiB. | [Native CUDA](docs/recipes/glm-5.3-flash-cuda.md), in 1.1.5 (#1050). NVIDIA's mixed-precision checkpoint: NVFP4 routed experts with compressed block scales, MXFP8 and 128×128-block FP8 for most of the rest. As close to BF16 as NVFP4 at its smaller size: at TP=2 its NLL is 1.11% above BF16's against NVFP4's 1.37% ([contributed receipt](docs/receipts/glm-fp4local-80517fc3/README.md) on #1050's head, not rerun on main), a difference the per-document intervals in [XSFP4's card](https://huggingface.co/Bizuayeu/GLM-5.3-Flash-XSFP4#measured) do not resolve. |
 | GLM-5.3-Flash, XSFP4 | `tensorfold pull Bizuayeu/GLM-5.3-Flash-XSFP4@88736f916f8939607fde8a4b881dcae749f7551d` | Two GB10s at TP=2, or three at TP=3 over a ring of direct links: on main `8086c205c` @Bizuayeu loaded it in 370.7 s at TP=2 (rank 0's model 96.97 GiB at a 524,288-token window) and in 313.8 s at TP=3 (63.60 GiB at 1,048,576), and both served. 165.7 GiB of weights, 0.4 GiB more than FP4-Local's. | [Native CUDA](docs/recipes/glm-5.3-flash-cuda.md), in 1.1.5 through FP4-Local's path. FP4-Local with its routed experts' NVFP4 blocks fitted again from the BF16 weights, a smoothing folded into them, so it runs on the same kernels. Closer to BF16 than FP4-Local and NVFP4 at TP=2 by all three measures in [its card](https://huggingface.co/Bizuayeu/GLM-5.3-Flash-XSFP4#measured), on #1050's code: top-1 agreement 93.26%, NLL 0.85% above and a KL bound of 0.0431, against FP4-Local's 92.82%, 1.11% and 0.0510. |
 | GLM-5.3, EXLR8 `home` | `tensorfold pull kindlingai/glm-5.3-exlr8-k2-k3-k4@f459f5d0272a529403dd9b3fd32f771f4250a0d4 --exlr8 home --node 0/2 --dir ~/models/glm-5.3-exlr8-home` | Two M5 Ultra Macs, 256 GB each; about 160 GB resident per Mac at 8k context / four streams. | [Two-Mac recipe](docs/glm-5.3-exlr8.md). Use `--node 1/2` on the other Mac; Thunderbolt 5 and MCDMA required. No CUDA backend. |
-| LFM2.5-24B-A2B, MLX 4 / 6 / 8-bit | `tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-4bit@667408d494bde191c2cb9dd4f4549827816b9275`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-6bit@1deec53eb49571e21cdcbf82fd5839248f07ed88`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-8bit@67ab417dd02d0c9545802a726df668620eae049d` | One Mac; M4 Pro checked. | [Native recipe](docs/recipes/lfm2.5-24b-a2b.md), [release history](CHANGELOG.md). |
+| LFM2.5-24B-A2B, MLX 4 / 6 / 8-bit | `tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-4bit@667408d494bde191c2cb9dd4f4549827816b9275`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-6bit@1deec53eb49571e21cdcbf82fd5839248f07ed88`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-8bit@67ab417dd02d0c9545802a726df668620eae049d` | One Mac (M4 Pro checked), or one NVIDIA GPU with the 4-bit export (GB10 checked). | [Native recipe](docs/recipes/lfm2.5-24b-a2b.md) ([CUDA](docs/recipes/lfm2.5-24b-a2b.md#cuda)), [release history](CHANGELOG.md). Weights: LFM Open License v1.0. |
 | Qwen3.6-35B-A3B, MLX with MTP | `tensorfold pull TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP@81169a9bc511a27c1b4eedb77a2cd98ced431847` | One GB10; 20.9 GB checkpoint. | [Qwen3.6](docs/recipes/qwen3.6-moe.md). Native CUDA; the recipe's Metal path belongs to Python. |
 | Gemma 4 31B, MLX 4-bit | `tensorfold pull mlx-community/gemma-4-31B-it-4bit@696d436c404745a59f30e4939a658162b0a9e57f`<br>`tensorfold pull z-lab/gemma-4-31B-it-DFlash@eabd648301ce28583cc14757912e5e0f84e152e1` | One M5 Mac; M5 Ultra checked. M1-M4 refused. | [Native 31B](docs/recipes/gemma-4-native.md). Name the drafter with `--drafter`; images with `--vision`. |
 | Gemma 4 12B, converted MLX 4-bit | `hf download google/gemma-4-12B-it --revision 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 --local-dir ~/models/gemma-4-12b-hf`<br>`hf download google/gemma-4-12B-it-assistant --revision 46d4c6f13f0ac0ad827b915669b8df9b81c64c51 --local-dir ~/models/gemma-4-12b-assistant` | One M5 Mac; converted text weights 6.3 GB, image weights 100 MB, plus assistant/caches. M1-M4 refused. | [Convert and serve](docs/recipes/gemma-4-12b-native.md). Upstream download needs `hf`; conversion needs Python, mlx and mlx-lm. Serving is native. |

diff --git a/docs/kernel-contract.md b/docs/kernel-contract.md
@@ -188,6 +188,13 @@ GLM's Metal window projections register narrow and wide rows at 2, 3, 4, 5, 6 an
 Independent FP64 linear cases accompany byte replacements against frozen pre-window-optimization kernels.
 The gathered SwiGLU keeps its predecessor's clipped BF16 output and nonfinite rules; gate/up taps qualify its dots separately.
 Controls dispatch real source mutations that couple rows or select the wrong expert; host tests validate their witnesses.
+CUDA also checks LFM2.5-24B-A2B's kernels: the router's fp32 logits (`tf_lfm2m_route`) and its top-4 rule, Liquid's
+3-tap short conv with its per-row state snapshots and the state commit (`liquid_conv.cuh`), each against the host's
+replay of its fp32 and bf16 steps (three controls: ties to the higher expert, bx unrounded, every stream committing into
+the first stream's slot). Their paged attention at head dim 64 runs over shuffled 512-key pages with several streams
+a launch: the decode rows' chunks and merge (`tf_paged_attn_chunk_segs_64`, `tf_paged_attn_merge_segs_64`) and the
+prompt rows (`pattn_kernel<64, 8, 4, 8>`), head by head within `attention_ref`'s float64 bound (two controls: the
+decode reading its values from the neighbouring lane's dims, the prompt from the neighbouring 8 dims).
 CUDA also checks the affine-3 experts' split decode pair, bit-equal to the pair kernel it replaces (two controls,
 one dropping a last round of K groups short of the warps, one chaining the groups in reverse).
 CUDA also checks Flash Next's NVFP4 experts on prompt tiles with the decode form's sums (SwiGLU gate-up and fp32

diff --git a/docs/recipes/lfm2.5-24b-a2b.md b/docs/recipes/lfm2.5-24b-a2b.md
@@ -1,4 +1,4 @@
-# LFM2.5-24B-A2B, native Metal
+# LFM2.5-24B-A2B, native Metal and CUDA
 
 This recipe covers [`LiquidAI/LFM2.5-24B-A2B-MLX-4bit`](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-4bit)
 at revision `667408d494bde191c2cb9dd4f4549827816b9275`. The numbers below are measured on this 4-bit export only.
@@ -41,3 +41,44 @@ the shared-round kernels on it. The M4 Pro is not yet measured on a quiet machin
 
 Every reply was token for token equal before and after, and equal to its solo run and to `"draft": false`.
 Cold prefill at 2k, 8k and 32k tokens and peak memory did not change.
+
+## CUDA
+
+`lfm2_moe` also serves the 4-bit export above (same revision, `667408d494bde191c2cb9dd4f4549827816b9275`) on one
+NVIDIA GPU, through the lane host. Checked on a GB10 (sm_121).
+
+```bash
+zig build native -Doptimize=ReleaseFast -Dnvcc=/usr/local/cuda/bin/nvcc
+zig-out/native/bin/tensorfold-native serve "$HOME/models/lfm2.5-24b-a2b-4bit" --port 8080 --parallel 8
+```
+
+The default window is 32,768 tokens (`--context`), and `--parallel` holds up to 128 streams if the memory budget fits
+them. Streams lease 512-position pages of keys and values from one pool as they need them
+(`TENSORFOLD_LFM2_KV_PAGES=0` gives each stream fixed pages instead). The prompt cache keeps a chat's state at its
+history marks, the conv states and the K/V rows before the mark, in pages allocated once inside its budget
+(`--prompt-cache-gib`). The family reads MLX affine 4-bit weights in groups of
+64, with the embedding at 4, 6 or 8 bits. The 6- and 8-bit exports are refused by name and stay on Metal. Drafts
+are the lane core's copy drafts from the prompt's own text; there is no draft head yet.
+
+Most launches are shared CUDA operators: the affine linears and experts, the norms, `qk_rows`, the MoE combine, the
+pick and the draws. The family adds three kernels: the router (fp32 weights, sigmoid plus bias, top-4) in
+`lfm2_moe.cu`, and the 3-tap short conv and its state commit in `liquid_conv.cuh`. It also adds two instances of shared
+templates at head dim 64: `pattn_kernel<64, 8, 4, 8>` for prompt rows, and `tf_paged_attn_chunk_segs_64` with
+`tf_paged_attn_merge_segs_64` (`paged_attn_segs.cuh`) for decode rows. All of them are under the kernel contract
+(see [the contract](../kernel-contract.md#coverage)).
+
+Receipt (GB10, nvcc 13.0, `--parallel 8`, greedy):
+
+- drafted == plain and concurrent == solo on 8 of 8 prompts, and the token hashes are the same after a restart;
+- a float64 forward of the same weights on the CPU (`tools/lfm2_moe/ref64.py`) picks the engine's greedy token on
+  421 of 435 teacher-forced prompt rows and 123 of 128 reply rows; `tools/lfm2_moe/REF64.md` lists every miss with
+  both tokens and margins;
+- decode, total tok/s: 87 at 1 stream, 194 at 4, 303 at 8;
+- cold TTFT: 0.32 s at 2.8k prompt tokens, 1.32 s at 11.2k.
+
+## Licence
+
+The weights are under Liquid AI's LFM Open License v1.0 (`license: other`, `lfm1.0` on the model card; the text is the
+`LICENSE` file in the checkpoint). Use, changes and redistribution are allowed; commercial use is licensed only to
+an organisation under US$10 million in annual revenue, and a non-profit may use it for research or other
+non-commercial purposes regardless. Read the licence before serving it commercially.
diff --git a/tools/lfm2_moe/REF64.md b/tools/lfm2_moe/REF64.md
@@ -0,0 +1,65 @@
+# Float64 reference receipt: LFM2.5-24B-A2B on CUDA
+
+- Checkpoint: [LiquidAI/LFM2.5-24B-A2B-MLX-4bit](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-4bit) at revision
+  `667408d494bde191c2cb9dd4f4549827816b9275`.
+- Engine: this PR's serving commit, before the rebase onto main (`c6645063a`). The rebased head serves the same
+  tokens: the token hashes of 8 prompts and of a chat conversation are equal.
+  Served on a GB10 (sm_121), greedy, drafts off, `--parallel 8 --context 16384`.
+- Reference: `ref64.py` on the CPU in float64 (Python 3.12, torch 2.14, safetensors 0.8, tokenizers 0.23), 2026-10-28.
+
+Reproduce (the reference pass takes about 10 minutes and 17 GB of host memory):
+
+```
+zig-out/native/bin/tensorfold-native serve MODEL_DIR --port 8080 --parallel 8 --context 16384
+python -I tools/lfm2_moe/ref64.py collect http://127.0.0.1:8080 MODEL_DIR engine.json
+python -I tools/lfm2_moe/ref64.py MODEL_DIR engine.json --bf16
+```
+
+Rows. A prompt row is the engine's one-token greedy pick after the prefix [0, t] of a prompt, one request per prefix.
+A reply row is a token of the engine's 32-token greedy reply, teacher-forced on that reply. `t` is the logits row after
+token t, so prompt row P - 1 and reply token 0 are the same position. The prompts (`prompts()` in the script) are
+capitals (13 tokens), code (19), a chat turn (19) and items (384, a list of 30 items and a question).
+
+The float64 pass dequantizes the MLX weights exactly. The `--bf16` pass adds the engine's bf16 storage points (see the
+script's docstring). It shows which side of a miss rounding alone lands on; it is not bit-exact.
+
+## Result
+
+- f64: prompt rows 421/435, reply rows 123/128
+- bf16: prompt rows 424/435, reply rows 123/128
+
+Float64 top-1 misses (margin = f64 logit of its top minus f64 logit of the engine's token; bf16 margin = the same difference in the bf16-points pass, < 0 means it prefers the engine's token):
+
+| case | row | t | engine | float64 | f64 margin | bf16 margin | bf16 picks |
+|---|---|---|---|---|---|---|---|
+| capitals | prompt | 1 | 589 "'s" | 124901 "<think>" | 0.3389 | -0.1250 (pre-round -0.1026) | engine's |
+| capitals | prompt | 3 | 1394 " each" | 278 " the" | 0.0704 | -1.0625 (pre-round -1.0871) | engine's |
+| capitals | reply | 39 | 278 " the" | 17755 " Greece" | 0.0095 | +0.0000 (pre-round -0.0276) | engine's |
+| code | prompt | 7 | 1288 " return" | 21744 " \"\"\"" | 0.3008 | +0.2500 (pre-round +0.2923) | f64's |
+| code | reply | 26 | 4876 "_m" | 71 "_" | 0.3628 | +0.3125 (pre-round +0.2956) | f64's |
+| code | reply | 35 | 73 "a" | 2762 "add" | 0.0687 | -0.3750 (pre-round -0.4420) | engine's |
+| chat | prompt | 13 | 90978 " Explanation" | 3103 "  \n" | 0.7478 | -0.8125 (pre-round -0.7755) | engine's |
+| chat | prompt | 15 | 25 "1" | 3103 "  \n" | 0.1301 | +0.0625 (pre-round +0.0216) | f64's |
+| items | prompt | 3 | 22 "." | 34 ":" | 0.9143 | +0.8750 (pre-round +0.8393) | f64's |
+| items | prompt | 5 | 2719 " problem" | 3451 " question" | 0.1379 | +0.0625 (pre-round +0.0963) | f64's |
+| items | prompt | 6 | 9402 " stone" | 24062 " brick" | 0.1680 | +1.0000 (pre-round +0.9824) | f64's |
+| items | prompt | 8 | 229 " " | 267 " a" | 0.0823 | -0.1250 (pre-round -0.0781) | engine's |
+| items | prompt | 314 | 27 "3" | 1035 "14" | 0.1292 | -0.3750 (pre-round -0.3704) | engine's |
+| items | prompt | 338 | 25 "1" | 26 "2" | 0.3699 | +0.3750 (pre-round +0.4515) | f64's |
+| items | prompt | 371 | 437 " are" | 296 " in" | 0.0565 | -0.6250 (pre-round -0.5979) | engine's |
+| items | prompt | 375 | 342 " (" | 8187 " item" | 0.3365 | -1.1250 (pre-round -1.1544) | engine's |
+| items | prompt | 377 | 342 " (" | 41501 " Item" | 0.6276 | -2.3750 (pre-round -2.3544) | engine's |
+| items | reply | 387 | 41501 " Item" | 342 " (" | 0.7665 | +0.6250 (pre-round +0.5957) | f64's |
+| items | reply | 396 | 2959 " red" | 6542 " blue" | 0.2816 | +0.5000 (pre-round +0.5314) | f64's |
+
+bf16-points misses where float64 agrees with the engine (margin = bf16 logit of its pick minus the engine's token; f64 margin = how far float64 prefers the engine's token):
+
+| case | row | t | engine | bf16 pick | bf16 margin | f64 margin |
+|---|---|---|---|---|---|---|
+| capitals | prompt | 10 | 10125 " Italy" | 6398 " Germany" | 0.0000 (pre-round -0.0632) | 0.0079 |
+| code | reply | 44 | 1113 " add" | 1619 " main" | 0.6250 (pre-round +0.5908) | 0.1500 |
+| code | reply | 47 | 71 "_" | 65296 "_sum" | 0.1250 (pre-round +0.1222) | 0.3751 |
+| items | prompt | 7 | 22 "." | 20 "," | 0.0000 (pre-round +0.0882) | 0.0279 |
+| items | prompt | 9 | 30 "6" | 27 "3" | 0.5000 (pre-round +0.4381) | 0.1054 |
+| items | prompt | 36 | 440 " The" | 41501 " Item" | 0.1250 (pre-round +0.1183) | 0.1051 |
+| items | prompt | 379 | 8187 " item" | 229 " " | 0.2500 (pre-round +0.2420) | 0.0420 |