lfm3: serve LFM3-1.7B NVFP4 on CUDA, weight-only (stacked on #1513, #1492 and #1503)

README.md

@@ -173,7 +173,7 @@ Conversions are explicitly marked: their upstream weights need the linked conver
 | GLM-5.3-Flash, FP4-Local | `tensorfold pull nvidia/GLM-5.3-Flash-FP4-Local@76348ddb2be2e35e39ecfc00e75dc07703f4f3d5` | Two GB10s at TP=2, or three at TP=3 over a ring of direct links: @Bizuayeu loaded it as stored and served it at both sizes. 165.3 GiB of weights, 13% less than NVFP4's 190.4 GiB. | [Native CUDA](docs/recipes/glm-5.3-flash-cuda.md), in 1.1.5 (#1050). NVIDIA's mixed-precision checkpoint: NVFP4 routed experts with compressed block scales, MXFP8 and 128×128-block FP8 for most of the rest. As close to BF16 as NVFP4 at its smaller size: at TP=2 its NLL is 1.11% above BF16's against NVFP4's 1.37% ([contributed receipt](docs/receipts/glm-fp4local-80517fc3/README.md) on #1050's head, not rerun on main), a difference the per-document intervals in [XSFP4's card](https://huggingface.co/Bizuayeu/GLM-5.3-Flash-XSFP4#measured) do not resolve. |
 | GLM-5.3-Flash, XSFP4 | `tensorfold pull Bizuayeu/GLM-5.3-Flash-XSFP4@88736f916f8939607fde8a4b881dcae749f7551d` | Two GB10s at TP=2, or three at TP=3 over a ring of direct links: on main `8086c205c` @Bizuayeu loaded it in 370.7 s at TP=2 (rank 0's model 96.97 GiB at a 524,288-token window) and in 313.8 s at TP=3 (63.60 GiB at 1,048,576), and both served. 165.7 GiB of weights, 0.4 GiB more than FP4-Local's. | [Native CUDA](docs/recipes/glm-5.3-flash-cuda.md), in 1.1.5 through FP4-Local's path. FP4-Local with its routed experts' NVFP4 blocks fitted again from the BF16 weights, a smoothing folded into them, so it runs on the same kernels. Closer to BF16 than FP4-Local and NVFP4 at TP=2 by all three measures in [its card](https://huggingface.co/Bizuayeu/GLM-5.3-Flash-XSFP4#measured), on #1050's code: top-1 agreement 93.26%, NLL 0.85% above and a KL bound of 0.0431, against FP4-Local's 92.82%, 1.11% and 0.0510. |
 | GLM-5.3, EXLR8 `home` | `tensorfold pull kindlingai/glm-5.3-exlr8-k2-k3-k4@f459f5d0272a529403dd9b3fd32f771f4250a0d4 --exlr8 home --node 0/2 --dir ~/models/glm-5.3-exlr8-home` | Two M5 Ultra Macs, 256 GB each; about 160 GB resident per Mac at 8k context / four streams. | [Two-Mac recipe](docs/glm-5.3-exlr8.md). Use `--node 1/2` on the other Mac; Thunderbolt 5 and MCDMA required. No CUDA backend. |
-| LFM2.5-24B-A2B, MLX 4 / 6 / 8-bit | `tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-4bit@667408d494bde191c2cb9dd4f4549827816b9275`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-6bit@1deec53eb49571e21cdcbf82fd5839248f07ed88`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-8bit@67ab417dd02d0c9545802a726df668620eae049d` | One Mac; M4 Pro checked. | [Native recipe](docs/recipes/lfm2.5-24b-a2b.md), [release history](CHANGELOG.md). |
+| LFM2.5-24B-A2B, MLX 4 / 6 / 8-bit | `tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-4bit@667408d494bde191c2cb9dd4f4549827816b9275`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-6bit@1deec53eb49571e21cdcbf82fd5839248f07ed88`<br>`tensorfold pull LiquidAI/LFM2.5-24B-A2B-MLX-8bit@67ab417dd02d0c9545802a726df668620eae049d` | One Mac (M4 Pro checked), or one NVIDIA GPU with the 4-bit export (GB10 checked). | [Native recipe](docs/recipes/lfm2.5-24b-a2b.md) ([CUDA](docs/recipes/lfm2.5-24b-a2b.md#cuda)), [release history](CHANGELOG.md). Weights: LFM Open License v1.0. |
 | Qwen3.6-35B-A3B, MLX with MTP | `tensorfold pull TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP@81169a9bc511a27c1b4eedb77a2cd98ced431847` | One GB10; 20.9 GB checkpoint. | [Qwen3.6](docs/recipes/qwen3.6-moe.md). Native CUDA; the recipe's Metal path belongs to Python. |
 | Gemma 4 31B, MLX 4-bit | `tensorfold pull mlx-community/gemma-4-31B-it-4bit@696d436c404745a59f30e4939a658162b0a9e57f`<br>`tensorfold pull z-lab/gemma-4-31B-it-DFlash@eabd648301ce28583cc14757912e5e0f84e152e1` | One M5 Mac; M5 Ultra checked. M1-M4 refused. | [Native 31B](docs/recipes/gemma-4-native.md). Name the drafter with `--drafter`; images with `--vision`. |
 | Gemma 4 12B, converted MLX 4-bit | `hf download google/gemma-4-12B-it --revision 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 --local-dir ~/models/gemma-4-12b-hf`<br>`hf download google/gemma-4-12B-it-assistant --revision 46d4c6f13f0ac0ad827b915669b8df9b81c64c51 --local-dir ~/models/gemma-4-12b-assistant` | One M5 Mac; converted text weights 6.3 GB, image weights 100 MB, plus assistant/caches. M1-M4 refused. | [Convert and serve](docs/recipes/gemma-4-12b-native.md). Upstream download needs `hf`; conversion needs Python, mlx and mlx-lm. Serving is native. |

build.zig

@@ -95,6 +95,7 @@ pub fn build(b: *std.Build) void {
         @import("zig/build/step5_host.zig").steps(b, step5_mods.metal, step5_mods.engine, step5_mods.lanes);
     }
     @import("zig/build/h3_host.zig").tests(b, target, test_step);
+    @import("zig/build/lfm3_host.zig").steps(b, test_step);
     @import("zig/build/digest.zig").steps(b, test_step, draft_ids, build_options);
     @import("zig/build/flashnext_budget.zig").steps(b, test_step);
     @import("zig/build/flashnext_batch_host.zig").tests(b, test_step);

docs/kernel-contract.md

@@ -188,6 +188,13 @@ GLM's Metal window projections register narrow and wide rows at 2, 3, 4, 5, 6 an
 Independent FP64 linear cases accompany byte replacements against frozen pre-window-optimization kernels.
 The gathered SwiGLU keeps its predecessor's clipped BF16 output and nonfinite rules; gate/up taps qualify its dots separately.
 Controls dispatch real source mutations that couple rows or select the wrong expert; host tests validate their witnesses.
+CUDA also checks LFM2.5-24B-A2B's kernels: the router's fp32 logits (`tf_lfm2m_route`) and its top-4 rule, Liquid's
+3-tap short conv with its per-row state snapshots and the state commit (`liquid_conv.cuh`), each against the host's
+replay of its fp32 and bf16 steps (three controls: ties to the higher expert, bx unrounded, every stream committing into
+the first stream's slot). Their paged attention at head dim 64 runs over shuffled 512-key pages with several streams
+a launch: the decode rows' chunks and merge (`tf_paged_attn_chunk_segs_64`, `tf_paged_attn_merge_segs_64`) and the
+prompt rows (`pattn_kernel<64, 8, 4, 8>`), head by head within `attention_ref`'s float64 bound (two controls: the
+decode reading its values from the neighbouring lane's dims, the prompt from the neighbouring 8 dims).
 CUDA also checks the affine-3 experts' split decode pair, bit-equal to the pair kernel it replaces (two controls,
 one dropping a last round of K groups short of the warps, one chaining the groups in reverse).
 CUDA also checks Flash Next's NVFP4 experts on prompt tiles with the decode form's sums (SwiGLU gate-up and fp32

docs/recipes/lfm2.5-24b-a2b.md

@@ -1,4 +1,4 @@
-# LFM2.5-24B-A2B, native Metal
+# LFM2.5-24B-A2B, native Metal and CUDA
 
 This recipe covers [`LiquidAI/LFM2.5-24B-A2B-MLX-4bit`](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-4bit)
 at revision `667408d494bde191c2cb9dd4f4549827816b9275`. The numbers below are measured on this 4-bit export only.
@@ -41,3 +41,44 @@ the shared-round kernels on it. The M4 Pro is not yet measured on a quiet machin
 
 Every reply was token for token equal before and after, and equal to its solo run and to `"draft": false`.
 Cold prefill at 2k, 8k and 32k tokens and peak memory did not change.
+
+## CUDA
+
+`lfm2_moe` also serves the 4-bit export above (same revision, `667408d494bde191c2cb9dd4f4549827816b9275`) on one
+NVIDIA GPU, through the lane host. Checked on a GB10 (sm_121).
+
+```bash
+zig build native -Doptimize=ReleaseFast -Dnvcc=/usr/local/cuda/bin/nvcc
+zig-out/native/bin/tensorfold-native serve "$HOME/models/lfm2.5-24b-a2b-4bit" --port 8080 --parallel 8
+```
+
+The default window is 32,768 tokens (`--context`), and `--parallel` holds up to 128 streams if the memory budget fits
+them. Streams lease 512-position pages of keys and values from one pool as they need them
+(`TENSORFOLD_LFM2_KV_PAGES=0` gives each stream fixed pages instead). The prompt cache keeps a chat's state at its
+history marks, the conv states and the K/V rows before the mark, in pages allocated once inside its budget
+(`--prompt-cache-gib`). The family reads MLX affine 4-bit weights in groups of
+64, with the embedding at 4, 6 or 8 bits. The 6- and 8-bit exports are refused by name and stay on Metal. Drafts
+are the lane core's copy drafts from the prompt's own text; there is no draft head yet.
+
+Most launches are shared CUDA operators: the affine linears and experts, the norms, `qk_rows`, the MoE combine, the
+pick and the draws. The family adds three kernels: the router (fp32 weights, sigmoid plus bias, top-4) in
+`lfm2_moe.cu`, and the 3-tap short conv and its state commit in `liquid_conv.cuh`. It also adds two instances of shared
+templates at head dim 64: `pattn_kernel<64, 8, 4, 8>` for prompt rows, and `tf_paged_attn_chunk_segs_64` with
+`tf_paged_attn_merge_segs_64` (`paged_attn_segs.cuh`) for decode rows. All of them are under the kernel contract
+(see [the contract](../kernel-contract.md#coverage)).
+
+Receipt (GB10, nvcc 13.0, `--parallel 8`, greedy):
+
+- drafted == plain and concurrent == solo on 8 of 8 prompts, and the token hashes are the same after a restart;
+- a float64 forward of the same weights on the CPU (`tools/lfm2_moe/ref64.py`) picks the engine's greedy token on
+  421 of 435 teacher-forced prompt rows and 123 of 128 reply rows; `tools/lfm2_moe/REF64.md` lists every miss with
+  both tokens and margins;
+- decode, total tok/s: 87 at 1 stream, 194 at 4, 303 at 8;
+- cold TTFT: 0.32 s at 2.8k prompt tokens, 1.32 s at 11.2k.
+
+## Licence
+
+The weights are under Liquid AI's LFM Open License v1.0 (`license: other`, `lfm1.0` on the model card; the text is the
+`LICENSE` file in the checkpoint). Use, changes and redistribution are allowed; commercial use is licensed only to
+an organisation under US$10 million in annual revenue, and a non-profit may use it for research or other
+non-commercial purposes regardless. Read the licence before serving it commercially.

docs/recipes/lfm3-cuda.md

@@ -0,0 +1,50 @@
+# LFM3-1.7B NVFP4, native CUDA
+
+`lfm3` serves LiquidAI's ModelOpt NVFP4 text export,
+[`LiquidAI/LFM3-1.7B-NVFP4`](https://huggingface.co/LiquidAI/LFM3-1.7B-NVFP4) at revision
+`b30025bbbbcbf62f6c110b40d51d36af6579f6a2`, on one NVIDIA GPU through the lane host. Checked on a GB10 (sm_121).
+The MLX affine and FP8 exports and the VL model are not loaded on CUDA yet, and Metal refuses this export by name
+(`Lfm3ModeloptNvfp4NotOnMetal`).
+
+```bash
+hf download LiquidAI/LFM3-1.7B-NVFP4 --revision b30025bbbbcbf62f6c110b40d51d36af6579f6a2 \
+  --local-dir "$HOME/models/lfm3-1.7b-nvfp4"
+zig build native -Doptimize=ReleaseFast -Dnvcc=/usr/local/cuda/bin/nvcc
+zig-out/native/bin/tensorfold-native serve "$HOME/models/lfm3-1.7b-nvfp4" --port 8080 --parallel 8
+```
+
+## Weight-only
+
+The mode is weight-only. The FFN's FP4 weights (w1, w3, w2, E2M1 in groups of 16 with e4m3 block scales and one fp32
+tensor scale) are used as stored, and every activation stays bf16. The export also carries static FP4 input scales;
+they are read, because the admission requires them, but not applied. The mixers (conv, q/k/v/gate, o), the embedding
+and the tied head are bf16 in the export and run as bf16.
+
+## What runs
+
+- Conv layers (13): Liquid's 7-tap short conv with its rolling state (`liquid_conv.cuh`, shared with LFM2.5).
+- Full-attention layers (6): keys and values in pages of the core KV pool, partial RoPE through `qk_rows`' frequency
+  table, `pattn_kernel<128, 8, 4, 8>` for prompt rows and the shared paged chunk and merge kernels for decode rows.
+- Sliding-attention layers (5, window 256): a ring a stream through the shared window-attention kernels.
+- The lane backend, the kept states of the prompt cache and the native glue are shared with LFM2.5-24B-A2B
+  (`zig/src/families/liquid/`). Kept states live in pages allocated once inside the prompt-cache budget.
+- The default window is 32,768 tokens (`--context`); the model allows 262,144.
+
+## Receipt
+
+GB10, greedy, `--parallel 8`:
+
+- drafted == plain and concurrent == solo on 8 of 8 prompts, prompts of 300 to 2,960 tokens across the window,
+  prompt rows == one-row decode at 32 cut points, resumed == fresh, and the token hashes after a restart;
+- a float64 forward of the same weights on the CPU (`tools/lfm3/ref64.py`) picks the engine's greedy token on 825 of
+  829 teacher-forced prompt rows and 127 of 128 reply rows; `tools/lfm3/REF64.md` lists every miss with both tokens
+  and margins;
+- decode, total tok/s: 112.8 at 1 stream, 420.0 at 4, 767.4 at 8; cold TTFT 0.150 s at 2k tokens, 0.615 s at 8k,
+  2.84 s at 32k.
+
+## Licence
+
+The weights are under Liquid AI's LFM Open License v1.0 (`license: other`, `lfm1.0` on the model card; the text is the
+`LICENSE` file in the checkpoint), as the MLX exports are. Use, changes and redistribution are allowed; commercial
+use is licensed only to an organisation under US$10 million in annual revenue, and a non-profit may use it for
+research or other non-commercial purposes regardless. TensorFold's code stays Apache-2.0.

docs/recipes/lfm3-native.md

@@ -0,0 +1,72 @@
+# LFM3 native Metal bring-up
+
+Tracking issue: [#1491](https://github.com/ashhart/TensorFold/issues/1491).
+
+This branch adds host checks for LFM3 checkpoint configuration and text tensor layouts.
+It does not register a serving engine. Text generation and image input are not implemented or qualified here.
+
+## Inspect a checkpoint
+
+Use Python 3.11 or newer. The inspector uses the standard library. It resolves a full Hugging Face commit,
+reads the config and index, and requests only the safetensors headers. It refuses a server that returns a
+complete weight file instead of the requested byte range.
+
+```sh
+python3 tools/lfm3/inspect_checkpoint.py LiquidAI/LFM3-1.7B-MLX-4bit --output text4.json
+python3 tools/lfm3/inspect_checkpoint.py LiquidAI/LFM3-VL-2B-MLX-4bit --output vl4.json
+```
+
+Replace `4bit` with `6bit` or `8bit` to inspect those exports. Pass `--revision COMMIT` to repeat a pinned inspection.
+Use `--local-dir MODEL_DIR` instead of a repository name to inspect files already on disk.
+
+## Check the native text layout
+
+Use the Zig version in `.zig-version`.
+
+```sh
+zig build test-lfm3-host tf-lfm3-check
+./zig-out/bin/tf-lfm3-check text4.json
+./zig-out/bin/tf-lfm3-check vl4.json
+python3 -m unittest discover -s tools/lfm3 -p 'test_*.py' -v
+```
+
+The native checker reads the saved metadata, validates shard header geometry, and checks the LFM3 text decoder's
+required names, dtypes, shapes, and byte counts. It performs no projection, GPU allocation, or model inference.
+For VL, it checks the text decoder only. A passing result does not validate vision weights or image processing.
+
+The config reader preserves full-attention versus sliding-attention layers, distinct rotary settings, sandwich norms,
+and each module's affine width and group size. Unsupported geometry and conflicting quantization declarations fail.
+
+## Checkpoint details
+
+All six inspected MLX exports store the decoder under `language_model.model.*`, including the text-only model.
+Each decoder has 594 tensors. Convolution kernels are BF16 with shape `[2048, 7, 1]`.
+
+Text 4/6/8-bit and VL 6/8-bit use affine groups of 64. VL 4-bit uses groups of 32 and an 8-bit embedding override.
+The 138 additional VL vision/projector tensors are BF16 in all three exports. Global quantization metadata must not
+cause these dense tensors to be treated as packed affine weights.
+
+The header inspections used these revisions:
+
+- Text 4-bit: `d0ca73dcfa70ce18733754a492cdc1fb0123262d`.
+- Text 6-bit: `26be898c36b545d625d84ac0761c6c3eb664aef3`.
+- Text 8-bit: `ee7667f3cfccb2a41725f91820531823fcabc6b7`.
+- VL 4-bit: `c823fade98e35a3d6942ad4d38b1ea15f675cd12`.
+- VL 6-bit: `6d71bfae121abedc75f6d531fc7f6ceb4f5e3501`.
+- VL 8-bit: `83336d5be3874a2bdd3297bde77b2114dbd3a816`.
+
+## Next gate
+
+Implement the text operators against an independent reference before advertising generation.
+Check the 7-tap convolution, per-head attention gate, sandwich normalization, sliding window, and partial-RoPE formula.
+Then exercise actual 1.7B 4-bit weights through the shared lane engine, followed by 6/8-bit checkpoints.
+The VL encoder and projector need their own native Metal operators and reference checks after text bring-up.
+
+Agree model-fidelity and performance acceptance criteria on the tracking issue before claiming release readiness.
+
+## Weight licence and checked hardware
+
+The official text and VL MLX exports retain the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM3-1.7B-MLX-4bit/blob/main/LICENSE).
+The VL export carries its own [copy of that licence](https://huggingface.co/LiquidAI/LFM3-VL-2B-MLX-4bit/blob/main/LICENSE).
+TensorFold code remains Apache-2.0; admission does not change the weights’ licence.
+This slice validates metadata on the host and does not register inference or qualify a GPU.