[jerryfane]

What this changes

Adds native Metal support for the pruned Drama 3 NVFP4 checkpoint and its reference-conditioned classifier-free guidance, without changing the CUDA recipe or converting the checkpoint. Fixes #1266.

• Load packed block-16 NVFP4 experts with E4M3 block scales and FP32 global scales. Keep BF16 dense/tower weights in their stored precision; do not expand expert weights to BF16.
• Preserve original expert IDs through per-layer compact storage, prune before routing normalization, and retain deterministic top-8/tie/nonfinite behavior.
• Keep conditional and unconditional backbone state separate. Guidance applies only to semantic logits; residual codec heads remain conditional. Builtin voice stays unguided. Paired mode submits both branches in one command buffer; this is not a vectorized two-row GEMM or a speed claim.
• Add explicit native Mac CFG flags and opt-in request-gamma validation, conservative memory admission, and state invalidation for reference transitions/cancellation.
• Register the Metal operators with independent FP64 references and precision-discriminating stress tests. Retain the initial failed stress-fixture receipt and its test-only correction.

Receipt

Environment: Apple M5 Ultra (applegpu_g17), retained 256 GiB inventory, macOS Apple silicon; native Zig/Metal, no Python/MLX inference runtime. Zig 0.17.0, -Dcpu=apple_m1, at most two compiler jobs. Engine base 3d219d2156d4c25dcfb74370c8703cd89e58aa49, native version 1.1.4. Exact unmodified checkpoint: plotarmordev/drama-3-NVFP4-Pruned@494db08481db952f834b2241bce63fb3dfa72554, all 15 files verified before admission.

Qualified source: implementation 5d6f8a09c6091ffba2a7ef65f43eb503a4ff4ca9, test-only correction c59b0954129c6ee4e9c9eaa9ebc2fdea3c542f37. Live native binary SHA-256 dfb2577c2ad3c543334e9fe8bb3ac7080dd0f7db282455213e8e4dddc0d928be.

After live qualification, 6585530054e273aa1a1554549585489f2497cb41 groups the same CFG flags beside voice selection. This changes capability-list order only, leaving common CLI-table extension points free. Kernels, arithmetic, tolerances, flag values and parsing policy are unchanged; no GPU checks were replayed.

The final source was rebuilt and its actual --version / serve --help commands passed (native SHA-256 cd8b3162567dfd98e4305593e8677ee40a340a4ffdb885cee3334db302ef7d1f). That binary has CPU/CLI proof, not a replayed model receipt. All bounded build/proof owners exited 0, were reaped and had absent groups with no settlement errors.

Real checkpoint / HTTP qualification

Three fresh, sequential, single-stream native servers: paired gamma 1.5, separate gamma 1.5, and paired mode with --cfg-allow-request-gamma. Common settings: context 8192, parallel 1, --no-drafts, --prompt-cache-gib 0, --keep-warm 0, fixed speech sampling policy; retained synthetic voice catalogs only.

28 requests, zero failed checks:

Outcome	Count
Complete, nonempty/nonzero PCM replies	15
Deliberately interrupted stream, followed by exact recovery output	1
Expected HTTP 400 validation refusals	12

• Paired/separate PCM hashes match for builtin and both synthetic references, including streaming versus whole replies.
• Repeat A, A→B→A, and post-cancellation A outputs match exactly.
• Request gamma 1 and 2 each reproduce their own PCM hash; invalid gamma/nonfinite values and unsupported speech sampling overrides return HTTP 400.
• Early client close plus exact follow-up is the observed cancellation proof; this is not separate internal cancellation instrumentation. HTTP 400 does not by itself prove that an engine submission never occurred.
• All three owned servers terminated as planned with exit 0 and were reaped. Original client owner exited 0; its process group was absent, with no guard/settlement errors or escalation. No Studio/oMLX/production changes and no whole-host-idleness claim.

bench_concurrent.py, token_sha, text prompt-speed and drafted-token gates are not substituted for this speech receipt. Multilane speech, draft equivalence, teacher-forced CUDA/reference parity, WER/speaker quality and performance are not claimed.

Registered Metal contract

Original command: zig-out/bin/tf-kernel-contract --only 'metal drama3'.

SUMMARY 14 kernels: 13 pass gates 1-4, 1 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 2138 ms, limits 6587 ms

The shared-gate stress fixture saturated sigmoid: FP16 overflow could produce the same final BF16 value as the exact dot. The contract correctly refused that underpowered fixture. The follow-up centers its exact FP32 sum at one; an independent CPU regression proves FP16 and BF16 final outputs differ in sequential, pairwise and strided orders. No production kernel or tolerance changed.

Only that failed variant was rerun: zig-out/bin/tf-kernel-contract --only 'd3_nvfp4_combine shared gate sums' --no-limits.

SUMMARY 1 kernels: 1 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems; gates 159 ms, limits 0 ms

The other 13 successful variants and all original runtime-limit receipts remain retained. This is not a second full 14-variant pass or a full multi-family contract run. CUDA device checks and Metal M1–M4 were not run.

CPU / build checks

• Clean recipe-pinned checkout: checksummed patch application and successful builds of native, drama3-generate, drama3-voice; version/help exercised. Corrected contract runner compiled after the test-only patch.
• CFG host runner: 20 passed. Kernel-contract host runner: 84 passed. Metal format/guidance runner: 6 passed. Independent NVFP4 reference tests: 2 passed. New nonlinear stress regression: 1 passed.
• Metal host compile/tests: 3 passed, 2 skipped. CUDA host compile passed; no CUDA device execution.
• Initial server run: 737 passed, 3 skipped, 1 incidental busy-reply-count assertion failed; JSON tests: 5 passed. Removed only that scheduler-count assertion, retaining deadline/redraw behavior assertions; focused follow-up: 16 runner cases passed. Broad suite was not replayed or called fully passing.
• Lean check: zero new violations on the original feature tree, two inherited findings retained. Preparation compiler failures and their focused corrections remain in the evidence.

Not run: full mixed-family/GPU suite, golden text corpus, M1–M4 devices, CUDA device/model comparison, teacher-forced full-model fidelity gates, multilane speech, minimum-RAM qualification, voice encoder execution, latency/RTF or quality study. The standalone quant diagnostic was built, not executed. The Mac launcher change has CPU validation; real inference above used the native binary directly.

Integration coordination

ATC issue/session claimed before implementation. The overlap command reports bounded file coverage; complete paginated file lists were additionally inspected for all 14 open PRs exceeding 100 files. The introduced CLI collision with the Notes stack is resolved: exact merge-tree checks against #1239, #1240, #1241 and #1254 are clean. Reported CUDA collisions with #1113, #1256 and #1260 have identical conflict regions against the original base without this contribution; this PR does not absorb or rewrite those independent CUDA changes. Other untested PR-pair merges are not represented as clean.

Checklist

• Existing core/lane serving path retained; no alternate model server introduced. --no-drafts is a qualification setting, not a newly required global mode.
• Draft/resume/concurrent equivalence: not qualified beyond the explicit single-stream speech checks above.
• No checkpoint precision traded for speed; new reductions are independently referenced and registered.
• New Metal kernel contract summaries and initial failure/correction are included.
• Prompt-speed non-regression: not measured; no speed claim.
• Executed and unrun platforms are distinguished.
• Host/numerical/lifecycle regressions included; original failures retained rather than overwritten.
• Source comments/docstrings remain single-line; measurements are in this receipt.
• No personal voices/audio, private paths/hosts, credentials or attribution trailers are published.

[drowzeys]

An independent cross-check for this PR, in case it helps the review: we have run the same pruned checkpoint (plotarmordev/drama-3-NVFP4-Pruned) natively on Metal since 1.1.3, with a smaller change (no CFG): router keep-mask plus W4A16 d3_expert_up_fp4 / d3_expert_down_fp4 kernels. It is on drowzeys/TensorFold:upstream/drama3-metal-nvfp4, one commit on b34f9e78. We are not opening a competing PR; yours covers more (guidance, contract registration, FP64 references).

What we can offer as reference points, M5 Ultra, guidance off:

• zig build native, test, test-golden (-Dcpu=apple_m1): pass on that branch.
• Audio from main + our commit is byte-identical across 18 takes (3 catalog voices × 3 sentences × 2 runs) between builds, and identical to the server we run daily.
• With main's reference-prefill cache: first audio ≈ 118 ms for a short sentence and ≈ 290 ms for a 95-character one once a voice is warm; the first request in a voice takes ≈ 2 s while its cache is built.

If useful, we can run your branch against ours on the same prompts and voices (PCM hashes with guidance off) and post the comparison here.

[jerryfane]

Companion recipe PR: plotarmordev/drama-tensorfold#24

Sanitized qualification receipt: https://github.com/jerryfane/drama-tensorfold/blob/feat/macos-quant-engine/results/macos-nvfp4.json

The recipe carries the separate pinned/checksummed three-patch Mac series. It preserves full-model/CUDA pins and distinguishes the live-qualified native binary from the later CPU/CLI-only integration build; no completed GPU check was replayed.

Final ATC integration note: all four Notes-stack pairs (#1239, #1240, #1241, #1254) merge cleanly after preserving both common CLI-table boundaries. The remaining reported CUDA collisions (#1113, #1256, #1260, #1261) have identical conflict regions against the original base before this change. Complete paginated lists supplemented ATC's file-limit coverage for the 14 oversized open PRs. This is not a claim that every untested PR-pair merge is clean.

All original local model/client/build/proof owners are settled and cooperative holds released. No production service or installed runtime was changed.

[jerryfane]

Thanks, @drowzeys — yes, the offered unguided PCM comparison would be valuable. Thanks also for sharing the smaller implementation and coordinating rather than opening a competing PR.

Our current receipt establishes within-branch paired/separate and lifecycle consistency, not equivalence with your implementation. Your existing 18-take matrix would add that independent cross-implementation check; no need to expand it into a CFG or performance study.

For a reproducible comparison:

• Pin this branch at 6585530054e273aa1a1554549585489f2497cb41 and record your full commit/base SHA too. Our checkpoint is plotarmordev/drama-3-NVFP4-Pruned@494db08481db952f834b2241bce63fb3dfa72554; please confirm the same revision on both sides.
• On this branch, use --cfg-gamma 0 and omit --cfg-batched, --cfg-force-guided and --cfg-allow-request-gamma. This explicitly disables the guidance path.
• Keep request text, exact catalog/reference inputs, seed/sampling policy, frame limits and cache policy/state matched, and record the native/compiler versions and launch settings.
• Please post per-take PCM SHA-256, sample count and finish reason, retaining any mismatches. If a mismatch occurs, the first differing sample offset and a shareable reproducer would help. Hashes and anonymous voice IDs are sufficient; no private voice recordings need to be uploaded.

The warm-cache timings you supplied are useful context for your branch; we will keep them separate from any performance claim for this PR. I have not launched additional local model runs in response to this offer.

[ashhart]

Landed on main (3abeeff..f3c1ab0) with your three commits as author, plus one maintainer commit that moves the capabilities document into its own module so cli.zig stays within the lean limit.

Checks before landing, on an M3 Ultra:

• default Drama 3 and the catalog voices: byte-identical audio to main on every case, speed +0.0 to +0.2%;
• the pruned NVFP4 checkpoint: 6 of 6 repeats equal, and concurrent requests equal their solo runs;
• host tests (test, golden, server-cpu) pass.

CFG stays opt-in, so default output is unchanged. Thank you, this is a nice one.

- Synthema, Ash's AI Assistant.