Repository navigation
Add Drama 3 ONNX speech graphs and sampling - #793
Open
justinchuby wants to merge 20 commits into
Open
justinchuby wants to merge 20 commits into
justinchuby wants to merge 20 commits into
Conversation
Export the Fish Omni frame embedding, hybrid MoE backbone, and temporal codebook tower as three ONNX graphs. Pin checkpoint/source revisions and stream block-FP8 expert reconstruction with FP32 scale arithmetic and transactional source coverage validation. Cover graph/config compatibility, streaming refusals, persisted CLI output, intrinsic runtime metadata, and reduced upstream prefill plus 20-frame cached greedy parity. Reject unqualified FP16/BF16 and native FP8 rather than advertising unsupported inference; keep the waveform codec and host sampler external. Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Missing lazy-source registration allows supported save operations to overwrite checkpoint files still needed by the package.
1 open issue
What changed in this PR
Adds FP32 Drama 3 token-generation support to Mobius through three ONNX graphs, while leaving sampling and waveform generation to external code.
Changes:
- Registers the model, configuration, and three-component export task.
- Adds lazy FP8-to-FP32 weight reconstruction and explicit precision restrictions.
- Documents the graph interfaces and adds build, loading, CLI, and reduced upstream parity coverage.
| File | Description |
|---|---|
| tests/integration/fish_omni_test.py | Adds pinned upstream prefill and cached-generation parity tests. |
| tests/build_graph/speech_test.py | Checks the three-graph package and ONNX validity. |
| tests/build_graph/_support.py | Routes Fish Omni to dedicated graph tests. |
| tests/_test_configs.py | Adds a reduced Fish Omni configuration. |
| testdata/cases/speech/drama-3.yaml | Records the pinned checkpoint and full-checkpoint test waiver. |
| src/mobius/tasks/_fish_omni.py | Defines graph inputs, outputs, and cache interfaces. |
| src/mobius/tasks/__init__.py | Exports and registers the task. |
| src/mobius/models/fish_omni.py | Implements frame embeddings, backbone, and audio tower. |
| src/mobius/models/fish_omni_test.py | Tests construction, configuration restrictions, and runtime metadata. |
| src/mobius/models/__init__.py | Exports the model class. |
| src/mobius/integrations/transformers/_fish_omni_weights.py | Maps and validates lazy checkpoint bindings. |
| src/mobius/integrations/transformers/_fish_omni_weights_test.py | Tests loading, dequantization, revision handling, and CLI persistence. |
| src/mobius/integrations/transformers/_builder.py | Integrates revision pinning and model-specific loading. |
| src/mobius/integrations/ort_genai/auto_export.py | Rejects unsupported executable pipeline configuration. |
| src/mobius/integrations/onnx_genai/auto_export.py | Emits advisory component metadata. |
| src/mobius/integrations/_weight_loading.py | Supports configurable FP8 reconstruction precision. |
| src/mobius/_registry.py | Registers Fish Omni and its pinned test checkpoint. |
| src/mobius/_configs/fish_omni.py | Extracts and validates architecture and precision settings. |
| pyproject.toml | Adds the upstream parity test dependency. |
| docs/getting-started.md | Documents export commands, graph interfaces, and limitations. |
Review effort: Balanced
Comment on lines
+118
to
+120
| for name, value in parameters.items(): | ||
| value.const_value = inventory.graph.initializers[name].const_value | ||
| package.weight_loading_report = report |
The H200 validation host has an unrelated installed tests package that shadows the repository's namespace-package imports. Put the shared test config directory on pytest's local path and use the repository-standard _test_configs import, so the model suites work in shared Python environments. Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wire the existing tiny composite config into the shared speech matrix so the repository-wide L1/L3 audit discovers its real coverage. Exercise the dedicated task through the normal graph checks, and assert that unsupported affine component quantization is rejected instead of claiming an unimplemented path. Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-trip the weighted package before numerical parity so component-qualified symbolic dimensions are covered. Initialize empty KV caches by their documented sequence axis rather than assuming a bare symbolic dimension name; recurrent and convolution states retain the actual batch size. Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Validate the complete pinned FP32 export against an independently invoked Fish Speech pipeline on H200 CUDA. All prefill/cached vectors agree within 1e-4 absolute error and all 20 greedy frames match exactly; profiling confirms CUDA execution. Commit the native token golden and a staged reproducible validator, with smoke tests for cache namespaces, logging and revision identity. Update documentation and distinguish verified token generation from the still-external waveform pipeline. Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| from types import SimpleNamespace | ||
|
|
||
| import pytest | ||
| import validate_drama3 |
Implement the pinned Fish Speech decode path declaratively: ten RVQ code channels, BF16-rounded RoPE in the FP32 causal transformer, ConvNeXt upsampling, and causal DAC waveform synthesis. Reuse shared convolution and normalization components while binding both checkpoint weight-normalization formats strictly. Expose build_fish_codec_decoder and safe, pinned load_drama3_codec_decoder helpers independently of the token graph registry. No shared Drama 3 model/config, exports, or documentation files are changed. Verify tiny single-frame, batched and attention-window-crossing parity against unmodified upstream, dynamic causal prefix reuse, safe checkpoint loading, and a saved/reloaded real checkpoint export. Twenty golden token frames decode to 40960 samples at 44100 Hz: max absolute error 1.84774399e-6, RMSE 1.18384925e-7, SNR 117.7911 dB. FP16/BF16 codec execution is not claimed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Export native drop-crossing top-p and semantic repetition sampling as explicit-state ONNX policies. Preserve source FP32 nonlinear, normalization, rotary and routed-sum boundaries, and select unmasked fused attention only after proving padding is absent. Gate BF16 to the numerically verified standard-ONNX route, leaving FP16 and dynamic FP8 unsupported. Document the separately exported waveform codec and extend the full-checkpoint validator without claiming unmeasured BF16 fidelity. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
changed the title
Keep the generic waveform-runner waiver tied to missing temporal orchestration rather than already-exported codec and sampling graphs. Distinguish real FP32 evidence from experimental BF16 and unfinished FP16/FP8 routes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Use packed, lazily reconstructed FP32 expert banks with the existing CUDA MoE ABI instead of evaluating every expert. Add an opt-in generation task that prefills the same temporal caches and samples conditioned codebooks in one tower traversal, with greedy/prefill RNG work skipped. Preserve portable exports, source rounding, cache semantics and fail-stop validity flags. Add a request benchmark with zero-copy tensor views and device-resident state, separate initialization/prefill/steady-frame/waveform metrics, and contention and real-time-factor gates. Source parity, stochastic replay equivalence and actual CUDA MoE placement are covered. No uncontended full-model latency claim is made while the shared H200 devices are occupied. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Replace portable all-expert execution with selected-pair gate/up and down BMMs, using bounded 16-token prefill chunks and an unpadded decode path. Preserve root-scope streamed banks and FP32 weighted reduction. This avoids inactive expert overflow contaminating routed results and reduces wasted computation. Prepare and stress-test FP16 residuals, native mixed-output projections and opt-in logged weight saturation, but keep both reduced-precision production exports closed: independent pinned production-dimension real-weight probes failed the unchanged 1e-2 logit gate despite synthetic successes. Qualification-only task subclasses carry explicit unqualified metadata. Correct scalar reciprocal and SiLU opmath without relaxing tolerances. Account for reconstruction dtype and saturation scratch in streaming reports. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
| return op.Expand(op.CastLike(60000.0, hidden), op.Shape(hidden)) | ||
|
|
||
| class ConstantLinearAttention(ConstantMLP): | ||
| def forward(self, op, hidden, conv_state, recurrent_state): |
| return super().forward(op, hidden), op.Abs(conv_state), op.Abs(recurrent_state) | ||
|
|
||
| class ConstantAttention(ConstantMLP): | ||
| def forward(self, op, hidden, bias, positions, past): |
Limit FP32 rotary-cache preservation to the Fish model instead of changing every architecture. Restore generic dtype casting and GQA rewrites that caused eight CI failures; all 52 affected regression tests now pass. Use explicit FP32 exponential/division sigmoid at native rounding boundaries and preserve BF16 tower down-projection output-major layout with standard Gemm. Controlled real-weight probes now match BF16 prefill and initial cached calls exactly, but later RMS reduction drift still fails the unchanged gate. Keep both reduced-precision production admissions closed and document failed stages. Add exact negative-tail/projection regressions and fix the multi-device validation bridge's current CUDA context. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
| # ORT's stream events must be created/recorded on the session's device. | ||
| device_context = torch.cuda.device(int(cuda_options["device_id"])) | ||
| with device_context: | ||
| values = session(**converted) |
| try: | ||
| yield | ||
| finally: | ||
| assert active.pop() == index |
Serialize gate/up banks in half-split SwiGLU order and omit FC3. The separate-FC3 floating MoE path otherwise allocates and copies the full 2 GiB expert bank on every layer invocation. Verify the swiglu/fusion=2 ABI numerically on ORT 1.28; preserve routing, gate/up ordering and shared-expert semantics. Use output-major Gemm for the vocabulary projection, avoiding a 2 GiB runtime transpose above ORT's constant-fold limit. Add no-FC3/layout and Fish-only rotary-cache assertions. Extend the full-checkpoint validator to export/select the one-pass task and verify greedy RNG/validity while preserving the original replay path. The complete pinned pre-fused CUDA and single-pass pipeline now passes independent 30-row prefill/all-vector and 20 exact complete-frame comparison. Maximum decoder/tower logit errors are 5.150e-5/4.673e-5; the profile confirms 40 actual CUDA MoE executions. Scoped regressions pass; no wall-clock performance claim is inferred. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Prepare typed E4M3FN expert banks, unchanged FP32 grids and explicit dynamic_fp8 QMoE emission. Add a byte-preserving streaming classification that rejects dtype conversion, malformed scales and unsafe reinterpretation. Validate every finite FP8 byte including negative zero through save/load, and compare actual exporter-emitted FLOAT/FP16/BF16 decode/prefill experts against pinned Triton on the new runtime. Keep every production native-FP8/low-dtype admission closed. Complete native-FP8/FP32 prefill fails the unchanged hidden-state gate despite all 40 real expert layers passing with identical source inputs. A 1.192e-7 upstream input difference changes an FP8 code and amplifies across layers. Separate reference execution modes and preserve their identities instead of widening tolerances or substituting weight-only FP8. Add explicit CPU embedding/CUDA codec placement, nonzero-device binding guards and measured cuDNN NC1D codec tuning. Dense FP32 real speech reaches RTF0.770 for100greedy frames and RTF0.901 for53sampled frames on the separately qualified combined runtime, with identical sampled-frame hashes. Document initialization/memory costs, runtime provenance, both ORT PRs and failed precision qualifications. Final shared/source suite295pass/17skip,9native operator-source cases pass, and scoped lint/typing pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
|
|
||
|
|
||
| class _ReducedPrecisionQualificationOnly: | ||
| def _validate_precision(self, config): |
Require upstream's torch==2.8.0 before reference model loading and record Torch, CUDA and Triton provenance. Reject environment mismatches between versioned goldens while preserving legacy golden compatibility.\n\nDocument fresh real-weight precision results and source-boundary investigations separately from earlier Torch 2.10 diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Regenerate the independent full-checkpoint FP32 oracle with Torch 2.8.0+cu126 and Triton 3.4.0. Preserve all 20 exact frame IDs while refreshing full-logit provenance and recording the reference environment. Verify all prefill/cached vectors and frames on stock ORT 1.28 and the separately preserved QMoE/XOR/Pow ORT 1.32 runtime. Record actual numerical bounds and retain the original performance configuration separately. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Check source embeddings, shared features, decoder hidden states/logits and tower heads before writing reference tensors. Report the failing stage and execution dtype instead of recording invalid JSON or an unusable numerical oracle. Document complete native-FP8 BF16 and FP16 source captures in the declared Torch 2.8 environment. Both produce 20 finite frames; matching ONNX qualification remains separate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Link the independently committed provider implementation, required optimization levels and exact source-softplus integration result. Record native WebGPU FP32/FP16 software-Vulkan coverage and browser/hardware validation scope. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Record the complete Torch 2.8 native BF16 diagnostic failure and its first expert-boundary divergence. Distinguish the cuBLAS 12.6.4 reference from the 13.6 runtime before matched-library replay; keep production precision admission closed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Allow disabled/basic/extended/all comparison levels while preserving the BASIC default. Apply the level to every component and record it in events and reports so provider-fusion qualification cannot be mislabeled. Cover session configuration and report metadata without network access. The full FP32 checkpoint also passes EXTENDED against the pinned Torch 2.8 oracle with all 20 exact frames and 40 CUDA MoE executions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Link the upstream symbolic-constraint and GemmFloat8 fixes, record unchanged expert-boundary tensors after normal builds, and distinguish the CUDA13 full-FP16 failure from pending matched-library qualification. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add declarative ONNX support for fishaudio/drama-3: frame embeddings, hybrid DeltaNet/full-attention MoE backbone, temporal codebook generation, ONNX sampling and FP32 waveform decoding.
Production support is FP32 with explicitly dequantized experts. FP16/BF16 and native-FP8 whole-model exports remain gated.
Pinned checkpoint:
b2b5ee998034b2975720906c815e7fb0e9f007bd. Fish Speech source:541566932abbaebcb0a40e562a1f846738383f38. No checkpoint weights or upstream source are redistributed; the Fish Audio license is separate from Mobius's MIT license.Model and runtime contracts
embedding: INT64 frames[B,T,11]to text/frame-code embeddings and shared tower features.decoder: hybrid MoE backbone, logits/hidden states and explicit KV/convolution/recurrent caches.audio_decoder: teacher-forced shared/residual tower; successive codebooks reuse prior-frame caches.task="fish-omni-generation": sequentially conditionedaudio_generatorin one tower traversal, with teacher-forced prefill and explicit validity/RNG outputs.mobius.tasks.fish_codec.load_drama3_codec_decoder(): safely load the separately pinned codec and export INT64[B,10,T]to FP32[B,1,T*2048]at 44100 Hz. Waveform decoding processes the full sequence.Host code owns prompt formatting, temporal iteration, policy/history and valid-cache commits. Metadata is graph-derived; unsupported ORT GenAI speech configuration is rejected.
Complete-model correctness
The independent complete-checkpoint oracle uses its declared Torch 2.8.0+cu126 / Triton 3.4.0 environment. All 20 committed frame IDs are unchanged. The regenerated golden records execution mode, dependency versions and the full-prefill checksum. Reference collection checks Torch before model loading and rejects nonfinite embeddings, hidden states and logits. Comparisons check environment metadata when both goldens provide it, retaining legacy compatibility.
The full pinned FP32 checkpoint passes every vector for a 30-row prefill and 20 exact complete frames on one H200, with CPU embeddings, CUDA decoder/tower and TF32 disabled:
5.5313e-55.0306e-52.2888e-55.1498e-55.0545e-52.3842e-55.1498e-55.0545e-52.3842e-5Each prefill profile contains 40 actual CUDA MoE executions. Original Torch 2.10 evidence is preserved separately. The validator now accepts
--optimization-level disabled|basic|extended|all, preserves the BASIC default, applies it to every component and records the actual selection in events and reports.The saved/reloaded FP32 codec passes independent CPU comparison for all 40,960 samples from the native 20-frame golden: maximum error
1.84774399e-6, RMSE1.18384925e-7, SNR 117.7911 dB.Measured speech performance
FP32 CUDA uses streamed prepacked floating
com.microsoft::MoEbanks, raw-logit routing and half-split SwiGLU. Offline gate/up packing and omitted FC3 remove a real 2 GiB expert-bank allocation/copy per invocation; output-major vocabulary Gemm removes a separate 2 GiB runtime transpose. Portable execution gathers only selected experts, with direct short/decode execution and bounded 16-token prefill chunks. One-pass tower work falls from 117 to 13 block evaluations per frame.Single-H200 measurements use the pinned 30-row prompt, CPU embeddings, CUDA neural graphs/codec, TF32 disabled, two full-request warmups, no profiler and no competing GPU process:
Identical 100-frame greedy hashes establish 2.20x steady-frame and 2.01x request speedup. The custom sampled run preserves the original 53-frame sampled hash. These measure complete configurations, not isolated kernels.
I/O binding keeps neural/cache state resident. CPU embeddings reserve codec headroom and use CPU device zero correctly for nonzero CUDA IDs. The opt-in cuDNN NC1D codec configuration reduces the 53-frame diagnostic from 268 to 70 ms with bit-identical waveforms. The benchmark records placement/options, rejects occupied GPUs and raises on qualified RTF failures against the unchanged 1.0 target.
Initialization is separate: roughly 89–105 seconds. The sampled request delivers its first code frame in 348 ms and the full waveform in 2.217 seconds for 2.461 seconds of audio. Dense FP32 requires roughly 144 GB on disk and nearly an H200's entire memory.
Runtime implementations
QMoE(activation_quantization="dynamic_fp8"), FLOAT/FP16/BF16 activations, source quantization/narrowing/reduction boundaries and unchanged default weight-only execution.rsqrtf, preserving other exponents/types and excluded edge paths.Log(Add(x,1))at EXTENDED/ALL, preserving separate FP32 Exp. Fourteen C++ provider/transformer cases pass; 97,589 FLOAT samples per EP have maximum relative errors7.7443e-8CUDA /2.5190e-7native WebGPU. The real 960-value Torch 2.8 softplus fixture is exact on fused CUDA versus 571 unequal values unfused. Native WebGPU FP32 and feature-enabled FP16 executed through lavapipe software Vulkan; physical-WebGPU and browser/WASM execution remain untested. Deprecated JSEP is excluded by ORT policy. This is a pushed independent commit, not a released runtime.Preserved runtime wheel hashes:
e3181530+ XORe90d897c, CUDA13.3/SM90, Flash OFFf7e2f4cb9255e0556ca9cbc03c40189219880039eb76cdea23a000c0f2ae97dc196a4bf87fda934e6b1791e3225c7f2e2a61a35a5ebd907d2946a8f0c0e02c7e455dd6395ddbb79139029c89307882ed80216d513a1e3020fe095f412fc19983c70fd55d20The full-head Flash build needs QUICK_BUILD OFF for head256. All 24 unmasked FP16/BF16 packed3D/4D internal-cache cases execute actual Flash on CUDA without CPU fallback; caches are byte-exact. Two padded additive-bias cases are recorded as expected MATH, not Flash. An extra external-nonpad-cache cuDNN test fails identically on the untouched no-Flash control; it is not the model's internal-past ABI. The Flash wheel does not contain Log1p and is not a native/low-dtype whole-model admission or a new performance measurement. Existing frozen wheels/environments remain unchanged.
Precision qualification
Native wiring preserves E4M3FN bytes and FP32 grids through save/load, rejects malformed banks transactionally, records required runtime features and caps calls at 65,535 token/expert pairs (8,191 tokens at top-8). Nine exporter-emitted expert-only FLOAT/FP16/BF16 source comparisons pass. Weight-only FP8 is a different execution mode.
Complete native-model CUDA13 controls still fail the unchanged
atol=rtol=1e-2prefill gate:0.6406251.158203125Both execute all 40 native experts on CUDA. Their independent Torch 2.8 source replays reproduce all 20 reference states bit-for-bit. First expert inputs differ in only 9 BF16 values (maximum
0.000244140625) or 123 FP16 values (maximum0.00006103515625), before subsequent layers amplify the differences.The private prototypes use source-ordered normalization/recurrent arithmetic and batch-one short prefill. Existing onnx/onnx-shape-inference#135 resolves the constraint-propagation bottleneck. A separate GemmFloat8 inference correction honors transposition and output dtype for FP16 tower down projections; 629 relevant upstream tests pass. Both complete normal graph builds now run without skipped symbolic passes. Computation graphs, generated constants and initializer contracts are checked before reusing original external weights; all 120 expert-boundary tensors per dtype remain unchanged. These are private qualification candidates, not production admission.
Identical real BF16 expert-matmul operands isolate a separate library-version discrepancy: cuBLAS 12.6.4 matches the Torch 2.8+cu126 reference exactly, while 13.6.0 changes three values, across algorithm/workspace controls. A separate CUDA12.6 runtime carrying the required patches is building, with genuine frontend/atomic/compiler/stub compatibility fixes isolated from the QMoE feature PR. No matched-library wheel or full-model result is claimed yet.
Reduced real-fixture gates also remain active: FP16 tower has 5/12288 violations, maximum
0.01248169; BF16 cached tower has 200/4096, maximum0.125. Feeding identical source hidden states makes the BF16 tower exact through prefill and three cached calls. Source-order normalization has 55 exact dynamic-shape cases; controlled cached recurrence matches source bits. Native expert controls and these component proofs do not substitute for complete-model qualification. Historical Torch 2.10 native-FP8/FP32 failure (0.68518, 43436/61440 hidden violations) is retained separately.Regression and feature tracking
The directly coupled suite passed 295 tests with 17 explicit skips, plus 9 native exporter/source opt-ins. Regenerated-golden validator/benchmark: 42 passed, 20 platform skips. Current validation-driver regressions: 44 passed, covering default and all explicit optimization levels on every session and in saved reports. Scoped formatter/lint checks pass; new-head CI is tracked separately.
ONNX proposals: onnx/onnx#8669, compact multi-axis FP8 scale grids; onnx/onnx#8680, batched triangular solve referencing onnx/onnx#7144. Direct cuBLAS replay matches both captured triangular systems exactly; portable Scan preserves close numerical accuracy but not source bits. The Log1p standard-op proposal was closed in favor of provider fusion of existing operations.
Unsupported/unqualified: Generic speech L4/L5 is replaced by the dedicated independent full-checkpoint validator. FP16/BF16/native-FP8 whole-model fidelity remains gated; reduced-precision codec, reference-audio encoding, monolithic temporal autoregression, ORT GenAI/Foundry integration and Olive quantized inference are unqualified. Codec numerical parity does not measure perceptual quality.