[Akshay Sonawane (apsonawane)]

Main integration update

Merged repository main at 9a662b9395c7c5dc619ae4d050f2c9362d88eae3. Published head: abf747b4b71d02ede934978efd7b279e4ab792e8. Existing PR bases remain unchanged.

• The exact remote head contains main and all preceding updated stack layers.
• Used the qwen-flash conda environment; ran lintrunner -a before each new commit, followed by clean lint checks. Final branch-vs-main lint passes.
• Published with normal fast-forward pushes, without amendments or force pushes.
• No new benchmark/profiling/evaluation scripts or co-author trailers.
• Validation includes C++ schema syntax checks, targeted CUDA SM90/host compilation of the affected operator families, and compilation of sparse/GDN/QMoE regression tests.
• Preserved main's updated decode/API/IndexShare interfaces while adapting the local specializations; fixed merge-created declaration/template/adapter overlaps.
• No complete rebuild or GPU/model requalification has run on this integrated snapshot. Earlier benchmark/parity results below apply to the pre-main combined snapshot, not this merge. The earlier GDN state-output failure remains unresolved.

This update supersedes earlier notes saying main has not been integrated, but does not remove the draft/build/runtime qualification requirements.

Summary

Add exact low-precision FP8 projection paths, quantized MoE routing/maps and scheduling refinements, FP8-to-BF16 dequantization support, and focused operator/kernel tests.

Stack and main target

Depends on #33462.

This is layer 5/7 of the microsoft/onnxruntime stack. The current review base is asonawane/ported-ops-sparse-attention-indexer so this PR shows only its incremental group. The final destination is upstream main: land/reconcile prerequisites first, then rebase and retarget this layer to main. Keep this PR in draft until that base-specific validation succeeds. Do not merge the entire stack through an unvalidated intermediate base.

Cross-repository dependency: GenAI package/runtime integration requires the companion ORT contrib operators.

Validation and blockers

• Combined qualified source snapshot: 286/287 selected operator tests pass.
• Known blocker: GatedDeltaNetTest.RecurrentOutputsIncludeFirstRowState fails in the existing combined workspace.
• Three previously documented packed-indexer baseline failures were excluded: RejectsWrongOutputCount, QsaMalformedStateLengthsAreClamped, CsaMalformedStateLengthsAreClamped.
• Five MTP/fold kernel tests pass separately.
• Split schema translation units pass syntax checks at their individual stack stages.
• Combined tip is byte-identical to the original workspace snapshot. Individual branch builds and upstream-main CI are still required.

Scope

• Preserve the original dirty worktrees and existing PR branches.
• No benchmark runners, profiling utilities, evaluation drivers, model weights or local task metadata in the new commits.
• No Copilot co-author trailers in the new commit messages.
• Broad compatibility with other models is not established by local Qwen qualification.

Stack navigation

• Add exact normalization and rotary CUDA operator modes
• Add fused hyper-connection and tensor CUDA operators
• Extend GDN and causal convolution fused state handling
• Extend sparse attention caches and optimize packed indexing
• Optimize quantized MoE and block-scaled FP8 CUDA projections
• Add fused MTP MoE and speculative state fold CUDA operators
• Document generic CUDA operators and complete build compatibility

Companion repository stack:

• Add Qwen CUDA graph execution through Generator
• Add generic operator package rewrites and integration tests

[Akshay Sonawane (apsonawane)]

Closing this draft to replace the feature-based stack with focused branches rebuilt from current main. Existing branches are preserved; replacement PRs will be linked here.

[Akshay Sonawane (apsonawane)]

Replacement opened directly against current main: #33473. The old branch is preserved; family documentation is bundled with the replacement.