Skip to content

fix(inference): enable quantized SeaCache and correct Transfer control CFG - #282

Merged
pengcuo merged 2 commits into
mainfrom
pzeren/seacache-fp8-sync-20261008
Oct 9, 2026
Merged

pengcuo merged 2 commits into
mainfrom
pzeren/seacache-fp8-sync-20261008

Conversation

@pengcuo

@pengcuo pengcuo commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Quantized inference currently ignores --diffusion-cache, and Transfer does not declare its extra no-control forward to the sampler. Enable the existing SeaCache setting for ModelOpt FP8 and runtime MXFP8/NVFP4, and register the extra branch only while control guidance is active so branch histories and FSDP forward counts stay aligned.

Also sync the control-CFG composition fix: text guidance must scale the existing conditional prediction while the additional control increment remains (control_guidance - 1) * (full - no_control). With text guidance 3 and control guidance 1.5, the old formula subtracts 1.5 times the control delta instead of adding 0.5 times it. Use 1 + (control_guidance - 1) / text_guidance inside the conditional hook and reject zero text guidance when control-CFG is active.

Validation:

  • All 43 tests in inference_test.py, transfer_test.py, and diffusion_cache_test.py passed on GB200. Coverage includes distinct cached branch residuals, interval-boundary resets, and control-CFG composition with text guidance. The six new composition/zero-guidance cases fail before the formula fix.
  • Ruff lint/format and Pyrefly passed (0 type errors).
  • All GitHub checks passed at d91e640, including unit tests, inference smoke tests, and training regressions.
  • ModelOpt FP8 cache-off/cache-on generation both produced 121 frames at 1280x720, using 50 UniPC steps, seed 2026, text guidance 3 and control guidance 1.5. Both ran at this PR head with the same checkpoint and inputs, one GPU per run in a four-GPU allocation. Compilation and CUDA graphs were disabled. The cached run recorded 69 full and 81 skipped forwards.
  • Synchronized frame comparisons against the edge input show substantially better pose alignment after the control-CFG correction, including removal of the spurious bend at frame 75 in both cache modes. Cache-on versus cache-off PSNR is 35.36 dB; mean adjacent-frame pixel differences are 1.31 and 1.33 respectively (8-bit scale). These are single-clip diagnostics, not a broad quality guarantee.
  • Runtime MXFP8/NVFP4 switch behavior has unit coverage; their end-to-end quality has not been revalidated.

Historical reference validation: the saved September 9 no-cache result (245d2b9) used the earlier control-CFG semantics. Its control_guidance=1.5 with text guidance 3 corresponds to control_guidance=2.5 under this formula (new = 1 + text_guidance * (old - 1)). A separate cache-off/cache-on run uses that exact conversion with the original checkpoint, prompts, seed, and control input; the saved control videos decode identically across all 121 frames. Both runs produced 121 frames. The cached result has 32.62 dB mean frame PSNR against the historical no-cache video, and 32.54 dB against the current no-cache result. This explicitly checks the original reported case rather than relying only on agreement between two current outputs.

@pengcuo
pengcuo marked this pull request as ready for review October 8, 2026 12:35
@pengcuo
pengcuo marked this pull request as draft October 9, 2026 03:16
@pengcuo pengcuo changed the title fix(inference): enable SeaCache for FP8/NVFP4 inference fix(inference): enable quantized SeaCache and correct Transfer control CFG Oct 9, 2026
@pengcuo
pengcuo marked this pull request as ready for review October 9, 2026 03:25
@pengcuo
pengcuo enabled auto-merge (squash) October 9, 2026 03:52
@pengcuo
pengcuo merged commit 7c5d20c into main Oct 9, 2026
9 checks passed
@pengcuo
pengcuo deleted the pzeren/seacache-fp8-sync-20261008 branch October 9, 2026 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants