Skip to content

cuda: fused FWHT quantizer for 64-wide warps - #303

Merged
khosravipasha merged 1 commit into
prismfrom
fix/rocm-wave64-fwht-quantize
Oct 2, 2026
Merged

khosravipasha merged 1 commit into
prismfrom
fix/rocm-wave64-fwht-quantize

Conversation

@bri-prism

@bri-prism bri-prism commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

The Prism release build fails on ubuntu-22-rocm (run 37038350494). The fused FWHT + q8_1 quantizer in quantize.cu asserts QK8_1 == warp_size, so every instantiation fails to compile for the wave64 targets in that job (gfx908, gfx90a, gfx942). windows-hip passes because it only builds 32-wide RDNA targets.

A 64-wide warp holds two complete 32-blocks of the transform output. The quantizer already reduces with warp_reduce_max<QK8_1> / warp_reduce_sum<QK8_1>, which on HIP shuffle with width 32 and so reduce each half on its own. The only 32-wide assumption left is the block index iqs = lane. This changes it to lane % QK8_1 and relaxes the assert to warp_size % QK8_1 == 0. The shared butterfly (ggml_cuda_fwht_block_butterfly) is already generic in the warp size.

On 32-wide warps lane % 32 == lane, so NVIDIA and RDNA behave exactly as before.

Testing

  • ROCm 7.2.1 (rocm/dev-ubuntu-22.04:7.2.1-complete, same version as the release job) with the release job's HIP flags (-DHIP_PLATFORM=amd -DGGML_HIP_ROCWMMA_FATTN=ON):
    • prism at a14c7de, GPU_TARGETS=gfx90a: fails in quantize.cu with the same 32 == warp_size static assert as the release run.
    • This branch, GPU_TARGETS=gfx908;gfx90a;gfx942: ggml-hip builds and links, 0 errors.
  • Not run on wave64 hardware. The change is checked by reading the reduction width and the store indices.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES. Claude Code helped find the cause and write the change.

The fused FWHT + q8_1 quantizer asserted QK8_1 == warp_size, so it did not compile for wave64 HIP targets (gfx908, gfx90a, gfx942) and broke the ROCm release build. A 64-wide warp holds two complete 32-blocks and the 32-wide warp reductions already reduce each half, so index the block with lane % QK8_1 and only require warp_size to be a multiple of QK8_1. 32-wide warps are unchanged.
@bri-prism
bri-prism marked this pull request as ready for review October 2, 2026 19:58

@khosravipasha khosravipasha left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewing on behalf of the maintainers

The wave64 indexing fix looks correct and preserves wave32 behavior. Please update the PR with the ROCm compile results when available; wave64 hardware validation remains outstanding.

@khosravipasha
khosravipasha merged commit 2459f68 into prism Oct 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants