Skip to content

docs: recommend f32 scales for asymmetric codes above about 10 bits - #107

Merged
cursor[bot] merged 2 commits into
mainfrom
akshey/f32-zero-points-fefb
Sep 28, 2026
Merged

cursor[bot] merged 2 commits into
mainfrom
akshey/f32-zero-points-fefb

Conversation

@aksheyd

@aksheyd aksheyd commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

The README and the asymmetric.quantize docstring offered 2 to 16 bits with any scale type, with no caveat. But f16 and bf16 store each block's zero-point too, and at high bit widths their rounding of it outgrows the codes' own rounding.

Worst error relative to f32 scales, on 65,536 normal weights with blocks of 32:

bits asymmetric f16 asymmetric bf16 symmetric f16 / bf16
10 1.0× 1.0× 1.0×
11 1.0× 2.3× 1.0×
12 1.0× 4.6× 0.9×
14 2.1× 18× 1.0×
16 6.9× 71× 1.1×

On blocks far from zero, like LayerNorm gains around 1, it starts sooner: bf16 is 3× worse at 8 bits and f16 1.5× at 10.

  • the README's paragraph on what scales cost gets one line: above about 10 bits, use f32 scales with asymmetric.quantize
  • the asymmetric.quantize docstring says the same, and that blocks far from zero hit it sooner, as the Rust asymmetric docs already say
  • the Scale docstring says it too

Symmetric codes are unaffected, so quantize's docstring stays as it is. Adaptive codes stop at 8 bits, and its docstring already covers blocks far from zero.

PyPI keeps each release's README as uploaded, so this is worth having in 0.3.0. Merges cleanly with #101, #103, and #106. just lint, just test, and just python pass.

Open in Web Open in Cursor 

f16 and bf16 store each block's zero-point too, and their rounding grows with the bit width, while the README and docstrings offered 2 to 16 bits with any scale type. On normal weights, asymmetric bf16's worst error is 2.3 times f32's at 11 bits and 71 times at 16, and f16's is 7 times at 16. Symmetric codes are unaffected. The README, asymmetric.quantize, and Scale now say to use f32 scales there.
@aksheyd
aksheyd marked this pull request as ready for review September 28, 2026 04:50
@cursor
cursor Bot merged commit 379aa22 into main Sep 28, 2026
10 checks passed
@cursor
cursor Bot deleted the akshey/f32-zero-points-fefb branch September 28, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant