docs: make the README's adaptive and np.savez recipes work on every layer - #103
Merged
Merged
Conversation
…ayer The adaptive line stopped at the first layer with outliers, where a block needs more than 8 bits and ToleranceTooTightError is raised, and its np.std(weights) failed on PyTorch tensors. It now uses weights.std(), which works on arrays and tensors, and a short retry with the error's smallest_tolerance follows the list, saying it loosens every block. The np.savez recipe saved bits=None for an adaptive tensor, which np.load refuses, so it now leaves out whichever part is None, and the test runs that exact recipe for every kind and scale type.
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two README recipes failed as written on real models.
Adaptive. On layers with outliers, a block can need more than 8 bits, and
adaptive.quantizeraisesToleranceTooTightError. That's on purpose since #76, but the README didn't say so, so a loop over a model's layers stopped at the first outlier layer. Itsnp.std(weights)also failed on PyTorch tensors, since NumPy hands the call toTensor.std, which doesn't take NumPy's arguments.weights.std(), which works on NumPy arrays and PyTorch tensors, including ones that require grad or hold bf16. A plain list needsnp.stdsmallest_tolerance, which always passes, since it's half an 8-bit step of the widest block. The README says it loosens every block, not just the one with the outlierOn 1,000,000 Student-t weights with 3 degrees of freedom, a common stand-in for heavy tails, the first call raises and the retry works at 0.18 of a standard deviation.
np.savez. For an adaptive tensor,bits=q.bitssavedNone, whichnp.loadrefuses withoutallow_pickle=True. The recipe now saves every part and leaves out whichever ofbitsandblock_bitsisNone, so one recipe works for every kind:Run verbatim through a real file, it round-trips all 12 combinations of kind and scale type, empty tensors included. The existing test now runs this recipe instead of branching on the kind.
#98 changed the next two bullets, so this branch has
mainmerged in, keeping #98's wording. PyPI keeps each release's README as uploaded, so this is worth having in 0.3.0. Merges cleanly with #101.just lint,just test, andjust pythonpass.