feat: use LUT for AEF dequantization - #10
Conversation
jakenotjay
left a comment
There was a problem hiding this comment.
The change itself lgtm, but I'd like it applied to the lazy path too for dask, at scale this should improve speed.
| # Keep the DataArray path elementwise so dask-backed arrays stay lazy. | ||
| nodata_mask = data == nodata_value | ||
| normalized = data.astype(np.float32) / divisor | ||
| dequantized = (normalized**2) * np.sign(data) |
There was a problem hiding this comment.
I like this change, but for dask we're still applying the old method. The comment is misleading, we should just apply it via apply_ufunc! Worth benchmarking over a large area with dask, because if its a significant speedup, we'll see a performance improvement on dequantising at large scale.
29421cc to
49d5cf2
Compare
|
Thanks, agreed. I updated this so the Rebased on current uv run pytest tests/test_utils.py
uv run ruff check aef_loader/utils.py tests/test_utils.py
uv sync --group benchmark && uv run ty check |
49d5cf2 to
65d324a
Compare
|
I also ran a live dask-backed source.coop check against public 2018 AEF data. The dequantized dask result stayed lazy before compute and matched an explicit reference-formula calculation from the same raw window exactly ( One useful thing this exposed: with multiple tiles, current Validation after that hardening: uv run pytest tests/test_utils.py
uv run ruff check aef_loader/utils.py tests/test_utils.py
uv run ty check |
65d324a to
2fc8ea9
Compare
Problem
dequantize_aefrecomputed the same int8-to-float32 formula for every pixel value. AEF stores only 256 possible raw values, so large materialized or dask-backed windows spend avoidable time repeating(v / 127.5) ** 2 * sign(v)and applying a separate nodata mask.Fix
This adds a 256-entry lookup table and maps raw values through it with
lut[raw + 128]. The-128 -> NaNnodata mapping is folded into the table.The plain ndarray path uses the LUT directly. The
xr.DataArraypath uses the same LUT throughxr.apply_ufunc(..., dask="parallelized"), so dask-backed arrays stay lazy and apply the lookup per block. Float chunks with existing NaN gap-fill are handled by preserving NaNs and mapping finite raw int8-valued floats through the LUT.In local checks on a 2048 x 2048 x 64 materialized AEF window, this changed numpy dequantization from about 2702 ms to 1582 ms, while preserving identical output.
Tests
Adds
tests/test_utils.pycoverage asserting:Ran locally: