Skip to content

Stop project_volume from corrupting the caller's volume - #30

Merged
gouwens merged 2 commits into
mainfrom
worktree-fix-issue-20-sentinel
Aug 31, 2026
Merged

gouwens merged 2 commits into
mainfrom
worktree-fix-issue-20-sentinel

Conversation

@gouwens

@gouwens gouwens commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Fixes #20.

kind="max"/"min" has to keep the zero-padded entries of paths out of the reduction. It did that by writing the dtype's extreme value into volume.flat[0] and never restoring it, so the caller's array came back modified — and with hemisphere="both" the second pass wrote through the np.flip(volume, axis=2) view, clobbering volume[0, 0, -1] too.

The sentinel is now substituted into the gathered copy:

values = volume.flat[self.paths]
values[self.paths == 0] = min_val

self.paths == 0 is exactly the set of entries that previously read the overwritten voxel, so the reduction is unchanged. Both Isocortex2dProjector._project_volume_to_view and IsocortexEntireProjector.project_volume were affected and both are fixed. Read-only volumes — a memory-mapped reference atlas, say — now project instead of raising ValueError: assignment destination is read-only.

Why not save-and-restore

The issue offers either. Masking never touches the input at all, so it also covers read-only arrays and does not briefly publish a corrupted volume to a concurrent reader.

Cost

Measured at streamline scale (250k paths × 200 padded length, gathering from a 250M-voxel volume): results identical, peak memory unchanged, the max path ~3% slower. The bool mask is a temporary an order of magnitude smaller than the intp index array the existing .flat[paths] gather already allocates. The mean/sum branches directly below already mask this way, with a full second copy.

Tests

tests/test_project_volume_mutation.py, 30 cases over both projector classes:

  • the caller's volume is unchanged for every hemisphere (left/right/both), both kinds, and four dtypes covering the np.iinfo and np.finfo branches — volume.flat[0] and volume[0, 0, -1] are asserted by name so a regression says which cell;
  • a read-only volume projects;
  • and the other half of the contract: the reserved voxel is still excluded, checked as exact per-streamline equality against a reduction computed straight from the fixture geometry, with index 0 loaded as the unique global extreme so a leak cannot hide.

20 of the 30 fail against the unfixed source. The repeatability tests are not among them and say so in their docstrings: the old code wrote the same sentinel every call, and neither clobbered cell lies on a mini-CCF streamline, so a repeated projection could not see the corruption. They are labelled characterizations rather than left to look like detectors.

Full suite: 102 passed.

Note for #19

PR #19 pins this defect with three xfail(strict=True) tests. Once this merges they become XPASS failures — that is the intended signal to delete those markers on rebase. Its tests/test_projector_2d.py / tests/test_entire_projector.py coverage and this file overlap and can be consolidated there; I kept the fix on main rather than in the stack so it does not wait on #18.

docs/source/glossary.rst, the MiniCCF.volume docstring and the CLAUDE.md gotcha all described the mutation as current behaviour and are updated.

🤖 Generated with Claude Code

gouwens and others added 2 commits August 31, 2026 13:55
For kind="max"/"min" the reduction has to keep the zero-padded path entries
out of the result, and it did that by writing the dtype's extreme value into
volume.flat[0] and never restoring it. That modified the caller's array; with
hemisphere="both" the second pass wrote through the np.flip view, so
volume[0, 0, -1] was clobbered as well.

Substitute the sentinel into the gathered copy instead, in both
Isocortex2dProjector._project_volume_to_view and
IsocortexEntireProjector.project_volume. paths == 0 is exactly the set of
entries that used to read the overwritten voxel, so the reduction is
unchanged; read-only volumes now project as well. Measured at streamline
scale (250k paths x 200), results are identical, peak memory is unchanged and
the max path is ~3% slower.

Fixes #20

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gouwens
gouwens marked this pull request as ready for review August 31, 2026 21:01
@gouwens
gouwens merged commit 01a0f45 into main Aug 31, 2026
4 checks passed
@gouwens
gouwens deleted the worktree-fix-issue-20-sentinel branch August 31, 2026 21:11
ablot added a commit to znamlab/ccf_streamlines that referenced this pull request Aug 31, 2026
Brings lab up to date with main: uv conversion, src/ layout, mini-CCF
test fixtures and CI, and merged upstream PRs AllenInstitute#28/AllenInstitute#29/AllenInstitute#30.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

project_volume silently corrupts the caller's volume

1 participant