Repository navigation
Conversation
darkclad
force-pushed
the
main-contour-inram
branch
from
October 22, 2026 22:31
b471421 to
d3dfa57
Compare
added 3 commits
October 22, 2026 16:33
The in-memory contour builder now follows each border of the ROI with Suzuki and Abe's border following, the algorithm behind OpenCV's findContours: one closed walk in 8-connected steps per outer border of each fragment and per hole, over exactly the ROI pixels with a 4-neighbour outside the ROI. The walk comes out ordered as it is traced. The builder it replaces drew the borders into a raster and then rebuilt their order with a greedy nearest-neighbour walk. Wherever a boundary pixel had more than two boundary neighbours that walk could dead-end, and everything it had collected was then discarded. On a filled 9066-pixel blob with 495 edge pixels it kept 2, so PERIMETER was 2; on a 2x2 square it kept 3 of 4. Every traced pixel was also reported one pixel right and one pixel down of where it is, because the one-pixel pad the ROI is traced in was never subtracted. Consequences, all measured: - contour pixels are ROI pixels, in the frame of raw_pixels; the out-of-core builder drops the same pad, so the in-RAM and out-of-core contours of a ROI are the same pixel set (the out-of-core one stays relative to the bounding box) - PERIMETER is the summed length of the closed walks, with no chord joining one border to the next - merge_multicontour lists each contour pixel once, so the EDGE_* statistics and PERCENT_TOUCHING count a pixel the walk passes twice, or two borders share, once - the inscribing/circumscribing circle diameters measure from the centroid itself rather than from centroid - 1, and to the ROI's outer borders only (the external contour imea reads), so a hole no longer shrinks the inscribing circle to the hole - the distance-to-contour searches behind the radial distribution and the weighted moments (CPU and GPU) search each border's walk on its own and skip a border whose bounding box cannot beat the best distance so far; walks of up to 10 pixels are scanned exhaustively, so a 2- or 3-pixel contour is never half-skipped - the merged contour and the border boundaries are built once per ROI by the contour feature, and every consumer (neighbours, circles, radius, radial, moments, GPU cache) reads them - a one-pixel ROI is a one-pixel contour: the radial distribution puts its pixel in the first bin instead of dividing by a zero radius and writing far outside its bins - the whole-slide moment weighting reads its slide corners only when the deduplicated contour has all four New tests: closed forms (square, disk, ring, two fragments, one-pixel line, single pixel), a 400-ROI battery checked against the 4-adjacent boundary rule, OpenCV goldens on the blob and the shape2d mask, a one- and a two-pixel ROI and a two-fragment ROI through the radial distribution, the distance searches on short walks and on 100 random multi-border masks, two disk copies weighing twice one copy in the weighted moments, 1x1/1x7/7x1 whole slides, and BorderedContour refusing a temporary. The morphology, radial, moments and fractal pages, their worked examples and the pinned values in feature_extras.toml describe the new contour; the out-of-core notes stay, re-measured. Changed values, shape2d fixture unless stated: PERIMETER 26.935 -> 22.971 (= OpenCV), DIAMETER_EQUAL_PERIMETER 8.574 -> 7.312, GEODETIC_LENGTH 11.132 -> 8.384, THICKNESS 2.336 -> 3.101, ROI_RADIUS_MEAN/MAX/MEDIAN 0.6247/1.4142/1 -> 0.3077/1/0 (= the scikit-image inner-boundary reference), DIAMETER_CIRCUMSCRIBING_CIRCLE 12.332 -> 6.889, DIAMETER_INSCRIBING_CIRCLE 0.828 -> 3.605 (= OpenCV to the external contour), all 24 radial values, all 62 weighted moments (WEIGHTED_SPAT_MOMENT_00 on the 48x40 fixture 2498.71 -> 1895.00 against 1821.28 at exact distances), PERCENT_TOUCHING on the 2x2 neighbours 66.67/33.33 -> 50 (= CellProfiler on all 5 ROIs). Refs PN-243, PN-22, PN-169, PN-70.
The Richardson divider now walks every closed border of the ROI -- the outer border of each fragment and the border of each hole -- on its own, and sums the borders' lengths and steps at each stride. A border no longer than the stride is measured whole, at its closed length, and adds no ruler step, so the mean ruler comes only from the borders the stride walks. It used to walk all borders as one sequence, which measured a chord from the end of one border to the start of the next. A ROI with a single border is unchanged, the FracLac-vetted blob included. Two identical separate disks now have exactly the dimension of one disk (1.0448); walked as one sequence they gave 1.1300. A 17x17 square beside a 3-pixel line has dimension exactly 1; dropping the line at the strides it is too short for gave 1.0262. Both identities are the new invariant tests. The fractal page and its worked example describe the per-border walk. Refs PN-169.
…n oracle reproduce With the contour on the ROI's own edge pixels, three features that no external reference could reproduce now agree with one exactly: - ROI_RADIUS_MEAN agrees with the scikit-image inner-boundary reference on the three disks (2.7517, 6.0470, 12.6860), and is vetted there beside MAX and MEDIAN. - PERCENT_TOUCHING agrees with CellProfiler on all five ROIs of the neighbour scene. At a distance of one pixel CellProfiler's dilation is the 3x3 neighbourhood, so the two definitions count the same pixels; the 3-of-5 divergence on record came from 2x2 borders that had lost a pixel. - DIAMETER_INSCRIBING_CIRCLE and DIAMETER_CIRCUMSCRIBING_CIRCLE equal their closed forms on the r=15 circle, 2 sqrt((R-1)^2+1) and 2R, and keep them with a centred 3x3 hole; on two r=10 disks 40 apart they equal D - 2R and D + 2R. imea is not an oracle for these: its own values on that circle are 46.86 and 12.65. The invariants added with the contour and fractal fixes are registered on FRACT_DIM_PERIMETER, WEIGHTED_SPAT_MOMENT_00 and IMOM_WRM_00. The generators verify every new pin. The feature pages' Validation sections and the generated feature indexes name the new oracles. Registry rows, recipes, the morphology and neighbour matrices, not_covered.md and the audit reports describe the measured state. Refs PN-22, PN-70.
darkclad
force-pushed
the
main-contour-inram
branch
from
October 22, 2026 23:34
d3dfa57 to
2994234
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The in-memory 2D contour was wrong in three ways, and every contour-based feature inherited them:
PERIMETER2.0. A 2×2 square kept 3 of its 4.ROI_RADIUS_*, radial distribution, the inscribing/circumscribing circles, the weighted moments) measured against a shifted boundary.PERIMETER(and everything built on it) added the jump from the end of one border to the start of the next.Follow-ups are on Axle's PN Jira board (https://labshare.atlassian.net/jira/software/projects/PN/boards/851), which is where the PN keys below point. This closes PN-243, PN-22 and PN-169, and the in-memory half of PN-70.
Fixes #497. Fixes #516. The out-of-core contour (#509) is not changed here.
Change
Three commits.
1.
fix(contour): trace every ROI border in order, in absolute coordinates.ContourFeature::gather_multicontouris now Suzuki and Abe's border following (1985), the algorithm behind OpenCV'sfindContours. It returns one closed walk in 8-connected steps per outer border of each fragment and per hole, over exactly the ROI pixels that have a 4-neighbour outside the ROI.buildRegularContoursubtracts the pad.buildRegularContour_nontrivdrops it as well, so the two builders return the same pixel set. The out-of-core contour is still relative to the bounding box (PN-208).PERIMETERis the summed length of the closed walks, with no chord between them.LR::merge_multicontourlists each contour pixel once. A walk passes a one-pixel-wide neck twice, and two borders can share a pixel; theEDGE_*statistics andPERCENT_TOUCHINGcount such a pixel once.circle.cppmeasures from the centroid itself, to the ROI's outer borders only. It usedcentroid - 1, which put the centre two pixels off the shifted contour. A hole's border is not part of the outline (OpenCV's external contour, which imea reads), so a hole no longer shrinks the inscribing circle to the hole; a ROI of several fragments is measured to every fragment's outer border.RadialDistributionFeaturedivided by a zero radius and wrote toradial_count_bins[INT_MIN], an access violation on Windows. A one-pixel ROI now lands in the first radial bin.ContourFeaturebuilds the merged contour once per ROI (LR::contour_pixels_, with the border boundaries inLR::contour_border_ends_), and every consumer reads it: neighbours, circles, radius, radial, moments and the GPU cache.2.
fix(fractal): walk each border separately forFRACT_DIM_PERIMETER. The Richardson divider walks every closed border on its own and sums lengths and steps per stride. A border no longer than the stride is measured whole, at its closed length, and adds no ruler step, so a small hole or fragment keeps its length at every stride and the mean ruler comes only from the borders the stride walks. A 17x17 square beside a 3-pixel line has dimension exactly 1. A single-border ROI is unchanged, the FracLac-vetted blob included.3.
test(vetting): vet what the corrected contour lets an oracle reproduce.ROI_RADIUS_MEAN(scikit-image, three disks),PERCENT_TOUCHING(CellProfiler, 5 ROIs, residual 0), andDIAMETER_INSCRIBING_CIRCLE/DIAMETER_CIRCUMSCRIBING_CIRCLE(closed forms on the r=15 circle: 2√((R−1)²+1) and 2R, which hold with a centred 3x3 hole too; on two r=10 disks 40 apart, D − 2R and D + 2R). The invariants added in commits 1 and 2 are registered onFRACT_DIM_PERIMETER,WEIGHTED_SPAT_MOMENT_00andIMOM_WRM_00. The registry, recipes, matrices and audit reports describe the measured state.Docs. Each commit updates the pages that describe its change. Commit 1: the morphology, radial distribution, moments and fractal pages, their worked examples and the pinned values in
docs/feature_extras.toml, all re-measured; the out-of-core notes stay, re-measured. Commit 2: the fractal page's divider walk. Commit 3: the Validation sections and the generated feature indexes.Changed values
Measured with the Python API on
mainand on this branch. Simple convex ROIs do not move: on the disk R=20,PERIMETERstays 131.88225099390849, and so do every feature derived from it and theEDGE_*statistics.DIAMETER_EQUAL_AREAandFRACT_DIM_BOXCOUNTdo not move on any fixture.PERIMETERDIAMETER_EQUAL_PERIMETERPERIMETERCIRCULARITYPERIMETERGEODETIC_LENGTHPERIMETERTHICKNESSPERIMETERPOLYGONALITY_AVEHEXAGONALITY_AVEHEXAGONALITY_STDDEVFRACT_DIM_PERIMETEREDGE_INTEGRATED_INTENSITYEDGE_MEAN_INTENSITYEDGE_MIN_INTENSITYEDGE_MAX_INTENSITYEDGE_STDDEV_INTENSITYROI_RADIUS_MEANROI_RADIUS_MAXROI_RADIUS_MEDIANDIAMETER_CIRCUMSCRIBING_CIRCLEDIAMETER_INSCRIBING_CIRCLEDIAMETER_MIN_ENCLOSING_CIRCLEPERCENT_TOUCHINGFRAC_AT_D,MEAN_FRAC,RADIAL_CVMEAN_FRAC_722WEIGHTED_SPAT_MOMENT_002498.71Every changed code, one row each (104 codes)
CIRCULARITYDIAMETER_CIRCUMSCRIBING_CIRCLEDIAMETER_EQUAL_PERIMETERDIAMETER_INSCRIBING_CIRCLEDIAMETER_MIN_ENCLOSING_CIRCLEEDGE_INTEGRATED_INTENSITYEDGE_MAX_INTENSITYEDGE_MEAN_INTENSITYEDGE_MIN_INTENSITYEDGE_STDDEV_INTENSITYFRACT_DIM_PERIMETERFRAC_AT_D_0FRAC_AT_D_1FRAC_AT_D_2FRAC_AT_D_3FRAC_AT_D_4FRAC_AT_D_5FRAC_AT_D_6FRAC_AT_D_7GEODETIC_LENGTHHEXAGONALITY_AVEHEXAGONALITY_STDDEVIMOM_WCM_02IMOM_WCM_03IMOM_WCM_11IMOM_WCM_12IMOM_WCM_20IMOM_WCM_21IMOM_WCM_30IMOM_WHU1IMOM_WHU2IMOM_WHU3IMOM_WHU4IMOM_WHU6IMOM_WNCM_02IMOM_WNCM_03IMOM_WNCM_11IMOM_WNCM_12IMOM_WNCM_20IMOM_WNCM_21IMOM_WNCM_30IMOM_WRM_00IMOM_WRM_01IMOM_WRM_02IMOM_WRM_03IMOM_WRM_10IMOM_WRM_11IMOM_WRM_12IMOM_WRM_20IMOM_WRM_21IMOM_WRM_30MEAN_FRAC_0MEAN_FRAC_1MEAN_FRAC_2MEAN_FRAC_3MEAN_FRAC_4MEAN_FRAC_5MEAN_FRAC_6MEAN_FRAC_7PERCENT_TOUCHINGPERIMETERPOLYGONALITY_AVERADIAL_CV_0RADIAL_CV_1RADIAL_CV_2RADIAL_CV_3RADIAL_CV_4RADIAL_CV_5RADIAL_CV_6RADIAL_CV_7ROI_RADIUS_MAXROI_RADIUS_MEANROI_RADIUS_MEDIANTHICKNESSWEIGHTED_CENTRAL_MOMENT_02WEIGHTED_CENTRAL_MOMENT_03WEIGHTED_CENTRAL_MOMENT_11WEIGHTED_CENTRAL_MOMENT_12WEIGHTED_CENTRAL_MOMENT_20WEIGHTED_CENTRAL_MOMENT_21WEIGHTED_CENTRAL_MOMENT_30WEIGHTED_HU_M1WEIGHTED_HU_M2WEIGHTED_HU_M3WEIGHTED_HU_M4WEIGHTED_HU_M5WEIGHTED_HU_M6WEIGHTED_SPAT_MOMENT_00WEIGHTED_SPAT_MOMENT_01WEIGHTED_SPAT_MOMENT_02WEIGHTED_SPAT_MOMENT_03WEIGHTED_SPAT_MOMENT_10WEIGHTED_SPAT_MOMENT_11WEIGHTED_SPAT_MOMENT_12WEIGHTED_SPAT_MOMENT_20WEIGHTED_SPAT_MOMENT_21WEIGHTED_SPAT_MOMENT_30WT_NORM_CTR_MOM_02WT_NORM_CTR_MOM_03WT_NORM_CTR_MOM_11WT_NORM_CTR_MOM_12WT_NORM_CTR_MOM_20WT_NORM_CTR_MOM_21WT_NORM_CTR_MOM_30Notes on the values:
I · log(d + 0.001), withdits distance to the contour. On the 48×40 rectangle the exact distance to the true border givesWEIGHTED_SPAT_MOMENT_00= 1821.28. The old 2498.71 combined the shifted contour with the approximatemin_sqdistsearch; the remaining 4% (1895.00) is the search alone (PN-200). Searching each border on its own lowers the overestimate wherever a ROI has a hole or several fragments (shape2dWEIGHTED_SPAT_MOMENT_00−95.31 with one search over the merged contour, −117.08 per border). Edge pixels now sit atd = 0and weighI · log(0.001) < 0, so several weighted central moments on small fixtures are negative. That follows from the weighting formula and is recorded on PN-200.PERIMETER0 and every pixel is an edge pixel.test_bad_contournow assertsEDGE_MEAN_INTENSITYequals the speckles' mean, where it pinned 0.num_bins - 1scaling). Only the frame changed here, and their mechanics pins are re-measured.Oracles
findContours(RETR_LIST, CHAIN_APPROX_NONE)(4.13.0; 5.0.0 agrees): number of borders, the pixels on them, andPERIMETER, exact, on the PN-243 blob and on shape2d. New generatorgen_contour_opencv.py.measure.perimeter, on the circles fixture: still 999.259, unchanged.find_boundaries(connectivity=1, mode='inner'): the contour's pixel set on every fixture measured, andROI_RADIUS_MEAN/MAX/MEDIANon the three disks.EDGE_*on shape8 unchanged, andPERCENT_TOUCHINGnow agrees on all five ROIs. It used to differ on three because the 2×2 borders had lost a pixel.PERIMETERreproduceDIAMETER_EQUAL_PERIMETER,GEODETIC_LENGTHandTHICKNESS.imea's own inscribing/circumscribing diameters are not an oracle for Nyxus': on the r=15 circle imea reports 46.86 and 12.65.
Tests
New gtest cases, each failing on
main(measured):mainTEST_2D_CONTOUR_SQUARE_ANALYTIC,_DISK_ANALYTICTEST_2D_CONTOUR_RING_ANALYTICPERIMETERoff by +7.058 (chord between the borders)TEST_2D_CONTOUR_TWO_FRAGMENTS_ANALYTICTEST_2D_CONTOUR_LINE_ANALYTICTEST_2D_CONTOUR_SINGLE_PIXEL_ANALYTICTEST_2D_CONTOUR_NOISY_ROIS_BOUNDARY_RULE_INVARIANT(400 seeded ROIs)TEST_2D_CONTOUR_BLOB97_BOUNDARY_RULE_INVARIANT,_BLOB97_OPENCVTEST_2D_CONTOUR_SHAPE2D_OPENCVPERIMETER26.935 against 22.971TEST_2D_MORPHOLOGY_FRACTAL_PERIMETER_TWO_COPIES_INVARIANTTEST_2D_RADIAL_SINGLE_PIXEL_ROI_MECHANICS0xC0000005)Added with the per-border search, the outer-border circles and the short-border divider, each failing on this branch's code before that change (measured):
TEST_2D_CONTOUR_DISTANCE_SEARCH_SHORT_WALK_INVARIANTmin_sqdist101, exact 73TEST_2D_RADIAL_TWO_PIXEL_ROI_MECHANICSFRAC_AT_D_01, not 0.5)TEST_2D_RADIAL_TWO_FRAGMENTS_MECHANICSTEST_2D_MOMENTS_WEIGHTED_DISK_COPIES_INVARIANTTEST_2D_MOMENTS_WHOLESLIDE_THIN_SLIDE_MECHANICS(1x1, 1x7, 7x1)TEST_2D_MORPHOLOGY_CIRCLE_DIAMETERS_HOLED_DISK_ANALYTICTEST_2D_MORPHOLOGY_FRACTAL_PERIMETER_SHORT_BORDER_INVARIANTTEST_2D_CONTOUR_BORDERED_CONTOUR_BINDS_LVALUES_MECHANICSBorderedContourcould bind a temporaryTwo more pin conventions rather than defects:
TEST_2D_MORPHOLOGY_CIRCLE_DIAMETERS_TWO_FRAGMENTS_ANALYTIC(two disks: 20 and 60), andTEST_2D_CONTOUR_DISTANCE_SEARCH_BORDERS_INVARIANT, which checks on 100 seeded multi-border masks that skipping borders by their bounding box changes no distance.TEST_2D_CONTOUR_MULTI_CONNECTED_ANALYTICasserted 4 contours on a fixture that has 3 fragments and no holes (OpenCV agrees). It now asserts 3. The PN-243 blob is the fixturebench_blob97_filled.The re-pinned regression goldens are listed under "Changed values". The out-of-core guard for PN-70 is re-measured: both builders now return the same 112 edge pixels, and the remaining gap is the bounding-box frame of PN-208.
Gates:
pytest tests/python/262 passed, 9 skipped, 8 failed. The 8 failures are the@pytest.mark.arrowcases and thearrow_output.pydocs example of a build without Arrow.pytest tests/python/262 passed, 4 skipped (Arrow deselected); 0 sanitizer diagnostics.check_test_names.py,check_coverage.py,report_features.pyand everyscan_*_coverage.pypass with--check; the Sphinx build passes with-Wat every commit.The GPU changes (per-border search in the weighting kernels, the border-boundary buffer, the whole-slide corner guard) were built and run on a CUDA build: on 8 multi-border ROIs and on 1x1, 1x7, 7x1 and 5x4 slides the weighted raw moments match the CPU exactly. The weighted central moments differ between CPU and GPU by up to about 2x, the same before and after the per-border search; that is recorded on PN-242, with the GPU's missing pixel-size scaling.
Not in this PR
PERIMETER, PN-261, PN-244) are a separate change that starts from this one. On ROIs with holes or fragments the out-of-corePERIMETERis further from in-RAM than before, because in-RAM now includes the hole borders.