Skip to content

fix(fci): bounded FDHSI entry selection - #12

Open
TurielD wants to merge 5 commits into
SpectraWorksOSS:mainfrom
TurielD:fix/fci-entry-selection
Open

TurielD wants to merge 5 commits into
SpectraWorksOSS:mainfrom
TurielD:fix/fci-entry-selection

Conversation

@TurielD

@TurielD TurielD commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

FCI fetch currently reopens every BODY header over serial EUMDAC ranges for every repeat cycle. Reuse measured provider x/y grids on the same connector, guarded by collection, published product/processor versions, platform/encoding/processing identity, exact chunk set and requested channels. Later matching cycles select their bbox with zero metadata range calls. Changed/missing identity falls back to measured selection and records the reason in result, Dataset and canonical lineage.

Cold inspection requests demanded 4 KiB metadata pages through a bounded pool (default four workers). Inspection suspends on cache misses so network waits happen outside HDF5?s global lock. Range-refusing servers remain rejected. No hardcoded chunk layout is assumed.

Validation: 49 FCI tests pass, including 42 first-cycle versus 0 second-cycle metadata ranges for two channels; identical cached/measured entries, changed bbox and reversed entry order, version/processor/count/missing-version fallback, bounded concurrent ranges, and existing Range refusal. Full repository tests: 452 passed in an isolated editable-install environment (213 dependency warnings). Initial global-environment run had 451 passed and one installed-version mismatch (0.5.0 versus checkout 0.6.1); the isolated install resolves it.

Limits: real-header layout stability and the RefCal 25-cycle/30-minute budget remain unverified. The public PUG guide URL returned only the portal JavaScript shell. Reuse is explicitly documented as assuming stable coordinates within the guarded identity. No live EUMETSAT smoke, RefCal compute, merge or release was performed. RefCal must prove this commit cloud-side before adoption. Reuse the same FCIConnector across cycles; the cache is in memory only.

?? Generated with Claude Code

TurielD and others added 5 commits October 25, 2026 03:04
Cache measured provider coordinate layouts across repeat cycles with version and chunk-set guards. Fetch demanded metadata pages concurrently outside HDF5 locks and record selection lineage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add test_fetch_incomplete_version_reuses_only_checksum_bound_product: fresh-process exact-product reuse and checksum mismatch remeasurement. Keep versioned cross-cycle cache guards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Provider body enumeration is unordered. Sort all row-aligned values, flags and pixel times together. Keep native-coordinate gaps. Parameterize the native parity test over reversed entry order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant