Repository navigation
fix(runners): one flight that cannot be converted no longer stops the run - #71
Merged
Merged
Conversation
… run FleetRunner converted the whole JSON input with neats_json_to_flights before any per-flight error handling, so one flight with an invalid trajectory timestamp (or any malformed flight object) raised and stopped the run for every flight. neats_json_to_flights gains an opt-in `failures` list. When given, a flight that cannot be converted is left out and recorded as a FlightConversionFailure (input index, identifying attrs, error); without it, behaviour is unchanged. A top level that is an object or a string is a whole-file error and raises TypeError instead of being recorded flight by flight. Recorded errors carry no tracebacks, so they do not keep the input alive. FleetRunner._load_trajectories and scripts/run_json.py use the list: a failed flight becomes a "Failed at parsing" error record and the other flights go on. If every flight fails, the run stops before loading the weather and returns only error records. The run_json summary now prints the flight id of nested fleet error records. The default conversion path is unchanged: on 148 NEATS JSON files (85,128 flights) its output is identical to before. The units audit report records the status (D2 follow-up, D1 note). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #70 (units audit D2).
FleetRunnerconverted the whole JSON input before anyper-flight error handling, so one flight with an invalid trajectory timestamp (or any
malformed flight object) stopped the run for every flight.
neats_json_to_flightsgains an opt-infailureslist. When given, a flight that cannotbe converted is left out and recorded as a
FlightConversionFailure(input index,identifying attrs, error). Without it, behaviour is unchanged (Dataiku recipes and other
direct callers are not affected).
TypeError, not onefake failed flight (audit D1 stays open: accepting a bare object is not decided).
FleetRunner._load_trajectoriesandscripts/run_json.pyuse it: a failed flight becomesa
Failed at parsingerror record and the other flights go on. If every flight fails, therun stops before loading the weather and returns only error records.
Evidence that nothing breaks
exact comparison before/after: 0 differences.
(
ao_json_sample.json, trajectory stored as a list) now records its 5 flights asfailures instead of stopping the run.
with non-existent weather paths and a check that the input is freed. New end-to-end test:
5 golden flights + 1 bad flight → the 5 golden results unchanged + 1 error record.
ruff,ruff format, projectmypyclean.Known, not in this PR
(pre-existing); a structured marker would be more robust.
mypyhook is broken for everyone (numpy 2.5.4 stubs vs Python 3.11 target);commit made with
SKIP=mypy, project mypy passes. Separate chore PR to follow.