fix(curation,dataset): query check_runs_latest in coverage and dataset SQL

[k0shir0]

Fixes #676.

Datasets and coverage now read the latest check result from check_runs_latest, so a noncritical check that passes and then errors on replay withdraws the episode from the default dataset and withdraws its coverage. Adds separate regression tests for dataset selection and coverage, and updates the catalog guide and dataset policy docstring.

Validation in an isolated Ubuntu WSL checkout with Python 3.12.3, the locked development environment, pinned FFmpeg, and Python development headers:

• .venv/bin/python -m pytest -q tests/test_dataset.py tests/test_catalog_curation.py tests/test_stage_planning.py — 235 passed.
• uv run --locked --no-sync pytest -q -n 4 --tb=short — 2,506 passed, 8 skipped.
• .venv/bin/ruff check, .venv/bin/ruff format --check, .venv/bin/ty check, and git diff --check — passed.
• Both new tests fail at their final assertions when the original check_runs queries are temporarily restored, and pass with the fix.
• Markdown link check (Lychee 0.24.2 on Windows) — 0 errors:

lychee --no-progress --include-fragments --exclude '^https://github\.com/Hebbian-Robotics/hflow/(issues|security/advisories/new)$' --exclude-path 'references[\\/]mcap-spec\.md$' --exclude-path 'references[\\/]foxglove-CompressedVideo\.proto$' .

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Changes how dataset and coverage queries filter check results.

The PR appears safe to merge; no outstanding finding was established.

What we checked:

• Replay tests might skip the check: Yes. Without SYNC, the stage runs on the supplied recording, and the new tests check the resulting error.

Summary

Coverage and the default dataset now use each check’s latest run, so an older successful result no longer counts after a replay error. The episode stays in the coverage total, while the default dataset leaves it out if a required step’s latest run is not settled.

• Updates curation coverage and dataset selection to read latest check results.
• Adds replay-error tests and updates the catalog guide.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Check runs again] --> B[Latest run is an error]
  B --> C[Episode loses check coverage]
  B --> D[Episode leaves default dataset]

Loading

_{Reviews (2) · Last reviewed commit: "add replay error regression tests, clari..."}

[kstonekuan]

Thanks, the change is right. Please add the two regression tests from #676 so this can't quietly regress: a check passes, then errors on a replay, and the episode drops out of default_dataset_sql's result and out of coverage. test_a_step_that_errored_on_its_latest_run_is_planned_for_retry in tests/test_stage_planning.py shows how to set up the pass-then-error sequence.

[k0shir0]

@kstonekuan, I added both requested pass-then-error regression tests in 06bf4bc. They verify that a noncritical check's replay error removes the episode from the default dataset and withdraws its coverage while retaining the coverage denominator. Both tests fail with the original queries and pass with the fix; the policy documentation is updated too.

Validation: 2,506 tests passed, 8 skipped; lint, formatting, type checks, and Markdown links passed. Please review the updated PR when you have a chance.

[kstonekuan]

LGTM, thanks for the tests and the doc updates. Merging.