Skip to content

feat(gate): capture overrides the test-presence diff heuristic (#3502)#175

Merged
veksen merged 1 commit into
mainfrom
feat-untested-gate-capture
Jul 14, 2026
Merged

feat(gate): capture overrides the test-presence diff heuristic (#3502)#175
veksen merged 1 commit into
mainfrom
feat-untested-gate-capture

Conversation

@veksen

@veksen veksen commented Jul 13, 2026

Copy link
Copy Markdown
Member

What

The test-presence gate (evaluateTestPresence) is a pure diff heuristic — it flags a data-access change when no related data-layer test changed alongside it, matching "related" by directory or filename stem. That misfires when a repo's test doesn't mirror the source path, even though a real-DB test covers the change.

This teaches the gate to prefer capture over the guess: when the run captured new query surface, a real-DB test demonstrably executed the change against Postgres, so the flag is dropped.

Why

Capture is ground truth for the exact blind spot this gate exists to catch: a query that runs against Postgres in no test produces no captured query. If QD captured new queries, the diff heuristic's "no related test" conclusion is contradicted by observed execution.

Concrete case — Nutcracker #749

  • The PR adds findByUser to src/db/postgres.ts and a real-DB test in tests/pg/postgres.test.ts.
  • The gate fired "no related data-layer test": tests/**pg**/postgres.test.ts doesn't share a directory or stem with src/**db**/postgres.ts, so isRelated missed it.
  • Meanwhile QD captured 2 new queries in the same run (the LIMIT 50 shape — straight from the test's findByUser(u, 50), not the route's 100/500) and listed them in the very same comment that claimed nothing exercises them.

With this change, newQueryHashes.length > 0 → the verdict is null → the warning never posts.

How

  • evaluateTestPresence(files, config, capture?) gains an optional capture with newQueryHashes. Non-empty → return null.
  • main.ts feeds comparison.newQueries hashes into the gate (already computed before the gate runs).

Limitation (next rung, #3503)

Suppression is run-level: a PR that tests file A but ships an untested query in file B is fully cleared for now. Per-query→file attribution is #3503. This errs on the under-fire side the gate already favours (warn-only).

Tests

  • New: capture with new-query hashes clears the flag (the #749 shape); empty capture still flags. Full gate suite green (24), typecheck clean.

…502)

The test-presence gate is a pure diff heuristic: it flags a data-access change
when no related data-layer test changed alongside it. The "related" match is by
directory or filename stem, so a repo whose test doesn't mirror the source path
false-positives even when a real-DB test covers the change — e.g. Nutcracker
#749, where `tests/pg/postgres.test.ts` exercises `src/db/postgres.ts` against
Postgres but doesn't share its directory or stem.

Capture is ground truth for the blind spot this gate exists to catch: a query
that runs against Postgres in no test produces no captured query. So when the
run captured new query surface, a real-DB test demonstrably ran the change —
observed execution now overrides the diff guess and the flag is dropped.

Suppression is run-level for now; per-query→file attribution (which would let a
partially-tested PR still flag its one uncovered file) is the next rung, #3503.
It errs on the safe under-fire side the gate already favours.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor Analysis

3 queries analyzed

0 regressed · 0 improved · 0 new · 0 removed

2 pre-existing issues

Using assumed statistics (10000000 rows/table). For better results, sync production stats.

More detail → get_ci_run({ runId: "019f5dae-9758-7db3-a665-631f897d95c1" }) · view run · docs

@veksen veksen merged commit 3b71941 into main Jul 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant