Skip to content

fix(premerge): scope 4c's verify review to the polish, not the whole stack - #804

Open
TheFJK wants to merge 2 commits into
mainfrom
fix/premerge-4c-review-scope
Open

TheFJK wants to merge 2 commits into
mainfrom
fix/premerge-4c-review-scope

Conversation

@TheFJK

@TheFJK TheFJK commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

The defect

### 4c — VERIFY dispatched Skill("code-review", "<level> <PREMERGE_PR>") while its own prose claimed the call was "scoped to the refactor". <PREMERGE_PR> is the stack PR, so nothing was scoped — the verify step re-reviewed the entire stack, and the built-in reviewer's Phase 0 then treats that whole diff as its scope.

4c is not a discovery pass. It asks "did the polish do harm?". Asking a critic "what is wrong with this stack?" answers a different question, from an unbounded set.

Failure scenario. The loop gates STOP_GREEN at attempt N. Phase 4 applies behaviour-preserving polish. 4c re-reviews the whole PR at xhigh — on Opus that cell is 10 inline angles, no verify pass, an independent fresh sample. It rolls one correctness-class finding anywhere in the stack, in a file the polish never touched. assert-green appends blockers_remaining=1 → not_green → the reason is not waitable → revert the simplify commit, push, report simplify=reverted. A correct refactor is reverted, the operator is told the polish did harm when it did not, and the actual finding is discarded with it.

The fix

The scope comes from the checkout, not from an argument.

  • ### 4b commits the polish and deliberately does not push it.
  • ### 4c step 1 reviews with a level and no target, so the reviewer's own default git diff @{upstream}...HEAD — upstream still at the pre-simplify SHA, via Phase 0c's push -u — resolves to exactly the polish commit.
  • ### 4c step 2 is a new premerge-simplify-push fence; steps 3–4 are the existing triage and gate.

Passing the range explicitly was tried, reviewed, and rejected

The first version of this PR passed <parent>..<head> as the target and justified the residual as "benign — a model that ignores the range reviews the whole checkout." That was false. An ignored target falls back to the same no-target default, which on an already-pushed branch is an empty range over a clean tree: zero findings, verdict=green, chore(release) stamped — a vacuous green from the one phase that exists to catch a bad polish, strictly worse than the bug being fixed.

Read out of the shipped binary (2.1.241) rather than off the prompt's enumeration:

  • the parser (hLs → qau) joins every post-level token into one free-text target and rejects no shape;
  • o30 accepts only no-target or ^[@\w][@\w./~^-]*\.\.\.?[@\w][@\w./~^-]*$ and returns on everything else — a range is the only target form that code parses;
  • but o30 runs only when the cell sets finderBudgetHint, and O_e sets it false for o48-xhigh-v1 — the cell claude-opus-5 xhigh resolves to, and xhigh is PREMERGE_DEFAULT_LEVEL. On premerge's own default cell nothing parses the target.

RFC 0021 A3 C8 recorded "a two-SHA range is not in that surface". That is retracted — amended in place, with the bound recorded alongside the retraction, because the bound is why 4c passes no target at all.

Also fixed

  • The push is not gated on --no-post-simplify-review. Moving it into 4c initially put the only push of the simplify commit behind that flag: on --no-post-simplify-review --no-bump, or on any repo without the version ratchet, the polish would be committed locally, never pushed, absent from the stack PR, and destroyed by the next git reset --hard origin/.... Both commands/premerge.md and RFC 0021 document the flag as costing the verification, never the polish.
  • A third arm for ### 4b exiting non-zero (wrong branch, untracked files, absent allowed set, stray files). Each leaves the lens edits uncommitted in a dirty tree, where the empty-range fallback would sweep the scope-rejected edits into the review.
  • An arm for COMMIT=none REASON=no-edits, which previously sent 4c to verify a commit that was never made.
  • Two stale "4c uses a range" claims, wrong step numbers in the extracted reference, and a this file's ... pointer that became false when the text moved files.

Verification

  • tests/premerge.test.sh 280/0, with new section P23. The no-push row, both 4b-prose rows, the flag rows and the bare-PR row are mutation-tested — reintroducing each defect reds its specific row.
  • One of my own new rows was found vacuously passing (a fence body string handed to a helper that greps a file) and corrected to assert_in/assert_not_in.
  • Full .github/workflows/test.yml ubuntu run list: 132/132 green, clean single run.
  • skill-size ratchet holds unraised at 2486 — rationale was extracted into references/ rather than bumping the pin (test(skill-size): the premerge-pipeline waiver bought a ratchet bump instead of an extraction #779).
  • Version consistent at 0.56.2 across all six surfaces plus both CI ratchet locks.

Scope deliberately not taken

Scoping the loop's re-review (Phase 1 at attempt ≥ 2) is the larger half and is #796. Blockers are not carried across attempts — --carry-prior carries suggestions only — so a naive scope there drops an unfixed blocker out of the review window and gates green with it live, which is worse than the pump it would fix. That needs a blocker-carry change in plan first.

4c was safe to fix alone because it is a regression check on one commit, not a discovery pass, so it has no carry problem.

Parked

Not merged, no auto-merge. /merge when you want it.

Closes #795

Fixture added 2 commits August 24, 2026 13:32
…stack

`### 4c — VERIFY` dispatched `Skill("code-review", "<level> <PREMERGE_PR>")`
while its own prose claimed the call was "scoped to the refactor".
`<PREMERGE_PR>` is the stack PR, so nothing was scoped: the verify step
re-reviewed the entire stack and the reviewer treated that whole diff as its
scope.

4c is not a discovery pass — it asks "did the polish do harm". Asking a critic
"what is wrong with this stack" answers from an unbounded set, so any blocker
that sample rolled — including one present all along that the green gate had
not sampled — reached the third table arm as "a blocker the polish introduced"
and reverted a correct, behaviour-preserving refactor.

The scope now comes from the checkout rather than from an argument. 4b commits
the polish and deliberately does not push it; 4c reviews with a level and no
target, so the reviewer's own default `git diff @{upstream}...HEAD` — upstream
still at the pre-simplify SHA, via Phase 0c's `push -u` — resolves to exactly
the polish commit. A new `premerge-simplify-push` fence then pushes.

Passing the range explicitly was tried and rejected. On `o48-xhigh-v1` (the
cell `claude-opus-5` xhigh resolves to, and `xhigh` is the default level)
`finderBudgetHint` is false, so `o30` never parses the target and it reaches
the model as prose. An ignored target does not degrade to a wider review: it
falls back to that same default, which on an already-pushed branch is an empty
range over a clean tree — zero findings and a vacuous green from the one phase
that exists to catch a bad polish.

The push runs regardless of `--no-post-simplify-review`; only the review,
triage and gate are gated on it. It is the only push of the simplify commit,
and both the command file and RFC 0021 document that flag as costing the
verification, never the polish.

Also here:

- a third arm for 4b exiting non-zero (wrong branch, untracked files, absent
  allowed set, stray files), each of which leaves the lens edits uncommitted
  in a dirty tree where the empty-range fallback would sweep them in;
- an arm for 4b's `COMMIT=none REASON=no-edits` early exit;
- RFC 0021 A3 C8 amended: "a two-SHA range is not in that surface" was read
  off the review prompt's enumeration rather than executed. `o30` accepts only
  no-target or a range and returns on every other shape — a range is the only
  target form that code parses. The amendment records the bound as well as the
  retraction, which is why 4c passes no target at all;
- rationale extracted to references/ so the body stays on its pinned size
  ceiling rather than raising it (#779).

Closes #795
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c728ff52-e262-4a3f-b7b8-2dc2b5b9d75e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

premerge: 4c's verify review targets the whole stack, so a resampled blocker reverts a correct simplify commit

1 participant