Skip to content
Draft
16 changes: 12 additions & 4 deletions .github/workflows/agent-review-runtime-quality-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ name: Agent Review Runtime Quality CI

on:
pull_request:
branches: [main]
paths:
- ".github/workflows/agent-review-runtime-quality-ci.yml"
- ".github/workflows/noema-review.yml"
Expand Down Expand Up @@ -166,10 +165,13 @@ jobs:
id: affected_suites
shell: bash --noprofile --norc -e -o pipefail {0}
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
test "$(git rev-parse HEAD)" = "$HEAD_SHA"
git fetch --no-tags --prune origin "refs/heads/$BASE_REF:refs/remotes/origin/$BASE_REF"
change_base_sha="$(git merge-base "refs/remotes/origin/$BASE_REF" "$HEAD_SHA")"
test -n "$change_base_sha"
noema_suite=false
opencode_suite=false
strix_suite=false
Expand Down Expand Up @@ -334,7 +336,7 @@ jobs:
exact_artifact_suite=true
;;
esac
done < <(git diff --name-only "$BASE_SHA...$HEAD_SHA")
done < <(git diff --name-only "$change_base_sha...$HEAD_SHA")

{
echo "noema=$noema_suite"
Expand Down Expand Up @@ -582,9 +584,15 @@ jobs:
tests/test_exact_artifact_quality_single_runner.py

- name: Verify consolidated workflow contract
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
python -m pytest -q tests/test_agent_review_runtime_quality_consolidation.py
python -m compileall -q tests/test_agent_review_runtime_quality_consolidation.py
git diff --check "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}"
git fetch --no-tags --prune origin "refs/heads/$BASE_REF:refs/remotes/origin/$BASE_REF"
change_base_sha="$(git merge-base "refs/remotes/origin/$BASE_REF" "$HEAD_SHA")"
test -n "$change_base_sha"
git diff --check "$change_base_sha...$HEAD_SHA"
git diff --exit-code
18 changes: 17 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,20 @@
### Agent runtime quality compares the live base graph

- `Agent Review Runtime Quality CI` now derives changed paths and whitespace
checks from the merge-base of the fetched live base ref and the exact PR
head. Long-lived PR events can no longer make already-protected-main files
look like new PR whitespace, while exact-head checkout and fail-closed diff
checks remain unchanged. Exact failure evidence is `.github#1678` run
`36804488453`, job `110185716853`.
- Refetch the base ref immediately before both merge-base decisions. A base
advance during the quality job can no longer revive the same stale-diff
failure at the terminal whitespace gate.
- Preserve canonical parser/security/coverage owner `.github#2530@dc54310c` as
an ordinary second parent. The integrated PR exposed that this quality
workflow still admitted only pull requests targeting `main`, so its own
stacked exact head produced no quality run. A RED contract now requires
stacked-base admission; the minimal repair removes only that base filter.
CodeQL and review evidence remain mandatory; release admission stays HOLD.
### Authorized Draft reviews reach the exact-head receiver

- Carry an explicit `draft_review_only` boolean from the merge scheduler to the
Expand Down Expand Up @@ -126,7 +143,6 @@
from silently returning to the vulnerable versions. Protected integration,
immutable consumer-pin advancement, and fresh exact-head hosted security
Checks remain required before release admission.

### Intel macOS native archives are bound to x86_64 bytes

- The release prescreener now requires every native member in an Intel macOS
Expand Down
64 changes: 64 additions & 0 deletions docs/doctoring/agent-review-live-base-diff-20261001.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Agent review live-base diff RCA

Status: Proposed shared-workflow repair; hosted exact-head evidence and
independent review remain mandatory.

## Failure evidence

Draft `.github#1678` exact head
`b9651115be28f9dd46f8b93a2a753b2652c57970` failed Agent Review Runtime
Quality CI run `36804488453`, job `110185716853`, step `Verify consolidated
workflow contract`. All preceding contract suites succeeded. The terminal
`git diff --check` used event base `f250638827f8252b0d9e5cb2601f4d333f96162f`
and reported 407 whitespace violations across four files.

## Root cause and boundary

The PR head already contains protected `main@37b10243cec3d160ecc9c1be75c71428b160a703`
as an ancestor through ordinary owner integration. The pull-request event still
carried its older base SHA, so the workflow treated intervening protected-main
history as the PR delta. The four files are clean relative to the live base;
rewriting their historical contents in the consumer would hide the workflow
identity defect and duplicate the `.github` control-plane responsibility.

## Repair and verification

The changed-path selector and terminal whitespace gate refetch the exact base
ref immediately before computing its merge-base with the exact head. This
closes the second stale window where the base could advance during a long
quality job after checkout but before the terminal whitespace gate. The head
checkout assertion remains exact, and a failed fetch or missing merge-base
fails closed.

The original regression contract was changed first and failed in two cases
against the event-SHA implementation. A follow-up RED contract then failed
because neither merge-base decision refetched the base; the minimal repair adds
one fail-closed fetch at each decision. After the workflow repair, 36 focused
consolidation, single-runner, autofix-context, and runtime-budget tests pass.
Applying the same command to the real #1678 graph resolves the live change base to
`37b10243cec3d160ecc9c1be75c71428b160a703`; `git diff --check` succeeds.

Completion requires a dedicated owner PR, exact-current-head hosted Checks,
qualifying independent review, ordinary protected-main integration, ordinary
merge of the owner into #1678, and a fresh successful #1678 run. Pending,
queued, skipped, or predecessor results are not passing evidence.

## Canonical owner integration and stacked admission

Before dependent exact-head revalidation, the repair ordinary-merges canonical
parser/security/coverage owner
`.github#2530@dc54310c8c5ee6637274e7e82f5ea53d64ad91e6`. The resulting owner PR head
`b66036e3702b95d47fc7eac5eb6097e198d49997` is an ordinary two-parent merge,
but it produced no Agent Review Runtime Quality run: the workflow's
`pull_request` trigger still admitted only base `main`, while this PR now
targets the canonical owner branch.

The regression contract failed against that filter. The minimal repair removes
only the pull-request base restriction and retains path selection, read-only
permissions, exact-head checkout, and PR-stable concurrency. This permits the
owner workflow to produce exact-head evidence on canonical stacked bases; it
does not make a pending or skipped result passing evidence.

The merge preserves both lineages without force or rebase and keeps dependency
repair in the canonical owner. The resulting head must rerun all Checks;
predecessor success is causal evidence only, not admission evidence.
6 changes: 6 additions & 0 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@

이 문서는 제품·기술·운영 Gap을 현재 문서와 현재 GitHub 상태에 묶어 두는 기준선이다. 새 작업은 먼저 이 문서의 Gap ID를 PR 설명과 테스트 증거에 연결하고, PR의 정확한 exact HEAD·Checks·리뷰를 다시 수집한 뒤 구현한다. 표의 상태는 작성 시점의 관측값이므로, 병합 판단에는 재사용하지 않는다. 이 인벤토리는 스냅샷이며 merge authorization이 아니다.

### 2026-10-01 live-base diff incident delta

| Gap ID | 상태 | exact-head evidence | causal owner / next gate |
|---|---|---|---|
| CONTROL-AGENT-QUALITY-LIVE-BASE-01 | **Proposed — live-base source repaired; stacked-owner admission repaired; hosted verification pending** | `.github#1678@b9651115…`의 Agent Review Runtime Quality CI run `36804488453`, job `110185716853`은 이벤트에 고정된 과거 base `f2506388…`와 exact head를 비교해, 현재 protected `main@37b10243…`에 이미 존재하는 CSV CRLF·라이선스 fixture 공백 407건을 PR delta로 오인했다. Canonical owner `.github#2530@dc54310c…`를 일반 두-parent 병합한 #2547 head `b66036e3…`에서는 workflow path가 바뀌었는데도 `pull_request.branches: [main]` 때문에 owner workflow 자체가 시작되지 않았다. | Canonical workflow owner는 `.github/workflows/agent-review-runtime-quality-ci.yml`이다. changed-path 선택과 최종 whitespace gate는 각각 base ref를 즉시 다시 fetch한 뒤 exact head와의 merge-base를 사용한다. stacked-base 계약 RED 뒤 base 제한 한 줄만 제거해 path filter·read-only 권한·exact-head checkout·PR-stable concurrency를 보존했다. 새 exact-head hosted Checks·독립 리뷰·ordinary protected merge 후 #1678이 owner commit을 일반 병합하고 exact-head run을 다시 통과해야 한다. |

### 2026-09-30 central coverage owner stack delta

| Gap ID | 상태 | exact-head evidence | causal owner / next gate |
Expand Down
67 changes: 64 additions & 3 deletions tests/test_agent_review_runtime_quality_consolidation.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,16 @@ def test_changelog_only_edits_do_not_boot_the_consolidated_runner() -> None:
assert ' - "CHANGELOG.md"' not in trigger


def test_runtime_quality_admits_stacked_pull_requests() -> None:
"""A canonical owner base must not suppress exact-head quality evidence."""

pull_request_trigger = _workflow_text().split(" pull_request:\n", 1)[1].split(
"\nconcurrency:\n", 1
)[0]

assert "branches:" not in pull_request_trigger


def test_consolidated_workflow_preserves_all_contract_suites() -> None:
"""Keep the retired Noema, OpenCode, and Strix evidence in one job."""

Expand Down Expand Up @@ -128,16 +138,67 @@ def test_consolidated_workflow_preserves_all_contract_suites() -> None:
assert required_path in workflow


def test_exact_head_is_verified_before_selected_suites_run() -> None:
"""Reject a checkout that differs from the pull request's current head."""
def test_exact_head_uses_live_base_merge_base_for_changed_paths() -> None:
"""Ignore stale event base SHAs while preserving exact-head selection."""

workflow = _workflow_text()
selector = workflow.split(
"- name: Select affected contract suites", 1
)[1].split("- name: Install exact hash-verified base dependencies", 1)[0]

assert 'test "$(git rev-parse HEAD)" = "$HEAD_SHA"' in selector
assert 'git diff --name-only "$BASE_SHA...$HEAD_SHA"' in selector
assert "BASE_REF: ${{ github.event.pull_request.base.ref }}" in selector
assert (
'change_base_sha="$(git merge-base '
'\"refs/remotes/origin/$BASE_REF\" \"$HEAD_SHA\")"'
in selector
)
assert 'git diff --name-only "$change_base_sha...$HEAD_SHA"' in selector
assert "github.event.pull_request.base.sha" not in selector


def test_whitespace_gate_uses_live_base_merge_base() -> None:
"""Check only the current PR delta when an old event base is stale."""

workflow = _workflow_text()
self_test_step = workflow.split(
"- name: Verify consolidated workflow contract", 1
)[1]

assert "BASE_REF: ${{ github.event.pull_request.base.ref }}" in self_test_step
assert "HEAD_SHA: ${{ github.event.pull_request.head.sha }}" in self_test_step
assert (
'change_base_sha="$(git merge-base '
'\"refs/remotes/origin/$BASE_REF\" \"$HEAD_SHA\")"'
in self_test_step
)
assert 'git diff --check "$change_base_sha...$HEAD_SHA"' in self_test_step
assert "github.event.pull_request.base.sha" not in self_test_step


def test_live_base_is_refetched_before_each_merge_base_decision() -> None:
"""Prevent a base advance during the job from reviving stale diff evidence."""

workflow = _workflow_text()
live_base_fetch = (
'git fetch --no-tags --prune origin '
'"refs/heads/$BASE_REF:refs/remotes/origin/$BASE_REF"'
)
merge_base = (
'change_base_sha="$(git merge-base '
'\"refs/remotes/origin/$BASE_REF\" \"$HEAD_SHA\")"'
)

assert workflow.count(live_base_fetch) == 2
for workflow_section in (
workflow.split("- name: Select affected contract suites", 1)[1].split(
"- name: Install exact hash-verified base dependencies", 1
)[0],
workflow.split("- name: Verify consolidated workflow contract", 1)[1],
):
assert workflow_section.index(live_base_fetch) < workflow_section.index(
merge_base
)


def test_review_repair_suite_is_selected_and_conditionally_executed() -> None:
Expand Down
Loading