ci(noema): review binary-only document PRs via a base/head object diff

.github/actions/noema-review/two_phase.py

@@ -164,6 +164,7 @@ def prepare_verdict(repo: str, number: int, expected_head: str, path: Path) -> i
         return 0
 
     diff, truncated = gate.fetch_diff(repo, number)
+    diff, truncated, unobserved = gate.augment_binary_document_diff(repo, number, pull_request, diff, truncated)
     changed_files = gate.fetch_changed_files(repo, number)
     changed_paths = tuple(file_path for file_path, _status in changed_files)
     review_context = gate.build_review_context(repo, number, pull_request, changed_files)
@@ -177,6 +178,7 @@ def prepare_verdict(repo: str, number: int, expected_head: str, path: Path) -> i
             expected,
             review_context,
             changed_paths,
+            unobserved,
         )
     except gate.NoemaTransportError as exc:
         _emit_transport_capacity_outputs(exc, expected_head=expected)

.github/workflows/noema-review.yml

@@ -764,6 +764,10 @@ jobs:
           NOEMA_REVIEW_ACTOR: ${{ steps.noema_github_app_token.outputs['app-slug'] && format('{0}[bot]', steps.noema_github_app_token.outputs['app-slug']) || '' }}
           NOEMA_REVIEW_INSTALLATION_ID: ${{ steps.noema_github_app_token.outputs['installation-id'] }}
           NOEMA_TRANSPORT_RETRY_ATTEMPT: ${{ github.event.client_payload.transport_retry_attempt || 0 }}
+          # Exact allowlist of declared corresponding-author emails that a
+          # manuscript's front matter may carry; they reach the model only as
+          # [CORRESPONDING_AUTHOR_EMAIL]. Empty means every email fails closed.
+          NOEMA_CORRESPONDING_AUTHOR_EMAILS: ${{ vars.NOEMA_CORRESPONDING_AUTHOR_EMAILS || '' }}
         run: |
           set -euo pipefail
           if [ -z "${PR_NUMBER:-}" ]; then

CHANGELOG.md

@@ -1,3 +1,24 @@
+### Noema reviews binary-only document PRs through a base/head object diff
+
+- A DOCX/HWPX/PDF/image change reaches Noema as `Binary files … differ` with no hunk. `changed_diff_locations()` returned an empty set, so `validate_substantive_verdict()` raised "requires parseable changed-line evidence" for every approve or request_changes, and a binary-only document PR could never get a formal verdict. That empty textual patch is an unsupported textual diff, not "no change". Separately, `fetch_file_content_at_ref()` decoded PDF and image bytes with `errors="replace"` straight into the prompt.
+- New `scripts/ci/document_blob_diff.py` (stdlib only) materializes the base blob at the merge base and the head blob (contents API for the blob SHA, then the Git blobs API for the bytes) and extracts ordered objects. DOCX yields body paragraphs, tables and relationship-resolved figures; HWPX yields manifest/section paragraphs, nested tables and `binaryItemIDRef` figures; PDF and images are one opaque hashed `page` object. Every object carries a sha256. The objects are aligned into added, removed, modified and neighbouring unchanged objects, and a byte change with no object change becomes a package-level `style` object, never "no change". The result is emitted as `document_diff_review.v1` (contextual-orchestrator#1220 contract; all five fixture envelopes pass that PR's validator).
+- Fail-closed safety: blob size, member count, total expansion, per-member compression ratio, traversal, encrypted members, macros (`vbaProject.bin`, `Scripts/`, `macroEnabled`), external image/OLE/template/frame relationships, external HWPX manifest hrefs, DTD/entity XML, unresolved figures, participant-material directories, and email, resident-registration-number, mobile-number and secret patterns in extracted text all reject the review instead of redacting. Original bytes and media never leave the runner; figures and pages carry only hashes.
+- `noema_review_gate.augment_binary_document_diff()`, called from both the direct and the two-phase paths, replaces each binary document stanza with synthetic hunks whose LEFT/RIGHT line numbers are base/head object ordinals. The existing citation validator therefore accepts findings on document objects unchanged. Opaque binaries are no longer decoded into the changed-file context.
+- Review fixes on the same PR (contextual-orchestrator lead findings, reproduced with that lead's fixtures):
+  - Object text is now cut on UTF-8 byte boundaries (8 KiB per text) and the envelope total is counted in bytes (256 KiB). This matches contextual-orchestrator#1220, which had rejected a 5,000-character Korean paragraph (15,000 B) with 400 and a >256 KiB Korean envelope with 413. Changed objects claim the byte budget first, and unchanged context is dropped once bytes or object slots run out. XML-legal control characters (CR) are blanked.
+  - The participant/secret scan now covers every extracted base and head object, not only the diffed ones. The changed-file context path applies the same scan to the whole `extract_review_document` body of a DOCX/HWP/HWPX and withholds it on a match, so a phone number in an unchanged paragraph far from the edit no longer reaches the prompt.
+- Second review round:
+  - Fail-open fix: c6f4b49b kept changed objects with `""` text once the 256 KiB budget ran out, so a late substantive change ("결론: 효과 없음") never reached the reviewer. Now if any changed object's text would be cut by the 8 KiB per-object bound or the envelope budget, the envelope fails closed and no provider is called. Only unchanged context is cut, ending with `…[truncated]`, or dropped. A regression test checks that no changed object ever carries empty or truncated text.
+  - Figures carry the text of an adjacent "Figure N"/"Fig."/"그림 N" caption, so contextual-orchestrator's rule finding (changed image, unchanged caption) fires.
+  - HWPX sections follow the manifest spine; a spine section missing from the package fails closed.
+  - Emails still fail closed by default, with one narrow exception for a declared corresponding author. An address is replaced by `[CORRESPONDING_AUTHOR_EMAIL]` before hashing, diffing, context or prompt only when it is on the workflow's exact `NOEMA_CORRESPONDING_AUTHOR_EMAILS` allowlist and sits in a front-matter author-contact paragraph (before the abstract or introduction, within the first 20 paragraphs, declaring "Corresponding author"/"Correspondence"/"교신저자"). Allowlist mismatches, the same address elsewhere, emails after the abstract, undeclared contacts, extra addresses, table cells and participant phone numbers are all still rejected, and no error message contains the address.
+- Tests: `tests/test_document_blob_diff.py` (58). The budget fail-open, empty-changed-text invariant, caption, spine and corresponding-author cases fail on c6f4b49b.
+- Third round (exact-head review of e6f0671f and CodeRabbit):
+  - A changed figure (captioned or not), a PDF/image `page`, or a textless package object proves bytes changed but not what changed. `document_blob_diff.unobserved_changed_objects()` derives their `path#locator` ids from the envelope itself, and `augment_binary_document_diff` returns them as structured review metadata. `call_llm` lists them in the prompt and passes them to `validate_substantive_verdict`, which refuses any `approve` while one is present (`NoemaModelOutputError`); comment and request_changes stay available. The flag no longer comes from scanning rendered or truncated diff text, so neither a caption (8da729e4 bypass 1) nor a `MAX_DIFF_CHARS` cut (bypass 2) can hide an unobserved change.
+  - An added or removed package with no extractable body still gets a citable package object (`added`/`removed`).
+  - When `fetch_diff` already cut the diff, `augment_binary_document_diff` re-reads the full diff before searching for binary stanzas, so a document after the cut is no longer skipped.
+- Tests: 66. The captioned and uncaptioned image swap, diff-cut, PDF-only and prompt/leak cases fail on 8da729e4. A text-only DOCX edit is the negative control: no unobserved ids, and a valid approve passes. The binary-only request_changes integration test, the fail-closed augmentation test and the no-raw-decode test fail on the previous gate and pass now. The new module has 100% branch coverage.
+
 ### Noema transport capacity schedules a bounded continuation re-dispatch
 
 - After gateway failover, HTTP 429/5xx no longer end only as a permanent required-check failure with `caller attempts=1`. ADR-0031 classifies that class as `provider_capacity_unavailable`, keeps the single gateway request per job, surfaces `provider_attempt_count` from the orchestrator error envelope, and authorizes at most two same-head `repository_dispatch` retries after a capped `Retry-After` or deterministic 60–180 s jitter. Review is never skipped. Refs #2165.