Skip to content

feat(product): package source-scoped advisory PR summary for reuse - #228

Draft
fraware wants to merge 104 commits into
mainfrom
product/portable-advisory-pr-report-v1
Draft

fraware wants to merge 104 commits into
mainfrom
product/portable-advisory-pr-report-v1

Conversation

@fraware

@fraware fraware commented Oct 16, 2026

Copy link
Copy Markdown
Owner

Purpose — verified-scope evidence travels with the reviewer

Draft cumulative development PR based on #227 at fully qualified source 5a17d08884bfd8fffd0b3f18db2b82bf64d63391. The existing PRs #225/#226/#227 remain draft, unchanged, and unmerged. This PR targets main for the complete repository CI and reproducibility suites; do not merge or publish before the independent gates.

Directly observed adoption gap

The real successful Action dogfood in run 38840137045 produced a source-scoped, non-authoritative job summary, but the downloadable ovk-evidence-artifacts-action_default_diff_dogfood artifact 12021192187 contained only the earlier ovk-pr-comment.md as a human-readable report. That earlier Markdown names a source commit and says Merge recommendation: allow without prominently carrying the new unsiged, unsupported-scope and no-merge-authorization limits. The accurately scoped job summary was absent from the artifact archive.

This creates a concrete distribution/reuse failure: another engineer downloading or forwarding the artifact receives a less carefully bounded headline than the original job-summary reader.

Bounded implementation

  • scripts/render_source_bound_job_summary.py now accepts an optional --portable-output path and writes the exact same previously validated text to a plain Markdown file, while continuing to append to the native GITHUB_STEP_SUMMARY without overwriting existing job summaries.
  • The Action writes ovk-advisory-pr-summary.md only on its already-qualified internally prepared advisory PR path and includes this report in the existing artifact upload. No write-scoped GitHub permission, extra hosted service, source excerpt, user-supplied PR diff, or new trust claim is introduced.
  • The Action clears stale ovk-advisory-pr-summary.md before installation (the upload step runs if: always()). The renderer rejects output/source aliases and symlinks, and deletes a previous portable report before source validation so a failed run cannot accidentally reupload a stale report.
  • Tests prove equality between reusable report and appended job summary, refusal of stale altered diffs, refusal to overwrite evidence/receipt/source/job-summary inputs, and conditional Action generation/cleanup/upload.
  • docs/SOURCE_SCOPED_JOB_SUMMARY.md explains how a teammate examines the portable report and why it does not convey signed or transportable certification.

The original ovk-pr-comment.md remains an older, generic artifact. Consumers must not interpret either Markdown's allow text as an automatic merge authorization. No source-to-IR soundness, historical-evidence applicability, event/runner authenticity, third-party adoption or production strict guarantee follows from this feature.

Required gate

Exact candidate CI (including the real Action dogfood), six-platform reproducibility, and native backend matrices where triggered. Inspect the uploaded real job artifact on the exact final SHA for both the new portable Markdown and the correct merge-preview/subject identities. No green conclusion can be inherited from #227; it qualifies the source, not the edited candidate.

Independent security review, repository branch/tag protection #25, source-to-model research #195, and consenting natural-PR pilot #20 remain separate. No public installation or strict enforcement is authorized.

fraware commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

FINAL-HEAD ENGINEERING QUALIFICATION — 2026-10-15 (development only)

Frozen exact SHA: 837c931d059f74dcda06432d83f5bde759f53d0a. PR #228 is open, draft, unmerged. This is a cumulative integration candidate; earlier #225/#226/#227 remain separate, open, unmerged drafts. No public release or source/certificate transport authority follows.

Four full workflow suites completed SUCCESS at this exact SHA

Earlier qualifying heads do not substitute for this exact-head result.

Independent artifact inspection

Downloaded real CI dogfood artifact ovk-evidence-artifacts-action_default_diff_dogfood (id 12026102979) from an intermediate candidate and compared ovk-advisory-pr-summary.md, ovk-source-binding.json, and the exact bytes of ovk-evidence.json:

  • source/evidence subject agrees on the observed synthetic merge preview 9a0c65a0a6beae0a2d61c168813c0d0585a51ef3; original head fe49235b669e7c18b8fa79268e2c9a1d3dc3962f is distinct.
  • receipt tree 451b18b11a9c0aff31c036697b2d4148d5903405 matches the report.
  • uploaded evidence's SHA-256 7777c665216095dcee607426acf2f05b4cab907b6e5a56d7dcd7810e8f5871fc matches the report.
  • the 1,367-byte standalone Markdown includes the full observed IDs, advisory/reliance caveats, and An allow is not permission to merge disclaimer.
  • artifact contained no raw .ovk-pr.diff in that run. The final candidate explicitly removed the diff from the upload path; tests enforce this and the final exact-head Action dogfood passed. Do not infer content authenticity from matching self-generated digests.

Required independent reviewer checklist (before any merge or rollout)

  1. Event/subject/attestation boundary: trace the actual pull_request merge-preview checkout through prepare_verified_change_input.py, ovk check --source-binding, build_repository_context, EvidenceBundle subject, rendered report and optional GitHub Check attribution. Challenge base drift, stale checkout, maliciously edited receipt, mixed event IDs, caller diff, missing PR metadata, fork permissions, and plain push. Source-to-IR semantic completeness still unproved.
  2. No unbound publication: inspect both manifest and changed-files modes from security(action): prevent unbound manifest results becoming PR-wide checks #226. Neither may emit a public Check/comment, claim downstream Action-level allow, or reuse stale source artifacts. No strict mode on unbound manifests.
  3. Report as portable observation: --portable-output must not alias an input, job summary, symlink, or output directory; missing or mismatched exact diff bytes must prevent new/stale shared Markdown. Re-check source/evidence digests and tree IDs on a newly generated final-head artifact. No raw PR diffs or sensitive source text uploaded by the Action.
  4. Installer and release: confirm pinned runtime dependency provenance and fail-closed package install, then have an authorized admin protect main and v* tags under Release blocker: protect main and v* release tags before v1.3.0-rc.1 #25. Use a new, independently authorized exact-SHA release process, never reattribute unsigned historical v1.2.1.
  5. Empirical safety and adoption: independently adjudicate at least one natural external cohort with explicit unsupported/unknown denominator; no source-model or historical transport VALID from the present development suite. Issues research: bounded Lean semantic-support soundness and historical-certificate oracle #195, research: independently adjudicated RTK transport experiment — execution gate #177 and Run an external advisory-mode pilot #20 remain open.

GitHub PR review API currently shows zero submitted review decisions or inline review threads across #225–#228. This comment is an engineering handoff, not an independent security approval. Decision: automated qualification passed; independent review/release governance/external calibration NO-GO.

fraware commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

Final-head artifact-byte read-back (supplements qualification note above)

From exact-head CI 38853899851, I downloaded the completed action_default_diff_dogfood artifact 12026276346 and inspected the ZIP members with Python (zipfile and hashlib), independently of the checker's asserted success:

  • Archive has 9 entries: ovk-evidence.json, ovk-pr-comment.md, ovk-advisory-pr-summary.md, ovk-attestation.json, ovk-artifact-manifest.json, ovk-attestation-envelope.json, ovk-evidence-quality.json, ovk-branch-metadata.json, ovk-source-binding.json. No raw .ovk-pr.diff is present.
  • Receipt merge-preview evaluated_checkout_commit_sha=c1660d7f5467991266a2b56eef84051b6a5f680c equals receipt evidence-subject SHA equals ovk-evidence.json subject.head_sha.
  • The original contributor head is distinct: 837c931d059f74dcda06432d83f5bde759f53d0a. Report contains both identities and the checked-out tree SHA. The 64-hex diff digest is present.
  • Computed SHA-256 of exact uploaded ovk-evidence.json bytes matches the Local evidence file SHA-256 printed in the portable Markdown.
  • The document explicitly says An allow is not permission to merge. Source receipt flags remain independent_event_authentication=false, bound_into_signed_evidence_envelope=false, production_reliance_authorized=false.

This is a real GitHub Action execution plus independent byte comparison of its output artifacts, not an independent proof of GitHub event authenticity, sound source extraction or applicability of checks across repositories. Candidate remains draft, unmerged and unreleased. The uploaded legacy ovk-pr-comment.md still has a generic Merge recommendation: allow heading, so downstream reviewers must use the correctly scoped ovk-advisory-pr-summary.md as the primary short human report and inspect full evidence before any relying decision.

fraware commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

Independent reviewer entry point — 17 Oct 2026

Exact candidate: 837c931d059f74dcda06432d83f5bde759f53d0a. All four exact-head workflow suites passed; there is no submitted independent review. Open launch review protocol #230 for reviewer A (source/event/PR identity) and reviewer C (release provenance, licensing, security supply chain).

This cumulative PR changes 49 files across 104 commits against main. Treat all tests as development evidence. Review multi-PR ancestry carefully: #225 and #227 are ancestors, while #226 has one unshared history commit but matching shared contract-test bytes. Verify no unique fix was dropped.

High-priority adversarial review: ordinary PR merge-preview source versus actual EvidenceBundle subject; caller-supplied diff/manifest cannot publish Check/comment or downstream Action-level allow; stale artifacts never appear after failed installer; output Markdown does not represent unbound source as authorized. Confirm underlying source-to-IR coverage and reviewer assumptions separately. Source coverage for partial workflow YAML remains a pilot gate in issue #20, not a proven general source guarantee.

A qualified reviewer must attach actual GitHub identity, independence, precise tested SHA, reproducible negative cases, inspected artifacts, accepted limitations, and a formal review decision. No automatic merge/publication or PR draft promotion until effective branch/tag protections under #25 and independent sign-off.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant