Skip to content

Freeze portable findings before exposing pluggable publishing - #66

Open
jrh-mann wants to merge 5 commits into
mainfrom
codex/findings-handoff
Open

jrh-mann wants to merge 5 commits into
mainfrom
codex/findings-handoff

Conversation

@jrh-mann

@jrh-mann jrh-mann commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Findings handoff

One Inspect agent investigates, submits a portable immutable findings bundle, then continues publishing in the same conversation. inspect_audit/publish can start a fresh publishing agent from that bundle. Publisher plugins remain replaceable.

  • Bash/Python edits /workspace/findings.json; collected auditor records and subject remain host-authoritative.
  • Findings, reviews and ratings include explanations and evidence. Submission checks sample references, original auditor records and the selected publisher's requirements before freezing.
  • Publishing skills/tools are withheld until submission. Job tools disappear afterwards, and host launch code refuses new experiments.
  • The bundle preserves selected evidence and a checksum manifest. Native logs remain external references.
  • Checkpoints preserve the submitted claims, manifest and phase. Progress and publication share one interpretation of missing/unresolved sample/check results.

Default publisher: Laurence's GL LaTeX audit report

latex is now the default; publisher=gl retains the optional web publisher. The publisher downloads the actual template from Generality-Labs/audit_reports at ad7de89658b6794112c2f65847991c7ed41d9e30, verifying every file against publishers/template.json. The class, style, logo, scoring criteria, build configuration and Example structure are fetched into a host cache. No TeX template copy is vendored. Investigation uses generated analytical registry data from that revision, including the correct A.0 contribution numbering.

The output follows the upstream report: cover sheet, contents, A-D eval scorecard with nine dimensions and key contributions, datasheet, audit setup, dimension findings and scoring criteria.

  • Grade is frozen in investigator.ratings.overall; assessments use framework criterion IDs.
  • Findings carry their criterion, e.g. C or C.1. Contribution findings render as A.n boxes, and broader dimension findings as sequential F-n boxes.
  • Code generates rows, assessments, finding boxes, counts and evidence references from the bundle. The publishing agent supplies only escaped prose in prose.json.
  • latexmk runs on the publishing host or in the separate publishing image with networking disabled, never in the investigation sandbox.
  • Output preserves the PDF, generated source, prose, template provenance and original findings bundle.

The GL web plugin still supports predefined bars, stacked bars and histograms and emits HTML/PDF/SVG with data provenance.

Cleanup

The superseded LaTeX helpers and their separate findings schema are removed, along with review/finding/rate recording tools, assessments.json, register.json mirroring, old report sync, old publishing skills and templates. This is a publisher over the shared frozen bundle, not restoration of the old reporting pipeline. TeX is confined to publishing dependencies.

Validation and CI

Before the request to stop local heavy work:

  • 251 non-Docker tests passed, with the TeX integration case skipped in that run.
  • The real fixture compiled into an 11-page PDF in the separate publishing image. All pages were visually reviewed. Its scorecard, nine rows, assessments, both finding-box formats, sample counts and evidence were checked against the frozen fixture.
  • The PDF test's expected F-number was corrected to F-1 to match template dimension order; all assertions were checked against the existing PDF without recompiling.
  • Type checking and formatting hooks passed.

No more local heavy tests/builds. Heavy verification is delegated to CI. A dedicated LaTeX fixture job uses the audit-template environment and its read-only AUDIT_REPORTS_TOKEN secret. The dedicated job now fails immediately if that read-only secret is missing; it must not pass with its private-template tests skipped. No paid model calls were made.

README is untouched. Hawk 4 deployment remains unvalidated and tracked in docs/hawk-4-migration.md. See docs/findings.md for contracts, cache/auth setup and publisher commands.

Python line budget

Compared with main: +673 package Python lines and +78 test Python lines (+751 total) for the shared handoff and publisher functionality after the removals above. #67 reduces production Python by 5 lines and adds 33 lines of authorizer tests; #68 changes no Python.

Related PRs and follow-up

Registry and private-cache follow-ups (fa0ea20)

  • test_registry_is_regenerated_from_the_pinned_template compares the complete generated JSON with regeneration from verified upstream sources, including all 56 dimension/contribution definitions, all four grades, all six assessment descriptions and provenance. A stdlib-only check against the existing verified cache passed locally; no local heavy tests were run.
  • python -m inspect_audit.publishers._template --update-registry is the reproducible update command when the pin changes.
  • A named-context Dockerfile can layer the verified private template cache into the Python publishing runner image without putting a GitHub token in the image or pod. The compiler image alone does not provide the Python host's cache. The derived image must remain private.
  • CI includes an offline cache-image test with networking disabled and no runtime GitHub credential, alongside registry regeneration and PDF compilation. These heavy checks are CI-only; default pytest excludes both Docker and LaTeX markers.
  • The audit-template environment currently has no AUDIT_REPORTS_TOKEN. A fine-grained read-only token for audit_reports must be supplied by the operator; the personal gh-login token was not copied into CI. Missing access is now an explicit failing check.
  • Coordination: this branch does not move Claude's dispatch, log-resolution, pricing or launcher modules. The optional GL web publisher stays as previously requested. Role renaming remains a separate follow-up after Freeze portable findings before exposing pluggable publishing #66 merges, before real findings bundles are saved.

@jrh-mann
jrh-mann deployed to audit-template October 22, 2026 23:31 — with GitHub Actions Active

This branch had an error being deployed

1 failed deployment
audit-template — fa0ea207 Deployed Oct 22, 2026 by jrh-mann via LaTeX fixture #134
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants