Skip to content

feat(gate): add tamper-evident finding baseline ratchet - #63

Open
DivyamTalwar wants to merge 2 commits into
mainfrom
codex/finding-baseline-ratchet-20260824
Open

feat(gate): add tamper-evident finding baseline ratchet#63
DivyamTalwar wants to merge 2 commits into
mainfrom
codex/finding-baseline-ratchet-20260824

Conversation

@DivyamTalwar

Copy link
Copy Markdown
Owner

Summary

  • add structorium baseline capture for deterministic, version-controlled snapshots of active findings
  • add structorium baseline check with a distinct regression exit code and optional JSON evidence
  • report new, resolved, and unchanged finding identities
  • reject hand-edited or malformed baselines with a SHA-256 checksum
  • refuse implicit baseline replacement; require explicit capture --force
  • fix a pre-existing filesystem-order-dependent prompt test exposed by full CI

Why this is high leverage

Structorium's changed-line gate is ideal for source-local defects, but architecture findings and detector identities can move outside a diff's exact line range. A version-controlled finding ratchet lets mature repositories adopt enforcement immediately while making removed debt visible and preventing new debt.

Competitive research and provenance

This is original Structorium code informed by public patterns, not copied source:

Structorium's version is finding-ID-native, checksum protected, dependency free, and reports resolved debt alongside regressions.

Verification

  • focused baseline and CLI suite: 7 passed
  • exact deterministic prompt regression: 1 passed
  • compileall on new runtime modules: passed
  • Ruff on every changed Python file: passed
  • git diff --check: passed

CI disclosure

The deterministic prompt-test commit is intentionally included because the first next-wave CI run showed the same pre-existing flaky assertion in both core and full suites. It sorts generated prompts and checks the complete batch set; it does not weaken production behavior.

The repository's Security dependency-review job may remain blocked until the repository dependency graph is enabled. Secret scan, pip-audit, and CodeQL are the code-authoritative security checks and will be reported separately.

No existing PR or branch was changed. This PR intentionally remains open for review.

Copy link
Copy Markdown
Owner Author

Implementation evidence

Contract Evidence
Incremental adoption capture records only active, unsuppressed finding identities
One-way ratchet check separates new, resolved, and unchanged entries
Fail-closed integrity schema, unique/sorted fingerprints, count, and SHA-256 checksum are validated before comparison
Explicit governance existing baseline cannot be replaced without --force
CI semantics zero new findings allowed by default; --max-new is explicit; exit code 3 distinguishes a quality regression
Stable identity fingerprint excludes mutable tier/confidence, so reclassification does not masquerade as a new defect
Local evidence 7 focused tests plus the exact prior CI regression test passed; Ruff/compileall/diff hygiene passed

Competitive pattern sources:

The implementation is dependency-free and does not copy competitor code.

Latest reviewed head: 31d68a0f99f842813689dd8d1397c435ba5d156b.

This PR is intentionally open and unmerged. GitHub CI/Security on this exact head is the authoritative next check; dependency-review requires the repository dependency graph setting and is tracked as external configuration.

Copy link
Copy Markdown
Owner Author

Final authoritative verification

Latest head: 31d68a0f99f842813689dd8d1397c435ba5d156b

  • CI run 32663715050: success (all CI jobs green)
  • Security run 32663715073:
    • CodeQL job 97253689472: success
    • pip-audit job 97253689467: success
    • secret-scan job 97253689508: success
    • dependency-review job 97253689298: externally blocked because the repository dependency graph is disabled

The Security workflow's aggregate failure is therefore repository configuration, not a source, dependency, secret, or CodeQL failure. The implementation-authoritative CI and security analyzers are green on the exact latest head.

PR state: open, unmerged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant