Skip to content

docs: state what audit verify shows and how to anchor a chain head - #85

Open
chaksaray wants to merge 2 commits into
developfrom
docs/84-third-party-audit-verify
Open

chaksaray wants to merge 2 commits into
developfrom
docs/84-third-party-audit-verify

Conversation

@chaksaray

Copy link
Copy Markdown
Member

Closes #84

What and why

GATE-12b. bawbel-gate audit verify can be run by someone who does not trust the operator, and nothing said so or said where it stops. This PR states the property, pins it with tests, and documents how to record a chain head outside the log. It changes no code under src/.

  • tests/test_audit_third_party_verify.py (12 tests, own commit): runs audit verify as a separate process in an empty working directory with an empty home, so it cannot use a gate config, a manifest, or state from the writing host. It also asserts verify does not modify the file or write anything.
  • README.md: a "Verifying a log you did not write" section under "7. Audit and operate" (property paragraph, how to read the head, two ways to record it, how to check later), and a pointer from the "Audit log (M2)" paragraph.
  • SECURITY.md: a "Limits of the audit chain" section.

Governing spec

DESIGN.md section(s): 8.6 (chain head can be anchored externally; audit verify replays the chain). Not changed. docs/adr/0001-audit-chain-construction.md already says the same about anchoring; the new text matches it.

Output contract

  • Changes: README, SECURITY.md, one new test file.
  • Does not change: any file under src/, DESIGN.md, the CLI, the audit record format.
  • New or changed CLI flags, manifest fields, reason strings, audit record fields: none.

Verification (run this session on the branch)

pytest                          -> 358 passed in 12.05s
pytest tests/property/ -x       -> 26 passed in 1.57s
pytest tests/replay/            -> 32 passed in 0.32s
pytest tests/test_m4_corpus.py  -> 10 passed in 0.17s
pre-commit run --all-files      -> exit 0 (ruff, bandit, whitespace, eof, json, yaml, merge-conflict all Passed)

Runbook check, on the published package: a chain written by the development tree, copied alone into an empty directory, verified by bawbel-gate==0.1.0 from PyPI in a separate virtualenv with env -i and an empty HOME:

ok: 5 records, chain intact
chain verified: chain.jsonl
exit=0        (home directory left empty)

What verify does and does not detect, from the tests (each is one test):

edited field                      exit 1  seq 3: hash mismatch
deleted record                    exit 1  seq 4: prev mismatch
reordered records                 exit 1  prev mismatch
inserted record                   exit 1  prev mismatch
record cut off mid-line           exit 1  line 5
final records removed             exit 0  ok: 3 records, chain intact      (not detected)
log rewritten, hashes recomputed  exit 0                                    (not detected)

Both undetected cases change the head. The tests run the README's own commands (audit tail -n 1 for the head, grep -Fc for the comparison) and show the head differs and the anchored hash is absent in each.

Mutation check on a copy of the tree: skipping the hash comparison, skipping the prev comparison, making verify exit 0 on failure, making it read a config from the working directory, read ~/.bawbel, or write a file to the working directory, and making it detect tail truncation each fail at least one of the new tests.

Invariant ordering

This PR touches tests/ and docs only, not audit/. The test commit comes first:

Property tests: commit 6f02776 (tests only, no implementation change exists)
Implementation: none

Invariants redefined: none

Differences from the runbook

  • The runbook's SECURITY.md sentence says the chain proves records were not altered "by a party without write access to the log". That is not what the code does. Someone with write access can rebuild every hash and verify passes (the tests do exactly that), and removing final records also passes. A party without write access cannot alter the file at all, so that sentence would claim nothing. The new section states the tested behavior instead, and matches ADR 0001: "without an anchored head, whoever can rewrite the log can rebuild the chain".
  • The existing in-scope bullet in SECURITY.md reads "Any way to falsify, reorder, or gap the audit chain without detection by bawbel-gate audit verify". The two limits above are exactly that, so I added "other than the limits listed under 'Limits of the audit chain' below" to the bullet. That edit narrows what the policy invites reports about, so I am flagging it for you to confirm (see Decisions).
  • audit verify has no --expect-head flag, so the comparison is a grep -Fc on the file. A flag would be a code change under audit/ and needs its own DESIGN.md PR; not done here.

Decisions for the maintainer

  1. Is the narrowed in-scope bullet right? The alternative is to leave the bullet as it was and treat the two limits as reportable.
  2. SECURITY.md still says "an audit-chain forgery is treated as Critical regardless of how narrow the reproduction steps are". With the limits now documented, decide whether a rewrite by someone with write access counts as a forgery under that sentence. I did not touch it.

Not verified

  • The git note and daily-commit anchoring commands were run only up to the local commit (git notes add, git add/git commit in a throwaway repository). The git push steps were not run. Whether a given remote lets the gate host rewrite the note ref or branch is the operator's setting.
  • grep is required for the comparison command and for the last three tests; they are skipped where it is missing.
  • No CI workflow runs the test suite in this repo (the PR template says so), so the results above are local.

Checklist

  • Every commit carries a DCO sign-off (git commit -s)
  • DESIGN.md is still accurate after this change (not modified)
  • Replay test in tests/replay/ if this blocks an attack class (not applicable)
  • Docs updated for user-facing changes (CLI, reason strings, manifest fields)
  • No em dashes in docs, comments, or error strings

🤖 Generated with Claude Code

https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P

chaksaray and others added 2 commits October 23, 2026 05:49
Runs `bawbel-gate audit verify` as a separate process in an empty directory
with an empty home, so it cannot lean on a gate config, a manifest, or state
from the host that wrote the chain. It must not modify the file or write
anything.

Detected from the file alone: an edited field, a deleted record, reordered
records, an inserted record, a record cut off mid-line.

Not detected, and pinned as such: removal of the final records, and a log
rewritten with every hash recomputed. Both are exposed by comparing the head
hash against one recorded elsewhere, which the last two tests check with the
`audit tail -n 1` command the README documents. If a "not detected" test
starts failing, the verifier got stronger and the docs need updating.

Refs #84

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P
Signed-off-by: chaksaray <chaksaray@gmail.com>
README: a "Verifying a log you did not write" section. audit verify needs the
chain file alone. It detects an edited, deleted, inserted or reordered record
and a cut-off line, and does not detect removed final records or a log
rewritten with every hash recomputed. Both change the head, so the section
shows how to read the head, two ways to record it outside the log (a git note,
a daily commit to a separate repository), and how to compare later.

SECURITY.md: a "Limits of the audit chain" section, and the in-scope bullet on
undetected falsification now points to it.

Every command in the section is run by tests/test_audit_third_party_verify.py.

Closes #84

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P
Signed-off-by: chaksaray <chaksaray@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant