[chaksaray]

Closes #120

What and why

The README carried about 25 hand-typed counts, and the corpus grew from 80 to 87 records in a week. scripts/sync_record_counts.py now measures them and writes them in place, so a count can no longer be typed wrong.

• Every count sits in an inline marker: <!-- count:published -->87<!-- /count -->. HTML comments are invisible when rendered and work inside table cells and sentences, so the prose is unchanged. The README now has 23 markers: record totals and id range, detected and uncovered counts, runtime stages, schema and AIVSS versions, rules per engine, chains, and the badge
• The corpus is pinned by commit: <!-- counts:corpus ref=<40 hex> -->. --check reads the corpus at that commit, so it fails when a rule file or the README changes, not whenever AVE publishes a record. --update-ref moves the pin to the head of AVE main and rewrites
• Rule files are parsed, never imported, and the script is standard library only (the Stage 1 test job has no PyYAML). PATTERN_RULES is followed through its V1 and V2 lists, and a layout the script does not recognize exits 2 instead of miscounting
• --corpus <path|https URL> reads a local snapshot with no network. BAWBEL_AVE_CORPUS_URL sets the same thing, the name bawbel-mcp already uses. Exit 0 current, 1 stale (--check), 2 unreadable, and nothing is written on 2
• A Record Counts CI job runs --check. Actions pinned by SHA, no installs
• Inside the README's code diagrams the counts are dropped, because a comment in a code fence prints literally. The tables carry the numbers

Found by dry-running before editing the README: PATTERN_RULES holds 15 rules and PATTERN_RULES_V2 another 25, joined at pattern_engine.py:1065. Reading the first list gave 15 and a detection count of 40. The resolver follows list assignments and a + b in source order, and a test compares it with the module's runtime list.

Output contract impact

Fields added: none
Fields renamed: none
Fields removed: none

README, a script, a test, and a CI job. No scanner code changed.

Checks

• Golden fixture unchanged: findings 7 max CRITICAL risk 9.8
• Tests pass: 707 passed in 316.32s (71 are new)
• No em dashes added
• pre-commit run --all-files clean: exit=0, 14 hooks passed (black, flake8, bandit, the self-scan, and the full test suite among them). The first run caught three flake8 findings in the script, fixed here

$ python scripts/sync_record_counts.py --check
corpus: 87 records, sha256 7e983eb9badce598, https://raw.githubusercontent.com/aveproject/ave/6b9f1748c6ceedc1787af7a114361a216d3017d1/dist/ave-records-latest.json
README counts match the corpus and the rule files
$ python scripts/sync_record_counts.py --check --corpus https://raw.githubusercontent.com/aveproject/ave/main/dist/ave-records-latest.json
README counts match the corpus and the rule files

The README's hand-typed numbers were all correct against the corpus at the pinned commit, so this PR changes no number.

Measured against the independent figures from the claims register's verifier (bawbel/.github, run 2026-10-20): 87 published, 43 detected (42 static plus Magika), 8 metadata-only, 44 uncovered (27 static, 17 runtime), 40 + 39 + 41 rules, 12 chains. They agree.

Can the check fail? Mutation runs:

README: two stale numbers          -> exit 1, names README line 69 and 258
README: marker with no measurement -> exit 2
README: pin marker removed         -> exit 2
README: pin to a commit with no corpus -> exit 2 (HTTP 404)
local corpus file that is not JSON -> exit 2
script: skip the V2 list           -> 2 tests fail
script: do not filter by published ids -> 1 fails
script: never merge id ranges      -> 3 fail
script: write mode does not write  -> 2 fail
script: --check returns 0 when stale -> 1 fails

The tests compare the parsing with the scanner's own objects (PATTERN_RULES, FLOW_DEFINITIONS, AVE_META), and with real yara and yaml parses when those are installed (importorskip, as the engine tests do).

Not in this PR

• scanner/ave_meta.py: 51 hand-typed entries, synced from the retired host on 2026-06-21. The README counts do not depend on it beyond the "metadata only" figure
• A scheduled workflow that moves the pin. --update-ref does it by hand for now
• Counts in docs/ and the other markdown files: none were found

Merging with #117 and #119

The README merges cleanly with both. CHANGELOG.md conflicts with each, because all three add an entry under [Unreleased]. I will rebase the later ones after the first merges.

Found, not changed

• scripts/sync_records.py still syncs from the retired bawbel/ave repo into PiranhaDB (PIR-2)
• pyproject.toml lists "AVE Standard" = "https://github.com/bawbel/ave", and CONTRIBUTING.md lines 107 and 108 name the retired repository URLs as checklist text. Both belong to the scanner ECO-3 pass, after fix: AVE record URLs from one config setting; add ave_url #117

🤖 Generated with Claude Code

https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P