Skip to content

fix: stat block from the claims register, SCAN-3 comparison table - #23

Open
chaksaray wants to merge 2 commits into
developfrom
fix/22-numbers-and-compare
Open

chaksaray wants to merge 2 commits into
developfrom
fix/22-numbers-and-compare

Conversation

@chaksaray

Copy link
Copy Markdown
Member

Refs #22 (first of the ECO-2 pull requests)

What and why

The stat block said 51 AVE records and 121 detection rules, typed by hand. Every count on the site now comes from one place: the expect values in claims/register.yaml in bawbel/.github, which that repo's verifier checks against the code.

  • scripts/sync_site_numbers.py writes the stat block between <!-- stat:start --> and <!-- stat:end --> in index.html. --check exits 1 when the block is stale or a count is typed anywhere else on index.html, docs/index.html, or README.md, including a shields badge. <pre> samples are not checked. The register comes from --register (a path or an https URL), SITE_CLAIMS_REGISTER, or the raw GitHub URL. An unreadable register, a missing or non-integer value, or missing markers exit 2 and write nothing
  • .github/workflows/site-numbers.yml: on pull requests it runs the tests and --check. Weekly and on demand it regenerates the block on develop and opens a pull request when the register changed. Actions are pinned by SHA
  • Stat block: 87 AVE records published (C-01), 43 with a detection engine (C-03), 12 attack chains (C-06), 6 detection engines (C-08), 120 detection rules (C-07). "0 Auth Required" is dropped (it described PiranhaDB)
  • Counts removed from prose, the meta description, the ecosystem cards, and the README badge. "owasp_mcp is required on every AVE record" stays: AVE schema 1.1.0 lists owasp_mcp in required with minItems: 1, and 87 of 87 records in the published corpus carry it
  • Comparison table rendered from the scanner README "Why Bawbel" table (SCAN-3): NVIDIA SkillSpector added, ? for unverifiable cells, and the LLM-telemetry row where SkillSpector leads. Counts in two cells dropped ("87 records", "12 AVE capability chains"). ClawGuard is gone because the source table has no ClawGuard column
  • PiranhaDB card: "LIVE api.piranha.bawbel.io" and "14 endpoints" become "OFFLINE"
  • Pre-commit card: "Local mode for air-gapped environments" removed. bawbel/integrations .pre-commit-hooks.yaml defines bawbel-scan and bawbel-scan-all, both language: python installs from PyPI. The scanner's air-gapped local mode is the Docker sandbox image build

Decisions applied

D4: "87 published, 43 detected". D1 (what leads) and the gate section come in the next pull request. D5 is superseded: the CVSS-AI page was deleted in #20.

Not in this pull request

  • The title stays "Bawbel: Agentic AI Security Scanner". It already leads with the scanner (D1). The gate pull request revisits it
  • ave.bawbel.io and api.piranha.bawbel.io links: ECO-3, where D3 governs /records
  • The page and badge links to bawbel/ave (redirects to aveproject/ave): ECO-3

Verify

$ python scripts/sync_site_numbers.py
index.html: stat block updated: C-01 87, C-03 43, C-06 12, C-08 6, C-07 120
$ python scripts/sync_site_numbers.py            (again)
index.html: stat block already current: C-01 87, C-03 43, C-06 12, C-08 6, C-07 120
$ python scripts/sync_site_numbers.py --check
site numbers match the claims register: C-01 87, C-03 43, C-06 12, C-08 6, C-07 120   (exit 0)
$ python -m pytest tests -q
16 passed in 0.13s
$ ruff check scripts tests
All checks passed!
$ check-jsonschema --builtin-schema vendor.github-workflows .github/workflows/site-numbers.yml
ok -- validation done

The same typed_counts run against develop's three pages flags 20 counts, among them README.md:14: 'records-51' (the badge) and docs/index.html:791 ("all 51 records"). On this branch it flags none.

The values match a fresh measurement. scripts/verify_claims.py in bawbel/.github, run 2026-10-20 online (ave@6b9f174, scanner@91c5b35), measured C-01 87, C-03 43, C-06 12, C-07 120, C-08 6.

HTML: tags balance on both pages (html.parser walk). The comparison table has 1 header row and 11 body rows of 5 cells. New links return 200: scanner issue #79, the scanner README #why-bawbel anchor, and the raw register URL.

Not checked: rendering in a browser. The CSS classes are the existing ones (yes, no, col-bawbel, eco-status).

Follow-up in bawbel/.github (separate task)

Against this branch, the register's website surfaces resolve as follows. The stat-block patterns for C-01, C-06, C-07, and C-08 still match, with values 87, 12, 120, and 6. These ten no longer match because the counts are gone:

C-01 index.html, docs/index.html, README.md  (\d+) (published )?(AVE )?records
C-06 index.html  (\d+) built-in attack chains     docs/index.html  all (\d+) toxic flow chains
C-07 index.html  (\d+) rules total                docs/index.html  (\d+) detection rules
C-08 index.html  <h3>(\d+) Detection Engines      docs/index.html  (\d+) detection engines
C-17 index.html  (\d+) endpoints

Once this merges and reaches main, the verifier will report those surfaces as "pattern not found" until the register drops them. C-03 also gains a website stat surface.

🤖 Generated with Claude Code

The stat block said 51 AVE records and 121 detection rules. It is now
written by scripts/sync_site_numbers.py from the expect values in
bawbel/.github claims/register.yaml: 87 published, 43 with a detection
engine, 12 attack chains, 6 detection engines, 120 detection rules.
--check fails when the block is stale or a count is typed anywhere else
on the site, including a badge. Counts in prose, the meta description,
the ecosystem cards, and the README badge are removed.

The comparison table is rendered from the scanner README table (SCAN-3):
NVIDIA SkillSpector added, unverifiable cells as ?, the telemetry row
where SkillSpector leads kept. The PiranhaDB card no longer says LIVE.
The pre-commit card's "Local mode for air-gapped environments" is
removed: the published hooks have no such mode.

A workflow runs the tests and --check on pull requests, and weekly opens
a pull request against develop when the register changes.

Refs #22

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: chaksaray <chaksaray@gmail.com>
This pull request already rewrites the line (the count is gone), so
the link changes with it. The host migration in #25 makes the same
edit, so the two merge cleanly in either order. aveproject.org/registry.html
returns 200 and ave.bawbel.io is retired.

Refs #22

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: chaksaray <chaksaray@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant