Skip to content

docs: scoring history for the CVSS-AI to AIVSS v0.8 migration - #344

Closed
chaksaray wants to merge 1 commit into
developfrom
docs/341-scoring-history
Closed

chaksaray wants to merge 1 commit into
developfrom
docs/341-scoring-history

Conversation

@chaksaray

@chaksaray chaksaray commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Closes #341
Refs #342, #343

Drafted by an agent from the TASK-01 runbook, after the maintainer answered its open points: the reason for the migration, the table scope (all 45 records from git), and the artifact list. The maintainer reviews every statement before merge.

Type of change

  • Documentation improvement

What changed

Adds docs/scoring-history.md, and links it from README.md (AIVSS Scoring, under the formula), CHANGELOG.md (an entry placed after 1.0.0, since the migration predates it), and GOVERNANCE.md (Decision making: "Corpus-wide score changes").

Migration facts recovered from git

first spec_version 0.8 in records/:  9e7dd0b 2026-05-12 feat: migrate to OWASP AIVSS (#17)
migration PRs:                       #17 base=develop merged=2026-05-12T16:55:00Z
                                     #18 base=main    merged=2026-05-12T17:02:44Z
                                     git diff --name-only 9e7dd0b f2f4b40 -> 0 files (same change, merged to each branch)
old fields (8c50921, parent of #17): cvss_ai_score, cvss_ai_vector
old vectors:                         45 of 45 begin CVSS:4.0; 0 carry an agentic dimension (AS, HO, TA, PE, RW)
old formula:                         SPEC.md at 8c50921, section 6, defines five agentic dimensions and no formula
CHANGELOG entry existed:             no (CHANGELOG.md starts at [1.0.0] - 2026-06-18)

Verification

records with cvss_ai_score at 8c50921: 46 files; 1 is records/TEMPLATE.json (AVE-PENDING, 0.0); 45 records, all in the current corpus
all 45:  mean -2.75 median -2.80 range -5.1 to -0.5, band changes 41, downward 45, upward 0
00016-00040 (the 25 in the release notes): mean -3.11 median -3.20, band changes 24
runbook's 25 before values vs git: 0 mismatches; after values vs dist at 6b9f174: 0 mismatches
v1.0.0 and v1.0.1 release pages: "CVSS-AI" present, all 25 of AVE-2026-00016..00040 present
formula ((cvss_base + aars) / 2) x thm x mitigation_factor reproduces 87 of 87 scores; severity agrees with band 87 of 87
hypothesis "new == old x thm": 0/25, 1/45. "AARS revised down": implied AARS > 10.0 for 23/25 (36/45), max 21.4
current: 87 records, CRITICAL 1 HIGH 20 MEDIUM 64 LOW 2; mean 5.89 median 5.90 stdev 1.23 range 3.1 to 9.2
ThM 0.75 n=25 mean 4.94, 0.90 n=13 mean 5.75, 1.0 n=49 mean 6.41; bands at ThM 1.0: CRITICAL 1 HIGH 30 MEDIUM 56
ThM 0.75 records into HIGH: 5 at 0.90, 9 at 1.0

Both reproduce blocks in the document, run as written: 9.2 and 5.6 MEDIUM; 87 {'CRITICAL': 1, 'HIGH': 20, 'MEDIUM': 64, 'LOW': 2} and mean 5.89 median 5.90 stdev 1.23 range 3.1 to 9.2.

Repo checks (python:3.12-slim, the package needs 3.11 or later):

validate_records.py              exit=0 All 87 records valid against schema/ave-record-1.1.0.schema.json.
validate_crosswalks.py           exit=0 11/11 crosswalk(s) valid
check_fixtures.py                exit=0 All 87 records have positive and negative conformance fixtures.
check_confidence_signal.py       exit=0
check_vulnerability_taxonomy.py  exit=0 (existing warning: 80 of 87 missing taxonomy fields)
check_mutation_thm_trail.py      exit=0 (existing warning: 49 of 87 with no ThM-decision trail)
check_terms_sync.py              exit=0 docs/terms.md matches the live schema.
pytest tests/ -x -q              519 passed in 0.71s

A count of U+2014 (em dash) in docs/scoring-history.md returns 0, and there are no FILL markers.

Differences from the runbook draft

  • The table covers all 45 records git shows, not only the 25 from the release notes. Those 25 are marked (1), and their aggregate matches the runbook: -3.11, -3.20, 24 band changes
  • The current distribution is recomputed for 87 records. The runbook used 80: HIGH 15, ThM 0.75 n=26, and 78 percent MEDIUM
  • The maturity-independent value includes mitigation_factor, which is 0.83 on 34 records. The runbook's (cvss_base + aars) / 2 does not reproduce published scores
  • The ThM text says 20 percent at 0.90 and 33 percent at 1.0, with counts for each
  • A short "Later corrections" section points at the 1.1.0 "AIVSS score corrections" entry, the existing example of a logged correction

Not done in this PR

No record score changed. No release note body edited. No schema change. The comment on the scanner release pages is for the maintainer after merge.

Found while drafting, not changed: in schema/ave-record-1.1.0.schema.json the thm description reads "Threat and Heuristic Multiplier (0.5-1.5)". README, CONTRIBUTING, and CHANGELOG define ThM as Threat Maturity with the valid set {0.75, 0.90, 1.0}, and the document follows them. Versioned schema files are frozen, so a fix belongs in the next schema version.

🤖 Generated with Claude Code

Adds docs/scoring-history.md. Records moved from the project-defined
CVSS-AI metric to OWASP AIVSS v0.8 in #17 (develop) and #18 (main) on
2026-05-12. 45 still-published records changed score, all downward,
41 across a severity band, and nothing published explained it.

The document gives the before and after table for all 45 (before from
git at 8c50921, after from dist at 6b9f174), marks the 25 printed in the
bawbel-scanner v1.0.0 and v1.0.1 release notes, states the reason, the
current distribution, how ThM drives band assignment, and a stability
policy for future corpus-wide changes. Linked from README, CHANGELOG,
and GOVERNANCE.

Closes #341
Refs #342, #343

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: chaksaray <chaksaray@gmail.com>
@chaksaray chaksaray added the wontfix This will not be worked on label Oct 20, 2026
@chaksaray chaksaray closed this Oct 20, 2026
@chaksaray
chaksaray deleted the docs/341-scoring-history branch October 20, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wontfix This will not be worked on

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant