Skip to content

data: re-transcribe the NIST AI RMF registry from AI RMF 1.0 - #219

Merged
emmanuelgjr merged 3 commits into
GenAI-Security-Project:mainfrom
Prasad-desh:fix/nist-ai-rmf-registry
Oct 9, 2026
Merged

emmanuelgjr merged 3 commits into
GenAI-Security-Project:mainfrom
Prasad-desh:fix/nist-ai-rmf-registry

Conversation

@Prasad-desh

Copy link
Copy Markdown
Contributor

What this PR changes

The registry half of #217, per @a-moskvin's split in that thread. No mapping row is touched and no vulnerability ID is affected.

Files: data/frameworks/nist-ai-rmf.json, plus its generated mirror docs/frameworks-registry.js and data/stats.json.

Checking the five IDs in #217 against NIST AI 100-1 showed the drift was not confined to them, so I read every entry against Tables 1-4. 50 of the 85 entries did not match the published statement for their ID.

Kind of error Examples
Two IDs holding each other's text GV-6 / GV-6.1; MP-1.3 / MP-1.4
Statements merged MS-2.5 carried MEASURE 2.6's opening sentence ahead of its own; MS-2.6 likewise
Shifted one position MS-4.1 held MEASURE 4.2's text, MS-4.2 held MEASURE 4.3's
Wrong text outright GV-4.1, GV-5.1, MP-2.1, MS-1.3, MS-2.2, MG-1.1, MG-3.1
Paraphrase, not the published wording GV-1.2, GV-1.5, GV-1.6 ("per" for "according to"), GV-1.7
Not in the framework at all MP-3.5 held authored text about shadow AI and endpoint agents, and was missing its parent and function fields

Missing subcategories, now added

MP-3.3, MP-3.4, MS-2.12, MS-2.13, MS-4.3, MG-4.3 — absent because the registry only held IDs that existing mappings cite.

The Core defines 19 categories and 72 subcategories; all 91 are now present, so inventory_completeness states a real total instead of unknown.

Titles

description is the published statement verbatim. title is an editorial label for display and is not part of the framework. 30 titles named a different subject than their entry; each correction is checkable against the new description in the same object.

30 title corrections
ID was now
GV-4.1 Organizational practices and norms Critical thinking and safety-first mindset
GV-5.1 Engagement with external stakeholders External feedback collected and integrated
GV-6 Policies for third-party entities Third-party software, data and supply chain
GV-6.1 Third-party risk management Third-party entity risks, including IP infringement
MG-1.1 Risk treatment prioritization Determination to proceed with development or deployment
MG-1.4 Risk treatment resources Negative residual risks documented
MG-2.1 Risk response — planned Resources and non-AI alternatives considered
MG-2.2 Risk response — incident Value of deployed systems sustained
MG-2.4 Risk response — change management Supersede, disengage, or deactivate
MG-3 Communication and documentation Third-party risks and benefits managed
MG-3.1 Pre-deployment assessment Third-party resources monitored and controls applied
MG-3.2 Communication of incidents Pre-trained model monitoring
MP-1.3 Broad set of perspectives Mission and goals documented
MP-1.4 Assessment of positive and negative impacts Business value or context of use defined
MP-2.1 Intended use and known misuse Tasks and methods defined
MP-2.2 Technology-specific risk Knowledge limits and human oversight documented
MP-3.1 Benefits and costs of AI Potential benefits examined
MP-3.5 AI system impact Human oversight processes defined
MP-4 Risk indicators Risks and benefits mapped for all components
MP-5.1 Likelihood and severity Likelihood and magnitude of impacts
MS-2.2 Evaluations for bias, security, privacy Evaluations involving human subjects
MS-2.4 Values and comparisons Functionality and behavior monitored in production
MS-2.5 Adversarial testing Validity and reliability demonstrated
MS-2.6 AI system evaluation frequency Safety risks evaluated regularly
MS-2.8 Bias and fairness Transparency and accountability examined
MS-2.9 Environmental impact Model explained and validated
MS-2.11 Fairness assessment Fairness and bias evaluated
MS-4 Measurement results Feedback on measurement efficacy
MS-4.1 Measurement results communication Measurement approaches connected to deployment context
MS-4.2 Measurement results documentation Measurement results validated by domain experts

Needs a reviewer's judgment, not changed here

MP-3.5 previously described shadow AI, and DSGAI03 and DSGAI16 both cite it with rationale text that assumes that meaning. The registry now carries the published MAP 3.5 statement (human oversight processes), so those two rows need either a re-point or a fresh rationale. Changing which control a row cites is a mapping judgment, so per STRAT-04 both rows are left exactly as they are.

The 21 row re-pointings in #217 are likewise untouched.

Type of change

  • New mapping file
  • Update to existing mapping (content, controls, CVE refs) — framework registry only, no mapping rows
  • Bug fix (broken link, typo, incorrect cross-ref)
  • New recipe (shared/RECIPES.md)
  • New tool (shared/TOOLS.md)
  • Infrastructure (scripts, CI, templates)
  • Translation (i18n/)

Source / evidence

NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023 — Tables 1-4. https://doi.org/10.6028/NIST.AI.100-1

AI RMF 1.0 is still the current version and the registry already cites it. Every description in this PR is the published statement for that ID.

One caveat worth a spot-check: the text was read from the published PDF through tooling, so character-level typography (en-dash vs hyphen in the "– as identified in the MAP function –" constructions, curly vs straight apostrophes) is the most likely residual error. The wording itself was read twice per entry, independently, and agreed both times.

Checklist

The Content, Links & data and For new mapping files only sections are written for mapping .md files; this PR touches none, so they are left unchecked rather than ticked misleadingly.

Project hygiene

  • Changelog entry added at bottom of every modified file (nist-ai-rmf.json carries a 2026-10-08 entry; the other two files are generated)
  • CHANGELOG.md updated — n/a, not a new mapping file
  • README.md counts — npm run stats:check reports README, og-image and banner all current
  • Ran node scripts/validate.js locally and it passes

Verification

  • npm run build — 0 errors, 92 warnings, 328 passed. Warning count is identical to main.
  • npm run test:scripts — 95/95, same as main.
  • Generated files are committed in sync, so the "Assert generated output is current" step after generate.js is clean.

Supersedes #218, which carried the same tree on a branch with two auto-generated commit messages.

Refs #217

50 of the 85 entries did not match the published statement for their ID. GV-6/GV-6.1 and MP-1.3/MP-1.4 each held the other's text, MS-2.5 carried MEASURE 2.6's opening sentence ahead of its own, MS-4.1 and MS-4.2 were shifted one position, and MP-3.5 held authored text that is not in the framework and was missing its parent and function fields.

Adds the six subcategories the Core defines that were absent: MP-3.3, MP-3.4, MS-2.12, MS-2.13, MS-4.3, MG-4.3. All 19 categories and 72 subcategories are now present.

Descriptions are the published statements verbatim from Tables 1-4. 30 titles that named a different subject than their entry are corrected. No mapping row is changed.

Refs GenAI-Security-Project#217

Signed-off-by: Prasad Deshpande <officiallyprasad@gmail.com>
Output of npm run stats. The control count moves from 1184 to 1190.

Signed-off-by: Prasad Deshpande <officiallyprasad@gmail.com>
Output of node scripts/generate.js, so the committed bundle matches what the generator produces.

Signed-off-by: Prasad Deshpande <officiallyprasad@gmail.com>
Copilot AI balanced review requested due to automatic review settings October 8, 2026 10:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@emmanuelgjr emmanuelgjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Thanks, @Prasad-desh — this is careful work.

Independent verification of 512faf6e against NIST AI 100-1 Tables 1–4 (pdftotext -layout, NFKC, line-wrap rejoin):

  • 91 ids = 19 categories + 72 subcategories, matching the PDF. description wording matches the published statement on 91/91; parent and function are correct on all 91.
  • The "50 of 85 wrong" claim holds (43 wording errors + 7 typography-only). Ten of the listed errors spot-checked on main, all confirmed. MP-3.5 on main is indeed not NIST text.
  • url moving from the Executive Order page to the AI RMF DOI is a good fix. stats.json changes only the control counts (+6).
  • Locally: validate 0 errors / 92 warnings (same as main), 95/95 tests, stats:check current, second generate.js run produces no drift.

Non-blocking follow-ups (fine as a later PR):

  1. Typography: GV-1.3 and GV-1.7 use a straight apostrophe where the PDF has ’; MS-2.7 uses - where the PDF has en-dashes.
  2. About seven unchanged titles still name a different subject than their (now correct) description: GV-2.2, GV-3.2, GV-4.3, MP-1.5, MP-5.2, MG-4.1, MG-4.2.
  3. data/frameworks/nist-ai-600-1.json's $comment says MG-4.3, MP-3.4, MS-2.12 and MS-2.13 don't resolve; after this PR they do.

Heads-up for reviewers, not a blocker: with the correct text in the registry, about 77 existing rows now cite a control whose published meaning differs from how the row uses it (MS-2.5 ×29, MG-2.2 ×24, MG-2.4 ×11, MG-3.2 ×9, MS-4.1 ×2, MP-3.5 ×2). #217 covers the 21 agentic rows and this PR flags the 2 MP-3.5 rows; the remaining ~54 LLM/DSGAI rows will be tracked separately for SME re-pointing. This PR rightly changes none of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants