Skip to content

Add MODELS.md review lineup for AISVS research - #1199

Merged
jmanico merged 4 commits into
mainfrom
models-md
Oct 20, 2026
Merged

jmanico merged 4 commits into
mainfrom
models-md

Conversation

@jmanico

@jmanico jmanico commented Oct 20, 2026

Copy link
Copy Markdown
Member

Scope

Adds a root MODELS.md that adapts the CheatSheetSeries review lineup to AISVS research: the research wiki under <version>/research/ and research-backed requirement changes in 1.02-dev/en/. No standard, research, or workflow content changes.

What changed

  • Same lineup and review contract as CheatSheetSeries, re-scoped to AISVS. Quoted requirement rows must match en/, research never adds requirements or changes levels, and minor releases keep chapters, sections, and control objectives.
  • Evidence rules build on the 1.01 completion criteria and add explicit handling for attack claims, mitigation claims, CVE states, and CVSS sources.
  • Local checks that work in this repo, which has no npm scripts: markdownlint and cspell on changed files with the CI configs, lychee for local links and anchors, git diff --check, and a requirement-row sync check against en/.
  • The Daybreak slot uses the native Codex catalog entry gpt-daybreak-blue-latest. OpenAI documents it as a deprecated alias, so the file says so and requires recording the resolved snapshot as unknown when Codex does not expose it.

Validation

  • markdownlint and cspell pass on MODELS.md with the CI configs, lychee reports no broken links or anchors across root Markdown and 1.02-dev, and git diff --check is clean. The commands run as written under macOS bash 3.2 with -eu.
  • The sync script matches all 283 requirement rows on 1.01 and reports NOT RUN on 1.02-dev, which has no research folder yet.

Review record

The file was built with the lineup it describes.

Step Model Commit Result
1. Build gpt-6-astra (Codex) from main First draft
2. Second build claude-opus-5-5 (Claude Code) build draft Fixed the Daybreak slot, scoped lint to changed files (research pages carry thousands of inherited findings), removed an unsupported rule
3. Review gpt-6.1-sol (Codex) 7391913 REQUEST_CHANGES: Daybreak instruction contradicted the pinned model
3. Review gpt-daybreak-blue-latest (Codex) 7391913 REQUEST_CHANGES: three blockers on attack and mitigation claim scoping and unpinned npx --yes with an unquoted file list; CVE and CVSS handling
3. Review claude-mythos-5-1 (Claude Code) 7391913 REQUEST_CHANGES: Daybreak contradiction, Mythos access overstated, PR precedent misquoted
3. Review claude-sonnet-5-5 (Claude Code) 7391913 REQUEST_CHANGES: duplication and cuts
4. Decide gpt-6-astra (Codex) 7391913 34 findings: 17 accepted, 14 partly accepted, 3 rejected with evidence
Focused re-review gpt-daybreak-blue-latest (Codex) f187045 MERGE, no findings
Focused re-review claude-mythos-5-1 (Claude Code) f187045 MERGE, one nit, applied in the last commit

Known limitations

  • Codex did not expose the snapshot behind gpt-daybreak-blue-latest in either Daybreak run.
  • The reviewed commits were rebased onto main after Remove the 2.0-dev folder and focus on the 1.0x series #1198 merged; 7391913 and f187045 are now fc1b92a and 60c72e3 with identical content. The final one-sentence nit commit was not re-reviewed.
  • No research change has gone through this lineup yet, since 1.02-dev/research/ does not exist.

Use the native Daybreak Blue model from the Codex catalog, note the
Mythos and Daybreak access requirements, lint only changed files
because research pages carry inherited findings, and tighten wording.
@jmanico
jmanico merged commit 599e754 into main Oct 20, 2026
5 checks passed
@jmanico
jmanico deleted the models-md branch October 20, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant