Repository navigation
feat(anc-toml): declare confirmation flags, non-destructive subcommands, a JSON probe, and the schema command - #147
Merged
Conversation
p5-must-force-yes counts a destructive subcommand as confirmed when its --help lists --force, --yes, -y, -f, --auto-approve, --assume-yes, or --confirm. The three added names are common non-interactive bypasses: cdktf's --auto-approve, apt-get's --assume-yes, heroku's --confirm. The fail evidence names the accepted flags so a tool author sees what would clear the row.
A tool whose destructive subcommands confirm with a flag outside the built-in names (terraform's -auto-approve) declares it in .anc.toml:
[p5]
confirm_flags = ["-auto-approve"]
p5-must-force-yes counts a declared flag beside the built-ins, and only where the destructive subcommand's own --help lists it. A single-dash name matches as a whole word: the help parser reads `-auto-approve` as the short flag `-a`, so HelpOutput::advertises_flag reads such names from the flag line itself.
The setting follows the [p6] domain_verbs chain: every file from ~/.anc.toml down to the target contributes, an entry listed twice keeps its first position, and a file that fails to parse voids the chain. Each entry remembers the file that declared it, credited to the nearer file when two declare it. A pass that needed a declared flag carries evidence naming the subcommand, the flag, and the file (`destroy accepts -auto-approve via .anc.toml [p5].confirm_flags`); a fail lists the declared flags with their files among the accepted ones; a void chain ends the row's evidence with the parse error.
A Pass reaches the scorecard's evidence through the mitigation carrier, which becomes an enum: DomainVerbs keeps the 0.8 row fields, and Config carries the prose for every other setting. The scorecard composes a Pass row's evidence in one place: what the audit matched comes first, then the setting that credited the pass, joined by `; `, so neither replaces the other. No row field is added, so the schema stays at 0.9.
…tructive
p5-must-force-yes classifies a subcommand as destructive by its name, so a `clean` that clears regenerable caches (biome's removes daemon log files) must carry a confirmation flag it has no reason to have. A tool declares such names in .anc.toml:
[p5]
not_destructive = ["clean"]
The audit leaves a declared subcommand out of its destructive set. Entries compare with the lowercased subcommand name and merge across the chain the way confirm_flags does. The row's evidence names each subcommand left out and the file that declared it, in a pass (`declared not destructive: clean via .anc.toml [p5].not_destructive`) or at the end of any other status. When every destructive subcommand is declared, the row skips as it does for a tool with none.
p5-must-read-write-distinction shares the name classifier and does not read the setting: a subcommand declared not destructive still changes state, so it stays on the write side, and dropping it there would hide a write surface the tool has.
…JSON When a CLI advertises --output or --format, p2-json-output validates JSON through the only probes anc runs unprompted: the flag beside --help or --version. Most CLIs answer those in text whatever the output flag says, so a miss records what anc could not check rather than what the tool does. The row skips with the same evidence and drops out of the score; as a warn it cost the tool half the credit of a MUST row (kubectl, helm, rclone, biome, ollama, and opencode in the anc 0.6.0 corpus). p2-must-schema-print and p2-should-schema-file name p2-json-output as their antecedent, so the skip carries into both rows as `antecedent p2-json-output could not be measured`, per the propagation table, and the schema rows are no longer graded against output anc never saw.
When a CLI shows an --output or --format flag that anc cannot validate through --help and --version, p2-must-output-flag skips: anc runs nothing else unprompted. A tool names the read-only call that prints JSON in .anc.toml:
[p2]
json_probe = ["version", "--client", "-o", "json"]
anc runs exactly those arguments through the shared runner (no shell, same timeout, closed stdin, NO_COLOR=1). The row passes when the call exits 0 and its stdout parses as JSON, with evidence naming the call and the file (`kubectl version --client -o json` printed JSON; probe declared via .anc.toml [p2].json_probe), and fails with what the call did instead: exited 1, printed no JSON on stdout, timed out. The probe takes over only from the unverifiable skip: a pass on the safe probes stands, and a tool whose help shows no output flag stays opt_out, with a note that the declared probe does not stand in for the flag.
A probe that prints JSON also opens p2-must-schema-print's gate for a tool whose top-level help names no output format (kubectl lists `-o` only on its subcommands); the row then names the probe.
The nearest file in the chain that declares json_probe supplies it, an empty list declares nothing, and a value of the wrong type voids the chain like any other key.
probe_named captures `<bin> <name> --help` the way HelpOutput::probe captures the top-level help: partial output from a timeout or crash counts, and a child that could not spawn or printed nothing is dropped. probe_help holds that rule for any subcommand path, and probe_named and probe_subcommands call it for each first-level name, so a deeper probe reads help the same way.
…ting force_yes's not_destructive exclusions and schema_print's json_probe gate both record a declaration the same way: as prose in a Pass's evidence, after any prose already there, or as a note at the end of any other status. Verdict::crediting holds that rule once.
p2-must-schema-print accepts a `schema` subcommand or a `--schema` flag, at the top level or one level down, and fails a CLI whose schema surface has another name (kubectl's `explain`). A tool names that surface in .anc.toml as a subcommand path, one token per level:
[p2]
schema_command = ["explain"]
The declared path counts when each token is listed in its parent's --help; anc reads each parent's help through the shared --help probe and never runs the command. A listed path passes the row with evidence naming the command and the file (`kubectl explain` is the schema command declared via .anc.toml [p2].schema_command); a path the help does not list keeps the fail and says so. A built-in schema surface takes priority.
A token counts when the help parser lists it, or when an indented line's text before the description gap is the token, since kubectl's `Basic Commands (Intermediate):` heading is one the parser does not read.
The nearest file in the chain that declares schema_command supplies it, an empty list declares nothing, and a value of the wrong type voids the chain.
brettdavies
added a commit
to brettdavies/agentnative
that referenced
this pull request
Oct 6, 2026
…chema, exempt argument-free subcommands from examples, and list real audit IDs (#58) ## Summary Three MUST requirements read narrower than what they measure, and the principle files named audit IDs anc does not report. Measuring the curated CLI corpus with anc 0.6.0 surfaced each. - **P3 `p3-must-subcommand-examples`** applies to subcommands that take their own arguments or options. A subcommand whose `--help` shows no positional argument (a nested command list counts as one) and no option beyond `-h`/`--help` and the global flags every subcommand inherits is exempt: its usage line is its whole call shape, so an example would repeat it. The frontmatter applicability, the MUST text, and two Evidence lines change to match. - **P5 `p5-must-force-yes`** accepts a documented confirmation flag under another name. `--yes` and `--force` stay the recommended names; a flag under another name satisfies the requirement when the command's `--help` documents it as the flag that confirms without a prompt (`terraform destroy -auto-approve`). - **P2 `p2-must-schema-print`** accepts a documented introspection command under another name (`kubectl explain`). A `schema` subcommand or `--schema` flag stays the recommended surface. - **Audit IDs.** Each principle's closing line lists the audit IDs anc reports for it, in requirement order, checked against anc v0.6.0's source and `anc audit --principle N` output. Seven listed IDs did not exist (`p2-output-json`, `p2-output-format`, `p2-stderr-diagnostics`, `p3-after-help`, `p4-unwrap`, `p5-destructive-guard`, `p7-timeout`), P3's line lacked `p3-unprefixed-command-list`, and P8 had no closing line. - **README** points at `VERSION` instead of a pinned version number, and spells out "MUST requirements" where the prose linter blocked "MUSTs". The three MUST edits change requirement text and applicability without changing any level; they bump `last-revised` on P2, P3, and P5. ## Changelog ### Changed - Change P3 `p3-must-subcommand-examples` to apply to subcommands that take their own arguments or options; a subcommand that takes none (no positional argument, no nested commands, no option beyond help and the global flags) is exempt. - Change P5 `p5-must-force-yes` to accept a confirmation flag under another name when the command's `--help` documents it as confirming without a prompt, with `--yes` and `--force` as the recommended names. - Change P2 `p2-must-schema-print` to accept a documented introspection command under another name as the schema surface, with `schema` / `--schema` as the recommended surface. ### Fixed - Fix the audit IDs each principle lists as measuring it, which named seven IDs anc does not report and omitted others. ## Linked audit review - P3: brettdavies/agentnative-cli#142 (merged): exempts subcommands that take no arguments from `p3-subcommand-examples`, with the same rule. - P5 and P2: brettdavies/agentnative-cli#147: `.anc.toml` `[p5] confirm_flags` (a declared flag counts only when the destructive subcommand's own `--help` lists it), `[p2] schema_command`, and the built-in `--auto-approve`, `--assume-yes`, `--confirm`. - Audit-ID lines and README: no audit changes needed; prose only, the IDs are the ones anc already reports. ## Human reviewer **Reviewer:** @brettdavies (review pending before merge) ## AI disclosure An AI coding agent drafted every edit from the anc 0.6.0 corpus measurements and anc's source; the maintainer reviews before merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.anc.tomlgains four settings that let a tool's repository tell anc what its help cannot say by name alone, using the same mechanism as[p6] domain_verbs: files layer from the target's git root down plus~/.anc.toml(relocated byAGENTNATIVE_HOME_CONFIG), the nearer file is credited, a wrong type voids the whole chain, and the evidence names the setting and the file it came from.[p5] confirm_flags(for example["-auto-approve"]): confirmation flags that count beside the built-ins forp5-force-yes. A declared flag counts only when the destructive subcommand's own--helplists it, so the setting adds a name anc accepts but cannot vouch for a flag the help never shows. Single-dash long flags are matched by their whole name.[p5] not_destructive(for example["clean"]): subcommands the tool declares are not destructive drop out ofp5-force-yes; when none remain the row readsskip, as for a tool with no destructive subcommands.p5-read-write-distinctionstill treats them as writes.[p2] json_probe(for example["version", "--client", "-o", "json"]): a safe read-only invocation anc runs with exactly those arguments (no shell, the usual timeout and sandboxing).p2-json-outputpasses when it exits 0 with one JSON value on stdout and fails otherwise; it replaces only the case where anc cannot validate JSON on its own probes.[p2] schema_command(for example["explain"]): a schema command under another name that counts forp2-schema-printwhen the help lists it.Two changes need no configuration:
p5-force-yesaccepts--auto-approve,--assume-yes, and--confirmbeside--force,--yes,-y, and-f.p2-json-outputreadsskip, notwarn, when an output flag exists but anc's safe probes cannot validate JSON. The warn recorded anc's inability to check, not the tool's behavior. Antecedent rules carry the skip intop2-schema-printandp2-schema-file.A passing row's
evidencelists what the audit matched first, then the.anc.tomlsetting the pass relied on, joined by a semicolon. The scorecard schema stays at 0.9 with no new field. The shared subcommand--helpprobe is one helper (probe_help) used by every audit that probes subcommands.The spec's P5 and P2 wording that these settings implement is in brettdavies/agentnative (brettdavies/agentnative#58).
Changelog
Added
[p5] confirm_flagsto.anc.toml: confirmation flags under other names (-auto-approve) count forp5-force-yeswhen the destructive subcommand's own--helplists them.[p5] not_destructiveto.anc.toml: subcommands a tool declares are not destructive (a cacheclean) are left out ofp5-force-yes.[p2] json_probeto.anc.toml: a safe read-only invocation anc runs to verify JSON output when its own probes cannot.[p2] schema_commandto.anc.toml: a schema command under another name (explain) counts forp2-schema-print.Changed
p5-force-yesto accept--auto-approve,--assume-yes, and--confirmas confirmation flags.p2-json-outputto readskipinstead ofwarnwhen an output flag exists but no safe probe can validate JSON..anc.tomlsetting the pass relied on, after what the audit matched.Type of Change
feat: New feature (non-breaking change which adds functionality)fix: Bug fix (non-breaking change which fixes an issue)refactor: Code refactoring (no functional changes)perf: Performance improvementdocs: Documentation updatetest: Adding or updating testschore: Maintenance tasks (dependencies, config, etc.)ci: CI/CD configuration changesstyle: Code style/formatting changesbuild: Build system changesBREAKING CHANGE: Breaking API change (requires major version bump)Related Issues/Stories
p5-must-force-yes, P2p2-must-output-flag,p2-must-schema-printTesting
Test Summary:
left: Fail("destructive subcommand(s) without --force or --yes: destroy…") right: Pass;confirm_flagsprecedenceleft: [] right: [("-auto-approve", ".anc.toml"), ("--nuke", "~/.anc.toml")]; unverifiable JSONexpected Skip, got Warn("--output/--format flag detected but could not validate JSON…");schema_commandleft: Fail("…no schema subcommand…") right: Pass; pass-row evidenceleft: Some("delete (--force).") right: Some("delete (--force); destroy accepts -auto-approve via .anc.toml [p5].confirm_flags").cargo fmt --check, both clippy runs,cargo test(1191 passing),cargo test -- --ignored, and the pre-push battery pass; each commit builds and passes clippy and tests on its own.confirm_flags = ["-auto-approve"]p5-force-yesstill fails:terraform destroy -helpdoes not list-auto-approve(onlyapply -helpdoes); the evidence names the settingnot_destructive = ["clean"]p2-json-outputskip;p5-force-yesskip with the settingp2-json-outputskipjson_probe = ["version", "--client", "-o", "json"]p2-json-outputpass;p2-schema-printfail (kubectl has noschemacommand)schema_command = ["explain"]json_probe = ["repo", "list", "-o", "json"]p2-json-outputskip / pass;p2-schema-printskip / failp2-json-outputandp2-schema-printskipp2-json-outputskipA probe that does not work fails the row: kubectl
version -o jsonwithout--clientexits 1 (it reaches for a cluster), helmversion -o jsonexits 1 (unknown flag).Files Modified
Modified:
src/anc_toml/mod.rs,src/types.rs,src/scorecard/mod.rs,src/runner/help_probe/mod.rs,src/audits/behavioral/{force_yes,json_output,schema_print,standard_names,subcommand_help,read_write_distinction}.rsREADME.md,CLAUDE.md,schema/scorecard.schema.json(descriptions only)Created:
src/anc_toml/settings.rs,tests/anc_toml_settings_integration.rsRenamed:
Deleted:
Breaking Changes
Deployment Notes
Checklist
Additional Context
Not changed here: the shared flag parser reads a single-dash long flag such as
-force-copyby its first letter, so the built-in-fcan match it forp5-force-yes;HelpOutput::advertises_flag, added here for declared flags, is where that fix would go. kubectl's grouped command headings (Basic Commands (Beginner):) are not read as command lists, so kubectl's destructive and read/write rows skip.