Skip to content
Merged
43 changes: 25 additions & 18 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,22 +222,26 @@ Existing field semantics:
appends a post-summary hint via `BadgeInfo::text_hint()` when `eligible`; the same `tool.name` is used for the slug so
the JSON `embed_markdown` and the printed hint can never disagree.

`0.8` addition (`MitigationInfo` carrier on `AuditResult`):

- `MitigationInfo { using_domain_verbs, domain_match_count, domain_match_examples, builtin_match_count, subcommand_total
}` is attached to an `AuditResult` when the audit's verdict was assisted by a documented per-CLI opt-in. Today's only
producer is `src/audits/behavioral/standard_names.rs`: when `p6-standard-names` Passes because one or more subcommands
were recognized via `.anc.toml [p6] domain_verbs` (rather than the built-in `STANDARD_VERBS` list), the audit fills
`MitigationInfo` with the bifurcated match counts and the first `DOMAIN_MATCH_EXAMPLES_LIMIT` (5) matched domain-verb
names in encounter order.
- `AuditResultView` surfaces two top-level fields derived from the carrier: `using_domain_verbs: Option<bool>` and
`domain_match_count: Option<usize>`. Both use `skip_serializing_if = "Option::is_none"` so they are absent from rows
that did not consult `domain_verbs`. The Pass row's `evidence` field is populated (rather than `null`) via
`format_pass_evidence(&mitigation)`; rows without mitigation keep the historical `evidence: null` on Pass.
- The carrier shape is deliberately not audit-specific. Future audits that admit per-CLI mitigation (suppression profile
assistance, conditional-applicability config) can populate `MitigationInfo` with the same fields rather than growing
parallel typed carriers. The semantic contract is "this verdict depended on a self-declared opt-in; here is what
assisted."
`0.8` addition (`Mitigation` carrier on `AuditResult`):

- `AuditResult.mitigation: Option<Mitigation>` is set when a Pass depended on a `.anc.toml` setting. The semantic
contract is "this verdict depended on a self-declared opt-in; here is what assisted." `Mitigation` has two variants:
- `DomainVerbs(MitigationInfo)`: `MitigationInfo { using_domain_verbs, domain_match_count, domain_match_examples,
builtin_match_count, subcommand_total }`, filled by `src/audits/behavioral/standard_names.rs` when
`p6-standard-names` Passes because one or more subcommands were recognized via `.anc.toml [p6] domain_verbs`
(rather than the built-in `STANDARD_VERBS` list), with the bifurcated match counts and the first
`DOMAIN_MATCH_EXAMPLES_LIMIT` (5) matched domain-verb names in encounter order.
- `Config(String)`: every other setting. The prose names the setting, what it contributed, and the file that
supplied it, cited through `anc_toml::Sourced::cite` (`destroy accepts -auto-approve via .anc.toml
[p5].confirm_flags`). An audit that credits a setting returns a `types::Verdict { status, mitigation }` from its
core helper.
- `AuditResultView` surfaces two top-level fields from the `DomainVerbs` variant: `using_domain_verbs: Option<bool>`
and `domain_match_count: Option<usize>`. Both use `skip_serializing_if = "Option::is_none"` so they are absent from
every other row. `scorecard::pass_row_evidence` composes a Pass row's `evidence`: `AuditResult.pass_evidence` (what
the audit observed, such as the subcommands and flags `p7-limit` matched) first, then the mitigation prose
(`format_pass_evidence` for `DomainVerbs`, the prose itself for `Config`), joined by a semicolon. A Pass with
neither keeps `evidence: null`. A setting that shapes a non-Pass verdict says so in that status's own evidence
string.

`0.9` addition (`ConfigHint` carrier on `AuditResult`):

Expand Down Expand Up @@ -324,7 +328,10 @@ agentnative. Three rules guard the probe:
1. **Bare invocation prints help** (`cli.rs`): `arg_required_else_help = true` means children spawned with no args get
instant help output instead of running `audit .`. This is also correct CLI behavior (P1 principle).
2. **Safe probing only** (`json_output.rs`): Subcommands are probed with `--help`/`--version` suffixes only, never bare.
Bare `subcmd --output json` is unsafe for any CLI with side-effecting subcommands.
Bare `subcmd --output json` is unsafe for any CLI with side-effecting subcommands. The one call outside those
suffixes is a `.anc.toml [p2] json_probe`: `run_declared_probe` runs the declared arguments exactly as written,
through the same `BinaryRunner` (no shell, timeout, closed stdin), because the tool's repository or the operator
chose them. An empty declaration declares nothing, so the bare invocation never runs.
3. **Binary discovery picks the newer of release/debug by mtime** (`src/project/bins.rs::rust_artifact`): when both
`release/<bin>` and `debug/<bin>` exist in a target directory, the function returns the one with the more recent
mtime. The target directory is cargo's (`src/project/cargo_target.rs`): `CARGO_TARGET_DIR`, else what `cargo
Expand All @@ -336,7 +343,7 @@ agentnative. Three rules guard the probe:

**Rules for new behavioral audits:**

- NEVER probe subcommands without `--help`/`--version` suffixes
- NEVER probe subcommands without `--help`/`--version` suffixes, except through a declared `[p2] json_probe`
- NEVER remove `arg_required_else_help` from `Cli`; it prevents recursive self-invocation
- NEVER revert binary discovery to the always-prefer-release shape (rule 3); that pattern silently masked
`p2-must-schema-print` regressions during the v0.4.0 spec sync
Expand Down
79 changes: 73 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,74 @@ anc . -q

## Configuration (`.anc.toml`)

`.anc.toml` declares a CLI's own vocabulary so audits stop counting it against the CLI. It carries one setting today,
`[p6] domain_verbs`, which adds verbs to the standard list that `p6-may-standard-names` checks subcommand names against:
`.anc.toml` declares a CLI's own vocabulary so audits stop counting it against the CLI:

```toml
[p2]
json_probe = ["version", "--client", "-o", "json"]
schema_command = ["explain"]

[p5]
confirm_flags = ["-auto-approve"]
not_destructive = ["clean"]

[p6]
domain_verbs = ["post", "like", "repost", "timeline"]
```

| Setting | Audit | What it declares |
| ---------------------- | ----------------------- | -------------------------------------------------------------------------- |
| `[p2] json_probe` | `p2-must-output-flag` | A read-only call that prints JSON, for anc to run and check |
| `[p2] schema_command` | `p2-must-schema-print` | The subcommand that prints the output schema, when it is not `schema` |
| `[p5] confirm_flags` | `p5-must-force-yes` | Flags that confirm a destructive subcommand, beside the built-in names |
| `[p5] not_destructive` | `p5-must-force-yes` | Subcommands whose names read as destructive but are not |
| `[p6] domain_verbs` | `p6-may-standard-names` | Verbs added to the standard list that subcommand names are checked against |

### The settings

`json_probe`: when a tool's help shows an `--output` or `--format` flag, `p2-must-output-flag` checks it by passing
`json` to the flag beside `--help` and `--version`, the only calls `anc` makes unprompted. Most tools answer those in
text, and then the row is `skip`: `anc` could not check, so the row is not scored. A declared probe is the call `anc`
runs instead, exactly as written: no shell, with the timeout, closed stdin, and `NO_COLOR=1` of every other probe. The
row passes when the call exits 0 and its stdout parses as JSON, and fails otherwise, with evidence naming the call and
the file: `` `kubectl version --client -o json` printed JSON; probe declared via .anc.toml [p2].json_probe ``.
Declare a call that only reads and exits on its own; `kubectl version -o json` contacts a cluster and exits 1 without
one, while `version --client` stays local. A probe that prints JSON also shows `p2-must-schema-print` that the tool
emits structured output when its help does not say so. The probe never stands in for the flag: with no `--output` or
`--format` in the help, the row stays `opt_out`. The nearest file that declares `json_probe` supplies it, and an empty
list declares nothing.

`schema_command`: `p2-must-schema-print` looks for a `schema` subcommand or a `--schema` flag, at the top level and one
level down. A tool whose schema surface has another name, such as kubectl's `explain`, declares the subcommand path, one
token per level (`["emit", "schema"]` for `<bin> emit schema`). The declared path counts when each token is listed in
its parent's `--help`; `anc` reads a parent's help with `--help` and never runs the command itself. The row then
passes, and its evidence names the command and the file: `` `kubectl explain` is the schema command declared via
.anc.toml [p2].schema_command ``. A declared path the help does not list leaves the row failing, and the evidence says
so. A built-in `schema` surface takes priority. Entries are lowercase. The nearest file that declares `schema_command`
supplies it, and an empty list declares nothing.

`confirm_flags`: `p5-must-force-yes` requires each destructive subcommand's own `--help` to list a confirmation flag.
The built-in names are `--force`, `--yes`, `-y`, `-f`, `--auto-approve`, `--assume-yes`, and `--confirm`. A declared
flag counts beside them, and only where the subcommand's `--help` lists it, so a declaration names the flag and cannot
stand in for one. A single-dash name such as terraform's `-auto-approve` matches as a whole word. A pass that needed a
declared flag says so in the row's evidence, naming the subcommand, the flag, and the file: `destroy accepts
-auto-approve via .anc.toml [p5].confirm_flags`.

`not_destructive`: `p5-must-force-yes` treats a subcommand as destructive by its name (`delete`, `rm`, `purge`,
`clean`, and names built on them). A tool whose `clean` clears regenerable caches, for one, declares it here, and the
audit leaves it out of the destructive set. Entries are lowercase; they are compared with the lowercased subcommand
name. The row's evidence names each subcommand left out and the file that declared it: `declared not destructive: clean
via .anc.toml [p5].not_destructive`. When every destructive subcommand is declared, the row is `skip`, as for a tool
with none. A declared subcommand still counts as a write for `p5-must-read-write-distinction`, because clearing a
cache changes state.

`domain_verbs`: `p6-may-standard-names` passes when most subcommand names are standard verbs. A declared verb counts
beside the built-in list. Entries are lowercase; they are compared with the lowercased subcommand name. A pass that
needed a declared verb carries `using_domain_verbs` and `domain_match_count` on the row.

A key `anc` does not know is ignored. A known key with a value of the wrong type, such as `confirm_flags =
"-auto-approve"`, is a parse error.

### Where `anc` looks

`anc` finds `.anc.toml` from the audit target's location, never from the directory you run it in. The files that apply,
Expand All @@ -123,17 +183,20 @@ A binary counts the same whether you pass its path or `--command` resolves it on
a dev build linked onto `PATH` keeps its repository's config. The repository root is the nearest directory holding a
`.git` entry, so a linked `git worktree` checkout or a submodule is its own root.

The files merge: each one's `domain_verbs` adds to the ones above it, and a verb listed twice keeps its first position.
If any file in the chain cannot be read or parsed, no config applies, and the `p6-may-standard-names` warning names the
failing file, such as `could not parse .anc.toml at crates/cli/.anc.toml`.
The files merge. A list setting gathers every file's entries, each file's after the ones above it, and an entry listed
twice keeps its first position; evidence credits it to the nearer file, so a flag both your `~/.anc.toml` and the tool's
repository declare reads as the repository's. If any file in the chain cannot be read or parsed, no config applies: the
`p6-may-standard-names` warning names the failing file, such as `could not parse .anc.toml at crates/cli/.anc.toml`, and
a row another setting could have shaped ends its evidence with `No .anc.toml setting applied:` and the same message.

### `~/.anc.toml`

The file in your home directory applies under every audit, inside repositories too, as the lowest layer. Keep personal
vocabulary there. Two machines with different home files can score the same tool differently; CI runners have none.
`AGENTNATIVE_HOME_CONFIG` relocates the file. Evidence and the hint then name it `$AGENTNATIVE_HOME_CONFIG`, never the
path it holds, and when it names a file that does not exist, `anc` prints a `warning:` line on stderr and applies no
user-level file.
user-level file. A `json_probe` or `schema_command` here applies to every tool you audit, so point
`AGENTNATIVE_HOME_CONFIG` at a one-off file to declare either for a single run.

### A repository you fetched: `--repo`

Expand Down Expand Up @@ -565,6 +628,10 @@ and how. Each scorecard conforms to the JSON Schema emitted by `anc emit schema`
a slug exists, even below the floor, so the site renders an SVG for every scored tool (a regression below the floor
shifts color rather than 404s). `convention_url` always points at `https://anc.dev/badge`. Schema `0.5` addition.

- `evidence` on a `pass` row: what the audit matched, for an audit that names it (`p7-limit` names each list command
and its limit flag), then, when an `.anc.toml` setting decided the pass, the setting, what it contributed, and the
file that supplied it, such as `destroy accepts -auto-approve via .anc.toml [p5].confirm_flags`. The two parts are
joined by a semicolon. `null` when the pass carries neither. See [Configuration](#configuration-anctoml).
- `config_hint`: present only on a `p6-may-standard-names` warning when no `.anc.toml` declared `domain_verbs`. `files`
lists where the setting can go, each as `{file, scope}`: `file` is `.anc.toml`, `~/.anc.toml`, or
`$AGENTNATIVE_HOME_CONFIG` (never an absolute path), and `scope` is `repository` (the root of the repository this
Expand Down
4 changes: 2 additions & 2 deletions schema/scorecard.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@
},
"evidence": {
"type": ["string", "null"],
"description": "Structured evidence for non-Pass statuses: the suppression-table marker, the unrecognized-flag list, the missing-file path, the antecedent propagation reason. `null` for clean Pass."
"description": "Structured evidence for non-Pass statuses: the suppression-table marker, the unrecognized-flag list, the missing-file path, the antecedent propagation reason. On a Pass: what the audit matched, for an audit that names it (`list (--max)`), then, when a `.anc.toml` setting decided the Pass, the setting, what it contributed, and the file that supplied it, e.g. `destroy accepts -auto-approve via .anc.toml [p5].confirm_flags`, the two joined by a semicolon. `null` for a Pass that carries neither."
},
"confidence": {
"type": "string",
Expand All @@ -107,7 +107,7 @@
},
"using_domain_verbs": {
"type": "boolean",
"description": "Schema 0.8+. `true` when the verdict depended on a per-CLI opt-in: today, `p6-standard-names` passing because `.anc.toml [p6] domain_verbs` recognized at least one subcommand. Absent from every row that consulted no opt-in."
"description": "Schema 0.8+. `true` when `p6-standard-names` passed because `.anc.toml [p6] domain_verbs` recognized at least one subcommand. Absent from every other row; a Pass another `.anc.toml` setting decided names that setting in `evidence`."
},
"domain_match_count": {
"type": "integer",
Expand Down
Loading
Loading