fix(insights): the narrative names the budget that failed the run

[babaliauskas]

What this changes

The run-insights narrative (the AI-written summary at the top of report.html) is now told which budget failed the run. On a FAIL, a summary that blames a budget that held is rejected.

• Each budget shows its result in the prompt. Every gating budget, both the overall rows and the slice rows, enters the FACTS block as budget "<label>": limit <x>, <status>. The status is one of breached, within, measured nothing, or over its limit but not confirmed at this sample size.
• failed_budgets and verdict_reason. These are new FACTS lines: the breached rows, and the decision's own reason. The reason is reworded so it never quotes config keys (migration_policy.max_invariant_violations → "the trace-rule violations budget", scoped to the slice when the reason names one). Its figures join the number allow-list. Both keys are banned from prose, like the other FACTS keys.
• Instruction rule. Only a breached budget may be named as the cause of a FAIL or as something to fix. The verdict summary must agree with verdict_reason.
• validate_blame. This runs on a FAIL only, over verdict_summary and recommendation.
  • It splits the text into clauses and masks every budget label, longest first. Masking folds case, hyphens and quote marks.
  • It rejects a clause in which a within budget shares space with a blame word that isn't negated: breach/exceed/fail/violate/address/miss, or over/above/beyond plus a limit word.
  • A held budget is excused when a limit-relative holding phrase follows it ("within its limit", "under the +200% budget", "within policy", or a clause-final "held"/"passed"). It is also excused when it is a negated stand-alone mention ("not Cost increase or Trace-rule violations").
  • A rejection retries with the misblamed budgets named. A second rejection ships the templated prose.
• Docs. DOCS.md, llms-full.txt, docs/faq.md and CHANGELOG are updated, including the cache caveat: an existing run keeps its old narrative until its insights.json is deleted.

No change to the config, suite, state.json, scores.jsonl, insights.json or bundle schemas.

Why

Run r_20261012_converter_d418e5 failed on the latency ceiling alone (+175.3% against +30%), and every other budget held. Its narrative said the target model "failed to meet the budget for Trace-rule violations" and recommended "address the Trace-rule violations". That budget had passed at 0 against a limit of 1. The prompt had listed each budget's limit without its result, so the model guessed the cause. The existing number and identifier checks could not catch an accusation written in plain words.

Verified locally (CLI from this branch)

Run	Verdict	Narrative
r_20261012_planner_d51f9c	FAIL, trace rules 4/11 against 1	AI prose names the trace-rule budget, copying the plain-worded reason. It says the other 7 budgets held.
r_20261012_converter_c80f30	FAIL, latency +47.4% against +30%	Templated fallback (model: none): "the latency increase budget was breached". Nothing blames Trace-rule violations. Both AI attempts were rejected, and why is not yet known. The reason is printed in compare's warnings and was not captured. It is either a blame-check false match or the older invented-number check. Follow-up: rerun report and capture the warning.
r_20261012_triage_697d54	PASS	Correct. It also confirms #50 (semantic none with the tolerance note; agent_contract not attempted).

Known limits of the blame check

The blame check is a heuristic safety net; the prompt rule is the primary fix. The reviewer measured it on probe sets:

• Correct sentences wrongly flagged: 7 of 34 realistic correct FAIL-on-latency sentences. Each costs a retry; two in a row ship the templated prose without findings. Examples: "neither … nor … were breached", "held steady", "within bounds/tolerance".
• Accusations that still pass: 16 of 36 contrived ones. Examples: "stayed within budget at first, then exceeded it", "below the ceiling on 3 prompts, failed on the rest", "under the new pricing policy exceeded its limit".

The accepted gaps are pinned in tests and listed in DOCS.md.

Follow-ups (not in this PR)

• Tool budgets on a tool-free suite still read "2 measured nothing", and the AI suggests verifying tool calls on a suite that has none.
• When the source model broke the same trace rules, the narrative doesn't mention it (the decision's recommendations do).
• The templated budget-breach advice says "Address the blocking evaluators" even when there are none.
• Optional: add cap/bound/tolerance to the limit words (measured: 13/18 → 8/18 of "within bounds"-style false alarms).

Test plan

• pytest passes: 3093 passed, 95% coverage (make ci)
• ruff check, ruff format --check clean
• mypy --strict src/evalshift_cli clean
• CHANGELOG.md updated under ## [Unreleased]
• (If user-visible) docs updated
• Capture why the converter run's AI narrative was rejected twice

Release: 1.4.2, together with #50, via a separate chore(release) PR after merge.

🤖 Generated with Claude Code