fix(insights): the narrative names the budget that failed the run

CHANGELOG.md

@@ -30,6 +30,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   under its verdict) warns that the evaluator checked nothing because no example offered
   tools. Coverage is written by `evaluate`, so an older run needs `evaluate` re-run to
   drop the stale coverage; `analyze` alone keeps it, and the `conditional_pass` with it.
+- The run-insights narrative no longer guesses why a run failed. The prompt listed each
+  budget's limit but not its result, so a FAIL on the latency ceiling alone was explained
+  as "failed to meet the budget for Trace-rule violations" — a budget that held. Every
+  budget, slice rows included, now enters the prompt with its status, the breached ones
+  are listed as `failed_budgets`, and the decision's `reason` travels with them in plain
+  words, without `evalshift.yaml` keys. Only a breached budget may be named as the cause,
+  and on a FAIL a summary or recommendation that blames a budget that held is rejected
+  and retried, then replaced by the templated prose. Naming a held budget to clear it
+  ("Cost increase stayed within its limit", "not Cost increase") is not blame, and
+  neither is a figure "over the source". Narratives are cached on the run's
+  config hash and model, not the CLI version: delete a run's `insights.json` and re-run
+  `report` to rewrite an existing one.
 
 ## [1.4.1] - 2026-10-11
 

DOCS.md

@@ -781,6 +781,10 @@ CI wiring (on `analyze` and `compare`):
 
 **A gate that measured nothing is never counted as a gate that passed.** A budget handed an empty sample — a tool-divergence ceiling on a run that scored no divergence row, a cost ceiling on a run with no priced call — is within its limit by arithmetic: `0/0` is below every ceiling. Counted naively that renders "7 of 7 budgets passed", and a narrative restates it as "all hard constraints are met" while the report body two sections above names the blocking evaluators that scored nothing. So `budgets_passed` counts only budgets that were *measured*, the blind ones are named on their own `unmeasured_budgets` / `unmeasured_evaluators` facts with a digit-free `coverage_basis` saying why, and the instruction forbids claiming a clean sweep, a met constraint or a safe migration while that basis is present. The blind-gate set is the same one `migration_decision.json`'s own `recommendations` name, computed by one shared function, so the two surfaces cannot disagree. This is distinct from the rates rule above and not covered by it: that fires when the *whole run* measured nothing, this fires when only *some* gate did.
 
+**A FAIL's cause is read off the facts, not inferred.** Every gating budget, overall and slice rows alike, enters the prompt with its status beside its limit — `budget "<label>": limit <x>, <status>`, where the status is `breached`, `within`, `measured nothing`, or `over its limit but not confirmed at this sample size`. Beside them go the `failed_budgets` list (the breached rows, overall and slice, named as the decision prose names them: "'security' slice overall regression rate") and, when the decision carries one, its `reason` as `verdict_reason`, in plain words: the `evalshift.yaml` keys it quotes are reworded (`migration_policy.max_invariant_violations` becomes "the trace-rule violations budget", or "the 'security' slice trace-rule violations budget" where the reason quotes it beside that slice's limit, and `fail_on_dropped_params` / `drop_params` are put into words), and its figures join the permitted numbers. `breached` is the only status that may be blamed: the instruction allows only a breached budget to be named as why the run failed or as something to fix, forbids describing a `within` budget as breached, exceeded, failed, missed, violated or something to address, and requires the verdict summary to agree with `verdict_reason`. Both new keys are FACTS keys like any other, so echoing either in prose rejects the generation. A narrative once explained a FAIL as "failed to meet the budget for Trace-rule violations" on a run where that budget held and only the latency ceiling was breached: the prompt had listed limits without results, so the model guessed.
+
+On a `fail` the verdict summary and recommendation are also scanned for a held budget that shares a clause with a blame word (breach…, exceed…, fail…, violat…, address…, miss/missed/misses/missing, unmet, not met, broke, broken, and over/above/beyond followed by a limit phrase — an optional "the", "its" or "their", at most two words or figures, then a limit noun: limit, budget, ceiling, threshold, maximum, allowance or policy, singular or plural, as in "over budget" or "over the +200% cost limit" — so "+102% over the source" is not blame) that is not negated by a "not", "no", "never" or "without" in the 24 characters before it. Each mention of a held budget in that clause is judged on its own, and a mention is excused — it names the budget without accusing it — in two cases. A negation directly before it, with only other budget labels, "or", "and", "nor" or commas between, clears it when the name stands alone — the label, and any labels chained to it, is followed by a comma, the end of the clause, or "or", "and", "nor" or "but": "failed only because of Latency increase, not Cost increase or Trace-rule violations" passes, while "failed with no Cost increase headroom left" is still rejected, because there the negation belongs to the noun the label modifies. A holding phrase after it describes it when the phrase is measured against the limit — "within", "inside", "under" or "below" followed by the same limit phrase the blame words use ("within policy", "within budget", "under their limits", "under the +200% cost limit"), or a bare "held", "holds" or "passed" with nothing but punctuation after it to the end of the clause ("Trace-rule violations held.") — provided that phrase is not itself negated, no blame word stands between the two, and no other budget label does unless joined by "and", "or", "nor" or a comma. A verb or a preposition alone is not a holding phrase: "the migration failed on Latency increase alone, as Cost increase and Trace-rule violations all stayed within budget" passes, while "Cost increase remained above the +200% limit", "Cost increase held at +250%, exceeding its limit", "Cost increase under the target model breached its ceiling", "Cost increase, within hours, broke its ceiling", "Cost increase exceeded its limit and stayed within budget" and "failed on Cost increase as Latency increase stayed within its ceiling" are still rejected. Four misses are accepted: a holding phrase excuses the mention before it whatever follows, so "Cost increase stayed within budget at first, then exceeded it", "Address Cost increase so it stays within budget", "Cost increase below the target's ceiling failed the run" and "Cost increase, below the ceiling on 3 prompts, failed on the rest" pass. A held budget is blamed when one of its mentions is not excused and the clause has an un-negated blame word. Clauses split at sentence punctuation (`. ; : ! ?`), at while, but, whereas, although, though and however, at ", and", ", but", ", with" and ", however,", and at an em or en dash — not at a bare "and" or a bare comma, because "exceeded latency and failed to meet Trace-rule violations" accuses both. Every budget label, held or not, is located longest-first and masked before the scan, matching case-insensitively with hyphens, whitespace and quote characters folded: "checkout slice cost increase" is still the slice's label rather than a mention of the held "Cost increase" inside it, and "Trace-rule violations" does not read as a violation of whichever budget shares its clause. A match rejects the generation like an invented number, the retry names the misblamed budgets ("Your previous answer blamed budgets that held: …", followed by "No budget was breached" on a fail no budget caused), and two rejections ship the templated prose, which names the breached budget, or the overall blocking regressions when no budget was breached, and otherwise points the reader at the decision's recommendations and the budget table. The scan is a safety net, not a parser: it catches the shipped phrasing and its close variants, and the prompt change is the primary fix. A false match is not free: it costs the retry, which is shared with invented-number rejections, and a repeated one ships the templated prose, which has no findings. A narrative is cached per run (see below), and the cache key does not change with the CLI, so an existing run keeps its old prose until its `insights.json` is deleted.
+
 ```yaml
 defaults:
   insights_model: gemini-3.1-flash-lite-preview   # optional; falls back to judge_model

docs/faq.md

@@ -34,6 +34,11 @@ deterministic templated prose instead (shown as model `none`). So a figure in
 that block is the same figure as in the tables below it, or it is not there at
 all.
 
+The same goes for the cause of a FAIL. The model is told which budgets were
+breached and which held, and only a breached one may be named as the cause. On
+a failed run a summary or recommendation that blames a budget that held is
+rejected and retried the same way, then replaced by the templated prose.
+
 It costs one model call per run, is cached in `insights.json`, and skips
 itself when no API key is configured. Disable it with
 `evalshift report --no-insights` or `evalshift compare --no-insights`.