[E14] Failure harness on the fused checkpoint's per-example predictions

[sebasmos]

Task metadata

Field	Value
Task ID	E14
Category	Evaluation
Priority	High
Complexity	Low
Est. days	1
Related	#76 (W08), PR #74
Blocks	#76

Features / method

Produce the three per-example views from the fused checkpoint's predictions rather than from a separately fitted ridge.

Description

PR #74 gave the failure harness a --manifest route, so real taxonomy, complementarity and loud-vs-silent artifacts now exist instead of only the synthetic demo. That is a real step forward, but those artifacts describe a different model from the one the paper reports.

_run_manifest in scripts/run_failure_analysis.py builds its report from masked_ridge_predict_fn, a ridge harness fitted on the train split. Table 4, the fairness stratification, the calibration and the baseline-gap diagnosis all come from the cross-attention fused checkpoint at probes/fused/cross_attn_fused.pt. The paper's core contribution is the three per-example views, so demonstrating them on a ridge while reporting aggregate degradation for the fused probe leaves that contribution still not shown for the model under discussion.

The inputs already exist: evaluate_missing_modality.py persists per-example predictions for all three conditions, and since #74 it persists the val predictions too.

Steps:

1. Add a route that reads the per-example prediction dump the missing-modality eval already writes, for all three conditions, instead of fitting a fresh model.
2. Point analyze at those predictions and emit the failure taxonomy, the complementarity matrix and the loud-vs-silent dropout profile from them.
3. Carry the checkpoint and manifest SHA-256 into the failure report, matching the provenance the other artifacts gained in Real fairness, calibration, and baseline-gap artifacts from the canonical run #74.
4. Record the producing model in the artifact itself, so a ridge report and a fused report can never be mistaken for each other.
5. Regenerate docs/results/failure_report.json and label the existing ridge output rather than overwriting it silently.

Acceptance criteria:

• The three views exist for the fused checkpoint on the real test fold.
• Every failure artifact names the model that produced it.
• docs/results/ distinguishes the ridge harness output from the fused output.
• [W08] Rebuild paper failure figures from real artifacts (currently unlabeled synthetic demo) #76 can rebuild the paper figures from artifacts that match Table 4's model.