[Innixma]

Summary

A tuning diagnostics explorer joins the tuning-trajectories explorer: metric error, Elo on validation, baseline advantage and the test - validation gaps against time, and each metric on test against validation, so the 15 diagnostic pareto_n_configs_* figures are views of a page. With static_figures="uncovered" (the generate_all_figs default) no static tuning-trajectory figure is written any more.

Details

The page:

• plot_tuning_trajectories_from_leaderboard writes tuning_diagnostics_explorer.html and tuning_diagnostics.csv next to the trajectories explorer (not in website_only mode), from the columns the trajectory leaderboard already carries. generate_all_figs also writes the composite <output_dir>/tuning_diagnostics_explorer.html with a subset selector, under save_composite_trajectories_explorer.
• Y-axes: metric error, Elo on validation, baseline advantage, and the test - validation gaps of baseline advantage, Elo and Improvability, plus Improvability and Elo for the test against validation views. X-axes: the twelve time axes, then Elo, Improvability and baseline advantage on validation, and Elo and baseline advantage on test.
• Each static figure is one view: err_tot_* is metric error against total, train or inference time per task (mean); adv*, elo_val, *_overfit are the metric or gap against time; *_vs is a metric on test against itself on validation; *_overfit_v2 is a gap against the metric on test. The Elo gap is test - validation throughout, so elo_overfit (validation - test) is its mirror.

The template:

• Metric x-axes are linear. They draw no Pareto front or optimal arrow (a metric is no cost), and a y = x line ("Test equals validation" in the legend) when the y-axis shows the x-axis's metric on test. The axis limits accept negative values there.
• Metrics can opt out of the front ("front": False, the gaps).
• build_pareto_explorer_html(drop_incomplete=False) keeps points that lack a value; each view then draws the points with a value on both of its axes. The diagnostics page needs it: a system without validation results still shows on metric error and baseline advantage. The default keeps dropping incomplete points, so the existing pages are unchanged.

static_figures="uncovered" now returns before every static tuning-trajectory figure ("all" still draws all of them). On a 14-subset generate_all_figs run with trajectories, the PDFs left are the tuning-impact bar plots: 28 instead of 238.

Tests

• ruff check . && ruff format --check . (ruff 0.14.0)
• pytest tests/tabarena/contexts tests/tabarena/plot tests/tabarena/evaluation tests/tabarena/website -q: 402 passed, with the new test_the_diagnostics_explorer_offers_every_diagnostic_view, test_uncovered_static_figures_leave_only_the_explorers, test_a_metric_x_axis_is_linear_and_keeps_incomplete_points_on_request and test_the_composite_diagnostics_explorer_holds_every_subset.
• Driven in headless Chromium on the pages of a generate_all_figs run:
  • Diagnostics explorer: opens on metric error against total time with a front; the validation Elo view leaves out the methods without validation results; Elo on test against validation draws the y = x line and no front; a gap against test advantage has a linear axis that takes negative limits; a gap against time has no front, baseline advantage against time has one; a negative limit on a time axis is flagged. The composite page switches between its 14 subsets.
  • The checks of Explorers: saved views, chart size, hidden unselected methods, a win-rate title and renames #688 (chart size, hide unselected, saved views, the composite trajectories explorer, the win-rate explorer) still pass on the regenerated pages.
  • No page errors.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ELdutHiUqkvynzEP7EnPsi