Skip to content

Put uncertainty and budget roles on remaining figures #14

Description

@esaran1

Why

Ranking figures now can hold out --intermediate-budget, but the regret-vs-exploration curve still uses a single gate bootstrap stream (rng=123) and no error bars. Class crossover frequency is a normalized pair-participation rate, not a claim about a population of classes. Ensemble-disagreement figures exist; they still need the same budget-role contract as the audit.

What to do

  1. Pass the same --budgets / --intermediate-budget / --fit-form as the audit.
  2. Add interval or replicate bands to regret-vs-exploration, or label it as a diagnostic sweep rather than a paper figure.
  3. Write figure_metadata.json next to every PNG/PDF (CLI already started this).
  4. If Chinchilla figures cannot support the gate curve, skip and log that; do not draw a compute-power-law substitute.

Acceptance

  • Paper figures cite the same input SHA-256 as results/audit.json
  • Intermediate budget cannot leak into projection ranking plots
  • Any figure without uncertainty is explicitly labeled diagnostic

Files

  • asla/figures.py
  • asla/cli.py
  • tests/test_figures.py

Do not fabricate

  • Fill missing crossover roots with placeholder vertical lines

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpaperPaper tables, figures, and write-up

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions