ToggleSpecialistMethod: blended reference and block legs, symmetric selection, caller-stated toggle cycle

[aclerc]

Summary

ToggleSpecialistMethod becomes one method with two blended legs and a symmetric row selection,
and the caller now states the toggle cycle explicitly: its length (toggle_period) and its grid
(toggle_datum). A selection on a signal the treatment affects (for example, keeping only rows
where a control curve was in effect) must not leave the two toggle states with different wind
mixes. Pairing, cycle-level exclusion and per-stage accounting address that. The per-bin and
power_band options read a power the treatment cannot move.

The estimate

• Two legs, blended. A reference leg (sum) compares the test turbine with the summed power
  of a reference subset. A block leg (block_mean) compares each row with its own toggle cycle's
  mean test power, and needs no reference at all, so a one-turbine campaign still gets an estimate.
  Headline and per bin are blended by minimum-variance weights that account for the legs'
  correlation, so the blend is never worse than its better leg.
• Reference subset. The reference leg runs on every non-empty subset of the available references
  and keeps the one with the smallest sigma (ties to the larger subset). More than 8 references
  raises: pass the nearest or best-correlated.
• Decisions on permuted labels. The subset choice and the blend weights are judged on a
  bootstrap with the on/off labels shuffled within each toggle cycle, so neither can favour the
  alternative whose realised uplift came out low. The reported uplift and sigma are the actual ones.
• One normal-operation filter is applied to every turbine the estimate uses.

Row selection

• pairing_max_gap (optional): a used row is kept only if the other segment has a used row
  within the gap, inclusive. It must be a positive whole multiple of the timebase. This replaces
  v0's pairing_filter_timedelta_seconds, whose floor-to-timebase semantics silently dropped every
  row below 600 s on 10-min data.
• columns.exclude_block (new column role): a flagged row removes its whole toggle cycle from
  both legs, so both states lose the same conditions. It is the symmetric alternative to
  exclude_row.
• exclude_row now reads the references' flags as well as the test turbine's. A timestamp
  flagged on any turbine a leg uses is dropped from that leg, just as if that reference were down.
  ColumnSchema.exclude_row no longer promises "test turbine only"; it leaves which turbines'
  flags count to each method.
• power_band (optional, (lo, hi] kW, finite with lo < hi): each leg keeps only rows whose
  bin power lies in the band.
• MethodOutput.selection_accounting: rows kept per segment after each stage (segment,
  filters, exclude_row, pairing, block, and power_band when set), also written per leg to
  <run>_selection_<ts>.csv.

The toggle cycle, stated by the caller

• toggle_period (required): one full on + off cycle. It sets the block leg's cycles, the label
  permutation and exclude_block.
• toggle_datum (required): any boundary of the cycle grid, never taken from the data, so a
  missing first record cannot shift every cycle. A ToggleSchedule input is checked against it.
• Studies: a toggle StudyConfig needs a toggle_datum, and treatment starts are drawn only on
  cycle boundaries, as a clock-aligned controller toggles. This changes which starts a toggle
  study draws.

Power bins

• bin_power: the per-row power the bins and power_band read.
  • "baseline" (default): no upgraded row enters it. That is rho_base * ref_total on the
    reference leg, and the mean of the cycle's baseline rows on the block leg.
  • "both_states": the two states' mean, for a campaign in which neither state is a true
    baseline. Renaming which state is "on" then moves no row between bins.

Outputs

• The data-stats CSV has one set of all/baseline/upgraded rows per leg. The results CSV
  reports each leg's ratios and uplift and the blend weight (weight_sum). Each leg's uplift and
  the headline are re-derivable from the CSVs.
• The per-bin frame carries each leg's energy sums (sum_actual_<leg>,
  sum_counterfactual_<leg>) and the bin's weight_sum, so each bin's blend is re-derivable.
• The harness uncertainty_diagnostics frame has one row per (condition, condition_bin), each
  leg's columns suffixed _<leg>.
• A pairing-gap plot (plots/2_filter/<test>_pairing_gap.png, under save_plots): each used row's
  gap to the nearest used row of the other segment, before and after pairing.

Breaking changes

• toggle_period and toggle_datum are required.
• Per-bin sum_actual / sum_counterfactual and the results CSV's ratio_baseline /
  ratio_upgraded are replaced by per-leg columns.
• Default bins read baseline power: on the reference leg the bin scale moves by about
  1 / (1 + uplift/2) against v1's two-state mean (bin_power="both_states" keeps v1's).
• exclude_row now honours reference flags.
• A toggle StudyConfig needs toggle_datum, and its draws change.

Also

• Fixes to the toggle placebo end-to-end tests (10-min fixture, availability, a zero-spread
  resample in _paired_correlation).
• Pairing gaps are read at nanosecond resolution whatever the index unit.
• Windows path-separator fix in test_uplift_plots.py.

Test plan

• pair_within boundaries; parity with v0's pairing filter at 10/30/60 min
• Each leg recovers a known uplift; the block leg with no references; a reference outage does not cost the block leg rows
• exclude_block removes the whole cycle in both legs; no flags change nothing
• The subset choice reads the permuted bootstrap; the blend is never worse than its better leg
• toggle_datum: required; the cycle grid survives a missing first record; an off-grid schedule and a tz mismatch are refused
• bin_power: baseline bins do not move with the upgraded rows, both_states bins do; with both_states, swapping the state labels moves no row between bins
• power_band: every used row lies in the band; a constant uplift reads true inside it; bad bands refused
• Headline and every bin re-derivable from the per-leg sums and weights
• Study draws land on cycle boundaries
• tests/benchmarking (not slow): 1363 passed; ruff and mypy clean

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟢 Approval recommended

Only a non-blocking documentation nit remains; no approval-blocking issues were identified.

Review effort: Lite
Findings: None

What changed in this PR

Adds pairing-aware selection, per-stage accounting, imbalance warnings, and pairing-gap diagnostics to ToggleSpecialistMethod.

Changes:

• Adds validated pairing_max_gap filtering.
• Exports segment-level selection accounting.
• Adds pairing-gap plots and Windows-safe path handling.
• Review note: update the module docstring to document the fourth diagnostic plot.

File	Summary
tests/​benchmarking/​campaigns/​test_uplift_plots.py	Makes plot path assertions platform-independent.
tests/​benchmarking/​baselines/​test_toggle_specialist.py	Tests pairing, accounting, warnings, and plots.
benchmarking/​harness/​method.py	Adds additive selection_accounting output support.
benchmarking/​baselines/​toggle_specialist.py	Implements pairing, accounting, diagnostics, and output changes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Several diagnostics contradict the blended estimate, the promised imbalance warning is absent, and block conditioning introduces correctness risks.

8 open findings

• Combined estimate uses mismatched reference totals and usage · New
• Block origin ignores the toggle schedule start · New
• Unbounded subset enumeration can exhaust runtime and memory · New
• Block conditioning signal depends on post-treatment values · New
• Missing segment imbalance warning option and firing logic · New
• Validate power_band bounds during construction · New
• exclude_row behavior breaks the public test-only contract · New
• Copied reference ratios do not match the blended uplift · New

🧠 Review effort: Balanced