Repository navigation
Make the E2E specs added by #838 pass on the CI runner #878
Description
What happens
Several specs added by #838 pass locally (macOS, one worker) but fail on the CI runner (ubuntu-24.04-arm, software rendering). They fail even with one browser per runner. The traces show frames 15–50 ms apart, so this is not CPU starvation. The failing assertions measure pixels, timing or audio in ways that depend on the runner. Evidence: sharded full run 41178467669 and targeted run 41185022221 on #877.
| Spec | Failing assertion | Likely cause |
|---|---|---|
bmson-generic.spec.ts:37 (canvas/lr2/beatoraja, all key counts) |
notePixels > baseline + 5 within 1 s (line 74), or the hit count (line 99) |
Each CI screenshot takes hundreds of ms, so a 1 s poll gets about two samples, and the note may pass before one lands |
bga-key-bound.spec.ts:58 (canvas/lr2/beatoraja) |
correctPixels('first') after a fresh press (line 133; line 144 has a 500 ms window) |
Tight windows around 500 ms frame intervals, combined with slow screenshots |
bga-key-bound-nonstop.spec.ts:47, bmson-bga-nonstop.spec.ts:47, bmson-bga-sequences.spec.ts:128 |
Colour sequences (colors()) within 3–4 s |
Same: short polls around timed BGA frames |
wav-effects.spec.ts:6 (keep-pitch) |
Zero-crossing rate 1,559 Hz, expected 1,650–1,850 | WSOLA output measured on a runner whose AudioContext rate likely differs from 48 kHz; the pitch mode passes |
To do
- Replace screenshot-poll timing with checks that do not race the chart. For example, pause at a known chart time before sampling (the play hook already exposes seconds), or sample the BGA or note layer state through the test hook instead of a full-page screenshot.
- For
wav-effects, record the AudioContext sample rate and assert the frequency relative to it, or measure with an FFT peak rather than zero crossings. - Re-run the targeted specs on CI with
workflow_dispatch(grepinput from Shard the full E2E run and make CI-sensitive probes reproducible #877) until green, then confirm one nightly full run.
Activity
test(e2e): find the bmson BGA box with the same colour tolerance as i…