Repository navigation
Conversation
The script-smoke job ran its ~350 checks one after another, about 77 minutes on main, and every platform job then reran the same Python suite in sequence before its build (39 of the Linux job's 66 minutes). On the slower runners that left Windows ARM64 and the macOS Intel jobs cancelled at 150 minutes on main, and pull-request jobs, with 90-minute limits, cancelled regardless of the change. Each check is its own process that writes only to fresh temporary directories, so they can overlap: - openq4_validate.py runs its Python checks concurrently, as many as the machine has CPUs. --python-jobs N or OPENQ4_PYTHON_TEST_JOBS change that and --python-jobs 1 restores sequential runs. Output is still printed in list order, one section per check, with each check's duration. - tools/validation/run_parallel_commands.py does the same for a list of commands on standard input; both script-smoke steps feed their unchanged check list through it. - validation_hardening.py covers the parallel path's failure aggregation as well as the sequential one. On a 16-thread Linux host the validator's 350 checks took 2041 s in sequence and 311 s in parallel with the same results; the whole rewritten smoke step took 343 s. The only failures, identical either way, are host-toolchain ones (newer clang/libstdc++), not parallelism. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why CI takes so long
main(35–86 min on green runs since Oct 4).needs: script-smoke), then rerun the same Python suite in sequence before building. Onmain's last green Linux job that was 39 of its 66 minutes.main's last two completed runs, Windows ARM64, macOS Intel OpenGL and macOS Intel Metal were cancelled at 150 minutes, and Windows x64 took 117–133 minutes. PR platform jobs have a 90-minute limit, so they are cancelled regardless of the change (for example Keep HUD window state across GUI revisions; add save layout contract and restore matrix #180's earlier run and Rebuilt hub2 exceeds runtime surface limits after compiler crash fix #181).The slow part is mostly checks that compile C++ harnesses from production sources, often many times for
--mutations. Each check is its own process writing only to fresh temporary directories, so they can run concurrently.Change
openq4_validate.pyruns its Python checks concurrently, by default as many as the machine has CPUs.--python-jobs NorOPENQ4_PYTHON_TEST_JOBSsets the count;--python-jobs 1restores the sequential run.==> Python check:section per check, now with each check's duration.tools/validation/run_parallel_commands.pydoes the same for a list of commands on standard input. Both script-smoke steps feed their unchanged check list to it through a heredoc, so every test name still appears in the workflow andvalidation_hardening.py's wiring checks still apply.validation_hardening.pynow covers failure aggregation on the parallel path as well as the sequential one.BUILDING.mddocuments the option.Measurements (Linux x64, 16 threads)
openq4_validate.pyPython checks (350)Both modes produced the same failure set: 11 tests that this host's newer clang/libstdc++ rejects (
-Wunused-but-set-globalunder-Werror, missing<cstdint>). GitHub's runners don't hit these, and they are unrelated to parallelism.GitHub's standard runners have 4 vCPUs, so the CI gain will be smaller than 6×. Most of these checks are CPU-bound compiles, so I'd expect roughly 3×: script smoke in about 25 minutes, and about 12–15 minutes off each platform job.
Possible follow-ups (not in this PR)
--skip-python-tests(as one job already does), since script smoke covers the platform-independent checks.🤖 Generated with Claude Code