Repository navigation
Build only the MCP images a change can affect, instead of all four every time - #56
Merged
bowen-bd merged 1 commit intoOct 27, 2026
Conversation
The generative image compiles the PyG extensions from source with CUDA on
arm64, because PyG publishes no aarch64 wheels there. It takes about two
hours, every time: 115.7 and 115.5 minutes on the last two main builds, while
no other image exceeds 33.
It ran on every pull request anyway, because the workflow's path filter is
per-workflow, not per-image: any venv/*/uv.lock, including venv/mlip's,
matched. Two open pull requests that bump mlip and fairchem locks would
rebuild it twice for a dependency it cannot see.
An image installs its own uv projects from their locks, so diff the change and
drop the images it cannot affect. docker/render.py matrix --changed takes the
paths, attributes venv/<project>/{uv.lock,pyproject.toml} to the image that
installs that project, and keeps everything when any path is not attributable
to exactly one image — docker/, the workflow, and the sources the images copy
in all select the whole matrix. An empty or unavailable diff also builds
everything, so a new branch or a forced push stays on the safe side.
Building too much only costs time; building too little ships an image nobody
checked. That asymmetry is why the default is everything and why the tests pin
each unattributable case.
bowen-bd
force-pushed
the
ci/skip-unaffected-images
branch
from
October 27, 2026 02:56
12f57b0 to
2f1fee6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build (generative, linux/arm64)takes about two hours, every time — 115.7 min and 115.5 min on the last twomainbuilds, against at most 33 min for any other image. On aarch64 there are no PyG wheels, sodocker/Dockerfile.cudacompiles the extensions from source with CUDA. The GHA layer cache is already configured (cache-from/to: type=gha, mode=max); the cost is not a missing cache but a lock change invalidating the layer that does the compiling.The cost was also being paid for changes that cannot possibly affect that image. The workflow's
paths:filter is per-workflow, so anyvenv/*/uv.lockmatched it. Two open pull requests bumpvenv/mlip/uv.lockandvenv/fairchem/uv.lock, and both rebuild the generative image — twice — for a dependency it does not install.This narrows the matrix by diffing the change. An image installs its own uv projects from their locks, so only a change to those locks can alter it.
Measured
Changes
docker/render.py matrix --changed PATH(repeatable, or-for stdin) narrows the matrix to the images those paths can affect.select_images()attributesvenv/<project>/{uv.lock,pyproject.toml}to the image that installs that project; a change tovenv/adit/uv.lockselectsgenerative, and tovenv/mlip/uv.lockselectsmlip.build-images.ymlgains a step that diffs against the base —origin/<base_ref>for a pull request,github.event.beforefor a push — and feeds the list to the matrix.workflow_dispatchwith an explicit image list still wins over the diff.Safety
The question is whether skipping an image can let a broken image through. The rule is built on one asymmetry: building too much only costs time, while building too little ships an image nobody checked.
So the filter narrows only when every changed path belongs to exactly one image:
venv/mlip/uv.lockmlipvenv/adit/uv.lockgenerativedocker/Dockerfile(or.cuda,images.json)src/**,pyproject.toml, anything elseEach of those rows is a test in
tests/test_images_and_manifests.py(TestMatrixSelection), along with the case that motivated the rule — a change to the mlip and fairchem locks must not select the generative image.What is genuinely given up: nothing that the previous behaviour caught. The only inputs shared across images are the build recipes and the repository sources, and both select the whole matrix. A change restricted to one project's lock has never been able to alter a different image's contents.
Verification
tests/test_images_and_manifests.py: 34 passed, including eight new selection tests and a subprocess check thatmatrix --changedemits exactly the selected images.docker/render.py servers --check,plugin-mcp --checkandtools/sync_version.py --checkremain current;ruffis clean.Note for this pull request's own CI
It changes
.github/workflows/build-images.yml, which the filter treats as shared, so this run still builds every image once — about two hours, as before. That is the intended behaviour (a build-recipe change must rebuild everything) and doubles as a check that the unfiltered path still works.