ci(agent-image): eval-only pre-merge check for the compass-agent image - #204
Merged
Merged
Conversation
rigel-mintaka
force-pushed
the
compass-runner-1749-agent-image-eval-check
branch
2 times, most recently
from
August 7, 2026 05:12
a48d670 to
71e6ee5
Compare
…on project
agent-image/ was outside moon, so the CI gate never built the image —
it had zero pre-merge coverage, and an image-build break (a bun-drift
assert in toolchain.nix, a stale FOD hash or a broken bundle in
entrypoint.nix) surfaced only post-merge in the publish workflow, while
a consumer waited on a tag. This registers agent-image/ as the
compass-agent-image moon project so the gate builds the image on any PR
that affects its closure, pulling the whole image-build break class to
PR time.
Closes SEA-1749.
## How
agent-image/moon.yml defines a nix `build` task that realises the image
with the same fork-pinned derivation the publish lane ships and the
local dogfood load produces:
nix run path:../forks/devenv#devenv -- container build agent
so a green build proves the exact artifact that publishes still builds —
both an eval-time break (the bun-pin drift assert) and a realise-time
break (FOD-hash invalidation, a broken bundle), the full class. The `ci`
task depends on `build`; both carry cache:false (nix owns its store-path
caching, so moon must not false-green a build it did not re-run) and
runInCI:true, matching the two vendored fork projects' nix builds.
Registered in .moon/workspace.yml as compass-agent-image, which is what
puts it in the gate: moon discovers projects from that map, so ci.yml's
one-job `moon ci :ci` picks it up with no workflow edit.
## Cost, and why it is bounded
The image closure is the dominant CI cost (the reason ci.yml's timeout is
90m), but it is not paid on every PR. `moon ci` runs a PR's affected
projects only, and the build task's inputs scope it to the image closure:
the agent-image/ tree, packages/compass-agent/, both vendored forks, the
root package.json and bun.lock, and .prototools. A PR touching none of
those never builds the image; every push to main runs it in the full
sweep. The inputs MIRROR publish-agent-image.yml's on.push.paths — the
reviewed source of truth for what changes the published artifact — plus
.prototools (which publish excludes because a pin move there cannot change
the output, but which reddens the toolchain assert, exactly what this gate
catches at PR time). As a project in the one-job gate the image build is
now a required check: a build break blocks merge, the same posture as the
vendored forks.
## A benign hasher warn
moon's task hasher logs "Attempted to hash input
forks/devenv/tests/keycloak-certs/realms but only files can be hashed" —
the vendored devenv fork ships a directory symlink at that path (a
keycloak test fixture, never in the image closure). It is cosmetic and
cannot skew the gate: affected-scheduling is VCS-touched-file based (a
change to or repoint of any closure path still runs the build), and the
hasher's cache key, the only thing the unhashable node feeds, is unused
because the task is cache:false. Documented in the task's inputs comment.
## Verification
- `moon run compass-agent-image:build` realises the image green
(deterministic output /nix/store/...-image-compass-agent.json).
- moon resolves the build task's inputs to exactly the closure globs
above (plus moon's own .moon/*.{yml,...} config auto-input).
- Docs pass root:markdownlint (0 errors).
## Supersedes
An earlier eval-only variant of this check (a separate advisory
PR-triggered workflow running `nix eval` the image drv without realising
it) was superseded by the decision to build the full image in the gate:
eval-only caught only the eval-time bun-drift assert, whereas the moon
build also catches realise-time breaks and reuses the nix-in-gate posture
the vendored forks already establish. The frozen record's deferred-
hardening note (docs/designs/platform/compass-agent-image-publish.md) and
the publish workflow's failure-ownership comment are updated to match; the
durable operational shape is folded into docs/architecture/build-and-ci.md.
Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
rigel-mintaka
force-pushed
the
compass-runner-1749-agent-image-eval-check
branch
from
August 7, 2026 15:58
71e6ee5 to
9e71c4c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
agent-image/ is deliberately not a moon project, so the CI gate never
builds it — the image has zero pre-merge coverage, and an eval-time
break (a bun-pin drift against the toolchain.nix assert) surfaces only
post-merge in the publish workflow while a consumer waits on a tag. This
adds the cheap tripwire that pulls that eval-time class to PR time.
Closes SEA-1749.
How
The check forces the image derivation's .drvPath:
This evaluates the derivation — the .prototools read and the bun-drift
assert (agent-image/toolchain.nix:50-56) — and instantiates the .drv,
but realises none of the heavy closure. So it catches an eval-time break
(a bun-pin drift) in seconds and deliberately does not catch a
realise-time break (an entrypoint.nix FOD-hash invalidation), which only
the post-merge publish build exercises. That realise-time class stays
owned by the publish workflow's main-branch failure notification,
unchanged. This is the accepted tradeoff the frozen record names: a cheap
pre-merge tripwire, not a full image build on the hot path.
Placement
A separate PR-triggered workflow, contents:read only (an eval needs no
registry credentials), path-filtered to the image's evaluation inputs:
the exact second-source-of-truth staleness the ONE-JOB doctrine forbids.
carries no PR trigger, to keep every token/secret off fork PRs. This
check needs a PR trigger and no credentials, so it gets its own
least-privilege workflow.
Its paths filter is the deliberate inverse of the publish workflow's on
one glob: it includes .prototools (which publish excludes), because a
bun-pin move there can only fail the toolchain.nix assert, never silently
change the output — and failing that assert at PR time is precisely this
check's purpose.
The durable operational shape is folded into
docs/architecture/build-and-ci.md beside the publish section.
Verification
End-to-end, at the exact CI-faithful command (clean env, no direnv/proto,
as on a GitHub-hosted runner):
toolchain.nix "has drifted from the .prototools pin" assert message.
Workflow validated with actionlint (0 findings); docs pass
root:markdownlint (0 errors). moon ci green at the pushed head via the
jj-hp gate.
Open Questions
advisory. Implemented as a separate PR-triggered least-privilege
workflow, advisory (not a required check) — the stated assumption. It
matches the frozen record's own "separate workflow" reasoning for the
publish lane, and a nix-eval flake should never red a required merge
gate. The frozen record specifies the what (eval-only, nix eval, no
realise) but not the placement, so this is a genuine fork. If required
(branch-protection config) or a different placement is preferred, say
so and I'll adjust — not blocking, the check is correct and green either
way.
Co-authored-by: Matt Wilkinson matt@sealedsecurity.com