Skip to content

feat(release): one build per tag and a signed SHA256SUMS - #898

Draft
benw5483 wants to merge 7 commits into
mainfrom
ci/one-build-signed-checksums
Draft

benw5483 wants to merge 7 commits into
mainfrom
ci/one-build-signed-checksums

Conversation

@benw5483

@benw5483 benw5483 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Before the first release after this merges: someone with @ActualAI npm access adds release.yml as a trusted publisher on all five packages (@actualai/actual and the four platform packages) and ticks direct npm publish, since a new entry allows only staged publishing by default. npm checks the workflow's file name, so without the entry the npm publish job fails with ENEEDAUTH. The npm-publish.yml entry can go once that release is out.

Before this leaves draft: someone with admin rights in the Google Cloud project runs the one-time setup in docs/RELEASE_SIGNING.md, test signature included, and the two public keys it writes to release-keys/ land here. Until then those files are placeholders.

Summary

Each tag now gets one build, and every channel ships it. release.yml builds the four targets once, writes SHA256SUMS for the tarballs and attaches it to both release pages, reads the Homebrew formula's sha256 values from it, and publishes the five npm packages from the same tarballs. Before publishing, it checks that each platform package carries its tarball's binary, byte for byte. npm-publish.yml, which ran a build of its own, is deleted.

After a release, a member of the signing group runs cargo run -p release-sign -- release <tag>. It checks both release pages against the release run's own artifact, signs SHA256SUMS with an Ed25519 key in Cloud KMS that nobody can export, verifies the result against release-keys/ and uploads SHA256SUMS.minisig to both pages. Nothing in GitHub can make a signature.

The risk sits in the first release after merge, which is the first live run of the new npm and release jobs. This PR's own CI runs the workflow as a dry run that builds and checks everything and publishes nothing.

This PR also fixes the pre-release tag skip, which has never worked. Until now an -rc or -test tag would have published to every channel, and from here it stops at the tag check. That means a pre-release tag can't be used to try the new jobs out.

Audit matrix

Resource Rule Threat Covered
Release tarballs on both release pages Each is listed by sha256 in SHA256SUMS, which SHA256SUMS.minisig signs with the org key Someone who can replace release assets swaps in a tampered binary Yes for anyone who runs minisign -V; self-update relies on it once its own check ships
SHA256SUMS before it's signed The signing script signs only a SHA256SUMS that matches the release run's own copy, with every tarball matching it Assets replaced between publishing and signing get signed Yes
Primary signing key (Cloud KMS) Can't be exported; only the team's Google group holds the signer role; signing runs by hand, never in CI Someone who can run the release workflow, or holds a GitHub secret, gets a valid signature Yes. Any team member can sign, and the KMS audit log records who did
Spare signing key (Cloud KMS) No signers until the primary is retired; installed CLIs trust both public keys One misused login signs with both keys, leaving installed CLIs no clean key Yes
Signatures already made minisign can't revoke a signature; a later CLI release can stop trusting a key A signature made by a login that has since been removed stays valid Partial (only CLIs updated past that release stop trusting it)
Older signed releases The trusted comment names the tag; self-update checks it and only moves to a newer version An older release's signed files served as a newer one Yes, once self-update's check ships
npm packages Published by release.yml through npm trusted publishing A tampered npm package Out of scope (the signature doesn't cover npm installs)
Homebrew formula Its sha256 values come from the same build; the release workflow pushes it to the tap The formula pointed at a tampered tarball Out of scope (Homebrew doesn't check the signature)

One addition from the build: the basic Owner and Editor roles can sign with any key in their project, whatever the key's own policy says. So the setup doc has the key ring live in a project where only signers hold those roles, and it gives a command that lists everyone who can sign.

Test plan, design notes and review rounds

Test plan

  • cargo test -p release-sign -- --include-ignored on macOS: 55 passed, including the check with stock minisign -V 0.12. The tests sign with a local Ed25519 key in place of KMS; a fake KMS endpoint signs whatever data it's sent the way the real one does, and fake release pages cover a tampered tarball, an unlisted or missing asset, a page whose SHA256SUMS differs from the run's, a key release-keys/ doesn't hold and a failed upload. No test calls KMS or GitHub.
  • Negative controls on the format: writing the legacy Ed algorithm marker, or a global signature that leaves out the file signature, turns both the stock minisign -V test and the minisign-verify tests red.
  • An independent cross-check: an OpenSSL Ed25519 key standing in for KMS. release-sign public-key converted its SPKI PEM, OpenSSL made the two raw signatures, and stock minisign -V accepted the file.
  • cargo clippy -p release-sign --all-targets -- -D warnings, cargo clippy -- -D warnings, cargo fmt --all --check and cargo test --workspace --features integration --no-run all pass.
  • actionlint 1.7.12 with shellcheck reports nothing on the new release.yml.
  • The tag check, run under GNU grep 3.11 in an ubuntu:24.04 container, the runner's image. Before the fix every ref failed with the grep error and counted as stable. After it, v0.9.0, v0.10.0 and 898/merge stay stable, and -alpha.3, -beta, -beta2, -rc, -rc.1, -test and -test.4 tags skip. This PR's dry run on the fix logs Stable release: 898/merge with no grep error.
  • The new jobs' run: scripts, extracted from release.yml, ran in a Debian container with npm 11.21.1 and the actual-releases scaffolding at da65519. Both modes came out right. SHA256SUMS was written and checked, all five packages packed, the four binaries compared equal, npm publish --dry-run accepted all five, and the Homebrew step read its four values from SHA256SUMS. The publish job's unpacked packages were identical to the packed directories. Negative controls: a tarball that no longer matches SHA256SUMS, a package carrying a different binary, a binary without its executable bit and a SHA256SUMS missing a line each stop their step with an error naming the problem.

Design notes

  • npm 11 exchanges the OIDC token for a publish token even under npm publish --dry-run (the exchange runs before the dry-run check in lib/commands/publish.js). So the job that packs and checks the packages, which also runs on pull requests, has no id-token permission. A separate job, Publish to npm, holds id-token: write, runs only on a tag push, unpacks the checked packages and publishes them with the same npm publish --access public as before.
  • npm publish --dry-run still refuses a version the registry already holds, so a dry run packages 0.0.0-dry-run.<run id> and passes --tag dry-run.
  • The dry run triggers on pull requests that change release.yml, crates/release-sign or release-keys, since it builds on two macOS runners.
  • The npm scaffolding checkout uses the workflow's own token, since actual-releases is public, and keeps no credentials. The workflow's default permission is now contents: read, with contents: write on the GitHub Releases job alone.
  • Signers pass the KMS key version's full name with --kms-key. The key id isn't covered by either minisign signature, so the tool attaches the id of whichever committed key the signature verifies against, and signers need nothing beyond roles/cloudkms.signer.
  • The Linux arm64 cross-compiler install now times out after ten minutes, because with one build a hung job holds back npm as well. The gated macOS signing steps are unchanged.
  • The pre-release tag check's grep pattern starts with a dash and had no -- before it. GNU grep read the pattern as options, failed with invalid option -- '(', and the check counted every tag as stable. The v0.9.0 release run and this PR's earlier dry run both log the error. With -- added, an -alpha, -beta, -rc or -test tag skips the builds. Every publishing job sits downstream of them, so nothing publishes, which is what CLI_DISTRIBUTION.md already said. None of the repository's 13 tags is a pre-release, so no past release relied on the old behaviour.
  • The macOS builds now run in the dry run too, and their signing steps still key off the Apple certificate secret. No such secret exists yet, so they skip. Once the Developer ID signing work adds it, a pull request from this repository that changes the release pipeline would also sign and notarize during its dry run, and that work should decide whether it keeps that.
  • The coverage workflow's 100% per-file gate measures the root crate's library only, so release-sign isn't in it. Its tests run in this workflow's Signing Tool job, stock minisign -V included, and in Build and Test's cargo test --workspace.

Review rounds

  • Architecture review: clean. It raised three points. CodeQL read the word trusted as a secret, and renaming a loop variable and a test helper cleared those alerts. The other two, the coverage gate's scope and the macOS signing steps in the dry run, are covered in the design notes.
  • Code review, round 1: one finding, the pre-release tag check above, fixed here.
  • Code review, round 2: one new finding, fixed here. The tool printed a key id with {:X}, so an id whose top hex digit is 0 lost a digit, while stock minisign always shows 16. The id is now zero-padded, and the round-trip test, which uses a leading-zero id, expects 16 digits. Stock minisign 0.12 prints a leading-zero id as 02A5C0EA5B93F986 in the comment minisign -G writes and in the mismatch minisign -V reports. The test count is still 55.

Generated by the operator's software factory.
• City: factory-main · Agent: local-core.builder-2
• On behalf of: @benw5483

release-sign is a workspace crate, separate from the CLI, that a member of the signing group runs after a release. Its release command finds the push run of release.yml that built the tag and checks the SHA256SUMS in that run's release-assets artifact against the tarballs the run built. It then checks that both release pages, on actual-cli and the actual-releases mirror, carry that same SHA256SUMS, tarballs that match it and nothing else. Only then does it sign.

The key is an Ed25519 key in Google Cloud KMS, which signs the exact bytes it's given, so a minisign signature takes two asymmetricSign calls: one over the BLAKE2b-512 digest of SHA256SUMS, and one over that signature followed by the trusted comment, timestamp:<seconds>, file:SHA256SUMS and tag:<tag>, separated by tabs. The key id isn't signed, so the tool attaches the id of whichever key in release-keys/ the signature verifies against, checked with the minisign-verify crate, and signers need only the KMS signer role. It then uploads SHA256SUMS.minisig to both release pages. For the one-time setup, public-key turns a KMS public key PEM into a minisign public key and sign-file signs a local file.

release-keys/ holds placeholders until the setup exports the two keys. The tests sign with a local Ed25519 key in place of KMS, against a fake KMS endpoint and fake release pages, and an ignored test checks the output with stock minisign -V. No test calls KMS.

City: factory-main · Agent: local-core.builder-2
release.yml and npm-publish.yml each built the four targets on every tag, so npm shipped binaries from one build and the GitHub releases and the Homebrew formula from another. release.yml now builds once, and a Package job writes SHA256SUMS for the four tarballs. The GitHub Releases job attaches SHA256SUMS to both release pages, the Homebrew job reads the formula's sha256 values from it, and the npm work moves over from npm-publish.yml, which is deleted.

The npm work is two jobs. npm Packages fills the platform packages from the release tarballs, packs all five and checks that each platform package carries its tarball's binary, byte for byte. Publish to npm unpacks those checked packages and publishes them through OIDC trusted publishing, as before. It is the only job that holds id-token: write, because npm exchanges the OIDC token even for npm publish --dry-run. The scaffolding checkout uses the workflow's own token, since actual-releases is public, and keeps no credentials. The workflow's default permission drops to contents: read, with contents: write on the GitHub Releases job alone.

A pull request that changes release.yml, crates/release-sign or release-keys runs the workflow as a dry run: the builds, SHA256SUMS, the npm packages and their checks, npm publish --dry-run on a 0.0.0-dry-run version the registry can't already hold, and the signing tool's tests on macOS with stock minisign. The GitHub release, Homebrew and npm publish jobs are skipped. The pre-release tag skip and the gated macOS signing steps are unchanged. The Linux arm64 cross-compiler install now times out after ten minutes, because a hung build would now hold back npm too.

City: factory-main · Agent: local-core.builder-2
docs/RELEASE_SIGNING.md covers checking a download with minisign -V, the trusted comment's format, and signing a release with release-sign. It also gives the one-time Google Cloud setup: both keys, the signer role on the primary key for the signing group alone, Data Access audit logs for Cloud KMS, the public key export and a test signature. Finding who signed what and rotating to the spare key close it out. The basic Owner and Editor roles can sign with any key in their project, so the setup has the key ring live in a project where only signers hold them.

CLI_DISTRIBUTION.md now describes the single release workflow, SHA256SUMS, the signing step and the npm trusted publisher that release.yml needs. The README gets a short section on checking a downloaded release.

City: factory-main · Agent: local-core.builder-2
Comment thread crates/release-sign/src/main.rs Fixed
Comment thread crates/release-sign/src/main.rs Fixed
Comment thread crates/release-sign/src/main.rs Fixed
Comment thread crates/release-sign/src/release.rs Fixed
CodeQL's cleartext-logging query reads a variable named `trusted` as a secret, so it flagged the four messages that say which key signed. Those messages print the key's name and key id, and release-keys/ publishes both. The loop in encode_for_trusted_key now names each key `candidate`, and nothing else changes.

City: factory-main · Agent: local-core.builder-1
The same CodeQL heuristic matches any name containing `trusted`, and the test helper that builds a release key for a test signer was called `trusted`. The tests pass its keys through sign_file and sign_release to the "Signed with the ... key" message, so two of the four alerts outlived the loop variable's rename. The helper is now release_key, and only test code changes.

City: factory-main · Agent: local-core.builder-1
The stable-tag check's pattern starts with a dash, so GNU grep took it for options, printed "invalid option -- '('" and exited 2 on every run. The if fell through to its else branch and every ref counted as stable, so a pre-release tag would have published to every channel. Passing -- before the pattern makes -alpha, -beta, -rc and -test tags skip the release, as CLI_DISTRIBUTION.md already says, while stable tags and pull request dry runs still build.

City: factory-main · Agent: local-core.builder-1
minisign zero-pads a key id to 16 hex digits wherever it shows one, in the public key comment that minisign -G writes and in the key id minisign -V reports on a mismatch. key_id_hex used {:X}, so an id whose top hex digit is 0, about one in 16, came out a digit short in the .pub comment and in the tool's "Signed with" messages. RELEASE_SIGNING.md has users go by the key id minisign reports, and --key-id already takes 16 digits. The round-trip test, whose key id has a leading zero, now expects the padded form.

City: factory-main · Agent: local-core.builder-1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants