Skip to content

fix(workflow): isolate verify and publish #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
noctuum merged 3 commits into from
Oct 16, 2026
Merged

fix(workflow): isolate verify and publish #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
02f3374
fix(scripts): run no strategy in publish
noctuum Oct 16, 2026
80e5629
fix(scripts): read the AUR repo without a checkout
noctuum Oct 16, 2026
838a6e8
fix(workflow): run verify as builder
noctuum Oct 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 58 additions & 7 deletions .github/workflows/check-updates.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,8 @@ jobs:

- name: Hand over
run: |
# A process started by a PKGBUILD can outlive its step. Stop all
# of them before root reads anything builder wrote. A zombie runs
# A process builder started can outlive its step. Stop all of
# them before root reads anything builder wrote. A zombie runs
# nothing, and pid 1 of a job container never reaps one, so only
# the other processes count, by effective or real user. live exits
# with 0 for a running process, 1 for none and 2 when the list
Expand Down Expand Up @@ -207,7 +207,8 @@ jobs:
run: |
pacman-key --init
pacman -Sy --noconfirm --needed archlinux-keyring
pacman -Su --noconfirm --needed git jq go-yq
pacman -Su --noconfirm --needed sudo git jq go-yq
useradd -m builder

- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -220,14 +221,65 @@ jobs:
name: handoff
path: handoff

# git and curl read the AUR repo and the sources here, as builder,
# who cannot reach the Docker socket that root has in a job container
- name: Verify checksums
run: ./scripts/check-all.sh --verify handoff
run: |
chown -R builder:builder .
sudo -u builder ./scripts/check-all.sh --verify handoff

- name: Hand over
run: |
# A process builder started can outlive its step. Stop all of
# them before root reads anything builder wrote. A zombie runs
# nothing, and pid 1 of a job container never reaps one, so only
# the other processes count, by effective or real user. live exits
# with 0 for a running process, 1 for none and 2 when the list
# fails, and only 1 lets the step go on.
uid=$(id -u builder)
live() {
procs=$(ps -e -o uid=,ruid=,stat=) || return 2
printf '%s\n' "$procs" |
awk -v u="$uid" '($1 == u || $2 == u) && $3 !~ /^Z/ { n++ } END { exit !n }'
}
for _ in $(seq 30); do
rc=0
live || rc=$?
[ "$rc" -eq 1 ] && break
pkill -KILL -u builder || true
pkill -KILL -U builder || true
sleep 1
done
rc=0
live || rc=$?
if [ "$rc" -ne 1 ]; then
echo "::error::A process of builder is still running, or the process list failed"
exit 1
fi

# Copy regular files from the expected paths only, and nothing
# else gets uploaded. find does not follow symlinks; -print0,
# because a name may hold a newline. A package name never starts
# with a dot, so . and .. do not match.
mkdir upload
if [[ -d handoff && ! -L handoff ]]; then
pkg_re='[A-Za-z0-9@_+][A-Za-z0-9@._+-]*'
(cd handoff && find prepared status -type f -print0 2>/dev/null) |
while IFS= read -r -d '' f; do
if [[ "$f" =~ ^prepared/${pkg_re}/(PKGBUILD|\.SRCINFO|base)$ ]] ||
[[ "$f" =~ ^status/${pkg_re}$ ]]; then
[[ -f "handoff/$f" && ! -L "handoff/$f" ]] || continue
install -D -m 0644 "handoff/$f" "upload/$f"
fi
done
fi

- name: Upload verified handoff
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: verified
path: handoff/
path: upload/
# .SRCINFO starts with a dot
include-hidden-files: true
if-no-files-found: error
retention-days: 1
Expand Down Expand Up @@ -325,9 +377,8 @@ jobs:
env:
AUR_GIT_NAME: ${{ secrets.AUR_GIT_NAME }}
AUR_GIT_EMAIL: ${{ secrets.AUR_GIT_EMAIL }}
GITHUB_TOKEN: ${{ github.token }}
run: |
sudo --preserve-env=GITHUB_TOKEN -u builder env \
sudo -u builder env \
GPG_KEY_ID="${GPG_KEY_ID:-}" \
GIT_AUTHOR_NAME="${AUR_GIT_NAME:-aurtomator}" \
GIT_AUTHOR_EMAIL="${AUR_GIT_EMAIL:-aurtomator@users.noreply.github.com}" \
Expand Down
30 changes: 15 additions & 15 deletions docs/WORKFLOW.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,14 @@ flowchart TD
H5 --> H6[hand over PKGBUILD, .SRCINFO, AUR commit]
H -->|failure| J[write error to .status/]
D -->|failure| J
B2 -->|all done| V[verify job: no secrets]
B2 -->|all done| V[verify job: no secrets, as builder]
V --> V1[check-all.sh --verify]
V1 --> V2[verify-pkg.sh: download the sources, check the checksums]
V2 -->|failure| V3[take the update out, write update_failed]
V2 --> Q[publish job: SSH key, GPG key, write token]
V3 --> Q
Q --> R[check-all.sh --publish: statuses of the check job]
R --> R1[run strategy again]
R1 --> R2[publish-pkg.sh: check the prepared files against the AUR copy]
Q --> R[check-all.sh --publish: statuses and versions of the check job]
R --> R2[publish-pkg.sh: check the prepared files against the AUR copy]
R2 --> R3[git commit + push to AUR]
R3 --> K[update-readme.sh]
K --> L[commit via GitHub API]
Expand All @@ -63,7 +62,7 @@ flowchart TD
- **One YAML per package.** Adding or removing a package is one file.
- **Strategy-based version detection.** Each type of upstream source has its own script. A new source is a new script plus its entry in `scripts/validate-pkg.sh`.
- **Patch, do not generate.** aurtomator trusts your PKGBUILD and changes only `pkgver`, `pkgrel`, checksums and `.SRCINFO`. Build steps, dependencies and install files stay untouched.
- **Four jobs, one with secrets.** A PKGBUILD is code that every step reading it executes. `check` runs the strategies with the read-only workflow token, executes no PKGBUILD and decides what the run does. The jobs after it take the status of every package from `check`. `prepare` runs the PKGBUILD steps without secrets and without a token, and hands over only prepared files and update failures, because code from one PKGBUILD can write the files of every package there. `verify` downloads the sources of each prepared update and checks its checksums, without secrets, so code from one PKGBUILD cannot forge the checksums of another package. `publish` holds the SSH key, the GPG key and the write token, never executes a PKGBUILD and never downloads a source, and checks what `verify` passes on before it pushes. See [4.4](#44-prepare-and-publish).
- **Four jobs, one with secrets.** A PKGBUILD is code that every step reading it executes. `check` runs the strategies with the read-only workflow token, executes no PKGBUILD and decides what the run does. The jobs after it take the status of every package from `check`. `prepare` runs the PKGBUILD steps without secrets and without a token, and hands over only prepared files and update failures, because code from one PKGBUILD can write the files of every package there. `verify` downloads the sources of each prepared update and checks its checksums, without secrets and as an unprivileged user, so code from one PKGBUILD cannot forge the checksums of another package. `publish` holds the SSH key, the GPG key and the write token, never executes a PKGBUILD, runs no strategy and never downloads a source, and checks what `verify` passes on before it pushes. See [4.4](#44-prepare-and-publish).
- **Failure isolation.** A broken package does not block the others. Each failure is logged for its package and opens a GitHub Issue. Each package has a time budget, so one that hangs does not hold up the run either.

## 2. Package YAML Format
Expand Down Expand Up @@ -624,26 +623,26 @@ A PKGBUILD is a bash script, and every step that reads it executes it: the audit
**Verify**, `verify-pkg.sh <package-name> <dir>/<package-name>`, needs no secrets, never sources the PKGBUILD and never runs `makepkg`:

1. The prepared directory must pass step 1 of publish below.
2. Clone `https://aur.archlinux.org/<package-name>.git` anonymously. When `HEAD` is not `base`, the AUR repo moved after the update was prepared: exit code 3.
3. `PKGBUILD` and `.SRCINFO` in the clone must be regular files. The prepared files must hold no NUL byte and pass steps 4 and 5 of publish below, with the prepared `pkgver` as `<version>`. From here on, every source URL in the prepared `.SRCINFO` follows from the AUR copy.
2. Clone `https://aur.archlinux.org/<package-name>.git` anonymously with `--no-checkout`. When `HEAD` is not `base`, the AUR repo moved after the update was prepared: exit code 3. The entries at the top of `HEAD` are written to a separate directory with `git ls-tree` and `git cat-file`, so git applies none of the links, attributes or filters of the repo: a regular file with its content, a symlink as a link to nowhere, anything else as an empty directory. The steps below read the AUR copy there.
3. `PKGBUILD` and `.SRCINFO` in the AUR copy must be regular files. The prepared files must hold no NUL byte and pass steps 4 and 5 of publish below, with the prepared `pkgver` as `<version>`. From here on, every source URL in the prepared `.SRCINFO` follows from the AUR copy.
4. Each checksum array of the prepared `PKGBUILD` must hold the same values as the same key in the prepared `.SRCINFO`.
5. Every checksum other than `SKIP` must match its source, computed as makepkg does with `<algorithm>sum`. The file is the one makepkg uses. A file in the AUR repo with the name of the source comes first and must be a regular file. Otherwise a remote source is downloaded with curl, retried as makepkg does and limited to `http`, `https` and `ftp`, redirects included. A git source at `#tag=` or `#commit=` is cloned as a mirror over `https` or `http`, and its checksum is that of `git archive --format tar` of the tag or commit, with `export-subst` and `export-ignore` turned off, as makepkg computes it. A checksum for any other source, another VCS for example, cannot be checked and stops the verify.
6. Any mismatch stops the verify with exit code 1. Otherwise the exit code is 0.

**Publish**, `publish-pkg.sh <package-name> <version> <dir>/<package-name>`, never sources the PKGBUILD and never runs `makepkg`. It treats the prepared files as untrusted:

1. The prepared directory must not be a symlink, `PKGBUILD`, `.SRCINFO` and `base` must be regular files, and `base` a commit id.
2. When the prepared `pkgver` is not `<version>`, upstream moved after the update was prepared: exit code 3.
3. Clone over SSH. When `HEAD` is not `base`, the AUR repo moved after the update was prepared: exit code 3. `PKGBUILD` and `.SRCINFO` in the clone must be regular files, not symlinks.
2. The prepared `pkgver` must be `<version>`. `check-all.sh --publish` passes the version the check run found, and the prepare run was given the same one, so another `pkgver` stops the publish with exit code 1.
3. Clone over SSH with `--no-checkout`, and write the AUR copy as step 2 of verify does. When `HEAD` is not `base`, the AUR repo moved after the update was prepared: exit code 3. `PKGBUILD` and `.SRCINFO` in the AUR copy must be regular files, not symlinks.
4. The prepared `PKGBUILD` may differ from the AUR copy only in `pkgver=`, which must be `<version>`, in `pkgrel=`, which must be `1`, and in the values of the checksum arrays. The arrays keep their names and lengths, hold only quoted hex checksums or `SKIP`, and have a `SKIP` exactly where the AUR copy has one. Comments and line continuations may stand between the checksums, read as bash reads them: a `#` at the start of a line, right after the opening parenthesis, or after a space or a tab comments out the rest of the line, and a backslash that ends a line without a comment, as a word of its own, continues the array on the next line. A backslash on the line that closes the array stops the publish, because bash joins the next line to the assignment.
5. Each `source` line of the prepared `.SRCINFO`, with or without an arch suffix, must be the line of the AUR copy with the old version replaced by `<version>`. A source that is `SKIP` in every checksum array of its arch may also change the digits in the name it is saved under, the part before `::`, as `neovim-20261014.tar.gz` becomes `neovim-20261016.tar.gz` in a PKGBUILD that names a nightly download by `$(date -u +%Y%m%d)`. Its URL after `::` must still be the AUR URL with the version replaced. Verify never downloads a `SKIP` source, and makepkg takes the sources from the `PKGBUILD`. Any other line must either equal the line of the AUR copy or be that line with the version replaced in the same way, because a license, a dependency or a description can hold the old version without taking it from `pkgver`, as `GPL-2.0` does in a package at version `2.0`. The replacement also covers three forms a PKGBUILD often derives from `pkgver`: dots turned into underscores (`1_86_0`), the part before the first `+` (`0.13.0` of `0.13.0+dev+1853+g6edb38ca98`), and the version without its last component (`6.19` of `6.19.0`, as in `frameworks/${pkgver%.*}/` of KDE Frameworks). The last form counts only while it keeps a dot. Each part of a line is replaced at most once, so `5.1` inside an already replaced `5.10.0` stays. A line built from any other form of the version, a lone major version for example, stops the publish, and so does a line that has the version replaced in one place and kept in another. The package then needs a manual update. `pkgrel` and the checksums follow the rules of step 4.
6. Any other difference, and a NUL byte in either file, stops the publish with exit code 1. Otherwise steps 10 to 13 of [4.2](#42-step-by-step) follow. When the prepared files equal the AUR copy, only `current` is written to the YAML and the exit code is 2.
6. Any other difference, and a NUL byte in either file, stops the publish with exit code 1. Otherwise the index of the clone takes the tree of `HEAD` and only the two prepared files enter its work tree, so the commit keeps every other file of the AUR repo, and steps 10 to 13 of [4.2](#42-step-by-step) follow. When the prepared files equal the AUR copy, only `current` is written to the YAML and the exit code is 2.

`check-all.sh --prepare <dir>` runs no strategy and needs no `GITHUB_TOKEN`. It takes the versions from the statuses that a check run, `check-all.sh` without a mode, wrote to `.status/` before it, and reads them before the first PKGBUILD runs. It prepares every package with `new_version` or `available_downgrade` into `<dir>/prepared/` and copies the statuses to `<dir>/status/`. A failed preparation becomes `update_failed`, and a package without a status from the check run becomes `check_failed`. The other statuses stay as the check run wrote them.

`check-all.sh --verify <dir>` runs `verify-pkg.sh` for every package in `packages/` that has a directory in `<dir>/prepared/`, and runs no strategy. An update that fails is removed from `<dir>/prepared/`, its status in `<dir>/status/` becomes `update_failed`, and `<dir>/status/_error_count` grows by one. Exit code 3 counts as a failure as well: the prepare run writes `base` for every package, so a moved AUR repo looks the same as a wrong `base`. The next run prepares the package again.

`check-all.sh --publish <dir>` takes the statuses of the check run from `.status/`, as `--prepare` does, and writes `.status/` anew. `up_to_date` and `check_failed` stay as they are. A missing status, and one the check run does not write, become `check_failed`. For a package with `new_version` or `available_downgrade` and a directory in `<dir>/prepared/`, it runs the strategy once more and publishes with the version the strategy reports now. Exit code 3 leaves the status at `new_version` or `available_downgrade` without an error, and the next run prepares the package again. A package with `new_version` or `available_downgrade` and no prepared directory gets the `update_failed` status that the prepare or the verify run wrote for it to `<dir>/status/`, or `update_failed: prepare and verify handed over no update`. Nothing else is taken from `<dir>/status/`, and only a regular file without a NUL byte: the prepare run executes PKGBUILDs, and code from one of them can write the files of every package. `_error_count` is the number of `check_failed` and `update_failed` statuses.
`check-all.sh --publish <dir>` takes the statuses of the check run from `.status/`, as `--prepare` does, and writes `.status/` anew. `up_to_date` and `check_failed` stay as they are. A missing status, and one the check run does not write, become `check_failed`. For a package with `new_version` or `available_downgrade` and a directory in `<dir>/prepared/`, it publishes the version in that status and runs no strategy, so nothing contacts an upstream host while the AUR and GPG keys are in place. Exit code 3, an AUR repo that moved, leaves the status at `new_version` or `available_downgrade` without an error, and the next run prepares the package again. A package with `new_version` or `available_downgrade` and no prepared directory gets the `update_failed` status that the prepare or the verify run wrote for it to `<dir>/status/`, or `update_failed: prepare and verify handed over no update`. Nothing else is taken from `<dir>/status/`, and only a regular file without a NUL byte: the prepare run executes PKGBUILDs, and code from one of them can write the files of every package. `_error_count` is the number of `check_failed` and `update_failed` statuses.

## 5. CI Workflow: check-updates.yml

Expand Down Expand Up @@ -691,10 +690,11 @@ Every job starts with **Checkout:** `actions/checkout`, pinned by commit SHA lik

`verify`:

1. **Install dependencies:** `git jq go-yq`.
1. **Install dependencies:** `sudo git jq go-yq`, and the `builder` user.
2. **Download handoff:** the `handoff` artifact to `handoff/`.
3. **Verify checksums:** runs `check-all.sh --verify handoff`, see [4.4](#44-prepare-and-publish).
4. **Upload verified handoff:** uploads `handoff/` as the `verified` artifact with `include-hidden-files: true` and a retention of 1 day.
3. **Verify checksums:** gives the workspace to `builder` and runs `check-all.sh --verify handoff` as `builder`, see [4.4](#44-prepare-and-publish). git and curl read the AUR repo and the sources here. A job container has the Docker socket of the runner, which root can use and `builder` cannot.
4. **Hand over:** the same step as in `prepare`.
5. **Upload verified handoff:** uploads `upload/` as the `verified` artifact with `include-hidden-files: true` and a retention of 1 day.

`publish`:

Expand All @@ -704,7 +704,7 @@ Every job starts with **Checkout:** `actions/checkout`, pinned by commit SHA lik
4. **Setup workspace:** git identity for `builder` from the `AUR_GIT_NAME` and `AUR_GIT_EMAIL` secrets, default `aurtomator`.
5. **Setup AUR SSH:** writes `AUR_SSH_KEY` to `/home/builder/.ssh/aur`, writes the Ed25519 host key that <https://aur.archlinux.org> publishes to `known_hosts`, and writes an SSH config with `StrictHostKeyChecking yes` and connection multiplexing. A host that answers with another key fails with `Host key verification failed`. When the AUR changes its host key, update the key in `check-updates.yml`.
6. **Setup GPG:** with `GPG_SIGNING_KEY` set, imports the key for `builder`, passes `GPG_KEY_ID` to the later steps and enables commit signing. Without it, the step is skipped.
7. **Publish updates:** runs `check-all.sh --publish handoff` as `builder`, with `GPG_KEY_ID`, `GIT_AUTHOR_NAME`, `GIT_AUTHOR_EMAIL` and `GITHUB_TOKEN` set to the workflow token. It keeps the statuses of `check` and checks the prepared files before it pushes, see [4.4](#44-prepare-and-publish).
7. **Publish updates:** runs `check-all.sh --publish handoff` as `builder`, with `GPG_KEY_ID`, `GIT_AUTHOR_NAME` and `GIT_AUTHOR_EMAIL`, and without `GITHUB_TOKEN`, as it runs no strategy. It keeps the statuses of `check` and checks the prepared files before it pushes, see [4.4](#44-prepare-and-publish).
8. **Update README status table:** runs `update-readme.sh` as `builder`.
9. **Commit changes via API:** see [5.5](#55-verified-commit).
10. **Create issues for failed packages:** see [5.6](#56-failure-issues).
Expand Down
Loading
Loading