Skip to content

ci(cd): build each arch natively and smoke-test both - #69

Merged
torridfish merged 1 commit into
mainfrom
ci/native-arm64-build
Sep 15, 2026
Merged

torridfish merged 1 commit into
mainfrom
ci/native-arm64-build

Conversation

@torridfish

@torridfish torridfish commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

目的

#64 publishes linux/amd64,linux/arm64, but the arm64 half is built under QEMU on an amd64 runner and is never executed. Two problems fall out of that:

  • Wall clock. uv sync compiles pyopenjtalk's bundled OpenJTalk/HTS-engine C++ from sdist (it ships source-only on PyPI, no wheels for any arch — see Dockerfile:9-13). That is pure CPU-bound work, ~an order of magnitude slower under emulation: 9m42s (amd64 only) → 21m50s (both, cold cache), so ~+12 min.
  • Verification. buildx cannot load a multi-platform result, so the smoke test only ever ran the amd64 image. The arm64 image that actually runs on the Ampere deploy host shipped unverified.

方法/實作說明

Split CD into a per-arch build matrix on native runners plus a merge job. ubuntu-24.04-arm is free for public repos, so QEMU is gone entirely — and because each arch now has a single-platform image, each arch can load and smoke-test its own image natively.

  • 主要修改:
    • .github/workflows/cd.yml
  • 關鍵實作:
    • build matrix: ubuntu-latest/linux/amd64 and ubuntu-24.04-arm/linux/arm64. Each builds with load: true, runs the existing offline smoke test (--network none, /openapi.json + /api/MarkAccent/) on its own native runner, then re-exports from cache and pushes untagged, by digest (push-by-digest=true,name-canonical=true). fail-fast: false so both arches always report.
    • Per-arch cache scopes (type=gha,scope=cd-<arch>): the two matrix jobs run concurrently and would otherwise race writing the same scope.
    • merge job: docker buildx imagetools create stitches the per-arch digests (passed between jobs as artifacts) into the tagged OCI index. Tags are applied here and nowhere else, so dev/stable only move after both smoke tests pass — a broken arm64 image can never take a tag. imagetools copies the provenance/SBOM attestation manifests attached to each single-platform index into the merged one, so attestations survive.
    • Release serialisation moved to merge: only tagging is order-sensitive (an older run must not drag stable backwards), so the builds no longer block on an earlier run. It also now runs once per run instead of once per arch.
    • REGISTRY/IMAGE_NAME hoisted to workflow-level env since two jobs need them; docker/setup-qemu-action dropped.

關聯 Issue

Closes #67. Closes #68.

附註

  • Verified with actionlint 1.7.12 (clean); the merge job's imagetools create argument construction was dry-run locally.
  • The cache scope changed from the default to cd-<arch>, so the first run on main after this merges is a cold cache for both arches. They build in parallel natively, so that is roughly one native cold build of wall clock, not two.
  • The Inspect published index step at the end prints the merged manifest list — that is where to confirm both platforms and the two attestation entries are present on the first run.

#64 added linux/arm64 to the CD image build, but the arm64 half ran under
QEMU on an amd64 runner. pyopenjtalk ships source-only on PyPI and compiles
its bundled OpenJTalk/HTS-engine C++ during `uv sync`, so that CPU-bound work
cost roughly +12 min per cold run (9m42s -> 21m50s). The arm64 image was also
pushed without ever being run, since buildx cannot `load` a multi-platform
result for the smoke test.

Split the build into a matrix over ubuntu-latest/linux-amd64 and
ubuntu-24.04-arm/linux-arm64 (native arm64 runners are free for public
repos), each with its own cache scope. Every arch now builds, loads and
smoke-tests its own single-platform image on a runner of its own
architecture, then pushes untagged by digest. A merge job stitches the
per-arch digests into the tagged index with `imagetools create`, so dev and
stable are only moved once both smoke tests pass and a broken arm64 image
can never take a tag. imagetools copies the provenance/SBOM attestation
manifests attached to each single-platform index into the merged one.

The release serialisation wait moves to the merge job: only tagging is
order-sensitive, so the builds no longer block on an earlier run.

Closes #67
Closes #68

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

Copy link
Copy Markdown

🛡️ PR Quality Check Summary

✅ PR Title: Passed (Length: 52/75, Format: OK). ci(cd): build each arch natively and smoke-test both
✅ Branch Name: Follows naming convention (ci/native-arm64-build)
✅ Commit Messages: All 1 commit(s) passed (Length, Format, Case)
✅ Conflicts: No merge conflict markers found
✅ Python Quality: All checks passed.


🎉 All checks passed!

@wade00754 wade00754 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

審查結論:Approve,未發現需要阻擋合併的問題。

已檢視 43760ce 的完整 workflow 變更,確認各架構使用原生 runner、獨立 cache scope,並在各自離線 smoke test 成功後才上傳 digest;merge 依賴整個 build matrix 成功,再統一套用發布 tags。digest artifact 的上下載與 imagetools 參數組裝也一致。

驗證:GitHub 上的 Lint/type-check/pytest 與 Docker Build Check 均通過;本次以靜態審查及既有 CI 結果為依據,沒有另外觸發發布。PR 的 Docker 檢查仍只涵蓋 amd64,新加入的原生 arm64 與 manifest 發布流程需在首次 CD 執行時確認。此項不阻擋合併。

@torridfish
torridfish added this pull request to the merge queue Sep 15, 2026
Merged via the queue into main with commit 199d464 Sep 15, 2026
36 checks passed
@torridfish
torridfish deleted the ci/native-arm64-build branch September 15, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CD publishes the arm64 image without ever smoke-testing it CD: build arm64 natively instead of under QEMU (~12 min per cold run)

2 participants