CD: build arm64 natively instead of under QEMU (~12 min per cold run)

[torridfish]

Follow-up to #64, which added linux/arm64 to the CD image build so images run natively on the arm64 (Oracle Ampere) deploy host. It works, but the arm64 half is built under QEMU emulation on an amd64 runner, which is slow enough to be worth fixing.

Measured cost

Run	Platforms	Duration
34763820630 (pre-#64)	amd64	9m42s
34849446439 (post-#64, cold arm64 cache)	amd64 + arm64	21m50s

So the arm64 half costs roughly +12 min on a cold cache.

Root cause

It is not the UniDic download. That layer (Dockerfile:43, ~700 MB) is network-bound, so emulation barely affects it.

The dominant cost is uv sync at Dockerfile:24. As the comment at Dockerfile:9-13 notes, pyopenjtalk ships source-only on PyPI with no wheels for any arch, so it compiles its bundled OpenJTalk/HTS-engine C++ from sdist during the build. That is pure CPU-bound C++ compilation, and under QEMU it runs roughly an order of magnitude slower than native.

Why this is not urgent

Layer caching is already correct — every expensive step sits before COPY . . (Dockerfile:45) and is gated by uv.lock / download_unidic.sh, so an app-code-only commit reuses all of it. GHA cache usage is currently 3.64 GB / 10 GB across 41 entries, so LRU eviction is not yet a factor either.

The thing that will actually bite: GitHub evicts caches not accessed for 7 days. CD only runs on pushes to main, so any gap longer than a week means the next run pays the full cold-cache emulation cost again.

Proposal

This repo is public, so GitHub's native arm64 runners (ubuntu-24.04-arm) are free. Build each arch natively and merge the manifest, eliminating QEMU entirely.

Sketch:

1. Matrix job over {ubuntu-latest → linux/amd64, ubuntu-24.04-arm → linux/arm64}; each builds and pushes by digest (outputs: type=image,push-by-digest=true,name-canonical=true) with its own cache scope.
2. A merge job runs docker buildx imagetools create over the per-arch digests to produce the tagged multi-arch index, applying the docker/metadata-action tags.

Complication worth planning for

The current smoke test depends on the single-platform shape: it builds amd64 with load: true (buildx cannot load a multi-platform result), runs the container on the runner, then pushes both arches in a second build that reuses the cache. A matrix split changes that flow — but it also improves it, since each arch can then smoke-test its own image natively instead of only amd64 being exercised. The merge job should only publish tags once both arch smoke tests pass, so a broken arm64 image can never take the dev/stable tag.

Also note provenance: true / sbom: true currently attach attestations (the two unknown/unknown entries in the index); the merge step needs to preserve these.

Not blocking

#64 is deployed and correct — ghcr.io/sessatakuma/api-tools:dev is now an OCI image index with both linux/amd64 and linux/arm64. This issue is purely a CI wall-clock optimization.

[torridfish]

Absorbing #68: arm64 must also be smoke-tested (closing #68 as a duplicate of this issue)

#68 reported that CD publishes the arm64 image without ever running it. That is the same work item as this issue — #68's preferred fix ("native ubuntu-24.04-arm runner") is this issue's proposal, and the "Complication worth planning for" section above is exactly #68's problem. Folding it in here so one PR closes both.

The verification gap (from #68)

Since #64, CD publishes a linux/amd64,linux/arm64 manifest, but .github/workflows/cd.yml only ever executes the amd64 half:

• Build amd64 image for smoke test (L80-95) builds linux/amd64 with load: true — buildx cannot load a multi-platform result into the runner daemon.
• Smoke-test production lifecycle offline runs that amd64 image.
• Push smoke-tested image (amd64 + arm64) (L117-126) builds the arm64 half under QEMU and pushes it without ever running it.

So the image that actually runs on the OCI Ampere deploy host ships unverified. An arch-specific wheel resolving differently, or OpenJTalk / fugashi misbehaving on aarch64, goes undetected until deploy.

Why the native-runner plan fixes both at once

The matrix split proposed above is not just a wall-clock optimisation — it is also what makes per-arch smoke testing possible, since each arch then has a single-platform image it can load and run natively on its own runner. QEMU-based docker run --platform linux/arm64 (#68's option 2) would need the wait loops widened a lot and makes CD slower, so it is strictly worse on both axes.

Acceptance criteria

• Each arch builds natively on its own runner (ubuntu-latest / ubuntu-24.04-arm), no QEMU.
• Each arch runs the offline smoke test on its own native runner — arm64 included.
• The merge job publishes the dev / stable tags only after both arch smoke tests pass, so a broken arm64 image can never take a tag.
• provenance / sbom attestations survive the imagetools create merge.