Build Elixir and Phoenix apps without a Dockerfile - #1271
Conversation
aptInstall and the npm augmentation mounted /var/lib/apt/lists and /var/cache/apt/archives as shared BuildKit caches. apt takes an exclusive lock on both, so two stages running apt at the same time fail with "Could not get lock /var/lib/apt/lists/lock". No stack hit it until now because none ran apt in two stages at once; the Elixir stack installs build tools on its builder while its runtime image installs ERTS's shared libraries, and BuildKit runs those in parallel. Locked mounts make BuildKit hand the cache to one step at a time, which is what apt needs anyway.
![miren-code-agent[bot]](https://avatars.githubusercontent.com/u/174767976?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
478c4e8 to
65a1dd2
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
b0f49e1 to
d93d4c5
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Elixir teams evaluating Miren hit a wall at the build step: the runtime already handles WebSockets and BEAM distribution, but a Phoenix app needed a hand-written Dockerfile.miren. Now a mix.exs is enough. The stack builds a Mix release on hexpm/elixir and ships only the release, which bundles ERTS, on debian-slim, the same build-fat, run-slim shape as the Go stack. Deps are fetched and compiled from mix.exs, mix.lock and config/ alone so that layer survives code changes. mix compile runs before mix assets.deploy because Phoenix 1.8 generates colocated LiveView hooks at compile time and esbuild needs them. Phoenix apps get PHX_SERVER=true, and an assets/package.json gets its own npm install, since the root-level npm augmentation can't see it. hexpm tags pair an Elixir release with an OTP release and a dated Debian rebuild, so there is no floating "1.18" tag to point at. A table in imagerefs maps Elixir minors (1.16 to 1.20) and their OTP majors onto tags known to exist in one bookworm rebuild, which matches debian-slim's glibc and OpenSSL. build.version takes the forms version managers use (1.18, 1.18.4, 1.18.4-otp-27); without it, the stack reads the app's pin from .tool-versions or mise.toml, and otherwise builds Elixir 1.19 on OTP 28. A full bookworm hexpm tag still works as the escape hatch. The table lags rather than breaks when it goes stale, since hexpm keeps old tags; MIR-1960 covers refreshing it, along with every stack's defaults, on a schedule. The release name comes from a releases: block when there is one, else the OTP app name. Umbrellas need the releases: block, and without one the build fails with a message saying so rather than a Mix error. Phoenix is detected from mix.lock as well as the root mix.exs, since an umbrella declares it in its web app. Building an umbrella's assets is left to [build] onbuild for now (MIR-1959). Env detection follows the Rails pattern: Phoenix gets a generated SECRET_KEY_BASE and a PHX_HOST recommendation (LiveView's websocket origin check fails without it), postgrex and friends imply DATABASE_URL, and the code scan treats System.fetch_env! and `System.get_env(...) || raise` as required and plain get_env as optional. The scan skips comments (phx.new's runtime.exs is full of commented-out examples), deps/, _build/, and vars read only from dev/test config. The per-language "Want native support?" callouts in the guides now link to the index's table of auto-detected stacks instead of repeating the list, so adding a stack no longer means editing every guide. The base image pulls through oci.miren.cloud as hexpm/elixir, under its real name: a bare elixir would suggest Docker Hub's official image, which has different tags. The registry proxy's allowlist needs to include it before this ships. Fixes MIR-1404
d93d4c5 to
80434d9
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
evanphx
left a comment
evanphx
left a comment
There was a problem hiding this comment.
There are a couple changes to make using Phoenix even more smooth out of the box. I'm going to take the pen and add them because I was actually working on this last night!
There was a problem hiding this comment.
🍪 biscuit: ✅ ready to merge — auto-review, non-blocking
I think this is still ready. The new commits add DNSCluster clustering, PHX_HOST handling and more accurate env-var detection, and nothing in them should block.
DNSCluster. I checked the new start command against the runtime:
- Sandbox commands run through
/bin/sh -c(controllers/sandbox/sandbox.go), so${RELEASE_NODE:-…@$(hostname -i)}gets expanded. web.app.mirenreturns one A record per sandbox in the same app (handleAppMirenQueryinpkg/dns/dns.go), so DNSCluster can find each replica.setupHostsmaps the sandbox hostname to the sandbox IP in/etc/hosts. Sohostname -igives the same address the peers resolve, and each node'srelease@IPname matches what the others connect to.
Clustering only turns on when the lock has dns_cluster and the code reads DNS_CLUSTER_QUERY. That keeps it away from apps with their own discovery setup. The test that replaces the get_env call with nil shows the fallback to a plain start.
Env detection. The elixirCode anchor stops phx.new's commented-out System.get_env examples from being reported. The optional/required split handles the forms phx.new actually uses:
|| raisecounts as required, including when the raise is on the next line.|| "default",get_env/2andin ~w(...)count as optional.
Reads that only appear in dev/test config are dropped. deps/ and _build/ are no longer scanned; that change applies to every stack, but I can't see it hurting any of them. PHX_HOST shows up in miren init under "must be configured manually" and doesn't block, which matches the guide.
Still open:
- Hexpm tags: only the default tag from the version table has been pulled. It's worth running
docker manifest inspectonce over every entry. - No live deploy yet: no one has run a real
mix phx.new→ deploy. That matters a bit more now, because the default Phoenix app starts through a shell$(hostname -i)expansion that no test runs;TestElixirDetectonly compares the command string. One deploy with two replicas would check the node naming, the cluster join and the endpoint startup in one go.
You've accepted the releases: releases() limitation, so I'm leaving that out.
evanphx's changes-requested review is still on the PR. He left it before pushing his commits, and it's his to clear once he's happy with what landed.
🍪 full review note · reviewed at fa31454 · comment /biscuit review to run biscuit again.
2 participants
Elixir teams evaluating Miren kept hitting the same wall at the build step. The runtime side already worked (WebSockets, BEAM distribution), but getting a Phoenix app onto Miren meant copying a multi-stage Dockerfile out of the docs. Now
miren init && miren deployon amix.exsdoes it.The stack builds a Mix release on hexpm/elixir and ships just the release on debian-slim, the same build-fat, run-slim shape the Go stack uses. Phoenix apps get their assets built (including npm deps under
assets/),PHX_SERVER=true, a generatedSECRET_KEY_BASE, and aPHX_HOSTnudge, since LiveView's websocket origin check fails without it. Versions work like other stacks:build.versiontakes1.18or1.18-otp-27(mapped onto hexpm's exact tags, since hexpm has no floating ones), apps that pin Elixir in.tool-versionsormise.tomlget that version automatically, and the default is Elixir 1.19 / OTP 28.The first rev is a fix the Elixir stack flushed out: apt's cache mounts were shared, so two stages running apt in parallel (builder and runtime image, here) failed on each other's lock. They're locked now.
Tested with real BuildKit builds of a plain Mix app and one with npm deps, and end to end in the dev env with two apps: a stock
mix phx.newapp (Postgres addon, digested assets, a database query through the release) and a small LiveView app that runs migrations at boot and needs required secrets (LiveView's websocket connects through the router, and a redeploy reused the cached deps layers).The builder pulls through oci.miren.cloud as
hexpm/elixir, under its real name, since Docker Hub's bareelixiris a different image. That name is now allowlisted on the registry proxy, so builds work as soon as this ships.Closes MIR-1404