From f7ec953c135f82699cf3fa218e25915b5a10ff17 Mon Sep 17 00:00:00 2001 From: Bryan Gin-ge Chen Date: Wed, 21 Oct 2026 00:55:23 -0400 Subject: [PATCH] perf(docker): install the dev image's dependencies before the code The dev Dockerfile copied the code in before `pip install -e .`, so any code change reinstalled every dependency, about 190 MB a time. Compose checks rebuild the image on every run. Its base, python:3.12-slim, was unpinned, so each new release of it also reran the apt-get layer, build-essential and libpq-dev, another 436 MB. Over months these copies filled Docker's build cache. The image now follows Dockerfile.production: the same pinned base image and uv, the packages uv.lock pins installed into /opt/venv (outside /app, which Compose mounts over) from pyproject.toml and uv.lock alone, then the code, then the project itself, editable, as before. No apt-get: psycopg is psycopg[binary], and every locked package has a wheel. Dev containers now get uv.lock's versions rather than the newest that pyproject.toml allows, the same as production. A dependency change needs `uv lock` before the image builds (--locked), which AGENTS.md now says. docs/sandboxed_docker_setup.md lists the images it pulls. Tested: a code change rebuilds in about 2 s, with the dependency layer cached. scripts/repo_check_compose.sh passes with the new image (1,843 tests), skipping only step 2, the GraphQL validation on the host. python, celery and the editable queueboard package resolve from /opt/venv, and psycopg uses its binary implementation. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 2 ++ Dockerfile | 34 ++++++++++++++++++++++------------ docs/sandboxed_docker_setup.md | 4 ++-- 3 files changed, 26 insertions(+), 14 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ff359686..06b830b4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -103,6 +103,8 @@ Notes ## Containers & Volumes - Code is bind-mounted read-only into containers (`.:/app:ro`) to avoid writes into the repo from inside Docker. +- The dev `Dockerfile` installs the packages `uv.lock` pins into `/opt/venv` before it copies the code, so a code + change doesn't reinstall them. A dependency change in `pyproject.toml` needs `uv lock` before the image builds. - Runtime artifacts (Django `STATIC_ROOT`, `MEDIA_ROOT`, Celery beat schedule) write under `/data` backed by the `appdata` named volume. - Generate migrations on the host (e.g., `uv run python qb_site/manage.py makemigrations`) and commit them, rather than creating files from inside containers. - `Dockerfile.production` is the production image, separate from the dev `Dockerfile`/Compose setup: packages from diff --git a/Dockerfile b/Dockerfile index be2808b5..939430ee 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,28 +1,38 @@ # syntax=docker/dockerfile:1 -FROM python:3.12-slim +# Development image, for docker-compose.yml, which mounts the repo over /app. The production image +# is Dockerfile.production; this one starts from the same base image and the same uv.lock. + +FROM python:3.12.14-slim-trixie@sha256:f77ac9e44ae96ef2c90b8053ea08c31f8be030f824196b0ae4db6d462c84e51f + +COPY --from=ghcr.io/astral-sh/uv:0.12.19@sha256:04d046b13e60d6bcec73cbc5e1cad25d680dea90c8573340950a0ac2d1aef424 /uv /bin/uv ENV PYTHONDONTWRITEBYTECODE=1 \ PYTHONUNBUFFERED=1 \ - PIP_NO_CACHE_DIR=1 \ - PIP_DISABLE_PIP_VERSION_CHECK=1 \ - PYTHONPATH=/app + PYTHONPATH=/app \ + PATH=/opt/venv/bin:$PATH \ + UV_LINK_MODE=copy \ + UV_PYTHON_DOWNLOADS=never \ + UV_PROJECT_ENVIRONMENT=/opt/venv WORKDIR /app -RUN apt-get update \ - && apt-get install -y --no-install-recommends \ - build-essential \ - libpq-dev \ - && rm -rf /var/lib/apt/lists/* +# Dependencies first, so that a change to the code doesn't reinstall them. The environment is +# outside /app, which Compose mounts over. --locked fails the build if uv.lock is out of date with +# pyproject.toml. +RUN --mount=type=cache,target=/root/.cache/uv \ + --mount=type=bind,source=pyproject.toml,target=pyproject.toml \ + --mount=type=bind,source=uv.lock,target=uv.lock \ + uv sync --locked --no-dev --no-install-project -COPY pyproject.toml README.md uv.lock* ./ +COPY pyproject.toml README.md uv.lock ./ COPY src ./src COPY qb_site ./qb_site COPY scripts ./scripts -RUN pip install --upgrade pip && \ - pip install --no-cache-dir -e . +# The project itself, editable, which puts src/ on the path. +RUN --mount=type=cache,target=/root/.cache/uv \ + uv sync --locked --no-dev RUN chmod +x scripts/docker-entrypoint.sh diff --git a/docs/sandboxed_docker_setup.md b/docs/sandboxed_docker_setup.md index f7bee2dd..58be4fc8 100644 --- a/docs/sandboxed_docker_setup.md +++ b/docs/sandboxed_docker_setup.md @@ -50,8 +50,8 @@ which it treats as fatal. It dies before doing any credential work, and BuildKit generic "error getting credentials". The helper runs only because `~/.docker/config.json` sets `"credsStore": "desktop"`. Every image -this stack pulls is public — `python:3.12-slim`, `postgres:16-alpine`, `redis:7-alpine` — so -nothing here needs a credential helper. +this stack pulls is public — `python:3.12.14-slim-trixie`, `ghcr.io/astral-sh/uv`, +`postgres:16-alpine`, `redis:7-alpine` — so nothing here needs a credential helper. **Read-only access to those paths does not help.** The failure is a write.