Skip to content

ci: drop Docker, run tests directly on the runner - #127

Merged
tjduigna merged 4 commits into
release-date-prefixfrom
drop-docker
Oct 1, 2026
Merged

tjduigna merged 4 commits into
release-date-prefixfrom
drop-docker

Conversation

@tjduigna

@tjduigna tjduigna commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Removes the Docker build/test machinery and runs CI directly on the GitHub runner.

Why

PRs into major_refactor didn't rebuild the base image (their dockerfiles/base/* matches the target), so CI tested them on the stale published plinder-base:v0.2.27. That image carries a 2024 lockfile: numpy 1.26, biotite 1.0, cloudpathlib 0.19, an old DuckDB, gemmi/mmcif/networkx, and no peppr/voronotalt. Example: #126 failed 50 tests + 16 errors that pass on #118. The lockfile also contradicted the refactor's own pyproject.toml.

Changes

  • Removed: dockerfiles/ (base/main/test Dockerfiles, env.yml, the pinned requirements.txt), docker-compose.yml, flows/docker.py, flows/proc.py (only used by docker.py).
  • PR test job:
    • micromamba from environment.yml provides the conda-only tools (openstructure, mmseqs2, foldseek).
    • Then pip install -r requirements_data.txt and pip install -e ".[data,test,eval,plots]".
    • Then PLINDER_OFFLINE=true python -m pytest -n auto -v (same command the Docker test service ran), and the coverage comment.
    • No more changed-files gating; the job always runs.
  • main Package workflow: same environment setup, then version bump → tests → git tag → python -m build → PyPI publish → push tag → coverage comment. The tag is still created before the build, so setuptools_scm stamps the released version as before. The image build/promote/push steps are gone.
  • version.py (repo root): the bumpversion major|minor|skip logic moved out of flows/docker.py unchanged. It returns v0.2.28 on the current history, identical to the old implementation.
  • pyproject.toml: duckdb>=1.5. The refactor calls DuckDBPyConnection.to_arrow_table(), which doesn't exist before 1.5 (verified 1.1.3 → 1.4.0 lack it; 1.5.2 and 1.5.6 have it). Without the floor, pip keeps an older DuckDB if one is present, which is what broke the stale image.
  • environment.yml: drop reduce (unused).
  • tox -e py312-test: runs pytest directly with the same extras as CI.
  • Docs: removed the Docker sections from contribution/development.md and contribution/release.md.

Notes

  • CI now tests the dependency resolution users actually get from pip install, rather than a separate lockfile.
  • No more images are published to ghcr.io/plinder-org/plinder. Existing tags stay. flows/data_ingest_report.py still names plinder:v0.1.1 for its Metaflow step; untouched.
  • The main workflow can only be fully exercised on merge to main. The version-bump script was checked locally against the old implementation.

Follow-up: uv instead of pip, no requirements_data.txt

  • All dependencies come from pyproject.toml. The dev dependency group is the complete CI environment: plinder[dev,eval,loader,plots], the git-only pdb-validation and mmpdb, networkit (a wheel is required via [tool.uv] no-build-package), tabulate and semver.
  • PyTorch stays its own loader extra, mapped to the CPU-only index on Linux.
  • CI: micromamba installs only the conda-only tools (openstructure, mmseqs2, foldseek), then uv sync --inexact installs everything else into that env. The release workflow uses uv build.
  • No committed uv.lock (gitignored): CI resolves the latest versions allowed by pyproject.toml, which is what pip users get.
  • pandas 3 support: patched alignment shards keep their original large_string schema, and the query test checks missing values with isna().

- restore the original physical schema when DuckDB rewrites a patched
  alignment shard (DuckDB writes string; pandas 3 writes large_string)
- assert missing string values with isna() so the query test holds for
  pandas 2 (None) and pandas 3 (NaN)
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/plinder/data/pipeline
  tasks.py 2603
Project Total  

This report was generated by python-coverage-comment-action

@tjduigna
tjduigna changed the base branch from major_refactor to release-date-prefix October 1, 2026 12:12
…nts_data.txt

- dev dependency group is the complete CI/dev environment (adds plots
  extra and semver); networkit must come from a wheel via no-build-package
- commit uv.lock and install with uv sync --locked into the Conda env
  that provides the conda-only tools (openstructure, mmseqs2, foldseek)
- build release distributions with uv build
- pin the project Python to 3.12 with .python-version
@tjduigna
tjduigna merged commit 14bde64 into release-date-prefix Oct 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant