Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ jobs:
echo "PCAPKIT_PYPI_COMPLETE=$complete" >> "$GITHUB_OUTPUT"

# Anaconda evidence, same shape as PyPI's above but for the ``conda``
# matrix (2 OS x the supported tier read above). Unlike PyPI's, this
# matrix (linux-64 and osx-64, each Python read above). Unlike PyPI's, this
# endpoint reported ``cf-cache-status: DYNAMIC`` on every request checked
# while writing this, i.e. not cached -- so there is no staleness direction to
# reason about here. This is the job-level cost saver mirroring ``pypi``'s;
Expand All @@ -488,7 +488,9 @@ jobs:
run: |
set -euo pipefail

# Each of ``conda``'s two platforms, for each supported version.
# linux-64 and osx-64, for each supported version. osx-arm64 is
# published too and not counted, as it was not when ``macos-latest``
# uploaded it beside the osx-64 package it converted (#1822).
read -r -a builds <<< "$PCAPKIT_CONDA_BUILDS"
expected=$(( 2 * ${#builds[@]} ))
# Same coupling risk, and the same fix, as the PyPI check above.
Expand Down Expand Up @@ -1216,8 +1218,15 @@ jobs:
strategy:
fail-fast: false
matrix:
# #1822: each leg builds and publishes its own runner's platform only,
# since ``conda convert`` copies binaries unchanged. ``macos-latest`` is
# arm64, so osx-arm64, and ``macos-26-intel`` is Intel, so osx-64, per
# https://docs.github.com/en/actions/reference/runners/github-hosted-runners
# (GA in actions/runner-images#13739). The osx-64 leg ends with GitHub's
# Intel images, which actions/runner-images#13045 says will end.
os:
- macos-latest
- macos-26-intel
- ubuntu-latest
# - windows-latest # skip for now
# #1633: the supported tier only, from .github/python-versions.toml
Expand Down Expand Up @@ -1267,6 +1276,9 @@ jobs:

case "${{ matrix.os }}" in
macos-latest)
target_platform="osx-arm64"
;;
macos-26-intel)
target_platform="osx-64"
;;
ubuntu-latest)
Expand Down Expand Up @@ -1407,6 +1419,9 @@ jobs:

case "${{ matrix.os }}" in
macos-latest)
target_platform="osx-arm64"
;;
macos-26-intel)
target_platform="osx-64"
;;
ubuntu-latest)
Expand All @@ -1420,9 +1435,11 @@ jobs:
;;
esac

# #1822: never ``conda convert`` across architectures; see the matrix.
host_platform="$(basename "$(dirname "$host_package")")"
if [ -n "$target_platform" ] && [ "$target_platform" != "$host_platform" ]; then
conda convert "$host_package" -o "$out_dir" -p "$target_platform"
if [ "$host_platform" != "$target_platform" ]; then
echo "::error title=${host_platform} package on a ${target_platform:-unmapped} leg (#1822)::${{ matrix.os }} publishes ${target_platform:-nothing}, and conda built ${host_platform} here; a package is published only for the platform it was built on."
exit 1
fi

package_paths="$(find "$out_dir" -type f \( -name '*.tar.bz2' -o -name '*.conda' \) -print | sort)"
Expand Down
22 changes: 17 additions & 5 deletions .github/workflows/cron-conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -424,8 +424,11 @@ jobs:
strategy:
fail-fast: false
matrix:
# #1822: each leg builds and publishes its own runner's platform only,
# as create-release.yml's `conda` job says.
os:
- macos-latest
- macos-26-intel
- ubuntu-latest
# - windows-latest # skip for now
# #1633: the supported tier, from .github/python-versions.toml through
Expand Down Expand Up @@ -512,6 +515,9 @@ jobs:

case "${{ matrix.os }}" in
macos-latest)
target_platform="osx-arm64"
;;
macos-26-intel)
target_platform="osx-64"
;;
ubuntu-latest)
Expand All @@ -527,9 +533,13 @@ jobs:

echo "host_package=$host_package" >> "$GITHUB_OUTPUT"

# #1822: never `conda convert` across architectures. It relabels the
# package and copies its binaries unchanged, so the osx-64 package it
# made from the arm64 build carried arm64 extensions.
host_platform="$(basename "$(dirname "$host_package")")"
if [ -n "$target_platform" ] && [ "$target_platform" != "$host_platform" ]; then
conda convert "$host_package" -o "$out_dir" -p "$target_platform"
if [ "$host_platform" != "$target_platform" ]; then
echo "::error title=${host_platform} package on a ${target_platform:-unmapped} leg (#1822)::${{ matrix.os }} publishes ${target_platform:-nothing}, and conda built ${host_platform} here; a package is published only for the platform it was built on."
exit 1
fi

package_paths="$(find "$out_dir" -type f \( -name '*.tar.bz2' -o -name '*.conda' \) -print | sort)"
Expand All @@ -552,9 +562,11 @@ jobs:
# modules alone. This unpacks the package built for this runner and
# checks that every vendored distribution has every file its RECORD lists
# and imports from there, with no site-packages on the path, before
# anything is uploaded. setup-python's interpreter, not conda's: it is
# the matrix version the package was built for, which conda's base is
# not on 3.15.
# anything is uploaded. #1822: it also fails unless the package, and every
# native binary in it, is for this runner's platform, so a relabelled
# package cannot pass on the runner that built it. setup-python's
# interpreter, not conda's: it is the matrix version the package was
# built for, which conda's base is not on 3.15.
- name: Smoke-test the vendored packages
env:
PCAPKIT_CONDA_PACKAGE: ${{ steps.conda_build.outputs.host_package }}
Expand Down
48 changes: 25 additions & 23 deletions docs/source/contributing/releasing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,12 @@ Editing ``__version__`` by hand without also moving :file:`CITATION.cff`'s
``tests/project/test_bump_version.py``'s
``RepositoryCitationTests.test_the_citation_file_names_the_packaged_version``
(``:494-514``) asserts the two agree, and ``create-release.yml``'s
``unit-tests`` job (``:579-595``) calls ``unit-tests.yml`` with
``unit-tests`` job (``:581-597``) calls ``unit-tests.yml`` with
``gate-only: true``, which runs the **full** suite rather than the tiered
subset an ordinary push runs (its ``gate`` job, ``unit-tests.yml:1776-1846``).
That gate runs whenever a release will -- a moved ``__version__`` is exactly
what makes ``version_check``'s evidence read "not yet published" -- and every
publishing job requires it to have succeeded (``:619,806,972,1208``), so a
publishing job requires it to have succeeded (``:621,808,974,1210``), so a
citation left behind fails the gate well before any approval is requested.
Either run the script, or move both fields by hand in the same commit.

Expand All @@ -66,7 +66,7 @@ process: the ``changelog`` job (``unit-tests.yml:1624-1643``) fails outright if
``CHANGELOG.md`` has drifted from its source entry under
:file:`docs/source/changelog/`, and the release-body step warns, without
failing, if that entry's heading still reads "unreleased"
(``create-release.yml:675-676``). Both enforce what the changelog convention
(``create-release.yml:677-678``). Both enforce what the changelog convention
already asks for.

Two Ways to Start a Release
Expand All @@ -83,8 +83,8 @@ neither is a workaround for the other:
the commit the pushed tag points at, and reads ``pcapkit.__version__`` out
of *that* tree (``:327``). Every downstream tag and release name is built
from that output -- ``v${{ needs.version_check.outputs.PCAPKIT_VERSION }}``
is ``github``'s ``tag_name`` at ``:717`` and the release ``pypi`` and
``conda`` upload to at ``:1115`` and ``:1462``, always from
is ``github``'s ``tag_name`` at ``:719`` and the release ``pypi`` and
``conda`` upload to at ``:1117`` and ``:1479``, always from
``PCAPKIT_VERSION``, never from ``github.ref_name`` -- so the version
actually released is whatever ``__version__`` says on the tagged commit,
not the name of the tag that was pushed. The two agree only if the tag
Expand Down Expand Up @@ -144,26 +144,26 @@ days later, and tagged ``main``'s tip ``2c0008c3c`` as both ``v1.5.0b9`` and
the default branch, and ``tag`` pulled ``main`` before tagging.

* On ``workflow_run``, where ``github`` creates the ``v*`` tag, it passes
``target_commitish`` ``github.sha`` (``:730``); on a ``v*`` push the tag
``target_commitish`` ``github.sha`` (``:732``); on a ``v*`` push the tag
already exists and it passes none. Either way it then checks that the tag
names that commit.
* ``tag`` cuts ``conda-<version>+0`` at that commit (``:844-871``) before it
* ``tag`` cuts ``conda-<version>+0`` at that commit (``:846-873``) before it
reads ``main`` at all; the :file:`conda/build` reset is a separate commit on
top of ``main`` (``:893-946``), described under `Precautions`_.
top of ``main`` (``:895-948``), described under `Precautions`_.
* ``pypi`` and ``conda`` check out that commit and, before building, check
that every tag they publish under names it, as ``tag`` does for the ``v*``
tag. A run created for a *later* commit of the same version therefore never
publishes a second tree under that version. ``version_check`` resolves the
``v*`` tag the same way first (``:534-552``), and off a ``v*`` ref a tag
``v*`` tag the same way first (``:536-554``), and off a ``v*`` ref a tag
naming another commit skips the gate and all four publishers
(`#1748 <https://github.com/JarryShaw/PyPCAPKit/issues/1748>`__):
``Release status`` posts a ``::notice`` if that release is complete, or a
``::warning`` naming both commits and what is missing (``:1607-1617``), and
``::warning`` naming both commits and what is missing (``:1624-1634``), and
the run stays green. A ``v*`` push of such a version still fails with the
check's ``::error``. A partial release is finished by re-running the run that
tagged it.
* ``pypi`` and ``conda`` attach their files with ``gh release upload``
(``:1108-1145``, ``:1457-1492``), not ``action-gh-release``, which calls Update
(``:1110-1147``, ``:1474-1509``), not ``action-gh-release``, which calls Update
a release before every upload (3.0.2's ``src/github.ts:631-683``). A file
the release already has is left alone
(`#1640 <https://github.com/JarryShaw/PyPCAPKit/issues/1640>`__): the action
Expand Down Expand Up @@ -257,11 +257,11 @@ The owner's choice on `#1634
the release with a GitHub App installation token, which Create a release
accepts for a target that changes a workflow file when it has the "Workflows"
repository permission (write). "Mint the release App's token"
(``create-release.yml:690-698``) runs
(``create-release.yml:692-700``) runs
``actions/create-github-app-token@v3`` with ``permission-contents: write`` and
``permission-workflows: write`` and no ``owner`` or ``repositories``, so the
token is "scoped to only the current repository" and revoked in the action's
``post`` step (v3.3.0's README). Only Create Release spends it (``:734``).
``post`` step (v3.3.0's README). Only Create Release spends it (``:736``).
While ``vars.RELEASE_APP_CLIENT_ID`` is unset, which reads as "an empty
string", the step skips and Create Release spends ``GITHUB_TOKEN``, so the
workflow keeps working, refusals included, until the App exists. A PAT would
Expand Down Expand Up @@ -291,13 +291,13 @@ the release test gate, ask for a second approval and race the first run's
uploads from another concurrency group, so ``version_check`` skips a ``push``
whose ``github.actor_id`` is ``vars.RELEASE_APP_BOT_ID`` (``:290-294``), and
``release_status`` reports it with ``::notice title=Nothing to release``
(``:1569-1571``). A tag pushed by a person runs as before. Whether a tag made
(``:1586-1588``). A tag pushed by a person runs as before. Whether a tag made
through the Releases API fires ``push``, and with the bot as actor, is
unmeasured; if not, the guard never matches and the cost is one approval to
reject.

A refused Create Release (404, or 403 "Resource not accessible by
integration") is explained by "Explain a refused release tag" (``:767-779``)
integration") is explained by "Explain a refused release tag" (``:769-781``)
according to the token spent: with ``GITHUB_TOKEN`` it lists the setup above;
with the App's, it points at the App's permissions and installation. For a
final release, first delete any draft ``v<version>`` the refused run left:
Expand Down Expand Up @@ -336,9 +336,11 @@ checks *is* the one it is about to create:
KTLO legs' wheels (`#1633 <https://github.com/JarryShaw/PyPCAPKit/issues/1633>`__) are not counted. This job is
already idempotent (``skip-existing: true`` on both upload steps), so the
check exists only to save the cost of the build matrix, not for correctness.
* ``conda`` compares Anaconda's aggregate count against the expected one (a
file per OS and Python version of its matrix) as the same kind of job-level
cost saver, and *also*
* ``conda`` compares Anaconda's aggregate count against the expected one (the
``linux-64`` and ``osx-64`` files for each Python version of its matrix; the
``osx-arm64`` leg's are published but not counted,
`#1822 <https://github.com/JarryShaw/PyPCAPKit/issues/1822>`__) as the same
kind of job-level cost saver, and *also*
checks, per matrix leg, whether that leg's own platform/Python distribution
is already there -- this second check is the actual correctness fix, because
``anaconda/actions/upload-package`` has no ``skip-existing`` equivalent, only
Expand Down Expand Up @@ -423,7 +425,7 @@ The Release Pipeline
GH["github -- GitHub Release<br/>creates/attaches the v* tag<br/>THE ONLY APPROVAL"]
GH --> TG["tag -- Conda Tag<br/>conda-&lt;version&gt;+0 tag at github.sha<br/>then conda/build reset on main"]
GH --> PY["pypi -- build + publish<br/>OIDC trusted publishing"]
TG --> CD["conda -- build + upload<br/>matrix: 2 OS, per-leg evidence"]
TG --> CD["conda -- build + upload<br/>matrix: OS x Python, per-leg evidence"]
GH --> CD
VC --> RS
UT --> RS
Expand All @@ -447,8 +449,8 @@ Only ``github-release`` carries a required reviewer; ``conda-tag``,
removed on `#887 <https://github.com/JarryShaw/PyPCAPKit/issues/887>`__.
What still makes one approval mean *the whole release* is the ``needs:``
graph: ``tag`` and ``pypi`` both depend on ``github``
(``.github/workflows/create-release.yml:806,972``), and ``conda`` depends on
``tag`` and ``github`` (``:1208``) -- so nothing downstream of ``github`` can
(``.github/workflows/create-release.yml:808,974``), and ``conda`` depends on
``tag`` and ``github`` (``:1210``) -- so nothing downstream of ``github`` can
start before it is approved, and rejecting it leaves nothing tagged and
nothing published. Before that removal ``tag`` depended on ``version_check``
alone, so removing its reviewer without moving this dependency would have let
Expand Down Expand Up @@ -495,7 +497,7 @@ back, and neither is needed, since a plain re-run self-heals; see

``environment: pypi`` **stays even though its reviewer is gone.** ``pypi``
publishes through PyPI's OIDC trusted publishing (``environment: pypi`` at
``:967``, ``id-token: write`` at ``:971``), and PyPI's trusted-publisher
``:969``, ``id-token: write`` at ``:973``), and PyPI's trusted-publisher
configuration can be scoped to a GitHub Actions environment name. If this
project's publisher on PyPI is scoped that way, removing the ``pypi`` name
-- not just its reviewer -- would break the upload with a claim mismatch;
Expand All @@ -506,7 +508,7 @@ the name regardless, since there is no upside to removing it.
``conda-tag`` **writes to** ``main``\ **, so a release is not read-only on the
branch.** After cutting ``conda-<version>+0`` at the run's commit, the
``tag`` job pushes a commit setting :file:`conda/build` to ``0`` on top of
``main`` (``.github/workflows/create-release.yml:893-946``) -- a plain
``main`` (``.github/workflows/create-release.yml:895-948``) -- a plain
fast-forward push, never a rewrite -- so the next Conda Update bump of that
version is ``+1``. There is none when the run's commit is not on ``main``, nor
when ``main``'s tip already says ``0`` (the usual case, since
Expand Down
22 changes: 11 additions & 11 deletions docs/source/contributing/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ the caller's own workflow run, as one of the caller's own jobs, rather than
starting an independent run after the caller finishes. All four calls in this
repository target the same reusable workflow, with the same input:

* ``.github/workflows/create-release.yml:593`` -- job ``unit-tests`` calls
* ``.github/workflows/create-release.yml:595`` -- job ``unit-tests`` calls
``./.github/workflows/unit-tests.yml`` with ``gate-only: true``.
* ``.github/workflows/cron-conda.yml:50`` -- job ``unit-tests`` calls
``./.github/workflows/unit-tests.yml`` with ``gate-only: true``.
Expand Down Expand Up @@ -366,13 +366,13 @@ its ``verdict`` job found the Unit Tests verdict was for another commit -- which
the ``vendor_update`` job reads from that run's jobs first, as run 38650191283
was. ``!cancelled()`` keeps a skipped ``vendor_update``, on a push, from
skipping it. ``unit-tests`` then depends on that job with
``needs: [ version_check ]`` (``create-release.yml:581``) and an ``if:`` that
``needs: [ version_check ]`` (``create-release.yml:583``) and an ``if:`` that
calls no status-check function, so a skipped dependency skips it too -- and
every job past it is gated the same way: ``github`` needs ``[ unit-tests,
version_check ]`` (``:619``), ``tag`` needs ``[ github, unit-tests,
version_check ]`` (``:806``), ``pypi`` needs ``[ github, unit-tests,
version_check ]`` (``:972``), and ``conda`` needs ``[ tag, github,
unit-tests, version_check ]`` (``:1208``).
version_check ]`` (``:621``), ``tag`` needs ``[ github, unit-tests,
version_check ]`` (``:808``), ``pypi`` needs ``[ github, unit-tests,
version_check ]`` (``:974``), and ``conda`` needs ``[ tag, github,
unit-tests, version_check ]`` (``:1210``).

.. mermaid::

Expand All @@ -398,7 +398,7 @@ unit-tests, version_check ]`` (``:1208``).

This is a transitive reduction, as in :doc:`releasing`: ``version_check`` and
``unit-tests`` are also direct ``needs:`` of ``tag`` and ``pypi``
(``create-release.yml:806,972``) and of ``conda`` (``:1208``), but ``github``
(``create-release.yml:808,974``) and of ``conda`` (``:1210``), but ``github``
already implies those edges, since it depends on both itself. Drawing them
would add lines without changing which job can start before which, or which
job a skip in ``version_check`` reaches.
Expand Down Expand Up @@ -916,10 +916,10 @@ Deployment ``environment:``
Five jobs declare a deployment ``environment:`` and can therefore pause for an
approval, four in ``create-release.yml`` and one in ``cron-python-versions.yml``:

* ``github`` -- ``environment: github-release`` (``create-release.yml:616``)
* ``tag`` -- ``environment: conda-tag`` (``create-release.yml:804``)
* ``pypi`` -- ``environment: pypi`` (``create-release.yml:967``)
* ``conda`` -- ``environment: anaconda`` (``create-release.yml:1205``)
* ``github`` -- ``environment: github-release`` (``create-release.yml:618``)
* ``tag`` -- ``environment: conda-tag`` (``create-release.yml:806``)
* ``pypi`` -- ``environment: pypi`` (``create-release.yml:969``)
* ``conda`` -- ``environment: anaconda`` (``create-release.yml:1207``)
* ``pr`` -- ``environment: github-release``, for the release App's key (see
`Python Versions`_)

Expand Down
Loading
Loading