Skip to content

fix(ci): validate release source before building distributions - #5220

Merged
jbeckwith-oai merged 3 commits into
mainfrom
codex/sdk-726-release-checks
Sep 28, 2026
Merged

jbeckwith-oai merged 3 commits into
mainfrom
codex/sdk-726-release-checks

Conversation

@jbeckwith-oai

@jbeckwith-oai jbeckwith-oai commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This pull request restores tests and type checks against the exact release source before PyPI publishing. The full test suite runs on Python 3.10–3.14 with locked dependencies, followed by type checking, before building or uploading distributions. Stale lockfiles fail validation instead of allowing tests to run against outdated dependencies. Checks run in a separate read-only job. After they pass, the build job independently checks out and validates the same release SHA on a fresh runner, so test-side filesystem, process, or environment changes cannot affect the distributions. Failed checks stop publication while preserving release provenance validation, environment approval, and the artifact-ID handoff to the isolated publisher.

Test plan

  • Focused workflow and release-provenance tests: 32 passed on Python 3.14.
  • Regression tests execute the workflow shell for success, intermediate Python test failure, and typecheck failure.
  • An offline regression exercises actual uv synchronization: a fresh lock passes, a stale lock is rejected without mutation, and the previous frozen setting fails the regression.
  • An offline package probe confirms a test-mutated checkout contaminates built output; the workflow regressions require separate hosted jobs with independent provenance validation and no test artifacts or caches forwarded to build.
  • Two fresh independent security and compatibility reviews returned clean results for the final diff.
  • Required local verification passed: formatting, lint, mypy, Pyright, and full tests (11,562 passed; 69 skipped).
  • Native macOS sandbox tests were skipped by the mandated Codex sandbox configuration; the dedicated hosted CI job supplies that coverage.

Issue number

N/A

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

The repository's required independent implementation review was completed by two reviewers; the literal /review command was not invoked.

@jbeckwith-oai
jbeckwith-oai marked this pull request as ready for review September 28, 2026 15:54
@jbeckwith-oai
jbeckwith-oai requested review from a team, rm-openai and seratch as code owners September 28, 2026 15:54
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T16:32:36.392666Z 550787c New commits
🔒 Security Review ✅ Completed 2026-09-28T16:34:17.446682Z 550787c New commits

Security findings

Blocking findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e73b44c42

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/publish.yml Outdated

@markstuart-oai markstuart-oai left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1e73b44. The existing P2 about stale lockfiles remains actionable after an independent source check. Please use UV_LOCKED=1 or validate uv lock --check before the frozen matrix. The provenance validator checks commit/tag/version, and normal CI also uses frozen installs, so neither establishes that this release manifest matches the dependencies being tested. This is the same existing finding.

The Python 3.10–3.14 test loop, typecheck ordering, failure propagation, publisher isolation and artifact-ID handoff otherwise look sound. The shell regression exercises both intermediate test and typecheck failures. All 22 exact-head hosted checks passed. Source-only review with an independent workflow pass; no repository tests were run locally.

The frozen-versus-locked behavior is documented in uv locking and syncing.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: 20271c1b89

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread .github/workflows/publish.yml
@jbeckwith-oai
jbeckwith-oai enabled auto-merge (squash) September 28, 2026 16:39
@jbeckwith-oai
jbeckwith-oai merged commit 2747c1c into main Sep 28, 2026
22 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/sdk-726-release-checks branch September 28, 2026 16:41
@github-actions github-actions Bot mentioned this pull request Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants