Skip to content

Skip unneeded CI jobs for docs-only changes - #831

Closed
jbeckwith-oai wants to merge 2 commits into
mainfrom
codex/skip-unneeded-ci
Closed

Skip unneeded CI jobs for docs-only changes#831
jbeckwith-oai wants to merge 2 commits into
mainfrom
codex/skip-unneeded-ci

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Summary

  • add a fail-closed change-classification job to CI and CodeQL
  • skip build, test, API compatibility, runtime compatibility, and CodeQL analysis for documentation-only and repository-metadata-only changes
  • preserve the existing CI / required check and teach it to validate that every skipped job was expected
  • keep full CI and CodeQL enabled for new or unknown paths by default

Why

In the 50-PR sample from the CI performance analysis, 12% of merged PRs did not need the full CI suite and 16% did not need CodeQL. Those changes currently pay the same roughly 9–12 minute critical path as source changes. This adds a small classifier so safe changes should reach the required gate in roughly 1–2 minutes while preserving fail-closed behavior.

Safety

The skip policy is an explicit allowlist in .github/ci-path-filters.yml. Changes to ci.yml run full CI, changes to codeql.yml run CodeQL, changes to the policy run both, and any future unrecognized path runs both by default. The third-party classifier action is pinned to the immutable commit for v4.0.2 and receives only read permissions.

Validation

  • actionlint v1.7.12 on both modified workflows
  • classifier scenarios covering docs-only, source, mixed, workflow, policy, release metadata, and unknown paths
  • required-gate scenarios covering safe skips, full PR CI, Stainless push behavior, classifier failure, unexpected jobs, and runtime failure
  • git diff --check

The repository lint wrapper could not run locally because the only installed JDK is 25.0.2 and this Gradle build expects JDK 21; the PR workflow will run the full JDK-21 suite because the change includes ci.yml.

Copilot AI review requested due to automatic review settings July 31, 2026 21:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a path-based change-classification step in GitHub Actions so documentation-only / repo-metadata-only changes can skip the expensive CI and CodeQL jobs, while preserving a fail-closed required gate.

Changes:

  • Add a “change classification” job to both ci.yml and codeql.yml, and gate expensive jobs on its outputs.
  • Add .github/ci-path-filters.yml to define which paths are considered safe to skip.
  • Update CI / required to validate that any skipped jobs were skipped intentionally (and to fail if classification fails or results are unexpected).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
.github/workflows/ci.yml Adds a change-classification job and gates CI jobs; updates the required gate to validate expected skips.
.github/workflows/codeql.yml Adds a change-classification job and gates CodeQL analysis based on path classification.
.github/ci-path-filters.yml Defines the allowlist/filters used to decide when full CI and/or CodeQL should run.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/codeql.yml
Comment thread .github/workflows/codeql.yml Outdated
Comment thread .github/ci-path-filters.yml Outdated
Copilot AI review requested due to automatic review settings July 31, 2026 21:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.github/workflows/ci.yml:47

  • dorny/paths-filter is configured with base: ${{ github.ref }}, but on both push and pull_request events github.ref points at the current ref (e.g., refs/heads/main after the push, or refs/pull/<n>/merge), not the pre-change base. This can yield an empty/incorrect diff and misclassify changes (potentially skipping CI unexpectedly, or never skipping when intended). Prefer the action’s default base selection (push uses before..sha, PR uses the GitHub API) by removing the base override, or set base/ref explicitly to the event SHAs.
          # On push, compare only the commits in this push. Pull requests use the GitHub API
          # to compare the entire PR with its base branch.
          base: ${{ github.ref }}
          predicate-quantifier: every
          filters: .github/ci-path-filters.yml

.github/workflows/codeql.yml:40

  • dorny/paths-filter is configured with base: ${{ github.ref }}. For pull_request this is a merge ref (refs/pull/<n>/merge), and for push it’s the branch ref after the update, which can produce an empty/incorrect diff and misclassify changes (skipping CodeQL unexpectedly or never skipping). Prefer the action defaults by removing the base override, or set base/ref to the appropriate event SHAs.
        with:
          base: ${{ github.ref }}
          predicate-quantifier: every
          filters: .github/ci-path-filters.yml

.github/workflows/ci.yml:210

  • runtime_compatibility’s matrix is always expanded from needs.version_support_matrix.outputs.runtime-matrix. When full_ci is false, version_support_matrix is skipped, so this output will be empty; depending on GitHub Actions evaluation order, fromJSON('') can fail before the job-level if can skip the job. Provide a default JSON array so the workflow can safely skip runtime compatibility when the matrix job is skipped.
    needs:
      - changes
      - version_support_matrix
    if: needs.changes.outputs.full_ci == 'true'
    runs-on: ubuntu-24.04

@HAYDEN-OAI HAYDEN-OAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact head against the workflow triggers, pinned classifier implementation, branch-protection required check, push/PR event handling, and the private Stainless mirror. Two substantive issues remain: the editable classification policy can bypass both the only required CI gate and CodeQL, and private-mirror pushes cannot fetch their previous commit after checkout credentials are removed. Please address the inline findings before merge.

Comment thread .github/workflows/ci.yml
# to compare the entire PR with its base branch.
base: ${{ github.ref }}
predicate-quantifier: every
filters: .github/ci-path-filters.yml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Enforce policy changes independently of the PR-controlled policy

On pull requests, this reads .github/ci-path-filters.yml from the proposed checkout, so the PR can change the very policy that decides whether its changes require CI. For example, a PR can append - '!**' to both full_ci and codeql while also changing arbitrary Java source. With predicate-quantifier: every, no changed path—including the policy file itself—matches either filter, both classifiers return false, all expensive jobs and CodeQL are skipped, and the sole required CI / required check succeeds. This violates the promised invariant that policy changes always run both workflows. Detect policy-file changes through a trusted, non-overridable check and force both classifiers to run, or load the classification policy from the trusted base revision.

Comment thread .github/workflows/ci.yml
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve authenticated history for pushes to the private Stainless mirror

This workflow explicitly also runs in stainless-sdks/openai-java, which is private. On push, the pinned paths-filter action compares github.event.before with the pushed branch; checkout defaults to fetch-depth: 1, so the previous commit is absent and paths-filter executes git fetch --depth=1 origin <before-sha>. persist-credentials: false has already removed the token from local Git configuration, so that fetch cannot authenticate against the private origin. The classifier then fails and CI / required fails for inherited next, stl/**, and codegen/stl/** push workflows. Pull-request runs do not expose this because paths-filter uses the GitHub API there. Fetch complete history during the authenticated checkout, or retain read-only Git credentials until push classification completes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants