Skip unneeded CI jobs for docs-only changes - #831
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a path-based change-classification step in GitHub Actions so documentation-only / repo-metadata-only changes can skip the expensive CI and CodeQL jobs, while preserving a fail-closed required gate.
Changes:
- Add a “change classification” job to both
ci.ymlandcodeql.yml, and gate expensive jobs on its outputs. - Add
.github/ci-path-filters.ymlto define which paths are considered safe to skip. - Update
CI / requiredto validate that any skipped jobs were skipped intentionally (and to fail if classification fails or results are unexpected).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| .github/workflows/ci.yml | Adds a change-classification job and gates CI jobs; updates the required gate to validate expected skips. |
| .github/workflows/codeql.yml | Adds a change-classification job and gates CodeQL analysis based on path classification. |
| .github/ci-path-filters.yml | Defines the allowlist/filters used to decide when full CI and/or CodeQL should run. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (3)
.github/workflows/ci.yml:47
dorny/paths-filteris configured withbase: ${{ github.ref }}, but on bothpushandpull_requesteventsgithub.refpoints at the current ref (e.g.,refs/heads/mainafter the push, orrefs/pull/<n>/merge), not the pre-change base. This can yield an empty/incorrect diff and misclassify changes (potentially skipping CI unexpectedly, or never skipping when intended). Prefer the action’s default base selection (push usesbefore..sha, PR uses the GitHub API) by removing thebaseoverride, or setbase/refexplicitly to the event SHAs.
# On push, compare only the commits in this push. Pull requests use the GitHub API
# to compare the entire PR with its base branch.
base: ${{ github.ref }}
predicate-quantifier: every
filters: .github/ci-path-filters.yml
.github/workflows/codeql.yml:40
dorny/paths-filteris configured withbase: ${{ github.ref }}. Forpull_requestthis is a merge ref (refs/pull/<n>/merge), and forpushit’s the branch ref after the update, which can produce an empty/incorrect diff and misclassify changes (skipping CodeQL unexpectedly or never skipping). Prefer the action defaults by removing thebaseoverride, or setbase/refto the appropriate event SHAs.
with:
base: ${{ github.ref }}
predicate-quantifier: every
filters: .github/ci-path-filters.yml
.github/workflows/ci.yml:210
runtime_compatibility’s matrix is always expanded fromneeds.version_support_matrix.outputs.runtime-matrix. Whenfull_ciisfalse,version_support_matrixis skipped, so this output will be empty; depending on GitHub Actions evaluation order,fromJSON('')can fail before the job-levelifcan skip the job. Provide a default JSON array so the workflow can safely skip runtime compatibility when the matrix job is skipped.
needs:
- changes
- version_support_matrix
if: needs.changes.outputs.full_ci == 'true'
runs-on: ubuntu-24.04
HAYDEN-OAI
left a comment
There was a problem hiding this comment.
Reviewed the exact head against the workflow triggers, pinned classifier implementation, branch-protection required check, push/PR event handling, and the private Stainless mirror. Two substantive issues remain: the editable classification policy can bypass both the only required CI gate and CodeQL, and private-mirror pushes cannot fetch their previous commit after checkout credentials are removed. Please address the inline findings before merge.
| # to compare the entire PR with its base branch. | ||
| base: ${{ github.ref }} | ||
| predicate-quantifier: every | ||
| filters: .github/ci-path-filters.yml |
There was a problem hiding this comment.
[P1] Enforce policy changes independently of the PR-controlled policy
On pull requests, this reads .github/ci-path-filters.yml from the proposed checkout, so the PR can change the very policy that decides whether its changes require CI. For example, a PR can append - '!**' to both full_ci and codeql while also changing arbitrary Java source. With predicate-quantifier: every, no changed path—including the policy file itself—matches either filter, both classifiers return false, all expensive jobs and CodeQL are skipped, and the sole required CI / required check succeeds. This violates the promised invariant that policy changes always run both workflows. Detect policy-file changes through a trusted, non-overridable check and force both classifiers to run, or load the classification policy from the trusted base revision.
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
| with: | ||
| persist-credentials: false |
There was a problem hiding this comment.
[P1] Preserve authenticated history for pushes to the private Stainless mirror
This workflow explicitly also runs in stainless-sdks/openai-java, which is private. On push, the pinned paths-filter action compares github.event.before with the pushed branch; checkout defaults to fetch-depth: 1, so the previous commit is absent and paths-filter executes git fetch --depth=1 origin <before-sha>. persist-credentials: false has already removed the token from local Git configuration, so that fetch cannot authenticate against the private origin. The classifier then fails and CI / required fails for inherited next, stl/**, and codegen/stl/** push workflows. Pull-request runs do not expose this because paths-filter uses the GitHub API there. Fetch complete history during the authenticated checkout, or retain read-only Git credentials until push classification completes.
Summary
CI / requiredcheck and teach it to validate that every skipped job was expectedWhy
In the 50-PR sample from the CI performance analysis, 12% of merged PRs did not need the full CI suite and 16% did not need CodeQL. Those changes currently pay the same roughly 9–12 minute critical path as source changes. This adds a small classifier so safe changes should reach the required gate in roughly 1–2 minutes while preserving fail-closed behavior.
Safety
The skip policy is an explicit allowlist in
.github/ci-path-filters.yml. Changes toci.ymlrun full CI, changes tocodeql.ymlrun CodeQL, changes to the policy run both, and any future unrecognized path runs both by default. The third-party classifier action is pinned to the immutable commit for v4.0.2 and receives only read permissions.Validation
actionlintv1.7.12 on both modified workflowsgit diff --checkThe repository lint wrapper could not run locally because the only installed JDK is 25.0.2 and this Gradle build expects JDK 21; the PR workflow will run the full JDK-21 suite because the change includes
ci.yml.