Skip to content

ci(secu): stop gitleaks job failing on rate limits #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sc979 merged 1 commit into from
Oct 15, 2026
Merged

ci(secu): stop gitleaks job failing on rate limits #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 108 additions & 19 deletions .github/workflows/gitleaks-analysis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@ on:
workflow_call:

env:
# Same rule as the centreon-ubuntu-26.04 runner images (centreon-images,
# runners/base-ubuntu-26.04/Dockerfile): "latest" = newest stable release
# published at least 4 days before the run. Set a version (e.g. "8.30.1")
# to pin it.
# Version installed on the runners that do not ship gitleaks (see "Install
# Gitleaks"). Same rule as the centreon-ubuntu-26.04 runner images
# (centreon-images, runners/base-ubuntu-26.04/Dockerfile): "latest" =
# newest stable release published at least 4 days before the run. Set a
# version (e.g. "8.30.1") to pin it.
GITLEAKS_VERSION: "latest"
# Installed when "latest" cannot be resolved, so the scan still runs.
GITLEAKS_FALLBACK_VERSION: "8.30.1"
Expand All @@ -28,25 +29,114 @@ jobs:
echo "::error::pull_request_target is not allowed"
exit 1

# Pull requests only scan their own commits: fetch them plus the merge and base commits, full history otherwise
- name: Compute fetch depth
id: fetch-depth
# Plain git instead of actions/checkout: the job then downloads no
# action at "Set up job", where concurrent jobs behind the shared
# egress of the self-hosted runners get rate limited by
# codeload.github.com (429) and fail before any step runs. Same clone
# as actions/checkout with persist-credentials: false:
# pull_request: the PR commits plus the merge and base commits
# (depth = PR commits + 2), the merge commit checked out detached.
# workflow_dispatch: full history of all branches and tags, the
# dispatched commit checked out detached.
- name: Checkout Repository
env:
EVENT_NAME: ${{ github.event_name }}
EVENT_SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }}
FULL_REF: ${{ github.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_COMMITS: ${{ github.event.pull_request.commits }}
REPOSITORY: ${{ github.repository }}
SERVER_URL: ${{ github.server_url }}
FETCH_TOKEN: ${{ github.token }}
run: |
echo "value=$(( ${PR_COMMITS:-0} > 0 ? PR_COMMITS + 2 : 0 ))" >> "$GITHUB_OUTPUT"
shell: bash
set -euo pipefail

- name: Checkout Repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: ${{ steps.fetch-depth.outputs.value }}
persist-credentials: false
# Same authentication as the re-fetches of "Run secret scan": the
# header is set on each git process only, never written to
# .git/config.
FETCH_AUTH="$(printf 'x-access-token:%s' "${FETCH_TOKEN}" | base64 -w0)"
echo "::add-mask::${FETCH_AUTH}"
unset FETCH_TOKEN

# Retried like actions/checkout does: 3 attempts.
fetch() {
local try
for try in 1 2 3; do
GIT_CONFIG_COUNT=1 \
GIT_CONFIG_KEY_0="http.${SERVER_URL}/.extraheader" \
GIT_CONFIG_VALUE_0="AUTHORIZATION: basic ${FETCH_AUTH}" \
git fetch --no-tags --prune --no-recurse-submodules origin "$@" && return 0
if [ "${try}" -lt 3 ]; then
echo "::warning::git fetch failed (attempt ${try}/3); retrying in $((try * 10))s."
sleep $((try * 10))
fi
done
echo "::error::git fetch failed after 3 attempts: $*"
return 1
}

# Start from an empty workspace, like actions/checkout's default
# clean.
find . -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
git init -q .
git config --local gc.auto 0
git remote add origin "${SERVER_URL}/${REPOSITORY}"

case "${EVENT_NAME}" in
pull_request)
# Pull requests only scan their own commits: fetch them plus
# the merge and base commits.
DEPTH_OPT=()
if [ "${PR_COMMITS:-0}" -gt 0 ]; then
DEPTH_OPT=(--depth="$((PR_COMMITS + 2))")
fi
fetch "${DEPTH_OPT[@]}" "+${EVENT_SHA}:refs/remotes/pull/${PR_NUMBER}/merge"
git checkout -q --force --detach "refs/remotes/pull/${PR_NUMBER}/merge"
;;
workflow_dispatch)
fetch "+refs/heads/*:refs/remotes/origin/*" "+refs/tags/*:refs/tags/*"
if git cat-file -e "${EVENT_SHA}^{commit}" 2>/dev/null; then
git checkout -q --force --detach "${EVENT_SHA}"
elif [ "${FULL_REF}" = "refs/heads/${REF_NAME}" ]; then
# Force-pushed away since the dispatch: check out the live
# branch tip; "Run secret scan" falls back to it too.
echo "::warning::Commit ${EVENT_SHA} is gone; checking out the live ${REF_NAME} tip."
git checkout -q --force --detach "refs/remotes/origin/${REF_NAME}"
else
echo "::error::Commit ${EVENT_SHA} is not reachable from ${FULL_REF}."
exit 1
fi
;;
*)
# "Run secret scan" fails the job on unsupported events.
echo "Unsupported event '${EVENT_NAME}': nothing to check out."
exit 0
;;
esac
echo "Checked out $(git rev-parse HEAD)."

- name: Install Gitleaks
env:
# Same criterion as the centreon branch of runs-on: private and
# internal centreon repositories run on the centreon-ubuntu-26.04-*
# runners, which ship gitleaks. The quanta-security runners do not
# yet (MON-212266): they install it like the public runners.
RUNNER_SHIPS_GITLEAKS: ${{ github.repository_visibility != 'public' && github.repository_owner == 'centreon' }}
run: |
set -uo pipefail

# The runner image team owns the version of the shipped binary:
# only check that it runs, and fail closed if it is missing.
if [ "${RUNNER_SHIPS_GITLEAKS}" = "true" ]; then
if ! CURRENT="$(gitleaks version 2>/dev/null)"; then
echo "::error::This runner must ship gitleaks, but 'gitleaks version' failed; failing the job."
exit 1
fi
echo "Using the runner's gitleaks ${CURRENT}: $(command -v gitleaks)."
exit 0
fi

case "${RUNNER_ARCH}" in
X64) ARCH=x64 ;;
ARM64) ARCH=arm64 ;;
Expand Down Expand Up @@ -78,8 +168,7 @@ jobs:
fi
fi

# The centreon-ubuntu-26.04-* runners ship gitleaks, built
# with the same rule: keep it when it is the expected version,
# Keep a gitleaks already on PATH when it is the expected version,
# otherwise (missing, or any other version) install that release.
CURRENT="$(gitleaks version 2>/dev/null || true)"
if [ "${CURRENT#v}" = "${VERSION}" ]; then
Expand Down Expand Up @@ -139,7 +228,7 @@ jobs:
BASE_SHA="${PR_BASE_SHA}"
TIP_SHA="${EVENT_SHA}"

# persist-credentials: false leaves no token in .git/config, so the
# "Checkout Repository" leaves no token in .git/config, so the
# re-fetches below authenticate on their own. The header is set on
# each git process only, through env-scoped config
# (GIT_CONFIG_COUNT/KEY/VALUE): never written to disk, never on a
Expand All @@ -162,7 +251,7 @@ jobs:
# ref is missing.
ref_commit() { git rev-parse --verify -q "$1^{commit}"; }

# Pull requests are checked out shallow (see "Compute fetch depth"),
# Pull requests are checked out shallow (see "Checkout Repository"),
# where a boundary commit looks parentless: gitleaks would scan its
# whole tree. A parentless commit in base..head is either such a
# boundary or shared history the base side does not reach yet
Expand Down Expand Up @@ -226,7 +315,7 @@ jobs:
return 0
}

# actions/checkout can't be re-run in a loop, so re-fetch only what
# The checkout can't be re-run in a loop, so re-fetch only what
# the range needs: the PR head and base branch plus the PR head/base
# SHAs (pull_request), or all branches plus the dispatched commit
# (workflow_dispatch), if the server still serves them. Only the
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Because consumers pin to `@main`, changes to workflows and the blocklist take ef
- `dependency-analysis.yml` — `workflow_call` + runs on PR. Two responsibilities:
1. **Lockfile compliance** — enforces PNPM only (rejects `yarn.lock` / `package-lock.json`), requires a lockfile, and enforces a minimum `lockfileVersion` (`8.9.9`).
2. **Blocklist scan** — downloads `compromised-packages.txt` from `main` and greps every `pnpm-lock.yaml` for `name@version` matches.
- `gitleaks-analysis.yml` — `workflow_call`, supporting `pull_request` and `workflow_dispatch` only (org repos call it on both; `workflow_dispatch` is mostly for debugging the pipeline). Resolves the gitleaks version like the `centreon-ubuntu-26.04` runner images (`centreon-images`, `runners/base-ubuntu-26.04/Dockerfile`): `GITLEAKS_VERSION: latest` is the newest stable release published at least 4 days before the run (read from the public releases feed, `releases.atom`, without any token), falling back to `GITLEAKS_FALLBACK_VERSION` with a `gitleaks-version-fallback` warning when the feed cannot be resolved; set a version to pin it. Keeps the runner's own gitleaks when `gitleaks version` matches (the `centreon-ubuntu-26.04-*` runners ship it), and otherwise installs that release for the runner's architecture, verified against the release's checksums file. On a PR it scans `base..head`: every non-merge commit not already on the destination branch (whatever it is called, and whether or not the commits are on other branches), so it never fails on secrets already on the base and fails on a secret in one of the PR's commits. On `workflow_dispatch` it scans the ref's whole history. Pull requests are checked out shallow (`fetch-depth` = PR commits + 2): the scan deepens the base side until the range holds no parentless commit (a shallow boundary would be read as a root and its whole tree scanned). Event SHAs missing from the clone are re-fetched with the job token. Fails closed: leaks, gitleaks errors, unresolvable ranges and unsupported events (push, schedule, …) all fail the job. `pull_request_target` always fails (not compliant with the security posture), from a first step that runs before checkout.
- `gitleaks-analysis.yml` — `workflow_call`, supporting `pull_request` and `workflow_dispatch` only (org repos call it on both; `workflow_dispatch` is mostly for debugging the pipeline). Resolves the gitleaks version like the `centreon-ubuntu-26.04` runner images (`centreon-images`, `runners/base-ubuntu-26.04/Dockerfile`): `GITLEAKS_VERSION: latest` is the newest stable release published at least 4 days before the run (read from the public releases feed, `releases.atom`, without any token), falling back to `GITLEAKS_FALLBACK_VERSION` with a `gitleaks-version-fallback` warning when the feed cannot be resolved; set a version to pin it. On the `centreon-ubuntu-26.04-*` runners (private and internal `centreon` repositories), it uses the gitleaks the image ships without checking its version (the runner image team owns it) and fails closed if it is missing; elsewhere (public runners, and `quanta-security` until it ships gitleaks, MON-212266) it keeps a gitleaks on PATH when `gitleaks version` matches, and otherwise installs that release for the runner's architecture, verified against the release's checksums file. The repository is checked out with plain `git` instead of `actions/checkout`, so the job downloads no action at "Set up job" (where concurrent self-hosted jobs got rate limited by `codeload.github.com`). On a PR it scans `base..head`: every non-merge commit not already on the destination branch (whatever it is called, and whether or not the commits are on other branches), so it never fails on secrets already on the base and fails on a secret in one of the PR's commits. On `workflow_dispatch` it scans the ref's whole history. Pull requests are checked out shallow (`fetch-depth` = PR commits + 2): the scan deepens the base side until the range holds no parentless commit (a shallow boundary would be read as a root and its whole tree scanned). Event SHAs missing from the clone are re-fetched with the job token. Fails closed: leaks, gitleaks errors, unresolvable ranges and unsupported events (push, schedule, …) all fail the job. `pull_request_target` always fails (not compliant with the security posture), from a first step that runs before checkout.

**Warn-then-enforce gating** (in `dependency-analysis.yml`): findings are reported as `[WARNING]` until a deadline, after which they become `[ERROR]` and fail the build. Controlled by two repository-level GitHub variables:
- `OVERRIDE_DEPENDENCY_ENFORCEMENT_DATE` — the date enforcement begins.
Expand Down
Loading