Skip to content

alter pre gate to handle multiline secrets - #296

Merged
Saarett merged 3 commits into
masterfrom
chore/pre-gate-ff-fix
Aug 13, 2026
Merged

alter pre gate to handle multiline secrets#296
Saarett merged 3 commits into
masterfrom
chore/pre-gate-ff-fix

Conversation

@GillSami

@GillSami GillSami commented Aug 13, 2026

Copy link
Copy Markdown
  • Please check if the PR fulfills these requirements
  • Tests for the changes have been added
  • Docs have been added / updated
  • All CI checks are green

important

This optimization is controlled by the DETECT_SECRETS_PERF_PREGATE environment variable, which defaults to off. To enable it, set DETECT_SECRETS_PERF_PREGATE=1 in the environment before running the scan (e.g. shell export, Dockerfile ENV, or CI job variable) — no code changes are needed.


scan.py uses a pre-gate to skip lines that "obviously" can't contain a secret, before running the full (slower) plugin set on the remaining lines.
The previous gate was a single hand-written regex (_TRIGGER_PATTERN) that approximated each detector's trigger condition from memory. Because it was hand-transcribed rather than derived from the actual detectors, it drifted from them and silently skipped lines that did contain real secrets.
That's a false negative — the worst possible failure mode for a secrets scanner.
False negatives found in the old gate

Case

  1. Short Artifactory tokens (AKC... under 15 chars)
  2. URL passwords with @ in the userinfo (https://user:pass@host)
  3. Keyword-free high-entropy candidates
  4. Tenant-defined custom / prerun / multiline secret policies

Fix

The gate is now built generically from the plugins actually loaded for the current scan, instead of hand-transcribed (gate.py):

  • Keywords come directly from KeywordDetector.DENYLIST (imported, not retyped).
  • Structural triggers (AWS keys, Artifactory, JWT, etc.) are read generically from each loaded plugin's own .denylist attribute — and checkov's CustomRegexDetector stores tenant custom/prerun patterns in that same attribute, so custom policies are picked up automatically with zero special-casing.
  • The entropy trigger is corrected to match the real precondition (delimiter presence, no length threshold).
  • The gate rebuilds automatically whenever the plugin list changes (e.g. tenant policy update), so it can never go stale.

@GillSami
GillSami force-pushed the chore/pre-gate-ff-fix branch from 3752486 to d9eef27 Compare August 13, 2026 14:12
@GillSami GillSami self-assigned this Aug 13, 2026
@Saarett
Saarett merged commit 8fbb5ba into master Aug 13, 2026
20 checks passed
@Saarett
Saarett deleted the chore/pre-gate-ff-fix branch August 13, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants