Repository navigation
Foundation: hygiene entropy rule accepts STAGE1/SHA256-style constant names in keyword arguments (#243) - #333
Conversation
…naming STAGE1/SHA256-style constants (#243) The bare name=VALUE exemption now accepts a run of one-case letters followed by digits (STAGE1, SHA256, UTF8, V2) on both sides. Digits stay after the letters of a run, so random text is still scored. Mirrored copy and docs updated; tests prove secret-shaped values in keyword position stay findings. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opus review 2 of 2 — PR #333 (
|
| Probe | Result |
|---|---|
| Regex before / after | old _*(?:[A-Z]+|[a-z]+)(?:_+(?:[A-Z]+|[a-z]+|[0-9]+))*_*=…, new _*(?:[A-Z]+|[a-z]+)[0-9]*(?:_+(?:(?:[A-Z]+|[a-z]+)[0-9]*|[0-9]+))*_*=… — digits only ever trail the letters of a run, as the comment claims |
Issue probe 1, build(instructions=_STAGE1_INSTRUCTIONS) |
old [(1, 'high-entropy-token', 'inst****** (len 33)')] → new [] |
Issue probe 2, verify(digest_prefix=SHA256_DIGEST_PREFIX_NAME) |
old (len 39) finding → new [] |
Issue probe 3, build(instructions=_STAGE_1_INSTRUCTIONS) |
clean before and after (non-regression) |
Quoted value x="…26 upper + 6 digits…" |
still high-entropy-token |
Secret-named left side (api_key=, token=) |
still high-entropy-token plus secret-assignment |
Base64 padding, x=QUJD…123456== |
still high-entropy-token |
| Mirrored rule blocks | tools/check_hygiene.py:170-183 and src/jev_cookbook/fixtures/_scan.py:102-115 diff to zero bytes — the body's "byte-identical in these lines" holds |
Fail-closed, independently of the builder's fuzz
My own harness, my own seeds: 8 alphabets (base62, base36-lower, base36-upper, base64, base64url,
lower+digits+_, upper+digits+_, lower+digits+.) × 20 000 samples = 160 000, lengths 32 to
64, each assigned unquoted to one of 7 non-secret names (x, instructions, digest_prefix,
SHA256_DIGEST, utf8_name, value, payload), every line scanned by both scanners:
base62: 0 base36-lower: 0 base36-upper: 0 base64: 0
base64url: 0 lower+digits+underscore: 0 upper+digits+underscore: 0 lower+digits+dot: 0
Zero flagged→exempt regressions on my seeds. The builder reports 1 in 160 000 on its own seeds;
I reproduced that exact sample (S1), so the two results are consistent rather than contradictory.
Repository-wide effect — the question the brief asks
Three ways, all on this head:
-
Old scanner vs new scanner over the PR tree (611 files): old 3, new 0, no new-only
findings. All three old-only findings are prose/comment lines this PR adds (M1). Nothing in a
notebook, fixture, recipe or asset changes status. -
New scanner vs old scanner over the
39ddfcetree (611 files, exported with
git archive): 0 findings either way, zero difference. The new rule hides nothing that
exists onmain— there is nothing onmainto hide. -
Footprint, the sharper form of the question since
mainis clean. Across all 611 tracked
files of the PR tree I enumerated every_WORDof 32+ characters that the new
_IDENT_ASSIGNMENTexempts and the old one did not. There is exactly one such word in
the whole repository:words >=32 chars newly matching the exemption across 611 tracked files: 1 instructions=_STAGE1_INSTRUCTIONS -> ['docs/development.md', 'docs/fixtures.md', 'tests/test_hygiene.py']i.e. the PR's own documentation example. Nothing else in the tree newly stops being scored.
Scope, and the mirrored copy the issue's paths did not list
The issue names only tools/check_hygiene.py (plus tests and docs). The builder also edited
src/jev_cookbook/fixtures/_scan.py, saying it "has to change in the same commit because a test
fails when the two diverge". Necessary, and the parity test really does enforce it. I reverted
only the mirror to 39ddfce, left the new tools/check_hygiene.py in place, and ran the suite:
FAILED tests/test_fixtures.py::test_the_packaged_scanner_has_the_same_rules_as_the_repository_scanner
- AssertionError: _IDENT_ASSIGNMENT
10 failed, 511 passed
test_the_packaged_scanner_has_the_same_rules_as_the_repository_scanner
(tests/test_fixtures.py:1447-1467) compares _IDENT_ASSIGNMENT.pattern, and because
_NAME_RUN/_RUN are interpolated into that pattern the new helpers are covered by it even
though they are plain strings not listed in SCANNER_PATTERNS. Belt and braces: every new test in
the exemption block also asserts through packaged_scan.scan_text, so a divergence fails in
tests/test_hygiene.py too. Tree restored to a3b5c3b afterwards — git status clean,
git rev-parse HEAD = a3b5c3bf….
No other path is touched. recipes/39-*'s _STAGE_1_INSTRUCTIONS naming is untouched, as the
issue permits and the body states. This is a foundation/* branch, so the recipe allowlist does
not apply, and tools/check_recipe_scope.py agrees: scope ok: foundation pull request: the recipe allowlist does not apply, exit 0.
Docs against the code
docs/development.md:339, docs/fixtures.md:246-251, the module docstring at
tools/check_hygiene.py:29-31 and the inline comments in both scanner copies all describe the new
grammar, and I found no stale description anywhere in the tree (grepped *.md, *.py, *.ipynb,
*.yml for "letters of one case", "no run mixes", KERNEL_START_TIMEOUT, IDENT_ASSIGNMENT,
STAGE1; .pre-commit-config.yaml, .github/workflows/ and the rest of docs/ carry no second
copy of the rule). Both docs examples
(startup_timeout=KERNEL_START_TIMEOUT, instructions=_STAGE1_INSTRUCTIONS) are exempt under the
shipped regex, as claimed, and instructions=_STAGE1_INSTRUCTIONS was a finding on main, which
makes the docs line self-enforcing. The only inaccuracy is S2, which the old wording shared.
Tests: do they bite, are they slow, are they flaky
- Red, both scanner copies at
39ddfce:11 failed, 510 passed— exactly the body's number.
The failures are the 9test_keyword_arguments_with_letter_digit_constant_names_are_not_findings
cases,test_the_three_probes_of_the_issue_through_the_command_line, and
test_the_repository_itself_is_clean. So the new tests prove the rule, not the status quo. - Every positive case is guarded against guarding nothing:
_scores_without_the_exemption
(tests/test_hygiene.py:1232) asserts each line inDIGIT_RUN_KEYWORDSwould score if the
exemption were removed, so none of the 9 cases can silently go vacuous. - The three issue probes go through the real CLI:
test_the_three_probes_of_the_issue_through_the_command_line(tests/test_hygiene.py:1253)
writesprobe_bad.py,probe_good.pyandprobe_sha.pywith the issue's exact contents and
runstools/check_hygiene.pyas a subprocess under a neutral account, asserting exit 0 and
Hygiene check passed. All three also appear in the in-processDIGIT_RUN_KEYWORDScases, which
additionally check a notebook scan,run()on a.pyfile and the packaged copy. - Fail-closed is proved by mutation:
_LOOSE_EXEMPTIONS(tests/test_hygiene.py:1336)
monkeypatches_IDENT_ASSIGNMENTto each of the issue's two suggested rules and shows
instructions=<KEY>becomes unreported — andinstructions=<KEY>is itself one of the 40
SECRET_KEYWORD_LINEScases, so the mutants demonstrably break a test that is actually in the
suite. - The 160 000-case fuzz is NOT in the suite. Confirmed by reading the whole
tests/test_hygiene.pydiff: the only new loops are 9 + 40 + 6 + 2 deterministic
parametrizations plus a two-assertion backtracking guard. The fuzz lives only in the PR body. - Runtime (3.14,
-p no:cacheprovider):tests/test_hygiene.pyalone 9.63 s (309 passed);
the three hygiene/fixture files together 13.28 s (532 passed). The slowest test is the
pre-existingtest_the_repository_itself_is_cleanat 5.2-5.7 s. The slowest new test is
test_the_three_probes_of_the_issue_through_the_command_lineat 0.18 s; every other new test
is under 0.01 s. Not slow. - Not flaky. Every value is derived from a fixed seed —
_random_token(seed=7),
_random_text(..., seed=21..24),base64.b64encode(random.Random(11).randbytes(36)). Nothing
reads the clock, the environment or an unseeded RNG. I rantests/test_hygiene.pyfour times
(alone, combined, and twice under reverted scanners) with identical results, and it passes
identically on 3.10 and 3.14. - Backtracking. The guard at
tests/test_hygiene.py:1356only exercisesfullmatch, so I also
timed the new rule through the real scanner on three long near-misses:scan_texton 10 001,
15 001 and 24 001-character words returned in 0.001 s, 0.003 s and 0.004 s. The nested
quantifiers are safe because neither_+nor_RUNcan match empty, so a partial run
consumption fails at the next_+in O(1) instead of multiplying.
Checks run
Environment: fresh venvs under /home/agent/pr333-r2/ (uv venv --python 3.14 and
--python 3.10, uv pip install --link-mode symlink -e ".[dev]"), everything under
flock .claude/heavy.lock nice -n 10 with no nested flock, pytest always -p no:cacheprovider,
long runs via a background wrapper with log polling. free -m showed 2.0-4.8 GB available
throughout. Both venvs and all scratch deleted at the end; nothing else removed.
Check (docs/development.md) |
Result |
|---|---|
ruff check . |
All checks passed |
ruff format --check . |
401 files already formatted |
python tools/check_hygiene.py (3.14) |
Hygiene check passed: 611 file(s) scanned. exit 0 |
python tools/check_hygiene.py (3.10) |
Hygiene check passed: 611 file(s) scanned. exit 0 |
python tools/render_catalog.py --check |
exit 0 |
python tools/check_recipe_assets.py |
Image-asset check passed: 461 file(s) under recipes/ checked. exit 0 |
full pytest (3.14) |
5952 passed, 49 skipped, 50 deselected, 31 xfailed in 195.17s |
pytest tests/test_hygiene.py tests/test_fixtures.py tests/test_check_hygiene_generic_usernames.py (3.14) |
532 passed in 13.28s |
pytest tests/test_hygiene.py tests/test_fixtures.py (3.10) |
521 passed in 13.32s (+ 11 in test_check_hygiene_generic_usernames.py = the body's 532) |
pytest tests/test_hygiene.py (3.14, timed) |
309 passed in 9.63s |
python tools/notebook_ci.py fixtures |
51 recipe folders, 0 failed, exit 0 |
python tools/check_recipe_scope.py --base origin/main --head HEAD --branch foundation/hygiene-kwarg-entropy --body-file <body> |
scope ok: foundation pull request: the recipe allowlist does not apply, exit 0 |
red run, both scanners at 39ddfce |
11 failed, 510 passed (matches the body, test for test) |
divergence run, only the mirror at 39ddfce |
parity test fails AssertionError: _IDENT_ASSIGNMENT; 10 failed, 511 passed |
old-vs-new cross-scan, PR tree and main tree |
see "Repository-wide effect" |
| newly-exempt-word footprint over 611 files | exactly 1 word, the PR's own docs example |
| independent 160 000-sample fail-closed fuzz | 0 flagged→exempt regressions |
No notebook is affected by this diff (no path under recipes/ changes), so no notebook was
executed; the whole-tree hygiene scan and tools/notebook_ci.py fixtures cover the notebook and
fixture files as data. pytest was never run in the primary checkout and no branch there was
touched.
PR body against the diff, claim by claim
| Claim | Verdict |
|---|---|
Closes #243 and nothing else |
✅ Closes appears once; #243 is the only issue reference in the whole body. The other fix/closed hits are prose ("fails closed", "fix round") with no issue number after them, so nothing else can close |
| 5 files: the two scanners, the tests, the two docs | ✅ matches git diff --stat |
| exemption now accepts one-case letters followed by digits, on both sides | ✅ |
new _NAME_RUN, with _RUN built from it |
✅ |
digits may only follow letters, so a1b2c3… still does not match |
✅ by fuzz and by targeted cases |
| both copies byte-identical in these lines | ✅ zero-byte diff of the two blocks |
secret-named left sides, quoted values, mixed-case runs, digit-before-letter, + / = still scored |
✅ all re-verified independently |
red: 11 failed, 510 passed, and which tests fail |
✅ reproduced exactly, test names included |
green on 3.14 and 3.10: 532 passed each |
✅ (3.14 directly; 3.10 as 521 + 11) |
| issue probes old-vs-new, lengths 33 and 39 | ✅ exact lengths reproduced |
| 40 fail-closed cases; two mutant tests | ✅ 8 values × 5 names = 40; 2 mutants |
the 1-in-160 000 residual x=sjtpcm32_xiroprmvbaooqgii_asprlpn |
✅ reproduced: flagged on 39ddfce, exempt now |
re-scan: Hygiene check passed: 611 file(s) scanned., exit 0, on 3.14 and 3.10 |
✅ both |
| old scanner on this branch "flags exactly two lines, both in the docs", "nothing in the other 609 files" | ❌ three lines in three files, 608 others — M1. The conclusion ("no existing finding is hidden") is nevertheless true and I verified it independently |
ruff check/ruff format --check clean, 401 files |
✅ |
check_recipe_assets.py passed, 461 files |
✅ |
full pytest: 5951 passed, 49 skipped, 50 deselected, 31 xfailed |
✅ for the head it was written against. I measure 5952 passed with the other three numbers identical; the +1 is test_decision_result_membership_and_iteration_by_question_name, the single test #332 added, which arrived with the origin/main merge after the body was written. Expected drift, not an error — and CONTRIBUTING forbids editing the body to chase it |
notebook_ci.py fixtures: 51 recipe folders, 0 failed |
✅ |
render_catalog.py --check exit 0 |
✅ |
Acceptance criteria of #243, one by one
-
"
instructions=_STAGE1_INSTRUCTIONSanddigest_prefix=SHA256_DIGEST_PREFIX_NAMEpass."
MET. Both werehigh-entropy-tokenfindings under39ddfce(len 33 and len 39) and are
clean on this head throughscan_text, through a notebook scan, throughrun()on a.py
file, through the packaged copy, and through the real CLI (exit 0,Hygiene check passed). -
"A real high-entropy literal still fails, proved by mutation rather than asserted."
MET.test_secret_shaped_values_in_keyword_position_are_still_findingsholds 40 cases
(base62, base36 lower/upper, base64, base64url,STAGE1_+random,SHA256_+random,
settings.+random, each ×instructions,digest_prefix,SHA256_DIGEST,utf8_name,x),
asserted through four entry points plus the packaged copy. The mutation is real:
test_a_looser_exemption_would_hide_a_secret_which_is_what_the_tests_above_provepatches the
exemption to each of the issue's own two suggestions and shows the secret goes unreported, so
the fail-closed assertions provably bite. Quoted literals and secret-named left sides stay
scored (test_secret_names_and_random_names_stay_scored_with_letter_digit_constantsplus the
pre-existingtest_one_case_values_and_long_left_sides_cannot_hide_a_secret), and my own
independent 160 000-sample fuzz found no flagged→exempt regression. -
"The three probes above exist as regression tests."
MET, and through the real CLI.test_the_three_probes_of_the_issue_through_the_command_line
(tests/test_hygiene.py:1253) writes the issue'sprobe_bad.py,probe_good.pyand
probe_sha.pyverbatim and runstools/check_hygiene.pyas a subprocess on each. The two
failing probes plus the 9 parametrized cases are 10 of the 11 red failures, so they are genuine
regression tests and not tautologies. -
"The published set is re-scanned under the new rule and the result recorded here; recipe 39's
_STAGE_1_INSTRUCTIONSnaming may then be left as-is or reverted, as its author prefers."
PARTLY MET. The re-scan was really done and I reproduced it (611 files, exit 0, zero
findings on 3.14 and 3.10; zero status change for any file that exists onmain). But the
recorded result is wrong — three flagged lines in three files, not two in two, and 608 other
files, not 609 (M1) — and nothing is recorded on Foundation: check_hygiene rejects ordinary keyword arguments whose constant name contains a letter-digit run (STAGE1, SHA256) #243 itself, whose only comment is the
2026-10-20 triage note, while AC4 says "recorded here". The second half of the criterion is
met: recipe 39 is untouched and the body says so explicitly.
Verdict
The code, the tests and the documentation are in good shape and I found nothing wrong with any of
them: the grammar is the narrowest one that fixes the issue, the mirror edit is necessary and
mechanically enforced, fail-closed is proved by mutation and survives an independent 160 000-case
fuzz, the repository-wide footprint is a single word this PR itself introduces, and the suite is
fast and deterministic on both 3.10 and 3.14. I would approve this head unchanged the moment the
re-scan record is corrected. I withhold approval only on M1: the miscounted re-scan is the whole
of the evidence recorded for acceptance criterion 4 of #243, it concerns a gate every pull request
passes, and the correction costs one pull-request comment and one issue comment — no code, no new
head, no re-run.
OPUS-REVIEW: CHANGES-NEEDED a3b5c3b
|
Correction to the PR description's re-scan record (the head is final, so this is a comment, not a body edit), for acceptance criterion 4 of #243 and as the audit trail for relaxing the secret scanner. Run on head a3b5c3b (611 files): the |
Opus review 2 — addendum on the re-scan correction, PR #333
M1 is resolved. I checked every figure and quotation in both comments against what I measured on OPUS-REVIEW: APPROVE a3b5c3b |
Review of PR #333 — reviewer 1 of 2 (independent)
Diff (vs What the change actually is
used at line 207 as I read the grammar as: left side =
Is the loosening safe? (my own probing)I tried to build a realistic secret that the new exemption hides. All probe values are synthetic, Realistic vendor formats are all still caught. Mixed case (Stripe Quoted literals are still caught — the usual way a secret reaches source. Fuzz, old rule vs new. 160,000 samples, 32-64 chars, 8 flat alphabets (base62, base36 lower, The leak class the builder describes ("a chain of one-case letter+digit-suffix runs joined by No new false positives: in every fuzz cell the new-rule finding count equalled the old-rule Backtracking. Linear, measured. I also checked whether a tighter grammar was available and it is not. Bounding the digit suffix Overall: I think the loosening is proportionate, the grammar is the right discriminator, and the Must changeM1. The one rule this PR invents is not pinned by any test
The PR's whole justification over the issue's simpler suggestions is "Digits may only follow the and ran
Why the existing net misses it: all 8 values of Issue #243's acceptance criterion 2 is "A real high-entropy literal still fails, proved by What to change — either is enough, both is better, and both are small:
M2.
|
mutant of _NAME_RUN |
result |
|---|---|
[A-Za-z0-9]+ (free alnum runs) |
51 failed, 470 passed — all 40 secret-shaped cases fail |
[A-Za-z]+[0-9]* (mixed case in a run) |
3 failed, 518 passed — caught by the pre-existing MIXED_34 cases |
| `[0-9]*(?:[A-Z]+ | [a-z]+)[0-9]*` (digit before the letters) |
Base-scanner runs:
- Both scanner files restored to
39ddfce,pytest tests/test_hygiene.py tests/test_fixtures.py
→11 failed, 510 passed, exactly the tests the PR body names. - Base scanner driven over the head tree → 611 files, 3 findings (listed in M2).
Independent probing (all synthetic, under /home/agent/r333/, nothing written into the repository):
issue-probe replay through the CLI; 13 hand-built grammar-shaped secret shapes; 13 position probes
(source and notebook-output) for kwarg/assignment/default-arg/dict-literal/f-string/export/.env/repr,
quoted and unquoted; vendor-rule interaction probes; a 160,000-sample flat-alphabet old-vs-new fuzz
plus 220,000 further samples of _- and .-grouped shapes; a secret-name-bypass sweep over 17
names; and a backtracking/timing sweep to 100,001 characters.
CI on the exact head a3b5c3bf…: Lint (ruff), Tests (py3.10), Tests (py3.14),
Catalog (README is current), Hygiene (secrets and notebook outputs), Fixtures (validate),
Scope (recipe pull requests), Notebooks (discover), Notebooks (execute) and all 29
per-notebook jobs: success. Commit signature verified.
Verdict
The rule is the right one and better than either option the issue suggested; the implementation
matches its stated grammar; it fails closed on every realistic secret format I could construct; it
does not backtrack; the mirrored copy and its parity test are correct; the red/green evidence in the
body reproduces. I am asking for changes only because the single discriminator this PR invented —
"digits may only follow the letters of a run" — is the one thing the 40-case fail-closed net cannot
see, in a gate that every pull request passes and whose false negatives are the expensive kind, and
because this file was explicitly made independent of its own exemption once before (M2). Both fixes
are a few lines and need no redesign.
OPUS-REVIEW: CHANGES-NEEDED a3b5c3b
…ets and keep its test file independent of it (#243) Review fix round for PR #333. Adds chains of short underscore-joined chunks to the secret-shaped values (digits before letters, interleaved, mixed case) and a mutant test per relaxed property, so dropping "digits only trail the letters" now fails. Rewrites the comment and the _STAGE_1 probe so tests/test_hygiene.py scans clean with the exemption switched off, and adds a test that proves it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opus review 2 — delta re-review, PR #333 new head
I accept the delta. Checks rerun (fresh 3.14 and 3.10 venvs under One thing the delta changed that is worth recording, not blocking. I re-derived the re-scan on Scratch and both venvs deleted; nothing I did not create was removed. OPUS-REVIEW: APPROVE 400277e |
|
Audit-trail update for acceptance criterion 4 of #243, with the final head's figures (400277e, which supersedes the figures in the earlier correction comment): the |
Re-review of PR #333 — reviewer 1 of 2 (independent), fix round 1
The merge
Coherent, nothing lost, no stray conflict markers ( Finding M1 — resolved, more thoroughly than I askedI asked for one I re-applied my own mutants to
The failure sets are exactly what the design predicts, which is how I know the new values are doing
Each mutation was applied to The mutant that was green on The values are properly discriminated rather than all killed by the widest mutant, which is what
The three mutants are nested, so
Finding M2 — resolved, and the builder found a second hit I had missedTwo things were wrong on
I verified both halves by routes of my own, not by running the builder's tests:
The two docs lines left as they are — I agree
Did I find a secret the shipped rule hides that the new tests do not cover?No new one. The scanner is unchanged, so the exempt set is exactly what I characterised last round: The one axis still unpinned is the Suggestions (not blocking, carried over or new)
PR body and fix-round record against the diff
Acceptance criteria of #243
Checks I ran on
|
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opus review 2 — delta check, PR #333 new head (merge of #334)
I accept the delta. It is content-free with respect to this pull request. Checks (fresh 3.14 venv under CI on this exact head is complete and every check is Nothing from my earlier reviews is reopened, and nothing new is blocking. The optional notes from OPUS-REVIEW: APPROVE 581d275 |
Delta re-review of PR #333 — reviewer 1 of 2, round 2 merge of
|
1 participant
Closes #243
What this adds
tools/check_hygiene.pyand its mirrored copysrc/jev_cookbook/fixtures/_scan.py: the barename=VALUEexemption from the high-entropy-token rule (_IDENT_ASSIGNMENT) now accepts a run that is one-case letters followed by digits (STAGE1,SHA256,utf8,V2), on both sides. Before, a run could not mix letters and digits at all, soinstructions=_STAGE1_INSTRUCTIONS(33 characters, scored as one token) was a finding. New helper regex_NAME_RUN;_RUNis built from it. Digits may only follow the letters of a run, soa1b2c3...style text still does not match. Both copies are byte-identical in these lines (tests/test_fixtures.py::test_the_packaged_scanner_has_the_same_rules_as_the_repository_scannerstill passes unchanged).+,/,=) are still scored.tests/test_hygiene.py: regression cases for the three probes of the issue (including through the real CLI), fail-closed cases (secret-shaped values in keyword position), two "mutant" tests, and a backtracking guard. Details below.docs/development.md,docs/fixtures.mdand the module docstring oftools/check_hygiene.py: the rule description now states the run grammar.Recipe 39's
_STAGE_1_INSTRUCTIONSnaming is untouched (out of scope; it may stay as is or be reverted by its author).Decisions for the reviewers
name=plus any 40-character alphanumeric secret that starts with a letter (the testtest_a_looser_exemption_would_hide_a_secret...runs that mutant and showsinstructions=<random base62>goes unreported); "a run may mix letters and digits in any order" does the same. Requiring digits to trail the letters of a run is the discriminator that keeps random text scored while covering STAGE1, SHA256, UTF8, V2, BASE64._or.(for exampleSTAGE1_SHA256_...) under a non-secret-named left side is exempt. In a 160 000-sample fuzz of random secrets (base62, base36 lower/upper, base64url, lower/upper plus_, 32 to 64 chars, assigned to seven non-secret names, scanned with the old and the new rule) exactly one sample went from flagged to exempt:x=sjtpcm32_xiroprmvbaooqgii_asprlpn, a random lowercase/digit/underscore string whose underscores happened to fall at word-like boundaries (1 of 20 000 samples of that alphabet; 0 of 20 000 for each of the other seven alphabets). Hex, all-letter and all-digit values were never findings of this rule before either.How it was verified
Tests were written first and run against the unmodified scanners (red), then against the change (green).
tools/check_hygiene.pyandsrc/jev_cookbook/fixtures/_scan.pyrestored toorigin/main(eba0506),pytest tests/test_hygiene.py tests/test_fixtures.pygives11 failed, 510 passed: the 9test_keyword_arguments_with_letter_digit_constant_names_are_not_findings[...]cases,test_the_three_probes_of_the_issue_through_the_command_line, andtest_the_repository_itself_is_clean(the docs now name_STAGE1_INSTRUCTIONS).pytest tests/test_hygiene.py tests/test_check_hygiene_generic_usernames.py tests/test_fixtures.pygives532 passedon each.scan_text:build(instructions=_STAGE1_INSTRUCTIONS)andverify(digest_prefix=SHA256_DIGEST_PREFIX_NAME)werehigh-entropy-tokenfindings (len 33, 39) and are now clean; the_STAGE_1_probe was and stays clean.test_secret_shaped_values_in_keyword_position_are_still_findings(40 cases: base62, base36 lower, base36 upper, base64, base64url,STAGE1_+ random,SHA256_+ random,settings.+ random; each assigned toinstructions,digest_prefix,SHA256_DIGEST,utf8_name,x) asserts ahigh-entropy-tokenfinding from the script, a notebook scan,run()on a.pyfile and the packaged copy.test_secret_names_and_random_names_stay_scored_with_letter_digit_constantscoversapi_key=,auth_token=,db_password=,client_secret=with ordinary constants and a random left side with a constant. The two mutant tests monkeypatch_IDENT_ASSIGNMENTto each looser rule from the issue and showinstructions=<KEY>becomes unreported, i.e. the fail-closed tests do bite.python tools/check_hygiene.pyon this branch:Hygiene check passed: 611 file(s) scanned., exit 0 (3.14 and 3.10). Theorigin/mainscanner run on this branch flags exactly two lines, both in the docs added here (docs/development.md and docs/fixtures.md nameinstructions=_STAGE1_INSTRUCTIONS); it flags nothing in the other 609 files, so no existing finding is hidden by the new rule.ruff check .clean,ruff format --check .clean (401 files), fullpytest:5951 passed, 49 skipped, 50 deselected, 31 xfailed,python tools/check_recipe_assets.pypassed (461 files),python tools/render_catalog.py --checkexit 0,python tools/notebook_ci.py fixtures:51 recipe folders, 0 failed.