Skip to content

fix: keep suppression comment text out of content analysis - #123

Open
chaksaray wants to merge 2 commits into
developfrom
fix/122-suppression-text
Open

chaksaray wants to merge 2 commits into
developfrom
fix/122-suppression-text

Conversation

@chaksaray

@chaksaray chaksaray commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Closes #122

What and why

A suppression comment is free text: ids, a reason, a reviewer. The pattern engines read it like any other line, so the comment could raise findings of its own, and suppressing one finding could create another.

Reproduced on develop (e0ae94d) before changing anything. The runbook's own snippet does not trigger anything, so the issue gives working cases:

                                 before                                          after
inline.md   (default)            findings: AVE-2026-00007 line 1, AVE-2026-00002   findings: none, suppressed: 1 (the real 00001)
block.md    (default)            findings: none, suppressed: 1 (the comment text)  findings: none, suppressed: 0
inline.md   (--no-ignore)        00001, 00007, 00002                               unchanged
block.md    (--no-ignore)        00001 at line 3 (the comment body)                unchanged

The text of bawbel-ignore and bawbel-accept comments (also -start and -end) is now replaced with spaces before the engines run, in scanner/core/preprocessor.py. Newlines and columns are kept, so finding line numbers do not move. scan() passes the masked text to the pattern, YARA, Semgrep, LLM and sandbox engines and leaves the original for the suppression parsers, so every suppression keeps working.

How it is kept narrow:

  • Only when suppressions are honored. Under --no-ignore (audit mode) a comment is ordinary text and is scanned in full
  • Only well-formed directives. An HTML comment must close within 1000 characters and hold no second <!--. A # or // marker must start the line or follow whitespace, so https://bawbel-ignore.example/ is not a comment. # reason: style lines count as metadata only straight after a directive. An unterminated or oversized comment is scanned, so the mask cannot swallow content after it. The limit is SUPPRESSION_COMMENT_MAX_CHARS in scanner/config/default.py

Why this does not widen what a hostile file can hide: when suppressions are honored, a file can already suppress findings on its own line with bawbel-ignore, or wrap anything in bawbel-ignore-start and -end. Masking the comment text adds no capability beyond that, and --no-ignore is still the audit mode. This is the one design judgement in the PR, so it is spelled out for review.

bawbel.yml and .bawbelignore: neither raises findings today, even with attack text in a .bawbelignore comment (checked), and nothing in the scanner reads bawbel.yml as configuration (bawbel init only writes it). The acceptance asked for them to be excluded from self-scan, so collect_files skips them at the root of a directory scan (SCANNER_CONFIG_FILES) and scripts/self_scan.sh drops bawbel.yml from its targets. The same names deeper in a tree, or named on the command line, are still scanned, so a hostile pack cannot hide a payload by naming a file bawbel.yml.

Output contract impact

Fields added: none
Fields renamed: none
Fields removed: none

Findings and suppressed findings that existed only because of a suppression comment's text no longer appear in default mode. No field changes.

Checks

  • Golden fixture unchanged: findings 7 max CRITICAL risk 9.8
  • Tests pass: 671 passed in 303.79s (35 are new, in tests/unit/test_suppression_text_not_scanned.py)
  • No em dashes added
  • pre-commit run --all-files clean: exit=0, 14 hooks (black, flake8, bandit, the self-scan and the full suite among them)
  • bash scripts/self_scan.sh exits 0
  • Negative fixture tests/fixtures/skills/clean/self_referential_suppression.md: no findings by default, and AVE-2026-00001 and AVE-2026-00007 appear under --no-ignore

The tests failed before the fix (the new names did not exist, so collection errored) and pass after. Eleven mutations of the implementation, each run against the final 35 tests, are all caught:

mask applied under --no-ignore too                 -> 5 fail
mask does nothing                                  -> 15 fail
line marker may follow non-space (URLs)            -> 1 fail
HTML comment may contain a second opener           -> 2 fail
HTML comment has no length bound                   -> 1 fail
metadata lines masked without a directive          -> 2 fail
metadata run never stops                           -> 1 fail
masking changes line breaks                        -> 2 fail
config files skipped at any depth                  -> 1 fail
config files never skipped                         -> 2 fail
LLM engine gets raw text on the Magika path        -> 1 fail

The LLM engine needs an API key and cannot run here, so a test captures what run_llm_scan receives on both call sites in scan(): the usual path, and the path taken when Magika reports a content-type finding that is not dangerous. That second call was uncovered at first; the last mutation above found it.

Not changed

  • Where findings from suppression comments are reported in bawbel accept output and the JSON audit trail: unchanged, since those read the original content
  • The multi-line "justified" parser's continuation heuristic treats any line starting with *, #, // or <!-- as part of the comment. That is why masking uses a stricter rule: a bullet after a directive stays content. Found, not changed

Merging with other scanner PRs

Trial merges (git merge, no conflicts) with #117, #119 and #121. The CHANGELOG entry sits at the bottom of [Unreleased] for that reason.

🤖 Generated with Claude Code

https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P

chaksaray and others added 2 commits October 22, 2026 07:29
A bawbel-ignore comment is free text: ids, a reason, a reviewer. The
engines read it like any other line, so a reason that quoted the pattern
it suppressed created a second finding that the comment did not cover,
and a block comment's own text was listed as a suppressed finding.

The text of bawbel-ignore and bawbel-accept comments (also -start and
-end) is replaced with spaces before the engines run. Newlines and
columns are kept, so line numbers do not move; the suppression parsers
still read the original content. Only when suppressions are honored:
under --no-ignore the comment is ordinary text and is scanned. Only
well-formed directives are masked. An HTML comment must close within
1000 characters and hold no second opener, a line marker must start the
line or follow whitespace, and metadata lines count only right after a
directive, so an unterminated or hostile comment cannot swallow content.

bawbel.yml and .bawbelignore are the scanner's own configuration. They
are skipped at the root of a directory scan and dropped from the
self-scan targets; the same names deeper, or named on the command line,
are still scanned.

Closes #122

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P
Signed-off-by: chaksaray <chaksaray@gmail.com>
scan() calls the LLM engine from two places. The second runs when Magika
reports a content-type finding that is not dangerous, and no test reached
it, so a change there to pass the raw text went unnoticed. The test now
runs both paths, with and without such a finding, in honored and audit
mode. With the second call changed to pass the raw content it fails.

Refs #122

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015T9K217kxBqsUZ3omKYv9P
Signed-off-by: chaksaray <chaksaray@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant