Skip to content

fix(detectors): extract clean text from list content blocks in PIIDetector (#127) - #608

Open
rohanbuilds-ai wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rohanbuilds-ai:fix/pii-detector-list-content-resolution
Open

rohanbuilds-ai wants to merge 1 commit into
GenAI-Security-Project:mainfrom
rohanbuilds-ai:fix/pii-detector-list-content-resolution

Conversation

@rohanbuilds-ai

Copy link
Copy Markdown

Summary

Fixes #127: Extract clean text from structured message content blocks in PIIDetector._resolve_field to prevent str() from producing Python repr strings that break downstream PII regex evaluation.

Problem

In finbot/ctf/detectors/primitives/pii.py, _resolve_field() resolved response_content by calling str(content) on assistant messages:

if msg.get("role") == "assistant":
    content = msg.get("content", "")
    if content:
        return str(content)

When assistant messages use the standard OpenAI / Anthropic structured content block format (e.g. [{"type": "text", "text": "Your SSN is 123-45-6789"}]), calling str(content) returns the Python representation string "[{'type': 'text', 'text': 'Your SSN is 123-45-6789'}]".

Because PII patterns evaluate against clean text strings, running regexes against the Python repr string fails or produces false negatives, allowing sensitive data (SSNs, TINs, account numbers) to leak without detection.

Solution

  • In PIIDetector._resolve_field():
    • If content is a list of blocks, extract text from items with item.get("text") or plain string items and join them with space.
    • Safely ignore non-text blocks (such as tool_use blocks with no text) without crashing or raising errors.
    • Apply the same safe extraction logic to top-level event["response_content"] lists.
    • Preserve standard string content extraction and dotted path navigation (event["tool_call.arguments.account_number"]).

Verification & Test Evidence

  1. Baseline Reproduction: Running the new test suite against unmodified origin/main baseline confirmed 3 test failures:
    • test_prm_pii_012_response_content_list_format_extracted_as_text: assert "[{'type': 'text', 'text': 'Your SSN is 123-45-6789'}]" == 'Your SSN is 123-45-6789'
    • test_prm_pii_012_detection_on_list_content: AssertionError: assert False where False = DetectionResult(detected=False, ... message='Found 0 PII match(es), need at least 1').detected
    • test_multiple_content_blocks_joined: assert "[{'type': 'text', 'text': 'Vendor EIN:'}, {'type': 'text', 'text': '12-3456789'}]" == 'Vendor EIN: 12-3456789'
  2. Post-Fix: All 9 unit tests in tests/unit/ctf/test_pii_detector.py pass in 0.39s.
  3. Full CTF Suite: All 36 existing passing tests in tests/unit/ctf/ continue to pass cleanly (1 skipped: google sheets integration).
  4. Linters & Style: isort --check-only, black --check, and git diff --check are 100% clean.

…ector

- Handle structured content block lists in assistant messages during _resolve_field
- Prevent str() coercion of lists from producing Python repr strings that break PII regex matching
- Support joining multiple text blocks, plain strings, and top-level response_content lists
- Safely filter out non-text blocks (e.g. tool_use) without errors
- Add unit test suite covering PRM-PII-012, list extraction, and primitive scanning in tests/unit/ctf/test_pii_detector.py

Fixes GenAI-Security-Project#127
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug_030_MUST_FIX: Test Case PRM-PII-012 PIIDetector._resolve_field coerces list content via str() instead of extracting text

1 participant