Skip to content

fix(strix): accept a changed file named by its path suffix - #2500

Open
seonghobae wants to merge 2 commits into
mainfrom
fix-strix-report-scope-basename
Open

seonghobae wants to merge 2 commits into
mainfrom
fix-strix-report-scope-basename

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Why

#2474 made a completed PR scan fail closed unless penetration_test_report.md contains the full repo-relative path of a changed file. Scanners routinely abbreviate a scoped file to its basename. On #2357 (head 4748a022), Strix reported Vulnerabilities 0, and its report text assessed check_telemetry_ownership.py and opencode_repository_dispatch_targets.json by name. The gate still failed:

ERROR: Strix report scope: scan report does not identify a changed source file
Strix completed without a report tied to a changed source file; failing closed.

(run 36393793492, job 108888653481)

The scope dir keeps the repo layout (scope_root / relative_path in copy_changed_file_into_scope), so the model could see the full path but shortened it. The contract rejects a report that names the right file, so this failure is not caused by #2357.

What

names_changed_path accepts the full path or any component-boundary suffix of it (ci/x.py, x.py). It still rejects:

  • test_x.py / i/x.py (the name continues past a path boundary)
  • x.pyc (a longer extension)
  • unrelated reports (the existing OpenSSH negative case)

The full path is also accepted under an absolute scan root (/workspace/scope/scripts/ci/x.py), which the old substring check allowed.

The module is now at 100% in-process coverage. Before this PR it was exercised only through a subprocess and measured 0%. validate also gets the docstring it was missing.

Relation to open PRs

Verification

  • pytest tests/test_strix_report_scope.py: 4 passed (fixtures already include the four finish-tool fields fix(strix): reject template reports and request PR source evidence #2492 will require); coverage report --include=scripts/ci/strix_report_scope.py: 100%
  • interrogate scripts/ci/strix_report_scope.py: 100%
  • bash scripts/ci/test_strix_quick_gate.sh: PASS
  • Local full coverage run -m pytest tests (measured before the fixture follow-up commit): 5096 passed. I did not use the local repo-wide TOTAL as evidence. CI reports coverage per module with --include, and my local environment is missing some fixtures (4 tests skipped).

Follow-up (not in this PR)

run_strix_scan returns 1 on a scope mismatch without trying another model, so one weak-model report fails the whole required check.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dnm2df79qqeCJtSxfttJxW

#2474 required a completed PR report to contain the full repo-relative
path of a changed file. Scanners routinely abbreviate a scoped file to its
basename (`check_telemetry_ownership.py`), so #2357 failed closed with
"Vulnerabilities 0" even though the report assessed exactly the changed
files. Match any component-boundary suffix of the changed path instead,
rejecting partial names such as `test_x.py` for `x.py` or `x.pyc`.

Also bring the module to in-process coverage and add the missing
`validate` docstring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dnm2df79qqeCJtSxfttJxW
@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 41 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 9d8ea489-4b0f-4ffb-9b06-610f60dc27fe

📥 Commits

Reviewing files that changed from the base of the PR and between 3295c25 and d04865f.

📒 Files selected for processing (2)
  • scripts/ci/strix_report_scope.py
  • tests/test_strix_report_scope.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Keeps the new cases valid if #2492's placeholder check lands first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dnm2df79qqeCJtSxfttJxW
@seonghobae

Copy link
Copy Markdown
Contributor Author

Second independent reproduction, from contextual-orchestrator#1227 at head 0b4d2503f1f72aba4dc0cd05ce5f1504425b3e36.

  • Strix run 36409604379, job 108886973092: Vulnerabilities 0, then ERROR: Strix report scope: scan report does not identify a changed source file.
  • The changed source paths were contextual_orchestrator/orchestrator.py, contextual_orchestrator/provider_errors.py and contextual_orchestrator/review_gateway.py. The report named `orchestrator.py` by basename only.
  • I ran this PR's names_changed_path (head d04865f14) against the job log text. It returns True for contextual_orchestrator/orchestrator.py and False for the other two, so this PR would have admitted that report. The current main matcher rejects it.

This is local evidence from one job log, not hosted acceptance of this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant