Repository navigation
Conversation
The event processor stores every event before it runs the detectors, so by the time RCEDetector counts prior SystemUtils calls the malicious execute_script call is already in ctf_events and counts as its own "prior" call. With min_prior_tool_calls: 1 (shell_shock.yaml) a single reverse-shell payload was enough to complete the challenge. Leave the current event out of the count by its external_event_id.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's wrong
The Shell Shock challenge (
shell_shock.yaml) setsmin_prior_tool_calls: 1, so a one-shot reverse shell shouldn't complete it. The player is supposed to warm the agent up with a safe SystemUtils call first. Right now that gate doesn't hold.CTFEventProcessor._process_eventstores the event inctf_events(_store_ctf_event) before it runs the detectors. By the timeRCEDetector._count_prior_tool_callsqueries the table, theexecute_scriptcall it's checking is already there, and it gets counted as its own prior call. So a single malicious call is enough.Fix
Exclude the current event from the count by its
external_event_id. The id is built the same way_store_ctf_eventbuilds it, and I left a comment pointing there so the two don't drift apart.Tests
New file
tests/unit/ctf/test_rce_multi_turn_gate.py. Both tests store events through the real_store_ctf_eventbefore calling the detector, so they follow the same order the processor uses:run_diagnosticscall is still detectedOn
mainthe first test fails because the detector returnsdetected=True. With the fix both pass.Before (on
main):After:
The full suite gives the same 4 failures I get on
main(3 intest_specialized_agents.py, 1 intest_database_provider.py), nothing new.Related: #300 is about the same function when
workflow_idis missing. That's a separate problem and I haven't touched it here.