Skip to content

fix(ctf): don't count the current event in the RCE multi-turn gate - #604

Open
Sahil-u07 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
Sahil-u07:fix/rce-gate-self-count
Open

Sahil-u07 wants to merge 1 commit into
GenAI-Security-Project:mainfrom
Sahil-u07:fix/rce-gate-self-count

Conversation

@Sahil-u07

@Sahil-u07 Sahil-u07 commented Oct 17, 2026 •

Copy link
Copy Markdown

What's wrong

The Shell Shock challenge (shell_shock.yaml) sets min_prior_tool_calls: 1, so a one-shot reverse shell shouldn't complete it. The player is supposed to warm the agent up with a safe SystemUtils call first. Right now that gate doesn't hold.

CTFEventProcessor._process_event stores the event in ctf_events (_store_ctf_event) before it runs the detectors. By the time RCEDetector._count_prior_tool_calls queries the table, the execute_script call it's checking is already there, and it gets counted as its own prior call. So a single malicious call is enough.

Fix

Exclude the current event from the count by its external_event_id. The id is built the same way _store_ctf_event builds it, and I left a comment pointing there so the two don't drift apart.

Tests

New file tests/unit/ctf/test_rce_multi_turn_gate.py. Both tests store events through the real _store_ctf_event before calling the detector, so they follow the same order the processor uses:

  • a single reverse-shell call with nothing before it is not detected
  • the same call after one run_diagnostics call is still detected

On main the first test fails because the detector returns detected=True. With the fix both pass.

# main
FAILED tests/unit/ctf/test_rce_multi_turn_gate.py::test_single_shot_rce_does_not_pass_gate
1 failed, 1 passed

# this branch
2 passed

Before (on main):

before

After:

after

The full suite gives the same 4 failures I get on main (3 in test_specialized_agents.py, 1 in test_database_provider.py), nothing new.

Related: #300 is about the same function when workflow_id is missing. That's a separate problem and I haven't touched it here.

The event processor stores every event before it runs the detectors, so
by the time RCEDetector counts prior SystemUtils calls the malicious
execute_script call is already in ctf_events and counts as its own
"prior" call. With min_prior_tool_calls: 1 (shell_shock.yaml) a single
reverse-shell payload was enough to complete the challenge.

Leave the current event out of the count by its external_event_id.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant