Skip to content

runner: keep only standard descriptors in spawned children (#79) - #81

Open
novelKR wants to merge 2 commits into
mainfrom
codex/runner-descriptor-hygiene
Open

novelKR wants to merge 2 commits into
mainfrom
codex/runner-descriptor-hygiene

Conversation

@novelKR

@novelKR novelKR commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Issue

Refs #79. This is the fix for the defect that #79's inventory found and its bounded run measured. It is an independent product defect fix, justified by CodeSpace's own correctness. It is not progress on CS-RG or on any DevGuard integration milestone.

Summary

The defect. macOS cannot create a pipe, a socket pair or a pseudo-terminal close-on-exec atomically, and a spawn passes on every descriptor that is not close-on-exec at that instant. A child spawned while another thread is between creating a descriptor and marking it therefore inherits it. #79's baseline run on rust-v0.154.0 (results) observed:

  • PTY pairs. Pipe children held other executions' PTY master and slave pairs: 39/200, 33/100, 60/200 and 39/100 pipe strays per case, and 21/100 and 27/100 pipe targets.
  • Stdin pipes. Two pipe children held another execution's stdin pipe.
  • Lifetime extension. A held master kept a finished execution's terminal allocated for the holder's remaining lifetime (up to about 2 s with the run's synthetic holders).
  • No delayed EOF, but a real mechanism. No stdout or stderr write end was observed held from outside. The same mechanism with a same-execution holder delayed completion by the holder's whole lifetime (PC1), which is what a foreign holder of a write end would do.

What this guarantees. Every child that the runner host spawns without child-side exclusion keeps only its standard descriptors (0–2): it holds no other descriptor of the Gateway's or the worker's. That covers pipe executions, the patch helper, the Linux sandbox probe and prepare helper, and the Gateway's UDS worker. Commands already get a cleared environment; now they also get no other descriptor.

How. A pre-exec step marks every descriptor above 2 close-on-exec in the child (crates/runner/src/descriptors.rs):

Platform Mechanism
Linux one close_range(3, ~0, CLOSE_RANGE_CLOEXEC)
macOS the child lists its own descriptors with proc_pidinfo(PROC_PIDLISTFDS) into a buffer allocated before the fork, sized from the descriptor table plus a margin
Fallback (another platform, a failed or possibly truncated listing, or a kernel before Linux 5.11) a walk of every descriptor number up to min(RLIMIT_NOFILE, kernel limit)
  • Async-signal-safe. The step makes only system calls on memory allocated before the fork. It does not allocate, lock or panic.
  • Standard descriptors (0–2) are kept.
  • Descriptors already close-on-exec, such as std's exec-error pipe, are left alone, so a failed exec is still reported as a spawn error (ProcessSpawnFailed).
  • The PTY path is unchanged. portable-pty already sweeps descriptors above 2 in the PTY child, and Investigate descriptor inheritance between concurrent spawns on macOS #79 measured 0/730 PTY children holding anything foreign.

What does not change:

  • the Codex pin (6b9826e, rust-v0.154.0), every dependency, and Cargo.toml/Cargo.lock;
  • process ownership, reaping, output pumps, completion and lease release, timeouts, the PTY stack, and the InProcess and UDS models;
  • env_clear(), stdio wiring, the working directory, and kill_on_drop.

One mechanical difference: because a pre-exec step is installed, std spawns these children through fork/exec instead of posix_spawn. The verification below reports the timing effect next to the baseline.

Alternatives considered:

Alternative Why not here
posix_spawn with POSIX_SPAWN_CLOEXEC_DEFAULT the strongest form, but std and tokio do not expose it. It would mean a CodeSpace-owned spawner replacing tokio's child management, which is an execution-structure change
Codex's DescriptorPolicy::Explicit not at the pinned rust-v0.154.0; it would need a pin change
A CodeSpace-wide lock around every spawn closes only the windows of code that takes the lock: not the HTTP accept path, library-internal creators, or an embedding process. It also serializes spawns

Contract changes

None to tool schemas, error codes, status values or IDs. Behaviour change: a spawned command, the patch helper, the Linux helpers and the UDS worker no longer inherit any descriptor above 2 from the CodeSpace process.

Tests

New regression tests:

  • The mechanism (descriptors::tests).
    • An unguarded control: a child must see a deliberately inheritable descriptor (held held), proving detection.
    • A guarded child keeps only its standard descriptors (held clear).
    • The fallback walk alone excludes it.
    • A failed exec is still reported (NotFound).
    • The descriptor limit and the macOS listing size are checked.
  • The product path (process::tests::pipe_child_keeps_only_its_standard_descriptors). A real pipe execution through InProcessRunner::exec reports held clear for a deliberately inheritable descriptor; fd 1 is the positive control.
  • The worker command (runtime::tests::worker_keeps_only_its_standard_descriptors). It exercises the exact command the Gateway uses to start the UDS worker.
  • Both product-path tests fail without the fix. Removing the guard makes them report held held, which was checked locally on macOS.

Behaviour preservation. These existing tests still pass:

  • missing_executable_is_process_spawn_failed and tty_missing_executable_is_process_spawn_failed (spawn errors);
  • spawn_hello_then_drop_kills_worker, with the real worker binary;
  • the runner and server suites (below).

Run locally on macOS (Apple M1, macOS 27, Homebrew Rust 1.98.0):

Check Result
cargo test -p codespace-runner lib 88 passed; fs_watch 12; isolation_files 2; patch_helper 1; uds_runner 12
cargo test -p codespace-server, with CODESPACE_RUNTIME_BIN set to a built codespace-codex-runtime lib 26 passed, including both runtime tests, plus every integration suite (apply, approvals, e2e, http_contract, inbox, operations, policy_contract, preflight, process, protocol_compat, read_find, recovery, rollback, security, stdio_contract, transport_contract)
cargo clippy --all-targets -- -D warnings (root) passed
cargo fmt --all --check passed

CI on this head (8afb701): all green (9 passed, 3 skipped as planned).

  • On Linux, the clippy legs compiled the close_range branch cleanly under -D warnings.
  • The Rust / Integration leg ran every new test, and each passed: the mechanism tests, including the unguarded control; the product-path test through the sandboxed pipe path; and the worker test. It also ran spawn_hello_then_drop_kills_worker and the spawn-error tests.
  • The macOS CI leg runs only the pty and file-system crates, so the macOS path is verified by the local runs above and by the comparative run below, not by CI. Adding the runner to the macOS leg is a CI change this PR does not make.

Comparative verification against the rust-v0.154.0 baseline: every expectation was met.

How it ran:

  • Harness. The sealed Investigate descriptor inheritance between concurrent spawns on macOS #79 harness ran unchanged: 8 files byte-identical to the baseline's.
  • Inputs. It ran against this commit with Codex 6b9826e and Rust 1.95.0, as the baseline did, under a protocol frozen beforehand (SHA-256 d1fe0bda…).
  • Run. 06:59–07:01 UTC, 1,590 executions, 0 errors, no stop.
Measure Baseline (ab0341b) This PR (8afb701)
Pipe strays holding a foreign descriptor (P8 / T8 / P64 / T64) 39/200, 33/100, 60/200, 39/100 0/200, 0/100, 0/200, 0/100
Pipe targets holding a foreign descriptor (P8 / P64) 21/100, 27/100 0/100, 0/100
A deliberately inheritable file reaching pipe children (NC1) 5/5 0/5
PTY targets whose master was held by another execution (T8 / T64) 14/50, 14/50 0/50, 0/50
PTY targets whose slave was held by another execution (T8 / T64) 12/50, at least 3/50 0/50, 0/50
PTY children holding anything foreign 0 0
Case wall time (P8, T8, P64, T64) 76.7, 38.5, 10.8, 6.2 s 76.5, 38.3, 10.8, 6.2 s
B0 delay, median / max 18.5 / 21.8 ms 18.3 / 20.5 ms
PC1 delay (a same-execution holder; mechanism unchanged) 2014–2017 ms 2005–2013 ms

Notes:

  • Detection. The frozen analysis marks NC1 detection inconclusive, which the protocol anticipated: excluding that file is the fix's purpose. Detection is established instead by the probe instrument check (12/12) and by the tests' unguarded controls.

  • Timing has no pass mark. Tail delays at limit 64 differed in both directions:

    • P64: p95 67 ms against 22 ms; maximum 92 ms against 100 ms.
    • T64: p95 57 ms against 36 ms; maximum 99 ms against 139 ms.

    The host was more loaded this time, with a load average of 6.5 against 2.7.

  • Evidence (DevGuard, local): cs79-verify-2026-10-01, MANIFEST.json SHA-256 ea3e1e43…. The baseline is cs79-2026-09-30, 10beb5fd….

Security scenarios

  • Cross-execution descriptor leakage. Another execution's PTY master or slave, or its pipe ends, can no longer reach a pipe child, the patch helper or the worker. That closes:
    • reading or writing another execution's terminal;
    • extending its terminal's lifetime;
    • delaying its completion and lease release through a held output write end;
    • holding its stdin write end.
  • The Gateway's own descriptors. Descriptors the Gateway holds without close-on-exec no longer reach commands or the worker. These include an accepted HTTP client socket inside its creation window, and anything inherited from the launching client. Store files were already close-on-exec.
  • Unchanged:
    • path escape, token handling, workspace isolation and rollback;
    • the PTY child's existing sweep;
    • what a command can open itself.

Out of scope

🤖 Generated with Claude Code

macOS cannot create a pipe, a socket pair or a pseudo-terminal
close-on-exec atomically, and a spawn passes on every descriptor that is
not close-on-exec at that instant. #79's baseline run on rust-v0.154.0
found pipe children holding other executions' PTY master and slave pairs
(about 20-39% of pipe strays per case) and, twice, another execution's
stdin pipe. A holder can delay an execution's completion, keep its
terminal allocated, or read and write it.

Every spawn the runner host makes without child-side exclusion now marks
each descriptor above 2 close-on-exec in the child, before exec: pipe
executions, the patch helper, the Linux sandbox probe and prepare helper,
and the Gateway's UDS worker. On Linux this is one close_range(2) call; on
macOS it lists the child's descriptors with proc_pidinfo into a buffer
allocated before the fork; elsewhere, and as the fallback, it walks every
descriptor number up to the limit. Standard descriptors are kept, and
descriptors that are already close-on-exec (std's exec-error pipe) are
left alone, so a failed exec is still reported as a spawn error.

The PTY path already sweeps in portable-pty and is unchanged. No pin,
dependency or execution-structure change; pipe spawns now take std's
fork/exec path instead of posix_spawn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the macOS descriptor listing and the walk bounded by
RLIMIT_NOFILE. A descriptor can stay open above a soft limit lowered
after it was opened, so the limit is no coverage bound. The child now
walks every descriptor number below its own descriptor-table size:
proc_pidinfo(PROC_PIDLISTFDS) without a buffer on macOS, which returns
(fd_nfiles + 20) entries, and FDSize from /proc/self/status on Linux
when close_range(CLOSE_RANGE_CLOEXEC) is refused. Every open descriptor
indexes that table.

Fail the spawn instead of continuing when the table size is unavailable
or an open descriptor cannot be inspected or marked: only EBADF means a
number is not open. Other platforms always fail this way.

On Linux the fallback reads /proc/self/status through syscall() (openat,
read, close) into a stack buffer, so the step makes no allocating or
cancellation-point call after fork.

Add tests that fail when each protection is removed: a descriptor above
a lowered limit with the table walk forced (run in a fresh copy of the
test binary), an unavailable table size failing the spawn before the
child runs, the errno classification, and FDSize parsing.

Run the descriptor-hygiene tests in the existing macOS CI stage
(macos-core): the runner's descriptor and spawn-error tests and the
server's worker-command test. The macOS leg's compiles list follows the
crates those commands build, so changes to them select the leg.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant