Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
af51222
feat(docker): 1/5 — add network: llm_only and egress_allowlist to Doc…
CarlesUIPath Oct 1, 2026
27a62c4
feat(docker): 2/5 — add the stdlib-only egress proxy for network: llm…
CarlesUIPath Oct 1, 2026
01f61b4
feat(docker): 3/5 — derive the llm_only egress allowlist on the host
CarlesUIPath Oct 1, 2026
ef3e112
feat(docker): 4/5 — run llm_only containers behind a per-task egress …
CarlesUIPath Oct 1, 2026
d8c989d
docs(docker): 5/5 — document network: llm_only and add a negative egr…
CarlesUIPath Oct 1, 2026
781d89c
docs(tasks): list docker_egress_llm_only among the smoke members
CarlesUIPath Oct 1, 2026
23c16dc
fix(docker): code review fixes for network: llm_only
CarlesUIPath Oct 1, 2026
b9c2457
fix(docker): allow only the model hosts each llm_only task uses
CarlesUIPath Oct 1, 2026
79a3d33
fix(docker): code review fixes for network: llm_only
CarlesUIPath Oct 1, 2026
d24918b
test(tasks): add a thorough egress probe task for network: llm_only
CarlesUIPath Oct 2, 2026
cc0cf72
test(tasks): probe the LiteLLM model host in the egress task
CarlesUIPath Oct 2, 2026
ca78e6c
fix(deps): bump urllib3 to 2.8.0 and virtualenv to 21.7.13 for pip-audit
CarlesUIPath Oct 2, 2026
2aae6d7
test(docker): resolve the CodeQL alerts in the egress tests
CarlesUIPath Oct 2, 2026
b76722d
test(docker): make the bad-region setup-error test independent of loc…
CarlesUIPath Oct 2, 2026
26fe491
test(docker): consolidate the egress tests
CarlesUIPath Oct 2, 2026
9dd135d
docs(docker): record the OpenCode and Pi llm_only runs
CarlesUIPath Oct 2, 2026
93418f0
test(docker): keep side effects out of asserts in the egress runner t…
CarlesUIPath Oct 2, 2026
ff443ac
chore(docker): remove redundant parts of the llm_only change
CarlesUIPath Oct 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/harness-candidates.md
Original file line number Diff line number Diff line change
Expand Up @@ -1032,3 +1032,7 @@ re-derive from scratch.
stayed at 1 so `max_turns` never tripped — all silently, with `make verify` green.
A static rule cannot see an SDK field go dead; a live-telemetry smoke (one real turn,
assert `token_usage` is non-empty) in the harness-bump checklist would have.
- [ ] Validation regexes anchored with `$` and applied with `.match` (not `.fullmatch`) accept a trailing newline — `"evil.com\n:443"` passed `normalize_egress_target` until it moved to `fullmatch`. A lint rule needs to tell a validation regex from a search regex; not cheap — caught in the network: llm_only Phase 1 review.
- [ ] `raise X(...) from exc` where `exc` came from parsing an env/URL value leaks the value through `__cause__` in a traceback (urlsplit put a URL password in its port error). Needs data-flow from env reads; not cheap — caught in the network: llm_only Phase 3 review.
- [ ] `await asyncio.to_thread(subprocess.run, ...)` that CREATES a resource inside a try whose `finally` removes it: a cancel leaves the worker thread running, so teardown races the create and leaks it (fixed in `isolation/egress.py` by `_run_to_completion`). Detecting "creates a resource" is not mechanical — caught in the network: llm_only Phase 4 review.
- [ ] The single-`asyncio.shield` join in `fs_permissions.set_permissions` does not survive a SECOND cancel of the caller; `isolation/egress._run_to_completion` loops on the shield. Promote that helper to a shared module and use it in both places — caught in the network: llm_only Phase 4 review.
123 changes: 123 additions & 0 deletions .claude/notes/isolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -1036,3 +1036,126 @@ separator a value beginning with `-` is parsed as an OPTION rather than a reposi
(`--upload-pack=…` runs a command of the caller's choosing). That URL is task-authored, and
since `evaluate <run_dir>` rebuilds the task from a shareable run directory it is no longer
necessarily the operator's own string.

## The egress sidecar (network: llm_only)

`network: llm_only` puts the task container on a per-task `--internal` network whose only exit
is a proxy sidecar that forwards to an exact `host:port` allowlist. The plan and its spike log
are in `c/2026-10-01-docker-no-internet-egress.md` (not committed); the facts that shaped the
design are below.

### Why an explicit proxy, not a transparent one

Harbor's design (a `gost` sidecar, nftables `redirect`, SNI sniffing, the task in the sidecar's
netns with `NET_ADMIN`) also catches tools that ignore proxy env. Phase 0 found no built-in
harness that needs it: Claude Code (Bun native binary), Codex (Rust reqwest), Antigravity (Go
`ProxyFromEnvironment`) and Pi (undici `EnvHttpProxyAgent` with `NODE_USE_ENV_PROXY=1`) all
honour `HTTPS_PROXY`. An explicit proxy needs no capability, no third-party image and no SNI
parser, and a tool that ignores it has no route, so it fails closed. iptables inside the task
container was rejected because it needs `NET_ADMIN` in the agent's own container, which lets the
agent undo it.

### Why a per-task network

A shared internal network lets task A use task B's sidecar (and its allowlist) and reach task
B's ports. One network per task costs one address pool each: a default Docker Desktop ran out at
the 30th network (`all predefined address pools have been fully subnetted`), which is why the
error names `--max-parallel` and `default-address-pools`.

### Why inhibit_ipv4 and ip_forward=0

On native Linux a default `--internal` bridge still holds a gateway IP on the host, and the task
container reached every host service bound on `0.0.0.0` through it (dockerd 20.10, 24 and 29,
both firewall backends; on 20.10 and 24 also `docker0` and the host NIC). The
`com.docker.network.bridge.inhibit_ipv4=true` option removes the gateway; the sidecar then takes
the `.1` address and the embedded DNS alias still works. `gateway_mode_ipv4=isolated` was
rejected: Docker 24 ignores it silently. Docker Desktop never showed the hole.
`--sysctl net.ipv4.ip_forward=0` on the sidecar is defence in depth: Docker enables forwarding in
the sidecar's netns, so a task container with `NET_ADMIN` could try to route through it. No
config field grants the task container run-time capabilities, and the measured route did not
reach the internet, but the sysctl costs nothing.

### Why a black-hole DNS server and --ipv6=false

The spikes ran on dind, which has no loopback resolver, so they could not show CVE-2024-29018:
on daemons before 26.0 / 25.0.5 / 23.0.11 the embedded DNS forwards an internal network's
queries from the host namespace when the host resolver is loopback (systemd-resolved), which is a
DNS tunnel out. The task container's `--dns 192.0.2.1` (TEST-NET-1, never routed) keeps the
embedded DNS answering the sidecar alias while every external forward goes nowhere.
`inhibit_ipv4` removes only the IPv4 gateway, so `--ipv6=false` keeps a daemon whose
`default-network-opts` enable IPv6 from giving the internal bridge an IPv6 gateway. IPv6
link-local reach to a host service was not measured.

`NET_RAW` is dropped from the task container for the same reason: with raw sockets an agent could
write frames for the bridge's own MAC address and reach a host service over UDP without the
sidecar. That path was not measured; the cap drop removes it at no cost, since no tool needs raw
sockets under `llm_only`.

### Why the sidecar log is a separate, atomically written file

The first version appended the sidecar log to `docker.log` after the container exited. That file
is in the run directory, which is bind-mounted writable into the container at the SAME path, so
the agent could replace it with a symlink to a host file (for example a shell rc file) and get a
line it chose (`DENY $(cmd):443 CONNECT` passes the visible-ASCII check) appended there by the
host. Container stdout could also forge `ALLOW` / `DENY` lines in it. The log is now `egress.log`,
written once by `write_text_atomic`, whose `os.replace` replaces a planted symlink instead of
following it. `BAD` lines carry only the method and length, because a refused target can hold
credentials in its userinfo or query.

### Why the framework image and a bind-mounted module

The sidecar image is the framework image, never the task image: a task image is task-authored and
may be a runtime-kit image with a different Python. The proxy code is NOT taken from the image:
the host bind-mounts its own `egress_proxy.py` read-only, so the code under test is the boundary
that runs and image skew cannot change it. That is also why the module is stdlib-only (five
imports, guarded by a test): the image only supplies `python3`.

### Why the sidecar watches the heartbeat

A host SIGKILL skips every `finally`. The task container already exits on a stale host heartbeat;
the sidecar reads the same file through a single-file `:ro` bind (Docker Desktop VirtioFS saw
every in-place counter write) and stops itself with the same counter-or-mtime rule as
`heartbeat_is_alive`. It has no `--rm`, so a crashed or stale-stopped sidecar keeps its log until
teardown reads it; `docker network rm` works with a stopped container still attached, so a later
prune needs no ordering. The watchdog returns from `serve` instead of calling `os._exit`, because
CE052 gates process-lethal calls on `IN_CONTAINER_ENV`, which a stdlib-only module cannot import.

### Why every docker call runs to completion

`_docker` runs `subprocess.run` in a worker thread, and cancelling the await does not stop the
thread. A Ctrl-C during `docker create` once let teardown run `docker rm -f` before the create
finished, leaking a created-but-never-started sidecar that no heartbeat would ever stop. So every
call is joined across any number of cancels (`_run_to_completion`) before the cancel propagates,
and teardown is joined the same way.

### The log line format is a security boundary

`docker.log` is the only evidence the boundary leaves, and operators grep it for
`^(ALLOW|DENY|FAIL)`. A request line with a control or non-ASCII byte is refused before anything is
logged, because a lone `\n` in a method once forged a separate `ALLOW` line.

### Why the allowlist follows the model callers, not the forwarded env

The first version allowed the API backend's hosts for every agent and the host of every forwarded
`*_URL` variable. The real runs showed the cost: a Claude task could reach the Azure
`CODEX_BASE_URL` host, and a Codex, Antigravity or Pi task could reach `api.anthropic.com`,
because those variables are on the default passthrough list and the default backend is `direct`.
Each host was a model provider, but none was needed. So the backend's hosts are added only for a
component that uses `API_BACKEND` (`uses_api_backend` on the agent config, an enabled simulator,
an `llm_judge` / `agent_judge` criterion), and a URL variable counts only where its owner reads it
(`LITELLM_BASE_URL` for the litellm backend, `CODEX_BASE_URL` in `CodexAgentConfig.egress_hosts`).
Any other host is explicit in `egress_allowlist`.

### What Phase 0 measured

- Every built-in harness in the image and every judge route (`llm_judge`, `agent_judge`,
litellm with aiohttp) works through the proxy with an exact host set.
- Claude Code on Bedrock calls the control plane `bedrock.<region>.amazonaws.com` at every start;
it is allowed so the CLI behaves as under `bridge`. `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC`
does not remove that call, and on direct its Datadog `DENY` did not slow the CLI, so it is not set.
- Claude Code OAuth refresh goes to `platform.claude.com` (binary string `TOKEN_URL`), which is on
the direct backend list because `agent_judge` runs the CLI too.
- `LITELLM_LOCAL_MODEL_COST_MAP=True` costs no latency either way (the `403` is immediate); it is set
to remove one `DENY` per process and a network-dependent cost map.
- busybox `wget` sends absolute-form `GET https://…` to a proxy instead of `CONNECT`; the proxy
answers `400` with a `BAD … absolute-form https (use CONNECT)` line rather than originate TLS.
Loading
Loading