[draft] Design: workspace.exec runs in the container's initial mount namespace #3

Draft — review before filing upstream. Design discussion; no code.

What we found

workspace.exec does not go through the daemon. layerfs-node/src/exec.rs says "the daemon is not involved"; layerfs-api/src/sandbox/exec.rs says "This sends no daemon request". It creates a Docker Engine exec in the container's initial mount namespace, with the Workspace directory as the working directory (registry.rs -> the mount point). backend.daemon.process.start uses the same path.

Why it matters

Any change that gives the daemon a private mount namespace (see the experiment in Ephemeral-AI-Lab#499; root cause in Ephemeral-AI-Lab#460) removes /workspaces/* from the namespace where exec starts, so every command would run in an empty mountpoint. That is not an external-debugging trade-off; it is the product's command path — a use-case boundary move, not a mount flag.

Options

1. Daemon exec: the daemon spawns the command (or via a helper that setns-es into its mount namespace) and owns output/exit; the Engine exec path is no longer used.
2. setns helper: a small in-container binary joins the daemon's namespace and execs; keeps the Engine exec entry for launching.
3. Don't move the mount: keep the abort path (fix(daemon): end a stalled normal unmount at Join with its own abort write (#426, #460) Ephemeral-AI-Lab/layerfs#496 and fix(fuse): bound the receive-loop join by a 1 s grace (#426, #460) Ephemeral-AI-Lab/layerfs#497) as the answer.

Recommendation

Prefer option 2 (setns helper) as the first experiment: it keeps the Engine exec entry, output streaming, cancellation and exit codes unchanged, and only adds a namespace join before the command. It needs CAP_SYS_ADMIN and the same user namespace (the Sandbox has both) and must run before chdir into the Workspace. A one-day check should prove it against the same 6/6 reproducer.

If upstream prefers one thread, this issue can be folded into Ephemeral-AI-Lab#499's "Known break" section instead of staying separate.

Note on compatibility: we found no supported workflow that relies on docker exec seeing the mount; one benchmark doc explicitly forbids docker exec as an editing path (core/docs/benchmark/fs-bench-pro/exec2edit.md). The compatibility question is narrower than it first looks.

Prototype

Implemented in #5 (branch feat/ns-exec-helper, 8545a284): a hidden layerfs-daemon ns-exec --pid-file … --directory … --uid … --gid … -- bash -c <cmd> that opens /proc/<pid>/ns/mnt, setns(CLONE_NEWNS), re-enters the directory, drops to the command identity, then execs; the daemon gains an opt-in private_mount_ns and writes its pid file; the API wraps the Engine argv only when the Sandbox record carries that pid file.

What the prototype changes vs this design:

• wrapped Execs start as root (the command identity cannot open the namespace fd — measured EPERM), and the helper drops to the ordinary uid/gid before exec;
• the helper takes --directory because the Engine's WorkingDir resolves in the initial namespace, where the mountpoint is only the covered directory.

E2E run (2026-10-28, fd passing, bundle 8fdaaf92, LAYERFS_PRIVATE_MOUNT_NS=1): PASSED

• early mount, n=10: workspace.exec 10/10 exit 0, pwd = /workspaces/1, marker written/read/committed 10/10; exec p50 54.2 ms; unmount p50 15.0 ms (7.2–35.0), 10/10 Unmounted; close 10/10.
• the command runs in the daemon's namespace: readlink /proc/self/ns/mnt = the daemon's mnt:[4026533424]; the FUSE row is visible; uid 501 / gid 20; exit codes and stdout/stderr pass through.
• control, flag off, n=5: exec OK, unmount 5/5 refused after p50 5036 ms (the old wedge).
• delayed mount, n=5: exec p50 101.5 ms, unmount p50 20.0 ms; normal path, n=5: exec p50 37.4 ms, unmount p50 10.8 ms.
• old record across the upgrade: clear Unsupported refusal; recreate the Sandbox.

The earlier pid-file E2E failed 10/10 with EACCES; that route is removed.

Outcome: the fd-passing route is implemented in #5 (ready for review); the daemon-spawned-exec alternative remains unbuilt. The open decision is whether upstream wants this direction at all — otherwise the abort path (Ephemeral-AI-Lab#496/Ephemeral-AI-Lab#497) stays the answer.

What a decision needs

• which option keeps output streaming, cancellation and exit codes unchanged;
• a real-topology test: daemon mount ns != PID 1's; /workspaces/* absent in the init ns; workspace.exec still lands in the workspace;
• whether docker exec visibility is a supported behaviour today (docs/tests).

Diagrams

Overview

Sequence