Repository navigation
feat(sandbox): 用具体的 Runtime enum 统一 Docker / Linux 原生 / macOS 原生，并新增 Linux namespaces 后端（B1/B2/B3，按 #515，叠在 #427 上） - #498
Closed
jiangkoumo wants to merge 96 commits into
Closed
jiangkoumo wants to merge 96 commits into
jiangkoumo wants to merge 96 commits into
Conversation
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/Cargo.toml (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 108,451 -> 108,451 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production). No tests changed or deleted. Production LOC: 108,451 -> 108,460 (delta +9) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production). No tests changed or deleted. Production LOC: 108,460 -> 108,505 (delta +45) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production). No tests changed or deleted. Production LOC: 108,505 -> 108,576 (delta +71) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production). No tests changed or deleted. Production LOC: 108,576 -> 108,834 (delta +258) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production). No tests changed or deleted. Production LOC: 108,834 -> 109,057 (delta +223) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production). No tests changed or deleted. Production LOC: 109,057 -> 109,166 (delta +109) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production). No tests changed or deleted. Production LOC: 109,166 -> 109,167 (delta +1) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,167 -> 109,167 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/runtime.rs (production). No tests changed or deleted. Production LOC: 109,167 -> 109,274 (delta +107) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/local.rs (production). No tests changed or deleted. Production LOC: 109,274 -> 109,527 (delta +253) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/registry.rs (production). No tests changed or deleted. Production LOC: 109,527 -> 109,527 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/discovery.rs (production). No tests changed or deleted. Production LOC: 109,527 -> 109,644 (delta +117) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/exec.rs (production). No tests changed or deleted. Production LOC: 109,644 -> 109,633 (delta -11) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/sandbox.rs (production). No tests changed or deleted. Production LOC: 109,633 -> 109,641 (delta +8) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/workspace.rs (production). No tests changed or deleted. Production LOC: 109,641 -> 109,636 (delta -5) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/lib.rs (production). No tests changed or deleted. Production LOC: 109,636 -> 109,638 (delta +2) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/tests/unit/runtime.rs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,638 -> 109,638 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/src/client.js (production). No tests changed or deleted. Production LOC: 109,638 -> 109,656 (delta +18) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/src/index.js (production). No tests changed or deleted. Production LOC: 109,656 -> 109,658 (delta +2) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/test/local_deployment.mjs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,658 -> 109,658 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/README.md (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,658 -> 109,658 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production). No tests changed or deleted except additions. Production LOC: 109,658 -> 109,696 (delta +38) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/groups.rs (production). No tests changed or deleted except additions. Production LOC: 109,696 -> 109,818 (delta +122) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production). No tests changed or deleted except additions. Production LOC: 109,818 -> 109,835 (delta +17) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production). No tests changed or deleted except additions. Production LOC: 109,835 -> 109,868 (delta +33) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production). No tests changed or deleted except additions. Production LOC: 109,868 -> 109,882 (delta +14) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production). No tests changed or deleted except additions. Production LOC: 109,882 -> 109,870 (delta -12) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production). No tests changed or deleted except additions. Production LOC: 109,870 -> 109,871 (delta +1) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production). No tests changed or deleted except additions. Production LOC: 109,871 -> 109,872 (delta +1) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
… forwards (H27)
Unify layerfs-node behind DockerProvider + ProviderCreate/ProviderSandbox,
sharing the create_provider_file startup and installation pipeline with
the native provider, and remove the deprecated H26 compatibility forwarders
from layerfs-api.
- layerfs-node:
- sandbox.rs: start_claimed routes Docker launches through DockerProvider
and create_provider_file, using ProviderCreate with ProviderLaunchOptions::Docker;
abandon/settled adapt to ProviderFailure while keeping identical refusal
detail, settled criteria, and stop(1) cleanup semantics.
- runtime.rs: Runtime::Docker wraps Box<dyn SandboxInstance>, aligning with
Local; Runtime::run unifies on sandbox.run(...) with identical streaming,
exit code observation, and absence of manual exec creation boilerplate.
- discovery.rs: Recorded::Docker holds DockerProvider, ContainerId, and
SandboxRecord; reopen and reopen_stopped route through DockerProvider;
sandbox.record persistence formats and labels remain byte-for-byte identical.
- layerfs-api:
- Delete deprecated H26 thin forwarders: SandboxApi::new(Docker),
SandboxCreate, ManagedSandbox, SandboxFailure, create_file, close_managed,
WorkspaceApi::from_managed, and internal::control_managed/split/exec.
- Streamline SandboxApi to wrap Arc<dyn SandboxProvider> directly; prune
unused SandboxCause::Create/Upload/Listener variants.
- Behavior preservation:
- Exec streaming/exit codes, 5-second graceful sandbox.close timeout,
1-second create abandon stop, sandbox.record schema, and typed refusals
remain strictly preserved.
Production LOC: 110441 -> 110143 (delta -298)
- engine: 66362 -> 66064 (delta -298)
- layerfs-api: 2375 -> 2074 (delta -301)
- layerfs-node: 3720 -> 3723 (delta +3)
- core: 41400 -> 41400 (delta 0)
- api: 2679 -> 2679 (delta 0)
- reference absent (retired R9c; last counted 65,417 at tree 498dd19)
Method: python3 tools/production_loc.py --root against parent 70ec59e and committed tree.
Poll endpoint through the Sandbox that owns the Child before asserting that no zombie remains. Bound the observation by WAIT and preserve the stopped and not-gone assertions without changing product behavior. Validation: cargo +1.85.1 fmt --all -- --check; cargo +1.85.1 clippy --locked -p layerfs-sandbox --all-targets -- -D warnings. Both passed on macOS with this worktree private target. Linux-only runtime tests were not executed; Hark owns Linux verification. Production LOC: 109210 -> 109210 (delta +0). Scope: core 41400, engine 65131, API 2679; reference absent (retired R9c; last counted 65,417 at tree 498dd19). Tests, tools, docs, dependencies and generated artifacts excluded; inline test code excluded by the counter. Method: tools/production_loc.py at Git blob 3341e4d; export parent 3147916 and final staged tree d97b40d using git archive, then run python3 -B tools/production_loc.py --root <each-export> --json. Both invocations exited 0.
… test fix) into B1
jiangkoumo
marked this pull request as ready for review
October 29, 2026 07:13
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
…ds by platform - Replace the SandboxProvider/SandboxInstance traits with `backend::Runtime` (Docker, MacosNative, LinuxNative and the Windows placeholder) plus the launched `Sandbox` handle. Each backend keeps its concrete type and one `match` dispatches. `Runtime::Windows` refuses with the existing `RuntimeError::Unsupported` before any effect; no public error code is added. - Move the shared native lifecycle to backend/native, the Darwin process, mount and force-detach facts to backend/macos, the /proc facts and lazy detach to backend/linux, and add backend/windows/README.md as the placeholder plan (WSL2 worker and VM placement routes). - layerfs-api takes `Arc<Runtime>` (`with_runtime`) and keeps the handle in ProviderSandbox; layerfs-node keeps the handle in its Sandbox state and reads the record, image, scratch directory and process tracking through it. - Behaviour is unchanged: Docker stays the default, and Sandbox records, refusals, ordinary exec streams and the one-second close grace are identical. No test is deleted; the Docker, native provider and lifecycle tests now drive the enum. Production LOC: 110211 -> 110222 (delta +11); engine 66132 -> 66143 (delta +11); reference absent (retired R9c; last counted 65,417 at tree 498dd19). Scope: first-party production source (core/engine/api src; tests, examples, docs, tools and manifests excluded) at parent 5e5486f -> this tree, counted with the unchanged tools/production_loc.py.
Add the Linux namespaces runtime backend to layerfs-sandbox, placing each Sandbox in dedicated mount, PID, and IPC namespaces with a private cgroup v2 hierarchy, and executing commands under non-root uid:gid with no capabilities and no-new-privileges. - Introduce `Runtime::Namespaces(NamespaceProvider)` and `Sandbox::Namespaces` in `backend/runtime.rs`, guarded with `#[cfg(target_os = "linux")]`. - Add `backend/namespaces/` managing private root tmpfs construction, cgroup v2 lifecycle with `cgroup.kill` teardown, and process group supervision. - Implement `layerfs-sandbox-helper` single-threaded binary in safe Rust via nix, handling namespace transitions, pivot_root, and dropping command privileges. - Enforce host preconditions before state changes: root execution, cgroup v2, FUSE character device access, and blocked user namespaces (`user.max_user_namespaces == 0`). - Wire `ProviderLaunchOptions::Namespaces` and `SandboxRecord::Namespaces` across `layerfs-api` and `layerfs-node`, allowing selection via `sandboxBackend`. Production LOC: 110222 -> 111918 (delta +1696) Scope: first-party production source counted by tools/production_loc.py Breakdown: core: 41400 -> 41400 (delta +0) engine: 66143 -> 67837 (delta +1694) api: 2679 -> 2681 (delta +2) reference: absent (retired R9c; last counted 65,417 at tree 498dd19)
The Linux-only namespaces record reader mapped the provider precondition failure to a refusal that dropped the cause, leaving its binding unused (clippy -D warnings on Linux). Carry the provider error into the message so a host without root, cgroup v2, /dev/fuse or blocked user namespaces says which condition refused the Sandbox. Production LOC: 111918 -> 111921 (delta +3) Scope: first-party production source counted by tools/production_loc.py Breakdown: core: 41400 -> 41400 (delta +0) engine: 67837 -> 67840 (delta +3) api: 2681 -> 2681 (delta +0) reference: absent (retired R9c; last counted 65,417 at tree 498dd19)
jiangkoumo
changed the title
Collaborator
Author
|
更新：按 #515 的方向改完了，分支新加了 3 个提交（5e5486fc8 → bdc341a，没有 force-push）。
验证：Linux（x86_64 和 aarch64 VM）上 namespaces 三组测试 11/11，安全项（uid、CapEff、NoNewPrivs、unshare -U、Sandbox 互相隔离、无残留）全部实测通过；macOS / Linux 的全仓 clippy、各项测试和 SDK smoke 都过；真实 Docker 9 个场景和基线一致。 下一步打算做 runtime 侧的 provider adapter（Microsandbox / Apple Container），按 #515 在宿主机起 VM 并打通私有通道。 |
…#515) Add the `vm:<provider>` placement to @layerfs/runtime (LAYERFS_PLACEMENT, or the `placement` setting; `local` stays the default and is unchanged): one Linux VM per home holds every project's Store on a named ext4 disk volume, serves the home's worker, and hosts the Sandboxes as Linux namespaces (LAYERFS_SANDBOX_BACKEND=namespaces). The host reaches only the worker's loopback-published port, with one attempt per call and no replay. The adapter (api/runtime/src/platform/vm/) attempts each of Ephemeral-AI-Lab#515 §6's five steps once: ensure the VM exists and runs (create or start, with the Store disk and a bind mount of the home); copy the sealed bundle in once per bundle hash; block user namespaces (V4, written and read back) and start the worker as root with its secret in a 0600 guest file, never in a command line; write the endpoint useWorker reads from the VM's private port (V5: Sandboxes share the VM's network, so daemon ports stay inside); and stop or delete the VM (V8: a delete is refused while any project of the home exists, because its Store is on that disk and nowhere else and this build has no remote Store push). Providers: microsandbox (`msb`, verified end to end in a real microVM) and apple-container (`container`, UNVERIFIED: that CLI is not installed on the machine this was written on; its inspect mapping refuses a shape it does not know instead of guessing, so a changed CLI is an error and never a VM the adapter believes is running). Runtime changes: four Store hooks (makeStore, storeHolders, releaseStoreHolders, removeStore) replace the inline Docker calls of project init and removal, with byte-identical Docker implementations for the local platforms; status() reports `placement`, the VM and `docker: { level: "not-used" }`; the Docker-shaped helpers refuse a VM home by name (volume.list/remove, sandbox.list, list_leftovers, clean_leftovers) and sweep answers `{ host: false }`; runtime.vm adds status/stop/delete for a VM home only. Also: the SDK client reads LAYERFS_SANDBOX_BACKEND once and sends it with every call, so a Sandbox made with the namespaces backend is not reopened as the other local backend (a real VM refused exactly that on project.commit); and the Linux sealed bundle gains layerfs-sandbox-helper, which the namespaces backend needs beside the daemon (and LAYERFS_SANDBOX_HELPER for a Sandbox reopened from its record). Tests: api/runtime/test/vm_placement.mjs (settings, the five steps, bundle hash, V8 delete, host-Docker refusals, apple-container's flags; stand-in provider CLIs), api/runtime/test/verify/vm_real.mjs (a real microVM: Init through the VM worker, a namespace Sandbox, exec as 501, the V4 block, a Commit, the V8 refusal, VM stop/start, a worker restart, a second isolated Sandbox), the existing api/runtime tests 14/14, and the macOS smoke 48/48 with placement unset. Production LOC: 111921 -> 112431 (delta +510). Scope: first-party production source under core/, engine/ and api/ (each api package's src/), counted by tools/production_loc.py; tests, docs, tools and node_modules excluded. api 2681 -> 3191; core and engine unchanged. Reference: absent (retired R9c; last counted 65417 at tree 498dd19).
… layer Production LOC: 111,918 -> 113,125 (delta +1,207) core: 41,400 -> 41,400 (delta 0) engine: 67,837 -> 69,037 (delta +1,200) api: 2,681 -> 2,688 (delta +7) reference: absent (retired R9c; last counted 65,417 at tree 498dd19) Counting method: python3 tools/production_loc.py at first parent bdc341a and staged tree.
Strictly follow proposal Ephemeral-AI-Lab#515's "one Linux VM per Store" (V1): - Each project gets its own Linux VM (<prefix><project>-vm), its own Store disk (<prefix><project>-vm-store), host port, bundle copy, and worker secret. - runtime.project.start_init creates the project's VM and disk; runtime.project.remove removes both together. - runtime.worker.start(project) and restart(project) act on that project's VM. - runtime.vm.status(project?), stop(project), delete(project) operate per project; status() lists every project VM with its state, worker and disk free bytes. - useWorker widen to accept an endpoint resolver (project => file) so one client process can direct Store calls to each project's VM worker. - deploymentEnv sets WORKER_ENDPOINT for subprocess agents to that project's VM. - Reserve V7 bundle Sandbox image paths (LAYERFS_IMAGE / LAYERFS_IMAGE_ARCHIVE) for forthcoming H38 namespaces backend merge. - Verified against Apple Container 1.6.1 (aa8d76b) on macOS 27.2: fixed inspect JSON array output (no --format flag), volume sizing, -i on exec stdin, and remount /proc/sys rw for V4 user namespace sysctl block. E2E verified on both microsandbox and apple-container real VMs. Production LOC: 112433 -> 112509 (delta +76) - core: 41400 -> 41400 (+0) - engine: 67840 -> 67840 (+0) - api: 3193 -> 3269 (+76) - reference: absent (retired R9c; last counted 65,417 at tree 498dd19) - combined: 112433 -> 112509 (delta +76)
…meral-AI-Lab#515) The `vm:<provider>` placement now runs the namespaces backend the way Ephemeral-AI-Lab#515 §4.7 V7 (H38) requires: the Sandbox root is the bundle's docker-archive image, its per-Sandbox overlay upper must sit on a local filesystem, and the client a VM's agent process runs as is given the image's digest *and* its archive. - `settings.js`: `GUEST.local` is `/layerfs/store/local`, on the project VM's ext4 Store disk instead of its root. Microsandbox's guest root is an overlay (`stat -f` / mountinfo: `overlay` over `/.msb/rootfs`), which `require_overlay_upper` refuses as an overlay upper, so every namespaces Sandbox in a Microsandbox VM would have refused to start; Apple Container's root is ext4, and the Store disk is ext4 on both. The Store's own directory (`/layerfs/store/global`) stays outside it, as `check_store` requires. - `platform/vm/index.js`: `deploymentEnv` also gives clients `LAYERFS_IMAGE_ARCHIVE` (the guest path of this VM's own bundle copy), which the namespaces backend needs beside `LAYERFS_IMAGE`; the worker already had both. Docs record the guest layout and that the image is required. - Tests: `vm_placement.mjs` (flow) asserts the archive path and the Store-disk Sandbox directory. `vm_real.mjs` proves, on both providers: the command's root is the image (`/etc/os-release`, the marker, no `/bin/node`, the VM's `/layerfs` not visible), the image's directories are not writable by the command while its own layer is, two Sandboxes' writes stay each one's, one digest is unpacked once (cache inode/mtime unchanged, mode 555, no `.tmp-*`), `LAYERFS_LOCAL_DIR` is on ext4, and a VM restart leaves a stopped Sandbox record that is not reopened onto a new root, is reclaimed by the next `sandbox.create` (its number returns) and runs from the same unpack. Merge: this branch also merges `h38-oci-rootfs` (27d92c5) with `--no-ff` (commit 9c8c1e0); no file conflicted, so there is no conflict resolution to record. One integration gap surfaced there and is fixed here (the archive path above) plus the `/layerfs/local` -> Store-disk move that H38's `require_overlay_upper` makes necessary in a real VM. Production LOC (tools/production_loc.py, nonblank non-comment first-party product source; tests, docs and tools excluded): vs H37b head 06ae574 (the branch's first parent's tree before the merge): core 41,400 -> 41,400 (delta 0) engine 67,840 -> 69,037 (delta +1,197) H38's namespaces rootfs work api 3,269 -> 3,276 (delta +7) H38's image-archive deployment reference absent (retired R9c; last counted 65,417 at tree 498dd19) combined 112,509 -> 113,713 (delta +1,204) vs H38 base bdc341a: core 41,400 -> 41,400 (delta 0) engine 67,840 -> 69,037 (delta +1,197) api 2,681 -> 3,276 (delta +595) H37b's per-Store VM work reference absent (retired R9c; last counted 65,417 at tree 498dd19) combined 111,921 -> 113,713 (delta +1,792) this commit alone (9c8c1e0 -> here): api 3,276 -> 3,276 (delta 0): a path value and one more field on an existing line; comments and tests are excluded. Checks: cargo fmt --check; clippy --all-targets -D warnings (macOS); clippy --target x86_64-unknown-linux-gnu -p layerfs-sandbox -D warnings; tools/check_product_boundary.py PASS (1029 files); tools unittest 155 OK; api/runtime/test/runtime.mjs 14/14; vm_placement.mjs 7/7; api/agent/test/packaging.mjs 8/8 (one case's `await using` spelled out for Node 22's parser in a temporary copy; the file is unchanged); macOS smoke 48/48 with this tree's release binaries; vm_real.mjs microsandbox 12.3 s and apple-container 47.5 s, both ok.
…meral-AI-Lab#515) Adds api/runtime/src/platform/vm/ssh.js so a LayerFS home can be placed on an existing Linux host reachable over SSH (EC2, local VM, or bare metal; Ephemeral-AI-Lab#515 §6, V3): - platform/vm/ssh.js: implements the five adapter steps over SSH with the system ssh client. The host kernel is verified (uname -s is Linux), the user-provided Store disk is checked (findmnt or blkid is ext4) and attached to /layerfs/store, the bundle is uploaded via tar streaming, the worker runs as root with its secret and user namespaces blocked, and the private stream is an ssh -L port forward with -o ExitOnForwardFailure=yes. StrictHostKeyChecking defaults to yes; key-only authentication via identity file paths (no passwords, no raw keys in argv/logs). One host serves one Store (refuses Busy on collision). Stops and deletes clean up the forward, bundle, and worker state while preserving the user's Store disk. - platform/vm/apple-container.js: cleans up newly created named Store volumes if create fails (e.g. port conflict), fixing the orphan volume issue from H39v. - platform/vm/microsandbox.js: cleans up ~/.microsandbox/sandboxes/<name> if create fails, fixing the hidden state issue from H39v. - settings.js: registers vm:ssh provider, validates host, user, port, identityFile, storePath, and strictHostKeyChecking settings. Missing host is reported in missing(). - Tests: vm_placement.mjs adds ssh and cleanup suites (9/9 pass). New test/verify/ssh_faults.mjs verifies the complete failure matrix (port collision, missing store path, non-ext4, non-Linux, unreachable drop, disconnect & reconnect, remote host reboot & reuse, delete with and without unpushed Store, shared host collision). All 9 scenarios pass. Real E2E runs against live Linux VM over SSH (7.46 s total, agent phase 1.26 s), verifying workspace scenario parts, image root read-only, two Sandboxes isolated, single unpack reuse, and ext4 filesystem. Production LOC (tools/production_loc.py, nonblank non-comment first-party product source; tests, docs and tools excluded): vs 5d9dc29 (HEAD of h40-ssh): core 41,400 -> 41,400 (delta 0) engine 69,037 -> 69,037 (delta 0) reference absent (retired R9c; last counted 65,417 at tree 498dd19) api 3,276 -> 3,619 (delta +343) dsh 0 -> 0 (delta 0) combined 113,713 -> 114,056 (delta +343) Crate breakdown (api scope): api/runtime/src/platform/vm/ssh.js +284 (new) api/runtime/src/settings.js +32 (85 -> 117) api/runtime/src/platform/vm/apple-container.js +13 (78 -> 91) api/runtime/src/platform/vm/microsandbox.js +12 (60 -> 72) api/runtime/src/platform/vm/index.js +2 (310 -> 312) Checks: cargo fmt --check; cargo clippy --workspace --all-targets --no-deps (0 warnings); tools/check_product_boundary.py PASS (1030 files); tools unittest 56 OK; api/cli/test/cli.test.mjs 75/75 PASS; api/runtime/test/runtime.mjs 14/14 PASS; api/runtime/test/vm_placement.mjs 9/9 PASS; api/runtime/test/verify/ssh_faults.mjs 9/9 PASS; vm_real.mjs microsandbox 12.1 s and apple-container 51.0 s, both OK; real Linux VM E2E over SSH (e2e-ssh.mjs) 7.46 s, all checks PASS.
Collaborator
Author
|
更新：按 #515 把 VM 这部分做完了，分支新加 6 个提交（bdc341ac8 → 9aa10e6，没有 force-push）。
验证：Mac 上 Microsandbox / Apple Container 真 VM E2E 全过， |
This was referenced Oct 31, 2026
Collaborator
Author
yifanxuaaa
pushed a commit
that referenced
this pull request
Oct 31, 2026
… B3) Integrate #427 at its branch head fa02a1b, keeping its 78 commits and their authorship, and place the macOS native backend on B1's single Runtime/Sandbox enum as Runtime::MacosNative(NativeMacosProvider). This is the B3 slice of the #498 split: the platform-independent implementation (backend/native/*, the node pipeline, the api/agent settings) is already in B1 a32da57, so this merge only adds macOS platform facts plus #427 itself. Conflicts (16 files) follow the H42a-2 P2-A rule: layerfs-node/src's shared files (local, discovery, runtime, sandbox, workspace, exec, registry) take the B1 side, unmount.rs takes the #427 side, and every other file takes the 9aa10e6 text, which is what B3 is specified to add. #427's backend/local/ (9 files) is removed: native/ + macos/ + B1's linux/ replace it. The merged tree equals the rehearsal-b3 tree (fee449554, tree ed2356142) except for five files that come from #427's three commits after the split's comparison base 2518769: the noatime mount option and its mounter test in layerfs-daemon/layerfs-nfs, and the noatime/WAL documentation in api/docs/agent.md, docs/README.md and docs/general/sqlite-on-network-filesystems.md. Those are #427 content and stay; they disappear from this PR's diff once #427 is on main. Per-workspace unmount leases, NFS export control, layerfs-nfs, the macOS Store install and kernel guard are #427 unchanged. D2 macOS stays as #427 implements it (Store on the host, loopback NFSv3 mount); D13 is NFSv3 now with FSKit as follow-up. No Durable execution. Deleted tests: none. Retained #427 test targets: layerfs-nfs's 8 binaries and mounter, layerfs-node's Linux-gated native_* tests (0 tests on macOS). Validation: macOS 27.2 ARM64, Rust 1.85.1, locked dependencies, repository ARM64 flags, every Rust test command under a 120 s wall timeout after a --no-run build. cargo fmt --check, cargo check --workspace --all-targets, Clippy -D warnings for the workspace and for layerfs-sandbox at x86_64-unknown-linux-gnu, product-boundary guard (1009 files) and the 155 tooling self-tests passed. layerfs-sandbox 45 passed/1 ignored (local_stop_mount needs a real mount), layerfs-api 10 passed, layerfs-daemon 6 passed/2 ignored, layerfs-nfs 18 passed, layerfs-node 0 tests on macOS (Linux-gated), api/cli 75 passed, agent calls/exec_output and native_deployment passed. LAYERFS_SANDBOX=native release smoke (addon and daemon built with CARGO_PROFILE_RELEASE_STRIP=none): 48/48, Z1 m=0 d=0 e=0. Disposable only; Durable NOT_RUN (owner-disabled). Docker route NOT_RUN on this host (no Docker Engine); its behaviour is covered by the mock-engine and lifecycle tests above, which run as Runtime::Docker. No benchmark. Production LOC: 106022 -> 110634 (delta +4612) First-parent comparison: b1-interface a32da57 to prepared merge tree fcd69b9. Scopes: core 38395 -> 41400 (+3005, layerfs-nfs; the counter files it under core, the boundary guard calls it engine), engine 64938 -> 66545 (+1607: daemon +1122, node +231, sandbox +226, store-service +28), api 2689 -> 2689 (0), dsh 0 -> 0. Method: tools/production_loc.py blob 3341e4d, --root <exact exported tree> --json for both snapshots; identical counter and scope for both; first-party production Rust, runtime SQL and JavaScript; tests, docs, tooling, fixtures, generated and third-party files excluded. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
对应 #446 B 区的 B1（统一 Sandbox 接口）、B2（Linux 原生） 和 B3（macOS 原生） 的接口部分，并按 #515 的方案做了 Namespaces 后端、VM 放置（每个 Store 一台 Linux VM）、Sandbox 镜像根，以及 Microsandbox / Apple Container / SSH 三个 VM adapter（Daytona 之后再做）。按 #515 和 engine/AGENTS.md，原来的
SandboxProvider/SandboxInstance两级 trait 已改成具体的 enum：Runtime::{Docker, MacosNative, LinuxNative, Namespaces, Windows}Sandbox::{Docker, Native, Namespaces}backend/{docker, native, macos, linux, namespaces, windows}；windows目前只有 README，选它会返回已有的Unsupported错误layerfs-api和layerfs-node只走这一条分派路径。本 PR 叠在 #427 上面：分支从 #427 的
d55acc15f开始，之后 merge 过 #427 的251876979。评审时只看本 PR 自己的提交就行。#427 合并后，这部分 diff 会只剩下本 PR 自己的改动。main 目前是35d36575c，不需要再 merge。提交
ExecRequestNativeMacosProvider/NativeMacosInstanceNativeProvider<const LINUX: bool>DockerProvider/DockerInstance（create/inspect/run/log/stop/remove，只尝试一次，typed refusal）GET /version应答（上游 73d974d 加版本协商之后，夹具的回复错位，测试报 ConnectionRefused）ProviderLaunchOptions::Docker；用SandboxRecord枚举统一 local/docker 记录（磁盘格式不变）；去掉reopen_dockerSandboxApi只走 providerDockerProvider，删掉旧入口（ManagedSandbox、SandboxCreate、SandboxApi::new(Docker)、internal::split/exec等）Runtime/Sandboxenum（match 分派），backend/local按平台拆成 native / macos / linux，windows 占位layerfs-sandbox-helper（全部 safe Rust，用 nix，#![deny(unsafe_code)]）docker save未压缩 tar），digest 校验、只解压一次、whiteout 和路径/软链/设备节点拦截，每个 Sandbox 独立 overlay 上层runtime.vm.status/stop/delete(project)，删项目时一并回收 VM 和盘vm:sshadapter：用户自带 Linux 主机，tar 流上传 bundle、ssh -N -L私有转发、检查 Linux 内核和 ext4；顺带修 Apple Container 失败遗留卷、Microsandbox 遗留目录保持不变的行为
sandbox.record磁盘格式和 label 不变，不做迁移close仍是 5 秒，创建失败清理仍是 1 秒/proc检查和进程组清理验证（本地；上游 CI 结果以本 PR 页面为准）
macOS（M2，macOS 27.2，rustc 1.85.1）
cargo fmt --all -- --check、cargo clippy --locked --all-targets -- -D warnings：通过layerfs-sandbox、受影响的layerfs-api测试目标、layerfs-node：通过（跳过 10 万文件规模的 sdk_merge_scale）node --test api/cli/test/cli.test.mjs：75/75CARGO_PROFILE_RELEASE_STRIP=none）SDK smoke 48/48，PATH inherit 7/7、fallback 4/4Linux（Debian 13，root，真实 FUSE）
-D warnings、layerfs-sandbox全过layerfs-node：native_discovery 6/6、native_admission 2/2、native_local 7/7真实 Docker（Docker 29.8.2，runtime/sandbox 镜像
layerfs-352-runtime:node24）在 runtime 镜像里用
node api/agent/test/verify/<场景>.mjs inner跑（环境变量和tools/worktree_run.py一样，--network host）：两边的残留容器数也一致（idle_close 等场景按设计留下一个已停止的 Sandbox）。
Linux namespaces（Debian 13 x86_64 root；另在 Mac 的 Microsandbox Debian 13 aarch64 VM 里也跑过）
namespaces_protocol5/5、namespaces_userns1/1、namespaces_lifecycle5/5CapEff=0、NoNewPrivs=1；unshare -U和mount都会被拒绝stop用cgroup.kill回收 setsid 后台进程，结束后没有残留的 mount、daemon 或 cgroupuser.max_user_namespaces不是 0 时直接拒绝启动以上 macOS / Linux 的结果已在
bdc341ac8上重跑：fmt、全仓 clippy、sandbox / api / node 测试、CLI 75/75、SDK smoke 都过。真实 Docker 9 个 verify 场景已在9aa10e65c上重跑，结果和残留都和基线一致。VM 放置（#515，Mac M2）
/layerfs只看得到 Store、两个 Sandbox 的上层互相看不到、镜像只解压一次（inode/mtime 不变）、Store 盘是 ext4NoSuchSandbox，下次 create 时回收vm:ssh：经 SSH 转发对真实 Linux VM 跑通 SDK Workspace 14 步、隔离和只读镜像根检查，全程 7.5 秒；端口占用、Store 路径不存在、非 ext4、非 Linux、断线等故障都被拦下且无残留vm_placement9 组（settings/flow/two/bundle/delete/hosts/apple-container/ssh/cleanup）、ssh_faults5 项、runtime 14/14、packaging 8/8 全过设计说明
docker save的未压缩 tar），放在 bundle 的images/<digest>.tar；标准 OCI layout 和 gzip 层之后再补