Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
133145f
node: one Sandbox name for every backend (#527 step 5)
yifanxuaaa Oct 31, 2026
527c32f
sandbox: the provider alone chooses FUSE or NFS (#527 step 6)
yifanxuaaa Oct 31, 2026
c6f5dd2
sandbox: provider/ into types/, LaunchFailure into the runtime (#527 …
yifanxuaaa Oct 31, 2026
27d1784
runtime: vm: homes refuse Store-level calls before any effect (#527 s…
yifanxuaaa Oct 31, 2026
18b0810
runtime: both deployments name the Sandbox backend (#527 step 3)
yifanxuaaa Oct 31, 2026
4083e75
runtime, agent: refuse VM providers and native backends on hosts they…
yifanxuaaa Oct 31, 2026
7080ac9
runtime: platform capabilities replace platform.name branches in Init…
yifanxuaaa Oct 31, 2026
9952d16
docs: fix the platform documentation drift of F7 (#527 step 1)
yifanxuaaa Oct 31, 2026
24c3c5f
sandbox: unpack a Debian image's root entry and long link targets
yifanxuaaa Oct 31, 2026
781e1c0
runtime: vm:ssh copies carry no AppleDouble files
yifanxuaaa Oct 31, 2026
38ad8ab
node: reopen requires the deployment's own backend (#527 step 4)
yifanxuaaa Oct 31, 2026
50e2c2b
test: the twelve-cell runner for the #527 platform pass
yifanxuaaa Oct 31, 2026
cb7dc81
agent: refuse namespaces off Linux before reading its image settings
yifanxuaaa Oct 31, 2026
c8f5d83
runtime: the macOS worker releases a Store before its volume goes
yifanxuaaa Oct 31, 2026
44d41ef
sandbox: a namespaces Sandbox reopens with its own helper copy
yifanxuaaa Oct 31, 2026
ab00731
sandbox: drop Runtime::Windows, build layerfs-sandbox for Windows (#5…
yifanxuaaa Oct 31, 2026
ad7cb42
sandbox: backend/ into platform/, json/ and root files (#527 step 9b)
yifanxuaaa Oct 31, 2026
2d6770f
sandbox: Docker's files by role, docker/client/, os, ExecExit (#527 s…
yifanxuaaa Oct 31, 2026
34d49eb
agent: refuse a backend by where its Sandbox runs, not by this OS
yifanxuaaa Oct 31, 2026
9b307f3
runtime: the local placement's Docker group under platform/docker/ (#…
yifanxuaaa Oct 31, 2026
3f28127
runtime: Store hooks keyed by Store; createStore makes every Store vo…
yifanxuaaa Oct 31, 2026
9a241c0
runtime: F11's duplicates land once (#527 step 10c)
yifanxuaaa Oct 31, 2026
72bd04c
runtime: sweep finds ended Sandboxes by their labels, not their names…
yifanxuaaa Oct 31, 2026
915522d
runtime: Apple Container copies a folder through the home bind mount …
yifanxuaaa Oct 31, 2026
89226d7
node, agent, runtime: one Sandbox setting, LAYERFS_SANDBOX=docker|nat…
yifanxuaaa Oct 31, 2026
d82d6a3
node, runtime: every backend's Sandbox records listed, and local disc…
yifanxuaaa Oct 31, 2026
a1f91f0
runtime: Windows releases a Store quietly when no worker holds it
yifanxuaaa Oct 31, 2026
a1bf3fd
test: a cell's setup is bounded by the runner's deadline
yifanxuaaa Oct 31, 2026
09bf9d1
docs: AGENTS.md names native macOS as a direct Store exception (#527 D5)
yifanxuaaa Oct 31, 2026
d610a82
test: the cell runner's vm-lima and vm-firecracker rows
yifanxuaaa Oct 31, 2026
b39550b
runtime: vm:lima placement (#527 platform pass)
yifanxuaaa Oct 31, 2026
a8cb674
runtime: vm:firecracker placement (#527 platform pass)
yifanxuaaa Oct 31, 2026
5431c7e
test: the cell runner names namespaces with the one Sandbox setting
yifanxuaaa Oct 31, 2026
e9b9646
agent: a vm: home refuses a Docker deployment before sending it
yifanxuaaa Oct 31, 2026
97d408c
test: the vm-docker refusal cell expects the documented answers
yifanxuaaa Oct 31, 2026
3c3ddb9
docs: #527 platform pass, twelve cells on every platform
yifanxuaaa Oct 31, 2026
b02242f
docs: #527 platform integration and consolidation design
yifanxuaaa Oct 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
17 changes: 17 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Fast lane: runs after a push and reports. It is never a required check and
# never blocks a push or merge; agents do not wait for it (AGENTS.md §4).
# Slow and FUSE mount tests run in nightly.yml. Selection: tools/ci_tests.py.
# windows-build only compiles the crates gated for Windows (#527 D6) and
# reports; it runs no test and its failure fails nothing (continue-on-error).
# The Node binding (engine/crates/layerfs-node) and the JavaScript packages in
# api/ stay outside CI (owner, 2026-10-23): no path trigger, excluded from
# Clippy/tests and skipped by the guard. The Rust layerfs-api crate is included.
Expand Down Expand Up @@ -76,6 +78,21 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
- run: cargo +1.85.1 clippy --manifest-path Cargo.toml --locked --workspace --exclude layerfs-node --all-targets -- -D warnings

# Windows runs the Linux build through WSL2 or a VM (#527 D2); this lane only
# reports whether the crates gated for Windows still compile there. Today
# that is layerfs-sandbox (its shared types; the backends are Unix only).
# layerfs-node stays outside CI, and layerfs-api cannot build for Windows
# yet (layerfs-store-service is Unix only).
windows-build:
name: windows build (reports only)
runs-on: windows-2025
timeout-minutes: 15
continue-on-error: true
steps:
- uses: actions/checkout@v7.0.1
- run: rustup toolchain install 1.85.1 --profile minimal --component clippy
- run: cargo +1.85.1 clippy --manifest-path Cargo.toml --locked -p layerfs-sandbox -- -D warnings

test:
runs-on: ubuntu-24.04
timeout-minutes: 30
Expand Down
20 changes: 15 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,15 @@ remains a separate MEMORY/OFF/EXCLUSIVE database file.
native handles and project discovery. This is independent of the macOS
adapter and changes neither the product SDK nor the daemon. A lost reply
is never replayed, and an uncertain Init retains its Linux home and volume.
- **Additional exception (owner, 2026-11-01, #527 D5): native macOS.** With
`LAYERFS_SANDBOX=native` on macOS, the SDK loads the macOS addon in its own
process and opens the project's Store directly on a local apfs or hfs
directory (`kernel_guard` admits those two there, #427). Its Sandboxes are
daemon processes serving loopback NFS to this Mac's own client, one user,
no container (see the agent guide's native Sandboxes). Every opener is a
process on this one Mac, so the one-kernel, local-filesystem rule holds;
network and other filesystems are still refused. The Docker, worker and
`vm:` routes on macOS are unchanged.
- **Plugin-neutral LayerFS (owner, 2026-10-24).** LayerFS is agnostic to dsh
and every other plugin. The macOS and Windows worker routes above are open
to any integration: `@layerfs/runtime` starts the worker on those platforms
Expand All @@ -170,19 +179,20 @@ remains a separate MEMORY/OFF/EXCLUSIVE database file.
setting), with `local` the default and unchanged. **One Linux VM per Store**
(V1): a project's VM holds that project's Store on its own ext4 disk, its
Sandboxes are Linux namespaces inside it
(`LAYERFS_SANDBOX_BACKEND=namespaces`, documented in
(`LAYERFS_SANDBOX=namespaces`, documented in
[the agent guide](api/docs/agent.md#namespace-sandboxes-linux)), and its
worker runs the real SDK as root with its own secret and its own
loopback-published port; the host reaches only each VM's worker, with one
attempt per call and no replay, the same rules as the macOS and Windows
routes. Store calls therefore leave the caller's process on Linux too, under
`vm:` placement only, and a process routes them per project
(`useWorker((project) => endpoint)`, or that project's
`LAYERFS_WORKER_ENDPOINT`). `project.init` makes the project's VM and
`project.remove` deletes it with its Store disk; a bare VM delete is refused
`LAYERFS_WORKER_ENDPOINT`). `project.start_init` makes the project's VM and
`project.delete` deletes it with its Store disk; a bare VM delete is refused
while the project exists, because that Store is on that disk only and no
remote Store push exists (#506). Implemented and verified providers:
`microsandbox` (`msb`) and `apple-container` (`container` 1.6.1).
remote Store push exists (#506). Implemented providers: `microsandbox`
(`msb`), `apple-container` (`container` 1.6.1) and `ssh`; verified with
stand-in providers; real runs: see the #527 platform pass.
`@layerfs/runtime`'s guide and `api/runtime/src/platform/vm/` hold the details.
- **One global database, one writer, short holds (owner decision 2026-10-18).**
All committed state lives in one SQLite Store file; no sharding, second
Expand Down
79 changes: 51 additions & 28 deletions api/agent/src/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,21 +60,25 @@ export function sandboxArg(sandbox) {
}

/** Deployment settings from the client options, else the environment
* (LAYERFS_<NAME>). A Workspace's Sandbox needs
* every one but the container prefix and daemon host, and, on a PostgreSQL
* Store (`postgresql`), the volume: its daemon opens the server. With
* `sandbox` (LAYERFS_SANDBOX) `native` (Linux or macOS), a Sandbox is a host
* daemon: Linux uses FUSE with a root daemon and nonroot commands (501:20);
* macOS uses loopback NFS and this user. It needs the daemon and
* `localDir` (LAYERFS_LOCAL_DIR) and no Docker setting. Unset or `docker` is
* Docker; any other value is refused. */
export function deployment(options = {}, postgresql = false) {
* (LAYERFS_<NAME>). `sandbox` (LAYERFS_SANDBOX) picks the backend (#527 D3):
* unset or `docker`, a Docker container, which needs every Docker setting but
* the container prefix and daemon host, and, on a PostgreSQL Store
* (`postgresql`), the volume: its daemon opens the server. `native` (Linux or
* macOS) and `namespaces` (Linux) are host daemons (see `localDeployment`).
* The binding is the one validator: it maps the former
* `sandboxBackend` (LAYERFS_SANDBOX_BACKEND), kept for one release, and
* refuses an unknown value or a conflicting pair, so this only checks that
* each value's fields are present and passes both settings on. `host` is
* the platform the Sandbox runs on (`sandboxHost`): native and namespaces are
* refused where they cannot run before anything is read. */
export function deployment(options = {}, postgresql = false, host = process.platform) {
const env = (name) => process.env[`LAYERFS_${name}`] || undefined;
const sandbox = options.sandbox ?? env("SANDBOX");
if (sandbox === "native") return nativeDeployment(options, env, sandbox);
if (sandbox !== undefined && sandbox !== "docker") {
throw new LayerFSError("InvalidInput", `deploy.sandbox (LAYERFS_SANDBOX) is docker or native (Linux or macOS), not ${sandbox}`, null);
}
const backend = options.sandboxBackend ?? env("SANDBOX_BACKEND");
const named = { ...(sandbox !== undefined ? { sandbox } : {}), ...(backend !== undefined ? { sandboxBackend: backend } : {}) };
if (sandbox === "native" || sandbox === "namespaces") return localDeployment(options, env, named, host);
// another value is the binding's to refuse, naming the values it takes
if (sandbox !== undefined && sandbox !== "docker") return named;
const deploy = {
docker: options.docker ?? env("DOCKER") ?? "/var/run/docker.sock",
image: options.image ?? env("IMAGE"),
Expand All @@ -90,27 +94,41 @@ export function deployment(options = {}, postgresql = false) {
if (!deploy.volume) delete deploy.volume;
if (!deploy.containerPrefix) delete deploy.containerPrefix;
if (!deploy.daemonHost) delete deploy.daemonHost;
return deploy;
}

/** A native Sandbox's settings: the daemon, the directory its Sandboxes
* live in, the optional name prefix, and the backend inside that directory
* (`sandboxBackend`, LAYERFS_SANDBOX_BACKEND: `native` when unset, or
* `namespaces` on Linux). The namespaces backend gives each Sandbox a root
* filesystem from an image, so it also needs `image` (LAYERFS_IMAGE, the
* same digest Docker deploys) and `imageArchive` (LAYERFS_IMAGE_ARCHIVE, the
* docker-archive the bundle carries). */
function nativeDeployment(options, env, sandbox) {
// a former LAYERFS_SANDBOX_BACKEND beside Docker is the binding's to refuse
return backend === undefined ? deploy : { ...deploy, sandboxBackend: backend };
}

/** A host-daemon Sandbox's settings (`sandbox` native or namespaces): the
* daemon, the directory its Sandboxes live in and the optional name prefix.
* Native on Linux uses FUSE with a root daemon and nonroot commands (501:20);
* on macOS loopback NFS and this user. Namespaces (Linux) gives each Sandbox
* a root filesystem from an image, so it also needs `image` (LAYERFS_IMAGE,
* the same digest Docker deploys) and `imageArchive` (LAYERFS_IMAGE_ARCHIVE,
* the docker-archive the bundle carries). The former `native` with
* `sandboxBackend` `namespaces` needs the same. `host` is the platform the
* Sandbox runs on: this process's, or Linux when a worker makes it (see
* `sandboxHost`). */
function localDeployment(options, env, named, host) {
const namespaces = named.sandbox === "namespaces" || named.sandboxBackend === "namespaces";
const setting = named.sandbox === "namespaces" ? "deploy.sandbox (LAYERFS_SANDBOX) namespaces"
: namespaces ? "deploy.sandboxBackend (LAYERFS_SANDBOX_BACKEND) namespaces" : "deploy.sandbox (LAYERFS_SANDBOX) native";
// #527 step 8: Windows runs Docker Sandboxes through its WSL2 worker; it has no native or namespaces backend
if (host === "win32") {
throw new LayerFSError("Unsupported", `${setting} is not supported on Windows: a Windows Sandbox is Docker, through the WSL2 worker; native runs on Linux or macOS, namespaces on Linux`, null);
}
// namespaces is Linux only: refused before its image settings are read, so
// a macOS deployment hears Unsupported, not a missing LAYERFS_IMAGE
if (namespaces && host !== "linux") {
throw new LayerFSError("Unsupported", `${setting} is Linux only; use native`, null);
}
const deploy = {
sandbox,
...named,
daemon: options.daemon ?? env("DAEMON"),
localDir: options.localDir ?? env("LOCAL_DIR"),
containerPrefix: options.containerPrefix ?? env("CONTAINER_PREFIX"),
};
const backend = options.sandboxBackend ?? env("SANDBOX_BACKEND");
if (backend !== undefined) deploy.sandboxBackend = backend;
const required = [["daemon", "DAEMON"], ["localDir", "LOCAL_DIR"]];
if (backend === "namespaces") {
if (namespaces) {
deploy.image = options.image ?? env("IMAGE");
deploy.imageArchive = options.imageArchive ?? env("IMAGE_ARCHIVE");
required.push(["image", "IMAGE"], ["imageArchive", "IMAGE_ARCHIVE"]);
Expand All @@ -122,6 +140,11 @@ function nativeDeployment(options, env, sandbox) {
return deploy;
}

/** The platform a client's Sandboxes run on: a worker (the macOS and
* Windows routes, or a `vm:` placement's VM) runs them on its Linux host;
* the addon in this process runs them here. */
export const sandboxHost = (client) => (transportOf(client).worker ? "linux" : process.platform);

/** Refuses every call on a closed client (or one no factory opened). */
export function live(client) {
if (client?.[STATE]?.closed !== false) {
Expand Down
16 changes: 9 additions & 7 deletions api/agent/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -111,15 +111,17 @@ export class LayerFS {
* settings read LAYERFS_HOME, _DOCKER, _IMAGE, _VOLUME and _DAEMON; a
* Sandbox this client starts runs _IMAGE
* on _VOLUME. LAYERFS_CONTAINER_PREFIX (default `layerfs-`) names every
* container it starts `<prefix><project>-<n>`. On Linux or macOS,
* LAYERFS_SANDBOX=native starts each Sandbox as a host daemon in a directory
* container it starts `<prefix><project>-<n>`. LAYERFS_SANDBOX picks the
* backend: `docker` (unset), `native` (Linux or macOS) or `namespaces`
* (Linux). Native starts each Sandbox as a host daemon in a directory
* under LAYERFS_LOCAL_DIR. Linux requires root for FUSE and runs commands
* as 501:20; macOS uses loopback NFS and the caller's identity. No Docker
* setting is read; native provides no isolation from the host.
* LAYERFS_SANDBOX_BACKEND picks the backend inside that directory: `native`
* (unset) or, on Linux, `namespaces` (namespaces, cgroups, nonroot commands).
* It is read for every call, so a Sandbox made with one backend is never
* reopened as another.
* setting is read; native provides no isolation from the host. Namespaces
* adds Linux namespaces, a cgroup and an image root (LAYERFS_IMAGE,
* LAYERFS_IMAGE_ARCHIVE). The former LAYERFS_SANDBOX_BACKEND=namespaces
* beside LAYERFS_SANDBOX=native still means namespaces, for one release.
* The backend is read for every call, so a Sandbox made with one backend
* is never reopened as another.
* LAYERFS_DAEMON_HOST is the host this process
* dials for a daemon's port, published on the Docker host's 127.0.0.1 only;
* set it when the client runs in a container (`host.docker.internal` on
Expand Down
4 changes: 2 additions & 2 deletions api/agent/src/sandbox/create.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@
// first reclaims the project's Sandboxes that closed themselves. The new one
// closes itself two minutes after it has no Workspace and no process, like
// every Sandbox; `sandbox.close` stops it now.
import { deployment, invokeClient } from "../client.js";
import { deployment, invokeClient, sandboxHost } from "../client.js";
import { LayerFSError } from "../errors.js";
export async function create(client, options = {}) {
if (options === null || typeof options !== "object" || (options.image !== undefined && typeof options.image !== "string")) {
throw new LayerFSError("InvalidInput", "options are { image? }, an image digest", null);
}
const deploy = deployment({ ...client.options.deploy, ...(options.image ? { image: options.image } : {}) }, Boolean(client.options.postgresql));
const deploy = deployment({ ...client.options.deploy, ...(options.image ? { image: options.image } : {}) }, Boolean(client.options.postgresql), sandboxHost(client));
const value = await invokeClient(client, "sandbox.create", { deploy });
return { sandbox: value.sandbox, image: value.image };
}
7 changes: 7 additions & 0 deletions api/agent/src/transport/worker_client.js
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,11 @@ export function workerClient(file) {
async function send(fn, op, args) {
let info;
try { info = endpointOf(file); } catch (error) { return refused("Failed", `the LayerFS worker is not running for this home: ${error.message}`); }
// a vm: placement's worker has no Docker Engine: its Sandboxes are
// namespaces inside its VM, so a Docker deployment is refused before it is sent
if (info.vm && args?.deploy && (args.deploy.sandbox ?? "docker") === "docker") {
return refused("Unsupported", `deploy.sandbox (LAYERFS_SANDBOX) docker is not available on a vm:${info.vm.provider} home: its Sandboxes are namespaces inside the project's VM ${info.vm.name}; leave LAYERFS_SANDBOX to the runtime`);
}
const deploy = {
...args.deploy,
...(info.daemonHost ? { daemonHost: args.deploy?.daemonHost ?? info.daemonHost } : {}),
Expand All @@ -150,6 +155,8 @@ export function workerClient(file) {
return connection(info).call(request);
}
return {
/** Calls run in a Linux worker, so its Sandboxes run on Linux. */
worker: true,
call: (op, args) => send("call", op, args),
backendCall: (op, args) => send("backendCall", op, args),
/** The Store opens in the worker; here only its clients are counted. */
Expand Down
4 changes: 3 additions & 1 deletion api/agent/test/fixture/fake-native.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
// (comma-separated) is refused with code CLI_FAKE_CODE (default Conflict);
// `initRefusal`, when set, is what project.init refuses; a policy depth over
// 255 is refused as the addon refuses it.
// - Answers: `sandboxList` is the project's running Sandboxes,
// - Answers: `sandboxList` is the project's running Sandboxes, `recordList`
// (by project) every backend's records (runtime.sandbox.records),
// `workspaceList` a Sandbox's daemon records. An exec command "exit <n>"
// ends with exit code n, stdout "out:<command>" and stderr "err"; any other
// command ends with code 3 and 5000 bytes of stdout.
Expand Down Expand Up @@ -68,6 +69,7 @@ function answer(op, args) {
case "sandbox.list_workspaces": return { value: { kind: "ok", version: "1", workspaces: exports.workspaceList } };
case "sandbox.close": return { value: { kind: "Closed", sandbox: args.sandbox } };
case "runtime.sandbox.reclaim": return { value: { kind: "ok", reclaimed: exports.reclaimed ?? [] } };
case "runtime.sandbox.records": return { value: { kind: "ok", items: exports.recordList?.[args.project] ?? [] } };
// #506 Store-first: Stores, their projects, remotes and transfers
case "runtime.store.create": return { value: { kind: "Created", store: args.store, storePath: args.storePath, bytes: 4096, sqlite: "3.50.0" } };
case "runtime.store.add_project": return { value: { kind: "Initialized", project: args.name, store: args.store, layer: "lay0", root: "r00t", storePath: args.storePath } };
Expand Down
Loading
Loading