Skip to content

feat: macOS 原生适配——daemon 内置 loopback NFSv3 导出,普通用户 mount_nfs 挂载 Workspace(无 Docker / 无 macFUSE / 无 root) - #427

Closed
jiangkoumo wants to merge 78 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/macos-native-on-main
Closed

jiangkoumo wants to merge 78 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/macos-native-on-main

Conversation

@jiangkoumo

@jiangkoumo jiangkoumo commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

jiangkoumo:feat/macos-native-on-main → main。本 PR 只做 macOS 原生适配,自成一体:它用到的本地 daemon 运行时(最早在已关闭的 #402 里提出)已经作为本 PR 的一部分收进来,不依赖其他 PR。
分支头 d55acc15f,已 merge 到 main 582bc786f。历史只用 merge,不 rebase、不 force-push。

这个 PR 做什么

LayerFS 在 Linux 上用 FUSE 把 Workspace 呈现为普通目录。macOS 没有内置 FUSE,现在的选择是 macFUSE(要内核扩展,Apple Silicon 上要降低安全级别)、FUSE-T(挂载不稳定)或 Docker Desktop 里的 Linux(要装 Docker,I/O 全过虚拟机文件共享)。

本 PR 走另一条路:Sandbox 的 daemon 作为当前用户的普通进程运行,自己提供一个只监听 127.0.0.1 的 NFSv3 服务,macOS 自带的 /sbin/mount_nfs 以普通用户身份把 Workspace 挂到 Sandbox 目录下。 和 Meta EdenFS 在 macOS 上的做法一致。不需要 root、内核扩展、虚拟机或 Docker。

  • 开关:LAYERFS_SANDBOX=native(仅 macOS),另需 LAYERFS_DAEMON、LAYERFS_LOCAL_DIR(0755、属于当前用户)和放在本地 apfs/hfs 上的 LAYERFS_STORE_PATH;不读任何 Docker 设置。
  • 命令以当前用户身份运行,沿用调用方的 PATH(Homebrew、~/.local/bin、nvm 的 node 都能找到)。它不做隔离,隔离交给调用方自己的沙箱(例如 Seatbelt)。
  • Commit 前 daemon 先把客户端缓冲的写回;Commit/merge 换根后让客户端立即丢掉被替换文件的属性缓存。
  • 记录、空闲 2 分钟关闭、回收、创建意图、sandbox.close 等语义和 Docker Sandbox 一致;daemon 被杀后,下一次 sandbox.create / runtime.sandbox.reclaim 会强制卸掉它留下的挂载再删目录。
  • 文档:api/docs/agent.md 的 "Native Sandboxes (macOS)";生命周期设计:core/docs/architecture/nfs-native-lifecycle.md。

平台范围

  • macOS:唯一的新部署方式 native。
  • Linux:保持 main 的 Docker 路线不变。LAYERFS_SANDBOX 只接受不设、docker、native;native 在非 macOS 上由 sandbox.create 在做任何事之前拒绝为 Unsupported;其他值(包括早先草案里的 local)拒绝为 InvalidInput,不会被悄悄当成 Docker。没有任何部署会在 Linux 上启动本地 daemon 或启用 daemon 的 NFS 路线,Linux daemon 仍走 FUSE。
  • 早先草案(feat(sandbox): 本地进程 Sandbox 后端 LAYERFS_SANDBOX=local——在没有 Docker 的 Linux VM / CI 里使用 workspace.* #402)里的 Linux 对外部署模式 local 已经去掉。为了不重写已验证的代码,engine/crates/layerfs-sandbox/src/backend/local/ 这个本地进程后端整体保留(native 用的就是它的进程、exec、进程组、挂载和记录代码),里面 Local::new 和两个 Linux 测试(local_runtime、local_stop_mount)只作为这份共享代码的测试留着,不再有部署入口。
  • 旧记录:runtime: "local" 的 Sandbox 记录(只会来自早先未合并的构建)任何部署都不使用,返回 InvalidInput("never mixed … neither used nor deleted"),不删除它指向的任何东西,需要手动清理。

和 2.0 统一 provider 设计的关系

作者提到 2.0 要把 Docker、原生 Windows/macOS/Linux 和其他沙箱 provider 一起设计。本 PR 可以当作其中 macOS 原生 provider 的参考实现:它把"provider 怎么启动 daemon、怎么挂载、怎么跑命令、怎么记录和回收"在 macOS 上完整走通了一遍,NFS 导出(engine/crates/layerfs-nfs、daemon 的 control/export.rs)和 provider 外壳是分开的。2.0 的 provider 接口定下来以后,我愿意按新接口调整或拆分本 PR。

主要改动

  • engine/crates/layerfs-nfs(新):NFSv3/MOUNT 协议服务、AppleDouble 处理、写回与属性刷新,只听 loopback。
  • engine/crates/layerfs-daemon:control/export.rs、export_helper.rs 把 Workspace 以 NFS 导出并调用挂载/卸载助手;Commit 前 flush、换根后刷新;macOS 上 daemon 的进程保护(umask 077、禁 core dump)。
  • engine/crates/layerfs-node:runtime.rs(Docker / 本地 daemon 两种运行时)、local.rs(native Sandbox 的启动、失败清理)、discovery.rs(native 记录的读写与安全校验)、unmount.rs(Workspace 级卸载租约:daemon 等待卸载助手时不占住整个 Sandbox 的锁)。
  • engine/crates/layerfs-sandbox/src/backend/local/:本地进程后端(macOS 用 proc_pidinfo/sysctl/mount 读进程和挂载,umount -f 强制卸载)。
  • engine/crates/layerfs-store-service:macOS 上 Store 只允许放在本地 apfs/hfs;native 的安装者就是 daemon 的用户。
  • SDK api/agent/src:LAYERFS_SANDBOX / LAYERFS_LOCAL_DIR。
  • 生产代码行数(tools/production_loc.py --root .):本分支 109,142(去掉 local 部署的提交 −16)。

验证

上游 CI(本 PR 头 d55acc15f):fmt and guards、ruff、clippy、test 全部通过。

Linux(x86_64,cargo 1.85.1 --offline --locked,Node 22.23.3),同一台机器上对照 main 582bc786f:

检查 本分支 main
cargo fmt --all --check ✅ ✅
tools/check_product_boundary.py(完整 / --skip-sdk) ✅ / ✅ ✅ / ✅
check_fuser_integrity.py、tools 单测 ✅ ✅
CI clippy(--workspace --exclude layerfs-node --all-targets -D warnings)、node clippy ✅ ✅
Darwin 交叉 clippy(--target aarch64-apple-darwin,nfs/sandbox/node/daemon/store-service,-D warnings) ✅ —
fast lane(tools/ci_tests.py targets fast,测试二进制 / 通过 / 失败 / 忽略) 256 / 1160 / 2* / 12 247 / 1136 / 1* / 12
node 测试 native_local(native 记录、旧 local 记录拒绝) 7/7 不适用
JS(假 binding):native_deployment、calls、exec_output、trace×9、runtime×11、cli、doc-tables、backend×4 全过 全过

* 失败都与本 PR 无关:layerfs-api::sdk_method_golden(两边都失败:测试机 umask 002,文件 mode 436≠420)、layerfs-node 的 native_admission 2 例和 native_discovery 3 例(mock Engine 连接被拒,main 同样)、packaging.mjs(两边都失败,需要 Node 24)。本分支多出的 1 个失败是 layerfs-persistence::concurrent_first_installs,已有的并发抖动:各跑 30 次,本分支 3/30、main 5/30 失败。

macOS(Apple silicon,macOS 27.2,CARGO_PROFILE_RELEASE_STRIP=none):

  • release 构建 daemon + node addon + daemon_nfs 示例:✅,addon 可加载(5 个导出)。
  • 单测:layerfs-nfs 17 / layerfs-sandbox 37(1 个 Linux 限定忽略)/ layerfs-store-service 7,全过;layerfs-node 测试按上游设计只在 Linux 运行。
  • Mac clippy -D warnings(nfs/sandbox/node/daemon/store-service/persistence/overlay,--all-targets):✅ 0 警告。
  • SDK smoke LAYERFS_SANDBOX=native:48/48(连跑两次),无挂载、无 daemon 残留。
  • PATH 检查 11/11(沿用 PATH 7/7,无 PATH 回落 4/4);卸载租约真机回归 api/agent/test/verify/native_unmount_real.mjs PASS。
  • 第一次 smoke 是在 release 构建刚结束、负载很高时跑的,workspace.unmount 遇到一次 macOS 客户端返回 Resource busy(之后的连带失败都源于这个挂载没卸掉);之后两次完整重跑均 48/48。这个偶发情况我会继续观察。

复现命令:

export CARGO_PROFILE_RELEASE_STRIP=none   # macOS 27 + rustc 1.85.1 必须关 strip,否则 dyld 拒绝加载 layerfs.node
cargo +1.85.1 build --release --locked -p layerfs-daemon -p layerfs-node
cargo +1.85.1 test --locked -p layerfs-nfs -p layerfs-sandbox -p layerfs-store-service
mkdir -p ~/lfs-try/local && chmod 755 ~/lfs-try/local
export LAYERFS_SANDBOX=native LAYERFS_LOCAL_DIR=~/lfs-try/local LAYERFS_HOME=~/lfs-try/home \
       LAYERFS_STORE_PATH=~/lfs-try/vol/global/store.sqlite \
       LAYERFS_DAEMON=$PWD/target/release/layerfs-daemon LAYERFS_NATIVE=$PWD/layerfs.node
cp target/release/liblayerfs_node.dylib layerfs.node   # Node 只加载 .node 后缀
# 然后用 api/agent/src 的 LayerFS:init → branch.fork → sandbox.create → workspace.mount(sandbox, branch) → exec → commit → unmount → sandbox.close

问题反馈请附:sw_vers、uname -m、rustc +1.85.1 -V、node -v、失败输出、mount | grep 127.0.0.1、pgrep -lf layerfs-daemon、LAYERFS_LOCAL_DIR/<sandbox>/daemon.log 末尾。

已知限制

  • macOS 上后台进程(exec 的 background)仍依赖 /proc,暂不可用。
  • 大量小文件写入比直接写 APFS 慢约 6 倍。
  • @layerfs/runtime 的 Docker 侧 runtime.sandbox.list/sweep 看不到 native Sandbox。
  • 只在 Apple silicon + macOS 27.2 上验证过。

…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/Cargo.toml (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 108,451 -> 108,451 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production).
No tests changed or deleted.

Production LOC: 108,451 -> 108,460 (delta +9)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted.

Production LOC: 108,460 -> 108,505 (delta +45)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production).
No tests changed or deleted.

Production LOC: 108,505 -> 108,576 (delta +71)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted.

Production LOC: 108,576 -> 108,834 (delta +258)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 108,834 -> 109,057 (delta +223)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,057 -> 109,166 (delta +109)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production).
No tests changed or deleted.

Production LOC: 109,166 -> 109,167 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,167 -> 109,167 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/runtime.rs (production).
No tests changed or deleted.

Production LOC: 109,167 -> 109,274 (delta +107)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/local.rs (production).
No tests changed or deleted.

Production LOC: 109,274 -> 109,527 (delta +253)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/registry.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,527 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/discovery.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,644 (delta +117)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,644 -> 109,633 (delta -11)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 109,633 -> 109,641 (delta +8)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/workspace.rs (production).
No tests changed or deleted.

Production LOC: 109,641 -> 109,636 (delta -5)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/lib.rs (production).
No tests changed or deleted.

Production LOC: 109,636 -> 109,638 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/tests/unit/runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,638 -> 109,638 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/client.js (production).
No tests changed or deleted.

Production LOC: 109,638 -> 109,656 (delta +18)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/index.js (production).
No tests changed or deleted.

Production LOC: 109,656 -> 109,658 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/test/local_deployment.mjs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/README.md (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,658 -> 109,696 (delta +38)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/groups.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,696 -> 109,818 (delta +122)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,818 -> 109,835 (delta +17)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,835 -> 109,868 (delta +33)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,868 -> 109,882 (delta +14)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,882 -> 109,870 (delta -12)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,870 -> 109,871 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,871 -> 109,872 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…er selectable (Ephemeral-AI-Lab#427)

Ephemeral-AI-Lab#427 is the macOS native adaptation. The local-daemon runtime it builds
on (first proposed in the closed Ephemeral-AI-Lab#402) is now part of this PR, and its
only selectable form is `native`: on macOS a daemon of this user serves
its Workspaces over loopback NFS to the host's own client. Linux keeps
the Docker route exactly as on main.

- deploy.sandbox (LAYERFS_SANDBOX) accepts unset, `docker` or `native`;
  `local` is refused InvalidInput like any other value, in the SDK
  (api/agent/src/client.js) and in the binding (runtime::local).
- local::start refuses `native` off macOS as Unsupported before any
  effect, so no deployment starts a local daemon on Linux. The root
  check and the Local::new (Linux FUSE) branch go: native always takes
  Local::native, LAYERFS_KERNEL=nfs and this user's identity.
  discovery::root() had no other caller and goes too.
- Records: a new record's runtime is always `native`. A native record is
  used only by a native deployment; a record of the former `local`
  runtime (an earlier, unmerged build) is used by no deployment, refused
  InvalidInput ("never mixed", "neither used nor deleted"), and nothing
  it names is deleted.
- Tests: engine/crates/layerfs-node/tests/native_local.rs writes native
  records (record handling is the same code on Linux) and gains
  a_record_of_the_former_local_runtime_is_never_used_nor_deleted;
  api/agent/test/local_deployment.mjs becomes native_deployment.mjs with
  `local` among the refused values.
- api/docs/agent.md drops the "Local Sandboxes" section and keeps its
  layout, record and reclaim facts under "Native Sandboxes (macOS)".

The local backend in layerfs-sandbox (engine/crates/layerfs-sandbox/src/
backend/local) is kept as it is: native uses its process, exec, group,
mount and record code; Local::new and its Linux tests stay as tests of
that shared code.

Production LOC: 109158 -> 109142 (delta -16)
(tools/production_loc.py --root .)
@jiangkoumo jiangkoumo changed the title [Draft] feat: macOS 原生支持(长期适配分支)——内置 loopback NFSv3 导出,普通用户 mount_nfs 挂载 Workspace(无 Docker / 无 macFUSE / 无 root) feat: macOS 原生适配——daemon 内置 loopback NFSv3 导出,普通用户 mount_nfs 挂载 Workspace(无 Docker / 无 macFUSE / 无 root) Oct 28, 2026
@jiangkoumo
jiangkoumo marked this pull request as ready for review October 28, 2026 03:14
@jiangkoumo

Copy link
Copy Markdown
Collaborator Author

新提交 d55acc1:改成独立的 macOS 原生适配,并 merge 到 main 582bc78

• 不再依赖 #402:它用到的本地 daemon 运行时已经作为本 PR 的一部分收进来。
• 去掉了 Linux 对外的 local 部署:LAYERFS_SANDBOX 只接受不设、docker、native;native 只在 macOS 可用,其他平台由 sandbox.create 直接拒绝为 Unsupported。Linux 仍是 main 的 Docker 路线,没有变化。
• 旧的 runtime:"local" 记录任何部署都不使用、也不删除,返回 InvalidInput。
• merge main 的两处冲突来自 #481(强制卸载):force.rs 先查 NFS 的 ExportKept 再走新的强制卸载例外;workspace.rs 保留卸载租约,把 force 选项放进发给 daemon 的请求。

验证:上游 CI 全过;Linux 上 fmt / 边界检查 / clippy / fast lane / node 与 JS 测试和 main 对照,没有新增失败;Mac(Apple silicon,macOS 27.2)上 release 构建、单测、clippy、native smoke 48/48、PATH 检查 11/11、卸载租约回归都通过。详细结果已更新到 PR 描述。

jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Oct 29, 2026
Poll endpoint through the Sandbox that owns the Child before asserting that no zombie remains. Bound the observation by WAIT and preserve the stopped and not-gone assertions without changing product behavior.

Validation: cargo +1.85.1 fmt --all -- --check; cargo +1.85.1 clippy --locked -p layerfs-sandbox --all-targets -- -D warnings. Both passed on macOS with this worktree private target. Linux-only runtime tests were not executed; Hark owns Linux verification.

Production LOC: 109210 -> 109210 (delta +0). Scope: core 41400, engine 65131, API 2679; reference absent (retired R9c; last counted 65,417 at tree 498dd19). Tests, tools, docs, dependencies and generated artifacts excluded; inline test code excluded by the counter.

Method: tools/production_loc.py at Git blob 3341e4d; export parent 3147916 and final staged tree d97b40d using git archive, then run python3 -B tools/production_loc.py --root <each-export> --json. Both invocations exited 0.
jiangkoumo added a commit to jiangkoumo/layerfs that referenced this pull request Oct 29, 2026
Mount the macOS native export with noatime. The export drops SETATTR's
access time and exposes the stored mtime for all three timestamps, so
client access-time updates add work without persisting a distinct atime.
Keep the Linux mount options unchanged. Document the native default and
cover the macOS helper option while preserving actimeo=60.

Exploratory Git scans reduced per-command SETATTR from 4831 to 2; wall
time was noisy and does not establish a speedup. Imported Git index stat
mismatches and attribute-less READDIRPLUS remain separate limitations.
No Git stat checks, stored timestamps, inode identities or formats change.

Validation: fmt; locked warning-denying Clippy for layerfs-daemon and
layerfs-nfs all targets; NFS tests (17 passed before adding the helper
test); final helper test (1 passed); product boundary guard; native mount
observed noatime. No tests deleted. Linux native execution not run.

Production LOC: 109210 -> 109210 (delta +0)
Scope: core 41400, engine 65131, API 2679; dsh absent.
Method: tools/production_loc.py at parent 2518769, identical counter on
git archive of HEAD and staged tree 86a1874;
nonblank/noncomment product Rust, runtime SQL and JS, excluding tests.
reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Document the system SQLite VFS restriction, safe pre-import journal conversion and cache placement. Link the guide from the SDK and documentation index. System nfsd comparison remains blocked by unavailable administrator authentication.

Production LOC: 109210 -> 109210 (delta +0). Method: python3 -B tools/production_loc.py --json, counter blob 3341e4d. First-parent and staged product trees are identical; only documentation changes. Counter scopes: core 41400, engine 65131, API 2679; layerfs-nfs is classified under core by this pinned counter. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Production LOC: 109210 -> 109210 (delta +0). First-parent comparison; core 41400, engine 65131, API 2679. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).

Method: tools/production_loc.py blob 3341e4d; git archive of first parent 2518769 and final staged tree 86a1874, then python3 -B tools/production_loc.py --root <export> --json. Both exited 0; tests, docs, tools and inline test code excluded.
Production LOC: 109210 -> 109210 (delta +0). First-parent comparison; core 41400, engine 65131, API 2679. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).

Method: tools/production_loc.py blob 3341e4d; git archive of first parent 212ac9aec31642138b6afa617178eb1f18bc04e5 and final staged tree cc1e2d1, then python3 -B tools/production_loc.py --root <export> --json. Both exited 0; tests, docs, tools and inline test code excluded.
@jiangkoumo

Copy link
Copy Markdown
Collaborator Author

更新:在 macOS 原生路线上补了两处小改动(新头 fa02a1b,普通 merge,没有 force-push)。

  1. macOS 默认挂载加 noatime(layerfs-daemon/src/control/export.rs,Linux 参数不变)
    导出层只存一个时间戳,三个时间都报它,sattr 也会丢掉 atime,所以客户端发来的访问时间更新都是空跑。change.rs 的注释里其实已经写了 "the mount is noatime",只是挂载参数里漏了。
    在约 4800 个受控文件的仓库上,导入后第一次 git status:NFS 请求从 27605 降到 22154(约 −20%),SETATTR 从 4831 降到 2。墙钟时间改善不明显(当时机器负载较高,一组 −6.6%,另一组没差别),所以只算去掉了无用请求,不算提速。新增了只在 macOS 上跑的测试 tests/mounter.rs,断言挂载参数里有 noatime 和 actimeo=60。Linux 要不要也加,听您的。

顺带查清了两点,没有改代码:
• 导入后第一次 git status 会把文件全部重读一遍,起因是 ctime(索引里记的是宿主的 ctime,挂载里报的是 mtime),不是 inode/dev;重新挂载和 Commit 之后都稳定。改 ctime 语义会影响 Git 的缓存判断,建议集成方导入后跑一次 git update-index --refresh。
• 全仓 find 慢,是因为客户端对每个名字都要发一次 LOOKUP。只给 READDIRPLUS 补属性,实测没有收益;要省掉这一步得同时返回 handle,涉及引用保管,先只记下设计。

  1. 新文档 docs/general/sqlite-on-network-filesystems.md(生产代码 0 改动)
    WAL 模式的 SQLite 库在 macOS NFS 挂载里能 open,但第一次查询就报 SQLITE_CANTOPEN(14)。原因是 SQLite 在 macOS 上遇到 nfs 会自动改用 nfsIoMethods,而它不提供 xShmMap。这是 SQLite 客户端的策略,导出层修不了。DELETE 模式的库正常,新建的库会自动回落到 delete,也正常。文档里写了导入前安全转换的方法,也注明了还没用系统 nfsd 做对照(需要管理员权限)。

验证:cargo fmt --check、clippy -p layerfs-daemon -p layerfs-nfs --all-targets -D warnings、cargo test -p layerfs-nfs(18 passed)、check_product_boundary.py、文档链接检查,在 macOS 27.2 / M2 上全部通过。

yifanxuaaa pushed a commit that referenced this pull request Oct 31, 2026
… B3)

Integrate #427 at its branch head fa02a1b, keeping its 78 commits and
their authorship, and place the macOS native backend on B1's single
Runtime/Sandbox enum as Runtime::MacosNative(NativeMacosProvider). This is
the B3 slice of the #498 split: the platform-independent implementation
(backend/native/*, the node pipeline, the api/agent settings) is already in
B1 a32da57, so this merge only adds macOS platform facts plus #427 itself.

Conflicts (16 files) follow the H42a-2 P2-A rule: layerfs-node/src's shared
files (local, discovery, runtime, sandbox, workspace, exec, registry) take
the B1 side, unmount.rs takes the #427 side, and every other file takes the
9aa10e6 text, which is what B3 is specified to add. #427's
backend/local/ (9 files) is removed: native/ + macos/ + B1's linux/ replace
it. The merged tree equals the rehearsal-b3 tree (fee449554, tree
ed2356142) except for five files that come from #427's three commits after
the split's comparison base 2518769: the noatime mount option and its
mounter test in layerfs-daemon/layerfs-nfs, and the noatime/WAL
documentation in api/docs/agent.md, docs/README.md and
docs/general/sqlite-on-network-filesystems.md. Those are #427 content and
stay; they disappear from this PR's diff once #427 is on main.
Per-workspace unmount leases, NFS export control, layerfs-nfs, the macOS
Store install and kernel guard are #427 unchanged.

D2 macOS stays as #427 implements it (Store on the host, loopback NFSv3
mount); D13 is NFSv3 now with FSKit as follow-up. No Durable execution.
Deleted tests: none. Retained #427 test targets: layerfs-nfs's 8 binaries
and mounter, layerfs-node's Linux-gated native_* tests (0 tests on macOS).

Validation: macOS 27.2 ARM64, Rust 1.85.1, locked dependencies, repository
ARM64 flags, every Rust test command under a 120 s wall timeout after a
--no-run build. cargo fmt --check, cargo check --workspace --all-targets,
Clippy -D warnings for the workspace and for layerfs-sandbox at
x86_64-unknown-linux-gnu, product-boundary guard (1009 files) and the 155
tooling self-tests passed. layerfs-sandbox 45 passed/1 ignored
(local_stop_mount needs a real mount), layerfs-api 10 passed, layerfs-daemon
6 passed/2 ignored, layerfs-nfs 18 passed, layerfs-node 0 tests on macOS
(Linux-gated), api/cli 75 passed, agent calls/exec_output and
native_deployment passed. LAYERFS_SANDBOX=native release smoke (addon and
daemon built with CARGO_PROFILE_RELEASE_STRIP=none): 48/48, Z1 m=0 d=0 e=0.
Disposable only; Durable NOT_RUN (owner-disabled). Docker route NOT_RUN on
this host (no Docker Engine); its behaviour is covered by the mock-engine
and lifecycle tests above, which run as Runtime::Docker. No benchmark.

Production LOC: 106022 -> 110634 (delta +4612)
First-parent comparison: b1-interface a32da57
to prepared merge tree fcd69b9.
Scopes: core 38395 -> 41400 (+3005, layerfs-nfs; the counter files it under
core, the boundary guard calls it engine), engine 64938 -> 66545 (+1607:
daemon +1122, node +231, sandbox +226, store-service +28), api 2689 -> 2689
(0), dsh 0 -> 0.
Method: tools/production_loc.py blob 3341e4d,
--root <exact exported tree> --json for both snapshots; identical counter and
scope for both; first-party production Rust, runtime SQL and JavaScript;
tests, docs, tooling, fixtures, generated and third-party files excluded.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
yifanxuaaa pushed a commit that referenced this pull request Oct 31, 2026
…anch (Area B3)

engine/crates/layerfs-sandbox/tests/local_runtime.rs arrived in B3 with the
#427 merge (fa02a1b). It is gated with #![cfg(target_os = "linux")] and
tests the Linux native process table, which B3 does not carry: B3's
backend/linux/ is byte-identical to B1's placeholder, so all nine cases fail
on Linux CI with Io(Unsupported, "Linux native process table"). The real
implementation and the file itself belong to B2 (b2-linux-native-v2), and
merging B2 into B3 keeps B2's copy unchanged, so removing it here neither
loses coverage nor changes the merged tree.

local_search_path.rs and local_stop_mount.rs are macOS-only and stay.

References: none in Cargo.toml, tools/ci_tests.py, workflows or docs. The only
remaining "local_runtime" text is layerfs-node's
a_record_of_the_former_local_runtime_is_never_used_nor_deleted test, which
covers former `local` sandbox records and is unrelated to this target.

Deleted tests: local_runtime (9 cases on Linux); the behaviour they check is
covered by the same file on B2.

Validation (macOS 27.2 ARM64, Rust 1.85.1, this worktree's own target):
cargo fmt --all --check exit 0; cargo clippy --locked --workspace --exclude
layerfs-node --all-targets -- -D warnings exit 0. tools/ci_tests.py targets
fast no longer emits --test local_runtime (259 -> 258 targets) and filter
fast is byte-identical. merge-tree --write-tree of b2-linux-native-v2 and
this branch is conflict-free and keeps local_runtime.rs identical to B2; the
resulting tree diffs empty against the H43e reference 5c6a238b.

Production LOC: 110634 -> 110634 (delta 0)
Scope: first-party production Rust, runtime SQL and published JavaScript;
tests, docs, tooling, fixtures and generated files excluded. Method:
tools/production_loc.py --root <exported parent tree | this tree> --json,
identical counter and scope for both snapshots. Parent comparison:
1dba37b to this commit. The change deletes one test file only.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
yifanxuaaa added a commit that referenced this pull request Oct 31, 2026
…fs_v2_0_0

B3 (layerfs-connect#47): native macOS Sandbox on B1's interface,
including #427 (daemon-served loopback NFSv3 export, mounted by the user
with mount_nfs; new crate layerfs-nfs). Merged at PR head fccce8f
onto the B2 merge 7d55f02. Conflicts resolved:

- layerfs-store-service install.rs: B3 gives installer_user a macOS
  variant (no root daemon to match) and gates the Linux one; v2 made it
  pub(crate) for remote/pull.rs (#506). Both variants are pub(crate).
- layerfs-sandbox tests/native_provider.rs: keeps the optional
  `Some(...)` Store volume from the B2 merge.
- layerfs-node tests/README.md: lists both v2's native_connection_wait
  and B3's native_local.

Verified on macOS (Rust 1.85.1): workspace warning-denying Clippy, all
targets; format check (only the existing minio.rs drift). Tests, each
group under a 120 s cap: layerfs-nfs, -sandbox and -node 69 passed,
0 failed, 1 ignored (B2's local_search_path and native_provider now
pass with B3's macOS process facts); layerfs-daemon and -store-service
50 passed, 0 failed, 3 ignored; layerfs-overlay and -persistence 200
passed, 0 failed, 2 ignored. Boundary guard PASS with self-tests;
tools/test_ci_tests.py OK; agent calls, exec_output and packaging
surface/sandbox/lifecycle PASS. Not run: Linux build and tests (no
Linux C toolchain on the host, Docker not running), a live mount_nfs
mount, Docker and FUSE E2E. Disposable only; Durable NOT_RUN, disabled
by owner.

Production LOC: 115210 -> 119822 (delta +4612)
First-parent comparison: layerfs_v2_0_0 7d55f02 (tree c17f04f) to
prepared merge tree 63d9a3c.
Scopes: core 38400 -> 41405 (+3005), engine 73515 -> 75122 (+1607),
api 3295 -> 3295 (+0). By crate: layerfs-nfs +3005 (new; the counter
does not yet know this crate, so it falls in core by its default, though
it sits in the engine layer), layerfs-daemon +1122, layerfs-node +231,
layerfs-sandbox +226, layerfs-store-service +28. Matches the +4612 in
#525's description.
Method: tools/production_loc.py (blob 686d269)
--root <exported tree> --json for both snapshots; first-party production
Rust, runtime SQL and JavaScript; tests, docs, tooling, generated and
third-party files excluded.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
Deleted tests: none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yifanxuaaa

Copy link
Copy Markdown
Contributor

@jiangkoumo This PR's head fa02a1b2f8 is now in layerfs_v2_0_0 through B3 (#525, merge commit a1cd85c). Closing here because this PR targets main. Thank you!

本 PR 的头 fa02a1b2f8 已通过 B3(#525,合并提交 a1cd85c)合入 layerfs_v2_0_0。本 PR 的目标分支是 main,因此在此关闭。感谢!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants