Skip to content

[upstream issue #446] LayerFS 2.0 roadmap #31

Open

Description

@mengmengmengqiang

[upstream issue Ephemeral-AI-Lab#446] LayerFS 2.0 roadmap

Original body (excerpt)

LayerFS 2.0. Agents tool call side effects in LayerFS. Git for task checkpoint.

From a filesystem to an agent application

LayerFS 1.0 delivered the core: one SQLite Store, Workspaces that mount a whole project without copying it, a Commit per call, Branches, and a Node SDK and CLI. What 1.0 left unclear is the application level: how an agent uses LayerFS day to day, and what a person sees and gets from it.

LayerFS 2.0 answers that with an agent application, LayerFS Connect, built on LayerFS. The work falls into seven areas:

Area Repository
A Store backends: SQLite and PostgreSQL layerfs
R Remote storage for the SQLite Store file layerfs
B Sandbox: Docker and native Linux, macOS and Windows layerfs
P API: Node, Python, CLI and MCP layerfs
U LayerFS Connect UI layerfs-connect
C pi agents and sessions in Connect layerfs-connect
M Models and providers in Connect layerfs-connect

LayerFS stays neutral to plugins: Connect and pi use only LayerFS's general, published API, and no LayerFS code names them. When a Connect issue needs a new LayerFS call, the LayerFS issue describes the general need and the Connect issue is marked blocked by it.

Progress is tracked on the LayerFS 2.0 project, which covers both repositories.

LayerFS 2.0 architecture: pi agents with an API plugin; file calls go straight to the Store, Bash calls run in each agent's sandbox, one Workspace per call; the SQLite Store pushes and pulls to remotes and PostgreSQL is a second backend; finished tasks publish to Git and GitHub.

LayerFS commit history growing: many agent Branches fork from main and from each other, one Commit per tool call, and finished work merges back as Layers L14 to L17.

Note

Status: proposal. Nothing below is released or committed to a date. Items have IDs (A2, R5 and so on) for issues and PRs to cite; an item's issue title starts with its ID, such as [A2]. Decisions (D) are owner rulings, recorded in AGENTS.md by D7. The pictures above predate this plan: publishing to Git is now under Later.

Tip

Where the Connect items come from. The Connect prototype already has most of these screens, specified in docs/ui. Its code is a scratch prototype: everything is redesigned (U1) and rebuilt. Several LayerFS items below (P7–P14, B8) replace workarounds the prototype uses today, such as tailing trace files, polling every 2 s, and running docker exec … stat to measure the Store.

D · Decisions

  • D1 · PostgreSQL as a second live Store backend: confirm, and choose its persistence profile. Overturns the one-SQLite-file rule (#494)
  • D2 · How native macOS and Windows sandboxes reach the Store: the Linux worker, PostgreSQL, or reopen the Linux-only rule
  • D3 · The 2.0 UI is LayerFS Connect, in layerfs-connect, redesigned from scratch (owner, 2026-10-28; replaces the dsh-plugin UI rule once recorded)
  • D4 · S3 client: bring back layerfs-s3, or add the object_store crate
  • D5 · The pi extension lives in layerfs-connect (owner, 2026-10-28)
  • D6 · Cloud sandbox to explore first: E2B, Daytona, Modal or AWS Fargate
  • D7 · Docs-only PR putting the rulings into AGENTS.md (after the other decisions)
  • D8 · Split the repository into core, engine and api layers (D8 · Refactor core/ into core, engine and api layers Ephemeral-AI-Lab/layerfs#463)
  • D9 · Two machines, one remote: pull replaces the local Store, and push refuses when the remote changed since the last pull
  • D10 · Python SDK: a native binding over the Rust API (probably PyO3, a new dependency), or a client of the worker only
  • D11 · MCP server: built on the Node or Python SDK; stdio or HTTP
  • D12 · How a cloud sandbox reaches the Store: PostgreSQL only (as the Connect prototype assumes), or download, run and push back (B7)
  • D13 · Native macOS mount: loopback NFSv3 (feat: macOS 原生适配——daemon 内置 loopback NFSv3 导出,普通用户 mount_nfs 挂载 Workspace(无 Docker / 无 macFUSE / 无 root) Ephemeral-AI-Lab/layerfs#427), or macFUSE/FSKit (as the Connect prototype assumes)
  • D14 · How Connect reaches a PostgreSQL Store: directly, or over HTTP. HTTP needs a Store server, which today's rules forbid
  • D15 · Scripted scenarios and demo projects: keep them as an end-to-end test suite and onboarding tour, or drop them. The dsh demo bundle is disabled permanently today
  • D16 · Where project details (title, description, created, origin) live: in the Store, or in Connect

A · Store backends: SQLite and PostgreSQL

PostgreSQL becomes a second backend for the global Store. Each Workspace's overlay stays its own local SQLite file. layerfs-persistence already has a BackendSelection::Postgres placeholder; everything behind it is SQLite today.

  • A1 · Backend interface in layerfs-persistence: reads, transactions, admission and the publication lane behind one seam, with SQLite as the first implementation
  • A2 · PostgreSQL schema and object storage; check pack sizes against the 1 GB bytea limit (after A1, D1)
  • A3 · PostgreSQL publication lane: one writer through an advisory lock, short holds (after A2)
  • A4 · PostgreSQL persistence profile, declared like Disposable (after D1)
  • A5 · Init into PostgreSQL, or copy a SQLite Store into PostgreSQL (after A2)
  • A6 · The same test suite on both backends; speed and storage measured against SQLite (after A3)
  • A7 · Hosted PostgreSQL checked: Cloudflare, AWS, Google Cloud (after A6)
  • A8 · Delete a Branch and reclaim its history. Failed agent runs leave Branches behind today (was in Later)
  • A9 · Project details in the Store: title, description, created, origin (after D16)

Done when: one project runs on SQLite and on PostgreSQL with identical history, and both are measured for speed and storage.

R · Remote storage for the SQLite Store file

A remote holds the whole Store file. Push uploads it; pull downloads it. LayerFS never opens SQLite over the network. PostgreSQL projects don't need a remote: their database is already remote.

  • R1 · Research Google Drive, Cloudflare R2, MinIO and AWS S3: maximum file size, single versus multipart or resumable upload, conditional overwrite, download resume, cost and sign-in
  • R2 · A consistent snapshot of

…(正文截断;完整内容见上游链接)

Sync note

这是只读镜像,不是上游仓库的替代。上游 issue/PR 仍是唯一真源;本镜像不复制评论、不复制代码分支。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    upstream-issueMirrored upstream issueupstream-syncMirror of Ephemeral-AI-Lab/layerfs upstream

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions