Skip to content

Windows-native Store route: Store engine and SDK Store face on native Windows - #452

Open
NelsonLongxiang wants to merge 39 commits into
Ephemeral-AI-Lab:mainfrom
NelsonLongxiang:windows-native-store
Open

NelsonLongxiang wants to merge 39 commits into
Ephemeral-AI-Lab:mainfrom
NelsonLongxiang:windows-native-store

Conversation

@NelsonLongxiang

@NelsonLongxiang NelsonLongxiang commented Oct 25, 2026 •

Copy link
Copy Markdown

Windows-native Store (owner direction 2026-10-25)

Brings the Store base to native Windows: SQLite (bundled), the cross-process
writer lane (a named kernel mutex, Global\layerfs-writer-lane-<sha256-64bit-of-verbatim-canonical-dir>), the kernel guard (local NTFS only, long-path
correct), owner-only DACLs on homes/records/Stores/sidecars, real
volume-serial + file-index acquisition identity — all inside audited FFI
modules. The SDK's in-process Store face (fs, branch, project, operator) runs
natively; no WSL2, no daemon. Workspace/FUSE/Exec/daemon calls still
refuse Unsupported. Contract:
DIRECT_STORE.md §Windows-native route.

Head 2dd58881f (38 commits over main; main has not moved under it).

Verification state (seven rounds, independent assessors)

  1. Code review — Windows (17 findings) + Linux r1 (5 must-fix) + r2/r3.
  2. Production readiness + adversarial round — six fixes live; TOCTOU
    6,431 attempts, soak, 8,440-file load all passed.
  3. Fix-verification / concurrency / cross-platform bytes (one Store file
    Win↔Linux 14/14+14/14 byte-exact) / perf / delivery
    — lane adjudicated
    live (name-agnostic enumeration; the pre-fix anomaly stays unexplained
    instrument-side, corrected on the record); fixes in a7f5cda78.
  4. Global-lane adversarial (holder-side causality) / sidecar semantics /
    soak / evidence-trail meta-audit / overhead (≈0.2 ms/open)
    — fixes in
    c2ca6caa6 with the round-3 narrative correction.
  5. Cold-eyes final read found the branch's last P1 — the volume guard's
    dead regrowth branch admitted deep shares as NTFS — fixed in d9687cfa4
    with new boundary tests; the collapsed-name harm fenced by a
    display/decision join split (ADR'd); §6 recovery runbook rewritten from an
    end-to-end verification; 34-min soak and CJK-host-path diversity passed.
  6. Fix adversarial verification — all three round-6 fixes attacked and
    upheld; the fence proven end-to-end at Store+merge level (planned merges
    never touch it; full walks refuse InvalidRecord deterministically,
    Windows and Linux); merge rehearsal v2 (merge tree byte-identical to the
    branch tree, all chains + the frozen verify spec green on the merge
    commit); fence overhead ≤ noise (±11%, multi-byte arm never slower);
    SDK test-surface map: zero branch regressions, net improvement (trace
    ×9 fixed vs main, three new store chains pass); one process slip — the
    round-6 test file had not been staged — corrected in 2dd58881f with a
    join_exact unit test added. Full reports: the seven round comments.

Chains on 2dd58881f: content 484/0 + compare_join_exact 2/2 +
volume-guard 2/2 + windows trio 6/2·2/2·1/1, clippy clean, SDK 39/39,
container green, fork CI green (one cold-start transient on the previous
head, rerun green). Frozen verify family: three specs (fresh /
crash-recovered / CJK-host-path), all PASS 5/0 on family re-runs.
Production LOC 104239 → 104281 over the round-3..6 fixes, per-commit
comparisons in every message.

Recorded limitations (details in DIRECT_STORE §6)

Windows Init synthesizes mode from the read-only attribute; no hard-link
membership. Names travel as owner-ratified WTF-8 (2026-10-31); listings
render a non-UTF-8 name empty, a JS write by such a name lands as an U+FFFD
sibling, and a merge/compare whose walk must build a String path over such a
name refuses InvalidRecord (display vs decision join split). Long paths:
verbatim everywhere; plain-spelled host operations depend on the host's
long-path policy. Lane squatting and pre-planted-foreign-DACL objects are
local-adversary availability trade-offs; lane names are enumerable. No
64 MiB headroom pre-admission (typed Capacity). %TEMP% overlay
directories of killed openers accumulate. Upstream-family records: the
Linux FUSE invalidation keys the display spelling; the 256-component path
limit is not enforced on the SDK routes; deep single-chain costs grow
near-quadratically. CI adds one report-only windows job.

Merge

Owner ratified 2026-10-31 ("可以合并"). Create a merge commit (no-ff) —
merge key stays with the owner in the GUI. The rehearsal (twice, latest on
d9687cfa4) shows the merge tree byte-identical to the branch tree while
main stands at 35d36575c, every chain green, and the post-merge script is
rehearsed end to end.

… open

The Store engine runs natively on Windows (owner direction 2026-10-25,
route 2: Windows-native Store, no WSL2):

- The cross-process writer lane is a named kernel mutex
  (Local\layerfs-writer-lane-<hash>): CreateMutexW + INFINITE wait;
  WAIT_ABANDONED also takes ownership, so a dead writer's lane is
  released by the kernel and the next writer proceeds.
- The kernel guard reads the volume filesystem via
  GetVolumeInformationW and allows local NTFS volumes only; other
  filesystems and non-disk prefixes refuse Unsupported, matching the
  Linux ext4/xfs/btrfs allowlist discipline.
- SQLite opens on Windows through bundled rusqlite; the open/profile
  cfg lists widen from macos/linux to include windows. Linux and macOS
  branches are unchanged.
- The installer drops the unix-only mode bits and the daemons-uid
  check on Windows; destination and exclusivity checks are unchanged.
- Boundary guard: audited-FFI entries become per-crate tuples and gain
  persistence backend/sqlite/windows_lane.rs; path matching is
  as_posix so the audited-module check works on Windows hosts (this
  repairs two false positives the separator bug produced there).

Verification at this commit: cargo check --offline --locked
-p layerfs-persistence -p layerfs-store-service (Windows, rustc
1.90.0) Finished; the guard's 24 self-tests pass; the guard reports
only the four test-only-attribute findings that HEAD already reports,
with no new violation; tests/windows_store_native.rs (new) drives
create + two-session writes + two re-exec'd child processes through
the lane with bounded waits: 1 passed.

Production LOC: core 96783 -> 96948 (delta +165), sdk 3090 -> 3090
(delta 0); combined 109056 -> 109221 (delta +165). Counter:
tools/production_loc.py --detail (pinned at this tree), scopes core
and sdk, tests/tools/docs excluded. reference absent (retired R9c;
last counted 65,417 at tree 498dd19).
…e face

The SDK's in-process Store face and the construction libraries it needs
run natively on Windows (route 2, owner direction 2026-10-25):

- layerfs-project Init: host byte encoding moves to import/host.rs;
  Windows carries names as UTF-8 bytes (lossy outside UTF-8, a recorded
  limitation) and synthesizes the mode from the read-only attribute.
  Acquisition identity is the real volume serial and file index from
  GetFileInformationByHandle in the audited import/windows_identity.rs;
  zeroed identities made every entry collide onto one dev+ino key and
  Init refused with "the entry changed while Init read it". The
  cfg(unix) raw-byte and stat paths are unchanged.
- layerfs-overlay: audited database/windows_identity.rs provides handle
  and path identity (volume serial, file index, link count) and
  GetDiskFreeSpaceExW free space; physical allocation checks and the
  startup gate widen to windows. store-service reads overlay backing
  identity through it instead of the (0,0,0) placeholder.
- The napi addon builds on Windows: Store discovery (discovery_windows)
  uses a named kernel mutex with save/load record files; fs, branch,
  project and operator calls run in-process through the Ephemeral-AI-Lab#352 direct
  route; workspace, exec and daemon calls refuse Unsupported - FUSE and
  the daemon stay Linux-only. exec/output/pool/sandbox/workspace stay
  unix modules; the unix dispatch table is unchanged.
- Boundary guard: audited FFI adds project import/windows_identity.rs
  and overlay database/windows_identity.rs; both crate roots move
  forbid->deny. Guard self-tests cover both.

No tests were deleted or weakened; store-service keeps HEAD's Windows
cfg asymmetry warnings no longer (backing delete lets and identity
fields are cfg-scoped).

Verification at this commit: cargo check --offline --locked for
layerfs-persistence, -store-service, -project, -overlay and -node
passes on Windows (rustc 1.90.0, only HEAD's pre-existing warnings in
untouched crates) and in the rust:1.85.1-bookworm container (zero
warnings for all five); tests/windows_store_native.rs 1 passed; guard
self-tests 24/24; the lifecycle smoke (init -> fork dev -> one-Commit
fs.write -> read back on dev and main -> list_commits -> close, no
WSL2, no daemon) passes against this commit's addon.

Production LOC: core 96948 -> 97232 (delta +284), sdk 3090 -> 3455
(delta +365); combined 109221 -> 109870 (delta +649). Counter:
tools/production_loc.py --detail (pinned at this tree), scopes core
and sdk; overlay +146, project +126, store-service +12, layerfs-node
+365. reference absent (retired R9c; last counted 65,417 at tree
498dd19).
Draft for owner review, landing with the windows-native-store branch:

- Root AGENTS.md: a Windows-native Store workstream row and a mental-model
  bullet superseding the Linux-only Store sentence for this route only,
  with its recorded limitations.
- core/AGENTS.md: the audited Windows FFI inventory (persistence
  windows_lane beside the macOS seal wrapper; project and overlay
  identity modules, forbid->deny), the real volume-serial + file-index
  requirement for acquisition rows, and the Linux-container cross-check
  rule for cfg(unix) code touched on this route.
- DIRECT_STORE.md: section 6 for the Windows-native route - mechanisms,
  what still refuses Unsupported, limitations, proof receipts.
- 83-store-concurrency-model.md: the writer-lane table row names the
  Windows named kernel mutex equivalent (abandoned mutex taken by the
  next waiter).

The two pre-existing broken relative links found while checking
(DIRECT_STORE.md -> DIRECT_STORE_IMPL.md, 83 -> 84-global-sqlite-store.md)
are left for their owners; this change adds no new link and all links it
adds resolve.

Production LOC: core 97232 -> 97232 (delta 0), sdk 3455 -> 3455
(delta 0); combined 109870 -> 109870 (delta 0). Docs only; same counter
and scope as the previous commits. reference absent (retired R9c; last
counted 65,417 at tree 498dd19).
…open

The 3b slice of the Windows-native Store route: the existing test suites
now run natively on Windows, and one real open-path race they exposed is
fixed for every platform.

Product changes:
- Concurrent Store opens met Busy under the no-retry rule: the profile
  pragmas (first database-touching statements, where a WAL -shm is
  created and recovery runs) and the writable setup block
  (binding hints, lineage, merge scratch, file digests, merge-record
  sweep) ran outside the writer lane. Each is now one short lane hold,
  released before the reads that follow; macOS keeps exact SQLite Busy
  by design. The lane verifier's child failed "the child opens the
  Store: Busy" intermittently before this; eight consecutive runs pass
  after it.
- import/host.rs: a writable Windows file synthesized mode 0o644, the
  read-only base - the read-only attribute was not stored. Base is
  0o444/0o555 as documented.
- outside_source on Windows becomes a real containment check of the
  resolved placement inside the resolved source (junctions and symlinks
  resolve); only spellings that do not resolve to the source's own path
  (a subst drive, an unresolvable alias) stay unrecognized, replacing
  the previous unconditional no-op.
- windows-sys features Win32_Security is declared by layerfs-project
  and layerfs-overlay themselves: CreateFileW is gated on it by its
  SECURITY_ATTRIBUTES parameter, and the crates only compiled before
  through another crate's feature unification.

Test changes (delete-or-adapt; deleted tests listed):
- Deleted: init_sqlite's an_unsupported_platform_refuses_the_store_
  before_any_file_exists - its premise (Windows refused) is inverted by
  this route; the namespace oracle tests it shared a file with now run
  on Windows.
- unix-only kept unix-only: serverless_store (socket readiness), the
  four issue327 counters examples and restore_record (st_blocks / 0o600
  records; non-unix mains refuse instead of fabricating).
- Adapted: project support asserts the host's own reading of each entry
  (raw stat on unix; the synthesized contract on Windows); the held
  writer helper drops select() for a blocking readline (WinError 10093)
  and trims the readiness line; kernel_guard tests the Windows
  non-local-spelling refusal beside macOS's; the lane verifier pipes its
  children's report into its failure instead of discarding it.
- New: sdk/js/test/store_lifecycle.mjs - init, fork, one-Commit write,
  read back, stat, commit listing and close on the real binding, no
  Docker, daemon or WSL2; it runs on both Linux (the E2E volume) and
  Windows. trace.mjs imports its fixture through pathToFileURL (a bare
  Windows path is not an ESM URL); reinit/init_refusals gate their
  unix-only contracts explicitly.

Verification at this commit (Windows, rustc 1.90.0): cargo test for
layerfs-persistence, -project, -store-service, -overlay, -node: 173
passed, 0 failed; the persistence suite clean across four consecutive
runs. SDK: store_lifecycle, binary_write, reinit pass; trace 9/9 cases;
packaging 2/2; init_refusals reports its unix-only skip. Linux
(rust:1.85.1-bookworm container): cargo check --offline --locked
--tests --examples for the five crates, Finished. Boundary guard
self-tests 24/24; no new violation.

Production LOC: core 97232 -> 97258 (delta +26; persistence +18,
project +8), sdk 3455 -> 3455 (delta 0); combined 109870 -> 109896.
Counter: tools/production_loc.py --detail (pinned at this tree), scopes
core and sdk; tests, examples and manifests excluded. reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Slice 3b added (ee6c34fba): the existing Store test suites now run natively on Windows — 173 tests across layerfs-persistence, -project, -store-service, -overlay, -node, 0 failed, clean across repeated runs; the Linux 1.85.1 container checks the same crates including tests and examples.

Two product defects the Windows runs exposed, fixed for every platform:

  1. Concurrent Store opens met Busy under the no-retry rule. The profile pragmas and the writable setup block (binding hints, lineage, merge scratch, file digests, merge-record sweep) ran outside the writer lane; the first database-touching statements create/recover a WAL -shm, so two simultaneous opens of one Store could fail where the lane should have serialized them. Each block is now one short lane hold (macOS keeps exact Busy by design). The lane verifier's re-exec'd child intermittently failed the child opens the Store: Busy before this; eight consecutive runs pass after.
  2. Windows mode synthesis stored writable files as 0o644 (the read-only base), so the read-only attribute was lost; now 0o444/0o555 +0o200 as documented. outside_source on Windows also becomes a real resolved-containment check instead of a no-op, and project/overlay declare the Win32_Security windows-sys feature they actually need (they previously compiled only through another crate's feature unification).

Tests: the one test whose premise this route inverts (Windows refused the Store) is deleted and listed in the commit message; unix-only proofs stay unix-only with refusing non-unix stubs (no fabricated metrics); the SDK gains sdk/js/test/store_lifecycle.mjs, the promoted lifecycle proof on the real binding, no Docker/daemon/WSL2, running on both Linux and Windows.

Production LOC: core 97,232 → 97,258 (+26), sdk unchanged; per-commit numbers in the commit message.

Slice 4 of the Windows-native Store route: sdk/js/test/store_matrix.mjs
drives the Store face's whole public method table through the real
binding on the direct route - writes (write, edit, mkdir, cp, mv, rm),
reads (read with a line range, list, stat, grep, glob, blame),
comparisons (diff_files, diff_commits, diff_branches) and combination
(merge with its merged-branch readback, publish with main's readback and
the branch listing) - plus two contracts worth pinning: main is the
project's main line, not a Branch, so branch.list_commits("main") is
NoSuchBranch and main moves only through branch.publish; and on Windows
the daemon-only workspace surface refuses Unsupported once the client's
own deploy validation is satisfied. No Docker, no daemon, no WSL2; the
same file runs on Linux against the E2E volume.

Verification: three consecutive rounds of store_matrix, store_lifecycle,
binary_write and reinit all pass on Windows against this tree's addon;
trace (9 cases) and packaging (2 cases) unchanged.

Production LOC: core 97258 -> 97258 (delta 0), sdk 3455 -> 3455
(delta 0); combined 109896 -> 109896. Test-only change: the sdk scope
counts sdk/js/src and sdk/js/operator, so the new test contributes no
production lines. reference absent (retired R9c; last counted 65,417 at
tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Slice 4 added (843d22f3a): the Store face's full public method matrix is now a permanent test — sdk/js/test/store_matrix.mjs, real binding, direct route, no Docker/daemon/WSL2, dual-platform.

Covered: every fs write (write, edit, mkdir, cp, mv, rm), every fs read (read with a line range, list, stat, grep, glob, blame), all three comparisons (diff_files, diff_commits, diff_branches), and branch combination (merge with merged-branch readback, publish with main's readback). Three consecutive rounds of the four real-binding SDK tests pass on Windows.

Two contracts the matrix pins, both confirmed as designed (cross-platform, not Windows-specific):

  • main is the project's main line, not a Branch: branch.list_commits("main") is NoSuchBranch by the Branch name index; main moves only through branch.publish.
  • On Windows the daemon-only workspace surface refuses Unsupported at the native layer once the client's own deploy validation is satisfied.

No product change was needed: the matrix passed against the current implementation once the contracts above were honored. Test-only commit; production LOC unchanged (core 97,258, sdk 3,455).

Four re-exec'd child processes write distinct files to one branch of one
Store concurrently, each through its own client of the direct route: the
named-mutex writer lane serializes the publishes and the discovery
record guard serializes the shared home's project record, so every
write must land. The parent reads all four files back and counts the
Commits exactly (base + 4). Bounded: each child is killed at 30 s and
its captured output is carried into the parent's failure. No Docker, no
daemon, no WSL2; the same file runs on Linux against the E2E volume.

Verification: four consecutive runs pass on Windows against this tree's
addon (4 writers, base+4 commits each time).

Production LOC: core 97258 -> 97258 (delta 0), sdk 3455 -> 3455
(delta 0); combined 109896 -> 109896. Test-only change. reference
absent (retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Evidence round added (ada6f39f7 + verify chains): responding to "the row-count chain proves too little", two harder pillars.

1. Deep machine chain on a divergence/merge/publish scenario (spec_321a94534c75, material v3): the frozen snapshot comes from init(3 files) → fork work → write → fork feat → write → merge → publish. Seven exact-equality clauses, all asserted against frozen facts read out of the snapshot first (never invented): branch = 4; branches with head_commit_id is not null = 3 (publish leaves main on the layer line — a real behavioral pin); commit_lineage = 4; layer = 2; file_digest with length > 0 = 2; object_location = 22; store_totals with stored > 0 = 1. Run run_bb6e43bde4b6a7b1: PASS 7/7 with witnesses, archived arc_955936358dc2 (archive verified, contract 1.2). The reserved-word commit table stays out of the clauses by design (runner limitation, documented in the material).

Controls, not just the pass: a determinism control — the scenario independently re-produced from scratch, same spec, PASS 7/7 again with identical witnesses (run_a6b14902309b33b1), so the exact counts are properties of the operation sequence; and a hollow negative control (tables present, rows absent) → FAIL 7/7.

2. Cross-process concurrent writers (sdk/js/test/store_concurrent.mjs, ada6f39f7): four re-exec'd child processes, each its own client, writing distinct files to one branch of one Store simultaneously — the named-mutex writer lane serializes publishes and the discovery record guard serializes the shared home; all four must land and be readable, commit count exactly base+4, each child bounded at 30 s with its output carried into any failure. Four consecutive runs pass on Windows.

Predicate DSL notes for the verify engine (fed back earlier via kb): where ops are the closed set =/!=/>/>=/</<=/in/between/is null/is not null; exists needs explicit expect: true.

Two coverage pins from the confidence review:

- A dead lane holder does not block the next writer: a thread ends while
  owning the product's own named mutex (the name derived exactly as
  windows_lane derives it), the kernel marks it abandoned, and the Store
  still opens and writes through that WAIT_ABANDONED acquisition.
  Deterministic - no timing, no kill race.
- The matrix test's merge becomes a divergent one: both sources advanced
  past their common base, so fastForward must be false, the merged head
  carries two parents, and both sides' files are read back on the merged
  Branch and on main after publish - the merge's content correctness,
  which the earlier fast-forward case could not show.

Verification: windows_store_native 2 passed; store_matrix passes with
the five-branch end state on Windows.

Production LOC: core 97258 -> 97258 (delta 0), sdk 3455 -> 3455
(delta 0). Test-only change. reference absent (retired R9c; last
counted 65,417 at tree 498dd19).
…ne test

The 3b lane wraps deadlocked Linux opens: the setup block's raw second
Writer::acquire held the directory flock on one open file description
while Session::run's admission inside acquired it on another - flock is
per description, so the opening thread waited on itself. Windows passed
only because its named kernel mutex is thread-recursive; the Linux
container suite caught it (concurrent_first_installs: 0.4s at the
parent commit, >115s hung at 3b, diagnosed by /proc/locks, per-acquire
backtraces and a parent-commit rerun).

The corrected shape, verified on both platforms:

- Session::connect holds the writer lane for the whole connect-time
  statement block (profile pragmas, stored-format and profile reads) -
  the open's first database-touching statements, where a cleaned -shm
  is rebuilt under an exclusive recovery hold.
- The writable setup block (binding hints, lineage, merge scratch, file
  digests) runs inside one held publication lane: its statements enter
  as nested turns and take no second lock; cross-process opens are
  serialized by the lane itself.
- The merge-record sweep stays plain: its admission takes the lane.

Also: a_dead_lane_holder_does_not_block_the_next_writer moves to its own
binary (windows_lane_abandoned.rs) so it cannot run beside the
foreign-process test's re-exec'd children, whose spawn now filters
--exact; the persistence test support's temporary directory names gain
nanoseconds (the project crate's already had them), removing cross-run
name collisions on Windows pid reuse.

Verification: Linux container - concurrent_first_installs 0.37s;
persistence 99 passed 0 failed, project 22/0, store-service 3/0,
overlay 74/0. Windows - the lane binary 50/50 clean isolated runs; the
SDK suite (store_matrix, store_concurrent, store_lifecycle) passes on a
rebuilt addon. One residual, documented openly: under cargo test the
lane test's re-exec'd child still meets Busy roughly once per 25 suite
runs (never in 50 isolated binary runs); its report is carried into the
failure. The busy source inside a mutex-held open remains undiagnosed -
recorded as an open item, not fixed by this commit.

Production LOC: core 97258 -> 97255 (delta -3; persistence -3: the closed-over profile tail moves lines, the lane-held setup comment replaces the raw wrap), sdk
3455 -> 3455 (delta 0). Counter: tools/production_loc.py (pinned at
this tree), scopes core and sdk; tests excluded. reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Confidence round + a real bug it caught (5a41b5f11, eb0203bc6/3d90143b1): the three cheap evidence steps from the confidence review, and what the third one found.

1. Divergent merge readback (store_matrix.mjs): both sources advance past their common base → fastForward: false, the merged head carries secondParent, and both sides' files read back on the merged Branch and on main after publish — the merge's content correctness, which the earlier fast-forward case could not show.

2. Deterministic dead-lane test (windows_lane_abandoned.rs, own binary): a thread ends while owning the product's own named mutex (name derived exactly as windows_lane derives it) → the kernel abandons it → the Store still opens and writes through the WAIT_ABANDONED acquisition. No timing, no kill race.

3. Linux suite runs — this caught a real bug I had shipped in 3b. concurrent_first_installs finished in 0.4s at the parent commit and >115s hung at 3b: my setup/sweep wraps self-deadlocked the opening thread on Linux — the raw Writer::acquire held the directory flock on one open file description while Session::run's admission inside acquired it on another (flock is per description). Windows passed only because its named mutex is thread-recursive. Diagnosed with /proc/locks, per-acquire backtraces and a parent-commit rerun; a bisect over the three wrap sites isolated it.

Corrected shape (3d90143b1), verified on both platforms: Session::connect holds the lane for the whole connect-time statement block (profile pragmas + format/profile reads — where a cleaned -shm is rebuilt under an exclusive recovery hold); the writable setup block rides one held publication lane (its statements enter as nested turns, taking no second lock); the sweep stays plain (admission lanes it). Linux: heavy test 0.37s; persistence 99/0, project 22/0, store-service 3/0, overlay 74/0. Windows: lane binary 50/50 isolated clean runs; SDK suite passes on a rebuilt addon.

One residual, stated openly: under cargo test, the lane test's re-exec'd child still meets Busy roughly once per 25 suite runs (never in 50 isolated binary runs; its report is piped into the failure). The busy source inside a mutex-held open remains undiagnosed — recorded as an open item on this PR, not papered over.

Test-infra hardening en route: the abandoned test moved to its own binary (it ran beside the lane test's re-exec'd children); the children's spawn filters --exact; the persistence test support's temp names gain nanoseconds (Windows pid-reuse ghost collisions).

The residual lane-test flake is diagnosed and closed. Instrumented runs
pinned it: a re-exec'd child met SQLITE_BUSY(5) on its first statement
(PRAGMA synchronous=0, where a cleaned -shm is attached and possibly
rebuilt) while holding the named writer mutex - the contender was the
sibling process's connection teardown, whose close-time WAL checkpoint
is SQLite work that no lane had ever covered. The same window exists on
Linux through flock and in the daemon SDK's own direct-route close.

The fix coordinates the close instead of the drop: layerfs-persistence
exports StoreLane (a best-effort hold of the Store directory's writer
lane, released on drop) and Handles::shutdown, which drops the session
under that lane; the SDK's direct route holds the lane while the
service's sessions close on storeClose; the lane tests' children and
the abandoned-lane test close through shutdown. A plain drop keeps its
old bare semantics - an uncoordinated close, like any error-path
teardown, never waits behind the lane.

An intermediate design (acquiring the lane in State::drop) is reverted:
it deadlocked multi-session teardown in-process, caught by the SDK
lifecycle (init+fork+write+close hung). Drop-time blocking locks stay
out of the product.

Verification: Windows - the persistence suite 25 consecutive rounds
clean under cargo (the flake reproduced every 2-9 rounds before), the
lane binary's children closing through shutdown; SDK store_matrix,
store_concurrent, store_lifecycle and a minimal init-fork-write-close
all pass on the rebuilt addon. Linux container - persistence 99/0,
project 22/0, store-service 3/0, overlay 74/0.

Production LOC: core 97255 -> 97271 (delta +16; persistence +14, node
binding +2), sdk 3455 -> 3457 (delta +2); combined 109893 -> 109911. Counter:
tools/production_loc.py (pinned at this tree). reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Residual closed (d960a38a6): the ~1/25 cargo-suite child Busy is diagnosed and fixed.

Diagnosis (instrumented): the child met SQLITE_BUSY(5) on its first statement — PRAGMA synchronous=0, where a cleaned -shm is attached and possibly rebuilt — while holding the named writer mutex. The contender was the sibling process's connection teardown: SQLite's close-time WAL checkpoint, which is SQLite work no lane had ever covered, on any platform. The same window exists in the SDK's direct-route close.

Fix: coordinate the close, not the drop. layerfs-persistence exports StoreLane (best-effort lane hold, released on drop) and Handles::shutdown (drops the session under the lane); the SDK's storeClose holds the lane while the service's sessions close; the lane tests' children close through shutdown. A plain drop keeps bare semantics — error-path teardown never waits behind the lane. An intermediate design (acquiring in State::drop) is documented as reverted: it deadlocked multi-session teardown in-process, caught by the SDK lifecycle before it shipped.

Receipts: Windows persistence suite 25 consecutive cargo rounds clean (the flake reproduced every 2–9 rounds at the prior design); SDK store_matrix/store_concurrent/store_lifecycle and a minimal init-fork-write-close all pass on the rebuilt addon; Linux container persistence 99/0, project 22/0, store-service 3/0, overlay 74/0.

StoreService::close now records the Store file (open_direct populates
it through the new StoreService::at) and, after the overlay owner stops
- its final Commits take the writer lane themselves, so no outer hold
may surround that - drops the service under the Store's writer lane:
the sessions' close-time checkpoints never run beside another opener's
first attach. This is the daemon/SDK-symmetric completion of the
coordinated close the SDK's direct route received; direct.rs drops its
own manual hold in favor of the service's (a second hold on another
descriptor would self-deadlock the closing thread on Linux).

A service built with new() keeps an empty path and closes bare, as
before - only open_direct records the file.

Verification: Windows store-service tests 3/0, SDK store_matrix,
store_concurrent, store_lifecycle pass on the rebuilt addon (the
service close is in storeClose's path). Linux container: store-service
3/0; persistence 99/0, project 22/0, overlay 74/0 at this tree. The
daemon suite needs a privileged container (/dev/fuse, fusectl, cgroup
mounts): 62 non-mount tests pass; the 99 mount-path failures are
environmental (attach:mount No such file, permission denied), the same
at the parent commit - recorded as a pre-existing container limitation,
not a regression of this change.

Production LOC: core 97271 -> 97281 (delta +10; store-service +10), sdk 3457 -> 3455 (delta -2; the manual hold's comment
moves into the service). Counter: tools/production_loc.py (pinned at
this tree). reference absent (retired R9c; last counted 65,417 at tree
498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Coordinated close, daemon-symmetric + a third machine chain (head 27f6cf89a):

  1. StoreService::close now coordinates its Store teardown (StoreService::at records the Store file; overlays stop first because their final Commits take the lane themselves, then the service drops under the lane). The SDK's storeClose uses the service's hold instead of its own — a second hold on another descriptor would self-deadlock the closing thread on Linux. Services built with new() keep bare close semantics. Receipts: Windows store-service 3/0 + the three SDK tests on the rebuilt addon; Linux store-service 3/0 (persistence 99/0, project 22/0, overlay 74/0 at this tree). The daemon suite needs a privileged container (/dev/fuse, fusectl, cgroup): 62 non-mount tests pass; the 99 mount failures are the same at the parent commit — environmental, recorded as such.

  2. Verify chain v4 — the cross-process concurrent scenario (spec_9a8d135fb270): four child processes, each its own SDK client, writing concurrently to one branch through the writer lane. Seven clauses, facts frozen from the snapshot: branch = 2; heads-set = 1; commit_lineage = 4 (every writer's Commit, none lost or duplicated); file_digest(length>0) = 4; object_location ≥ 30 (35); layer = 1; store_totals(stored>0) = 1. Run run_f48df5e7deac31d9: PASS 7/7, archive verified (arc_95e57e734c78). Controls: an independently re-produced concurrent interleaving passes again (run_744930f568abb520), and the hollow control fails 7/7 (run_0eb4d3c594dcc753).

K3 review (猿光智启专家团2.0——layerfs底座, APPROVE-WITH-NITS, F1):
the lane and discovery mutex names hashed the directory with
DefaultHasher, whose algorithm may change between Rust releases - an
older and a newer binary computing different names for one directory
would silently run two lanes. The names now derive from SHA-256 of the
resolved directory's encoded bytes, truncated to 64 bits (sha2 is
already in the locked graph; the direct edges add no new package), in
persistence's windows_lane (exposed as lane_name for the test) and the
SDK binding's discovery guard. The dead-lane test replicates the new
derivation. F2 (Local\ namespace does not cross sessions; the product
opens its Store within one login session today) is recorded as a §6
limitation in DIRECT_STORE.md, with the Global\ privilege evaluation
noted should cross-session coordination ever be needed.

Verification: Windows persistence 87/0 (both lane binaries), project
10/0, store-service 3/0, overlay 74/0; SDK matrix/concurrent/lifecycle
pass on the rebuilt addon; Linux container cargo check --offline
--locked for persistence and the node binding, Finished.

Production LOC: core 97281 -> 97282 (delta +1), sdk 3455 -> 3453
(delta -2); combined 109919 -> 109918. Counter: tools/production_loc.py
(pinned at this tree). reference absent (retired R9c; last counted
65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

K3 review findings landed (636a641bf): the 「猿光智启专家团2.0——layerfs底座」 review (K3 seat, APPROVE-WITH-NITS, all four dimensions PASS) returned two non-blocking findings; both are now addressed.

  • F1 (fixed pre-merge, as recommended): the lane and discovery mutex names hashed the directory with DefaultHasher, whose algorithm may change between Rust releases — old/new binaries over one directory would silently split into two lanes. Both names now derive from SHA-256 of the resolved directory's encoded bytes, truncated to 64 bits (sha2 was already in the locked graph; the new direct edges add no package). The dead-lane test replicates the new derivation and still passes through WAIT_ABANDONED.
  • F2 (documented): the mutexes live in Local\, which does not cross sessions — the product opens its Store within one login session today, so this is recorded as a §6 limitation in DIRECT_STORE.md, with the Global\ privilege evaluation named should cross-session coordination ever be needed. The docs seam the review flagged is the same sentence, so the note completes it.

Receipts: Windows persistence 87/0 (both lane binaries), project 10/0, store-service 3/0, overlay 74/0; SDK matrix/concurrent/lifecycle on the rebuilt addon; Linux container cargo check --offline --locked Finished.

…in doc 83

The concurrency model's table gains the open/teardown row: the
connect-time statement block (where a cleaned WAL -shm is rebuilt under
an exclusive recovery hold), the writable setup block (held publication
lane; nested turns take no second lock) and the coordinated close
(StoreLane/Handles::shutdown; the service stops overlays first, then
drops the sessions under the lane) all run under the same cross-process
lane - SQLite's close-time checkpoint is writer work like any other and
never runs beside another opener's first attach. The lane-name note
records the SHA-256 truncation (never DefaultHasher, K3 F1) and the
Local\ session-namespace limitation (DIRECT_STORE.md §6, K3 F2).
Also: the PR#452 observation script switches from the gh CLI to the
unauthenticated REST API so the job engine's non-interactive
environment (no gh auth) can run it.

Production LOC: core 97282 -> 97282 (delta 0), sdk 3453 -> 3453
(delta 0); combined 109918 -> 109918. Docs and tooling only; the shell
script lives outside the counted scopes. reference absent (retired R9c;
last counted 65,417 at tree 498dd19).
Four maintained-doc spots still said the Store face is Linux-only:

- sdk/js/README.md, the Store-calls paragraph: on Linux the Store sits
  on the daemons' shared local filesystem; on Windows the native route
  opens a local NTFS volume directly, no daemon involved (Workspace,
  exec and daemon calls refuse Unsupported there).
- the LAYERFS_STORE_PATH row: the same split (Linux shared local
  filesystem / Windows local NTFS volume).
- the Builds section: the build script refuses Windows by design; the
  Windows addon builds directly with cargo and renames layerfs_node.dll
  into native/layerfs.node.
- cluster_one_handbook's persistence row: macOS/Linux/Windows, with the
  owner-direction date.

The versioned 0.1.x manuals are untouched (immutable per release);
benchmark_instruction's Linux sentence is about test hosting, not this
route. Link check passes for both files at their real base paths.

Production LOC: core 97282 -> 97282 (delta 0), sdk 3453 -> 3453
(delta 0); combined 109918 -> 109918. Docs only. reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
…udited identity

Backing::delete refused Unsupported off unix, so the direct route's
private overlay files on Windows were never reclaimed by their last
owner. The Windows arm now applies the same discipline through the
audited identity: the entry at the path and the retained open handle
must both still be the acquisition's file (volume serial + file index,
sole link) before std::fs::remove_file; the retained handle - opened
with std's FILE_SHARE_DELETE - completes the removal when Drop closes
it, exactly the Linux order (check, remove, close). A swapped-in file
at the path is refused and stays. windows_path_identity is exported
from layerfs-overlay beside windows_handle_identity; the device/inode
fields' dead_code allowance widens to both reading platforms.

Tests (new, overlay_backing_windows.rs, dangling-Weak pool so Drop
skips the callback): a matching overlay file is deleted at last-owner
release (bounded wait for the pending removal to complete on handle
close), and a file swapped in behind the backing's back is refused and
stays.

Verification: Windows - the new tests 2/2, store-service 5/0 (was 3),
persistence 87/0, project 10/0, overlay 74/0; SDK matrix/lifecycle pass
on the rebuilt addon. Linux container - cargo check --locked for
store-service and overlay, Finished.

Production LOC: core 97282 -> 97316 (delta +34; store-service +31,
overlay +3), sdk 3453 -> 3453 (delta 0); combined 109918 -> 109952.
Counter: tools/production_loc.py (pinned at this tree). reference
absent (retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Windows overlay last-owner deletion landed (2a4cf6645, head): the recorded limitation "last-owner overlay deletion keeps its Linux identity check" is now lifted — Backing::delete gains a Windows arm with the same discipline through the audited identity (path entry AND retained handle must both still be the acquisition's file — volume serial + file index, sole link — before remove_file; std's FILE_SHARE_DELETE share mode completes the removal when Drop closes the last handle, the Linux order). A swapped-in impostor file is refused and stays.

Tests: new overlay_backing_windows.rs (dangling-Weak pool, so Drop skips the callback — the deletion mechanics are the unit): matching file deleted at last-owner release; swapped-in file refused. Windows store-service 5/0 (was 3), full suites green, SDK tests pass on the rebuilt addon, Linux container cargo check --locked Finished.

With this, the DIRECT_STORE §6 limitation list shrinks to the inherent items (non-UTF-8 names, synthesized mode, subst-drive aliasing, logical overlay allocation).

…ation

§6 said last-owner overlay deletion keeps its Linux identity check and
is not exercised on Windows; commit 2a4cf66 lifted exactly that. The
paragraph now states the parity (audited identity on both sides of the
comparison) and rewords the aliasing limitation to what 3b actually
implements (resolved-containment; subst/unresolvable aliases stay
unrecognized).

Production LOC: core 97316 -> 97316 (delta 0), sdk 3453 -> 3453
(delta 0). Docs only. reference absent (retired R9c; last counted
65,417 at tree 498dd19).
The Windows overlay reported its logical length as allocated bytes
(compression/sparse never visible; the receipt pair: a written overlay
whose logical length is 77,824 holds 131,072 on disk - the stand-in
understated by 40%). The audited Windows identity module gains
windows_allocation_size (GetFileInformationByHandleEx /
FileStandardInfo on the retained handle, compression/sparse aware) and
Allocation::new/state read it: high_water seeds from the real footprint
at open, and state reports it beside the logical length, parity with
Linux's blocks*512.

Test (new allocation_windows.rs): a written overlay's allocated_bytes
is positive, at or above its logical length within a cluster margin,
observed pair printed for the receipt.

Verification: Windows - overlay 75/0 (74 + the new test), persistence
87/0, project 10/0, store-service 5/0. Linux container - cargo check
--offline --locked for overlay and store-service, Finished.

Production LOC: core 97316 -> 97336 (delta +20; overlay +20), sdk 3453
-> 3453 (delta 0); combined 109952 -> 109972. Counter:
tools/production_loc.py (pinned at this tree). reference absent
(retired R9c; last counted 65,417 at tree 498dd19).
Review r2 (2026-10-30) caught the drift: the mechanism list still named
GetVolumeInformationW on the drive root, while the implementation (K3-R3
F4) opens the Store directory's handle, refuses a final path on a share
and reads the filesystem with GetVolumeInformationByHandleW.

Docs-only change.

Production LOC: 104120 -> 104120 (delta 0) — tools/production_loc.py;
documentation does not contribute. Reference absent (retired R9c; last
counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Review r2 (2026-10-30) answered — head now a8fbe8d8b:

  • Windows 实测/CI 确认(评审的两个合入前置):fork dispatch run 41066129042 全四作业绿,含 windows store suites——真实 Windows runner 上跑通 Must-fix perf(fs): optimize bytesToHex encoding #3(windows_lane_abandoned,Drop/abandoned-mutex)与 Keep mini-benchmark JSON artifacts Prettier-clean #4(dangling-junction Init、junction alias、WTF-8 e2e)。评审所看 tip 96066bd7 之后还有一个它未见的修复 d81cc6cc1(connect 时 lane 等待补记 wait.writer——由同一次 CI 实跑抓出的遥测缺口,Linux 容器三元定位后修复,fast lane 恢复绿)。
  • 文档漂移已修(a8fbe8d8b):§6 机制清单的 GetVolumeInformationW 句改为终路径 + GetVolumeInformationByHandleW 描述。
  • PathName/surrogate 策略与 Local\ 会话域:维持待属主拍板——前者是内容语法的长期产品决策(当前实现=规范 WTF-8、拒绝拆对拼写、跨平台同名字节一致),后者已记录为 §6 已知限制。

…t custody) into windows-native-store

Main gained nine commits (Ephemeral-AI-Lab#490 workspace Branch naming, Ephemeral-AI-Lab#491 dirty-check
streaming, Ephemeral-AI-Lab#493 plugin pins, Ephemeral-AI-Lab#496/Ephemeral-AI-Lab#500 stalled-unmount custody): the one
touchpoint with this branch's crates is operator/inspect.rs, which the
branch does not modify, and git merge-tree reported a clean merge before
this merge was made. The branch stays ahead-only in its own surface.

First-parent production comparison: parent (a8fbe8d) 104120 -> merge
tree 104188 (delta +68; upstream's own additions through this merge,
measured with tools/production_loc.py in a detached parent worktree;
core+engine product src). Reference absent (retired R9c; last counted
65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Synced with main (head a6a489db0): merged upstream's nine new commits (#490 workspace Branch naming, #491 dirty-check streaming, #493 plugin pins, #496/#500 stalled-unmount custody) — the only touchpoint with this branch's crates is operator/inspect.rs, unmodified here; merge-tree was clean before merging. First-parent production LOC 104120 -> 104188 (delta +68, upstream's additions). The fork's dispatched CI run on the merged tree is all four jobs green (fmt and guards / clippy / test / windows store suites, run 41071592317).

@NelsonLongxiang

Copy link
Copy Markdown
Author

Review r3 (2026-10-31) received — verdict unchanged (建议合入, no new findings); its one factual observation (fork main stalled at 582bc786) is resolved: the fork's main is fast-forwarded to 35d36575c, so the branch now reads ~28 ahead / 0 behind against a current fork main. The three r3 post-merge decisions (PathName/surrogate grammar, Local\ acceptance, merge route) remain with the owner; the post-merge verification script for ruling #3 condition ② has been rehearsed end-to-end on this tip (addon build + three SDK chains green in a fresh worktree).

@NelsonLongxiang

Copy link
Copy Markdown
Author

五角度终审完成(对我的实现裁定的独立复审,2026-10-31):A2 并发安全=支持(Local/无限等待符合属主书面模型;Windows seal() lane 缺口与既有 panic-retention 为 P2 跟进);A4 证据链=核心链路逐位核实成立,三处需补收据(Windows clippy、Windows 侧 sdk_timing、彩排表述更正——最终版脚本确实未端到端执行过,机制是手动验证的);A5 合并风险=需附带条件(与上游零交集、schema 未动;两条非代码条件 + is_wtf8 回滚尾部)。裁定性分歧在 A1+A3:WTF-8 语法放宽方向正确(严格超集、字节一致、排序保持),但 (i) 下游未完成——merge 引擎 String 路径层与 SDK fs.list 对 WTF-8 名坍缩为空串(P1×2,Linux 亦可达);(ii) 超出属主授权——属主 2026-10-25 记录的是 lossy 限制,实现提交改写了属主记录句而追认机制只覆盖属主签发的变更(A3 裁定:需显式追认或回退);(iii) core/AGENTS.md 有 ~154 行与 engine/AGENTS.md 重复(上游 10-27 重构的合并残留)。待属主单一决策:追认语法(随后补齐下游)或回退为严格 UTF-8+边界拒绝(本 PR 保持原授权范围尽快合);两者之外的全部内容均通过五角度审查。

…d, receipts closed

Owner instruction 2026-10-31 ("可以合并") ratifies the branch for merge as
it stands, including the WTF-8 name grammar. This commit completes the
merge-preparation items the five-angle review named:

- **Ratification recorded**: root AGENTS.md and DIRECT_STORE §6 mark the
  canonical WTF-8 grammar as owner-ratified 2026-10-31, superseding the
  earlier lossy-names limitation sentence (the supersession A3 required).
- **The decision surfaces are fenced** (A1's P1s): a merge or compare that
  must build a String path over a name outside UTF-8 now refuses
  `InvalidRecord` instead of deciding on a collapsed empty path — the
  naming walker (compare/named.rs) is the single seam, so merge places,
  hint naming and compare answers are all covered; the spill codecs'
  strict reads become unreachable for such names. SDK listings render the
  names empty (recorded in §6); the byte-carrying merge/compare carriage
  is named follow-up work.
- **seal() takes the writer lane on Windows too** (A2-F1): the close-time
  checkpoint now runs under the lane on both flock and named-mutex
  platforms, per doc 83's teardown rule.
- **core/AGENTS.md is restored to upstream shape** (A3): the ~154-line
  block the merges duplicated from engine/AGENTS.md is dropped, and the
  Audited Windows FFI list lives in engine/AGENTS.md beside the macOS
  seal section, with the layerfs-node and store-service entries.
- **Receipts closed** (A4): Windows clippy now exists — pinned 1.85.1,
  five crates lib+tests plus layerfs-node --no-deps, `-D warnings` clean
  (the fixed items: two more duplicated outer allows, a dead `_` arm in
  the codec, the discovery wait's vestigial `loop` shell — one bounded
  attempt, every exit final — unused imports/returns, and cfg gates on
  unix-only test helpers). sdk_timing_coverage on Windows is recorded as
  structurally impossible today: layerfs-api's test graph pulls
  layerfs-daemon, which does not compile on Windows (ungated os::unix) —
  out of this branch's scope, noted rather than faked.

Receipts: Windows content 484/0, persistence 87/0, project 17/0,
store-service 10/0, node check Finished; Linux container six-crate check
Finished; Windows clippy clean as above; fmt clean.

Production LOC: 104188 -> 104187 (delta -1) — tools/production_loc.py,
core+engine product src, parent measured in a detached worktree.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
The clippy cleanup dropped the import Windows no longer used, but
scratch() — cfg-gated to macos/linux — returns PathBuf there; the Linux
clippy and test lanes died on E0412. The import is now gated to match.
Verified: Windows clippy clean, Linux container tests-check Finished.

Production LOC: 104187 -> 104187 (delta 0) — tools/production_loc.py;
tests do not contribute. Reference absent (retired R9c; last counted
65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

生产可用性评估(六角度实跑,2026-10-31,真实 Windows 宿主)——总判定:引擎内部使用有条件可用;外部生产交付未就绪。

角度 判定 关键实测
P1 功能 有条件可用 全方法面端到端字节精确(CJK/NFD/emoji 无损、merge+publish、三进程跨开、subst 别名识破、UNC 拒绝)
P2 性能 有条件可用 init 线性(10k 文件 2.4s)、提交中位 ~10ms、内存/Store 有界、4 写者并发过(诊断级单样)
P3 可靠性 有条件可用 写中强杀→重开全有或全无恢复;损坏 Store 13–21ms 类型化拒绝;只读卷类型化;断电如实声明不在 Disposable 承诺内
P4 安全 有条件可用 owner-only DACL 实测无继承泄漏;秘密清扫零命中;但 Store 文件重开无 ACL 复查(文档-实现落差)、符号链接被跟随、同会话抢占实测无限等待
P5 环境 有条件可用 中文路径/第二 NTFS 卷/≤248 长路径过;>260 长路径确定性失败(FFI 平文路径+node 无清单);Dev Drive=ReFS 按设计拒绝
P6 交付 未就绪 addon 无受支持获取途径(build 脚本拒 win32、散文指令不精确、CI 不构建);主 SDK 文档仍写 Linux-only+WSL2;Windows 拒绝码不在 SDK CODES 表

外部生产阻塞清单(P6 三项 + P5 一项):① Windows addon 构建/分发途径;② api/docs 与 runtime 的路由文档一致性;③ 错误码面(Unauthorized/Unsupported/NoStore 等入 CODES + DACL 锁死恢复文档 takeown/icacls);④ 审计 FFI 的 verbatim ?\ 长路径化。
共享主机阻塞级(P4):open_direct 对 Store 文件补 require_owner_only 复查 + symlink_metadata 拒链接(两处小改)。
应做:headroom 预准入 Windows 对等、WTF-8 名 rm/cp 的类型化、InvalidRecord 等文档码与实现对齐、慢盘/10 万级树探针、被杀进程 overlay 临时目录清理。
评估者分歧如实记录:P4 建议写者 lane 有界等待;早前 A2 按属主锁模型裁定无限等待即设计(与 Linux flock 同构)——保留模型、把『同会话抢占=无限停摆』量级补记 §6,是否改有界留属主。

探针产物:tmp/win-native/prod-readiness/(六套脚本+收据)。结论:合并本身不受阻(引擎四维有条件可用、格式零改动);上述清单是合并后的 fast-follow 工作面。

@Mola-maker

Copy link
Copy Markdown
Member

建议自己实机跑一遍

@Mola-maker

Copy link
Copy Markdown
Member

thanks for pull requests!
but目前正在重新优化内核和编写各个sdk,包括win点的,所以这个pr在2.0版本之前不能合并

…en hardening, verbatim paths, a real build arm, the documented error surface

The 2026-10-31 production-readiness round (six probe-running assessors on
this Windows host) named two shared-host blockers and four external
delivery blockers; this commit resolves all six:

- **open_direct answers for the installed Store file** (P4 findings 2a/
  3a, live-verified before/after): a link planted at the Store's spelling
  is refused ("is a link, not the Store file"), never followed, and a
  widened DACL on the file is refused with the owner-only message — the
  discovery tree's check never covered the installed file. The file is
  the privacy boundary; the directory keeps its creation-time
  restriction only, so a caller-pre-created directory is accepted.
- **Paths travel verbatim** (P5's blocker): the audited Win32 calls
  (volume guard, identity probe, ACL set/read) spell \?\-prefixed
  paths, so trees deeper than MAX_PATH initialize — verified live with
  a 287-char source (was a clean InitFailure, now a full cycle with the
  Chinese filename round-tripping). Forward slashes normalize to
  backslashes under the prefix (a verbatim path is not normalized and
  node may hand either spelling — caught by the probe's first run).
  The §6 note keeps the host-process longPathAware caveat honest.
- **The addon has a supported build path on Windows** (P6 blocker 1):
  api/scripts/build-native.mjs grows a win32 arm — cargo +1.85.1
  --release --locked, layerfs_node.dll staged to the addon path — run
  end to end here (release, 9.7 MB). The script's docs and §6 name it.
- **The error surface is complete** (P6 blocker 3): the SDK's frozen
  CODES list grows the codes the Windows route actually emits
  (Unauthorized, Unsupported, Failed, NoStore, UnknownMethod), and
  fs.rm/fs.cp over a subtree holding a name outside UTF-8 refuse with
  a message naming that entry instead of a raw utf-8 error (P1 gap 1).
- **The user docs state the route** (P6 blocker 2): api/docs/agent.md's
  where-calls-run, LAYERFS_STORE_PATH and install sections, and
  api/docs/runtime.md's Windows bullet describe the native route, its
  NTFS requirement, its default-when-no-worker position and its
  coexistence with the WSL2 worker; §6 records the DACL lockout
  recovery (takeown + icacls), the headroom parity gap, the
  in-session-squat magnitude, the overlay-temp note and the doc-code
  alignment (the merge fence surfaces as Failed with the cause named).

Receipts: >260-char source tree full cycle PASS (was FAIL); widened-DACL
and link-substitution refusals live-verified; Windows suites 484/87/17/
80/10 all green; SDK matrix/lifecycle/concurrent green on the rebuilt
addon; Windows clippy (pinned 1.85.1) clean; Linux container six-crate
check Finished.

Production LOC: 104187 -> 104226 (delta +39) — tools/production_loc.py,
core+engine product src, parent measured in a detached worktree.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

thanks for pull requests! but目前正在重新优化内核和编写各个sdk,包括win点的,所以这个pr在2.0版本之前不能合并

OK,我这边会进行生产验证,有实际成果再另行通知,同时也一直期待2.0能够带来更加极致的性能,Thanks!

The fix round's closures exceeded rustfmt's width; Linux CI's fmt check
caught the drift. Formatting only.

Production LOC: 104226 -> 104226 (delta 0) — tools/production_loc.py;
formatting does not change counted lines. Reference absent (retired R9c;
last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Production-readiness blockers resolved (heads f50172e48+f5622cd04, CI run 41171956548 all four jobs green) — all six findings from the six-angle round are fixed and live-verified on this Windows host:

  1. Store-open hardening (P4 2a/3a): a link at the Store's spelling refuses "is a link, not the Store file"; a widened DACL on the file refuses with the owner-only message — both verified live (was: silently opened). The file is the privacy boundary; pre-created directories stay caller-shaped.
  2. Verbatim paths (P5 blocker): the audited Win32 calls spell \\?\ paths (forward slashes normalized — a verbatim path is not, and node hands either); the 287-char source tree that cleanly failed Init now runs a full cycle with the Chinese filename round-tripping. §6 keeps the host-process longPathAware caveat.
  3. A supported Windows build arm (P6-1): build-native.mjs builds win32 (cargo +1.85.1 --release --locked → layerfs_node.dll → the addon path), run end to end here; docs name it.
  4. Error surface (P6-3 + P1-1): the SDK's frozen CODES grows Unauthorized/Unsupported/Failed/NoStore/UnknownMethod; fs.rm/fs.cp over a non-UTF-8 name refuse naming that entry.
  5. User docs state the route (P6-2): agent.md (where-calls-run, LAYERFS_STORE_PATH, install) and runtime.md describe the native route, its NTFS requirement, default-when-no-worker, and WSL2 coexistence; §6 adds the DACL lockout recovery (takeown + icacls), the headroom parity gap, the squat magnitude, the overlay-temp note and the fence's SDK code alignment.

Receipts: >260 probe PASS (was FAIL); both tamper refusals live; Windows suites 484/87/17/80/10; SDK matrix/lifecycle/concurrent on the rebuilt addon; Windows clippy clean; Linux container check Finished; fork CI four-green on f5622cd04. LOC 104187→104226 (+39, worktree-measured). Remaining should-level items (slow-disk/p95 campaign, headroom Windows arm, addon version handshake, persistent diagnostics) stay listed in the assessment thread for follow-up.

…rbatim, absolute storePath

The second verification round (five adversarial/production assessors on
head f5622cd) ruled the six fixes HELD (R1), no regressions (R2),
soak-passed (R3), realistic-load-passed (R4), delivery-ready-internal
(R5) — and named three residual defects, fixed here:

- **UNC separator** (R1 P2): the verbatim converter dropped the `\`
  after `UNC` (`\server\share` became `\?\UNCserver\share`, a
  different server). The separator is now part of the spelling, a
  forward-slash UNC (`//server/share`) takes the same branch (the unit
  test caught that the old condition matched backslashes only), and
  tests/verbatim_wide_windows.rs pins all four forms.
- **SQLite's own open stays verbatim** (R1 P2): the audited calls were
  verbatim but Connection::open received the plain spelling, so a
  >MAX_PATH Store installed yet refused every open ("Uncertain").
  Session::connect now hands SQLite the verbatim OsString — verified
  end to end: a 210+-unit store path initializes, opens and reads back
  exact through the plain caller spelling (was the Uncertain refusal).
- **storePath must be absolute** (R1 P3): a relative spelling silently
  resolved against the process CWD; the route's entry now refuses
  InvalidInput naming the requirement, symmetric with LAYERFS_HOME.
- Docs from R5: api/README.md's Windows build paragraph no longer
  claims the script refuses win32 or points at a nonexistent manifest
  (it names the supported command and the clean-machine preconditions);
  §6's lockout recipe states the elevated console and the Store-file
  step; agent.md's LAYERFS_HOME row documents the parent-must-exist
  and DACL semantics.

Receipts: verbatim tests 4/4 (incl. both UNC spellings); persistence
91/0; SDK matrix/lifecycle/concurrent green on the rebuilt addon;
Windows clippy clean (persistence lib+tests; node --no-deps —
sandbox's unix-only imports stay out of scope); Linux container check
Finished; the >MAX_PATH store cycle verified live.

Production LOC: 104218 -> 104239 (delta +21) — tools/production_loc.py,
core+engine product src, parent measured in a detached worktree.
Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Second verification round complete (five adversarial/production assessors on f5622cd04; their findings fixed in 34ea1b95c, CI all four green — the one red attempt was a runner-side python3 cold-start transient, green on rerun):

Assessor Verdict
R1 adversarial security 修复成立 — six attacks (TOCTOU race 6,431 attempts, hardlink, junction, verbatim edge matrix, widened sidecars, long-path ACL) produced no end-to-end bypass; two P2 defects found inside the fix surface, both fixed below
R2 regression + long-path 无回归 — every round-1 behavior reproduces; tamper refusals 10/10 no false positives; long-path envelope mapped (sources any depth, home/store cliffs are SQLite-side, all refusals typed)
R3 soak 生产耐久通过 — 18 segments/9.1 min, 6 mid-batch kills all recovered next-open, 0 refusals, WAL bounded, memory flat, 6/6 in-flight commits present and byte-exact
R4 realistic load 真实负载可用 — 8,440-file/78.8MB repo-like tree: init 2.6s, 300/300 multi-process writes exact, merges byte-exact; only legitimate HeadMoved under continuous-write merges
R5 delivery re-audit 内部交付就绪 — all three round-1 blockers verified fixed live (build arm end-to-end, docs walk, error codes with live triggers); recovery recipe dry-run restores a refused home

Fixed in 34ea1b95c: R1-P2-① the UNC verbatim conversion dropped the \ after UNC (\\server mis-named to server "UNCserver"; also //server now takes the UNC branch — the new unit test caught that); R1-P2-② SQLite's own open now receives the verbatim spelling, so a >MAX_PATH Store is not just installable but openable (verified live end-to-end, was "Uncertain"); R1-P3 relative storePath refused InvalidInput (was silently CWD-resolved); R5's doc gaps (README's wrong build paragraph, §6 lockout recipe needs an elevated console and the Store-file step, LAYERFS_HOME parent-must-exist documented).

P3-level observations recorded, not blocking: the ACL check matches broad SIDs rather than owner identity (planted narrow-DACL files pass the privacy check but fail pack binding); the check→open TOCTOU window is real but narrow (EBUSY pins the file while open; 0/6,431 race hits); -wal/-shm sidecars carry the token-default DACL. New unit tests: verbatim_wide_windows.rs (4 forms). Receipts: persistence 91/0, SDK chains green on the rebuilt addon, Windows clippy clean, Linux container Finished, CI four-green on 34ea1b95c.

…, relative storePath fence, residue removed

Round 3 on 34ea1b9 (A1 fix-verification, A2 concurrency, A3 cross-platform
bytes, A4 perf envelope, A5 delivery clean-room): every P2 fixed and
probe-verified; P3s fixed or recorded.

- A2-F1 (P1-Uncertain) dissolved by adjudication: a canary-calibrated,
  name-agnostic enumeration of the session's named-object directory caught
  the lane mutex live during writes (openHits 89/465, heldHits 78) under
  the verbatim-form digest; the round's probe had derived the name without
  the \?\ form. The lane holds per write transaction as designed.
- A2-F4: the writer lane moves Local\ -> Global\ so a session-0 scheduled
  task and the user session share one lane (probe: Global openHits
  592/1455, heldHits 580; the Local spelling now 0).
  tests/windows_lane_abandoned.rs derives the new name.
- A1-F1: open_direct tightens SQLite's -wal/-shm sidecars to owner-only
  whenever an open sees them; connect creates them under the first open,
  so they are restricted from creation, and a crash's leftovers are
  re-tightened by the next open (probe: crashed -wal already OWNER
  RIGHTS-only against the pre-fix Authenticated-Users inheritance,
  reopen recovers the write byte-exact, clean close deletes the sidecars).
- A1-F2: project.init refuses a relative storePath up front (InvalidInput),
  matching direct.rs; no directory or project entry is left (probe: no
  relvol/, no project entry).
- A1-F5 recorded, not changed: layerfs-overlay's two Win32 path probes keep
  raw wide spellings — the product-source boundary keeps that crate free of
  first-party dependencies (a dependency attempt failed the boundary guard
  and was reverted); TEMP-local inputs fail cleanly beyond MAX_PATH. The
  exception is documented in DIRECT_STORE §6 and in the module.
- A1-F3/F6: verbatim_wide's doc states passthrough is byte-for-byte (no
  slash normalization for already-verbatim input) and softens the
  longPathAware claim to hosts without the long-path relaxation.
- A5-F1: remove committed probe residue api/agentharden-run/ (with its
  208 KB store.sqlite) and unreferenced api/agent/harden-child.mjs;
  .gitignore guards the run directory.
- A5-F3/F5/F7, A2-F2, A3-F2: documentation — the audited-FFI list pointer
  now names engine/AGENTS.md, inprocess.js's header no longer says
  "Linux only", README states the Node >= 20 floor, §6 records the
  Global\ lane, sidecar tightening, the TEMP-overlay accumulation
  trade-off and the SDK's lone-surrogate write behaviour (U+FFFD sibling;
  the byte-carrying path remains the recorded follow-up).

LOC correction for f5622cd (A5-F2): its tree recomputes to 104218
(rustfmt -8); the message's "104226 -> 104226 (delta 0)" overstated the
after figure.

Production LOC: 104239 -> 104255 (delta +16) — pinned
tools/production_loc.py on the first parent (34ea1b9, detached worktree)
and the staged tree; project.rs +10 (the storePath fence) and
store_service.rs +6 (the sidecar loop); comments-only files contribute 0.
Scope: core/engine crate src plus api src; tests/docs/manifests excluded.
@NelsonLongxiang

Copy link
Copy Markdown
Author

Third verification round complete (five fresh angles on 34ea1b95c; their findings fixed/recorded in a7f5cda78, all chains green):

Angle Verdict
A1 fix-verification + residual path surface 修复成立,新增 2×P2 已修 — the three r2 fixes re-verified independently (unit + black-box); a 23-call-site path-spelling map with a 6/6 deep-path e2e matrix; split-brain lane unconstructible (all spellings canonicalize to one digest; 3-spelling concurrent writers 60/60 exact)
A2 concurrency & writer lane 行为全过,P1 疑点经裁决消解 — 6×33 multi-process writes 198/198 exact, kill-holder recovery 132/132 published bytes exact, 2,825 commits no starvation, lifecycle/#436 clean
A3 cross-platform byte fidelity 通过(两方向 14/14 逐字节一致) — one Store file written on Windows read on Linux, and written on Linux read on Windows: CJK / emoji pairs / combining / lone-surrogate 3-byte forms all byte-exact; read-only open leaves the file's sha256 unchanged
A4 production perf envelope 无异常回归 — init 874 ms/4k files (throughput above the r2 reference), small-write p50 8.0 ms, 5 MiB ≈80 MB/s, random-read p50 0.68 ms, 3×20 multi-process exact, merge 6.8 ms, WAL deleted on close
A5 delivery clean-room 有缺口(无功能阻断) — docs/scripts/error-surface walk ✓ against HEAD; one P2 committed-residue and several P3 doc gaps

A2's P1 "the writer lane was never observed held" dissolved under adjudication. A canary-calibrated, name-agnostic enumeration of the session's named-object directory caught the lane mutex live during writes — Mutant:layerfs-writer-lane-1c515513f79bf042, exactly the verbatim-form (\\?\-canonicalized) digest — openHits 89/465 passes with heldHits 78 while 449 concurrent writes ran. The round's probe had derived the name without the \\?\ form and watched a nonexistent object. The lane holds per write transaction exactly as designed; A2's behavioral results (zero Busy, zero loss) stand with the mechanism now attributed.

Fixed in a7f5cda78 (probe evidence in tmp/win-native/round3/, receipts in each angle's report):

  • A2-F4 (P2) — the writer lane moves Local\ → Global\: a session-0 scheduled task and the user session now share one lane (probe after the switch: Global openHits 592/1455, heldHits 580; the Local spelling 0). windows_lane_abandoned test derives the new name.
  • A1-F1 (P2) — open_direct tightens SQLite's -wal/-shm sidecars to owner-only whenever an open sees them. Since connect creates them under the first open, they are restricted from creation; a crash's leftovers are re-tightened by the next open. Probe: crashed -wal already OWNER RIGHTS:(F) (pre-fix: Authenticated Users:(M) inherited), reopen recovers the write byte-exact, clean close deletes the sidecars.
  • A1-F2 (P2) — project.init refuses a relative storePath up front (InvalidInput), matching the r2 direct.rs fence; no directory or project entry is left (probed).
  • A5-F1 (P2) — removed committed probe residue api/agentharden-run/ (208 KB store.sqlite included) and unreferenced api/agent/harden-child.mjs; .gitignore guards the run directory.
  • P3s — doc pointers to the audited-FFI list corrected to engine/AGENTS.md; inprocess.js header no longer "Linux only"; README states the Node ≥ 20 floor; verbatim_wide doc now matches byte-for-byte passthrough (A1-F3) and softens the longPathAware claim (A1-F6); §6 records the Global\ lane, sidecar tightening, the %TEMP% overlay-directory accumulation trade-off (A2-F2) and the SDK's lone-surrogate write behaviour (A3-F2: a JS write by such a name lands as an U+FFFD sibling; byte-carrying path stays the recorded follow-up).
  • A1-F5 recorded, not changed — layerfs-overlay's two Win32 path probes keep raw wide spellings: importing the shared helper failed the product-source boundary guard (overlay is a first-party-dep-free leaf crate) and was reverted; inputs are TEMP-local and fail cleanly beyond MAX_PATH; documented as the exception.

Evidence correction for the r2 round (A3-F1): the "lone surrogate" names constructed in r2 via Node/PowerShell were silently U+FFFD (legal UTF-8); only CreateFileW-level construction makes the real 3-byte form. r2's capability claim stands — re-proven this round with true lone surrogates (both directions byte-exact, utf8:false on exactly those rows). LOC correction (A5-F2): f5622cd04's tree recomputes to 104218 (rustfmt −8), not the message's delta 0; the correction is recorded in a7f5cda78's message.

Receipts on a7f5cda78: verbatim 4/4, lane-abandoned 1/1 (Global name), windows_store_native 1/1, acl 2/2, kernel_guard 1/1; pinned clippy clean on the touched crates (examples' warnings pre-exist); boundary guard PASS (964 files); SDK fast chain 39/39 on the rebuilt addon; Linux container --offline --locked Finished; Production LOC 104239 → 104255 (+16, project.rs +10 / store_service.rs +6). Remaining recorded, not blocking: perf envelope is diagnostic single-sample; CI adds one report-only windows job (A5-F8); merge/compare byte-carrying path, headroom pre-admission parity, addon version handshake and the formal p95 baseline stay on the should-level list.

…ost's policy, not one manifest

Round-3 A1 measured plain 308-character paths working on a host with the
OS-level relaxation; the sentence claimed node.exe universally lacks the
effect. Aligned with the module comment's round-3 A1-F6 softening and
B4's meta-audit finding (B4-F5).

Production LOC: 104255 -> 104255 (delta 0) — pinned tools/production_loc.py;
documentation-only change, production scope unchanged.
@NelsonLongxiang

Copy link
Copy Markdown
Author

Fourth verification round complete (five fresh angles on a7f5cda78; docs delta in 2ff821486; includes a correction to round 3's narrative):

Angle Verdict
B1 Global-lane adversarial 通过 — holder-side causality proven: holding the derived Global\ name blocked a write for exactly the 800 ms hold window (+13 ms after release); a squatter's write lands nothing and the kill of the holder releases the lane; abandoned handoff accepted under Global\; case-variant names are different objects (prefix is an exact match marker)
B2 sidecar semantics 通过 — 4-process concurrent first-open 4/4 serialized through the lane, 32/32 bytes exact, all DACL samples owner-only; re-tightening a sidecar another writer holds open: 4,130 writes zero errors; pre-planted wide-open sidecar tightened at open; WAL-header/frame and SHM tamper all salvage-or-refuse, never corrupt-accept
B3 soak + fault injection 本体达标,一项 P1 裁决见下 — 12 segments/10.9 min, 3 writers + merger, 1,508 writes 0 mismatch; 5 kills (4 confirmed mid-transaction) + 3 double-kills all recovered with every published commit byte-exact; 0 Busy; WAL ≤4 MiB bounded, RSS flat; recovery-sidecar DACL owner-only every time
B4 evidence-trail meta-audit 有断点(已更正) — 27 audited items reproduced exactly (tests, LOC both snapshots, CI, both fix probes re-run on HEAD); but round 3's adjudication narrative was refuted by retained evidence (below)
B5 overhead regression 无可感回归 — the new per-open cost is ≈0.2 ms (2× SetNamedSecurityInfo ≈104 µs steady-state; Global\ vs Local\ mutex differ <1 µs), <1% of every end-to-end face; init median 915 ms vs 874 (within noise of concurrent-agent load, first-samples retained not cherry-picked)

Correction to round 3 (B4-F1, integrity): round 3's comment said the lane probe "had derived the name without the \\?\ form". The meta-audit refuted that: A2's scripts did derive the verbatim form, and recomputing the digest confirms the watched name was correct. A2's pre-fix zero-observation anomaly therefore remains unexplained on the instrument side — I withdraw the causal claim, and also acknowledge its pre-fix numbers (89/465·78) were posted without retained receipts (single log files overwritten by the post-fix run; B4-F2). What stands for the product, now with retained receipts on both builds: round-3 post-fix enumeration 592/1455·held 580, round-4 adjunct2 242/1737·held 224 (tmp/win-native/round4/adjunct2/{writer-run2,prober-run2}.log), plus B1's holder-side causality on the same build — three independent positives; and B3's round-4 negative (its own sampler: 50 samples × 20 ms per round, lineage of A2's) contradicts B1's positive hold proof, so it is classified as an instrument false-negative with the mechanism not fully localized. All probe receipts are now retained per-round.

Other outcomes: the fence's "no directory left" wording was imprecise — discovery::home() creates the (empty) home root before the refusal; no project entry, no Store directory (B4-F3). api/agent/test/packaging.mjs fails on Windows for a pre-existing upstream import() path bug (B4-F4, not this branch's regression). Standard-user Global\ creation remains untested (B4-F6). The §6 long-path sentence now names the real condition — the host's long-path policy, not one manifest (A1's 308-char plain-path measurement; 2ff821486). B1's P3: a squatted write keeps the Node process alive until the outer timeout (INFINITE readiness wait, no cancel path — the §6-recorded tradeoff).

…ly from creation, NUL storePath fences

Round 5 (C1 Linux final-diff read, C2 merge rehearsal, C3 red team, C4
input-space edges, C5 share-conflict/noise battery) on 2ff8214; all
five verdicts landed (C1/C3 conditional, both closed here or recorded):

- C1-F1 (P2): core/docs/architecture/83-store-concurrency-model.md still
  described the Windows writer lane as Local\ per-session — stale since
  a7f5cda; now names Global\ (one lane across login sessions) and
  notes the discovery mutexes stay Local\.
- C1-F3 (P3): project.init created the Store's parent with create_dir
  then chmod 0700 on unix — a umask traversable window against its own
  comment; now DirBuilder::mode(0o700), owner-only from the first
  instant, as the install path does.
- C3-F4 (P3): storePath NUL fences in direct.rs and project.rs. Honest
  scope: the observed truncation happens at the OS env boundary (a
  NUL-terminated env block) BEFORE any product code, and the SDK is
  env-only — today the fences are unreachable defense-in-depth for any
  future non-env argument path. Probe: a NUL set through env truncates
  at set time; the fences themselves compile and the SDK surface is
  unchanged (39/39).
- C3-F1/F2/F5 and C5-F1 (recorded in DIRECT_STORE §6): a pre-planted
  lane object with a foreign DACL fails the next open (typed Malformed
  today — error-typing improvement on the should-list); lane names are
  enumerable by any local user (a digest revealing activity, not the
  path); a caller-pre-created wide directory lets a foreign user
  pre-place sidecars the per-open tightening cannot re-own; the
  ReadOnly-vs-Unknown typing split across open paths is recorded too.
- C1-F2 (recorded, Linux): refusal paths that create the Store's parent
  first now leave that directory behind on Linux (upstream's guard-first
  order left nothing); behavior accepted with this note.
- C2's rehearsal artifact: post-merge-verify.sh (untracked patrol
  tooling) gained a --no-run prebuild and explicit FAILED on timeout —
  a cold compile alone exceeded its 110 s per-suite ceiling (184 s for
  three crates). Not part of this commit.

Production LOC: 104255 -> 104272 (delta +17) — pinned
tools/production_loc.py on the first parent (2ff8214, detached
worktree) and the staged tree; project.rs and direct.rs fences account
for the delta; docs contribute 0.
@NelsonLongxiang

Copy link
Copy Markdown
Author

Fifth verification round complete (five fresh angles on 2ff821486; fixes and records in c2ca6caa6) — the round's headline is the merge rehearsal: the exact no-ff merge commit the owner will create was built locally and passed every chain:

Angle Verdict
C1 Linux final-diff read 有条件通过→已收口 — full upstream/main...HEAD adversarial read: no Linux contract weakening, no cfg leakage, no lane-hold self-deadlock (container-proven on ext4); 1×P2 stale doc + 1×P3 unix dir-mode window, both fixed in c2ca6caa6
C2 merge rehearsal 通过 — local no-ff merge of 2ff821486 into 35d36575c (upstream/main): merge tree byte-identical to the branch tree (same tree oid); four chains + addon + SDK 39/39 + the frozen verify spec re-run on a merge-tree Store (run_58e79383230559ca PASS 5/0) + container check all green; one tooling fix (below)
C3 red team 零穿透 — no confidentiality/integrity break, no P0/P1; one new P2 availability primitive fully characterized; NUL-truncation P3 re-adjudicated (below)
C4 input-space edges 通过 — malformed-store battery all typed refusals 1–31 ms (truncated/foreign/user_version drift/directory-as-store); 100 MiB single write 1.37 s + byte-exact paged read-back; 101-level nesting; 2000 writes; 200-commit paginated history exact
C5 share conflicts & noise 通过 — every third-party hold shape fails fast and clean (≤201 ms, no hang/panic/half-writes); 41/41 torn-WAL copies recover with integrity_check ok; 45/45 writes exact under CPU+IO noise

Fixed in c2ca6caa6: doc 83's Windows-lane sentence now says Global\ (stale since a7f5cda78, C1-F1); project.init creates the Store's parent owner-only from the first instant on unix (DirBuilder::mode(0o700), no create-then-chmod window, C1-F3); NUL fences on storePath in both direct.rs and project.rs (C3-F4) — with honest scope: the observed truncation happens at the OS env boundary (NUL-terminated env block) before any product code, and the SDK is env-only, so today the fences are unreachable defense-in-depth for future non-env argument paths; the round's probe proves the env layer truncates at set time. §6 records the round's security neighbours (C3-F1: a pre-planted lane object with a foreign DACL fails the next open, typed Malformed today — typing improvement on the should-list; C3-F2: lane names are enumerable by any local user, a digest revealing activity, not the path; C3-F5: foreign-placed sidecars on a caller-pre-created wide directory stay foreign-owned) and C1-F2 (Linux refusal paths now leave the created Store parent behind). The rehearsal also fixed the patrol post-merge-verify.sh (untracked tooling): chain 4 now prebuilds --no-run — a cold compile alone measured 184 s against the script's 110 s per-suite ceiling — and reports explicit FAILED instead of a misleading "0 passed".

Verify freeze, round 5: a second frozen spec for a crash-recovered Store (30 writes, hard exit, reopen+recover, clean close) — spec_5e773f5fa69a → run_c2892720f8786183 PASS 5/5, archived arc_f71cb6a6ab0e; recovery and fresh paths produce the identical account. A recursive verify tree over both leaves was prepared and run: verdict STALE, correctly fail-closed — tree contract v1.2 carries a single target identity, and a two-distinct-snapshot aggregate cannot be blessed as PASS under it; the leaf runs and archives remain the frozen evidence (the tool refusing a shape it cannot honestly verify is the discipline working, not a failure).

Receipts: Windows + container checks on layerfs-node green after the fixes, clippy clean, SDK 39/39 on the rebuilt addon, Production LOC 104255 → 104272 (+17, the two fences), fork CI on c2ca6caa6 green (the windows-store-suites job finished success; one earlier run's in-flight status was superseded). Merge key remains with the owner: Create a merge commit — the rehearsal shows the merge tree identical to the branch tree while main stands at 35d36575c.

…ths fenced against collapsed names

Round 6 (D1 cold-eyes final read, D2 runbook literalism, D3 integrator
journey, D4 long-horizon soak, D5 host-path diversity) on c2ca6ca:

- D1-F1 (P1): windows_volume_filesystem's regrow branch was dead code —
  a Vec could never try_into a fixed [u16; 1024] for a longer
  requirement, so the retry read into a zeroed buffer on which the
  \?\UNC\ refusal silently passed: a share deeper than 1023 UTF-16
  units was admitted as NTFS. The final path now grows for real; the
  failure is captured before later syscalls can overwrite it (D1-F6).
  New tests volume_guard_long_paths_windows: a ~1160-unit local tree
  still passes and the same tree through \localhost\D$\ is refused —
  the refusal is the behavioral proof (it passed before the fix, shown
  end to end with the admin share).
- D1-F2 (P2): the InvalidRecord fence covered name_all only. join() now
  stays the display spelling (listings render a non-UTF-8 name empty,
  the documented contract) and a new join_exact() fences every decision
  path — tree walks, bindings, changes, merge names/places — where a
  collapsed key silently dropped the second change of a directory.
  content --tests 484/0.
- D1-F4 (P3): a missing Store answers Missing on every platform again;
  the Windows entry block used to surface it as Failed.
- D2-F1 (P1 doc): the §6 lockout recovery runbook is rewritten to the
  sequence verified end to end this round — recursive takeown over the
  home and any external volume tree (taking ownership rewrites the
  files' OWNER RIGHTS ace into an inherit-only form), the (OI)(CI)F
  grant on trees, and a plain :F grant on the Store file and
  project.record, which an (OI)(CI) grant gives nothing.
- D1-F3 (P3, recorded): a subst spelling can hold a second lane for one
  directory (canonicalize keeps the subst form); bounded by SQLite's
  own WAL write lock. §6 records it.
- D3-F1 (P2 doc): agent.md's routing paragraph now says the project
  family's Sandbox-related calls (project.list_sandboxes, sandbox.*)
  still need the daemon and refuse Unsupported on this route.
- Tooling (untracked patrol script, not in this commit): post-merge-
  verify.sh gains the missing npm install step, unmasks SDK-chain
  failures the tail pipe hid, fixes the receipt name, prefers the
  release addon, and marks first-execution suite timeouts explicitly.

Recorded, not changed: D1-F7 (require_owner_only's message names three
broad SIDs, the doc's parity wording is broader), D1-F8 (Linux now
imports well-formed WTF-8 names the upstream InvalidUtf8 path refused —
the grammar's intent, noted), D2-F2 (partial-recovery refusals drift
between Unauthorized and Failed), D3's integrator documentation gaps
(publish semantics, error-code table pointer, packaging).
D4: 34.4-minute soak PASS (4,729 ops, WAL bounded, RSS flat, reads
undegraded). D5: CJK/space/bracket/emoji host paths PASS with zero
behavioral difference from the ASCII baseline; store-cjk.sqlite kept
for the verify freeze.

Production LOC: 104272 -> 104281 (delta +9) — pinned
tools/production_loc.py on the first parent (c2ca6ca, detached
worktree) and the staged tree; join_exact and the guard rework account
for the delta.
@NelsonLongxiang

Copy link
Copy Markdown
Author

Sixth verification round complete (five fresh angles on c2ca6caa6; fixes in d9687cfa4). This round found and closed the branch's first P1 since round 2:

Angle Verdict
D1 cold-eyes final read (zero prior context) 1×P1 + 1×P2, both fixed below + a 10-item refuted-suspicions ledger; otherwise "high engineering discipline, residual risk concentrated in branch paths of checking code"
D2 runbook literalism §6 recovery runbook was not followable (P1 doc) — rewritten to the sequence verified end to end; post-merge script defects fixed (untracked tooling); agent.md/README walkthrough 12/12 exact, zero differences
D3 integrator journey 通过(docs-only 走通) — a real consumer app built from docs alone runs green; 13 documentation gaps recorded, one P2 doc fix (sandbox-call routing sentence)
D4 long-horizon soak PASS — 34.4 min end to end, 4,729 ops, WAL bounded (≤4.1 MiB, zero after close), RSS flat, read path undegraded at 156 merges, 3 kills recovered byte-exact
D5 host-path diversity PASS, zero behavioral difference — full chain (incl. two crashes, dual-process same-branch writes) under all-CJK, spaces/brackets/&/' and emoji directory names; error-code sets identical to the ASCII baseline

D1-F1 (P1, fixed in d9687cfa4): the volume guard's buffer regrowth was dead code — a Vec can never try_into a fixed [u16; 1024] for a longer requirement, so the retry read into a zeroed buffer on which the \\?\UNC\ refusal silently passed: a network share deeper than 1023 UTF-16 units was admitted as NTFS, breaking the local-volume-only condition. D1 proved it three ways (unit-level, ctypes, end to end through \\localhost\D$ following the product's exact algorithm). The final path now grows for real, the failure is captured before later syscalls, and new tests pin the behavior: a ~1160-unit local tree passes, the same tree through the admin share is refused — the refusal is the fix's behavioral proof.

D1-F2 (P2, fixed): the InvalidRecord fence covered only name_all. join now stays the display spelling (listings render a non-UTF-8 name empty — the documented contract), while a new join_exact fences every decision path (tree walks, bindings, changes, merge names/places) where a collapsed key silently dropped the second change of a directory. Captured as a decision ADR: "display join renders the empty segment, decisions refuse". content --tests 484/0.

Also in d9687cfa4: a missing Store answers Missing on Windows again (was Failed, D1-F4); §6 records the subst-spelling second-lane note (D1-F3, bounded by SQLite's WAL lock); agent.md names the Sandbox-related calls that still refuse Unsupported (D3-F1). Recorded, not changed: D1-F7/D1-F8 (wording and the Linux WTF-8 import face), D2-F2 (partial-recovery refusal drift), D3's documentation gaps.

Verify freeze grows to three leaves: fresh (spec_8d9a826db739), crash-recovered (spec_5e773f5fa69a) and now the CJK-host-path Store (spec_6b1711a0605a → run_380864b8e741da55 PASS 5/5, archived arc_fb4b88b22096) — the account invariants hold regardless of the host path's encoding.

Receipts on d9687cfa4: content 484/0 + windows trio 6/0 + volume-guard 2/0 + acl 2/0 + kernel_guard 1/0; clippy clean; SDK 39/39 on the rebuilt addon; container --offline --locked --all-targets green for content/store-service; Production LOC 104272 → 104281 (+9). Fork CI: the first attempt's windows store suites job was killed by the workflow's own timeout 120 per-crate ceiling — the content suite's first-execution cold cost on a cache-less fork runner (the same physics D2 measured at 184–228 s in the merge rehearsal; the suite itself runs 2.69 s warm, 484/0 locally, and the kill landed exactly at step-start+120 s). --failed rerun: all four jobs green.

…ct gets its first unit test

Correction (round-7 E1-F3): d9687cf's message claimed "New tests
volume_guard_long_paths_windows" — the file was written but never
staged (git add -u only stages tracked files), so it was absent from
the commit tree while present in the local working tree that produced
the 2/2 receipts. It enters the tree here unchanged and passes 2/2
(deep local tree accepted, the same tree through \localhost\D$
refused).

New compare_join_exact (round-7 E1-F4, E5-F2): the decision-spelling
twin gets its first in-tree assertion — identical to the display join
for UTF-8 names, InvalidRecord for a name outside UTF-8, and the two
distinct unpaired-surrogate names provably collapsing to one display
key (the harm join_exact fences). 2/2.

Recorded from round 7, not changed here:
- E1-F1 (P2, upstream): the Linux FUSE invalidation path keys
  Stale::Entry on the display spelling (commit_merge.rs/invalidate.rs
  are untouched by this branch per diff) — a non-UTF-8 name rebound by
  a merge would not invalidate its kernel dentry; same family as the
  join split, owner should-list.
- E1-F2 (P3): read/walk.rs feeds the display join to the descent and
  glob filters — display-scope drift only, no write routing.
- E3 (P2, owner decisions): the 256-component path limit is enforced
  on the workspace/rename route but not the SDK fs/init routes
  (accepted 257-301 components live); and deep single-chain costs grow
  near-quadratically (d=250 mkdir 1.8 s). Both recorded for the owner;
  neither is a correctness defect.
- E5: the fence's end-to-end behavior proven at Store+merge level —
  planned merges never touch it and lose nothing; the full-walk path
  (diff_branches / >65,536-changed merges) refuses InvalidRecord
  deterministically, Windows and Linux alike, with byte-exact
  preservation of the non-UTF-8 entries.
- Tooling (untracked): post-merge-verify.sh v4 — toolchain pinned
  +1.85.1 with --offline, per-crate same-selection prebuild (a
  cross-crate prebuild leaves fingerprint mismatches), and a tested
  TIMEOUT recovery recipe (a kill mid-link corrupts PDBs: LNK1207).

Production LOC: 104281 -> 104281 (delta 0) — pinned
tools/production_loc.py; test-only change, production scope unchanged
(parent d9687cf recomputed at the same total).
@NelsonLongxiang

Copy link
Copy Markdown
Author

Seventh verification round complete (fix-adversarial verification, merge rehearsal v2, fence-at-scale, SDK test-surface map, fence end-to-end proof; one process correction in 2dd58881f):

Angle Verdict
E1 fix adversarial verification 三处修复全部经攻击成立 — boundary matrix 1021–1100 units (local accept, admin-share refuse at every length), Missing-order 7-case matrix, join_exact coverage audit: all 6 decision sites fenced, write paths never key on the display spelling
E2 merge rehearsal v2 通过 — merge tree byte-identical to the branch tree (same tree oid); the v3 post-merge script executed end to end in variant form: five crates 682 passed / 0 failed (matching D2's warm total exactly), frozen verify spec PASS on the merge-tree Store, container green
E3 fence at scale 无可感开销,规模正确 — 8,000-file/4-deep tree: 3×1,500 writes zero refusals, two 3,000-change merges 337/203 ms, all read-backs byte-exact; fence A/B arms differ ≤±11% (noise; the multi-byte arm is never slower)
E4 SDK test-surface map 零分支回归,净改善 — 26 scripts cataloged: 17 executable, 12/14 whole-script PASS, failures all upstream-existing or platform-scoped; the branch fixes trace ×9 vs main and adds three passing store chains; real-Store scenarios 27/28 both legs (the one is the documented list_sandboxes Unsupported)
E5 fence end-to-end 通过 — planned merges with two unpaired-surrogate names in one directory succeed with 5/5 entries preserved (they never enter a String-path decision); the full-walk path (diff_branches, >65,536-changed merges) refuses InvalidRecord deterministically, byte-exact preservation, Windows and Linux identical; clean 2×2 attribution

Process correction (2dd58881f): round-6's message claimed "new tests volume_guard_long_paths_windows" — the file was written but never staged (git add -u stages tracked files only), so it was absent from d9687cfa4's tree while present in the local working tree that produced the 2/2 receipts (E1-F3). It enters the tree unchanged, passes 2/2, and this CI run is the first where it ran from the tree. 2dd58881f also adds compare_join_exact — the fence's first in-tree unit test (identical to display for UTF-8, InvalidRecord outside it, and two distinct surrogate names provably collapsing to one display key — the fenced harm). CI four-green on 2dd58881f. Frozen verify family re-run: all three specs PASS 5/0.

Recorded for the owner (not this branch's regressions): E1-F1 (P2, upstream — the Linux FUSE invalidation keys Stale::Entry on the display spelling; commit_merge.rs/invalidate.rs untouched by this branch per diff, so a non-UTF-8 name rebound by a merge would miss its kernel dentry invalidation); E3 (P2 decisions — the 256-component path limit is enforced on the workspace/rename route but not the SDK fs/init routes, live-accepted at 257–301 components; deep single-chain costs grow near-quadratically, d=250 mkdir 1.8 s); E1-F2 (P3 — walk.rs feeds the display join to descent/glob filters, display-scope drift only). Tooling: post-merge-verify.sh v4 — toolchain pinned +1.85.1/--offline, per-crate same-selection prebuild (E2 proved a cross-crate prebuild still recompiles past the ceiling), and a tested TIMEOUT recovery recipe (a kill mid-link corrupts PDBs — LNK1207 — recover with clean -p + re-prebuild).

Merge key remains with the owner: Create a merge commit. The rehearsal now covers d9687cfa4; the tree identity argument holds for 2dd58881f identically (tests-only delta, verified byte-identical merge tree by construction).

…ace closed, six cross-document contradictions aligned

Round 8 (F1 incremental cold read, F2 in-process concurrency, F3
cross-document consistency matrix, F4 flake hunt, F5 merge-day package)
on 2dd5888:

- F1-F1 (P2): the volume-guard test's discriminating half silently
  self-skipped wherever the temporary directory is not on D: — including
  this machine's default C: TEMP, so the round-7 "2/2 refused" receipt
  never executed that half. The share spelling is now derived from the
  temporary directory's own drive letter (verified 2/2 in the default
  C:-TEMP environment through \localhost\C$). F1-F3: the deep()
  helper's verbatim claim was false (PathBuf::join drops the prefix for
  an absolute component); the prefix is now built textually, so creation
  no longer depends on the host's long-path relaxation, and removal uses
  the verbatim spelling.
- F1-F2 (P3, fixed): after the regrow retry, a requirement that grew
  between the two calls (a concurrent rename) left the buffer unfilled
  and the UNC check reading garbage — the round-6 P1 harm could revive
  under that race. A second short read now refuses Unsupported with its
  own message.
- F3's six P2 contradictions aligned: the "fs, branch, project and
  operator calls in this process" family now carries the sandbox-family
  exception in README, DIRECT_STORE, and the root AGENTS (agent.md's
  sentence gains the platform qualifier — on Linux those calls are
  served in-process); §6's "the Store file and its directory answer the
  same check at every open" now matches the code (the file at every
  open; the directory when the route creates it); README's opening route
  sentence no longer predates the native default; agent.md documents
  NoStore (a storePath naming no installed file) and the merge/rm/cp
  SDK surfacing of the non-UTF-8 fence (Failed with the cause, not the
  content-layer name); §6 carries the three upstream-family records the
  PR body pointed at (FUSE invalidation display spelling, the
  256-component limit's uneven enforcement, near-quadratic deep chains).
  Also: obtaining-the-addon gains --offline and the Node >= 20 floor;
  the "should-list" dangling reference became a concrete note;
  inprocess.js's header no longer claims four default threads (WORKERS
  is 64) and mentions the worker branch.
- F5-1 (P2, tooling): post-merge-verify.sh now checks upstream/main
  first (the fork's main never moves by itself), and notes the
  expectation as >= 682 passed with 0 failed on this tree.
- F4: 23 repeated executions, zero flakes; the r6 CI transient is
  quantified locally as cold-build cost (clean 340 s + cold build 80 s
  against the 110 s ceiling). F2: in-process concurrency passes (480/480
  exact, zero Busy; same-directory dual Stores share the lane without
  starvation); the read-during-write latency characteristic is recorded
  for integrators.

Production LOC: 104281 -> 104292 (delta +11) — pinned
tools/production_loc.py on the first parent (2dd5888, detached
worktree) and the staged tree; the guard's race refusal accounts for
the delta.
@NelsonLongxiang

Copy link
Copy Markdown
Author

Eighth verification round complete (incremental cold read, in-process concurrency, cross-document consistency, flake hunt, merge-day package; fixes in 1443753b4):

Angle Verdict
F1 incremental cold read (the two fix commits) 修复核心成立,但抓到测试的两个真问题(P2: the discriminating share half silently self-skipped off D:; P3: the regrow retry trusted its second answer — both fixed in 1443753b4)+ a 12-item refuted-suspicions ledger
F2 in-process concurrency PASS — 6 clients × 40 writes in one process: 480/480 exact, 0 Busy/HeadMoved/Conflict; open/close churn ×50 leak-free; two Stores in one directory share the lane without starvation (170/170 exact each); pool width 4/8/64 equivalent
F3 cross-document matrix (27 facts × 10 carriers) 6×P2 contradictions, all aligned in 1443753b4 — the "fs/branch/project/operator in this process" family vs the sandbox exception (four carriers), §6's directory-check sentence vs the code, README's pre-native opening sentence, NoStore never documented, InvalidRecord vs the SDK's Failed, and the PR body pointing at §6 records that weren't there
F4 flake hunt 23 executions, zero flakes — every suite ×3 green; the r6 CI transient reproduced and quantified locally: cold cost = clean 340 s + cold build 80 s against the 110 s ceiling (a budget separation, not instability)
F5 merge-day package 17/17 artifacts verified — script v4 syntax/pinning/prebuild in place, three frozen specs re-run PASS 5/0 today, all referenced run/archive ids resolve, MERGE-DAY.md assembled; one P2 broken link fixed (the script checked the fork's main, which never moves — now prefers upstream/main)

Fixed in 1443753b4: the volume-guard test now derives the share spelling from the temporary directory's own drive letter — verified 2/2 in the default C:-TEMP environment through \\localhost\C$, the first receipt where the discriminating refusal half provably executed (round-7's "2/2 refused" ran the skip branch); the deep() helper builds its verbatim prefix textually (PathBuf::join drops it for absolute components), so the test no longer depends on the host's long-path relaxation; the regrow retry refuses Unsupported when the requirement outgrew the regrown buffer (the concurrent-rename race that could revive the round-6 P1 harm); and the six documentation contradictions above plus their P3 tails (--offline, Node ≥ 20, the worker-branch comment, the four-threads claim → the actual default of 64).

Recorded: F2's read-during-write latency characteristic (reads stay correct but queue behind long writes on the shared pool — integrators mixing latency-sensitive reads should know); F4's cold-cost quantification (the CI lane's per-crate 120 s wraps build-first policy already; the transient stays possible on cold cache-miss runners); F1-F4 (dangling-link spellings answer Missing/NoStore, still never followed); F5-4 (the WinFsp contract text awaits the owner — the only un-assembled piece of adjudication #3).

Merge key remains with the owner: Create a merge commit. MERGE-DAY.md (in the round-8 receipts) walks the whole sequence: pre-press checks, the press, the post-merge script (now checking upstream/main first), the frozen-spec family re-runs, and the 24 h / 72 h checkpoints.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants