Skip to content

feat: self-upgrading MCP supervisor, permanent daemon residency, and a repo health pass - #588

Merged
githubrobbi merged 12 commits into
mainfrom
mcp-supervisor
Aug 12, 2026
Merged

feat: self-upgrading MCP supervisor, permanent daemon residency, and a repo health pass#588
githubrobbi merged 12 commits into
mainfrom
mcp-supervisor

Conversation

@githubrobbi

Copy link
Copy Markdown
Collaborator

Twelve commits in three themes. Every commit was validated locally before push (strict clippy on the pinned and floating-nightly toolchains, cross-target clippy for x86_64-pc-windows-msvc, all 2424 tests, and the full lint-pre-push gate including cargo-vet and the audit-discipline check).

Closes #584 and #587 via commit trailers.

1 — Zero-downtime MCP binary swaps

uffsmcp stdio mode is now fronted by a supervisor that owns the client's pipes for the whole session and serves through a hidden --worker child it can replace. When the binary on disk is swapped (an installer, just use-local), a ticker notices the changed (len, mtime) identity, waits for a quiet moment with nothing in flight, spawns a fresh worker, replays the captured initialize/initialized handshake (swallowing the duplicate response), reroutes traffic, kills the predecessor, and nudges the client with notifications/tools/list_changed.

The AI host's MCP connection never drops — a new binary can be swapped in without restarting any downstream session. The same replay path restarts a crashed worker, rate-limited to 3 crashes/60s. It is deliberately a byte pump, not a re-framing proxy: each JSON-RPC line is parsed only enough to classify it, and the original bytes are always forwarded. Plain std::process + threads, so no async runtime in the supervisor process and identical behavior on Windows.

An end-to-end test drives a real session across a simulated binary replacement over real pipes and asserts two different worker PIDs on the same connection plus the list_changed nudge.

Overrides: UFFS_MCP_WORKER_EXE, UFFS_MCP_WATCH_PATH, UFFS_MCP_POLL_MS.

2 — Permanent residency (uffs --daemon resident)

The tiering ladder's final rung (idle auto-retire) removed uffsd from the process list, so the next search paid a full restart. resident on registers a per-user login item — Windows HKCU Run key, macOS launchd LaunchAgent with KeepAlive.SuccessfulExit=false, Linux systemd user unit with Restart=on-failure — starting uffsd --no-retire, and starts the daemon immediately when none is running.

Windows deliberately uses the Run key rather than a scheduled task or service: it needs no Administrator, so the non-elevated resident daemon keeps reading the MFT through the Access Broker and the zero-UAC story is preserved end to end. The elevation gate exempts resident explicitly, since it only touches per-user artifacts.

Residency also survives crashes and manual stops on every platform via an auto-spawn marker (resident.args, beside the PID file). Every implicit daemon spawn merges it in at the single spawn_daemon choke point, so a revived daemon keeps --no-retire. Merging is conservative and unit-tested: caller flags always win, marker flag groups append only when absent, and ephemeral --ephemeral-id instances pass through untouched so a snapshot daemon can never inherit --no-retire and leak.

3 — Repo health pass

Nightly canary (#584). Two independent regressions on the floating nightly. min_ident_chars briefly extended to macro-generated single-letter constants and tripped on the DriveLetter A–Z constants — which are the domain model; that is now declared posture via allowed-idents-below-min-chars so the same churn cannot break the canary again, without weakening the lint elsewhere. It did catch one genuine offender: BloomFilter::k()hash_count(). Separately, the new pedantic assert_is_empty lint: all 36 assert!(x.is_empty()) sites across 20 files became typed assert_eq!/assert_ne! so failures print the offending value — a test-quality upgrade, no suppressions. A redundant_field_names firing inside thiserror 2.0.19's generated code was fixed at the dependency level (2.0.20), per the ethnum precedent.

Dependency refresh. clap 4.6.6, futures 0.3.34, globset 0.4.20, rmcp 3.1.2, devicons 0.6.13, plus the darling 0.24 train. Every non-trusted delta got a real source-diff review recorded as a cargo-vet delta audit with Vet-Reviewed-Diff trailers (15 new audits; exemptions 238 → 234).

Toolchain pin 2026-07-23 → 2026-08-11, taken via just toolchain-sync, which gates on host clippy plus Windows (cargo-xwin) and Linux (cargo-zigbuild) cross-compiles before writing the pin. The canary's green floating-nightly result is exactly the precondition that made this safe.

sccache shim (#587). .cargo/config.toml named sccache directly as build.rustc-wrapper, making the repo unbuildable anywhere sccache is absent — which is why Dependabot's cargo updater has never opened a PR here (four red runs a month) and why fresh clones break before cargo install sccache. The wrapper now points at a shim that forwards to sccache when present and execs rustc directly when not. Verified with the real toolchain cargo, a PATH carrying rustc but not sccache, and a fresh target dir: before → could not execute process 'sccache rustc -vV' (exit 101); after → clean build and a full workspace cargo metadata resolve. With sccache present, sccache --show-stats records the compile requests, so caching is genuinely preserved. Bug B (dev-flow.md §5.2) is unaffected — both cache keys still live together in one atomic config; only the wrapper's value moved behind the shim.

Advisory bookkeeping. Corrected the standing assumption that a polars bump would clear the two quick-xml advisories: object_store 0.14.1 does now require the fixed quick-xml ^0.41, but polars-io still pins object_store ^0.13.1 even in polars 0.55.2, so the blocker is entirely upstream. Reachability remains nil (cloud-store XML LIST parsing; UFFS never touches a cloud object path).

…-swap

uffsmcp stdio mode becomes a tiny supervisor that owns the client's
pipes for the whole session and serves through a '--worker' child it
can replace. A ticker stats the on-disk binary; when a rename-replace
install changes its (len, mtime) identity and nothing is in flight,
the supervisor spawns a fresh child, replays the captured initialize/
initialized handshake (swallowing the duplicate response), atomically
reroutes traffic, kills the old child, and nudges the client with
notifications/tools/list_changed. The client's MCP connection never
drops: a new binary can be swapped in place without restarting any
downstream host session.

The same replay path restarts a crashed worker (rate-limited to 3
crashes per 60s window before the session ends honestly), so the
supervisor doubles as a crash hatch.

Deliberately a byte pump, not a re-framing proxy: each JSON-RPC line
is parsed only enough to classify it (request/response id, the two
handshake messages) and the ORIGINAL bytes are always forwarded.
Plain std::process + threads — no async runtime in the supervisor
process, identical behavior on Windows.

Overrides: UFFS_MCP_WORKER_EXE (worker program), UFFS_MCP_WATCH_PATH
(swap trigger file), UFFS_MCP_POLL_MS (ticker interval, default 5000).
Spawns the real uffsmcp supervisor over real pipes and drives one MCP
session across a simulated binary replacement (the watch file is
touched). The worker is a scripted JSON-RPC responder (the real worker
needs a live daemon) that stamps every reply with its own PID, making
the swap directly observable: the same connection sees two different
worker PIDs, and the notifications/tools/list_changed nudge arrives.

Unix-gated: the fake worker is a sh script; the supervisor logic under
test is platform-neutral (plain std::process + threads).
User-manual section for the zero-downtime upgrade path: what the
supervisor does on a binary replacement, the list_changed nudge, the
crash hatch, and the three environment overrides.
The test module is unix-gated (the fake worker is a sh script), so on
x86_64-pc-windows-msvc the file-level lint expectations were
unfulfilled and serde_json went unused — both rejected by the
cross-target lint lane. Gate the expectations with cfg_attr(unix) and
acknowledge serde_json on non-unix.
…autostart

The tiering ladder's final rung (idle auto-retire) removes uffsd from
the process list, making the next search pay the full restart cost.
'resident on' flips that trade: it registers a per-user login item
that starts uffsd with --no-retire — never exits on idle, while the
memory tiers still park unused drives down to a few MB — and starts
the daemon immediately when none is running. 'off' removes the item;
'status' (default) reports both halves.

Per platform:
- Windows: HKCU Run value — no Administrator at any point; the
  non-elevated resident daemon reads the MFT through the Access
  Broker, preserving the zero-UAC story end to end.
- macOS: launchd LaunchAgent via 'launchctl bootstrap' with
  KeepAlive.SuccessfulExit=false — a crash relaunches, a clean
  'uffs --daemon stop' sticks; ThrottleInterval tames pathological
  loops.
- Linux: systemd user unit, Restart=on-failure.

The login item bakes an absolute --log-file (login-started processes
have no useful cwd), pulling the shared path from uffs-security's
log_dir — new direct dependency for the CLI. A running daemon is
never touched: residency takes over at the next start, and the
command says so. The elevation gate exempts Resident explicitly: it
manages per-user artifacts only.
User-manual section under Idle Retirement: what 'resident on/off/
status' does per platform, the zero-UAC Windows story, and why
residency is nearly free (tiering still parks unused drives).
…resident

'resident on' now records the resident daemon argv in resident.args
(sibling of daemon.pid); every implicit daemon spawn — the next
search, an MCP tool call — merges it in at the single spawn_daemon
choke point, so a crashed or manually stopped resident daemon comes
back with --no-retire instead of silently falling back to the 2-hour
idle-retire lifetime. This closes the Windows gap where the Run key
cannot relaunch a crashed daemon mid-session; on macOS/Linux it
additionally covers daemons started outside launchd/systemd.

Merging is conservative and fully unit-tested (new daemon_resident
module, split out to respect the 800-LOC ceiling on daemon_spawn):
the caller's flags always win (an explicit --data-dir suppresses the
marker's), marker flag groups append only when absent, and ephemeral
job-scoped instances (--ephemeral-id) pass through untouched — a
snapshot daemon must never inherit --no-retire and leak forever.

'resident off' deletes the marker; 'resident status' reports the
auto-spawn lifetime alongside the login item.
… + posture

Fixes #584 (nightly-canary upcoming-toolchain regression).

The floating-nightly canary tripped on two fronts:

1. min_ident_chars briefly extended to macro-generated single-letter
   constants (2026-08-09 nightly), breaking on the DriveLetter A-Z
   constants — which ARE the domain model. Posture decision, encoded
   in clippy.toml: single UPPERCASE identifiers are explicitly allowed
   via allowed-idents-below-min-chars ('..' keeps the default
   lowercase list), so canary stays green across upstream churn
   without weakening the lint for ordinary identifiers. The one real
   offender it did catch is renamed for clarity: BloomFilter::k() →
   hash_count().

2. The new pedantic assert_is_empty lint: every
   assert!(x.is_empty()) / assert!(!x.is_empty()) in the workspace
   (36 sites across 20 files) is rewritten to assert_eq!/assert_ne!
   with a typed empty Vec / "" so the offending value prints on
   failure — a genuine test-quality upgrade, not a suppression.

Also bumps thiserror 2.0.19 → 2.0.20: the new nightly's
redundant_field_names fired inside thiserror 2.0.19's generated From
impls; 2.0.20 (released 2026-08-08) emits shorthand initialization.
cargo vet: covered by the existing dtolnay trust entry.

Verified: cargo +nightly clippy --workspace --all-targets
--all-features --locked --no-deps -- -D warnings (the exact canary
command) is fully clean on nightly 2026-08-10; pinned-toolchain
clippy and all 2424 tests pass unchanged.
…y vetted

Direct-dep refresh (all semver-patch or -minor):
  clap 4.6.5 -> 4.6.6 (+ clap_builder; epage trust entry)
  futures family 0.3.33 -> 0.3.34 (9 crates, delta-audited)
  globset 0.4.19 -> 0.4.20 (+ regex-automata 0.4.18; BurntSushi trust)
  rmcp / rmcp-macros 3.1.0 -> 3.1.2 (delta-audited: SSE stack-overflow
    fix, WWW-Authenticate auth-refresh mapping, RFC 8414 issuer-
    comparison fix, MRTR extractors, SEP-2549 cache hints)
  devicons 0.6.12 -> 0.6.13 (delta-audited: dotted-directory icon fix)
  transitives: darling 0.23 -> 0.24 train (delta-audited: syn-3
    migration, serde-style const-block codegen; no token injection)

Every non-trusted delta got a real source-diff review recorded in
supply-chain/audits.toml (15 new delta audits; exemptions 238 -> 234).
cargo vet clean; cargo audit unchanged (the two known quick-xml
advisories remain polars 0.54.4 transitives, gated on a polars
release carrying the nightly-API patches).

reqwest stays intentionally pinned at 0.12.28 — re-verified earlier:
0.13.x still ships no rustls-tls-native-roots feature.

All 2424 tests pass; pinned and floating-nightly clippy both clean.

Vet-Reviewed-Diff: futures-core@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-io@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-sink@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-task@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-macro@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-channel@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-executor@0.3.33->0.3.34
Vet-Reviewed-Diff: futures-util@0.3.33->0.3.34
Vet-Reviewed-Diff: futures@0.3.33->0.3.34
Vet-Reviewed-Diff: darling@0.23.0->0.24.0
Vet-Reviewed-Diff: darling_core@0.23.0->0.24.0
Vet-Reviewed-Diff: darling_macro@0.23.0->0.24.0
Vet-Reviewed-Diff: rmcp@3.1.0->3.1.2
Vet-Reviewed-Diff: rmcp-macros@3.1.0->3.1.2
Vet-Reviewed-Diff: devicons@0.6.12->0.6.13
Re-verified the upstream state behind the two ignored quick-xml
advisories (RUSTSEC-2026-0194/0195). Half the recorded removal
condition is now met, and the other half was mis-attributed:

  * object_store 0.14.1 DOES require quick-xml ^0.41.0 (the fixed
    line). The note claiming 0.14.0 still needed ^0.40.1 was true on
    2026-07-02 and is now superseded.
  * polars-io still pins object_store ^0.13.1 — including in the
    newest polars 0.55.2 — so bumping polars off 0.54.4 would NOT
    clear these advisories, contrary to the standing assumption that
    a polars bump was the fix. The blocker is entirely upstream:
    polars-io must adopt object_store 0.14.

Reachability is unchanged and still nil: the vulnerable code is
object_store's cloud-store XML LIST parsing, and UFFS never touches a
cloud object path. cargo deny check advisories: ok.
Bumped via `just toolchain-sync`, which gates a candidate on the host
clippy run plus the Windows (cargo-xwin) and Linux (cargo-zigbuild)
cross-compile phases before writing the pin — so the shipped targets
are validated, not just the dev host. Exit 0 on the first candidate.

The nightly-canary early-warning signal (#584) is what made this safe
to take: the workspace is clippy-clean on the floating nightly as of
the preceding commit, which is exactly the precondition the canary
exists to establish before a pin bump.

Re-verified on the new pin: strict clippy across the workspace
(--all-targets --all-features) clean, all 2424 tests pass, rustfmt
clean.

Also stops the header comment from restating the pin date in prose —
it claimed nightly-2026-07-06 while the channel key had already moved
to 2026-07-23. The channel key is the single source of truth.
Fixes #587.

`.cargo/config.toml` named `sccache` directly as `build.rustc-wrapper`,
which made the repo unbuildable anywhere sccache is absent — every
cargo invocation died before doing any work:

    error: could not execute process `sccache rustc -vV` (never executed)
    Caused by: No such file or directory (os error 2)

That is why Dependabot's cargo updater has never opened a PR here (its
container has no sccache): four red update runs a month, and every Rust
dependency bump left to a human. It is also the wall a fresh clone hits
before `cargo install sccache`, which has bitten us interactively before.

The wrapper now points at a shim that forwards to sccache when it is
installed and execs rustc directly when it is not. Caching is unchanged
where sccache exists; where it doesn't, the build simply runs uncached
instead of failing.

Verified end to end (real cargo binary, PATH carrying rustc but not
sccache, fresh target dir):
  * before — `could not execute process 'sccache rustc -vV'`, exit 101
  * after  — `cargo check` succeeds and a full workspace `cargo metadata`
             resolves (exit 0), which is what Dependabot needs
  * sccache still present — 7 compile requests recorded by
    `sccache --show-stats`, so caching is genuinely preserved
  * exit codes propagate (a bad rustc flag still fails the build)
Workspace clippy clean and all 2424 tests pass.

Bug B (dev-flow.md §5.2) is unaffected: `rustc-wrapper` and
`incremental = false` still live together in `.cargo/config.toml` as one
atomic config; only the wrapper's value moved behind the shim.

The config path resolves relative to the workspace root — verified
empirically; a `../`-prefixed form does not resolve. A Windows `.cmd`
twin ships alongside for developers who invoke cargo directly rather
than through `just` (which exports RUSTC_WRAPPER="" on every platform
anyway); point RUSTC_WRAPPER at it there.
@githubrobbi
githubrobbi enabled auto-merge August 12, 2026 00:54
@githubrobbi githubrobbi changed the title feat(mcp,daemon): self-upgrading MCP supervisor, permanent residency, and repo health pass feat: self-upgrading MCP supervisor, permanent daemon residency, and repo health pass Aug 12, 2026
@githubrobbi githubrobbi changed the title feat: self-upgrading MCP supervisor, permanent daemon residency, and repo health pass feat: self-upgrading MCP supervisor, permanent daemon residency, and a repo health pass Aug 12, 2026
@githubrobbi
githubrobbi added this pull request to the merge queue Aug 12, 2026
Merged via the queue into main with commit b0e2fe1 Aug 12, 2026
22 of 25 checks passed
@githubrobbi
githubrobbi deleted the mcp-supervisor branch August 12, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐤 Nightly Canary: upcoming-toolchain regression — 5ff597a

1 participant