Skip to content

Hold sink exports behind a bounded pick-pending marker on fresh enroll (LLP 0093)#287

Merged
philcunliffe merged 3 commits into
masterfrom
feat/pick-pending-export-hold
Jul 8, 2026
Merged

Hold sink exports behind a bounded pick-pending marker on fresh enroll (LLP 0093)#287
philcunliffe merged 3 commits into
masterfrom
feat/pick-pending-export-hold

Conversation

@philcunliffe

@philcunliffe philcunliffe commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Stacked on #286 (fresh-enroll registry refresh) - the diff here is only the export hold. When #286 merges, GitHub retargets this to master.

Problem

The issue #281 reordering (PR #283) left a one-time forwarding window it explicitly deferred (the "Revisited-by #281" notes in LLP 0069/0080): on a fresh auto-daemon enroll, the daemon attaches, backfills, and can start exporting backfilled rows before the user finishes the local-only pick. Withholding is enforced at export-read time and the server has no retraction path, so rows from a directory the user was about to withhold forward once, permanently. That narrowed LLP 0069 R6's "recorded here, never forwarded - not even once" on exactly the path the picker exists for.

Decision (new LLP 0093)

The enrolling login writes a machine-local pick-pending marker before enrollCentralSink, and the kernel sink driver skips whole export ticks while that marker is fresh.

  • Marker: usage-policy/pick-pending.json under HYP_HOME state, co-located with the local-only list it guards. Content is debug metadata; freshness is mtime, so a torn write can neither wedge nor extend the hold.
  • Login-owned lifecycle: written on the fresh-enroll fork only (a re-login edits a list whose daemon already forwards), before enrollCentralSink so no daemon tick can beat it onto disk; cleared in one finally covering every exit (pick landed, cancelled, non-TTY skip, enroll error). Write and clear are best-effort - neither can fail the login (LLP 0072).
  • Bounded, never a kill switch: 10-minute TTL (comfortably past the login's two 30s waits plus a human reading the picker); stale markers are treated as absent and unlinked; unreadable markers fail open - deliberately the opposite polarity of the local-only list (which fails loudly), because the marker is a timing hint, not the privacy signal, and failing closed would turn a crashed login into an unbounded export outage.
  • Driver-wide enforcement: the driver cannot know which sinks leave the machine without a new registration concept, and briefly deferring a local sink is harmless where one missed forward hold is a permanent leak. Held rows stay in the cache; watermarks make the post-pick tick a plain catch-up.

Rejected alternatives (recorded in LLP 0093): backfill-before-forward reordering (does not close the window - the pick needs a human, and forwarding would resume the moment backfill finished if the user walked away), off-machine-only gating, unbounded hold.

Tests

  • Marker module: fresh/clear/stale-unlink/injected-now/fail-open.
  • Driver: a fresh marker holds the whole tick (held: 'pick_pending', no exportBatch call); a cleared or expired marker exports normally.
  • Login lifecycle: marker lands before enroll, persists through the pick, and is cleared on return, on enroll error, and on non-TTY; never written on the re-login fork.

npm test: 2140 pass. build:types clean. Smokes: core_boot_noop, daemon_foreground_start_stop, local_only_export_withhold all ok.

🤖 Generated with Claude Code

philcunliffe and others added 2 commits July 8, 2026 13:09
#281 follow-up)

After the #283 reordering the picker ran at the right time but queried
through the wrong registry: hyp remote login boots its kernel at dispatch
time, before enrollment, and on a first-run box the local config names no
plugins. The login process's query-registry snapshot therefore never
registers ai_gateway_messages - that registration arrives with
@hypaware/ai-gateway, which the org config-control pull enables only after
enrollCentralSink installs the daemon. listCapturedDirectories then failed
with "unknown dataset" (best-effort null), waitForCapturedDirectories read
null as "enumeration cannot run, stop now" (no 30s poll), and the picker
took its enumeration_failed durable-hint path on exactly the fresh enroll
it exists for. The #283 tests stubbed waitForCaptured/listCandidates, so
the gap was never exercised; a re-login on a populated box worked, which is
why manual verification passed.

Fix: freshenCaptureEnumeration (src/core/cli/remote_commands.js). On the
fresh-enroll interactive fork, after waitForClientAttach returns a client,
re-boot one fresh kernel (config profile, the same layered resolution any
subsequent hyp command runs) and hand its registry's enumeration to the
capture wait. One re-boot is sufficient by construction: attach markers are
written by the reconciler that runs only after a confirmed config apply, so
attach means the pulled central layer is on disk - and the attach handler
requires the gateway capability, so that layer enables the plugin. The
snapshot-already-has-it case (re-login on a populated box) skips the boot
entirely; a failed re-boot or a still-missing dataset falls back to the
prior behavior. Best-effort throughout: the refinement never breaks the
login it refines (LLP 0072). The handle is disposed (sources.stopAll,
mirroring dispatch) after the picker so no boot-started source keeps the
login process alive.

local_only.js exports CAPTURE_DATASET so the registry check and the
enumeration SQL cannot drift. LLP 0069/0080 carry forward-ref notes per
the immutable-record convention.

Tests: seam-level coverage of the refresh wiring (fresh enumeration is
polled, disposed after the picker; null refresh falls back), unit coverage
of freshenCaptureEnumeration (snapshot no-op, success, still-missing,
boot-throw), and - the #283 lesson - one deliberately unstubbed test that
runs the REAL bootKernel against an on-disk central layer naming
@hypaware/ai-gateway and asserts enumeration RUNS (empty list, pollable)
instead of failing to null.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…l (LLP 0093)

Closes the one-time forwarding window the issue #281 reordering left open:
after PR #283 deferred the local-only picker past enrollCentralSink, the
daemon that enroll installs could attach, backfill, and start exporting
rows before the user finished the pick. Withholding is enforced at
export-read time, and the server has no retraction path, so rows from a
directory the user was about to withhold forwarded once, permanently -
narrowing LLP 0069 R6's "not forwarded, even once" on exactly the
fresh-enroll path the picker exists for.

Decision (LLP 0093): the enrolling login writes a machine-local
pick-pending marker (usage-policy/pick-pending.json, co-located with the
local-only list) BEFORE enrollCentralSink, so no daemon tick can beat it
onto disk; the kernel sink driver skips whole export ticks while the
marker is fresh (held: 'pick_pending' in the tick report); the login
clears the marker in one finally covering every exit from the
fresh-enroll fork. Freshness is mtime with a 10-minute TTL and stale or
unreadable markers read as absent, so a crashed or abandoned login can
never stall exports indefinitely - the hold is a bounded refinement,
never a kill switch (LLP 0072). Held rows stay in the cache and export on
the first tick after the pick; watermarks make that a plain catch-up.

Enforcement is driver-wide rather than per-sink: the driver cannot know
which sinks leave the machine without a new registration concept, and
briefly deferring a local sink is harmless where one missed forward hold
is a permanent leak. Rejected alternatives (backfill-before-forward
reordering, off-machine-only gating, unbounded hold) are recorded in the
decision doc. Re-logins and --no-forward/query-only logins never write
the marker.

Tests: marker module (fresh/clear/stale-unlink/injected-now/fail-open),
driver hold (fresh marker holds the whole tick, cleared or expired marker
exports), and login lifecycle (marker lands before enroll, persists
through the pick, cleared on return, on enroll error, and on non-TTY;
never written on re-login).

Stacked on the issue #281 follow-up fix (fresh-enroll registry refresh).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@philcunliffe philcunliffe added the neutral:adopt Foreign PR adopted into neutral's reconcile scope label Jul 8, 2026
@philcunliffe

Copy link
Copy Markdown
Contributor Author

neutral review — adopted PR (full-heal, stacked on #286), head f628806

Verdict: clean. No blockers, majors, or minors. Reviewed the incremental delta on top of fix/picker-fresh-enroll-registry. Mergeable ∧ green (typecheck + test SUCCESS) ∧ reviewed.

Egress / withholding invariant (no leak found): driver.tick is the sole production forwarding chokepoint. Both callers — src/core/daemon/runtime.js:478 (daemon loop) and src/core/commands/sink.js:41 (hyp sink) — enter through it, and the marker check is the first thing after the tick counter (driver.js:58), returning { sinks: [], held: 'pick_pending' } before any listHandles/exportBatch. No sink (including the central forward sink) can export while the marker is fresh. The columns-projection/query read path is a separate seam already covered by LLP 0080 read-time withholding; this hold correctly targets the forwarding-tick race that seam can't cover on a fresh enroll. Marker path aligned: login writes ${HYP_HOME}/hypaware, daemon driver reads the same file.

Bounded (no kill switch needed): TTL 10 min via file mtime (pick_pending.js:96); stale marker reads absent + is opportunistically unlinked; stat failure fails open. An abandoned/crashed login can never wedge exports past the TTL — correctly the opposite polarity to the local-only list (which fails closed).

Ordering / lifecycle / idempotency: Marker written BEFORE enroll/enrollCentralSink (remote_commands.js:728), so no daemon tick beats it to disk. finally clears it on every exit from the fresh-enroll fork (pick landed, cancelled, non-TTY skip, enroll-throw, connectedElsewhere), gated by holdingExports so only a landed marker is cleared. Write and clear are best-effort (degrade to pre-hold behavior / TTL backstop; never fail login). Re-login/re-seed writes no marker.

Test quality (proves withhold-while-pending + release-after-pick): test/core/pick-pending.test.js:96 drives the real createSinkDriver: fresh marker → held==='pick_pending', 0 exports; clear marker → next tick held===undefined, 1 export — a true fail-before/pass-after in one test. Plus TTL-stale, injected-now freshness, and fail-open-on-unreadable coverage. remote-login-command.test.js proves marker present at enroll time, cleared on return/throw/non-TTY, never written on re-login. 72/72 tests pass.

@ref: driver.js:57, pick_pending.js:22/89 cite LLP 0093 / #bounded (anchor exists). LLP 0069/0080 edits are forward-ref notes only. No [inferred] claim survives in Accepted 0093.

@philcunliffe philcunliffe added the neutral:approved neutral reviewed this adopted PR and approves; held for maintainer merge (LLP 0025) label Jul 8, 2026
@philcunliffe

Copy link
Copy Markdown
Contributor Author

neutral verdict: approved ✅ — mergeable ∧ green ∧ reviewed at head f628806. Stacked on #286 — merge #286 first, then #287. Held for you to merge.

Base automatically changed from fix/picker-fresh-enroll-registry to master July 8, 2026 20:39
# Conflicts:
#	llp/0069-local-only-dir-selection.spec.md
#	llp/0080-local-only-dir-selection.design.md
#	src/core/cli/remote_commands.js
#	test/core/remote-login-command.test.js
@philcunliffe

Copy link
Copy Markdown
Contributor Author

neutral review (round 2) — adopted PR, head 2fc31aa

Verdict: clean. Re-review after conflict resolution. Mergeable ∧ green ∧ reviewed.

Why re-reviewed: #286 squash-merged to master and its branch was deleted; GitHub retargeted this PR's base to master, which left it CONFLICTING (master's #286 squash commit vs this branch's original #286 commits + #287's additions on the four shared files). neutral (full-heal, canPush) merged master in and resolved, keeping #287's superset on each shared file.

Re-verification of the resolution (nothing lost, nothing masked):

Green: full suite 2140 pass, 0 fail, 1 skipped; the two directly-touched files 72/72; npm run build:types (tsc) exit 0 — no type-import breakage.

Same clean bill as the round-1 review of f628806; the only change is the conflict resolution, which preserved all reviewed content.

@philcunliffe

Copy link
Copy Markdown
Contributor Author

neutral verdict: approved ✅ (round 2, head 2fc31aa) — conflict with master resolved, mergeable ∧ green ∧ reviewed. Now that #286 has landed, #287 targets master directly and is held for you to merge.

@philcunliffe philcunliffe merged commit 7ea4b06 into master Jul 8, 2026
4 checks passed
@philcunliffe philcunliffe deleted the feat/pick-pending-export-hold branch July 8, 2026 20:52
philcunliffe added a commit that referenced this pull request Jul 9, 2026
…289)

The fresh-enroll candidate wait returns at the FIRST non-empty
enumeration, racing the daemon's still-running backfill: on the
2026-07-09 enroll it offered a handful of directories while the cache
held 126 distinct captured cwds, presenting a race-ordered slice of
history as the full set.

Rather than patch the wait again (#283, #286, #287), disable the
login-time trigger wholesale behind ENROLLMENT_PICKER_ENABLED = false
until the enumeration source and presentation are redesigned. Every
fork that would have prompted prints the durable-command hint instead;
the backfill wait, the registry-refresh re-boot, and the pick-pending
marker are skipped as pure overhead while the trigger is off.

Unchanged: hyp ignore --local-only, the machine-local list, existing
lists' enforcement at the export seam, and the picker module + its
unit tests, so re-enabling is one flag flip alongside the redesign.

Suspended-by forward-refs appended to LLP 0069 #trigger, 0072, 0080,
and 0093.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:adopt Foreign PR adopted into neutral's reconcile scope neutral:approved neutral reviewed this adopted PR and approves; held for maintainer merge (LLP 0025)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant