Cut cold-read latency and probe storms; instrument isolate residency

[RhysSullivan]

Production traces (Axiom, Cloudflare analytics, PostHog, 24h) put the console's slow reads on a few specific causes. This PR fixes the ones that are code, and adds instrumentation for the one that is not yet explained.

Latency fixes

• Tools read no longer waits on TTL-expired remote catalogs (packages/core/sdk). Every remote-catalog connection expired on the same 15-minute clock, so a read that tripped the TTL paid one tools/list handshake per connection (p50 2.2s on GET /api/tools; p50 380ms when no sync fired). Expiry is a guess, not a signal: expired catalogs now re-list behind the read (stale-while-revalidate), while stale-marked and config-revised rebuilds are still awaited within the grace budget. toolsSyncGraceMs: null keeps strict mode. New span attributes: executor.tools.sync_awaited, executor.tools.sync_deferred, executor.tools.sync_trigger.
• Cloud keeps the request's postgres socket open for deferred rebuilds. DbServiceShape.keepAlive retains the detached work; the scope finalizer hands sql.end to the platform waitUntil after every retained promise settles, including ones registered while earlier ones were settling. makeScopedExecutor folds the handle's keepAlive into ExecutorConfig.waitUntil. Long-lived-driver hosts are unaffected.
• Cold isolates race the JWKS store against the upstream fetch (apps/cloud/src/auth/jwks-cache.ts). The Workers Cache API read measured p50 1.4s / p90 2.8s on cold isolates while the upstream fetch measured p50 26ms; the store was read first, so every cold verify paid ~1.5s. Whichever answers first wins; the store still covers a slow or dead key server.
• OAuth discovery documents are answered at the Worker entry (apps/cloud/src/server.ts). They were dispatched through the app plane and paid its first evaluation on cold isolates (p95 5.0s cold vs 0ms warm).
• Members list runs its lookups concurrently and fetches listOrgMembers once instead of twice; per-member getUser fan-out raised from 5 to 20. A listOrgMembers failure still surfaces as an AccountError; an Autumn or invitations failure or defect still only degrades seats.
• Automatic connection health probes are deduplicated across remounts (packages/react). One session was posting /health once per second per connection for hours (12k/day per connection). A module-scope memory keyed by user, org, and connection with a 30s floor now survives remounts. A reconnect that clears the server verdict still probes, whether it lands while mounted or not, and suppression is bounded by an in-mount retry timer rather than by the next remount.

Trace volume

• Dropped mcp.host.register_tool (1.5M spans/day, always 0ms), mcp.host.register_resource, mcp.host.register_search_invoke, and mcp.request.read_json_rpc (460k/day, 0ms). mcp.namespace_search.count moved onto the enclosing span.

Instrumentation for the memory kills

The Worker is killed for exceededMemory ~24k times/day, taking every co-resident MCP session down with it. McpSessionDO.init reports resident runtimes up to 139 against a soft cap of 32, and the cap cannot evict a session with an open stream. New on every residency span: mcp.isolate.resident_evictable, resident_pinned, resident_streaming, resident_eviction_pending, in_flight_cold_builds. New unsampled log line mcp_isolate_cap_overflow whenever an init finds nothing evictable, with the same breakdown.

Review

Six Codex review passes. Findings fixed in follow-up commits: deferred-rebuild socket lifetime (including late registrations), JWKS blocking-fetch count on the failure path, seat defect fallback in listMembers, health memory partitioned by identity, reconnect-clear detection while unmounted, and bounded floor suppression that survives effect reruns. Final verdict: ready on correctness.

Verification

• Unit tests pass for every touched package: cloud (db, account, jwks-cache, api.request-scope, mcp/*, app-paths: 170 tests), core sdk (executor, connections, health-check: 113), core api server (10), plugin-mcp catalog-sync (9), host-mcp tool-server (61), cloudflare agent-session-durable-object (51), react use-connection-health (20).
• Typecheck, oxfmt, and oxlint clean on the changed files. Cloud build succeeds.
• Not run: e2e. Not yet measured: the production effect, which needs a deploy.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	executor-cloud	0265b4f	Sep 16 2026, 06:36 PM

[github-actions]

Cloudflare preview

Console	https://executor-preview-pr-2030.executor-e2e.workers.dev
MCP	https://executor-preview-pr-2030.executor-e2e.workers.dev/mcp
Deployed commit	0436089

Sign-in is Cloudflare Access (one-time PIN to an allowed email). The preview has its own database and encryption key; it is destroyed when this PR closes.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	executor-marketing	0265b4f	Commit Preview URL Branch Preview URL	Sep 16 2026, 06:36 PM

[pkg-pr-new]

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@2030

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@2030

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@2030

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@2030

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@2030

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@2030

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@2030

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@2030

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@2030

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@2030

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@2030

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@2030

executor

npm i https://pkg.pr.new/executor@2030

commit: 0436089