Superset/agent session phase1 - #71
Merged
Merged
Conversation
Superset Agent Session Plan Phase 1: a gen2_superset_runs table CoDev owns as the source of truth for a Superset terminal-agent session's identity, credential lease, and lifecycle state, plus a gen2_superset_run_events audit trail. Registration is idempotent on (workspaceId, idempotencyKey) so a client retry resumes the same run instead of starting a second provider process. No credential material is stored; connectionId only points at the existing encrypted provider_credentials row. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Superset Agent Session Plan Phase 2: a provider-agnostic credential profile primitive for codev-guestd, generalizing the existing TemporaryCodexHome pattern in guest.rs into a handle-based registry a caller can hold across a Superset-launched process rather than only across one exec call. A profile is a fresh 0700 run-scoped directory under the guest daemon's private /tmp (already outside /workspace and outside Superset's shared /var/lib/codev-superset via guestd's own ProtectSystem=strict + PrivateTmp=true systemd sandboxing), holding only the credential files one launch recipe needs, removed on handle release or by a TTL sweep for anything a VM restart left behind. Only the handle -- never a raw path or credential payload -- is meant to cross into a Superset launch request. Implements the Codex auth-cache recipe only, per the plan's "start with the existing Codex official auth-cache format." Not yet wired into GuestService: Phase 3 adds the Superset launch endpoint that will actually create a profile and resolve a handle back into a launch environment. cargo is not installed on this machine, so this module is unverified by me: no cargo fmt/clippy/test. Needs a real Rust toolchain run before merge. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's rust job caught these; fixing before clippy/test can run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
0055_snapshot.json and 0056_snapshot.json were missing despite journal entries and applied .sql files, leaving 0057's prevId pointing straight at 0054. drizzle-kit diffed against the stale 0054 snapshot and re-emitted a CREATE TABLE for gen2_yjs_documents on the next db:generate. Reconstructed both snapshots from the 0055/0056 .sql files and re-chained ids so db:generate reports no schema changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…web slice)
Superset Agent Session Plan Phase 3, scoped to the apps/web piece only
-- the Superset host-service (vendor/superset, bun) and orchestrator/
guestd (Rust) routes this adapter calls do not exist yet, and
vendor/superset has no local or CI verification available in this
environment, so that side is deliberately deferred.
- superset-agent-sessions-feature.ts: CODEV_SUPERSET_AGENT_SESSIONS_ENABLED,
independent of the existing terminal/Git/worktree runtime flag.
- superset-agent-orchestrator-client.ts: thin HTTP client for the
/v1/sandboxes/{id}/superset-agents routes Phase 3 adds to the
orchestrator, following the existing codex-execs route convention.
Explicitly marked provisional -- the wire contract is this adapter's
best guess, not confirmed against a real Rust route.
- superset-agent-runtime.ts: wires Phase 1's durable run state machine
(register/claim-lease/mark-started/finished/failed/recovery-required)
around start/poll/cancel/reconcile calls into that client, mirroring
the lease-claim and rollback shape already used by
apps/web/lib/gen2/agent.ts. Not called from any route yet.
Fully typecheck/test/lint verified locally (19 new tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…Rust slice, unverified)
Superset Agent Session Plan Phase 3: the orchestrator/guestd half of
"Add corresponding fixed routes in guestd, the orchestrator, and a
server-only CoDev runtime adapter" (the adapter itself landed
separately in apps/web). vendor/superset's own /codev/agents endpoints
this ultimately calls do not exist yet -- that piece stays deferred,
per prior discussion, since it uses bun with no local or CI
verification available here.
New route family, mirroring the existing codex-execs plumbing at every
layer:
- model.rs: SupersetAgentStart/Input/Poll request/response types.
- guest.rs: /v1/superset-agents routes. Unlike start_codex_exec, this
guest does not execute the provider process itself -- Superset's
host-service does, so there is no local session table. Each handler
validates and forwards to Superset's existing bridge
(superset_bridge_request, already used by the file/Git/worktree
proxy) at /codev/agents. Deliberately does NOT use the Phase 2
provider-profile primitive here: that primitive materializes
credentials under this guest daemon's *private* /tmp
(PrivateTmp=true in the codev-guestd systemd unit), a mount
namespace Superset's own service cannot see, so it would not help
Superset build its own launch environment. Forwarding the validated
request once over this bridge-secret-authenticated, guest-internal
channel is itself the "trusted guest/host launch path" Phase 2
allows to see a credential -- documented at start_superset_agent.
- guest_client.rs, backend/mod.rs (Backend enum + FakeBackend),
backend/firecracker.rs: host-side plumbing from the orchestrator's
HTTP API down to one VM's guestd, one-to-one with the codex-exec
family's existing shape.
- http_api.rs: POST/DELETE/GET /v1/sandboxes/{id}/superset-agents...,
with a permissive ID pattern (Superset mints the agent ID, not this
guest, so it isn't pinned to the guest's own "prefix-digits-digits"
session ID convention).
New tests: guest.rs request-validation cases plus a bridge-unreachable
503 proof; http_api.rs end-to-end router test against Backend::fake().
cargo is not installed on this machine -- as with Phase 2's
provider_profile.rs, none of this has been compiled, clippy'd, or
tested by me. Needs a real Rust toolchain run (a PR, as Phase 2 used)
before merge; expect at least cargo fmt formatting fixes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI's rust job caught these; fixing before clippy/test can run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Request::builder().uri() rejects a literal space before the request ever reaches Axum's router (InvalidUriChar), so the "bad agent ID" case needs a percent-encoded space to actually exercise validate_superset_agent_id instead of failing at request construction. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… the plan's route set, unverified) Superset Agent Session Plan Phase 3's last piece: the bridge-secret- protected agent endpoints orchestrator/guestd's new /v1/superset-agents routes forward to (POST /codev/agents, /agents/:id/input, /agents/:id/poll, DELETE /agents/:id, GET /agents/:id/recovery), in vendor/superset/packages/host-service/src/codev/agents.ts, registered in app.ts alongside the existing file/runtime bridges. Key finding that shaped the design: terminal-agents/store.ts's TerminalAgentBinding is populated by hook events an agent CLI reports as it runs (see its doc comment) -- there is no headless "launch a process" primitive in this package to call. A Superset terminal-agent is an agent CLI running inside a tracked terminal. So this reuses the exact same terminal primitives registerCoDevRuntimeBridge's /codev/terminal routes already use (createTerminalSessionInternal, writeFramedInputToSession, snapshotSession, disposeSessionAndWait), delivering the agent's command via initialCommand instead of starting an empty interactive shell, and does not attempt to fake hook-based TerminalAgentStore tracking. "Recovery" is therefore a liveness check against the terminal session (still-live via snapshotSession's adopt path), not true resume-candidate tracking -- a real gap against the plan's Phase 5 language, left for whoever builds that. Other decisions worth a reviewer's attention: - The credential (when present) is written to a fresh 0700 directory under SUPERSET_HOME_DIR (this service's own private, writable home; see bootstrap-host.sh), never under the shared /workspace disk, and removed on close/failure. - initialCommand types the agent's command into a live shell, so each argument is POSIX-single-quote-escaped (posixShellQuote) before joining -- required because a command's last argument is a member's prompt text verbatim, an injection vector otherwise. - The launch command ends with `; exit $?` so a one-shot exec's shell terminates and the terminal subsystem's existing PTY-exit handling sets endedAt, which poll/recovery read. - idempotencyKey dedup here is defense-in-depth: CoDev's own Phase 1 run registration already prevents most retries from reaching this far, but a proxy-level retry between guestd and this bridge would still reattach instead of relaunching. vendor/superset uses bun, is not in this repo's pnpm workspace, and has no CI job in this repo that touches it (deploy-runtime-azure.yml only runs on push to main). This file has had zero automated verification -- no bun install, typecheck, lint, or test -- and no sibling file in codev/ has tests to model one on. Needs a real bun toolchain run before this can be trusted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Keeps startGen2AgentTurn/pollGen2AgentTurn/cancelGen2AgentTurn's browser
contract unchanged; when CODEV_SUPERSET_AGENT_SESSIONS_ENABLED is set they
now delegate to a per-chat Superset terminal-agent run instead of the direct
codex-exec sandbox path, reusing the already-verified Phase 1-3 run state
machine and orchestrator routes.
New in superset-agent-runtime.ts: startGen2SupersetAgentTurn provisions one
worktree per chat (idempotent, so concurrent chats get distinct worktrees),
starts the run, and persists the user message + durable turn row the same
way the sandbox path does; pollGen2SupersetAgentTurn and
cancelGen2SupersetAgentTurn are thin delegates to the existing Phase 3
primitives. agent.ts's three exported functions now just flag-branch and
translate errors, matching the direct path's existing error shapes.
Fixes a real wire-format bug found while wiring this up: the Phase 3
orchestrator client expected poll chunks as `{ dataBase64 }` (base64), but
both the vendor host-service route and the already-verified terminal-bridge
convention (orchestrator-superset-runtime.ts's terminalPollSchema) use plain
`{ data }` text. Fixed the schema to match; added toCodexExecChunks() to
adapt that into the CodexExecChunk shape the browser's chat-panel.tsx already
merges and decodes for a live reply.
Also fixes a correctness gap: Superset's poll returns the full terminal
buffer on every change, not an incremental append, so reusing
recordGen2TurnChunks's accumulate-and-append logic would have re-stacked
duplicate snapshots turn over turn. Added recordGen2SupersetRunOutput in
turns.ts, which replaces the stored output with each poll's latest snapshot
instead of appending -- reduceCodexTurn's existing idempotent re-parse
(turn-events.ts) still gives stable item ids and the correct final reply.
Extracted buildGen2CodexCommand into codex-command.ts (both paths need it;
agent.ts <-> superset-agent-runtime.ts would otherwise import each other).
Verification: apps/web typecheck and eslint are clean on every touched file;
90 tests pass across 18 suites. superset-agent-runtime.test.ts and 4 other
gen2 suites fail to even load in this sandbox on a pre-existing,
unrelated @azure/arm-compute ESM/CJS resolution error (confirmed present on
the pre-Phase-4 baseline via git stash, so not introduced here) -- the new
Phase 4 test cases added to that file are unexercised locally. The Rust
orchestrator/guestd routes and the vendor/superset host-service endpoints
this calls remain unverified end-to-end, per the Phase 3 commits' own
caveats -- nothing here has run against a real Firecracker guest or Superset
host yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`providerSurfaceCapability` counted a connected `codev claude-auth` setup-token towards rooms readiness, and `surfaceFlags` defaults an unset `enabledForRooms` to true. A member whose only Claude login was that token therefore saw a green "Ready for chat rooms", while every reply failed: `resolvePersonalChatSubscription` resolves Claude solely through `getConnectedClaudeRuntime`, which reads the browser runtime in `claude_connection_sessions` and never this token. There is no toggle in the UI to turn the claim off, so the member had no way out of it. The token stays what it is — a workspace-only login, consumed by `resolveClaudeCliTokenForIde` — and the card now reads "Connect from your terminal below" for rooms, which is true. Step 0 of the provider-connection cleanup. The credential lease's matching asymmetry (Gen 2 claims and ignores it, rooms hard-fail on it) is deliberately left alone: with no reaper, the 16-minute TTL is the only thing releasing a stranded lock, so it is replaced wholesale by run-derived leases in a later step rather than tuned here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`saveProviderCredential` refuses an `anthropic` OAUTH_TOKEN that did not come from the CLI, so every route in the Claude browser-OAuth flow — start, session, manual-code completion, callback — could only ever return 400. No UI called them: Claude connects through the official login runtime or `codev claude-auth`. The Codex `GET /api/auth/oauth/ codex` and its callback had no caller either; the live device-code pair (`session` + `poll`) stays. Removing Claude from `OAuthProvider` makes the manual-code flow mode and `parseManualAuthorizationCode` dead, and leaves no configuration whose provider is `anthropic`, so the two `anthropic` special cases in the authorize URL and the JSON-bodied token exchange go with them. Cursor and Codex are untouched. Also drops the org Bedrock card: no executor has ever run a `bedrock` credential. The provider enum keeps the value for now — `packages/ide` and `apps/mobile` still reference it, and removing it there would force an embedded-bundle rebuild for a Gen 1 concern. Test infrastructure, because this area could not be verified: three `@azure/*` packages take a named import from their own CommonJS state module, which Node cannot synthesise, so any test reaching `lib/platform/azure-kms` or `lib/runtime/azure` died at import. Nine files in `lib/providers` were failing that way, including the OAuth tests this change edits. Vitest now aliases those packages to stubs that throw if a test genuinely reaches Azure. `lib/providers` goes from 19 passing files (102 tests) to 28 (148). Step 1 of the provider-connection cleanup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four places answered "where can this credential run", by hand, and drifted: `surfacePredicates` in credentials.ts, the table in provider-surface-capability.ts, the ternaries in provider-account-card.tsx, and whatever `enabledFor` each connect flow happened to pass. Three resolvers then answered "which credential runs this turn" — Gen 1's `resolveAgentCredential`, rooms' `resolvePersonalChatSubscription`, and Gen 2's `resolveGen2Codex` — and none consulted the table the settings page rendered from. A readiness badge was a guess about what a different function would do. `registry.ts` now holds the one table. It is keyed by credential *kind* rather than "subscription vs API key", because Claude's two subscription logins are different credentials that run in different places, and calling them one thing is what produced the mismatch. `resolve.ts` walks it: each provider's kinds in preference order, filtered to what the target executor can run, one loader per kind. Readiness is the same walk with `dryRun`, so it cannot disagree with what a turn does — and it decrypts nothing. Executors are separated into `rooms`, `workspace` (Gen 1 Orca) and `gen2` because they genuinely differ: the Gen 1 host takes six named credential fields, while the Gen 2 guest route takes `codexAuthCacheJson` alone and has no channel for an environment variable. That last fact — not a policy — is the only reason Claude cannot run in Gen 2, and it is now one boolean in the registry instead of an unwritten assumption. `LaunchProfile` (files + env) is the neutral shape that replaces passing a Codex auth cache by name; Gen 2 unwraps it back into the named field for now, and that shim is what the guest change deletes. Per P3-C, `enabled_for_rooms` and `enabled_for_workspace` are replaced by `allow_in_shared_workspaces`. The two they replace claimed to gate surfaces but were read on one path of three — the subscription resolvers ignored them entirely — so flipping a settings toggle changed a badge and nothing else. Where a credential *can* run belongs to the registry; what was genuinely the member's to decide is whether their own subscription may fund a turn inside a workspace other people can see, which every resolution path now reads. Migration 0058 adds it and carries over the old workspace flag; 0059 drops both. Two reporting bugs fall out of the rewrite: an API key no longer claims to be rooms-ready (the rooms executor has no API-key path), and a Cursor login no longer claims rooms readiness at all (`roomReplyOptions` has only ever offered Claude and Codex). The snapshot keeps its `enabledForRooms`/`enabledForWorkspace` shape, now derived from the registry rather than stored, so the copy of this logic in `packages/ide` keeps working without an embedded-bundle rebuild. It goes when the settings redesign changes that contract deliberately. Step 3 of the provider-connection cleanup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`claimHostedCodexExecution` set `unavailable_until` sixteen minutes ahead. Nothing reaped it, so the stamp was the reaper: a turn that died without releasing took the member's subscription out of service for a quarter of an hour — including their chat-room replies, which refuse outright on a busy credential. Gen 2, meanwhile, claimed the seat, caught the busy error and ran anyway, so the one lock meant "wait" on one surface and nothing at all on another while still blocking the first. A seat is now a row in `provider_credential_runs`, owned by a run and refreshed as that run is polled. Polling is what holding it means, so a long turn keeps its seat for exactly as long as it runs and a crashed one gives it up after two minutes instead of sixteen. `ref` records which run holds it, so a release from an earlier turn cannot free the seat a newer one took, and a retried start reattaches to its own seat rather than deadlocking against itself. A turn that starts before it has a durable id claims under its idempotency key and re-tags onto the session or run id, which is the identifier the poll and cleanup paths share. Every executor now asks the same question and waits rather than refusing: a member's second turn is a queueing problem, not an error. When the wait does run out they are told what holds the connection instead of getting a bare failure. Gen 2 and Superset no longer swallow a busy seat — Superset marks the run failed rather than putting two turns on one login. `includeBusy` is gone with it. Resolution answers "is one connected", never "is one free"; reporting a mid-turn subscription as missing sent members off to reconnect a perfectly good credential. Claude's browser runtime keeps its own `expiresAt` lease for now. It is fenced into `claude-runtime-execution`'s run reference and sandbox expiry, and that module is deleted when the browser login runtime is retired — rewriting it here would be work with a one-step shelf life. Step 6 of the provider-connection cleanup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`provider_credentials` carried two conventions for "shared" at once: `scopeType: "WORKSPACE"` for the older fallback key pool, and `scopeType: "ORGANIZATION"` whose scope id was *also* a workspace id for the `--org` logins. Reading a row meant knowing which mechanism had written it, `scoped-credential-sharing.ts` needed a paragraph to apologise for it, and `defaultSharingEnabled` existed only to tell the two apart. They are one scope now: a `WORKSPACE` credential belongs to that workspace's members, gated by membership as before. `sharing_enabled` goes with them. No caller ever set it — every write took the default implied by the scope — so it recorded nothing a member had decided while reading as though it did. The choice a member does make is `allow_in_shared_workspaces`, on their own credential, which every resolution path reads. Migration 0061 relabels the ORGANIZATION rows before narrowing the enum, dropping any that would collide with a WORKSPACE row already holding that (workspace, provider, type) — the fallback-pool entry is the one the resolvers have been reading. 0062 drops the column. The org page now says where these actually apply: a coding workspace, never a chat room or a Gen 2 turn. Both of those run on the credential of whoever asked, so a shared login could not fund them even before this change — the page just never said so. Step 7 of the provider-connection cleanup. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The exec route took `codexAuthCacheJson` by name, so the guest knew what
a Codex credential was and every new provider meant another field in the
model, the guest, and the host request carrying it — the Gen 1 IDE start
request already carries six. That naming is also the only reason Claude
cannot run in a Gen 2 workspace: its setup-token is an environment
variable, and the route had no channel for one.
A launch profile is files plus environment, which is the whole of what a
CLI needs. The guest writes each file 0600 inside the 0700 directory it
already creates per run, injects the environment last so a profile can
override the defaults, and knows nothing about which provider any of it
belongs to. Environment values may contain `{{profileDir}}`, expanded to
that directory — one substitution, and the reason a caller can say
`CODEX_HOME` without knowing where the guest puts things.
`codexAuthCacheJson` still works and is converted into the equivalent
profile at the edge, so there is a single code path below it and a
control plane that has not been redeployed keeps running. Nothing in
`apps/web` sends a profile yet; flipping Gen 2 over is the next step,
once this is deployed. The refresh read-back follows the profile rather
than assuming `auth.json` at the root, so a caller that nests the cache
under its own `CODEX_HOME` still has a refreshed token captured.
`provider_profile.rs` gains the Claude recipe alongside Codex's — it was
written for exactly this and never wired up — plus the validation the
routes share: relative paths only, caps on file count and size and on
environment names and values, and rejection messages that name a key or
path but never a value, since a rejection is the one place a credential
could otherwise reach a log.
Not included: capturing a `claude setup-token` from the login sandbox.
That belongs with retiring the browser login runtime, and guessing at
the CLI's output format is not something to land untested.
Step 2 of the provider-connection cleanup. Rust checks were not run
locally — no cargo toolchain on this machine — so CI is the first
compile.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cargo fmt --check` failures from CI: I hand-wrapped two assignments that fit inside the 100-column limit, so rustfmt wanted them joined. Also collapsed three constructs where I was guessing at rustfmt's choice rather than knowing it — a struct literal with an if/else in a field, an iterator chain building a Vec of tuples, and a borderline 99-column continuation — into plain statements that leave nothing to decide. Same behaviour, and the next formatting run has no opinion to disagree with. No toolchain on this machine to run `cargo fmt` against, so CI remains the check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adding `LaunchProfile`/`LaunchProfileFile` to `use crate::model::{...}`
shifts every later item, and rustfmt refills the whole list greedily
rather than only the line I touched. Applied its output verbatim from
the CI diff.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PHASE 1 - 2 (so far)