sean-brydon · Oct 13, 2026 · Oct 13, 2026
diff --git a/design/structured-agents/plan.md b/design/structured-agents/plan.md
@@ -0,0 +1,164 @@
+# Structured agents: plan
+
+Status: 2026-10-13. Phase 0 (spike) done: go. Phase 1 (driver interface) in progress. The spike's recordings join the repo as test fixtures in phase 3, once reviewed.
+
+## Why
+
+Today every agent is its TUI in tmux, and the conversation view is rebuilt from
+the agent's own record files (`~/.claude/projects/*.jsonl`, Codex rollouts),
+stat-polled, plus screen-scraping (`screen-status.ts`, `screen.ts`) and
+keystroke-driving (`answer.go`, `controls.go`, `orchestrate.go` paste+Enter).
+
+Every comparable app (t3code, Vibe Kanban, Sculptor, Paseo, Superset, Nimbalyst,
+Orca's structured mode) instead drives the agent over a structured channel and
+keeps its own record:
+
+- Claude: `claude -p` stream-json in/out (Vibe Kanban, Sculptor) or the Agent SDK
+  on top of the same protocol (t3code, Paseo, Superset).
+- Codex: `codex app-server` (JSON-RPC over stdio), unanimously.
+- Others: ACP (Agent Client Protocol).
+- Own store (SQLite / log); only the agent's session id is kept, for `--resume`.
+- CLI transcripts are read only for history import, never as the live feed.
+
+Policy (checked 2026-10-13): running the real, unmodified `claude` binary with the
+user's own login, including `claude -p` and stream-json, is allowed and draws
+from subscription limits (support article 15036540, updates 2026-06-15 and
+2026-10-07). Never read, copy or proxy the user's credentials. Risk: the paused
+June change would have moved `-p`/SDK usage off subscription limits; if it comes
+back, the TUI path is the fallback, so the driver stays a setting.
+
+## Decisions (from the user)
+
+| | |
+|---|---|
+| Default | Chat first; "Open in terminal" hands the same session to the TUI via `--resume` |
+| Claude link | Raw CLI stream-json spoken by berthd in Go (no Node, no SDK) |
+| v1 scope | Claude + Codex (app-server) + a generic ACP driver for the rest |
+| ACP agents | Gemini CLI, opencode, Cursor/Copilot/others: one config-driven driver |
+| History | berthd's own event log per box; agent session id kept for resume |
+| Approvals | Default bypass (berthd auto-allows each request in default mode; works as root); a global setting (repo/kit can override) |
+| Rollout | Spike first, then phased PRs behind Labs |
+
+## Shape
+
+```
+app ──SSE/API──▶ berthd ──unix socket──▶ berthd agent-host (in tmux) ──stdio──▶ claude -p / codex app-server / ACP agent
+                   │                            │
+                   └── index (SQLite)           └── events.jsonl (append-only, berth's own format)
+```
+
+- The **driver** turns agent protocol into `transcript.Item` upserts (the model
+  the app already renders) plus live state (busy/idle/waiting, partial text,
+  status) and requests (approval, question, plan).
+- The **agent-host** owns the agent's stdio inside a tmux session, so an agent
+  survives a berthd restart/upgrade or a dropped link (none of the desktop apps
+  solve this; it's the Berth-specific part). berthd reconnects and replays the
+  log from an offset.
+
+## Phases
+
+### 0. Spike (throwaway, scratch dir, against claude 2.1.296 and the boxes' codex)
+
+Answer, with recorded protocol fixtures (sanitised) kept for tests:
+
+- [ ] Long-lived multi-turn `claude -p --input-format stream-json --output-format stream-json --verbose --include-partial-messages`
+- [ ] Control requests: interrupt, set permission mode, set model
+- [ ] `--permission-mode bypassPermissions --permission-prompt-tool stdio`: do AskUserQuestion and ExitPlanMode still reach us (Vibe Kanban relies on this)?
+- [ ] Bypass when the box user is root (VPS): does Claude refuse, and what does it need?
+- [ ] `/compact`, custom commands, skills; `system/init.slash_commands`; which built-ins refuse
+- [ ] Images/attachments as content blocks
+- [ ] berth's installed hooks: do they fire under `-p`? (avoid double turn events)
+- [ ] Session hand-off: stream session → `claude --resume <id>` TUI → back; `--fork-session`; `--resume-session-at <uuid>`
+- [ ] Kill mid-turn, resume: what's lost
+- [ ] Codex `app-server`: thread/start, turn/start, turn/interrupt, approvals, resume
+- [ ] ACP handshake with `gemini --experimental-acp` and `opencode acp`
+
+Exit: a go/no-go note per item; any no-go changes the phases below.
+
+### 1. Driver interface (no behaviour change)
+
+- `internal/agentdrv`: `Driver{Start, Send(prompt, attachments), Interrupt, SetMode, SetModel, Respond(requestID, decision), Stop, Events()}`.
+- Wrap today's path as `ptyDriver` (tmux keys + transcript tail) so the
+  conversation pane talks to one API regardless of driver.
+- Box capability `driver`; the app keeps today's code path for older boxes.
+
+### 2. Agent-host + event log
+
+- `berthd agent-host` in the agent's tmux session: spawns the agent, owns stdio,
+  appends to `<state>/conversations/<id>/events.jsonl` (versioned), serves a
+  unix socket. Lock: one agent process per conversation, ever.
+- Keep streamed partial text in the log (a killed agent loses it). Stop =
+  interrupt, then SIGTERM, then kill the process tree (SIGKILL leaves Bash
+  children running).
+- SQLite index: conversation id, location/worktree, agent, driver, agent session
+  id, title, status, cost, timestamps.
+- Push to the app via the events bus (items in the event, not a "changed" ping);
+  no stat-polling for driver agents.
+- Attaching to the tmux session shows a readable live render, so `berth attach` keeps working.
+
+### 3. Claude driver
+
+- Spawn: the flags above + `--resume <id>` when continuing. Always
+  `--permission-mode default --permission-prompt-tool stdio`; "bypass" is berthd
+  auto-allowing each request (Claude refuses the bypass flag as root without
+  `IS_SANDBOX=1`). Questions and plans always come to us.
+- Set an env var (e.g. `BERTH_DRIVER=structured`) so `berthd hook` stays silent
+  for these sessions; the user's other hooks keep running.
+- Read commands/models from the `initialize` control request; follow
+  `session_id` changes (`/clear` starts a new one). Merge assistant messages by
+  `message.id` (one per content block).
+- Map messages → Items, reusing `internal/transcript/claude.go` where shapes
+  match; partial messages → live text (replaces `screen-status.ts` scraping).
+- Approvals, AskUserQuestion, ExitPlanMode → cards (the `question` item exists);
+  no keystroke form-driving for these agents.
+- Subagents via `parent_tool_use_id` → crew items. Usage/cost from `result` →
+  usage plugin.
+- Slash commands from init; terminal-only ones offer "Open in terminal".
+- Fork (`--fork-session`) and rewind (`--resume-session-at`).
+
+### 4. Terminal hand-off
+
+- "Open in terminal": at idle (or after interrupt) the host stops the driver
+  and runs `claude --resume <id>` in the same tmux session; the pane shows the
+  terminal and the conversation falls back to the transcript tail.
+- "Back to chat": the TUI exits, the driver resumes the session; turns made in
+  the TUI are imported once from the transcript into the event log.
+
+### 5. Codex driver: `codex app-server`
+
+thread/start|resume, turn/start|interrupt, approvals → cards, items → Items.
+
+### 6. Generic ACP driver
+
+- Agents defined in config (box, kit, repo):
+  `{ "gemini": "gemini --acp", "opencode": "opencode acp", "cursor": "cursor-agent acp" }`.
+- Per-agent quirks: `authenticate` when asked (cursor), resume via
+  `session/resume`, pick permission options by `kind`, usage optional, ignore
+  unknown vendor requests (`cursor/create_plan`).
+- session/new|load, session/prompt, session/update → Items,
+  session/request_permission → cards (or auto-allow in bypass), session/cancel.
+- Agents without ACP stay terminal-only.
+
+### 7. Flows onto the driver
+
+Judges, reviewers and `.ask` steps (`headless.go`) use the same driver, so they
+get the event log, live view and cost. Read-only mode = a tool allow/deny list.
+
+### 8. Make it the default
+
+- New agents open in chat; Labs toggle removed.
+- Settings: approval mode (global, default bypass), default driver
+  (structured | terminal: the fallback if `-p` billing changes).
+- Transcript tailing stays for terminal-mode agents and importing old sessions.
+
+## Cross-cutting
+
+- **Protocol drift:** stream-json control messages are undocumented. Contract
+  tests on recorded fixtures, plus a scheduled CI job against the latest
+  `claude`/`codex` that fails loudly on unknown message types.
+- **Bypass default:** this is a change from today (berth never adds
+  skip-permissions). Worktrees limit the blast radius; show the mode on every
+  agent; repo/kit can force `ask`.
+- **Credentials:** never touch `~/.claude`/`~/.codex` auth; login stays the
+  user running `claude`/`codex` themselves (terminal hand-off covers `/login`).
+- **Naming:** the Claude name/logo can't be part of the product's name or logo.
diff --git a/docs/reference/app-api.mdx b/docs/reference/app-api.mdx
@@ -221,7 +221,9 @@ failed after it went out (the box may have acted on it). The box API:
 `waiting`, `finished`), plus `preset` (the agent preset it was started
 with), `turn` (its current or last turn), `state_seq` and `fidelity`
 (`hooks`, `partial` or `screen`), `queued` (prompts held until it is idle)
-and, while it waits, `ask` (what its hooks said it asks for). `exited` is
+and, while it waits, `ask` (what its hooks said it asks for). `driver`
+says how the box reaches the agent: `terminal` (its own screen in tmux) for
+every agent today; boxes that say it list `driver` in `capabilities`. `exited` is
 true once the program has ended. `title` names the work: the first line of
 the prompt it started with, or of the first prompt it was sent or typed, cut
 to about 48 characters; `PATCH sessions/{name}` with `{ title }` renames it

diff --git a/docs/reference/plugin-sdk.mdx b/docs/reference/plugin-sdk.mdx
@@ -333,7 +333,7 @@ Go structs have them, in `packages/plugin-sdk/src/types.ts`:
 | `BoxStatus` | `name`, `address`, `network?`, `fingerprint`, `state` (`connecting`, `online`, `offline`, `untrusted`), `error?`, `latency_ms?`, `since`, `link?` (`slow?`, `reason?`, `max_ms?`, `jitter_ms?`, `path?`) |
 | `Location` | `name`, `path`, `repo`, `worktrees?`, `scripts`, `agents?`, `remote?`, `slug?`, `default_branch?`, `repo_trust?` (`none`, `trusted`, `untrusted` or `changed`) |
 | `Worktree` | `name`, `path`, `branch?`, `head?`, `main?`, `setting_up?`, `port?`, `title?`, `parent?` (the path of the worktree whose agent handed this one off) |
-| `Session` | `name`, `location?`, `dir`, `command?`, `created`, `attached`, `exited`, `agent?`, `agent_state?`, `state_since?`, `preset?`, `turn?`, `state_seq?`, `fidelity?`, `queued?`, `ask?`, `title?` (what the work is called: the first line of its first prompt, or a rename) |
+| `Session` | `name`, `location?`, `dir`, `command?`, `created`, `attached`, `exited`, `agent?`, `agent_state?`, `state_since?`, `preset?`, `turn?`, `state_seq?`, `fidelity?`, `queued?`, `ask?`, `title?` (what the work is called: the first line of its first prompt, or a rename), `driver?` (`terminal`) |
 | `Turn` | `id` (`"<session>#<n>"`), `session`, `agent?`, `n`, `origin?`, `sent_seq?`, `end_seq?`, `state` (`queued`, `pending`, `running`, `waiting`, `finished`, `exited`, `lost`), `queued?`, `started?`, `ended?`, `waits?` (each `{ start, end?, reason?, ask? }`), `fidelity?`, `idem_key?`, `status?` |
 | `Ask` | `tool?`, `input?` (the command or file path, at most 300 characters), `why?`, `message?` |
 | `QueuedPrompt` | `turn`, `preview` (the first 280 characters), `length`, `origin?`, `at` |

diff --git a/internal/box/answer.go b/internal/box/answer.go
@@ -58,7 +58,7 @@ func (b *Box) answerSession(w http.ResponseWriter, r *http.Request) error {
 	if req.Tool == "" {
 		return badRequest("say which question to answer (tool)")
 	}
-	sess, path, err := b.claudeRecord(r)
+	sess, err := b.Sessions.Get(r.Context(), r.PathValue("name"))
 	if err != nil {
 		return err
 	}
@@ -73,24 +73,10 @@ func (b *Box) answerSession(w http.ResponseWriter, r *http.Request) error {
 	}
 	defer mu.(*sync.Mutex).Unlock()
 
-	res, err := reader().Read("claude", path, sess.Dir, 0)
+	shown, err := b.driverFor(sess).Answer(r.Context(), sess, req)
 	if err != nil {
 		return err
 	}
-	it, open := transcript.OpenQuestion(res, req.Tool)
-	switch {
-	case it.Kind == "":
-		return httpError{http.StatusNotFound, "that question isn't in Claude's conversation"}
-	case !open:
-		return httpError{http.StatusConflict, "those questions were already answered"}
-	}
-	shown, err := checkAnswers(it.Questions, req.Answers)
-	if err != nil {
-		return err
-	}
-	if err := driveAnswers(r.Context(), tmuxTerm{s: b.Sessions, target: "=" + sess.Name + ":"}, it.Questions, req.Answers); err != nil {
-		return err
-	}
 	writeJSON(w, AnswerResult{Answered: shown})
 	return nil
 }

diff --git a/internal/box/api.go b/internal/box/api.go
@@ -55,6 +55,9 @@ type Box struct {
 	// Turns is the turn ledger: what every agent session is doing, turn by
 	// turn. Without it agents read as running.
 	Turns *Turns
+	// Drivers replace a driver by kind (driver.go); nil, or a kind not in
+	// it, is the built-in one.
+	Drivers map[string]Driver
 	// Hooks, when set, may refuse actions through "before:" hooks.
 	Hooks *hooks.Runner
 	// Flows runs the box's and its repositories' automations.

diff --git a/internal/box/controls.go b/internal/box/controls.go
@@ -5,7 +5,6 @@ import (
 	"net/http"
 	"regexp"
 	"strings"
-	"time"
 
 	"github.com/cosscom/shipyard/internal/events"
 	"github.com/cosscom/shipyard/internal/integrations/adapters"
@@ -84,11 +83,10 @@ var (
 	interrupted = regexp.MustCompile(`(?i)\binterrupted\b`)
 )
 
-// interruptSession answers POST /v1/sessions/{name}/interrupt: it presses
-// Esc, as stopping an agent at its terminal does, waits for the agent to
-// stop, and ends its turn in the ledger. Claude Code's hooks say nothing
-// when a turn is interrupted, so without this the session would read as
-// working until its next prompt.
+// interruptSession answers POST /v1/sessions/{name}/interrupt: its driver
+// stops the agent (Esc, in a terminal), and the turn ends in the ledger.
+// Claude Code's hooks say nothing when a turn is interrupted, so without
+// this the session would read as working until its next prompt.
 func (b *Box) interruptSession(w http.ResponseWriter, r *http.Request) error {
 	name := r.PathValue("name")
 	sess, err := b.Sessions.Get(r.Context(), name)
@@ -105,22 +103,10 @@ func (b *Box) interruptSession(w http.ResponseWriter, r *http.Request) error {
 	if b.Turns != nil {
 		st, _ = b.Turns.State(name)
 	}
-	if err := b.pressKeys(r.Context(), name, "Escape"); err != nil {
+	stopped, err := b.driverFor(sess).Interrupt(r.Context(), sess)
+	if err != nil {
 		return err
 	}
-	// The agent stops within a moment: its "esc to interrupt" goes, and
-	// Claude Code says "Interrupted".
-	stopped := false
-	deadline := time.Now().Add(4 * time.Second)
-	for !stopped && time.Now().Before(deadline) {
-		time.Sleep(150 * time.Millisecond)
-		screen, err := b.Sessions.Screen(r.Context(), name, 0)
-		if err != nil {
-			break
-		}
-		tail := lastLines(screen, 14)
-		stopped = !working.MatchString(tail) && adapters.ScreenState(screen) != "waiting"
-	}
 	if stopped && b.Turns != nil && (st.State == "running" || st.State == "waiting") {
 		b.Events.Publish(events.Event{Type: adapters.Finished, Box: b.Name, Origin: origin(r), Data: map[string]any{
 			"session": name, "path": sess.Dir, "agent": controlAgent(sess), "source": "interrupt", "status": "interrupted",
@@ -182,28 +168,17 @@ func (b *Box) sessionControls(w http.ResponseWriter, r *http.Request) error {
 	if err != nil {
 		return err
 	}
-	agent := controlAgent(sess)
-	out := Controls{Agent: agent}
-	if !sess.Exited {
-		screen, err := b.Sessions.capture(r.Context(), sess.Name, 0)
-		if err != nil {
-			return err
-		}
-		if agent == "claude" {
-			out = claudeControls(screen)
-			out.Modes = []string{"default", "acceptEdits", "plan", "auto", "bypassPermissions"}
-		} else if m := screenLimit.FindAllString(lastLines(screen, 20), -1); m != nil {
-			out.Limit = strings.TrimSpace(m[len(m)-1])
-		}
+	out, err := b.driverFor(sess).Controls(r.Context(), sess)
+	if err != nil {
+		return err
 	}
 	writeJSON(w, out)
 	return nil
 }
 
-// setMode answers POST /v1/sessions/{name}/mode {"mode":"plan"}: it presses
-// Shift+Tab, as a person cycling Claude Code's modes does, until its screen
-// shows that mode, and says where it ended up. A mode the session can't
-// reach (bypass, unless it was started allowing it) is a 409.
+// setMode answers POST /v1/sessions/{name}/mode {"mode":"plan"}: the
+// session's driver switches the mode (Shift+Tab in a terminal), and it says
+// where it ended up. A mode the session can't reach is a 409.
 func (b *Box) setMode(w http.ResponseWriter, r *http.Request) error {
 	var req struct {
 		Mode string `json:"mode"`
@@ -219,58 +194,19 @@ func (b *Box) setMode(w http.ResponseWriter, r *http.Request) error {
 	if sess.Exited {
 		return ErrSessionExited
 	}
-	if controlAgent(sess) != "claude" {
-		return badRequest("only Claude Code's mode can be switched from here; use its terminal")
-	}
 	if b.Turns != nil {
-		// Shift+Tab at a permission prompt picks an option: never there.
+		// A mode change at a permission prompt can pick an option there.
 		if st, ok := b.Turns.State(name); ok && st.State == "waiting" {
 			return httpError{http.StatusConflict, "the agent is waiting for an answer; answer it first"}
 		}
 	}
 	if err := b.before(r, "session.send", map[string]any{"name": name, "key": "btab", "mode": req.Mode}); err != nil {
 		return err
 	}
-	read := func() (string, error) {
-		screen, err := b.Sessions.Screen(r.Context(), name, 0)
-		if err != nil {
-			return "", err
-		}
-		if adapters.ScreenState(screen) == "waiting" {
-			return "", httpError{http.StatusConflict, "the agent is showing a question; answer it first"}
-		}
-		return claudeControls(screen).Mode, nil
-	}
-	mode, err := read()
+	mode, err := b.driverFor(sess).SetMode(r.Context(), sess, req.Mode)
 	if err != nil {
 		return err
 	}
-	seen := map[string]bool{mode: true}
-	for presses := 0; mode != req.Mode && presses < 7; presses++ {
-		if err := b.pressKeys(r.Context(), name, "BTab"); err != nil {
-			return err
-		}
-		// The footer redraws within a moment.
-		next := mode
-		for wait := 0; wait < 10 && next == mode; wait++ {
-			time.Sleep(100 * time.Millisecond)
-			if next, err = read(); err != nil {
-				return err
-			}
-		}
-		mode = next
-		if mode != req.Mode && seen[mode] && presses > 0 {
-			break // round the whole cycle: not one this session has
-		}
-		seen[mode] = true
-	}
-	if mode != req.Mode {
-		msg := "this session doesn't offer that mode; it is in " + mode + " mode"
-		if req.Mode == "bypassPermissions" {
-			msg = "bypass needs Claude Code started with --dangerously-skip-permissions; this session is in " + mode + " mode"
-		}
-		return httpError{http.StatusConflict, msg}
-	}
 	b.publish(r, "session.mode", map[string]any{"name": name, "mode": mode})
 	writeJSON(w, map[string]any{"mode": mode})
 	return nil