From 8960e4bb0cf8bb8a82855173d2f2f532b04b3105 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 14:00:12 +0000 Subject: [PATCH] Make `assembly live --files` default on Filesystem access (read/write/run in the launch directory, sandboxed with write/run confirmation) is now on by default for the live voice agent; pass `--no-files` to opt out. Previously the agent silently ran against an empty in-memory filesystem unless `--files` was passed, so it would report the working directory as empty even when real files were present. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RQzGo8b3DKLg9aCYp9k32M --- REFERENCE.md | 4 ++-- aai_cli/AGENTS.md | 2 +- aai_cli/commands/agent_cascade/__init__.py | 10 ++++----- .../test_snapshots_help_run.ambr | 17 ++++++++------ tests/test_agent_cascade_files.py | 22 +++++++++++++++++++ 5 files changed, 40 insertions(+), 15 deletions(-) diff --git a/REFERENCE.md b/REFERENCE.md index 75f8164c..f052b6ce 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -160,8 +160,8 @@ missing `npx`/`uvx`, an offline host) drops only its own tools, so a single brok tool never sinks the session. MCP tools are a live-run feature and are not reflected in `--show-code` output. -`--files` lets the agent read, write, and run code in the directory you launch -it from (off by default). Reads run immediately; a write, edit, or command run pauses +The agent reads, writes, and runs code in the directory you launch it from (on by +default; pass `--no-files` to disable). Reads run immediately; a write, edit, or command run pauses the turn for confirmation in the voice TUI — press `y`/`n` (`a` approves the rest of the session) or just say it ("approve" / "run it" / "go ahead"; anything unclear is treated as a no). Destructive commands (e.g. `rm -rf`, `sudo`) ignore the spoken answer and require a diff --git a/aai_cli/AGENTS.md b/aai_cli/AGENTS.md index 5a0e3d8d..ee604863 100644 --- a/aai_cli/AGENTS.md +++ b/aai_cli/AGENTS.md @@ -151,7 +151,7 @@ heavily-reworked commands with long bodies; small commands keep the inline - **`streaming/`** + `client.stream_audio` — v3 realtime API. Event callbacks run on the SDK reader thread and guard against `BrokenPipeError` (`stdio.silence_stdout()`) so a closed pipe never dumps a thread traceback. - **`core/sync_stt.py`** + **`core/signals.py`** + `commands/dictate/` — `assembly dictate`: headless dictation over the **Sync STT API** (`Environment.sync_base`, one POST `/transcribe` per utterance with the required `X-AAI-Model: u3-sync-pro` header; 80 ms–120 s of PCM/WAV). It needs no terminal: recording starts immediately and `dictate_exec._record` polls `signals.stop_on_terminate` between ~100 ms mic chunks for a SIGTERM, which finishes the utterance (clean exit 0) — so a hotkey tool like Hammerspoon can launch it as a background task and `kill -TERM`/`task:terminate()` to transcribe. SIGINT (Ctrl-C) still cancels (exit 130). Both boundaries (the stop latch, mic, HTTP) are injectable, so the suite never needs a real signal or microphone (`tests/test_dictate_exec.py` scripts the SIGTERM latch). Contrast `signals.terminate_as_interrupt` (used by `stream`/`agent`/`speak`), which routes SIGTERM into the *cancel* path instead. - **`agent/`** — full-duplex voice agent (mic in, TTS out via `voices.py`). -- **`agent_cascade/`** + `commands/agent_cascade/` — `assembly agent-cascade`: the same live terminal conversation as `assembly agent`, but **client-orchestrated** — `engine.run_cascade` wires Streaming STT → the LLM Gateway → streaming TTS itself instead of talking to the Voice Agent endpoint, mirroring what the `agent-cascade` `assembly init` template does server-side. **Sandbox-only** (streaming TTS has no prod host; guarded via `tts.session.require_available`). Reuses the agent slice's `DuplexAudio`/`AgentRenderer` and `core.client.stream_audio`/`core.llm.complete`/`tts.session.synthesize`; the three network legs are injected through `engine.CascadeDeps` (the `tts/session.py` seam) so the cascade — greeting, clause-level streaming TTS, barge-in, history window — is unit-tested against fakes with no sockets/mic/speaker. The LLM leg is a deepagents graph (`brain.py`) streamed token-by-token via `brain.build_streamer` (`graph.stream(stream_mode="messages")`): the engine buffers `SpeechDelta`s, flushes complete clauses with `text.pop_clauses` (soft-separator clauses gated by `engine._MIN_CLAUSE_CHARS`), and synthesizes each clause with **streaming TTS** (`tts.session.synthesize(on_audio=…)`) so audio starts on the first frame instead of after the whole reply. The reply runs on a throwaway producer thread feeding a `queue.Queue` the worker drains under a monotonic deadline (the wall-clock backstop that replaced `_complete_within`), and an abandoned-on-timeout graph leg's langchain `ThreadPoolExecutor` worker is detached (`_detach_executor_threads_since`) so it can't wedge interpreter exit. A `ToolNotice` surfaces the "Searching the web…" affordance and drops any unspoken preamble. Under `-v` (`debuglog.active()`) `brain._stream_graph` logs each accumulated assistant line, tool call, and tool result as it streams. **Front-end:** an interactive mic session in human mode runs a **voice-only Textual TUI** (`agent_cascade/tui.py`, `LiveAgentApp`) by default — there's no text input (you can't type to it), just a transcript + an animated voice bar tracking listening/thinking/speaking. It uses its own `banner` wordmark, `messages` widgets, and `tui_status.voicebar_markup`/`VOICE_FRAMES` — all modules that now live in `agent_cascade/`; the blocking `run_cascade` runs on a worker thread and reaches the UI through a `_TuiRenderer` (the `engine.Renderer` protocol) that hops each call onto the UI thread, and a quit calls `DuplexAudio.close` to end the mic iterator and unblock that worker. `_exec._should_use_tui` gates it: file/sample input, `--json`/`-o text`, and a non-TTY all fall back to the plain `AgentRenderer` line output. **`--files`** (off by default) swaps the brain's in-memory backend for a real-cwd, sandbox-capable `SandboxedShellBackend` (`aai_cli/agent_cascade/sandbox.py`): file ops behave as before (traversal-blocked `virtual_mode`), and because it implements `SandboxBackendProtocol` deepagents binds a *functional* `execute` that runs commands OS-sandboxed in the real cwd — `sandbox-exec` (SBPL) on macOS, `bwrap` on Linux, refused (never an unconfined fallback) on any other platform or with the sandbox binary missing; the OS sandbox blocks the network, confines writes to cwd (+ the temp dir), and read-denies credential stores (`~/.ssh`/`~/.aws`/…, `.env*`, `.claude/`). The policy renderers are pure and the subprocess/capability boundaries injected, so the suite asserts *what we'd run* with no real sandbox. `write_file`/`edit_file`/`execute` are gated via `interrupt_on` + an `InMemorySaver`; `brain._stream_gated` detects the post-stream interrupt (`graph.get_state(config).interrupts`), asks an injected `Approver`, and resumes with `Command(resume=…)`, bracketing the human wait in `ApprovalPause` events so `engine._consume` suspends its reply deadline (`risk.py` surfaces a shell-risk warning on the prompt). The voice TUI supplies the approver via `agent_cascade.modals.ApprovalScreen` (`y`/`a`/`n`), which can *also* be resolved hands-free by voice: while a write awaits approval, `_consume` arms `_awaiting_approval` and `engine.on_turn` routes the next final transcript to `app.submit_voice_approval` → `ApprovalScreen.try_voice`, which applies `spoken_approval.spoken_decision` (an unambiguous affirmative approves, anything else rejects — fail-safe; destructive `risk.py`-flagged commands ignore the spoken answer and require a keypress). Headless runs auto-deny (`_exec._deny_writes`). `--files` also turns on durable per-project memory via deepagents' `MemoryMiddleware` (`memory=["./.deepagents/AGENTS.md"]`), distinct from the in-session `InMemorySaver`, and binds one gateway-bound, sandbox-backed general-purpose subagent (deepagents' `task` tool; spec in `agent_cascade/subagents.py`, omitting `model`/`tools` so it inherits both) for delegating a focused subtask. The subagent's own `interrupt_on` mirrors `_WRITE_TOOLS`, and a delegated `write_file`/`edit_file`/`execute` surfaces at the *parent* `get_state().interrupts` (so `_pending_writes` gates it too — verified by a HITL spike, locked in `tests/test_agent_cascade_subagents.py`). Reads (incl. `grep`) stay ungated. +- **`agent_cascade/`** + `commands/agent_cascade/` — `assembly agent-cascade`: the same live terminal conversation as `assembly agent`, but **client-orchestrated** — `engine.run_cascade` wires Streaming STT → the LLM Gateway → streaming TTS itself instead of talking to the Voice Agent endpoint, mirroring what the `agent-cascade` `assembly init` template does server-side. **Sandbox-only** (streaming TTS has no prod host; guarded via `tts.session.require_available`). Reuses the agent slice's `DuplexAudio`/`AgentRenderer` and `core.client.stream_audio`/`core.llm.complete`/`tts.session.synthesize`; the three network legs are injected through `engine.CascadeDeps` (the `tts/session.py` seam) so the cascade — greeting, clause-level streaming TTS, barge-in, history window — is unit-tested against fakes with no sockets/mic/speaker. The LLM leg is a deepagents graph (`brain.py`) streamed token-by-token via `brain.build_streamer` (`graph.stream(stream_mode="messages")`): the engine buffers `SpeechDelta`s, flushes complete clauses with `text.pop_clauses` (soft-separator clauses gated by `engine._MIN_CLAUSE_CHARS`), and synthesizes each clause with **streaming TTS** (`tts.session.synthesize(on_audio=…)`) so audio starts on the first frame instead of after the whole reply. The reply runs on a throwaway producer thread feeding a `queue.Queue` the worker drains under a monotonic deadline (the wall-clock backstop that replaced `_complete_within`), and an abandoned-on-timeout graph leg's langchain `ThreadPoolExecutor` worker is detached (`_detach_executor_threads_since`) so it can't wedge interpreter exit. A `ToolNotice` surfaces the "Searching the web…" affordance and drops any unspoken preamble. Under `-v` (`debuglog.active()`) `brain._stream_graph` logs each accumulated assistant line, tool call, and tool result as it streams. **Front-end:** an interactive mic session in human mode runs a **voice-only Textual TUI** (`agent_cascade/tui.py`, `LiveAgentApp`) by default — there's no text input (you can't type to it), just a transcript + an animated voice bar tracking listening/thinking/speaking. It uses its own `banner` wordmark, `messages` widgets, and `tui_status.voicebar_markup`/`VOICE_FRAMES` — all modules that now live in `agent_cascade/`; the blocking `run_cascade` runs on a worker thread and reaches the UI through a `_TuiRenderer` (the `engine.Renderer` protocol) that hops each call onto the UI thread, and a quit calls `DuplexAudio.close` to end the mic iterator and unblock that worker. `_exec._should_use_tui` gates it: file/sample input, `--json`/`-o text`, and a non-TTY all fall back to the plain `AgentRenderer` line output. **`--files`** (on by default; `--no-files` opts out) swaps the brain's in-memory backend for a real-cwd, sandbox-capable `SandboxedShellBackend` (`aai_cli/agent_cascade/sandbox.py`): file ops behave as before (traversal-blocked `virtual_mode`), and because it implements `SandboxBackendProtocol` deepagents binds a *functional* `execute` that runs commands OS-sandboxed in the real cwd — `sandbox-exec` (SBPL) on macOS, `bwrap` on Linux, refused (never an unconfined fallback) on any other platform or with the sandbox binary missing; the OS sandbox blocks the network, confines writes to cwd (+ the temp dir), and read-denies credential stores (`~/.ssh`/`~/.aws`/…, `.env*`, `.claude/`). The policy renderers are pure and the subprocess/capability boundaries injected, so the suite asserts *what we'd run* with no real sandbox. `write_file`/`edit_file`/`execute` are gated via `interrupt_on` + an `InMemorySaver`; `brain._stream_gated` detects the post-stream interrupt (`graph.get_state(config).interrupts`), asks an injected `Approver`, and resumes with `Command(resume=…)`, bracketing the human wait in `ApprovalPause` events so `engine._consume` suspends its reply deadline (`risk.py` surfaces a shell-risk warning on the prompt). The voice TUI supplies the approver via `agent_cascade.modals.ApprovalScreen` (`y`/`a`/`n`), which can *also* be resolved hands-free by voice: while a write awaits approval, `_consume` arms `_awaiting_approval` and `engine.on_turn` routes the next final transcript to `app.submit_voice_approval` → `ApprovalScreen.try_voice`, which applies `spoken_approval.spoken_decision` (an unambiguous affirmative approves, anything else rejects — fail-safe; destructive `risk.py`-flagged commands ignore the spoken answer and require a keypress). Headless runs auto-deny (`_exec._deny_writes`). `--files` also turns on durable per-project memory via deepagents' `MemoryMiddleware` (`memory=["./.deepagents/AGENTS.md"]`), distinct from the in-session `InMemorySaver`, and binds one gateway-bound, sandbox-backed general-purpose subagent (deepagents' `task` tool; spec in `agent_cascade/subagents.py`, omitting `model`/`tools` so it inherits both) for delegating a focused subtask. The subagent's own `interrupt_on` mirrors `_WRITE_TOOLS`, and a delegated `write_file`/`edit_file`/`execute` surfaces at the *parent* `get_state().interrupts` (so `_pending_writes` gates it too — verified by a HITL spike, locked in `tests/test_agent_cascade_subagents.py`). Reads (incl. `grep`) stay ungated. - **`tts/`** + `commands/speak.py` — `assembly speak` synthesizes text to speech over the sandbox streaming-TTS WebSocket (`streaming-tts.sandbox000.…`). **Sandbox-only:** `session.is_available()` is false in production (empty `Environment.streaming_tts_host`), so the command exits 2 with a `--sandbox` hint. `session.synthesize` drives a Begin→Generate→Flush→Audio→Terminate protocol with an injectable `connect` for hermetic tests (mirrors `agent/session.py`); `audio.py` plays the PCM (default) or writes a WAV (`--out`). The single-voice default-playback path **streams**: `synthesize`'s `on_audio(chunk, sample_rate)` callback is wired to `audio.PcmPlayer.feed`, so speech starts on the first Audio frame (it opens the device lazily, since the rate is only known at Begin) instead of after the whole text — the win for a long `--url` page. `--out` (needs the full buffer) and the multi-voice dialogue path (`synthesize_dialogue` → `_output_audio` → buffered `play_pcm`) stay buffered; `synthesize` still returns the complete PCM for the summary regardless. - **`code_gen/`** — backs `--show-code` on `transcribe`/`stream`/`agent`: builds a ready-to-run Python SDK script from exactly the flags passed (no API key needed; generated code reads `ASSEMBLYAI_API_KEY`). - **`auth/`** — browser-assisted `assembly login` via AMS + **Stytch B2B OAuth discovery** (`discovery.py`, `flow.py`, `loopback.py`, `ams.py`). Not Stytch Connected Apps. diff --git a/aai_cli/commands/agent_cascade/__init__.py b/aai_cli/commands/agent_cascade/__init__.py index 8265b7b8..2b377bbf 100644 --- a/aai_cli/commands/agent_cascade/__init__.py +++ b/aai_cli/commands/agent_cascade/__init__.py @@ -62,8 +62,8 @@ def _emit_voice_list(_state: AppState, json_mode: bool) -> None: "assembly --sandbox live --mcp-config ~/.config/mcp/servers.json", ), ( - "Let the agent read and write files in the current directory", - "assembly --sandbox live --files", + "Run a conversation without filesystem access", + "assembly --sandbox live --no-files", ), ("See available voices", "assembly --sandbox live --list-voices"), ( @@ -172,9 +172,9 @@ def live( rich_help_panel=_PANEL_TOOLS, ), files: bool = typer.Option( - False, - "--files", - help="Let the agent read, write, and run code in the current directory, sandboxed (writes and runs need confirmation)", + True, + "--files/--no-files", + help="Let the agent read, write, and run code in the current directory, sandboxed (writes and runs need confirmation). Use --no-files to disable", rich_help_panel=_PANEL_TOOLS, ), device: int | None = typer.Option(None, "--device", help="Microphone device index"), diff --git a/tests/__snapshots__/test_snapshots_help_run.ambr b/tests/__snapshots__/test_snapshots_help_run.ambr index bd1cfba2..d4971549 100644 --- a/tests/__snapshots__/test_snapshots_help_run.ambr +++ b/tests/__snapshots__/test_snapshots_help_run.ambr @@ -630,11 +630,14 @@ │ streaming fields │ ╰──────────────────────────────────────────────────────────────────────────────╯ ╭─ Tools ──────────────────────────────────────────────────────────────────────╮ - │ --mcp-config FILE MCP servers config JSON ({"mcpServers": {…}}) to │ - │ add (repeatable; none load by default) │ - │ --files Let the agent read, write, and run code in the │ - │ current directory, sandboxed (writes and runs need │ - │ confirmation) │ + │ --mcp-config FILE MCP servers config JSON ({"mcpServers": │ + │ {…}}) to add (repeatable; none load by │ + │ default) │ + │ --files --no-files Let the agent read, write, and run code │ + │ in the current directory, sandboxed │ + │ (writes and runs need confirmation). Use │ + │ --no-files to disable │ + │ [default: files] │ ╰──────────────────────────────────────────────────────────────────────────────╯ Examples @@ -646,8 +649,8 @@ $ assembly --sandbox live --system-prompt "You are a terse pirate." Add your own MCP servers (none load by default) $ assembly --sandbox live --mcp-config ~/.config/mcp/servers.json - Let the agent read and write files in the current directory - $ assembly --sandbox live --files + Run a conversation without filesystem access + $ assembly --sandbox live --no-files See available voices $ assembly --sandbox live --list-voices Print equivalent Python instead of running diff --git a/tests/test_agent_cascade_files.py b/tests/test_agent_cascade_files.py index e80b2a8d..b4bc2557 100644 --- a/tests/test_agent_cascade_files.py +++ b/tests/test_agent_cascade_files.py @@ -11,6 +11,7 @@ import types import pytest +from typer.testing import CliRunner from aai_cli.agent_cascade import brain, engine from aai_cli.agent_cascade.brain import ApprovalPause, SpeechDelta @@ -19,9 +20,30 @@ from aai_cli.commands.agent_cascade import _exec from aai_cli.commands.agent_cascade._exec import run_agent_cascade from aai_cli.core import config +from aai_cli.main import app from tests._cascade_fakes import make_session from tests.test_agent_cascade_command import _opts +runner = CliRunner() + + +@pytest.mark.parametrize( + ("argv", "expected"), + [([], True), (["--no-files"], False), (["--files"], True)], +) +def test_files_flag_resolves_into_options(monkeypatch, argv, expected): + # Filesystem access is on by default: omitting the flag yields files=True, and --no-files + # opts out. Pinned at the argv->options seam so the True default isn't a silent mutation. + captured = {} + + def fake_run(opts, state, *, json_mode): + captured["opts"] = opts + + monkeypatch.setattr(_exec, "run_agent_cascade", fake_run) + result = runner.invoke(app, ["live", *argv]) + assert result.exit_code == 0 + assert captured["opts"].files is expected + def test_deny_writes_always_rejects(): # The non-interactive approver declines every write (no channel to confirm one).