From 3968530d55b39bf161ae24104ab4ad1e4fa096be Mon Sep 17 00:00:00 2001 From: Hiroshi Shinaoka Date: Thu, 29 Oct 2026 20:11:32 +0900 Subject: [PATCH 1/3] docs: guide to running a two-level agent fleet Add docs/agent-fleet with a figure and a step-by-step guide to the maintainer's setup: one orchestrator Claude Code session (also reachable from a phone) driving herdr agents on several machines over Tailscale, the orchestrator operating rules, the recovery state layout, the structure of the maintainer's rule set, and security notes. Link it from the README. Co-Authored-By: Claude Opus 5.5 --- README.md | 5 + docs/agent-fleet/README.md | 251 +++++++++++++++++++++++++++++++ docs/agent-fleet/agent-fleet.svg | 125 +++++++++++++++ 3 files changed, 381 insertions(+) create mode 100644 docs/agent-fleet/README.md create mode 100644 docs/agent-fleet/agent-fleet.svg diff --git a/README.md b/README.md index 0f839d1..a947c1c 100644 --- a/README.md +++ b/README.md @@ -33,3 +33,8 @@ skills/ Skills are thin trigger and workflow layers; the rule files are the source of truth. + +## Guides + +- [`docs/agent-fleet`](docs/agent-fleet/README.md): how to run many coding + agents on several machines from one orchestrator session. diff --git a/docs/agent-fleet/README.md b/docs/agent-fleet/README.md new file mode 100644 index 0000000..8ba56a1 --- /dev/null +++ b/docs/agent-fleet/README.md @@ -0,0 +1,251 @@ +# Running a Two-Level Agent Fleet + +This page describes how the maintainer runs many coding agents at once, and +how to reproduce the setup. It is a guide, not a rule. Adapt it to your own +machines and budget. + +![Two-level agent fleet](agent-fleet.svg) + +## Architecture + +The fleet has three levels. + +- **Level 0: the human.** The maintainer decides, approves releases and gives + instructions in plain language. They talk only to the orchestrator. The + orchestrator session can also be continued from Claude Code on a phone, so + the maintainer can steer the fleet while away from the laptop. It is the same + session, not a second one. +- **Level 1: the orchestrator.** One Claude Code session on a laptop. It runs + rounds on a cron schedule, reads each agent's screen, dispatches tasks by + prompt, reviews and merges pull requests, and escalates decisions it should + not take. +- **Level 2: the machines.** Each remote machine runs + herdr, a terminal multiplexer for + coding agents, with several agents in its panes. Agents are Claude Code + sessions (Opus or Sonnet) or pi agents on cheaper models such as DeepSeek. + +Two shared services tie the levels together. + +- **GitHub.** Agents push branches and open pull requests. Issues are assigned + to the maintainer when work starts. Only the orchestrator merges. +- **A private state repository.** A small git repository holds the recovery + state, so a new orchestrator session can take over at any time. See + [Recovery state](#recovery-state). + +Each machine has a role. In the maintainer's fleet these are a quiet benchmark +host, an implementation host with a GPU, and a many-core scaling host. Adding a +machine adds a whole team of agents. + +## Reproduce the setup + +### 1. Network + +Install [Tailscale](https://tailscale.com/) on the laptop and on every machine. +The machines then reach each other on a private network and need no open ports +on the public internet. + +### 2. SSH key + +Create one dedicated ed25519 key without a passphrase. The orchestrator works +unattended, so it cannot type a passphrase. + +```sh +ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519_fleet -C fleet +``` + +Add the public key to `~/.ssh/authorized_keys` on each machine. Use this key +only for the tailnet. Give each machine a Host alias in `~/.ssh/config` on the +laptop: + +```sshconfig +Host host-a_vpn + HostName host-a.your-tailnet.ts.net + User you + IdentityFile ~/.ssh/id_ed25519_fleet + IdentitiesOnly yes +``` + +Check that `ssh host-a_vpn true` works without any prompt. + +### 3. herdr + +Install herdr on each machine and start it there. On the laptop, register each +machine once: + +```sh +herdr machine add host-a_vpn +herdr machine list +``` + +Adding and removing machines is the human's job. The orchestrator only uses +machines that are already registered. + +### 4. Orchestrator commands + +The orchestrator drives every machine with `herdr --machine