Skip to content

feat(config, llm, cli): Add tool call fan-out - #1182

Open
JeanMertz wants to merge 2 commits into
mainfrom
fs-tools-multi
Open

JeanMertz wants to merge 2 commits into
mainfrom
fs-tools-multi

Conversation

@JeanMertz

Copy link
Copy Markdown
Collaborator

A tool can now declare fan_out in its configuration, after which a single call to it carries several independent operations in an ops array. JP runs each one as its own invocation and folds the results into the one response the provider is waiting for. An assistant that already knows it wants fourteen file reads issues one call instead of fourteen, each of which would otherwise re-send the whole accumulated context.

fan_out = true runs every operation at once and reports each failure on its own. The table form takes concurrency, which set to 1 runs them in order, and on_error = "stop", which starts nothing further once one has failed. The folded result frames each operation with its position and names the ones that never ran, so a model that asked for five and reads three does not assume the other two succeeded.

Nothing changes until a configuration sets the key: a tool without fan_out keeps its schema, its arguments, and its response unchanged, and no tool ships with it enabled. Terminal output stays identical either way, because a call carrying three operations renders three headers and prompts three times, so what the user approves is what they were shown. The envelope wraps whatever schema a tool declared, so it reaches MCP servers JP does not control, which no per-tool schema change could do.

A tool can now declare `fan_out` in its configuration, after which a
single call to it carries several independent operations in an `ops`
array. JP runs each one as its own invocation and folds the results
into the one response the provider is waiting for. An assistant that
already knows it wants fourteen file reads issues one call instead of
fourteen, each of which would otherwise re-send the whole accumulated
context.

`fan_out = true` runs every operation at once and reports each failure
on its own. The table form takes `concurrency`, which set to `1` runs
them in order, and `on_error = "stop"`, which starts nothing further
once one has failed. The folded result frames each operation with its
position and names the ones that never ran, so a model that asked for
five and reads three does not assume the other two succeeded.

Nothing changes until a configuration sets the key: a tool without
`fan_out` keeps its schema, its arguments, and its response unchanged,
and no tool ships with it enabled. Terminal output stays identical
either way, because a call carrying three operations renders three
headers and prompts three times, so what the user approves is what
they were shown. The envelope wraps whatever schema a tool declared,
so it reaches MCP servers JP does not control, which no per-tool
schema change could do.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
A tool configured for fan-out no longer breaks the schema it is shown.
A `$defs` or `definitions` block now moves to the envelope's root, so
every `#/$defs/Name` reference still resolves: nesting the block under
`properties.ops.items` left each reference pointing at a root that no
longer held it, which reached Ollama as a property with no type at all
and other providers as a dangling pointer. MCP tools carry the server's
document verbatim, so this struck the population fan-out exists for.

Two failures stop being hidden from the assistant. A tool without
`fan_out` whose argument formatter fails once again answers with an
error rather than a success carrying error text, which Anthropic was
receiving as `is_error: false` and replay was rendering in the success
style. And `on_error = "stop"` now reads the tool's own outcome instead
of the response left after result-mode policy, so a failure suppressed
by `result = "skip"` or a declined `result = "ask"` prompt stops the
operations behind it as configured.

A turn can no longer stall. When a `stop` policy ruled out every
remaining operation before any had started, nothing was spawned and no
event would ever arrive, so the execution loop waited on a channel whose
senders it held itself; completion is now checked before the receive.
Ctrl-C during a fanned-out call also reaches the cancellation menu
again: every prompt state transition keys on the operation rather than
the shared call id, so an approved operation no longer strands an
`AwaitingPermission` entry that made `is_prompting` true for the rest of
the turn. Restarting and escalating now stop releasing queued
operations, matching what cancelling already did.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant