Skip to content

feat(mcp)!: Add an in-process tool server - #1170

Open
JeanMertz wants to merge 22 commits into
pr6-retry-on-resetfrom
anthropic-claude-code
Open

JeanMertz wants to merge 22 commits into
pr6-retry-on-resetfrom
anthropic-claude-code

Conversation

@JeanMertz

Copy link
Copy Markdown
Collaborator

JP executes local, built-in, and configured MCP tools through one
in-process MCP server. Ordinary JP queries and third-party MCP clients
use the same loopback Streamable HTTP handlers. Approvals, argument and
result editing, inquiry routing, and conversation recording remain with
the MCP Host through private channels. Tools requesting input run again
with accumulated answers; final delivery waits for recording.

Existing conversations retain their text/error storage format. MCP
resource attachments remain readable and removable, but cannot be
resolved for queries. Typed tool results preserve ordered content,
resources, annotations, structured data, and error details through Host
review. Unchanged reviews preserve that data; text edits replace the
delivered content.

Configured upstream MCP servers remain stdio-only. The HTTP endpoint
validates Host and Origin headers, supports scoped cancellation and
response resumption, and has no authentication. This implements
RFD 109 and provides the tool service required by RFD 110.

BREAKING CHANGE: MCP resource attachments cannot be resolved.

Conversations containing mcp+<server>+<scheme>:// attachments still load,
list, and print. Before querying them, remove those attachments with
jp attachment rm '<uri>' and supply their content through another
supported attachment source.

@JeanMertz
JeanMertz force-pushed the anthropic-claude-code branch from 6f7718d to 62d5f9c Compare September 13, 2026 06:21
@JeanMertz
JeanMertz added this pull request to stack #1173 September 14, 2026 17:44
`mcp+<server>+<scheme>://` attachments no longer fetch their contents.
A conversation carrying one still loads, lists, and prints, and the
attachment can still be removed, but resolving it for a query now fails
naming the attachment and the `jp attachment rm` invocation that clears
it. Every other scheme is unaffected.

Attachments also all resolve while the CLI context is still in hand,
rather than holding MCP-backed ones back until the MCP servers finish
starting. Declaration order is unchanged: each attachment still reaches
the provider as a document numbered by its position.

This drops the MCP client parameter from `jp_attachment`'s `Handler`
trait, so the base attachment crate no longer depends on `jp_mcp`.
Per [RFD 109], that dependency is what keeps `jp_mcp` from owning tool
execution: `jp_conversation` reaches `jp_mcp` through `jp_attachment`
today, and a server living under `jp_mcp` cannot sit beneath a crate
that depends on it. Plugin-based MCP attachments can be designed
separately; MCP tools and their resource results are untouched.

[RFD 109]: docs/rfd/109-in-process-jp-mcp-server.md

Signed-off-by: Jean Mertz <git@jeanmertz.com>
A tool's resolved description, its parameter schema, and the errors
raised while resolving or running one are tool-domain concerns, not
LLM ones. They move out of `jp_llm` into `jp_tool`: `ToolDefinition`,
`ToolDocs`, and `ParameterDocs` with the argument coercion, defaulting,
and validation that read a schema; `jp_tool::schema` for reading and
validating a parameter schema; and `jp_tool::Error` for the whole tool
error domain.

Per [RFD 109], `jp_mcp::server` becomes the owner of tool execution,
and it cannot depend on `jp_llm`. Lowering these contracts underneath
both is what lets that server describe and check a tool call without
pulling in an inference client. Building a schema from configuration
stays with the configuration types in `jp_llm::tool::json_schema`, and
execution stays in `jp_llm::tool` until that move.

No behaviour changes. `ToolDefinition::execute` becomes the free
function `jp_llm::tool::execute`, since the type it hung off no longer
lives in that crate. Eight `ToolError` variants that nothing
constructed are dropped, and the two MCP variants plus the template
variant carry a boxed source so `jp_tool` names neither `jp_mcp` nor
`minijinja`.

[RFD 109]: docs/rfd/109-in-process-jp-mcp-server.md

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`jp_tool::content` introduces the representation a tool result takes at
JP's internal boundaries: an ordered `Vec<ContentBlock>` of text,
resources, and input requests, alongside whether the tool failed and the
transience and trace it reported.

The shape follows MCP's, so a result arriving from an MCP server crosses
into JP without being flattened to a string first, and one JP assembles
can be handed back out. Fields MCP defines and JP does not act on
(annotations, resource titles and descriptions) are carried as data
rather than dropped. `ToolResult::to_text` flattens the whole thing for
a caller that wants the string a provider receives today.

An input request describes its answer with JSON Schema rather than JP's
closed `AnswerType`, which is what lets an MCP elicitation request and a
local tool's question become the same thing. Secrecy stays a typed field
instead of a schema keyword: a consumer that rewrites the schema for a
provider must not be able to drop the rule that the answer stays off
disk. `From<Outcome>` converts results from tools speaking the existing
protocol.

Nothing calls these yet. They are the contracts [RFD 109]'s execution
service is built against, introduced first so that phase moves execution
rather than also inventing the shape it speaks. Existing tools, stored
conversations, and rendering are untouched; the wider typed-content
migration remains [RFD 058]'s.

[RFD 058]: docs/rfd/058-typed-content-blocks-for-tool-responses.md
[RFD 109]: docs/rfd/109-in-process-jp-mcp-server.md

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Keep ordinary queries working while moving tool resolution, command
execution, upstream dispatch, and the built-in registry into
`jp_mcp::server`. The coordinator retains conversation ownership and
inquiry routing. The default `client` feature stays usable without the
server's execution dependencies.

Add the Phase 2 service from RFD 109: private, single-use Host replies
control admission, execution release, input, result review, and final
recording. Questions re-run tools with accumulated answers. Edited
arguments are revalidated, formatter execution respects approval and
visibility, and final results wait for recording acknowledgement.

Calls have separate identities and cancellation scopes. Host loss stops
work, progress cannot block required interactions, and shutdown drains
calls before closing upstream services. Dropping a result receiver does
not retry or cancel execution. HTTP transport and CLI adoption of this
service remain Phase 3 work.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
JP's ordinary query path uses its in-process MCP server for local,
built-in, and upstream tools. Approvals, inquiry routing, result editing,
and conversation ownership stay with the MCP Host. One MCP call spans
input requests and tool re-execution, and final delivery waits for the
Host to flush the recorded response.

Upstream stdio tools receive trusted context, options, and accumulated
answers through the `computer.jp/*` metadata keys. Recognize single-text
`Outcome` envelopes without flattening mixed native content or losing
unedited result metadata. Keep the existing text/error projection for
conversation storage.

Complete RFD 109 Phase 3 with a loopback Streamable HTTP endpoint, Host
and Origin checks, scoped cancellation, and shutdown cleanup. The Host
connection disables proxies, redirects, and transparent session
reinitialization. Authentication and the third-party-client readiness
work remain separate; no Anthropic subscription flow is enabled here.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
External MCP clients can read Host-supplied tool metadata through
`tools/list`, including opaque result-size hints. Empty metadata stays
omitted for ordinary JP calls, and incoming call metadata cannot change
descriptions or execution policy.

Complete RFD 109 Phase 4 with an independent JSON-RPC/SSE client and a
scripted MCP Host. Verify inquiry re-execution, argument and result edits,
recording barriers, concurrent callers, scoped cancellation, and Host
loss. Exercise response resumption through `Last-Event-ID` without a
second execution, and preserve large results and native upstream content
through the HTTP path.

No Claude Code process, subscription credentials, or transcript conversion
is involved. Agent-specific integration remains RFD 110 work.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Tool results retain ordered content, resources, annotations, structured
data, and error details through Host review and recording. Unchanged
reviews preserve that data; text edits replace the delivered content.
Existing conversation files and terminal output keep their text/error
projection.

Use typed execution, formatter, and recording errors, validated question
IDs, and distinct Host correlation keys. Retain original error sources
until rendering diagnostics, and preserve question context when decoding
legacy tool outcomes.

Remove Reqwest 0.13 and use the workspace's 0.12 client through rmcp's
HTTP transport interface. Session handling and SSE resumption remain in
rmcp. Mark RFD 109 Implemented and clear its satisfied dependency from
RFD 110.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
@JeanMertz
JeanMertz force-pushed the anthropic-claude-code branch from ce78607 to f8fa6ad Compare September 15, 2026 12:41
@JeanMertz
JeanMertz removed this pull request from stack #1173 September 15, 2026 12:41
@JeanMertz
JeanMertz changed the base branch from main to pr6-retry-on-reset September 15, 2026 12:41
Signed-off-by: Jean Mertz <git@jeanmertz.com>
JP speaks the Agent Client Protocol over its own JSON-RPC client instead
of `agent-client-protocol`. The SDK brought a second async runtime into
a Tokio-only binary: `async-io`, `async-process`, `blocking`, `polling`
and their dependencies were linked but never run, because JP spawns the
adapter itself and hands ACP the byte streams. Dropping the SDK and its
schema crate removes 26 crates from the dependency graph, all of which
`cargo vet` would otherwise have to audit.

`rpc` carries the protocol: a `Request` trait pairing each method with
its response type, a peer that correlates replies by id, and a driver
running the reader, writer and foreground sequence. `schema` declares
the ~25 fields JP reads or writes as plain serde structs; unknown fields
are ignored, so a field the adapter adds later costs nothing.

Recorded adapter traffic is what keeps those hand-written types honest.
`RECORD=1 cargo test -p jp_llm cache_reconstruction` captures a real
session to `tests/fixtures/acp/live.jsonl`, and every later run replays
it through the same driver, asserting the decoded events and usage. A
recording that is missing fails the test rather than skipping it, and
the account email, organization and working directory are stripped as
the recording is written.

Tool execution gains a matching shutdown order: admission stops, then
the client closes its MCP session, and only then does the listener go.
Closing them the other way round leaves the session open on a server
that can no longer hear the DELETE.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
The two tests needed a container runtime on the machine, so they carried
`#[ignore]` and never ran anywhere: not on a contributor's machine, not
on Linux CI, not on Windows. An ignored test reports nothing while
looking like coverage.

What they covered was real — that an install script reaches the built
image, and that a `:ro` mount is honoured by the runtime rather than
just requested. Neither can be established without a runtime to ask, and
mocking the runtime would only prove JP passes the flag, which is what
the remaining tests already assert.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Four intra-doc links named items rustdoc cannot see. `MockExecutor` and
`cassette::Recorded` are both `#[cfg(test)]`, so they do not exist in a
documentation build; `jp_test::mock::Vcr` is a dev-dependency, outside
the library's dependency graph. Each is now plain code formatting, which
still tells a reader where to look without asking rustdoc to resolve it.

`Delivery` was a stale name. It was a type this adapter briefly returned
to the coordinator, and it became `ExecutorResult` before the code
landed; the module doc kept the old spelling.

The qualification document pointed at `docs/README/providers.md` for
setup, but the site excludes `README/**` from its build, so the link
resolved to nothing published. The steps a qualification run needs are
short enough to state where they are followed, so they now appear under
their own heading rather than behind a reference.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`jp_attachment_agentic_shepherd` took `serde_json` with the workspace's
`default-features = false` and enabled no feature of its own, so nothing
in its dependency graph turned `std` on. serde_json 1.0.151 rejects that
combination with a `compile_error!` where earlier versions built, and
the lock moved to 1.0.151 in the previous commit.

Five other crates declare the dependency the same way and keep building,
because something else they depend on enables `std` for them. This one
has no such neighbour, so it says what it needs.

The failure only appears when the crate is built alone: a workspace
build unifies features across every member, which hides it. `cargo
shear` expands each crate on its own, which is how CI found it.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
The shutdown test was revived under `mcp_executor_shutdown_tests.rs`,
adapted to the current fixture and declared from `mcp_executor_tests.rs`.
The original file was left behind: nothing declares it, so it compiled
nowhere and its assertions ran nowhere.

The behaviour it covered is unchanged and still covered — that a client
can close its MCP session while the endpoint is shutting down, because
admission stops before the listener does.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`JobObject` in process-wrap 9 is a unit struct with no inherent `impl`,
so `JobObject::new()` names a function that does not exist and the
Windows build fails to compile. The wrapper beside it on the previous
line, `KillOnDrop`, is written the same way and always was.

The two other wrappers this function uses build on every platform, which
is why the error only appears on Windows: `#[cfg(windows)]` keeps the
line out of a macOS or Linux build entirely, so neither a local check
nor the Linux CI job ever type-checks it.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
A tool call whose approval prompt stays open now survives, however long
the answer takes. The Claude adapter aborted it after five minutes of
silence and handed the model a failure for a question nobody had
answered yet, which the model then read as a transient fault and
retried:

    MCP server "jp" tool "github_pr_review_add_comment" sent no
    response or progress for 300s; aborting.

Both of the adapter's per-call limits applied: an idle timer that
watches for silence, and a wall-clock ceiling on the whole call. JP now
turns the idle check off and raises the ceiling past any prompt a person
would leave open.

Progress notifications are the adapter's suggested remedy and do not fit
here. Its timers measure wall-clock time whether or not JP is scheduled
to run, so a laptop closed for an afternoon looks exactly like a hung
server, and the heartbeat that would have reset the timer is never sent.
Turning the check off is what survives suspension.

Deciding when to stop waiting stays with JP: the interrupt handler
cancels a call the user abandons, and the MCP Host holds the reply the
execution service is parked on until then.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
`Utf8TempDir` and `ReplyEditMode` are used only by
`remembered_denial_does_not_run_http_argument_formatter`, which carries
`#[cfg(unix)]` because its fixture spawns a shell. Their imports had no
such gate, so a Windows build compiled them with nothing to use them and
failed on `-D unused-imports`.

The other imports in the file already pair with their test this way. A
platform gate on a test has to reach the imports it alone needs, and the
macOS and Linux builds cannot report the mismatch: the item exists there,
so the import is used.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant