Skip to content

fix(mcpl): an inference/request result names the model membrane says served it - #484

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:fix/414-mcpl-actual-model
Oct 18, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:fix/414-mcpl-actual-model

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

TL;DR: an MCPL server's inference/request result now names the model membrane says served the request, which is what SPEC §11.3 defines model as ("Actual model used"). It named the model the host's routing chose. The mcpl:inference_complete trace adds the served model as actualModel, beside the routed one in model.

Problem

!Fixes #414. On main (9c1e7577), the router fills the result's model in both modes with the model its routing policy resolves (resolveModel): a model the policy names, or default without a policy. membrane names the model that served the request in details.model.actual, which can differ from the requested one where routing or fallback applies. So a server was told the routed model served its request, and the completion trace, which carries the usage, named only the routed model. §11.3 defines the result's model as "Actual model used" (anima-research/mcpl 761cb48, SPEC.md:1282), and the spec's own example shows the gap: §11.5's illustrative policy routes to claude-haiku-4-5 (:1323), and §11.3's result reports claude-haiku-4-5-20251001 (:1269). §11.5's guidance for hosts includes logging result.model for verification and cost attribution (:1316).

Changes

  • The result, in src/mcpl/inference-router.ts: resultFrom reports details.model.actual as model, through actualModelOf, which treats an empty value as none. When membrane names none, the result gives the routed model, so model stays the string MCPL requires. membrane's type says it always names one, but its Anthropic and Bedrock adapters pass the provider's response.model through as they get it (membrane 0.5.87, src/providers/anthropic.ts:852 and src/providers/bedrock.ts:959).
  • The completion trace: mcpl:inference_complete keeps model as the routed model, as mcpl:inference_start has it, and adds actualModel, the model the result reports. It leaves actualModel out when membrane names none, rather than name the routed model as served. membrane names none only from complete(), when the provider's response gives no model and the adapter passes that through, as its Anthropic and Bedrock adapters do. On its streaming paths (src/membrane.ts:1699, :2890 and :4242), and through its other adapters, membrane names the requested model itself when the provider names none. Those adapters pass an empty name through (response.model ?? requestedModel, as at src/providers/openai.ts:646), and the router counts it as none. Where membrane names the requested model, actualModel is the routed model, as the result's model is, and nothing marks that no provider named it. mcpl:inference_start, which can know only the routed model, and mcpl:inference_fail are unchanged (see Out of scope).
  • The shared test mock: createMockResponse in test/helpers/mock-membrane.ts sets details.model, which membrane's NormalizedResponse type requires and the router now reads, as it already sets details.stop. No test reads it through the mock today.

What callers will notice:

  • An MCPL server that sends inference/request gets the served model as the result's model, which can differ from the model the host's routing names. On 2026-10-17, GitHub code search for inference/request in anima-research and antra-tess finds the spec's documents, hosts that answer it (agent-framework, mcpl-cc-bridge, mcpl-harness, rlm), the two type packages, whose result types keep model a string (mcpl-core-ts src/methods.ts:312, mcpl-core src/methods.rs:294), and test servers that send one. The one other server it finds, tavern-mcpl, says it sends none (src/server.ts:433).
  • An operator reading traces finds the served model as actualModel on mcpl:inference_complete, and model there still the routed one. We found no reader of the mcpl:inference_* traces outside the router: on 2026-10-17, GitHub code search for mcpl:inference_complete in anima-research and antra-tess finds only src/mcpl/inference-router.ts, and connectome-host's main (16539782) names none of them.

What each part is for

  • actualModelOf and the result's model are the fix itself. One helper holds the rule that an empty value counts as none, so the result and the trace can't disagree about it.
  • The fallback to the routed model keeps model the string MCPL requires when membrane names none. Without it, such a result would carry an empty or missing model.
  • actualModel on the completion trace, beside model: §11.5's audit and cost attribution want the served model, and this is the trace that carries the usage. Changing model itself would have shifted its meaning between mcpl:inference_start and mcpl:inference_complete.
  • Leaving actualModel out when membrane names none keeps the trace from naming a model as served when nothing named one. Where membrane fills in the requested model itself, the router can't tell that apart, and the trace names it.
  • details.model in the shared mock and in the stop-reason stand-in: membrane's type requires it and the router now reads it, as fix(mcpl): answer each way a server's inference ends; a failure is the error, a completion its real result #380 gave the mock details.stop. The stand-in's tests need it, and no test reads the mock's yet.
  • Cut: the served model and usage on mcpl:inference_fail, and membrane's per-round roster on mcpl:inference_complete (see Out of scope).

Tests

At 4b2119b5, on main 9c1e7577, on Linux, Node 24.22.0, from a fresh install with no lockfile (membrane 0.5.87, context-manager 0.14.1, chronicle 0.4.0). The pull request's CI runs the whole suite on Node 20 and 24.

  • The router's tests, test/mcpl-inference-router.test.ts: 47 of 47 pass. The router runs on a real membrane with a scripted provider adapter that serves scripted-model while the router routes to default. In both of membrane's tool modes, native and XML:
    • a streamed result names scripted-model, and the start trace still names default;
    • a result answered whole names scripted-model;
    • the completion trace carries actualModel: 'scripted-model' beside model: 'default', streamed and whole;
    • a provider that names no model, answered whole, under a routing policy whose default is house-summarizer, gets house-summarizer as the result's model, and no actualModel in the trace. This one is a control. It passes before the change too, and fails once the fallback, or the trace's omission, is taken out, or the fallback is a fixed default.
  • Red-first: with main's router (9c1e7577) under these tests, 8 of 47 fail, the four tests in each tool mode that check the served model in a result or a trace. The control passes.
  • Removals, at this head: each of 9 pieces, taken out or changed alone, fails at least one test:
    • the result reporting the routed model;
    • no fallback for an empty value, or a fixed default as the fallback;
    • an empty value counted as a model;
    • the completion trace without actualModel, with it whatever membrane gave, or with the result's model as actualModel;
    • the completion trace's model as the served one;
    • the stop-reason stand-in's response without details.model.
  • Typecheck and build are clean.

Not verified

  • No live provider or MCPL server took part. The tests run membrane on a scripted provider adapter, and use a stand-in for the stop reasons membrane can't play.
  • No test plays a stream whose provider names no model. What the result and the trace give there, the routed model in both, rests on reading membrane's streaming paths in 0.5.87.

Out of scope

  • mcpl:inference_fail names only the routed model, as on main. Some failures come after membrane returned a response, when the router refuses how it ended: a tool call, a refusal, a tool-call block left open, tool-call markup, a provider's error finish, or a stop reason MCPL has no finishReason for. That response names a served model and carries usage, and the fail trace reports neither. MCPL inference/request reports the routed model alias, not the model that served the request #414 is about what a result names, and a failure sends the server no result.
  • On a streamed turn of several provider rounds, membrane's actual is the model of the last round, and its ModelInfo notes that a routed turn can change models between rounds (src/types/response.ts:157–180 in 0.5.87). A stream resumes for another round when a stop sequence lands inside an XML block the model opened (src/membrane.ts:1351 and :3849). The result's model and the trace's actualModel then name the last round's model, while the trace's usage sums every round. membrane's details.model.perRound lists each round's model and usage. The result can name only one model, and the trace doesn't carry the roster.
  • Three of membrane's adapters, when they stream, give as the response's model the one they asked for, not one read from the stream: Bedrock's (src/providers/bedrock.ts:907 in 0.5.87), the OpenAI-compatible one (openai-compatible.ts:336, :606) and the OpenAI completions one (openai-completions.ts:323, :575). A streamed result through them names the requested model. That's membrane's to change.

Written by Effie-2756 (Claude Opus 5.5) using the Rookery's Pi tooling, on a design Ada-1017 (Claude Opus 5.5) approved. Esther-2584 (Claude Opus 5.5) reviewed the change.

…served it

MCPL SPEC §11.3 defines the result's `model` as "Actual model used", and
§11.5's guidance for hosts includes logging it for verification and cost
attribution (anima-research/mcpl 761cb48, SPEC.md:1282, :1316). The router
filled it, in both modes, with the model its routing policy resolved: an
alias such as `claude-haiku-4-5`, or `default` with no policy. membrane
names the model that served the request in `details.model.actual`, which
can differ where routing or fallback applies (anima-research#414, from Anarchid's review
of anima-research#380).

- resultFrom reports `details.model.actual` as `model`, through
  actualModelOf, which treats an empty value as none. membrane's type says
  it always names one, but its anthropic and bedrock adapters pass the
  provider's `response.model` through as it is (membrane 0.5.87,
  src/providers/anthropic.ts:852, bedrock.ts:959). When membrane names
  none, the result gives the routed model, so `model` stays the string
  MCPL requires.
- mcpl:inference_complete keeps `model` as the routed model, as
  mcpl:inference_start has it, so the two pair on a field whose meaning
  doesn't shift, and adds `actualModel`, the value the result reports. It
  leaves `actualModel` out when membrane named none, rather than name the
  routed model as served. membrane names none only from complete(), through
  an adapter that passes the provider's absent or empty model through; its
  streaming paths, and its other adapters for an absent model, name the
  requested model themselves, and there actualModel is the routed model.
  mcpl:inference_start (which knows only the routed model) and
  mcpl:inference_fail (which answers no result) are unchanged.
- test/helpers/mock-membrane.ts: createMockResponse sets `details.model`,
  which membrane's NormalizedResponse type requires and the router now
  reads, as it already set `details.stop`. No test reads it through the
  mock today.
- test/mcpl-inference-router.test.ts: the scripted adapter takes the model
  the provider served (`scripted-model` unless given), and ask() takes a
  routing policy. In both tool modes: a streamed result names the served
  model, and the start trace still names the routed one; a whole result
  names it; the completion trace carries it as actualModel, streamed and
  whole; and a response whose provider names no model, under a policy
  whose default is `house-summarizer`, gets that routed model with no
  actualModel in its trace (a control, which a fixed `default` fallback
  fails). The stand-in for the stop reasons membrane can't play gets
  `details.model` too. The whole-mode test that pinned `model: 'default'`
  was a control; it now pins the model that served the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 18, 2026

Copy link
Copy Markdown

Required label not found on this PR.

nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
@nissa-seru
nissa-seru merged commit d9012cb into anima-research:main Oct 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCPL inference/request reports the routed model alias, not the model that served the request

1 participant