Repository navigation
fix(mcpl): an inference/request result names the model membrane says served it - #484
Merged
nissa-seru merged 1 commit intoOct 18, 2026
Merged
Conversation
…served it MCPL SPEC §11.3 defines the result's `model` as "Actual model used", and §11.5's guidance for hosts includes logging it for verification and cost attribution (anima-research/mcpl 761cb48, SPEC.md:1282, :1316). The router filled it, in both modes, with the model its routing policy resolved: an alias such as `claude-haiku-4-5`, or `default` with no policy. membrane names the model that served the request in `details.model.actual`, which can differ where routing or fallback applies (anima-research#414, from Anarchid's review of anima-research#380). - resultFrom reports `details.model.actual` as `model`, through actualModelOf, which treats an empty value as none. membrane's type says it always names one, but its anthropic and bedrock adapters pass the provider's `response.model` through as it is (membrane 0.5.87, src/providers/anthropic.ts:852, bedrock.ts:959). When membrane names none, the result gives the routed model, so `model` stays the string MCPL requires. - mcpl:inference_complete keeps `model` as the routed model, as mcpl:inference_start has it, so the two pair on a field whose meaning doesn't shift, and adds `actualModel`, the value the result reports. It leaves `actualModel` out when membrane named none, rather than name the routed model as served. membrane names none only from complete(), through an adapter that passes the provider's absent or empty model through; its streaming paths, and its other adapters for an absent model, name the requested model themselves, and there actualModel is the routed model. mcpl:inference_start (which knows only the routed model) and mcpl:inference_fail (which answers no result) are unchanged. - test/helpers/mock-membrane.ts: createMockResponse sets `details.model`, which membrane's NormalizedResponse type requires and the router now reads, as it already set `details.stop`. No test reads it through the mock today. - test/mcpl-inference-router.test.ts: the scripted adapter takes the model the provider served (`scripted-model` unless given), and ask() takes a routing policy. In both tool modes: a streamed result names the served model, and the start trace still names the routed one; a whole result names it; the completion trace carries it as actualModel, streamed and whole; and a response whose provider names no model, under a policy whose default is `house-summarizer`, gets that routed model with no actualModel in its trace (a control, which a fixed `default` fallback fails). The stand-in for the stop reasons membrane can't play gets `details.model` too. The whole-mode test that pinned `model: 'default'` was a control; it now pins the model that served the request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Required label not found on this PR. |
1 task done
nissa-seru
pushed a commit
to nissa-seru/agent-framework
that referenced
this pull request
Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: an MCPL server's
inference/requestresult now names the model membrane says served the request, which is what SPEC §11.3 definesmodelas ("Actual model used"). It named the model the host's routing chose. Themcpl:inference_completetrace adds the served model asactualModel, beside the routed one inmodel.Problem
!Fixes #414. On
main(9c1e7577), the router fills the result'smodelin both modes with the model its routing policy resolves (resolveModel): a model the policy names, ordefaultwithout a policy. membrane names the model that served the request indetails.model.actual, which can differ from the requested one where routing or fallback applies. So a server was told the routed model served its request, and the completion trace, which carries the usage, named only the routed model. §11.3 defines the result'smodelas "Actual model used" (anima-research/mcpl761cb48,SPEC.md:1282), and the spec's own example shows the gap: §11.5's illustrative policy routes toclaude-haiku-4-5(:1323), and §11.3's result reportsclaude-haiku-4-5-20251001(:1269). §11.5's guidance for hosts includes loggingresult.modelfor verification and cost attribution (:1316).Changes
src/mcpl/inference-router.ts:resultFromreportsdetails.model.actualasmodel, throughactualModelOf, which treats an empty value as none. When membrane names none, the result gives the routed model, somodelstays the string MCPL requires. membrane's type says it always names one, but its Anthropic and Bedrock adapters pass the provider'sresponse.modelthrough as they get it (membrane 0.5.87,src/providers/anthropic.ts:852andsrc/providers/bedrock.ts:959).mcpl:inference_completekeepsmodelas the routed model, asmcpl:inference_starthas it, and addsactualModel, the model the result reports. It leavesactualModelout when membrane names none, rather than name the routed model as served. membrane names none only fromcomplete(), when the provider's response gives no model and the adapter passes that through, as its Anthropic and Bedrock adapters do. On its streaming paths (src/membrane.ts:1699,:2890and:4242), and through its other adapters, membrane names the requested model itself when the provider names none. Those adapters pass an empty name through (response.model ?? requestedModel, as atsrc/providers/openai.ts:646), and the router counts it as none. Where membrane names the requested model,actualModelis the routed model, as the result'smodelis, and nothing marks that no provider named it.mcpl:inference_start, which can know only the routed model, andmcpl:inference_failare unchanged (see Out of scope).createMockResponseintest/helpers/mock-membrane.tssetsdetails.model, which membrane'sNormalizedResponsetype requires and the router now reads, as it already setsdetails.stop. No test reads it through the mock today.What callers will notice:
inference/requestgets the served model as the result'smodel, which can differ from the model the host's routing names. On 2026-10-17, GitHub code search forinference/requestin anima-research and antra-tess finds the spec's documents, hosts that answer it (agent-framework, mcpl-cc-bridge, mcpl-harness, rlm), the two type packages, whose result types keepmodela string (mcpl-core-tssrc/methods.ts:312, mcpl-coresrc/methods.rs:294), and test servers that send one. The one other server it finds, tavern-mcpl, says it sends none (src/server.ts:433).actualModelonmcpl:inference_complete, andmodelthere still the routed one. We found no reader of themcpl:inference_*traces outside the router: on 2026-10-17, GitHub code search formcpl:inference_completein anima-research and antra-tess finds onlysrc/mcpl/inference-router.ts, and connectome-host'smain(16539782) names none of them.What each part is for
actualModelOfand the result'smodelare the fix itself. One helper holds the rule that an empty value counts as none, so the result and the trace can't disagree about it.modelthe string MCPL requires when membrane names none. Without it, such a result would carry an empty or missingmodel.actualModelon the completion trace, besidemodel: §11.5's audit and cost attribution want the served model, and this is the trace that carries the usage. Changingmodelitself would have shifted its meaning betweenmcpl:inference_startandmcpl:inference_complete.actualModelout when membrane names none keeps the trace from naming a model as served when nothing named one. Where membrane fills in the requested model itself, the router can't tell that apart, and the trace names it.details.modelin the shared mock and in the stop-reason stand-in: membrane's type requires it and the router now reads it, as fix(mcpl): answer each way a server's inference ends; a failure is the error, a completion its real result #380 gave the mockdetails.stop. The stand-in's tests need it, and no test reads the mock's yet.mcpl:inference_fail, and membrane's per-round roster onmcpl:inference_complete(see Out of scope).Tests
At
4b2119b5, onmain9c1e7577, on Linux, Node 24.22.0, from a fresh install with no lockfile (membrane 0.5.87, context-manager 0.14.1, chronicle 0.4.0). The pull request's CI runs the whole suite on Node 20 and 24.test/mcpl-inference-router.test.ts: 47 of 47 pass. The router runs on a real membrane with a scripted provider adapter that servesscripted-modelwhile the router routes todefault. In both of membrane's tool modes, native and XML:scripted-model, and the start trace still namesdefault;scripted-model;actualModel: 'scripted-model'besidemodel: 'default', streamed and whole;house-summarizer, getshouse-summarizeras the result'smodel, and noactualModelin the trace. This one is a control. It passes before the change too, and fails once the fallback, or the trace's omission, is taken out, or the fallback is a fixeddefault.main's router (9c1e7577) under these tests, 8 of 47 fail, the four tests in each tool mode that check the served model in a result or a trace. The control passes.defaultas the fallback;actualModel, with it whatever membrane gave, or with the result'smodelasactualModel;modelas the served one;details.model.Not verified
Out of scope
mcpl:inference_failnames only the routed model, as onmain. Some failures come after membrane returned a response, when the router refuses how it ended: a tool call, a refusal, a tool-call block left open, tool-call markup, a provider'serrorfinish, or a stop reason MCPL has nofinishReasonfor. That response names a served model and carries usage, and the fail trace reports neither. MCPL inference/request reports the routed model alias, not the model that served the request #414 is about what a result names, and a failure sends the server no result.actualis the model of the last round, and itsModelInfonotes that a routed turn can change models between rounds (src/types/response.ts:157–180in 0.5.87). A stream resumes for another round when a stop sequence lands inside an XML block the model opened (src/membrane.ts:1351and:3849). The result'smodeland the trace'sactualModelthen name the last round's model, while the trace's usage sums every round. membrane'sdetails.model.perRoundlists each round's model and usage. The result can name only one model, and the trace doesn't carry the roster.src/providers/bedrock.ts:907in 0.5.87), the OpenAI-compatible one (openai-compatible.ts:336,:606) and the OpenAI completions one (openai-completions.ts:323,:575). A streamed result through them names the requested model. That's membrane's to change.Written by Effie-2756 (Claude Opus 5.5) using the Rookery's Pi tooling, on a design Ada-1017 (Claude Opus 5.5) approved. Esther-2584 (Claude Opus 5.5) reviewed the change.