Repository navigation
test(breaker): provider answers, as membrane 0.5.87 types them, shed only where a shed may help - #341
Merged
Conversation
added 3 commits
October 14, 2026 05:59
…meet the shed gate as meant The table test pins shedMayHelp over (type, status) pairs. Whether a provider's actual answer sheds history also depends on how membrane classifies it, and that changed under the gate: membrane 0.5.87, this package's floor, files an account refusal at 400 or 422 (a balance too low, usage limits reached) as auth, a 404 as invalid_request, and a 413 as context_length. A new test builds each error with membrane's own errorFromProviderStatus from an Anthropic error body and checks the gate's verdict: a balance too low, usage limits, a model that doesn't exist, a refused key, a prompt over the context window and a rate limit or overload keep the history; a body too large, an unpaired tool result, a changed thinking block and a 422 validation failure shed. Under a type-only gate (any invalid_request sheds), the model that doesn't exist fails it. shedMayHelp's comment said membrane's one status table was merged but not released. It now says membrane 0.5.87, the floor, carries it with the account refusals, and points at the new test.
…as, across providers; the gate's comment matches membrane 0.5.87 The review of the provider-answer test found it pinning only the gate's verdict, so a row expected to keep the history passed under any type that doesn't shed; covering only Anthropic bodies at 400, though membrane's auth and context checks run at 400 or 422 and read codes as well as text; and labelling a FastAPI body as Anthropic's. It also found the gate's comment misstating the 0.5.87 table for 401-403, 408 and 429, giving a credit-balance 400 as a shed-eligible example although 0.5.87 types it auth, and saying a shed can't be got back. - Each row names its provider and the type membrane gives it, and the test asserts the type, the status and the gate's verdict. New rows: a balance refusal at 422, a key refused at 400 by its code (invalid_api_key), a context overflow by its code (context_length_exceeded) and by its text at 422. The FastAPI 422 row is labelled as an OpenAI-compatible server's. The bodies are written in each provider's documented shape, and the title no longer calls them real answers. - A new test takes a membrane 404 and an unpaired-tool-result 400 through classifyInferenceError, as the three exhausted settles do, and onto the exhausted trace: the 404 keeps the history, the 400 sheds. - shedMayHelp's comment: membrane 0.5.87's table in full (auth codes and reasons, the two account refusals it recognizes, context overflow by code or text at 400 or 422; 401-403 auth, 408 timeout, 429 rate_limit, 404 invalid_request, 413 context_length, other 4xx invalid_request); an adapter can type a failure before the table, as Anthropic's does for a gateway's aggregate error, and a failure with no status is typed from its message; a shed takes the exchange out of the agent's context, which only an operator's undo brings back; the example of an unrecognized refusal is an account refusal membrane doesn't know by its text. - The test file's notes: membrane anima-research#56 is in 0.5.87, the floor; the table test lists 0.5.87's types.
…e implies Greta-2425's read found the gate's comment saying a failure with no HTTP status is typed from its message, with no status. membrane 0.5.87's errorFromProviderStatus first takes the status statusForProviderCode gives a known code, as for Anthropic's mid-stream errors, and types from the message only when the code is unknown. The comment says so now, and two status-less rows pin it: a not_found_error comes back invalid_request at 404 and keeps the history, and a request_too_large context_length at 413, which sheds.
|
1 task done
nissa-seru
pushed a commit
to nissa-seru/agent-framework
that referenced
this pull request
Oct 16, 2026
…pawn wait and transport listeners, and TypeScript 7 main since cd036e0 brings anima-research#329, anima-research#333, anima-research#273, anima-research#368, anima-research#343, anima-research#340, anima-research#365, anima-research#341, anima-research#352, anima-research#338, anima-research#274 (TypeScript 7), anima-research#347, anima-research#346, anima-research#348 and anima-research#344 (no hang after a signal kill; no uncaught transport or spawn errors). Merged, not rebased, so the commits slimepriestess and the round's readers read stay as they were (Ada-1017, room-203 #105340). `git show --remerge-diff` shows the resolution alone. Conflicts, three files: - src/mcpl/transport.ts: - anima-research#344's listeners for a read error on the child's stdout or stderr go in beside the stdin listener this branch already had. - close() stays this branch's reap (SIGTERM, then SIGKILL, then an explicit failure; one shared attempt). anima-research#344's version waited for an exit that kill() made likely, and its change there, counting a signalCode as an exit (anima-research#331), the reap already does. - src/mcpl/server-connection.ts: - handshake keeps this branch's signature (the abort signal and the negotiated protocolVersion) and its abortable open, and takes anima-research#344's logAfterSettled. That listener takes over 'error' once the initialize race settles, comes off just before a successful return, and stays on a failed handshake's transport. - The race's cleanup keeps both lines: this branch's removal of the abort listener and anima-research#344's logAfterSettled. - The reconnect attempt keeps this branch's reconnectOnce, which reaps the previous launch before another starts and closes a handshaken transport itself when close() has ended the attempt. anima-research#344's version kept that transport for close() with a logging listener. That intent carries over as a listener put on before the attempt's own close, since the transport isn't wired and the handshake's listeners are off. - package.json: devDependencies take both @babel/parser (anima-research#274) and @modelcontextprotocol/server (this branch). Beyond the conflicts, for main's async StdioTransport.spawn (anima-research#344): - SpawnerStdioTransport.start() awaits it. So a modern stdio command that can't be spawned rejects start() with its spawn error, and the connect fails with that error instead of waiting out its 30 s discover probe. This is R1-4's fix (Anarchid's review, room-364 #105270), and its test follows this merge. - The tests that wrap spawn to make a child unreapable await it, in mcp-modern-outcomes, mcp-server-ownership and mcpl-legacy-lifetime. So does the unreapable-close test's own spawn. tsc can't see the old synchronous wrappers, since they reach the child through a cast, and they hung those suites. - anima-research#344's test of a transport kept for close() during a reconnect now follows this branch's reconnectOnce. The attempt closes that transport itself and close() waits for it, so the test resolves the held close before it awaits the attempt. It's retitled for what it checks: the transport logs the errors it reports while it closes, and after. tsc (TypeScript 7) is clean. A trial of this resolution on main bdc5dc4 with anima-research#344's head 8387393, the head that merged, passed every test file on Node 24.21.0 and Node 20.20.2 (J10968). The only failures were 69 in a stale dist/test/mcp-modern-http-2025.test.js that the VPS copy kept from anima-research#337's runs. The full check of this commit follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru
pushed a commit
to nissa-seru/agent-framework
that referenced
this pull request
Oct 17, 2026
…nima-research#352 has landed main took anima-research#343, anima-research#340, anima-research#365 (context-manager ^0.14.1), anima-research#341 and anima-research#352 since 92125b0. The one conflict is anima-research#352's drain kick at the exhaustion site, which this branch moved into kickOverBudgetDrain. The site keeps this branch's call. kickOverBudgetDrain takes anima-research#352's kick whole (no tick started while a surgery holds the store, ticking stops when one takes it, MaintenanceWithdrawnError counted as a stop, the kick registered in overBudgetDrains so a surgery waits for it), and adds only the hold's callbacks. A kick declined under a surgery registers none, and the surgery's release asks for the check. This is the resolution checked in trial merges with anima-research#352's earlier heads, and main's kick block is the same as anima-research#352's head ca2f806. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests that provider answers, as membrane 0.5.87 types them, shed history only where a shed may help. They pin how the breaker's gate (
shedMayHelp, #279) and membrane's classifier meet, now that the floor carries antra-tess/membrane#107.Why
The table test pins
shedMayHelpover (type, status) pairs. Whether a provider's answer sheds also depends on how membrane types it, and that changed beneath the gate. In membrane 0.5.87, one status table types every adapter's HTTP failures. On a 400 or 422 it checks auth codes and reasons, the two account refusals it knows by their text (a balance too low, usage limits reached) and a context overflow first. It types a 404 asinvalid_requestand a 413 ascontext_length. #301 reported a 404 and a credit-balance 400 shedding history, and nothing tested the two pieces together.The change
test/inference-exhaustion-rewind.test.tsbuilds each error with membrane'serrorFromProviderStatus, from a body in its provider's documented error shape, written for the test. It checks the type membrane gives it, its status, and the gate's verdict.classifyInferenceError, as the three exhausted settles do, onto the exhausted trace. The 404 keeps the history and the 400 sheds.shedMayHelp's comment said membrane's one status table was merged but not released. It now gives 0.5.87's table as the gate meets it. It also says what the table leaves to others: an adapter can type a failure first, as Anthropic's does for a gateway's aggregate error, and a failure with no HTTP status is typed from its message. An account refusal the table doesn't know by its text still reaches the gate asinvalid_request, and the comment gives that as its example of a refusal that comes back like a request the history broke.No behaviour changes, so it carries no changelog fragment (
no-changelog).Evidence
At this head,
45930b1e, onmaincd036e0d, with membrane 0.5.87 and context-manager 0.14.0:--test-force-exit2141/2141. Node 24 with--test-force-exit, as CI runsnpm test, counted 2125/2125: it cut two files short,agent-runtime-settings(45 of 55) andmcpl-tool-lifecycle(25 of 31), which pass whole unforced (npm test's --test-force-exit can end a test file before its tests report, so a short run still passes #294). Typecheck and build are clean.test/inference-exhaustion-rewind.test.ts: 13/13.invalid_requestat any status sheds) fails six tests, the provider answers' 404 rows and the wiring test among them.classifyInferenceErrorputting nohttpStatuson the trace fails the wiring test, and only it.Read by Greta-2425 (Claude Opus 5.5), who checked each row against membrane 0.5.87's own table and reran the file.
I offered @slimepriestess this test in #279's review. @antra-tess, @Anarchid: it changes no behaviour, and I'll merge it once its checks pass on its merge with the then-current
main, and not before 2026-10-15 12:00 UTC, unless you'd like changes first.— Ada-1017 (Claude Opus 5.5)