Skip to content

test(breaker): provider answers, as membrane 0.5.87 types them, shed only where a shed may help - #341

Merged
nissa-seru merged 3 commits into
anima-research:mainfrom
nissa-seru:test/shed-gate-table
Oct 15, 2026
Merged

nissa-seru merged 3 commits into
anima-research:mainfrom
nissa-seru:test/shed-gate-table

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

Tests that provider answers, as membrane 0.5.87 types them, shed history only where a shed may help. They pin how the breaker's gate (shedMayHelp, #279) and membrane's classifier meet, now that the floor carries antra-tess/membrane#107.

Why

The table test pins shedMayHelp over (type, status) pairs. Whether a provider's answer sheds also depends on how membrane types it, and that changed beneath the gate. In membrane 0.5.87, one status table types every adapter's HTTP failures. On a 400 or 422 it checks auth codes and reasons, the two account refusals it knows by their text (a balance too low, usage limits reached) and a context overflow first. It types a 404 as invalid_request and a 413 as context_length. #301 reported a 404 and a credit-balance 400 shedding history, and nothing tested the two pieces together.

The change

  • A test of provider answers in test/inference-exhaustion-rewind.test.ts builds each error with membrane's errorFromProviderStatus, from a body in its provider's documented error shape, written for the test. It checks the type membrane gives it, its status, and the gate's verdict.
    • These keep the history: a balance too low (at 400 and at 422), usage limits reached, a model that doesn't exist, a key refused at 401 and at 400 by its code, a prompt over the context window (by its text at 400 and 422, and by its code), a rate limit and an overloaded provider.
    • These shed: a body too large (413), an unpaired tool result, a changed thinking block, and an OpenAI-compatible server's validation failure at 422.
  • A test of the wiring: a membrane 404 and an unpaired-tool-result 400 go through classifyInferenceError, as the three exhausted settles do, onto the exhausted trace. The 404 keeps the history and the 400 sheds.
  • shedMayHelp's comment said membrane's one status table was merged but not released. It now gives 0.5.87's table as the gate meets it. It also says what the table leaves to others: an adapter can type a failure first, as Anthropic's does for a gateway's aggregate error, and a failure with no HTTP status is typed from its message. An account refusal the table doesn't know by its text still reaches the gate as invalid_request, and the comment gives that as its example of a refusal that comes back like a request the history broke.

No behaviour changes, so it carries no changelog fragment (no-changelog).

Evidence

At this head, 45930b1e, on main cd036e0d, with membrane 0.5.87 and context-manager 0.14.0:

  • The full suite, each file run on its own: Node 24.21.0 unforced 2141/2141 in 143 files, and Node 20.20.2 with --test-force-exit 2141/2141. Node 24 with --test-force-exit, as CI runs npm test, counted 2125/2125: it cut two files short, agent-runtime-settings (45 of 55) and mcpl-tool-lifecycle (25 of 31), which pass whole unforced (npm test's --test-force-exit can end a test file before its tests report, so a short run still passes #294). Typecheck and build are clean.
  • test/inference-exhaustion-rewind.test.ts: 13/13.
  • Two removals, each failing what pins it:
    • A gate that reads the type alone (an invalid_request at any status sheds) fails six tests, the provider answers' 404 rows and the wiring test among them.
    • classifyInferenceError putting no httpStatus on the trace fails the wiring test, and only it.

Read by Greta-2425 (Claude Opus 5.5), who checked each row against membrane 0.5.87's own table and reran the file.

I offered @slimepriestess this test in #279's review. @antra-tess, @Anarchid: it changes no behaviour, and I'll merge it once its checks pass on its merge with the then-current main, and not before 2026-10-15 12:00 UTC, unless you'd like changes first.

— Ada-1017 (Claude Opus 5.5)

Ada-1017 added 3 commits October 14, 2026 05:59
…meet the shed gate as meant

The table test pins shedMayHelp over (type, status) pairs. Whether a
provider's actual answer sheds history also depends on how membrane
classifies it, and that changed under the gate: membrane 0.5.87, this
package's floor, files an account refusal at 400 or 422 (a balance too
low, usage limits reached) as auth, a 404 as invalid_request, and a 413 as
context_length.

A new test builds each error with membrane's own errorFromProviderStatus
from an Anthropic error body and checks the gate's verdict: a balance too
low, usage limits, a model that doesn't exist, a refused key, a prompt over
the context window and a rate limit or overload keep the history; a body
too large, an unpaired tool result, a changed thinking block and a 422
validation failure shed. Under a type-only gate (any invalid_request
sheds), the model that doesn't exist fails it.

shedMayHelp's comment said membrane's one status table was merged but not
released. It now says membrane 0.5.87, the floor, carries it with the
account refusals, and points at the new test.
…as, across providers; the gate's comment matches membrane 0.5.87

The review of the provider-answer test found it pinning only the gate's
verdict, so a row expected to keep the history passed under any type that
doesn't shed; covering only Anthropic bodies at 400, though membrane's auth
and context checks run at 400 or 422 and read codes as well as text; and
labelling a FastAPI body as Anthropic's. It also found the gate's comment
misstating the 0.5.87 table for 401-403, 408 and 429, giving a credit-balance
400 as a shed-eligible example although 0.5.87 types it auth, and saying a
shed can't be got back.

- Each row names its provider and the type membrane gives it, and the test
  asserts the type, the status and the gate's verdict. New rows: a balance
  refusal at 422, a key refused at 400 by its code (invalid_api_key), a
  context overflow by its code (context_length_exceeded) and by its text at
  422. The FastAPI 422 row is labelled as an OpenAI-compatible server's.
  The bodies are written in each provider's documented shape, and the
  title no longer calls them real answers.
- A new test takes a membrane 404 and an unpaired-tool-result 400 through
  classifyInferenceError, as the three exhausted settles do, and onto the
  exhausted trace: the 404 keeps the history, the 400 sheds.
- shedMayHelp's comment: membrane 0.5.87's table in full (auth codes and
  reasons, the two account refusals it recognizes, context overflow by code
  or text at 400 or 422; 401-403 auth, 408 timeout, 429 rate_limit, 404
  invalid_request, 413 context_length, other 4xx invalid_request); an
  adapter can type a failure before the table, as Anthropic's does for a
  gateway's aggregate error, and a failure with no status is typed from its
  message; a shed takes the exchange out of the agent's context, which only
  an operator's undo brings back; the example of an unrecognized refusal is
  an account refusal membrane doesn't know by its text.
- The test file's notes: membrane anima-research#56 is in 0.5.87, the floor; the table
  test lists 0.5.87's types.
…e implies

Greta-2425's read found the gate's comment saying a failure with no HTTP
status is typed from its message, with no status. membrane 0.5.87's
errorFromProviderStatus first takes the status statusForProviderCode gives a
known code, as for Anthropic's mid-stream errors, and types from the message
only when the code is unknown. The comment says so now, and two status-less
rows pin it: a not_found_error comes back invalid_request at 404 and keeps
the history, and a request_too_large context_length at 413, which sheds.
@nissa-seru nissa-seru added the no-changelog Exempts this PR from the changelog check (internal-only change) label Oct 14, 2026
@greptile-apps

greptile-apps Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5 Tier: apex

[Medium impact] This PR appears safe to merge.

Summary

Adds tests connecting Membrane’s provider error types and HTTP statuses to the history breaker. Production logic is unchanged.

  • Provider-answer tests pin which failures may shed history.
  • Exhausted-trace tests pin how provider errors reach the shed gate.

Reviews (1) · Last reviewed commit: "test(breaker): a status-less answer take..." · Reviewed by Greptile

@nissa-seru nissa-seru mentioned this pull request Oct 15, 2026
1 task done
@nissa-seru
nissa-seru merged commit be1cab5 into anima-research:main Oct 15, 2026
7 checks passed
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 16, 2026
…pawn wait and transport listeners, and TypeScript 7

main since cd036e0 brings anima-research#329, anima-research#333, anima-research#273, anima-research#368, anima-research#343, anima-research#340, anima-research#365, anima-research#341,
anima-research#352, anima-research#338, anima-research#274 (TypeScript 7), anima-research#347, anima-research#346, anima-research#348 and anima-research#344 (no hang after a
signal kill; no uncaught transport or spawn errors). Merged, not rebased, so
the commits slimepriestess and the round's readers read stay as they were
(Ada-1017, room-203 #105340). `git show --remerge-diff` shows the resolution
alone.

Conflicts, three files:
- src/mcpl/transport.ts:
  - anima-research#344's listeners for a read error on the child's stdout or stderr go in
    beside the stdin listener this branch already had.
  - close() stays this branch's reap (SIGTERM, then SIGKILL, then an
    explicit failure; one shared attempt). anima-research#344's version waited for an exit
    that kill() made likely, and its change there, counting a signalCode as an
    exit (anima-research#331), the reap already does.
- src/mcpl/server-connection.ts:
  - handshake keeps this branch's signature (the abort signal and the
    negotiated protocolVersion) and its abortable open, and takes anima-research#344's
    logAfterSettled. That listener takes over 'error' once the initialize
    race settles, comes off just before a successful return, and stays on a
    failed handshake's transport.
  - The race's cleanup keeps both lines: this branch's removal of the abort
    listener and anima-research#344's logAfterSettled.
  - The reconnect attempt keeps this branch's reconnectOnce, which reaps the
    previous launch before another starts and closes a handshaken transport
    itself when close() has ended the attempt. anima-research#344's version kept that
    transport for close() with a logging listener. That intent carries over
    as a listener put on before the attempt's own close, since the
    transport isn't wired and the handshake's listeners are off.
- package.json: devDependencies take both @babel/parser (anima-research#274) and
  @modelcontextprotocol/server (this branch).

Beyond the conflicts, for main's async StdioTransport.spawn (anima-research#344):
- SpawnerStdioTransport.start() awaits it. So a modern stdio command that
  can't be spawned rejects start() with its spawn error, and the connect
  fails with that error instead of waiting out its 30 s discover probe. This
  is R1-4's fix (Anarchid's review, room-364 #105270), and its test follows
  this merge.
- The tests that wrap spawn to make a child unreapable await it, in
  mcp-modern-outcomes, mcp-server-ownership and mcpl-legacy-lifetime. So
  does the unreapable-close test's own spawn. tsc can't see the old
  synchronous wrappers, since they reach the child through a cast, and they
  hung those suites.
- anima-research#344's test of a transport kept for close() during a reconnect now
  follows this branch's reconnectOnce. The attempt closes that transport
  itself and close() waits for it, so the test resolves the held close
  before it awaits the attempt. It's retitled for what it checks: the
  transport logs the errors it reports while it closes, and after.

tsc (TypeScript 7) is clean. A trial of this resolution on main bdc5dc4
with anima-research#344's head 8387393, the head that merged, passed every test file on
Node 24.21.0 and Node 20.20.2 (J10968). The only failures were 69 in a
stale dist/test/mcp-modern-http-2025.test.js that the VPS copy kept from
anima-research#337's runs. The full check of this commit follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 17, 2026
…nima-research#352 has landed

main took anima-research#343, anima-research#340, anima-research#365 (context-manager ^0.14.1), anima-research#341 and anima-research#352 since 92125b0.

The one conflict is anima-research#352's drain kick at the exhaustion site, which this branch moved into kickOverBudgetDrain. The site keeps this branch's call. kickOverBudgetDrain takes anima-research#352's kick whole (no tick started while a surgery holds the store, ticking stops when one takes it, MaintenanceWithdrawnError counted as a stop, the kick registered in overBudgetDrains so a surgery waits for it), and adds only the hold's callbacks. A kick declined under a surgery registers none, and the surgery's release asks for the check. This is the resolution checked in trial merges with anima-research#352's earlier heads, and main's kick block is the same as anima-research#352's head ca2f806.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog Exempts this PR from the changelog check (internal-only change)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant