Skip to content

test: pin that the [inference-failed] note keeps a failure's reason and type on one line, with notice openings marked - #893

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:ansel/inference-failed-marking-pins
Oct 29, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:ansel/inference-failed-marking-pins

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

This adds two tests that pin how the [inference-failed] note writes a failure's reason and type: on the note's one line, with an opening of a notice tag the framework knows marked. The note passes the reason through failureShown, and the type with its HTTP status through suppliedValue, in the note for a model call that failed and in the note for a turn that ended. Those wrappers came with #852, released in 0.29.0. This change touches only test/inference-failure-bounds.test.ts, +31 −1. With each of the three wrappers removed, or narrowed to oneLine alone, one of the two new tests fails, and every other test in the 13 test files whose text holds inference-failed passes.

Problem

At 0f11975e, src/framework.ts passes the note's supplied values through three wrappers:

  • :31558: ${failureShown(markerReason)}, the reason;
  • :31556: (${suppliedValue(cause)}) after "the model call … failed", the type with its HTTP status;
  • :31549: (${suppliedValue(cause)}) after "it failed", in the note for a turn that ended.

The runs below show that no test in the 13 test files whose text holds inference-failed checks them. At this head, with any of the three removed, every test in those files passes except one of the two new ones, and the rest of those files is as at 0f11975e.

Changes

  • test/inference-failure-bounds.test.ts gains two tests, and a sentence in its header comment saying what they check. Each gives the note a type that holds a line break and an opening of a notice tag the framework knows, with HTTP status 529. It checks that the note has no line feed or U+2028 and shows the type with its opening escaped.
    • "the marker gives a failure's reason and type on its one line, with a notice's opening in them marked" checks the note for a model call that failed. Its reason holds line breaks and such an opening too, and the test checks the reason the same way.
    • "the note for a turn that ended gives the failure's type on its one line, with a notice's opening in it marked" checks the note for a turn that ended, with a reason of one word.
  • No change outside that file: nothing in src/, no changelog fragment, and no text an agent or operator sees.

On October 29 at 06:08 UTC, the open pull requests that also touch test/inference-failure-bounds.test.ts are #190 and #844. By git merge-tree, each conflicts with main and with this head in the same files, and this test file isn't one of them.

Tests

On the files of this head, 41ea6529, from a clean checkout of a commit with the same tree that isn't in this pull request's history, on Linux with Node 24.22.1 and npm 11.19.1, after npm install with package-lock.json and node_modules removed, and npm run build. npm ls --depth=0 in that install gives @animalabs/chronicle 0.5.0, @animalabs/context-manager 0.24.1, @animalabs/mcpl-core 0.4.0, @animalabs/membrane 0.8.0, @babel/parser 7.29.11, @modelcontextprotocol/client 2.4.0, @modelcontextprotocol/server 2.4.0, @types/node 22.20.5, @types/ws 8.18.2, chokidar 4.0.3, discord.js 14.27.0, tsx 4.23.15, typescript 7.0.2 and ws 8.22.0.

  • The 13 test files whose text holds inference-failed, activation-request-sanitize, cancel-surfaces-as-error, default-error-policy, exhausted-turn-rearm, inference-bounds, inference-failure-bounds, inference-failure-observability, kv-unified-open-flight-retry, prose-drafts, provider-acceleration-cooldown, provider-waits, shutdown-not-user-attributed and user-interrupt-not-failure: 580 of 580 pass.
  • Six changes to src/framework.ts, each built and run against the same 13 files: each fails exactly one test, and the other 579 pass. Each removal changes one line. Each replacement by oneLine changes that line and adds oneLine to the import at :138.
    • The reason's failureShown removed, or replaced by oneLine: "the marker gives a failure's reason and type…" fails.
    • The failed call's suppliedValue removed, or replaced by oneLine: the same test fails.
    • The ended turn's suppliedValue removed, or replaced by oneLine: "the note for a turn that ended gives the failure's type…" fails.
  • Time: at the head, the two new tests took 1.00 and 0.76 ms.

Not verified

  • The whole suite at this head: these runs covered the 13 files above. CI's run on this pull request is its first whole-suite run at this head.
  • Openings of the framework's other notice tags in these values: each test gives an opening of one tag.
  • Platforms: every run was on Linux.

  • This change is test-only: it touches nothing in src/, and the no-changelog label applies.

Ansel-2428 wrote the tests. Desmond-2579 found, by runs at an earlier commit, that every test in the test files whose text holds inference-failed passed with the reason's wrapper or the ended note's wrapper removed, and he read the change, both before it was rebased onto main and in its final form. Daniel-2612 wrote this description.

…n and type on its one line, marked

The marker passes a failure's reason through failureShown, and its type
with its HTTP status, when it shows them, through suppliedValue, both in
the note for a model call that failed and in the note for a turn that
ended. Each wrapper puts the text on one line and marks any opening of a
notice tag the framework knows in it, so that text a provider supplies
can neither end the marker's line nor read as a notice; failureShown
leaves the framework's own placeholder for a failure it couldn't
describe as it is. No test pinned any of the three: with any one of
them removed, every test in the files that mention the marker still
passed.

Two tests now give the note a type holding line breaks and a notice's
opening, the first with a reason holding them too, and check that the
note keeps them on its one line with the opening marked. With each
wrapper removed in turn, or narrowed to oneLine alone, one of the two
fails and every other test in those files passes.
@nissa-seru nissa-seru added the no-changelog Exempts this PR from the changelog check (internal-only change) label Oct 29, 2026
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 29, 2026
@nissa-seru
nissa-seru merged commit 4159ab7 into anima-research:main Oct 29, 2026
14 checks passed
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 29, 2026
main 7aab5e3 brings anima-research#896 (the written-to test's host write fails
before the child's SIGTERM, in both engines), anima-research#871, anima-research#897, anima-research#893, anima-research#870 and
anima-research#869 since 0f11975. No file conflicts.
tel-0s added a commit that referenced this pull request Oct 29, 2026
…'s context retired, MCPL inference aborted at stop, the re-armed deadline cut, failure accounting released with its agent

main at 612f6fd is eight merges past 4a45766, with no release among them,
so package.json stays 0.30.0 and this branch's floors stand (chronicle
^0.6.0, context-manager ^0.25.0, membrane ^0.8.0), which
test/dependency-ranges pins. It brings #871 (a re-armed turn's deadline cut
follows the deadline-cut rule; an ended agent takes its re-armed deadline
cut with it), #896 (the race order of the MCPL down alerts), #897 (an ended
agent's failure accounting goes with it: forgetFailureAccounting), #893
(the [inference-failed] marker's pins), #870 (strict credential mode for
opt-in WebSocket dials), #895 (stop() aborts each MCPL server's
inference/request still running, and answers it -32000), #898 (a module's
context is retired once its stop() completes: its pushes dropped with a
line, its writes, calls and subscriptions refused, its unapplied response
and unmerged gatherContext answer left out) and #884 (the suite's speed:
setAbandonGraceForTests and CI).

One path conflicted (3 regions):

- src/module-registry.ts:
  - imports: main's boundedErrorMessage; heldWriteId stays out, as on this
    branch, which has no held-write handles.
  - the registry's constructor: both, this branch's window lookups
    (windowStatusOf, observes, viewKeeps, agentNameOf) and main's
    dropRetiredPush.
  - findMessageByExternalId's comment: this branch's (a '' names a write
    kept in memory, a surgery's hold or the wait for its agent's
    recovery; no held rows) with main's clause that a stopped module's
    context keeps its mapping as it was. The code merged cleanly: main's
    retired guards around the mapping's updates, without the held-row
    answer this branch removed.

Read onto this branch past the conflict:
- releaseAgentRuntime takes main's freshCallsCut.delete,
  forgetFailureAccounting and rearmDeadlineCuts.delete, beside this
  branch's forgetPresentation; an ephemeral run's disposal and
  disposeConversationAgent's name-only arm take them as main put them.
- ModuleContext's doc (types/module.ts): a retired context's reads list
  this branch's window lookups (windowStatusOf, observes, viewKeeps,
  agentNameOf), which stay open as main's reads do; main's clause that a
  read meeting a held write may record its landing goes, as this branch
  has no held rows to meet.
- applyProcessResponse is synchronous on main now; this branch's body has
  no await, so it takes main's form.

storeSees: no new site; main opens and derives no context manager.
agentSees: no new view decision (noteSpokenTo's existing check gains only
the re-arm's reset). Notice tags: none new; main's new texts ([framework],
[process] and [unstick] lines, the -32000 answer) go to stderr or an MCPL
server, not to an agent. #898's retire and #884's suite changes touch no
presentation point.

Changelog: no release in main's range, so no fragment was consumed; main's
five new fragments come in as written. retire-stopped-module-context's
"reads answer, and fail, as before" covers this branch's window lookups.

Tests adapted:
- retired-module-context, the findMessageByExternalId case: its '' entry
  comes from a write a surgery's hold keeps in memory, released before the
  module's removal, where main made it with a quiesce. A write while
  quiesced is stored at once here (#291), so it mapped to its id.

Tests: built with zero errors; no stale compiled test; fixtures copied.
main's 35 changed test files: 1522 tests, 1516 pass, 1 fail (the
retired-module-context case, adapted above; its file then 22 of 22), 5
skipped (Linux-only workspace cases). presentation*, window-*, dendrite-*,
stored-injections, audience-stamp*, dependency-ranges, held-* and
tune-out-* (44 files): 606 of 606. Full suite (npm test): 9518 tests,
9469 pass, 0 fail, 49 skipped (runtime-conditional: Linux-only rules, and
temp directories that fold case or normalization).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog Exempts this PR from the changelog check (internal-only change)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant