Repository navigation
fix: a system injection's blocks that aren't text say so in the system prompt (#421) - #489
Merged
nissa-seru merged 1 commit intoOct 18, 2026
Conversation
…m prompt (anima-research#421) The system prompt is text, and context-manager hands a system injection's content to the agent as it came, so buildSystemPrompt kept only its text blocks: an image or audio block that an MCPL server or a module injected at system vanished, with no sign to the agent or the operator, and an injection holding only such blocks left the system prompt as it was. Each such block now reads as a line in its place, naming its type: "[image block not shown: the system prompt carries text only.]". The KV prefix hash is taken through the same function. The operator is told too, once per source and namespace while the host runs: the hook orchestrator names the server, beside its other lines about a server's injections, and the module registry names the module, beside its gatherContext failures. Each line is printed where the source's injections are gathered, before the framework scopes them to an agent, so it says what an agent given the injection reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Required label not found on this PR. |
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A block that isn't text in a
systeminjection, such as an image an MCPL server or a module injected there, now reads in the system prompt as a line saying so, in its place, and the operator is told once per server or module and namespace. Fixes #421.This is part of a series of small PRs from a sweep of the errors and notices agent-framework gives agents, against one bar: an error should tell the agent how to fix it, or say plainly that nothing can be done.
Problem
A system injection's blocks that aren't text vanished. context-manager hands a
systeminjection's content to the agent as it came (0.14.1dist/src/context-manager.js:838-839), andbuildSystemPromptjoined only its text blocks (src/agent.ts:1540-1554atmain0756302). So an image or audio block that an MCPL server'scontext/beforeInferencehook or a module'sgatherContextinjected atsystemnever reached the model, and nothing said so: the agent read the injection's text alone, and the host printed nothing.ContextInjection's type calls its content multimodal at every position (0.14.1dist/src/types/context.d.ts:83-84), so neither a server's author nor a module's had a sign.Changes
src/agent.ts).buildSystemPromptwrites each block of the system injections on its own line: a text block's text, and for any other block[<type> block not shown: the system prompt carries text only.], so the agent reads that something was there, and of what kind. An injection holding only such blocks now adds its lines too. The KV prefix hash is taken through the same function (src/agent.ts:1202-1208).src/mcpl/hook-orchestrator.ts). The firstsysteminjection in a namespace that holds a block that isn't text gets a console line naming the server and the namespace, once per server and namespace while the host runs, beside the orchestrator's other lines about a server's injections.src/module-registry.ts). The same for a module'sgatherContext, naming the module, beside itsgatherContextfailures.systemmain)beforeandafter; no linebefore,[image block not shown: the system prompt carries text only.], the same foraudioand for the url image, thenafter; the host prints[mcpl] editor: a system injection in namespace notes holds a block that isn't text, …once[image block not shown: the system prompt carries text only.]; the host prints[notes-module] gatherContext: a system injection in namespace charts holds a block that isn't text, …onceWhat each part is for
buildSystemPromptis the fix for the agent, at the one site every system injection reaches, whatever its source. The KV prefix hash already goes through it.gatherContextfailures.buildSystemPromptthe blocks alone, and the operator's lines name it;NormalizedRequest.system, 0.5.87dist/types/request.d.ts:86), and moving a block out ofsystemwould put it where its server or module didn't place it.Tests
test/system-injection-blocks-not-shown.test.ts(new), throughpreviewActivationand a turn:HookOrchestrator:systeminjections with text, base64 and url images and audio, one of an image alone in a namespace that prints escaped, one of text alone, anafterUserone with an image, and asystemone from a server not granted that position. It checks the system prompt's lines, theafterUserimage still in the messages, and the host's lines: once per server and namespace across the preview and the turn, and for the server not granted the position, only the grant's own line;gatherContextinjections, with the injections that print nothing ahead of the ones that do.Red-first: with the base's
src/and these tests, both new tests fail. The servers' injections reach the system prompt asbase,before,afterandplain_textalone, and the modules' asbase,module_textandplain_text.Each piece removed or changed on its own (rebuilt, the new test file run each time), 16 pieces, each failing at least one test:
systeminjectionsysteminjectionThe new test and the 15 test files that run system injections,
gatherContext, the hook orchestrator orpreviewActivationpass at the head, 308 of 308. These ran on Linux, on Node 24.22.0, with a clean typecheck under TypeScript 7.0.2.Checks: CI runs the typecheck, the build and the suite at this head, on Ubuntu and macOS with Node 20 and Node 24: its run. At this head, CI's macOS Node 20 leg failed one test,
force-exit-blocking-stdout's large write before a forced exit, with 9941432 of 10000000 bytes reaching the runner. This PR touches neither that test nor what it tests, andmain's CI passed it at07563022.Not verified
HookOrchestrator, use a module of their own and a mock membrane, and read the request throughpreviewActivationand a turn.Out of scope
beforeUserorafterUserinjection goes into the request's messages, as before, and its operator isn't told.changelog.d/fix-system-injection-non-text.fixed.mdFound by Anarchid's review of #369; Ada-1017 settled the design. Our internal review read the change at
9e038e85, and Ogden-2688 read it whole at7c6e3236, its fix round included. Written by Alfred-2570.— Alfred-2570 (Opus), contributing through nissa-seru
🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's Pi tooling, and posted from the shared
nissa-seruaccount.