Repository navigation
feat(rutis): image tool results across ecosystems - #267
Merged
Merged
Conversation
…tis-agent
- bridge: call_tool results and after_tool edits take yoagent-shaped
content blocks (text, base64 images) instead of text; validated by
content_blocks
- dsh adapter: dsh image blocks (attachment references) become yoagent
images via the attachments service when one is loaded, else a text
placeholder
- rutis-agent example: a runner's {"content": [...]} value is read as
blocks (dot_picture)
- yoagent.d.ts ContentBlock; README, CLAUDE.md, CHANGELOG
- tests: content_blocks unit, TS/Python round trip, dsh with and without
a store; dsh_test no longer reads a mid-write empty abort file
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T7iq5hpndSiHQcnAsywKuG
…limits - content_blocks decodes base64 (STANDARD), takes png/jpeg/gif/webp only, and refuses images over 10 MB (websocket frame limit): an image a provider refuses would sit in history and fail every later request - dsh adapter: dsh's offloaded images and images over 3.75 MB (under Anthropic's 5 MB base64) stay text; an error result's images are not read - rutis-agent example: blocks only with at least one image - docs: limits, picking fields in after_tool edits, text join Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T7iq5hpndSiHQcnAsywKuG
An image identical to one in the output the edit was given passes as it is: yoagent's read_file takes bmp up to 20 MB, MCP tools other types, and keeping them must not withhold the result. New or changed images are checked as before. Tests: kept bmp passes, changed data does not; the size pre-check's own rejection. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T7iq5hpndSiHQcnAsywKuG
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Images in tool results, for every ecosystem the bridge connects.
Bridge (TypeScript / Python handlers).
call_toolresults andafter_tooledits takecontentblocks in yoagent's JSON shape — the same as pi's and MCP's — instead oftext:after_toolalready sawoutput.content; it can now keep, add or drop images when it edits a result.textandcontentare exclusive; anything else incontent(unknown block, non-image/*type, non-base64 data) fails the answer, as malformed answers already do. Rust handlers already returned full yoagent tool results.DSH. dsh image blocks are references into dsh's attachment store (
{type: 'image', attachment: ImageAttachmentRef}). The adapter reads the bytes with theattachmentsservice (readImage) when one is loaded — looked up per image, so the adapter still runs without a store; then, or when a read fails, the image is a text placeholder.rutis-agent. rutis-agent (v0.7.0) has text-only tool results: a runner's JSON value is serialized. The example adapter reads a runner's
{"content": [...]}value as yoagent blocks (a convention of this adapter; rutis-agent's own agent still sees the JSON text), shown by adot_picturetool.pi will follow in #265 (its images already have this shape).
Tests:
content_blocksunit test; a TypeScript + Python round trip (call_toolimages,after_tooledits keeping/dropping them); dsh with and without a store; the rutis-agent example checks the image. Also fixes a race indsh_test(an abort file read mid-write as empty). Full yoagent-rutis suite passes with the runtimes required; clippy clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01T7iq5hpndSiHQcnAsywKuG