Repository navigation
capture: share image history strings across turns to avoid quadratic memory - #1534
Open
caoxiaoyuyuyuyuyu wants to merge 1 commit into
Open
caoxiaoyuyuyuyuyu wants to merge 1 commit into
caoxiaoyuyuyuyuyu wants to merge 1 commit into
Conversation
Each chat request re-sends the whole conversation and is JSON-decoded on its own, so turn k holds a fresh copy of every image and long text from turns 1..k. RolloutGraph kept all of them, which makes memory grow quadratically with the number of turns in multimodal rollouts. add_turn now replaces long strings (>= 1024 chars) in request_messages, request_tools and response_message with one graph-local copy. Only immutable str objects are shared; dicts and lists are rebuilt per turn, so values and message structure are unchanged and exports are byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RolloutGraphkeeps every turn'srequest_messages. A chat request re-sends the whole conversation and each request body is JSON-decoded on its own, so turn k holds a fresh copy of every image (and other long text) attached in turns 1..k. For multimodal rollouts graph memory therefore grows quadratically with the number of turns: n turns with one base64 image each retain n(n+1)/2 image copies instead of n.This PR makes
add_turnreplace long strings (>= 1024 chars) inrequest_messages,request_toolsandresponse_messagewith a single graph-local copy.Type of Change
Reproduction (synthetic)
Build a graph of 12 turns where turn k re-sends k+1 user messages, each with a distinct 64 KiB
data:image/png;base64,...URL, and every request body is passed throughjson.loads(json.dumps(...))as the capture server does. Before this change the graph retains 78 distinct image string objects (~5 MiB); after it, 12 (~0.8 MiB).Why the output is unchanged
strobjects are shared; equal strings are swapped for an equal string, so every value is identical.RolloutGraph(not process-global), so it is released together with the graph.RFC Status
Test Plan
New
tests/envs/test_capture_graph_shared_text.py(fully synthetic data):export_session(..., include_messages=True)serialised withjson.dumpsis byte-identical to the export from the same graph with sharing disabled.Two of the three tests fail on
mainand all pass with this change.(
test_coding_codeact_env.pyfails to collect locally for an unrelated missing optional dependency.)Claude Code Review
N/A
🤖 Generated with Claude Code
Note
Low Risk
Memory-only deduplication of immutable strings at graph ingest; behavior and exports are covered by tests, with no auth or training-token logic changes.
Overview
RolloutGraph now interns strings ≥1024 characters when ingesting turns, so repeated base64 image URLs and other long text from re-sent chat history are stored once per graph instead of once per turn.
On
add_turn,request_messages,request_tools, andresponse_messageare passed through a recursive_share_texthelper that replaces matching long strings with a single graph-local copy while rebuilding lists and dicts per turn (only immutable strings are shared).Adds
tests/envs/test_capture_graph_shared_text.py: multimodal multi-turn graphs retain one object per distinct image, edits in one turn do not leak to others, andexport_sessionJSON is byte-identical with sharing disabled.Reviewed by Cursor Bugbot for commit 3265518. Bugbot is set up for automated code reviews on this repo. Configure here.