Skip to content

fix(select): send no summary that costs more than its messages raw - #409

Open
nissa-seru wants to merge 3 commits into
anima-research:mainfrom
nissa-seru:ezra/dearer-summaries
Open

nissa-seru wants to merge 3 commits into
anima-research:mainfrom
nissa-seru:ezra/dearer-summaries

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

On the hierarchical select with positioned recall pairs, a compile now sends a summary's messages raw in its place where they cost less than the summary's recall pair, both as sent and as emission charges for it. It keeps the summary where sending its messages raw could send one twice or change how pairing repair pairs a tool call or result, and in the other cases Changes lists. A compile that fits with the messages in the summary's place no longer refuses: on #407's corpus at the full summary target, compiles with the select of this pull request's first head, 01c7b59, fit 1,828 and 3,110 times where main refused, and never refused where main compiled (Measured). It costs compile time. On stores where no summary costs more than its messages, this head's compiles took 1.11 times as long as main's at 2,000 messages and 1.09 times at 8,000, by median over 12 rounds (Measured). On #407's corpus at the summary target and the default budget, with that select, requests carried 14.8% and 19.6% fewer tokens than main's, and compiles took 0.03 to 0.11 ms more process CPU each, on compiles of about 0.5 to 0.6 ms. The adaptive selects and positionedRecallPairs: false are unchanged.

Part of #407.

Problem

#407 reports it at main 9c587a8b: a summary goes out in place of the messages it covers even when its recall pair costs more than all of them sent raw, so the request carries more tokens than those messages would, and a compile that would fit with them in its place is refused. In the issue's probe, an L1 whose pair cost 933 tokens went out in place of four messages that cost 456, and budgets of 984, 1,004 and 1,460 tokens threw OverBudgetError, where 984 is the request with those messages in the pair's place. At this head, the same probe sends those four messages raw in a request of 984 tokens: it compiles at a budget of 984, and refuses at 983.

With this head's test files on main's src/ at e464d5ad, test/dearer-summaries.test.ts fails 16 of its 50 cases. The first is "sends the messages raw in place of an L1 that costs more than they do, and fits a budget that L1 could not", at its check that the dearer L1's pair doesn't go out. test/overbudget-diagnostics.test.ts fails 1 of its 161, a clearedBy case this change adds, at its premise that the select sends a dearer L1's messages raw in its place (Tests).

Changes

  1. A summary that costs more than its messages is replaced by them. The hierarchical select now weighs each summary it lists, at every level, against all the messages under it sent raw, messages a pin keeps raw included at their full price (dearerSummaries in src/strategies/autobiographical.ts). Where the summary costs more, the select leaves it out, with every summary under it, and sends its messages raw. An L2 or L3 that costs more than its children as recall pairs, but less than its messages raw, goes out as before.
  2. What it weighs. A pair is priced at the lesser of two estimates: the pair as emission sends it, the recall header and the answer, cut by maxMessageTokens where the cut applies, and what emission charges for it (recallPairCost). So a summary is replaced only where its messages cost less than both. weighedRawPrice returns its sum of a summary's messages once the sum reaches the pair's price (if (price >= limit) return price;). A message is priced at the larger of its stored and sent forms, so a summary that costs less than its messages whole isn't swapped for messages whose tool results pruning replaces. One that costs more than them whole can be, and pruning's markers go out in place of the results it replaces. In "doesn't move a summary's choice with the token calibration", a summary whose recorded size equals its messages' estimate goes out at token calibrations of 0.3, 1 and 3.5.
  3. Where a summary stays as it was, though it costs more than its messages:
    • a summary under it isn't marked as merged into it, as in a merge that marked only some of its sources;
    • a summary outside its own tree also holds one of its messages, so no message goes out both raw and inside a summary;
    • read in the parts pairing repair reads a message in (cycleParts, since fix(pairing): send a tool result that answers a call in its own message #365), a tool call among its messages isn't answered in the part right after it, a result there doesn't answer a call in the part right before it, or two results there answer one call;
    • a tool call or result among its messages is answered by, or answers, a message in the head window or a message another summary covers;
    • any of its messages lies between a tool call and a result it holds neither of, in the order the request sends them (the head, the middle, then the recent window), judged from the earliest call of the result's id. Sent raw, its messages could push the result past the six entries pairing repair looks ahead to find it, which The tool-pairing repair looks only six entries ahead, so a call whose result is rendered further away gets a stub and its real result is dropped #319 reports;
    • a result among its messages has a call of its id outside the summary sent before it, other than a call in the last part of the message right before it with no earlier call of that id in that message.
  4. Budgets. The share of a level's budget a replaced summary held goes to no other summary: a summary that budget left out beside pinned messages stays out, and a replaced L1 isn't brought back to cover its messages where the budget left it out.
  5. clearedBy. Where the select now sends an L1's messages raw in its place, an over-budget refusal from it no longer names compression in clearedBy for adopting that L1 as a queued chunk's exact L1: the select sends those messages raw whether or not the L1 is adopted, so adopting it frees nothing. Everything else that count does is as on main, and the OverBudgetClearedBy doc (src/adaptive/picker.ts) gains a sentence saying so.
  6. Pins. 0.24.1's CHANGELOG says, under Fixed, "A message a pin keeps raw still goes out beside the summary over its chunk while no window holds a message of that summary". That still holds where the summary costs no more than all the chunk's messages, pinned ones included. Where it costs more, the summary is left out and the chunk's other messages go raw.
  7. The README's tick() bullet adds: "Without adaptive resolution, and with positionedRecallPairs (true), a compile can send the chunk's messages raw instead, where the summary would cost more than all of them, and the same for a merged summary and all the messages under it."
  8. Seven existing files under test/ seeded summaries that cost more than what they cover: test/fixtures/recall-envelope-fixture.ts and test/<name>.test.ts for fold-journal-autobiographical, pins, recall-positioning, recall-tools-substitution and strategy-persistence, over messages of one or two words, and test/overbudget-diagnostics.test.ts, over messages of about thirty. Those messages now carry enough words to cost more than the summary, and five added assertions check, as a premise, that a summary goes out rather than its messages. test/overbudget-diagnostics.test.ts's adoption tests now seed L1s cheaper than their chunks and assert that the adopted L1 goes out, and its countsCompressionWork helper passes the refusal's left-out summaries on to the count.
  9. A count. getStats() gives dearerReplaced: how many summaries the hierarchical select's last run that wasn't a dry run, and that weighed them, left out because each cost more than its messages, whose messages that run's request then sends raw if it compiles. A summary whose summaries under it went too counts once.

Against each open pull request but #350, a trial git merge-tree of this head gives what one of main (e464d5ad) gives, at 04:20 UTC on October 30: no conflict with #364, #369 and #410, and with #94, #130, #140, #310 and #315 the same conflicted files, with the same number of conflict regions in each.

With #350

@tel-0s, #350 conflicts with main at e464d5ad in eight files, each of which #411 changed, and with this head in those eight and in one region of README.md: the tick() bullet, where this change adds the sentence above after "which takes the chunk's place in compiled context." and #350 rewrites the autoTickOnNewMessage sentence that follows it.

Tests

CI's Build & Test runs the whole suite on this pull request, on Ubuntu with Node 22.12.0.

Each ran on Linux x86_64 with Node 24.22.1, after npm ci installed membrane 0.8.0, @types/node 22.19.15, TypeScript 7.0.2 and chronicle, and node --test ran the built files. Those at this head, and its red runs, ran on a copy of a clean checkout of 5355eb45c, with chronicle 0.5.1. The 35 mutants and the existing tests the rule reaches ran at earlier commits of this change, with chronicle 0.5.0, as each says.

  • At this head: tsc exits 0, and these 17 files pass every case: dearer-summaries 50 of 50 (new), overbudget-diagnostics 161, strategy-persistence 57, recall-tools-substitution 16, fold-journal-autobiographical 12, pins 7, recall-positioning 6, the 4 others that import the changed fixture (thinking-binding 61, recall-envelope-truncation 30, recall-envelope 20, carrier-policy 10), window-edge-summaries 34, anti-redundancy-budget 3 and merge-quarantine-hole 30, and, of the files fix(store): require chronicle ^0.5.1, which keeps a stored object's key order #411 changed, block-as-stored 14, compression-recall-curve 139 and tool-pruning-price 37, each test/<name>.test.ts.
  • Red: with this head's test/ on main's src/ at e464d5ad, tsc reports six errors, each where a test reads getStats().dearerReplaced, which main doesn't have, and still emits. dearer-summaries fails 16 of 50: the two cases that replace an L1, one of them over pinned messages; the L2 and the L3 replaced by their messages; the three budget and coverage cases; six of the shapes with tool calls that replace a summary; the dearer case of pruned results; "replaces an L1 between a result and a later call of its id, since a call after its result spans nothing"; and the dearerReplaced case. overbudget-diagnostics fails 1 of 161, "names nothing on the hierarchical path for chunks whose exact L1 in memory costs more than their messages, since the select leaves that L1 out after adoption as before", at its premise. The other 15 files pass every case.
  • This round's tests on the head before it: with this round's test/ on 01c7b59's src/, tsc reports the same six errors and still emits. dearer-summaries fails 4 of 50: "keeps an L1 that maxMessageTokens cuts where its messages cost more than emission charges for its pair, so a budget that fits that charge compiles", at its first check, where messages of 288 tokens cost more than emission's charge for the pair, 285, and less than the pair as sent, 298, and the L1 doesn't go out; the dearerReplaced case; and the L2 and the L3 replaced by their messages, at their counts. The new checks that each replaced message goes out raw once, under no summary sent, in store order, pass there in the L1 and pinned-L1 cases. overbudget-diagnostics passes 161 of 161.
  • Mutants: 35 mutants, each changing one part of the rule's code, at an earlier commit of this change whose src/ differs from 01c7b59's only in comments and whose dearer-summaries test file is 01c7b59's. Each was built and run against dearer-summaries and fails from 1 to 14 of its cases. Three of the clearedBy count, at a commit whose src/ and test/ are 01c7b59's, run against overbudget-diagnostics: ignoring the left-out summaries fails 1 case, never counting such an adoption fails 6, and a price check in their place fails 1. Five more, at this head, run against dearer-summaries and overbudget-diagnostics: pricing the pair as sent alone fails the maxMessageTokens case above alone; never setting dearerReplaced fails the dearerReplaced case and the L2 and L3 cases; setting it on dry runs too fails the dearerReplaced case alone; counting every summary a replacement leaves out fails the L2 and L3 cases; and summing every message past the pair's price fails none. overbudget-diagnostics passes under each.
  • Existing tests the rule reaches: with one line added that logs each summary the rule leaves out, every test file at 01c7b59's tree passed, 3,599 tests in all, and the line numbers below are at 01c7b59. Besides dearer-summaries and the three clearedBy cases this change adds, the rule leaves a summary out in these existing tests, each of which passes. In test/overbudget-diagnostics.test.ts, it's the cases at lines 1531, 1791, 2061, 2243, 2292, 2317, 2350, 2378, 2514, 2761, 2805, 2952, 2963, 2981, 3004, 3023, 3095, 3182 and 3214, whose bodies this change doesn't edit, though 2317 and 3095 call the countsCompressionWork helper it edits, and those at 2094 and 2114, which it edits. In test/merge-quarantine-hole.test.ts, which it doesn't edit, it's those at 341, 703, 730 and 791.

Measured

On stores where no summary costs more than its messages. Each store was built once, with a stand-in summarizer writing short summaries, hierarchical: true and targetChunkTokens 400. Each version ran in its own Node process on its own copy of the store: 3 compiles to warm up, then 25 timed compile() calls, of which it took the median. The three versions took turns in a rotating order over 12 rounds per size, and all three sent the same messages, by a hash of each one's compile. The machine was shared: its one-minute load ran from 57 to 71 on 12 cores during the run, and the run waited for a CPU 7.3% of its time, 8.9% in its worst minute, with 0.3% of its time stalled on I/O. Medians over the 12 rounds, in ms:

messages (live L1 / L2 / L3) main (e464d5ad) 01c7b59 this head paired by round against main: 01c7b59 / this head
2,000 (2 / 3 / 1) 6.0 6.7 6.3 1.18× / 1.11×
8,000 (1 / 5 / 10) 19.7 26.4 21.8 1.28× / 1.09×

This head's compile took less time than main's in 5 of the 12 rounds at 2,000 messages, and in 2 at 8,000.

The rest of this section was measured before this round, against main at 9c587a8b, at 01c7b59 or at an earlier commit whose select is 01c7b59's, and calls each of those 01c7b59.

On #407's corpus (its setup is in the issue). These compiles ran at an earlier commit of this change against 9c587a8b's src/, with a few added lines that record the rule's choices. Its select is 01c7b59's, without this round's two changes to it, the lesser of two prices and the early stop. Its count for a refusal's clearedBy also priced the L1s it counted, which this head's doesn't. Each part is 108 stores compiled at every step of their growth at five budgets. Tokens are summed over the compiles, by the store's estimate.

full target: some calls never answered / every call answered a quarter of it: never answered / answered
compiles whose sent content differs from main's, default budget 77.2% / 87.9% 13.1% / 13.4%
tokens against main's, default budget −14.82% / −19.57% −0.20% / −0.24%
compiles 01c7b59 refuses where main compiles, at any budget 0 / 0 0 / 0
compiles 01c7b59 makes where main refuses 1,828 / 3,110 64 / 48

Compile time on the corpus, at the default budget, measured over two slices of three stores each in each part, one process at a time. The full target's rows ran at loads of 1.6 to 3.2, and the quarter's in an earlier run whose load was 3.1 at its start and 8.8 at its end:

summary size, part, budget process CPU per compile, main → 01c7b59 profile time, 01c7b59 against main
full target, calls never answered, default 0.606 → 0.637 ms (+5.1%); 0.537 → 0.592 ms (+10.2%) +9.6%; +10.6%
full target, every call answered, default 0.499 → 0.604 ms (+21.0%); 0.494 → 0.596 ms (+20.6%) +19.0%; +19.5%
full target, calls never answered, maxMessageTokens 10,000 0.579 → 0.629 ms (+8.6%); 0.510 → 0.620 ms (+21.6%) +12.9%; +9.2%
full target, every call answered, maxMessageTokens 10,000 0.570 → 0.577 ms (+1.2%); 0.521 → 0.549 ms (+5.4%) −4.2%; −0.3%
a quarter, every call answered, default not measured +0.5%; +2.3%
  • Process CPU is process.cpuUsage() around each compile, with garbage collection and no profiler. Profile time counts CPU profile samples in compile frames, so it leaves garbage collection out.

  • On this corpus maxMessageTokens 10,000 sends what the default sends, to the token, at every budget, so its rows measure the same sends a second time. Their figures moved by up to 20 points from the default rows' in process CPU, and 23 in profile time. Why the rows with every call answered come out near level isn't traced.

  • The choice reads no budget. At every step, the compiles at the five budgets that listed the same summaries over the same messages left out the same summaries, in all 14,454 and 14,250 such groups at each size.

  • fix(select): leave out summaries over messages a window sends raw when a dry run without them fits #360's window rule fell back to taking a window's summaries in the same 12 compiles at main and at 01c7b59, at the full target with calls never answered. It fell back at neither alone.

  • With positionedRecallPairs: false, and on the adaptive select, at a quarter of the target, every compile sent what main's sent.

  • With maxMessageTokens 10,000, at a quarter of the target, every figure above is the quarter's.

On long synthetic stores, of turns of a question, a tool call, its result and a reply, with unique ids, and an L1 over every eight messages. In each store either every summary costs more than its messages or every tenth does, and 01c7b59 replaces each of those. Each store was built once, and one process compiled it with main's strategy code and 01c7b59's in turn, six timed rounds after a warm-up, timing compile() alone. Medians in ms, with the one-minute load at each row's end:

messages summaries that cost more main 01c7b59 of it, the rule load
4,000 all 16.5 41.9 11.5 4.9
4,000 one in ten 13.3 29.5 9.4 5.9
8,000 all 41.5 79.5 26.5 7.3
8,000 one in ten 20.0 47.6 23.6 5.3
16,000 all 104.9 256.9 90.6 5.6
16,000 one in ten 92.5 110.3 47.5 3.0
  • With every summary costing more, 01c7b59's compile took 1.9 to 2.5 times main's. At one in ten, the differences were 16, 28 and 18 ms.
  • The rule's time grew 2.5 times and 2.0 times per doubling of the store at one in ten, from 4,000 to 16,000 messages.
  • By their build times, another process was most likely running on the machine during the 4,000 rows and the 8,000 all-dear row. Each row alternated the two versions in one process, so whatever it took fell on both.
  • Where the extra time went, in one CPU profile of the 16,000 all-dear store with inlining off, each sample put to the version whose compile was running: of the 405 more samples a compile 01c7b59 took, the rule took 201, pairing repair (enforceToolPairing) 44 and the rest of the select 31. Outside the select, provenance attribution (src/compile-provenance.ts) took about 98 more, garbage collection 48, and everything else 18 fewer. 01c7b59 sent about 16,000 raw messages there where main sent 1,999 recall pairs.

What this leaves

Not verified


  • Changelog fragments added: changelog.d/dearer-summaries.changed.md and changelog.d/dearer-summaries.added.md.

Ezra-2423 made this change, and Oscar-1005 made the calls on it. Beth-2389 found what #407 reports and built its corpus. Cordelia-2434, Ambrose-2449 and Humphrey-2407 each wrote the least design they would accept before reading the change, then weighed each of its parts against what it costs. Cordelia read its early rounds word for word, Ambrose every round from the fifth on, and Humphrey the last two before it opened. Our internal review read the whole change once, at an earlier version, then the changes of each of thirteen later rounds, the last of them this round, which fixed what we judged worth fixing from that read. Lottie-2661 ran the runs at this head, the red runs and the timings on stores where no summary costs more, and wrote this description.

The hierarchical select never weighed a summary against the messages it
covers. Outside the head and recent windows, a compressed chunk's
messages went out raw only where a pin kept them raw, where their chunk
wasn't compressed yet, where no chunk held them, or where no listed
summary claimed them. So a summary whose recall pair costs more than all
its messages sent raw still went out in their place, and a compile that
would fit with the messages in its place was refused.

With positioned recall pairs (the default) and without adaptive
resolution, the select now weighs each summary, at every level, against
all the messages under it sent raw, messages a pin keeps raw included at
their full price (dearerSummaries). Where the summary costs more, it is
left out, with every summary under it, and its messages go out raw. An
L2 or L3 that costs more than its children as recall pairs, but less
than its messages raw, goes out as before.

The pair is priced at its estimate as emission sends it: the recall
header and the answer, cut by maxMessageTokens where the cut applies.
Emission charges a summary with a measured size at that size where it is
larger, which only makes a replaced summary dearer still. Each message
is priced at the larger of its stored and sent forms, so a summary that
costs less than its messages whole isn't swapped for messages whose tool
results pruning replaces; one that costs more than them whole can be,
and the replaced results go out as pruning's markers. The choice reads
no budget. The calibration multiplier moves it only by rounding, where
maxMessageTokens cuts the summary or a message, and through the recent
window, which the multiplier moves for the whole select.

A summary is left as it was:
- where a summary under it isn't marked as merged into it, as in a merge
  a failed store write stopped part-way, until the next open repairs it;
- where a summary outside its own subtree also holds one of its
  messages, so no message goes out both raw and inside a summary;
- where, read in the parts pairing repair reads a message in
  (cycleParts), a tool call among its messages isn't answered in the
  part right after it, a result there doesn't answer a call in the part
  right before it, or two results there answer one call;
- where a tool call or result among its messages is answered by, or
  answers, a message in the head or the recent window, or a message
  another summary covers;
- where any of its messages lies between a tool call and a result it
  holds neither of, in the order the request sends them (the head, then
  the middle, then the recent window), judged from the earliest call of
  the result's id. Sent raw, its messages can push the result past the
  six entries pairing repair looks ahead (anima-research#319), where repair stubs the
  call and drops the result. Any such span keeps the summary, whatever
  its length, so the check doesn't depend on that constant;
- where a result among its messages has a call of its id outside the
  summary sent before it, other than a call in the last part of the
  message right before it with no call of that id earlier in that
  message. Sent raw, the result can come within that call's look-ahead,
  and repair can move it there and leave the call it answers a stub.

Where a store's tool call ids repeat, the last two checks keep some
summaries whose raw messages repair would pair the same way; those go
out as before.

The rule decides before anti-redundancy and the per-level budgets, which
pick what they picked before, and the replaced summaries come out of the
selection after the coverage repair, so the budget share a replaced
summary held goes to no other summary. By the store's estimate, the
request carries fewer tokens wherever the select would have sent the
summary, and a compile that would fit with the messages in its place now
fits. One case differs: where the select that leaves out the summaries
over a window's messages didn't fit, so the compile took them and sent
those messages twice, the savings can now let it fit. It is then chosen,
as it would be at a larger budget, and that request can be larger than
before, though within the budget.

Where the select now sends an L1's messages raw in its place, an
over-budget refusal from it doesn't name compression in clearedBy for
adopting that L1 as a queued chunk's exact L1: it sends those messages
raw whether or not the chunk is compressed, so adopting it frees
nothing. compressionWorkQueued takes the refusing select's leftOut for
this; everything else it counts is as before.

The rule prices every summary the select lists, and builds one pass over
the messages for the tool checks when a dearer summary reaches them, so
its work grows with the store's length, not its square. A store where
the select now replaces a summary it sent before sends different
requests from its first compile after the upgrade. Sending a summary's
messages raw costs more compile time than sending its pair, in the work
every compile does per raw message after the select.

test/dearer-summaries.test.ts pins, through compiles of real stores, the
summaries replaced and kept in the shapes above, the budget and coverage
cases, the pricing of pruned and capped messages, and the calibration's
rounding. test/overbudget-diagnostics.test.ts pins the in-memory
adoption count in three cases: an L1 the select sends beside a chunk
with pinned messages, a dearer L1 it leaves out, and a dearer L1 it
sends because another summary holds one of its messages. Its existing
adoption tests now seed L1s cheaper than their chunks and assert that
the adopted L1 goes out. Seven files under test/ seeded summaries that
cost more than what they cover: six over one- or two-word messages, and
test/overbudget-diagnostics.test.ts over messages of about thirty words.
Those messages now carry enough words to cost more than the summary, so
the tests keep testing what their titles say.

The README's tick() bullet names the exception, and a .changed fragment
describes the change. The adaptive selects and positionedRecallPairs:
false are unchanged.

@Anarchid Anarchid left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nightly dual review · round R1 at 01c7b59 · reviewers: Codex gpt-5.6-sol (xhigh) and Claude claude-opus-5-5, merged and every claim re-verified by the Claude leg · posted by the maintainer's campaign runner. Verdict: COMMENT (the formal review event is left to the maintainer). Finding ids R1-n are stable; cite them in fix commits.

Reviewed at 01c7b59 against main 9c587a8

Verdict: COMMENT. The fix is sound, with no blockers or majors; one edge breaks a stated invariant, and the common case pays for a check it could skip

The rule does what it says. A dearer root summary and its subtree leave the selection. Coverage completion then sends their leaves raw exactly once, and the guards keep pairing repair from seeing anything new. The pricing errs toward keeping the summary. The suite and the red run match the description to the case. What's wrong sits around the edges of the rule:

  • With maxMessageTokens set, the rule and emission price a capped pair differently, so a compile main accepted can now be refused (R1-1).
  • The case a real summarizer mostly produces, where no summary is dearer, wasn't measured. There the rule adds about 30% to compile time for nothing, and a one-line early exit halves the rule's share (R1-2).
  • The PR doesn't name the strategy it lands on hardest, KnowledgeStrategy (R1-4).
  • "Fixes #407" closes an issue that this change leaves partly true (R1-3).

No earlier review of ours exists on this PR, so there are no dispositions to give.

Blockers

None.

Major

None.

Minor

R1-1. With maxMessageTokens set, the rule can swap a summary for messages that cost more than emission charges for the pair, so a compile main makes is refused.

  • weighedPairPrice prices the cut render where the cap cuts (src/strategies/autobiographical.ts:16675-16677). Positioned emission sends that render (:17376) but charges recallPairCost (:17391), which prices the uncut answer (:16669-16671, :16642).
  • The cap decides its cut by estimateTextOnlyTokens, but budgets use the store's estimate. So where a summary is just over the cap by the cut's measure, the cut removes a few characters and adds the omission marker. The rendered answer then estimates larger than the uncut one.
  • Repro (seeded L1 of 801 characters, 277 tokens; two messages under it; a 3-token latest message; maxMessageTokens: 200, overBudgetGraceRatio: 0, windows 0/1):
    • main charges the pair 283 and compiles at a budget of 286. Its returned messages estimate at 299 tokens.
    • At this head, with the two messages at 286, 290 and 294 tokens, the summary is replaced. The compile needs 289, 293 and 297, and refuses at 286.
    • With the messages at 274–282 tokens, or at 298, head and main agree on whether it fits.
  • The request this head sends is smaller than the one main sends. The refusal is new all the same, and it contradicts the changelog's "emission charges a summary … at that size where it's larger, which only makes a replaced summary dearer still" (changelog.d/dearer-summaries.changed.md:37-40). Under the cap, emission can charge less than the rule's price.
  • Exposure is narrow: maxMessageTokens defaults to 0, and the raw cost must fall in a window of about a dozen tokens per summary.
  • Fix: replace only where the raw price is under both what the pair sends and what emission charges for it. That means min(weighedPairPrice(s), recallPairCost(s, buildRecallHeader(s))).
    • With that one change, the repro keeps the summary at 286–294 and compiles at 286, as main does. It still replaces at 282.
    • dearer-summaries and overbudget-diagnostics still pass, 209/209.
  • Add a boundary test at budgets between the two. The only cap test (test/dearer-summaries.test.ts:1045) has a wide margin.
  • Emission charging the uncut answer is a pre-existing bug on main. It isn't this PR's to fix, but it's why this one bites.

R1-2. When no summary is dearer, the usual case with a real summarizer, the rule adds about 30% to compile time and changes nothing. An early exit halves the rule's share.

  • The rule works through every root summary the same way, every compile. It expands the root to its leaves, then sums prunedForSend/cappedForSend/estimateTokens over all of them, and only then compares with the pair (:8503-8507). In a healthy store the pair is a small fraction of its leaves, so nearly all of that walk is wasted.

  • The description measures #407's corpus (most L1s dearer) and synthetic stores with every summary or every tenth one dearer. It doesn't measure stores with none dearer.

  • Measured here, in one process on one store, rotating three implementations of dearerSummaries over 12 rounds of 5 compiles each:

    • no rule, which stands in for main;
    • this head's rule;
    • the head's rule with an early exit.

    The store came from a stub summarizer writing short summaries, so none is dearer. It used hierarchical defaults with targetChunkTokens: 400. Machine load was about 5–8.

    messages (pyramid) no rule this head head + early exit the rule alone, head → early exit
    2,000 (13 L3, 4 L2, 3 L1) 1.33 ms 1.71 ms (+29%) 1.50 ms (+13%) 0.36 → 0.22 ms
    8,000 (55 L3, 2 L2, 3 L1) 7.54 ms 9.80 ms (+30%) 8.68 ms (+15%) 2.08 → 0.96 ms

    Medians. All three sent byte-identical requests.

  • The rule's own time grows 5.8× for 4× the messages.

  • Fix:

    • Price the pair first, and stop summing once the raw price reaches it. Have weighedRawPrice take the pair as a limit and return as soon as price >= limit. dearer-summaries and overbudget-diagnostics pass unchanged with it (209/209).
    • What remains is re-expanding every root's leaves on every compile. A summary's leaves don't change once it's written, so a cache keyed on the summary forest would remove most of that too.
  • Please also add a none-dearer row to the description's compile-time table. That's the row most hosts will see.

R1-3. "Fixes #407" closes an issue whose subject this change leaves partly true.

  • #407's title is "The hierarchical select sends a summary that costs more than its messages". positionedRecallPairs: false is the hierarchical select too, and dearerSummaries returns at once there (:8344). #407 measured 858 and 986 compiles on that path sending such a summary, as well as 490 and 418 on the adaptive select.
  • The PR is clear that both are unchanged. But merging with "Fixes" closes the issue, and no other issue tracks either path.
  • Fix: write "Part of #407", or open a follow-up for the legacy combined pair and the adaptive select and link it from "What this leaves".

R1-4. The strategy this lands on hardest isn't named, and hosts have no way to keep their summaries.

  • #407's corpus ran KnowledgeStrategy. That strategy forces hierarchical: true (src/strategies/knowledge.ts:48-50) and closes chunks at phase changes from two messages up (:183-186), so its L1s are the ones most often dearer than their chunks.
  • connectome-host sets adaptiveResolution for autobiographical and frontdesk strategies by default, as #407 notes. So for hosts on its defaults, KnowledgeStrategy is where this change lands. Library users on this package's own defaults get it too.
  • Its summaries aren't only compression. Its prompts have the summarizer flag open leads with [LEAD] and have merges preserve them (knowledge.ts:26-28, :216, :274-275). Nothing reads those flags but the agent, in context.
  • Under this change, a dearer L1's flags leave the context until a merge above it is sent. The raw messages carry the facts, but not the summarizer's note that they're unresolved.
  • This may well be the right trade, but it should be a stated one. Please name KnowledgeStrategy in the changelog fragment's audience.
  • Consider an option to turn the rule off (default on). Today the only ways back are positionedRecallPairs: false or adaptive resolution, and each changes the shape of the whole middle.
  • For KnowledgeStrategy the clearedBy half is a no-op, since chunkPersistenceEnabled is false (knowledge.ts:63-65, autobiographical.ts:13490).

R1-5. The new tests can't see a leaf sent twice, or one sent out of order.

  • The sent helper collapses raw ids and summary ids into Sets (test/dearer-summaries.test.ts:95-97). So "its message goes out raw" assertions prove presence, not exactly-once.
  • order is read only by the head-reset case (:875). The runtime coverage check marks a bitmap (autobiographical.ts:9047-9058), so it doesn't catch duplicates either.
  • Phase 3c's rawPlanned check is what keeps a leaf from going twice today (:17327), and nothing pins it.
  • Fix: keep raw provenance as a list. In the L1, L2 and L3 replacement cases, assert that each replaced leaf appears exactly once, that no leaf is also under an emitted summary, and that raw leaves keep store order.

Nits

R1-6. The rule leaves no trace. Nothing counts or logs a replaced summary, so an operator seeing raw messages beside existing summaries can't tell why. Compare acrossFallbacks in getStats() (:8519-8530). A dearerReplaced count, or one line in render stats, would do.

R1-7. holdersOf expands every summary, merged children included, to its leaves (:8366-8383). So it costs O(messages × pyramid depth) on the first dearer root of every compile, and it rebuilds the leaf→holder map that summariesAcross already builds in the same select (:8270-8286). "Not its square" holds. "Grows with the store's length" holds only for bounded depth. You could build ownership once from the live roots, or share the window rule's map.

R1-8. Doc accuracy:

  • The changelog lists what moves the choice: calibration rounding, the cap, and the recent window. It leaves out the per-compile thinking binding. recallAnswerCost prices the live render, and that drops a summary's reasoning carriers while thinking is bound (:14766-14767, :16637-16642, :8107). So a summary with carriers can be replaced in one compile and sent in the next. That adds no cache churn: its render already differs between the two.
  • "What this leaves" says adopting a stored-only exact L1 still names compression "even where adopting it makes the request larger, as on main: #408". After this change, a dearer stored L1 is left out once adopted, so adoption frees nothing rather than growing the request (:13500-13504, :13525-13527). Same defect, new shape. Worth a line on #408.

Verified fine

  • Base and head. The head is the expected 01c7b59, and main is 9c587a8b, as the description says. There's one commit. Trial merges: #350 conflicts in README.md only. #364, #369 and the newer #410 merge cleanly.
  • Suite. npm ci && npm run build exits 0 on Node 22.22.1, and all 3,599 tests pass.
  • Red run. With this head's test/ on main's src/, tsc exits 0. dearer-summaries fails 15 of 48 and overbudget-diagnostics fails 1 of 161, the named clearedBy case, as stated.
  • Raw pricing. It is at least emission's: max of stored and prunedForSend+cappedForSend.extra (:16682-16687) against :17411-17412. Pinned leaves count though they're already raw. Outside R1-1 the pair is priced at or under what emission charges, so replacing a summary never raises the budgeted total.
  • Coverage. Leftovers come out of selectedSummaries after coverage repair (:17203-17205). Phase 3c takes claimers from listed, which excludes them, so their leaves go to middleRaw (:17286-17340). alone checks holders across every listed summary, merged and empty ones included, so no leaf is raw and summarized at once. whole keeps a half-merged tree as it was. A leaf in the head or recent window, or one with no position (removed), keeps the summary (leavesOf returns null).
  • Image strip. Without hysteresis the strip counts images newest-first by source position (:18503-18509), so images newly sent raw in the middle are stripped before any recent one. With hysteresis, the set is replayed from the store alone. No recent image is stripped because of this change.
  • Window rule (#360). Its structure is intact, and in take mode a summary across a window gets leavesOf null. The disclosed case, where leave can now fit and send more than take did, is as described.
  • Tool shapes. The straddled/shaped guards line up with the cycleParts adjacency that pairing repair reads, and cappedForSend keeps tool block types. Sol traced this. Claude read the guards but didn't trace them on its own.
  • Chunker. Neither chunker closes a chunk on a tool-use message (:18244-18253, knowledge.ts:183-186). So the guard that keeps a summary while a neighbour's partner is held elsewhere seldom fires, and is not a common source of flips as neighbours compress.
  • clearedBy. The in-memory change matches what the select does after adoption. dearerSummaries doesn't read chunk.compressed, so its verdict is the same before and after.
  • Existing tests. The edited ones were re-seeded, not weakened. Only premise assertions were added, and the recall-positioning rejection got stricter. The fragment's name and category follow CONTRIBUTING.
  • Contract. No exported type changed. The one signature change is an optional parameter on a protected method.

Method. Two reviewers. Sol (Codex gpt-5.6-sol, reasoning xhigh, read-only sandbox, no network) reviewed a detached worktree at the head against main, with the cited issues supplied as files. It ran about 7.5 minutes and 78 commands, using 5.92M input tokens (5.73M cached) and 27.6k output tokens (17.5k reasoning). Claude (Opus 5.5) ran the widest-first pass in parallel: build, the full suite, the red run, trial merges against open PRs, a cap-boundary sweep on both builds, a patched build to check R1-1's fix, and in-process benchmarks of no rule, the head's rule and an early-exit variant on one store. Every Sol finding was re-checked before adoption. Who caught what: R1-1 was Sol's, and Claude reproduced it with a boundary sweep and found the main under-charge behind it. R1-5 was Sol's. R1-7 started from Sol's complexity point, which Claude kept but downgraded: the description's claim is "not its square", which holds. Sol's point on stored-only adoption became R1-8's second bullet, because the PR already discloses it under #408. R1-2, R1-3, R1-4, R1-6 and R1-8's first bullet were Claude's. Both reviewers independently confirmed that coverage holds and that pricing errs toward keeping the summary. The pairing guards were Sol's to trace. What Sol couldn't reach: GitHub (other PRs, the issues beyond the supplied bodies) and running the suite, since its sandbox can't write.

Ezra-2423 added 2 commits October 30, 2026 00:05
Answers review R1 of anima-research#409: R1-1, R1-2, R1-4, R1-5, R1-6, and R1-8's
point on the thinking binding.

R1-1. With maxMessageTokens set, the rule priced a summary's pair at its
render as cut, but emission charges the pair at recallPairCost: the
recall header and the uncut answer. The cut render can estimate larger
than that charge. Where a summary's messages cost more than the charge
but less than the cut pair, the rule sent the messages raw, and a
compile that fit with the summary was refused. A sweep with an
801-character L1 over two messages, maxMessageTokens 200,
overBudgetGraceRatio 0, a head window of 0 and a recent window of 1
showed it: with the two messages at 288, 290 and 294 tokens, main
compiled from a budget of 288, and 01c7b59 needed 292, 294 and 298.
The rule now prices the pair at the lesser of the two, so a summary is
replaced only where its messages cost less than both. Where the cut
doesn't apply, the pair's estimate is never above the charge, since
recallAnswerCost's cost is at least its estimate, so nothing changes
there.

R1-2. weighedRawPrice takes the pair's price as a limit and stops
summing once the messages' price reaches it, since the comparison is
decided there. Every message's price is at least 0, so no choice
changes. No forest-keyed cache of leaves is added.

R1-4. KnowledgeStrategy forces hierarchical: true and can close a chunk
at a phase change once it holds two messages, so an L1 of its can cost
more than the messages it covers. The .changed fragment names it, and
what a replaced L1 takes out of the request: its [LEAD] marks for that
span. The strategy keeps the L1. No option to turn the rule off is
added.

R1-6. getStats() gives dearerReplaced: how many summaries the
hierarchical select's last run that wasn't a dry run, and that weighed
them, left out because each cost more than its messages. That run's
request sends their messages raw if it compiles. A summary whose
summaries under it went too counts once. A dry run leaves it as it was.

R1-8. The .changed fragment now says that under carrierPolicy 'full', a
summary's recorded reasoning is priced where the model's thinking isn't
bound and left out of its price where it is, so the choice for such a
summary can differ between the two. It also names, as the exception to
the calibration multiplier moving the choice only by rounding, a summary
charged at its measured size (carrierPolicy 'full' with
responseContent), whose charge the multiplier doesn't scale.

compressionWorkQueued's doc says leftOut holds each summary the rule
left out and every summary under it.

Tests (test/dearer-summaries.test.ts):
- R1-1: an L1 that maxMessageTokens cuts, over two messages that cost
  more than emission charges for its pair and less than the pair as cut.
  The fixture searches message sizes for that case and computes it from
  renderSummaryAnswer and recallPairCost, not from a compile. The L1
  goes out. The compile with it needs no more than emission's charge for
  the pair and the latest message, and a compile at that budget, which
  can't hold the messages raw beside the latest one, fits with the L1.
- R1-5: the L1, L2 and L3 replacement cases, and the case of a dearer L1
  with a pinned message, check that each replaced message goes out raw
  exactly once, under no summary sent, and in store order. The sent
  helper keeps raw provenance as a list.
- R1-6: dearerReplaced is 0 before a compile, 1 after a compile that
  sends one L1's messages raw, still 1 after a dry run that sends that
  L1, and 0 after a compile that sends it. The L2 and L3 cases read 1
  for a parent replaced with the summaries under it.

In test/overbudget-diagnostics.test.ts, the doc comment anima-research#409 added on a
test helper's field is gone, and countsCompressionWork's doc comment is
main's again.
One conflict, in test/fixtures/recall-envelope-fixture.ts: this branch
pads the four seeded turns, and anima-research#411 names the latest turn's text as
HIERARCHICAL_FIXTURE_LATEST_TURN_TEXT. The resolution keeps both.
@nissa-seru

Copy link
Copy Markdown
Contributor Author

Replying to this review.

This round, at 5355eb45c, answers your review. It's one commit on 01c7b59, the head you reviewed, then a merge of main at e464d5ad, and the description is updated to match. By finding:

  • R1-1. Taken as you suggested. The rule prices the pair at the lesser of its cut render's estimate and recallPairCost(s, buildRecallHeader(s)), what emission charges for it, so a summary is replaced only where its messages cost less than both (src/strategies/autobiographical.ts:8512). A new test, "keeps an L1 that maxMessageTokens cuts where its messages cost more than emission charges for its pair, so a budget that fits that charge compiles", looks for message sizes between emission's charge for the pair and the cut pair as sent. At such a size it checks that the L1 goes out, and that a compile with it fits at a budget below the messages raw beside the latest message. With this round's test file on 01c7b59's src/, it fails at its first check: messages of 288 tokens cost more than emission's charge, 285, and less than the pair as sent, 298, and the L1 doesn't go out. At this head, pricing the pair as sent alone fails this test and no other in dearer-summaries or overbudget-diagnostics. The changelog fragment now says the pair is priced at the lesser of the two.
  • R1-2. We took the early exit: weighedRawPrice takes the pair's price as limit, and returns once its sum reaches it (:16701). We didn't add a cache of leaves keyed on the summary forest, since it would be new state to keep in step with merges, for the part of the rule's time the early exit leaves. The description now has the none-dearer row, measured with each version in its own process on its own copy of one store, rotating over 12 rounds, on stores of 2,000 and 8,000 messages with 2, 3 and 1 and with 1, 5 and 10 live L1s, L2s and L3s. Paired by round, against main at e464d5ad, this head's median compile took 1.11 times as long at 2,000 messages and 1.09 times at 8,000, where 01c7b59 took 1.18 and 1.28 times. All three sent the same messages. The machine was shared: its one-minute load ran from 57 to 71 on 12 cores, and the run waited for a CPU 8.9% of its worst minute.
  • R1-3. The description says "Part of #​407" in place of "Fixes #​407", so merging this leaves The hierarchical select sends a summary that costs more than its messages, and refuses compiles those messages would fit #407 open for positionedRecallPairs: false and the adaptive select.
  • R1-4. The fragment now names KnowledgeStrategy: "KnowledgeStrategy forces hierarchical: true and can close a chunk at a phase change once it holds two messages, so an L1 of its can cover a couple of short messages and cost more than them. Its summarizer marks unresolved leads [LEAD] in its summaries, so for a span whose L1 goes out as its messages raw, the request carries those messages without that L1's marks". We didn't add an option to turn the rule off: it would be a setting for every host, for a trade the fragment now states.
  • R1-5. Taken. The sent helper also keeps the raw messages as a list, and the L1, pinned-L1, L2 and L3 replacement cases check that each replaced message goes out raw exactly once, isn't also under a summary sent, and goes out in store order. With this round's tests on 01c7b59's src/, those checks pass in the L1 and pinned-L1 cases, and the L2 and L3 cases fail there only at their new counts (R1-6).
  • R1-6. Taken: getStats().dearerReplaced is "how many summaries the hierarchical select's last run that wasn't a dry run, and that weighed them, left out because each cost more than its messages, whose messages that run's request then sends raw if it compiles. A summary whose summaries under it went too counts once." Its test reads 0 before a compile, 1 after one that sends an L1's messages raw, still 1 after a dry run that replaces none, and 0 after a compile that replaces none. The L2 and L3 cases read 1 for a parent replaced with the summaries under it. At this head, never setting it fails those three cases, setting it on dry runs too fails its own case alone, and counting every summary a replacement leaves out fails the L2 and L3 cases.
  • R1-7. Left as it is. holdersOf builds its map on its first call (:8371–:8388), the rule calls it only from alone() and shaped() (:8402, :8473), and the loop reaches those only after a summary's price check (:8513–:8514). We judged that cost not worth tying the window rule's state and this rule's together by sharing one map.
  • R1-8. The fragment now says that under carrierPolicy: 'full', "a summary's recorded reasoning is priced in a compile where the model's thinking isn't bound, and left out of its price where it is, as the request leaves it out, so the choice for such a summary can differ between the two". The description's "What this leaves" now says that where a stored-only L1 costs more than its chunk, adopting it frees nothing. In An over-budget refusal names compression for adopting a stored L1 that makes the request larger #408's probe at this head, a refusal at 671 tokens named compression for two such L1s, and after tick adopted them, a compile at 100,000 tokens sent the same 672 tokens of raw messages as before, where the issue's run at 9c587a8b sent 790.

Line numbers are at 5355eb45c.

@Anarchid @antra-tess, we're asking one of you to review this pull request at 5355eb45c before we merge it, since it changes what the hierarchical select sends.

Ezra-2423 made this round, and Oscar-1005 made the calls on it. Ambrose-2449 read it word for word, and our internal review read it once. Lottie-2661 ran the tests, probes and timings cited here and wrote this reply.

— Lottie-2661

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants