diff --git a/CHANGELOG.md b/CHANGELOG.md index a2b40b8b..ee68e811 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,153 @@ Releases up to and including 0.6.2 predate this file; for their contents see ## Unreleased +## 0.22.0 — 2026-10-24 + +### Breaking + +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and that ceiling's limit was over. + - **Cost at a ceiling:** a window held at a ceiling strips its oldest images as new ones come, as under any limit, but from deep in the request, so the prompt cache is rewritten from there at every new image. At the store's default price of 1,600 tokens an image, a window held at the count ceiling rewrites about 800k tokens of it at each new image. The autobiographical strategy's `imageStripHysteresisRatio` strips in batches instead, so the cache is rewritten once a batch, and its line names that option when it's off. + - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. + - **Siblings:** no sibling's current range admits this release; each takes these changes on when it raises its range. + - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. + +### Added + +- `compile()` takes `thinkingBinding` and `identity` (`SelectOptions`) for a model that binds each signed thinking block to the request it was minted in (#155). On an account that enforces the binding, such a model refuses a request carrying a block whose request has changed since, with a 400 ("Invalid `signature` in `thinking` block. The block is bound to a different conversation…"), and with the provider's drop control on it sets the block aside instead. With `thinkingBinding.binds`, a compile leaves such blocks out. + - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. + - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message its last committing compile with `binds` sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. + - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. + - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. + - **A dry compile** reports what it would leave out and writes no record. + - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent and its stream noted keeps its thinking, and nothing past the record's end does. + - **A record the store can't read or write** is logged and never fails a compile or a note. One it can't read is read as none, so a committing compile with `binds` writes its own in its place; when it can't write a compile's record, the record before it, if any, stands. + - **While a request is streaming,** compile for its branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing, so its replies go without their thinking. + - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until a compile writes it again; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile with `binds` (a compile without it leaves the record, so a later note still writes), and on a branch with no record or one it can't read, or when the store can't write it. +- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the compile that wrote the branch's record, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. +- `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. +- On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. + +- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They count images as the window's limits do. They don't bound a whole request: an image inside an opaque provider item, and anything a caller injects at compile time or appends to a compiled request, aren't counted. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `ContextStrategy.imageCeilings()`, which both strategies implement, reports the two ceilings as the strategy applies them, a malformed one at its default. It reports configuration, not usage: a caller that wants a request within the ceilings counts that request's images itself. `ImageCeilings` is exported. + +- System changes: a message holding only membrane's `system_change` blocks + (`SystemChangeContent`, membrane's release that adds them) changes the tools + and system text the model is offered from its place on, and every strategy + compiles it unaltered, in its place among the raw messages. (Under the + autobiographical strategy's `positionedRecallPairs: false`, the middle's + summaries go ahead of its raw messages, as they always have.) The + autobiographical strategy, and with it `KnowledgeStrategy`, keeps it raw as + a pin keeps a message, so no fold takes it and no chunk holds or spans it; + after a head-window reset that leaves messages in the head window, the + changes stored before the new head are part of the head, ahead of the head + window, in order. `PassthroughStrategy` and + `WindowedPassthroughStrategy` keep one that falls before their window, + ahead of the window and priced into it, and the windowed strategy sizes a + re-anchored window as its fraction of the room the changes leave. None is + dropped: when the changes alone exceed the budget (beyond the + autobiographical strategy's grace, and on its hierarchical path only where + `enforceBudget` is on), every strategy refuses with `OverBudgetError`, + `clearedBy` empty, until the caller consolidates them into `tools` and + `system`; a dry compile reports where the strategy's other refusals do, + which for the passthrough strategies is nowhere. They also refuse when the + newest conversation message doesn't fit beside the changes, naming + `newer-message` while they leave room for one. + Compression never reads one. It's priced from its JSON. A write refuses one + beside other blocks or inside a tool result, and one whose `removeTools` or + `addSystem` isn't an array of strings, or whose `defineTools` isn't an array + of tool definitions, each with a non-empty string `name`, a string + `description` and an object `inputSchema` whose `type` is `'object'`; a read + leaves such a change out. A write also refuses a change while one of the + nearest calls before it has no result between it and the change (counting + back over other changes and over results, by id, to the nearest message + holding a call, every shard of it), + a tool result answering a call that a change already follows, and an edit + giving a message a `tool_use` that a change follows before its results: + membrane sends a change there after the result anyway, so write it after + the result. Ingress sharding never splits a message of them, since it holds + no text block. + Only membrane's `NativeFormatter`, on an adapter that sends system changes, + sends one; elsewhere membrane refuses the request. Rolling back to a + release without them isn't supported once a store holds one. +- `CompileResult.firstChanged`, optional: on a committing compile, the index + of the first compiled message, injections included, whose participant or + content bytes differ from the last committing compile's message at that + index, or that has none there, so after appends alone it is the first + appended index; an injection placed beside the newest user message moves + with each new user message, so after one it's that injection's old index. + It compares bytes, so it holds across a branch switch or a new strategy, + and it's `null` only for the manager's first committing compile. A dry + compile leaves it out and doesn't move it. It's compared when read, + so a compile whose caller never reads it serializes only its injections. + +### Changed + +- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and the limit that ceiling sets was over: at an image the render replaces, or, with `imageStripHysteresisRatio`, where it brought a trim on anywhere in the rule's replay of the history. The depth limit or a limit the host set may be stripping images there too. +- `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. + +### Fixed + +- A compression prompt carries at most `imageCeiling` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `imageByteCeiling`, and the line it logs when it replaces images names the count ceiling and the images kept. + +- A summarizer request no longer carries an empty turn, or a `---` separator + with nothing after it, where a message left empty for it sat, as one of + thinking alone was: merging consecutive messages passes over an empty one, + and its neighbours join as they would without it. +- With the head window disabled (`headWindowTokens: 0`), a compile after + `resetHeadWindow` under `adaptiveResolution` no longer sends history from + before the reset twice, and under kv-unified no longer throws + `CanonicalForestError` (`duplicate-chunk-id`) for it: the middle's ranges + overlapped each other, and the recent window when it starts before the + reset. +- A `WindowedPassthroughStrategy` compile that refuses no longer moves the + window's anchor; only a compile that sends does. So a retry with a larger + budget still sends the older turns that fit. +- A `KnowledgeStrategy` chunk no longer runs into or across an L1 summary's + span once a removal moves where its walk closes, which left the chunk after + it uncompressed for good and its messages sent raw beside the summary: a + chunk now closes where the L1 summary covering its messages changes, except + between a call and the result right after it. + +## 0.21.0 — 2026-10-24 + +### Added + +- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them), and the message ends with a sentence naming each pin and the tokens, and saying that whoever manages pins can unpin them, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent, and it comes last and under 140 characters, so a host that quotes a long message by its head and its last 140 characters, as agent-framework does, keeps it whole. Each pin's tokens are what it keeps raw, a part locked with `lockChunk` included, so they're what unpinning frees at most; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. + +### Fixed + +- **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. + - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. + - **Marks the summarizer's budget can't hold.** The parts of a marked message that no chunk held when it was marked stay out of compression while the mark keeps them raw. From the head window's end on, the compression requests after them carry them whole and don't price them against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made as a long message arrives leaves every part not yet chunked this way. Where such parts together come to more than that budget, a summarizer whose window is smaller would refuse those requests, and unless `compressionSourceOnly` or `compressionSourceOnlyFallback` is set, compression would stop making progress for that store. Unpinning the marks lets later requests leave them out; a chunk already refused in three request shapes can stay quarantined after that, and `clearCompressionRefusalQuarantine` clears it. + - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. + +## 0.20.1 — 2026-10-23 + +### Added + +- `AutobiographicalStrategy.setLoadRepairHandler(fn)` tells a host when a load's repair of the summary state drops summaries holding text, so whoever keeps the agent can learn that its memory changed from more than a log line (#175). A load runs at each initialization (an open, a branch switch, a fork, `setStrategy`) and at each take-up of what another instance on the store wrote. The handler gets one `LoadRepairReport` per such load. The report holds no summary text. For each dropped summary that held text, it gives the id, the level and the stored `tokens` (each null where it isn't a number), why it was dropped (`unreadable`, `names-missing`, `names-not-below` or `names-non-child`), and, where its range can be read, the messages it covered, with each end's stored time where the strategy's message view gives it. It also gives the sum of those tokens, how many empty summaries the same repair dropped, the namespace, and where the old summaries read back from: `getStateJsonAt(stateId, sequenceBefore)`, with `branch`, the Chronicle branch the repair wrote on, current. A repair that drops only empty summaries, or none, makes no report, and the repair's diagnostic line is written as before. A report made while a handler is set goes to it once its load's synchronous run has ended, never during it. Reports made while no handler is set, such as an open's where the host sets the handler after opening, are held in memory and handed to the handler, oldest first, when it's set. Each strategy instance holds its own. A handler that throws, or whose returned promise or thenable rejects, is logged, and stops neither the load nor the next report. `getLatestLoadRepairReport()` gives the newest report the instance has made, whether held or handed on, for a health surface to show. A load that drops text makes no report only where the store's branch or sequence can't be read, which is logged. + +- `CompileResult.renderStats`: each compile's result carries the render stats its own select recorded. A dry compile's (`dryRun`) are its own, which `getRenderStats()` never returns, so a host can show the composition of the request a dry compile built, beside the last live compile's, without sending anything. A live compile's are the object `getRenderStats()` then returns. Strategies supply them through an optional `RenderStatsCapableStrategy.getLastSelectStats()`; a strategy without it gives compiles none. + +### Changed + +- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). A source-only request (`compressionSourceOnly`) recalls none, before or after. + - **Before:** it recalled every summary on the frontier, as 0.15.0's entry said: "Only pins before the chunk being compressed count, so a summary after it is recalled as before." So a chunk compressed after later history had been summarized was given those later summaries too, ahead of it. That's the old head's chunk, which a head-window reset makes compressible late, or a pinned or quarantined chunk let go late. + - **Now:** a summary that starts after the history being summarized isn't recalled, and its messages stay out of the request, as before. Pins still count only before the chunk. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. When a refused request is retried with a recalled summary above L1 replaced by its children (the recall-curve fallback), a straddling summary's later child is recalled too: the same later part, as a summary of its own. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. + +### Fixed + +- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request for history from before the reset no longer gets the new topic's opening as its head, ahead of the history it summarizes (#317). A source-only request (`compressionSourceOnly` or `compressionMergeSourceOnly`) carries no head, so this didn't reach it. The other way later history reached a compression request, its recall pairs, is under Changed. + - **The head it gets now:** the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. + - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. + - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. + - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. + - **Unchanged:** what every compile selects and sends, and every summary already written. What an over-budget refusal names as clearing it (`clearedBy`) follows tick's deferral, which reads the request's head. + - **Prompt cache:** a request for older history after a reset carries on the prefix it had before the reset, where it used to restart with the new topic, and restart again each time the live head grew. + ## 0.20.0 — 2026-10-23 ### Breaking diff --git a/README.md b/README.md index 1f125ebc..ff9d1470 100644 --- a/README.md +++ b/README.md @@ -84,18 +84,24 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a pin or mark on any part of a long message stored in parts covers the whole message, in what the agent is sent, in compression and in the summarizer's prompt alike, less a part locked with `lockChunk` outside the span its ids name. As with any pin, a chunk recorded before it keeps its messages, so compression can still summarize those parts while the agent is sent them raw. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at. A compile whose budget can't hold the raw pins beside the rest refuses with `OverBudgetError`; where the adaptive plan's middle is over the budget, its message and `diagnostics.rawPins` name each pin keeping raw tokens in that middle. `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. - `getRenderStats` reports how much of the last compile was raw and how much was summaries, with counts for levels 1 to 3. It's `null` until a live compile has run since the strategy loaded its branch, which opening the store, `setStrategy()` and a branch switch each do; a dry compile or preview leaves it as it was. Each compile's result carries its own as `renderStats`, so a dry compile (`dryRun`) shows the composition of the request it built without sending it. +For a model that binds each signed thinking block to the request it was minted in, pass `{ thinkingBinding: { binds: true, dropBlock, participant }, identity }` as compile's third argument, with `participant` the name the model's own replies are stored under and `identity` digests of the request's system text, tools, model and the rest of its head. The compile then leaves out the thinking such a model can no longer take, against a per-branch record of the last request the branch sent, and says what it newly left out in `thinkingStrip`. Once the branch has a record, a reply keeps its thinking at the compile after it's stored only if the host has noted it with `noteRequestSent(rows, identity, compileId)`, naming the `provenance.compileId` of the compile its stream was built from: as it sends each later round of a stream, the ids of the stored messages the request carried after the compiled ones, every round's so far, in order, and once the stream has ended, those and its last reply's, with `{ unstored: true }` for a row it can't vouch was carried or returned as stored. A host that doesn't call it loses each reply's thinking at the compile after it's stored, and is never refused for it. While one of a branch's requests is still streaming, compile for that branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing (see `SelectOptions.thinkingBinding` in `src/types/strategy.ts`). + Every option is documented on `AutobiographicalConfig` in `src/types/strategy.ts`. ## Branches Chronicle keeps the store's history, so a conversation can branch. `branchAt(messageId)` creates a branch as of a message, and `switchBranch(name)` moves to it. `fork()` creates a branch at the current head and switches to it. The strategy's state, summaries included, follows the branch. +## Observing a manager's work + +Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. Outside that is a returned promise no JavaScript can give a handler, such as one whose `constructor` or species can't be used, or a proxy around one: its rejection goes unhandled, which ends a Node process by default. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. + ## Inspecting and repairing stores Among other tools, `scripts/` holds ones for inspecting and repairing stores. For example, `inspect-store.ts` writes out a store's messages, summary tree and resolutions, `audit-topology.ts` checks that each summary covers a contiguous run of messages, and `repair-topology.ts` plans a repair for one that doesn't, which it applies with `--apply`. Opening a store takes its lock and can write to it, so run them on a stopped agent's store or on a copy. `scripts/README.md` shows how to compile a script and describes several of them. diff --git a/changelog.d/175-load-repair-report.added.md b/changelog.d/175-load-repair-report.added.md deleted file mode 100644 index a7de72eb..00000000 --- a/changelog.d/175-load-repair-report.added.md +++ /dev/null @@ -1 +0,0 @@ -- `AutobiographicalStrategy.setLoadRepairHandler(fn)` tells a host when a load's repair of the summary state drops summaries holding text, so whoever keeps the agent can learn that its memory changed from more than a log line (#175). A load runs at each initialization (an open, a branch switch, a fork, `setStrategy`) and at each take-up of what another instance on the store wrote. The handler gets one `LoadRepairReport` per such load. The report holds no summary text. For each dropped summary that held text, it gives the id, the level and the stored `tokens` (each null where it isn't a number), why it was dropped (`unreadable`, `names-missing`, `names-not-below` or `names-non-child`), and, where its range can be read, the messages it covered, with each end's stored time where the strategy's message view gives it. It also gives the sum of those tokens, how many empty summaries the same repair dropped, the namespace, and where the old summaries read back from: `getStateJsonAt(stateId, sequenceBefore)`, with `branch`, the Chronicle branch the repair wrote on, current. A repair that drops only empty summaries, or none, makes no report, and the repair's diagnostic line is written as before. A report made while a handler is set goes to it once its load's synchronous run has ended, never during it. Reports made while no handler is set, such as an open's where the host sets the handler after opening, are held in memory and handed to the handler, oldest first, when it's set. Each strategy instance holds its own. A handler that throws, or whose returned promise or thenable rejects, is logged, and stops neither the load nor the next report. `getLatestLoadRepairReport()` gives the newest report the instance has made, whether held or handed on, for a health surface to show. A load that drops text makes no report only where the store's branch or sequence can't be read, which is logged. diff --git a/changelog.d/323-calibration-request-overhead.fixed.md b/changelog.d/323-calibration-request-overhead.fixed.md index 7c06e8fb..68050e01 100644 --- a/changelog.d/323-calibration-request-overhead.fixed.md +++ b/changelog.d/323-calibration-request-overhead.fixed.md @@ -5,16 +5,34 @@ estimate doesn't, so a tool-heavy agent's multiplier learned its schemas as a per-token rate and folded history the budget had room for, or sat at 1.00 rejecting every sample as out of band. A caller that passes nothing - gets the old comparison. + gets the old comparison, plus the compile's message-position injections + (below). - With the overhead declared, three out-of-band calibration samples in a row - whose implied multipliers agree within 10% are learned, outright, at the - clamp edge, instead of being rejected for good. At 1.00 the ratio and the - implied multiplier are the same number, so a store whose true ratio was - past 1.8, where every pricing-epoch discard restarts, could never learn. - Undeclared samples are rejected as before, and the rejection line says the - caller may be leaving out the system prompt and tools it sends. + whose implied multipliers agree within 10% of the first are learned, + outright, at the clamp edge, instead of being rejected for good. At 1.00 + the ratio and the implied multiplier are the same number, so a store whose + true ratio was past 1.8, where every pricing-epoch discard restarts, could + never learn. Undeclared samples are rejected as before, and the rejection + line says the caller may be leaving out the system prompt and tools it + sends. +- The sample also counts the compile's message-position injections + (`beforeUser` and `afterUser`), which the request carries and the window + estimate doesn't: `ContextManager.compile` passes their content to the + select as `SelectOptions.messageInjections`, and the select prices it with + the window. Left out, an agent whose injections are large beside its window + learned them as a per-token rate, or, with the overhead declared, at the + clamp. - **Host operators:** a multiplier learned before the caller declared its - overhead holds that overhead, and steps back down over about ten compiles - once samples declare it. To relearn in one sample, seed it as provisional - on the stopped store: `setStateJson('/autobio:calibration', - { multiplier, at, pricing, provisional: true })`. + overhead holds that overhead. Inside the calibration band, the first two + samples that declare it replace the multiplier with their mean when they + agree within 10%, and the EMA steps as before when they don't; either way + the record then carries `declaredOverhead: true`, so later declared + samples step by the EMA. (Out of the band, the three-sample run above + applies.) The first of the two is held in memory, so a restart between + them costs one more sample. An undeclared sample that moves the multiplier + writes the record without the flag. A store whose multiplier, above 1, + puts every compile over budget gets no sample to move it, as before: to + unwedge it, seed the record as provisional on the stopped store, and the + refused compile drops it to 1: `setStateJson('/autobio:calibration', + { multiplier, at, pricing: , provisional: true })`. + `RequestOverhead` is exported from the package. diff --git a/changelog.d/328-fresh-calibration-provisional.fixed.md b/changelog.d/328-fresh-calibration-provisional.fixed.md new file mode 100644 index 00000000..55485a4b --- /dev/null +++ b/changelog.d/328-fresh-calibration-provisional.fixed.md @@ -0,0 +1,10 @@ +- A store with no calibration record, or a record with no finite + multiplier, starts from a provisional 1, as a discard's restart does + (#328). Its first sample inside the calibration band sets the multiplier, + declared or not: one sample, without the two-sample hold. Learned, a 1 + stepped a fifth of the way per undeclared sample (nine applied changes, + each a reprice of every message and summary, to reach a true 1.4), and + took two declared samples, the first held. Until that + sample, render stats report `calibration.provisional: true`. A record + from a newer pricing epoch, or a read of the record that throws, still + loads 1 as before, not provisional. diff --git a/changelog.d/compile-render-stats.added.md b/changelog.d/compile-render-stats.added.md deleted file mode 100644 index 11342502..00000000 --- a/changelog.d/compile-render-stats.added.md +++ /dev/null @@ -1 +0,0 @@ -- `CompileResult.renderStats`: each compile's result carries the render stats its own select recorded. A dry compile's (`dryRun`) are its own, which `getRenderStats()` never returns, so a host can show the composition of the request a dry compile built, beside the last live compile's, without sending anything. A live compile's are the object `getRenderStats()` then returns. Strategies supply them through an optional `RenderStatsCapableStrategy.getLastSelectStats()`; a strategy without it gives compiles none. diff --git a/changelog.d/context-observer.added.md b/changelog.d/context-observer.added.md new file mode 100644 index 00000000..ce284195 --- /dev/null +++ b/changelog.d/context-observer.added.md @@ -0,0 +1,38 @@ +- `ContextManagerConfig.observer` (`ContextObserver`): a host is told, as each + happens, of every compile, of each refusal context-manager itself makes of + compression, and of each calibration sample. The manager hands the observer + to its strategy in `StrategyContext.observer`. Each callback is optional and + called synchronously. What it throws or rejects never reaches the work: the + manager logs a callback's first failure once through its diagnostics and + drops the rest, so a compile's result is the same with an observer as + without one. Outside that is a returned promise no JavaScript can give a + handler, such as one whose `constructor` or species can't be used, or a + proxy around one: its rejection goes unhandled, which ends a Node process + by default. A record's + error and render stats are the objects the caller gets, so an observer + treats them as read-only. + - `compile(record)`, once per `compile()`: the strategy, `dryRun`, + `startedAt` on the wall clock (`Date.now()`), the whole compile's + `durationMs` on the monotonic clock, `outcome` (`'compiled'`, `'refused'` + for one of `COMPILE_REFUSALS`, or `'failed'`), the thrown `error` itself, + and a compiled compile's `renderStats`, the object its result holds. + - `compression(record)`: a `reason` from `COMPRESSION_REFUSALS` + (`'admission-rejected'`, `'topology-violation'`, `'empty'`, `'incomplete'`, + `'malformed'`, the split rung's caps, `'chunk-call-cap'` and + `'window-call-cap'`, and `'overlap-guard'`, a chunk refused because an L1 + already covers part of it). A provider's refusal or error on a compression + call isn't one of these. + - `calibration(record)`, once per sample `reportRealInputTokens` takes: an + `outcome` from `CALIBRATION_SAMPLE_OUTCOMES` (`'out-of-band'`, + `'deferred'`, `'held'`, `'confirmed'`, `'replaced'`, `'stepped'`), the + `ratio`, and the `multiplier` after. +- `OverBudgetError.stageId` names which stage refused, from the exported + `OVER_BUDGET_STAGES`, so a host needn't parse the message. The library's own + strategies give it through the constructor's new `stageId` option, whose + words open the message when no `stage` is given. The constructor's `stage` + is unchanged: a strategy's own words, which open the message exactly as + before. Given words alone, `stageId` reads `'strategy-defined'`, a new id; + given a `stageId` beside them, it reads that id; given neither, it's + `'picker-exhausted'`. Every message reads as before, and + `AutobiographicalStrategy`'s protected `overBudgetError` is still given each + stage's words as `stage`. diff --git a/docs/adaptive-resolution-design.md b/docs/adaptive-resolution-design.md index 00d5a930..f92f74ea 100644 --- a/docs/adaptive-resolution-design.md +++ b/docs/adaptive-resolution-design.md @@ -373,6 +373,14 @@ This case should be rare in practice with unbounded L_n: the picker can almost a > pressure, but any such eviction that would silently lose content instead > **refuses the turn** (`UncoveredDropError`) — pressure never ships silent > loss. The philosophy stands: surface the overage, let the host decide. +> +> *(Raw pins)* The refusal of a plan whose middle is over the budget names +> each pin that keeps raw tokens there (`OverBudgetDiagnostics.rawPins`), with +> the tokens it keeps. Under adaptive resolution a pin on part of a message +> stored in parts covers the whole message, so a mark on a long message can +> be what doesn't fit: whoever manages the pins can unpin it, which lets +> compression fold it, or whoever sets the budget can raise it, where the +> model's window allows. ### 3.11 Recent-window slide diff --git a/package-lock.json b/package-lock.json index bfe407bd..8d141d99 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.22.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.22.0", "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", diff --git a/package.json b/package.json index 011bbc36..de4c1c2e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.22.0", "description": "Context management layer for LLM-based agents using Chronicle and Membrane", "type": "module", "main": "dist/src/index.js", diff --git a/src/adaptive/index.ts b/src/adaptive/index.ts index ea7311d3..d22dabac 100644 --- a/src/adaptive/index.ts +++ b/src/adaptive/index.ts @@ -151,6 +151,7 @@ export { accountFrontier, type OverBudgetClearedBy, type OverBudgetDiagnostics, + type OverBudgetRawPin, type PickerInputs, type PickerResult, type PickerChunk, diff --git a/src/adaptive/picker.ts b/src/adaptive/picker.ts index 4cb403a7..075b7b03 100644 --- a/src/adaptive/picker.ts +++ b/src/adaptive/picker.ts @@ -51,14 +51,47 @@ export interface OverBudgetDiagnostics { reserveForResponse?: number; /** Configured window minus the response reserve, before over-budget grace. */ inputBudget?: number; + /** + * The pins that keep messages raw in the adaptive plan's middle, which + * compression would otherwise fold: outside the head and recent windows. + * `tokens` is the raw tokens they keep there together, estimated as the + * plan prices them, and each pin gives its own; a message two pins cover + * counts once in `tokens` and toward each pin. Set only by the refusal of a + * plan whose middle is over the budget ("Adaptive picker exhausted but …"); + * every other refusal leaves it out, as does that one when no pin keeps + * any. + */ + rawPins?: { tokens: number; pins: readonly OverBudgetRawPin[] }; +} + +/** A pin that keeps messages raw in an over-budget refusal's middle (`OverBudgetDiagnostics.rawPins`). */ +export interface OverBudgetRawPin { + /** The pin's id, as `unpin` takes it. */ + pinId: string; + kind: 'pin' | 'document'; + /** Its name, when it was given one. */ + name?: string; + firstMessageId: string; + lastMessageId: string; + /** + * Under adaptive resolution, how many stored messages the pin covers when + * it covers more than its ids name: a pin on part of a message stored in + * parts covers that message's parts, less a part locked with `lockChunk` + * outside the span its ids name. Left out otherwise. + */ + parts?: number; + /** Raw tokens the pin keeps in the middle. */ + tokens: number; } /** * What clears an over-budget refusal at run time besides a larger budget, - * which clears any of them: waiting, or a hot setting. Cold configuration, + * which clears any of them but the structural repair's (a newest turn the + * repair dropped, which no budget brings back): waiting, or a hot setting. Cold configuration, * such as the head window's size, isn't named, and neither is a head-window * reset (`resetHeadWindow`), which clears a head that's over but starts a new - * topic, nor newer messages that would shrink a head holding images, since + * topic (the system changes stored before the new head stay in it, so a + * reset clears none of what they cost), nor newer messages that would shrink a head holding images, since * the image strip takes the oldest images first but may not take enough. * The strategy that refuses sets it, since it knows which part of the * context is over and what can shrink that part: @@ -120,14 +153,17 @@ export interface OverBudgetDiagnostics { * as before, since nothing says what it folds. Whether that work is * enough to fit isn't checked, so a host can tell an agent that * compression may make room, not that it will. - * - `'newer-message'`: the newest message alone is over the budget, and a - * newer message that fits moves the window past it. + * - `'newer-message'`: the newest message, with the system changes sent + * beside it, is over the budget, and a newer conversation message that + * fits beside them moves the window past it (a newer change doesn't). * - `'smaller-recent-window'`: the raw recent window, which compression never * touches, is what's over the budget, and the head fits beside the newest * turn as the compile must keep it raw, or beside everything from a - * compression hold if one starts earlier. That's the newest message (for - * one split into shards, its newest shard, since the earlier shards can - * fold), unless the newest message's body holds a tool_result, on its first + * compression hold if one starts earlier, and beside the system changes + * outside the head, which stay raw wherever the window starts. That's the + * newest conversation message, a system change being none, with the + * changes after it (for one split into shards, its newest shard, since the + * earlier shards can fold), unless the newest message's body holds a tool_result, on its first * shard: then it's that whole body and the whole call before it, which the * result must follow raw. A smaller * `recentWindowTokens` (the hot `tailTokens`) moves the rest out of the raw @@ -140,6 +176,109 @@ export interface OverBudgetDiagnostics { */ export type OverBudgetClearedBy = 'compression' | 'newer-message' | 'smaller-recent-window'; +/** + * Which stage of a compile refused (`OverBudgetError.stageId`), from a closed + * list a host can read without parsing the message: + * - `'picker-exhausted'`: the adaptive picker folded everything it could. + * - `'head-tail-reservation'`: the head and the recent window alone exceed + * the budget. + * - `'emission-overran-plan'`: the emitter spent more than the plan priced + * (planner/emitter estimator drift). + * - `'tail-emission-dropped'`: tail emission would have dropped messages + * reserved for the recent window. + * - `'repair-lost-newest-turn'`: structural repair didn't keep the newest + * turn. + * - `'hierarchical-emission-overran'`: the hierarchical emitter overran the + * budget. + * - `'hierarchical-system-changes'`: the hierarchical emitter's system + * changes alone exceed the budget. + * - `'passthrough-system-changes'`: `PassthroughStrategy`'s system changes + * alone exceed the budget. + * - `'passthrough-changes-and-newest-message'`: its system changes and the + * newest conversation message together exceed the budget. + * - `'windowed-newest-message'`: `WindowedPassthroughStrategy`'s newest + * message alone exceeds the budget. + * - `'windowed-changes-and-newest-message'`: its system changes and the + * newest conversation message together exceed the budget. + * - `'windowed-system-changes'`: its system changes alone exceed the budget. + * - `'strategy-defined'`: a strategy named its stage only in words of its + * own (the constructor's `stage`), which open the message. + */ +export const OVER_BUDGET_STAGES = [ + 'picker-exhausted', + 'head-tail-reservation', + 'emission-overran-plan', + 'tail-emission-dropped', + 'repair-lost-newest-turn', + 'hierarchical-emission-overran', + 'hierarchical-system-changes', + 'passthrough-system-changes', + 'passthrough-changes-and-newest-message', + 'windowed-newest-message', + 'windowed-changes-and-newest-message', + 'windowed-system-changes', + 'strategy-defined', +] as const; + +/** A stage of {@link OVER_BUDGET_STAGES}. */ +export type OverBudgetStage = typeof OVER_BUDGET_STAGES[number]; + +/** A stage context-manager's own strategies refuse at, which has words of its own: what the constructor's `stageId` takes. */ +type LibraryStage = Exclude; + +/** How each of context-manager's own stages opens the refusal's message. Every such stage has words, and nothing else does. */ +const OVER_BUDGET_STAGE_TEXT: Record = { + 'picker-exhausted': 'Adaptive picker exhausted', + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', + 'passthrough-system-changes': 'Passthrough kept only its system changes', + 'passthrough-changes-and-newest-message': 'Passthrough: its system changes and the newest conversation message', + 'windowed-newest-message': 'Windowed passthrough: the newest message alone', + 'windowed-changes-and-newest-message': 'Windowed passthrough: its system changes and the newest conversation message', + 'windowed-system-changes': 'Windowed passthrough: its system changes alone', +}; + +/** + * The options a library site refusing at `stageId` gives: the id, and its + * words as `stage`. AutobiographicalStrategy's sites go through its + * protected `overBudgetError` hook, whose override received the words as + * `stage` in 0.21.0 and still does; the message and the field are as for + * the id alone. + */ +export function overBudgetStage(stageId: LibraryStage): { stageId: LibraryStage; stage: string } { + return { stageId, stage: OVER_BUDGET_STAGE_TEXT[stageId] }; +} + +/** The sentence naming the pins that keep raw tokens in a refusal's middle, or nothing when none do. */ +function rawPinsText(raw: OverBudgetDiagnostics['rawPins']): string { + if (!raw || raw.pins.length === 0) return ''; + const { tokens: total, pins } = raw; + const each = pins.map((pin) => { + const what = pin.kind === 'document' ? 'document mark' : 'pin'; + const label = pin.name !== undefined ? `"${pin.name}" (${pin.pinId})` : pin.pinId; + const span = pin.firstMessageId === pin.lastMessageId + ? `message ${pin.firstMessageId}` : `messages ${pin.firstMessageId} to ${pin.lastMessageId}`; + const parts = pin.parts === undefined ? '' + : pin.firstMessageId === pin.lastMessageId ? `, covering ${pin.parts} stored parts of its message` + : `, covering ${pin.parts} stored messages`; + // A lone pin's own tokens are the total, already given. + return `${what} ${label}, ${span}${parts}${pins.length === 1 ? '' : `, ~${pin.tokens} tokens`}`; + }); + // Several pins can cover one message, so unpinning one of them frees none + // of it: the remedy speaks of them together, whose tokens count it once. + // It says who can act rather than telling its reader to, since a host can + // pass the text on to an agent that can do neither. It comes last and is + // kept under 140 characters, so a host that quotes a long message by its + // head and last quarter, cut at 600 characters, still quotes it whole. + const remedy = `Whoever manages pins can unpin ${pins.length === 1 ? 'it' : 'them'}, freeing up to those tokens; ` + + `whoever sets the hard budget can raise it if the model's window allows.`; + return `. ${pins.length === 1 ? 'A pin keeps' : 'Pins keep'} ~${total} tokens raw in the middle: ${each.join('; ')}. ${remedy}`; +} + /** * Error raised when the picker has folded everything it can and the * resulting context still exceeds the hard token budget. @@ -167,6 +306,13 @@ export class OverBudgetError extends Error { * {@link OverBudgetClearedBy}). Empty when nothing else is known to clear it. */ readonly clearedBy: readonly OverBudgetClearedBy[]; + /** + * Which stage refused (see {@link OVER_BUDGET_STAGES}): the `stageId` the + * thrower gave, `'strategy-defined'` when it gave only words of its own + * (`stage`), or `'picker-exhausted'` when it gave neither. A `stageId` that + * isn't one of the library's counts as none given. + */ + readonly stageId: OverBudgetStage; constructor(opts: { budget: number; @@ -174,9 +320,15 @@ export class OverBudgetError extends Error { diagnostics: OverBudgetError['diagnostics']; /** What clears the refusal besides a larger budget; empty when left out. */ clearedBy?: readonly OverBudgetClearedBy[]; - /** Which stage refused — defaults to the plan-stage wording. Emission - * sites pass their own so a planner/emitter drift refusal doesn't - * masquerade as picker exhaustion (it cost a debugging session once). */ + /** Which of the library's stages refused — defaults to + * `'picker-exhausted'`. Emission sites pass their own so a + * planner/emitter drift refusal doesn't masquerade as picker exhaustion + * (it cost a debugging session once). Its words open the message unless + * `stage` gives words. */ + stageId?: LibraryStage; + /** A strategy's own words for the stage that refused, which open the + * message exactly as given, as in 0.21.0; `stageId` then reads + * `'strategy-defined'` unless the thrower also gave a `stageId`. */ stage?: string; }) { const countLabels = { @@ -196,8 +348,16 @@ export class OverBudgetError extends Error { const depth = opts.diagnostics.deepestLevelKind === 'unplanned' ? 'fold plan=not computed' : `${depthLabel}=L${opts.diagnostics.deepestLevel}`; + // Words open the message exactly as 0.21.0 built it (`stage ?? default`), + // whatever they say. An id names a stage only as the map's own key, so a + // caller past the type who gives another value gives no id. + const words: unknown = opts.stage; + const named: unknown = opts.stageId; + const id = typeof named === 'string' && Object.hasOwn(OVER_BUDGET_STAGE_TEXT, named) ? named as LibraryStage : undefined; + const stageId: OverBudgetStage = id ?? (words != null ? 'strategy-defined' : 'picker-exhausted'); + const opening = words != null ? `${words}` : OVER_BUDGET_STAGE_TEXT[id ?? 'picker-exhausted']; super( - `${opts.stage ?? 'Adaptive picker exhausted'} but ${opts.actual} tokens still exceed hard budget ${opts.budget}` + + `${opening} but ${opts.actual} tokens still exceed hard budget ${opts.budget}` + ` (head=${opts.diagnostics.headTokens}, tail=${opts.diagnostics.tailTokens},` + ` middle=${opts.diagnostics.middleTokens}${middleCount}, ${depth}` + (opts.diagnostics.deepestAvailableLevel === undefined ? '' : @@ -209,7 +369,7 @@ export class OverBudgetError extends Error { (opts.diagnostics.configuredBudget === undefined ? '' : `, configured=${opts.diagnostics.configuredBudget}`) + (opts.diagnostics.reserveForResponse === undefined ? '' : `, response reserve=${opts.diagnostics.reserveForResponse}`) + (opts.diagnostics.inputBudget === undefined ? '' : `, input budget=${opts.diagnostics.inputBudget}`) + - ')' + ')' + rawPinsText(opts.diagnostics.rawPins) ); // Writable per Error convention so instanceof-by-name works across // iframe / vm boundaries; the field stays writable on the prototype. @@ -218,6 +378,7 @@ export class OverBudgetError extends Error { this.actual = opts.actual; this.diagnostics = opts.diagnostics; this.clearedBy = Object.freeze([...(opts.clearedBy ?? [])]); + this.stageId = stageId; } } diff --git a/src/canonical-json.ts b/src/canonical-json.ts new file mode 100644 index 00000000..2428d37b --- /dev/null +++ b/src/canonical-json.ts @@ -0,0 +1,14 @@ +/** + * JSON with each object's keys in one order whatever order they were set in + * (sorted, though integer-like keys come first, ascending, as in any object), + * and each array's items in their own order, so two values that differ only + * in the order their keys were set give the same text. An object the store + * keeps comes back with its keys sorted, while one built in code keeps the + * order it was built in. + */ +export function canonicalJson(value: unknown): string { + return JSON.stringify(value, (_key, inner: unknown) => { + if (inner === null || typeof inner !== 'object' || Array.isArray(inner)) return inner; + return Object.fromEntries(Object.keys(inner).sort().map((key) => [key, (inner as Record)[key]])); + }); +} diff --git a/src/context-log.ts b/src/context-log.ts index 510ce7ba..ed7120c5 100644 --- a/src/context-log.ts +++ b/src/context-log.ts @@ -342,6 +342,9 @@ export class ContextLog { case 'audio': case 'video': return 1000; + case 'system_change': + // Mirrors MessageStore.computeBlockTokensRaw: priced from its JSON. + return this.tokenEstimator(JSON.stringify(block)); default: // Mirrors MessageStore.computeBlockTokensRaw: membrane's XML-history // carriers (tool_attempt, tool_notice) are priced as the markup they diff --git a/src/context-manager.ts b/src/context-manager.ts index ea298d8c..6f7e9b13 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -32,6 +32,8 @@ import type { SummaryEntry, TimeRangeSummaryEntry, HotContextSettingsUpdate, + RequestIdentity, + ThinkingBinding, SelectOptions, PreviewResult, HotContextSettingsStatus, @@ -56,8 +58,14 @@ import { markStoreBranchSwitch, observeStoreBranch } from './branch-generation.j import type { StoreBranchGeneration } from './branch-generation.js'; import { randomUUID } from 'node:crypto'; import { attributeEntries, buildRenderedLayout, rawBodiesOf } from './compile-provenance.js'; +import { isCompileRefusal, isolateObserver, type ContextObserver } from './observer.js'; import { FoldJournal, branchRefOf, cachedStoreIdentity, existingStoreIdentity } from './fold-journal.js'; import { describeThrown, logThrown } from './describe-thrown.js'; +import { + UNVOUCHED, bindThinking, continueRecord, isThinkingBlock, markThinkingLeftOut, messageDigest, readBindingRecord, + refuseUnboundBinding, refuseUnreadableIdentity, registerBindingState, thinkingAsSent, thinkingSent, withoutThinking, writeBindingRecord, + type BindOutcome, type BindingRecord, +} from './thinking-binding.js'; import { type Diagnostics, namespaceDiagnostics } from './diagnostics.js'; import type { FoldQuery, FoldQueryResult, FoldReceipt, Presentation, ReceiptSource, RoundUsage } from './fold-journal.js'; import type { @@ -71,6 +79,37 @@ import type { type InitializationStamp = StoreBranchGeneration & { id: string; attempt: number }; +/** + * A compiled message as `firstChanged` compares it. An injection's content + * is the caller's, which may change in place, so its bytes are taken when + * it's compiled, also when a binding compile sends it without its thinking, + * as a new array of the caller's blocks. Other content is the store's, the + * strategy's or the manager's own, which nothing changes in place (as the + * store's own per-message caches assume), so the same array is the same + * content, and its bytes are taken only when a comparison with another array + * needs them. + */ +interface CompiledMessageKey { + readonly participant: string; + readonly content: readonly ContentBlock[]; + readonly injected: boolean; + bytes?: string; +} + +/** The index of the first of `keys` that differs from `before`'s at its index, or has none there (ContextManager.recordFirstChanged). */ +function firstDifference(keys: readonly CompiledMessageKey[], before: readonly CompiledMessageKey[]): number { + const bytesOf = (key: CompiledMessageKey): string => (key.bytes ??= JSON.stringify(key.content)); + let i = 0; + for (; i < keys.length && i < before.length; i++) { + const now = keys[i]!; + const then = before[i]!; + if (now.participant !== then.participant) break; + if (now.content === then.content && !now.injected && !then.injected) continue; + if (bytesOf(now) !== bytesOf(then)) break; + } + return i; +} + /** * Base configuration for ContextManager. */ @@ -103,6 +142,18 @@ interface ContextManagerBaseConfig { * inject a fake clock. */ now?: () => number; + /** + * Told of each compile, of each refusal context-manager itself makes of + * compression, and of each calibration sample, as each happens (see + * `ContextObserver`). The manager hands it to its strategy in + * `StrategyContext`. It's called synchronously, and what it throws or + * rejects never reaches the work: the manager logs a callback's first + * failure through its diagnostics and drops the rest. (A returned promise + * no JavaScript can give a handler is outside that; see + * `ContextObserver`.) A record's error and render + * stats are the objects the caller gets, so treat them as read-only. + */ + observer?: ContextObserver; /** * Strategy-facing exclusion predicate: return true to keep a message * visible. When set, the view handed to the strategy — for compile, @@ -254,11 +305,19 @@ export class ContextManager { private compressionHolds = new Set(); /** Summaries the strategy rendered but couldn't describe, already reported. */ private undescribedSummariesReported = new Set(); + /** + * The last committing compile's messages, for the next one's + * `firstChanged`, whatever branch or strategy it read; null before this + * manager's first committing compile. + */ + private lastCompiled: CompiledMessageKey[] | null = null; /** Sharded adds held on ingress: first-shard id -> every shard id. */ private compressionHoldGroups = new Map(); /** heldAt / expiresAt per held id (expiresAt absent = no timeout). */ private compressionHoldInfo = new Map(); private now: () => number = Date.now; + /** The host's observer, isolated (`ContextManagerConfig.observer`), or none. */ + private observer: ContextObserver | undefined; /** Hold options + shared heldAt for the addMessage currently appending. */ private holdingAddOptions: CompressionHoldOptions | undefined; private holdingAddAt = 0; @@ -430,6 +489,7 @@ export class ContextManager { // so strategies always have something to scope state IDs by, even when // the caller didn't supply a namespace. const strategyNamespace = config.namespace ?? 'default'; + registerBindingState(store, strategyNamespace); const messageStore = new MessageStore(store, { estimator: config.tokenEstimator, namespace: messageNamespace, @@ -484,6 +544,14 @@ export class ContextManager { auxiliaryStores, ); if (config.now) manager.now = config.now; + if (config.observer) { + manager.observer = isolateObserver(config.observer, (callback, error) => logThrown( + `[context-manager] the observer's ${callback} callback threw or rejected; the work it observes goes on, ` + + `and its later failures aren't logged:`, + error, + manager.diag, + )); + } try { await manager.initializeStrategy(); @@ -530,6 +598,9 @@ export class ContextManager { // later (appendMessage), once expired holds have been released, as at // any add. content = contentForWrite(content, 'Message refused'); + // So is a system change between a tool call and its result + // (MessageStore.refuseToolPairSplit). + this.messageStore.refuseToolPairSplit(content, 'Message refused'); return this.write(() => { this.expireCompressionHolds(); if (!options?.holdCompression) { @@ -1112,7 +1183,9 @@ export class ContextManager { // ========================================================================== /** - * Check if compile() will block waiting for background work. + * Whether the strategy has no background work pending (its + * `checkReadiness()`). compile() doesn't wait for that work: it compiles + * from what's there. */ isReady(): boolean { this.requireInitialized('isReady'); @@ -1146,14 +1219,56 @@ export class ContextManager { * - "afterUser": inserted after the last user message, or at the end when * the messages hold none * - * May block if strategy has pending work. + * With `opts.thinkingBinding.binds`, it then leaves out the thinking a + * model that binds thinking can no longer take, injections placed (see + * `SelectOptions.thinkingBinding`). + * + * With `ContextManagerConfig.observer`, it tells the observer of the + * compile as it returns or throws (`ContextObserver.compile`). + * + * It awaits nothing, so it doesn't wait for the strategy's background work. */ async compile( budget?: TokenBudget, injections?: ContextInjection[], opts?: SelectOptions ): Promise { + const observer = this.observer; + if (!observer?.compile) return this.compileNow(budget, injections, opts); + // A compile awaits nothing, so it's timed whole on one clock reading + // each side, and the observer hears of it as it returns or throws. The + // wall clock, not the manager's own, so an observer adds no read of now(). + const startedAt = Date.now(); + const start = performance.now(); + const strategy = this.strategy.name; + const dryRun = opts?.dryRun === true; + let result: CompileResult; + try { + result = this.compileNow(budget, injections, opts); + } catch (error) { + observer.compile({ + strategy, dryRun, startedAt, durationMs: performance.now() - start, + outcome: isCompileRefusal(error) ? 'refused' : 'failed', error, + }); + throw error; + } + observer.compile({ + strategy, dryRun, startedAt, durationMs: performance.now() - start, outcome: 'compiled', + ...(result.renderStats ? { renderStats: result.renderStats } : {}), + }); + return result; + } + + /** compile()'s work, which awaits nothing. */ + private compileNow( + budget?: TokenBudget, + injections?: ContextInjection[], + opts?: SelectOptions + ): CompileResult { this.requireInitialized('compile'); + // Before anything is selected or committed: a compile refused here leaves + // nothing behind. + if (opts?.thinkingBinding?.binds === true) refuseUnboundBinding(opts); this.expireCompressionHolds(); // Don't block the agent's turn on speculative compression — let it // run in the background. The strategy renders whatever's available @@ -1178,12 +1293,16 @@ export class ContextManager { const branch = branchRefOf(this.store); const view = this.strategyMessageView(); - // Get selected entries from strategy + // Get selected entries from strategy. The injections it will sit among + // go with it, for its calibration's count of the request (SelectOptions). + const messageInjections = (injections ?? []) + .filter((inj) => inj.position === 'beforeUser' || inj.position === 'afterUser') + .map((inj) => inj.content); const entries = this.strategy.select( view, this.contextLog.createView(), effectiveBudget, - opts + messageInjections.length > 0 ? { ...opts, messageInjections } : opts ); const selectionCause = this.strategy.takeSelectionCause?.(); // Read now, before anything else selects: these are this select's own. @@ -1234,11 +1353,43 @@ export class ContextManager { } const finish = (result: CompileResult, injected: ContentBlock[][]): CompileResult => { - result.rawBodies = rawBodiesOf(attribution); + const rawBodies = rawBodiesOf(attribution); + const compileId = randomUUID(); + const binding = opts?.thinkingBinding?.binds === true + ? this.bindThinkingIn(result, messageSources, opts, compileId) + : null; + result.rawBodies = rawBodies; if (renderStats) result.renderStats = renderStats; - const layout = this.renderedLayoutFor(viewMessages, entries, attribution.rawComplete, injected, selectionCause); + // A body sent without its thinking isn't carried whole, and the layout + // prices what was sent: thinking only where a stored body kept it. + let layoutEntries = entries; + let layoutInjected = injected; + let rawComplete = attribution.rawComplete; + if (binding) { + const without = new Set(binding.record.without); + if (without.size > 0) { + markThinkingLeftOut(messageSources, without); + // Keyed by each stored member of a body, as attribution keys it. + const complete = new Map(rawComplete); + for (const id of without) { + for (const member of rawBodies.get(id) ?? []) if (complete.has(member.id)) complete.set(member.id, false); + } + rawComplete = complete; + } + // Each entry keeps the thinking the compile sent of it: an entry the + // compile splits (a bundled tool cycle) is bound part by part, so one + // part can keep what another leaves out. The record's `without` is + // per body, so from the next compile such a body goes without whole. + const sent = thinkingSent(result.messages); + layoutEntries = entries.map((entry) => { + const content = thinkingAsSent(entry.content, sent); + return content === entry.content ? entry : { ...entry, content }; + }); + layoutInjected = injected.map((content) => (content.some(isThinkingBlock) ? withoutThinking(content) : content)); + } + const layout = this.renderedLayoutFor(viewMessages, layoutEntries, rawComplete, layoutInjected, selectionCause); result.provenance = { - compileId: randomUUID(), + compileId, namespace: this.strategyNamespace, branch, messages: messageSources, @@ -1248,6 +1399,36 @@ export class ContextManager { ...(layout ? { journalMark: this.foldJournal.mark() } : {}), }; if (this.debugLogContext) this.logCompiledContext(result); + // Last, once nothing of the compile can still fail: what a committing + // compile leaves behind for the next one. A compile that throws before + // here, which no caller sends, is never the one the next is compared with. + if (opts?.dryRun !== true) { + this.recordFirstChanged(result, messageSources); + if (binding) { + // Nothing here fails the compile: a record it can't write leaves + // the one before, if any, which this compile's stream can't note + // onto, so what it sends past that record goes without its + // thinking. The strategy's note only adjusts its calibration, so a + // strategy that throws there is logged and the compile stands too. + this.writeBindingRecordFor( + binding.record, + 'this compile\'s stream can\'t note onto the record before it, if any, so its replies go without their thinking until a compile with binds writes the record', + ); + if (binding.leftOut.length > 0) { + try { + this.strategy.noteThinkingLeftOut?.(binding.leftOut); + } catch (err) { + this.diag.error('[context-manager] the strategy failed to take note of thinking left out:', describeThrown(err)); + } + } + } + // A compile without binds neither reads nor writes the record. One an + // earlier binding compile left is compared at the next binding compile + // like any other: a message keeps its thinking only while everything + // up to it matches what that compile sent and its stream noted, a + // prefix the provider took with that thinking, and nothing past the + // record's end keeps it. + } return result; }; @@ -1320,6 +1501,82 @@ export class ContextManager { return finish({ messages, systemInjections }, injectedContent); } + /** + * Records a committing compile's messages for the next one's comparison + * (CompiledMessageKey), and gives `result` its `firstChanged`, compared + * when it's read: a caller that never reads it pays for no comparison. + * That is the index of the first compiled message whose + * participant or content bytes differ from the last committing compile's + * message at that index, or that has none there (so the first appended + * index after appends alone, and the message count when these messages are + * that compile's or a prefix of them); null before this manager's first + * committing compile. The comparison is by bytes, so it holds across a + * branch or strategy switch, which the manager can't always see (a direct + * switch of the store's branch away and back). Cache markers are message + * flags, not content, so they don't count. Read after later compiles, it + * still compares this compile with the one before it. Until it's read, a + * result keeps this compile's keys and the last one's. + */ + private recordFirstChanged(result: CompileResult, sources: readonly CompiledMessageSources[]): void { + // An injection is told by its source, not its array: a binding compile + // sends one that holds thinking as a new array. + const keys: CompiledMessageKey[] = result.messages.map((message, i) => sources[i]?.kind === 'injection' + ? { participant: message.participant, content: message.content, injected: true, bytes: JSON.stringify(message.content) } + : { participant: message.participant, content: message.content, injected: false }); + const before = this.lastCompiled; + this.lastCompiled = keys; + // The first read, or a write, settles the value and lets go of both + // compiles' keys, so the result then holds a number. It stays an + // accessor, so a frozen or sealed result reads it too. + let pending: { keys: CompiledMessageKey[]; before: typeof before } | null = { keys, before }; + let value: number | null | undefined; + Object.defineProperty(result, 'firstChanged', { + enumerable: true, + configurable: true, + get: () => { + if (pending) { + value = pending.before ? firstDifference(pending.keys, pending.before) : null; + pending = null; + } + return value; + }, + set: (next: number | null | undefined) => { + value = next; + pending = null; + }, + }); + } + + /** + * Leave out of `result.messages` the thinking the branch's record says a + * binding model can no longer take, and report it (see + * `SelectOptions.thinkingBinding`). Returns what a committing compile then + * leaves behind: the record, and the stored thinking it left out. + */ + private bindThinkingIn( + result: CompileResult, + sources: readonly CompiledMessageSources[], + opts: SelectOptions, + compileId: string, + ): BindOutcome { + const record = this.readBindingRecordFor('this compile goes on as if the branch had no record, and if it commits, its record takes that one\'s place'); + const outcome = bindThinking({ + messages: result.messages, + sources, + record, + dropBlock: opts.thinkingBinding?.dropBlock === true, + // Checked at the compile's start (refuseUnboundBinding). + identity: opts.identity!, + participant: opts.thinkingBinding!.participant!, + sequence: this.store.currentSequence(), + compileId, + }); + result.messages = outcome.messages; + if (outcome.strip) result.thinkingStrip = outcome.strip; + result.thinkingRecordedAfter = outcome.record.recordedAfter; + return outcome; + } + /** * The rendered layout of one compile, or null when the strategy does not * report its layout, or names a summary it cannot describe (a layout that @@ -1556,7 +1813,8 @@ export class ContextManager { /** * What the next compile sends for each of `ids`, messages this manager has * just stored, rendered as the newest of the window (see - * `ContextStrategy.renderAsNewest`). Give each message by the id + * `ContextStrategy.renderAsNewest`), before any thinking the thinking + * binding leaves out (`SelectOptions.thinkingBinding`). Give each message by the id * `addMessage` returned. For a body stored as shards that's its first * shard's, which stands for the whole body where the compile joins its * shards; where the compile sends each shard on its own, an id gives back @@ -1586,6 +1844,141 @@ export class ContextManager { return this.strategy.renderAsNewest(this.strategyMessageView(), ids); } + /** + * Note what a stream built from this branch's last committing compile with + * `binds` sent + * after the compiled messages (see `SelectOptions.thinkingBinding`): as it + * sends each later round, every row its request carried so far, and once + * the stream has ended, those rows and its last reply. Call it on the + * branch that compile was on, with `compileId`, that compile's + * `provenance.compileId`. Each row, in order, is the id `addMessage` + * returned for a stored message, which stands for the whole body, or + * `{ unstored: true }`. `identity` is the head the stream was sent with. + * The branch's record then holds these rows, so at the next compile they + * keep their thinking while nothing up to them differs. Past the record's + * end nothing keeps its thinking: a reply keeps its thinking at the next + * compile only once a note holds it. + * + * Note only what the host can vouch it sent, or for the last reply, got + * back: a stored row whose copy, as `renderAsNewest` renders it and the + * host's request then carries it, differs from what was sent or returned + * is `{ unstored: true }`, and so is a row the store doesn't hold that the + * next request won't carry where this one did. A request-only row the + * host puts back in the same place at every later request needs no row, + * since no compile holds it. A note sent with another head than the + * record's leaves the record vouching for nothing until a compile writes + * it again: the stream's earlier thinking was minted under the + * old head. A host that notes nothing keeps no reply's thinking past the + * compile after it was stored, and is never refused for it. + * + * Each stored row is digested as `renderAsNewest` renders it and as the + * compile turns that into messages: a tool result under another + * participant goes as the user's, and a bundled tool cycle as each of its + * parts (`splitMixedToolMessages`). So one it renders as null, or that the + * next compile renders otherwise, matches nothing there, and what follows + * goes without its thinking; a note that renders rows this way releases + * compression holds that have timed out, as `renderAsNewest` does. + * Without `renderAsNewest`, a row is its stored content, and a body stored + * as shards matches nothing, and so does an id the store doesn't hold, as + * after a removal. + * + * Returns false and writes nothing when `compileId` isn't the compile that + * wrote the branch's record, as for a stream whose note arrives after + * another committing compile with `binds`: its replies were minted under a + * prefix the record no longer holds, and go without their thinking. A + * compile without `binds` leaves the record, so a note after one still + * writes. It returns false too when the branch has no record, and, logging + * it, when the record is one this version can't read, or the store can't + * read or write it; a note it can't write is lost, and the record stands + * as the last one written. + * A dry compile writes no record, so it leaves a stream's notes as they + * were. Throws, writing nothing, for a row that isn't an id or the marker, + * a `compileId` that isn't a string, or a digest that isn't a string. + */ + noteRequestSent(rows: ReadonlyArray, identity: RequestIdentity, compileId: string): boolean { + this.requireInitialized('noteRequestSent'); + if (!Array.isArray(rows)) throw new TypeError(`noteRequestSent: \`rows\` must be an array, got ${typeof rows}`); + rows.forEach((row: unknown, i) => { + if (typeof row === 'string') return; + if (typeof row === 'object' && row !== null && (row as { unstored?: unknown }).unstored === true) return; + throw new TypeError(`noteRequestSent: row ${i} must be a message id or { unstored: true }, got ${JSON.stringify(row) ?? typeof row}`); + }); + refuseUnreadableIdentity(identity); + if (typeof compileId !== 'string') { + throw new TypeError( + `noteRequestSent: \`compileId\` must be a string, the provenance.compileId of the compile the stream was built from; got ${typeof compileId}`, + ); + } + const record = this.readBindingRecordFor('this note writes nothing, and the branch\'s next committing compile with binds replaces the record'); + // A note is for the stream of the compile that wrote the record; after + // another committing compile with binds its replies were minted under a + // prefix the record no longer holds. + if (record === null || record.compileId !== compileId) return false; + const view = this.mergedMessageView(); + // An id the store doesn't hold, as after a removal, is nothing the next + // compile sends: it vouches for nothing, as the marker does. + const ids = rows.filter((row): row is MessageId => typeof row === 'string' && view.get(row) !== null); + const rendered = ids.length > 0 ? this.renderAsNewest(ids) : []; + let next = 0; + const continued: string[] = []; + for (const row of rows) { + const stored = typeof row === 'string' ? view.get(row) : null; + if (stored === null) { + continued.push(UNVOUCHED); + continue; + } + // With no renderAsNewest, a row is its stored content, which for a body + // stored as shards is only its first shard's. + const content = rendered === undefined ? (stored.bodyGroupId === undefined ? stored.content : null) : rendered[next]; + next++; + if (content === null || content === undefined) { + continued.push(UNVOUCHED); + continue; + } + // As the compile turns it into messages: a tool result under another + // participant goes as the user's, and a bundled tool cycle as each of + // its parts. + for (const part of splitMixedToolMessages([{ participant: stored.participant, content }])) continued.push(messageDigest(part)); + } + return this.writeBindingRecordFor( + continueRecord(record, continued, identity), + 'this note is lost: the record stands as the last one written, so the rows this note adds go without their thinking unless a later note of the stream writes', + ); + } + + /** + * The branch's binding record. One this version doesn't read, or that the + * store can't give, is logged, with what follows from it, and read as + * none; a committing compile with `binds` then writes its own in its place. + */ + private readBindingRecordFor(consequence: string): BindingRecord | null { + let read: ReturnType; + try { + read = readBindingRecord(this.store, this.strategyNamespace); + } catch (err) { + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} couldn't be read (${describeThrown(err)}); ${consequence}`); + return null; + } + if (read.malformed) { + this.diag.warn( + `[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} isn't ` + + `one this version reads; ${consequence}`, + ); + } + return read.record; + } + + /** Write the branch's binding record; a write that throws is logged, with what follows from it, and false. */ + private writeBindingRecordFor(record: BindingRecord, consequence: string): boolean { + try { + writeBindingRecord(this.store, this.strategyNamespace, record); + return true; + } catch (err) { + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} couldn't be written (${describeThrown(err)}); ${consequence}`); + return false; + } + } + /** Read the active strategy's allowlisted live settings, if supported. */ getHotContextSettings(): HotContextSettingsStatus | null { this.requireInitialized('getHotContextSettings'); @@ -1606,11 +1999,14 @@ export class ContextManager { * Non-committing preview of the layout at a hypothetical budget and/or * settings. Commits nothing — see `AutobiographicalStrategy.previewContext`. * Returns null when the active strategy has no fold plan to preview. + * `opts.thinkingBinding` previews a compile with that option: a binding + * one prices a summary's carriers as such a compile renders them (see + * `SelectOptions.thinkingBinding`); without it, as configured. */ previewContext( budget: TokenBudget, overrides?: Record, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ): PreviewResult | null { this.requireInitialized('previewContext'); const s = this.strategy as unknown as { @@ -1619,7 +2015,7 @@ export class ContextManager { log: ReturnType, budget: TokenBudget, overrides?: Record, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ) => PreviewResult; }; if (typeof s.previewContext !== 'function') return null; @@ -1640,7 +2036,11 @@ export class ContextManager { * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel`. A raw pin, with no bound or a bound of 0, keeps the * messages out of compression and renders them raw at their original - * chronological position. A bound above 0 is read only under + * chronological position, a message stored in parts whole, as + * `ProtectedRange` describes. A compile whose budget can't hold them beside + * the rest refuses with `OverBudgetError`; where the adaptive plan's middle + * is over the budget, its message and `diagnostics.rawPins` name each pin + * keeping raw tokens in that middle. A bound above 0 is read only under * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution`: there the range is still compressed, since only a * summary can supply its level, and is held at `level`, or no deeper than @@ -1681,7 +2081,13 @@ export class ContextManager { * `kind: 'document'`, levels included (see `pinRange`): without a `level` * or `maxLevel` the message is kept raw, while * `markDocument(id, { maxLevel })` lets it fold no deeper than that level - * where the selector reads level bounds. + * where the selector reads level bounds. Under adaptive resolution, a mark + * on any part of a message stored in parts covers the whole message, less + * a part locked with `lockChunk` outside the span the mark's id names, as + * `pinRange` does. + * Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each + * summary it reaches, so a locked part that shares a summary with a part + * the mark covers renders at the level the mark holds that summary at. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { @@ -1977,6 +2383,7 @@ export class ContextManager { messageStore: this.strategyMessageView(), contextLog: this.contextLog.createView(), membrane: this.membrane, + ...(this.observer ? { observer: this.observer } : {}), currentSequence: this.store.currentSequence(), store: this.store, namespace: this.strategyNamespace, diff --git a/src/index.ts b/src/index.ts index af88b229..7aacfcc7 100644 --- a/src/index.ts +++ b/src/index.ts @@ -38,12 +38,24 @@ export type { ConfigLayer, ConfigResolutionSemantics, EffectiveConfigReport } fr // OverBudget drain breaker on these errors (AF PR #58, framework.ts // classifyInferenceError); exporting them from the root gives consumers a // real `instanceof` instead of stringly-typed `err.name` matching. -export { OverBudgetError, UncoveredDropError } from './adaptive/picker.js'; -export type { OverBudgetClearedBy } from './adaptive/picker.js'; +export { OverBudgetError, UncoveredDropError, OVER_BUDGET_STAGES } from './adaptive/picker.js'; +export type { OverBudgetClearedBy, OverBudgetStage } from './adaptive/picker.js'; + +// What a host observes as it happens (ContextManagerConfig.observer). +export { COMPILE_REFUSALS, COMPRESSION_REFUSALS, CALIBRATION_SAMPLE_OUTCOMES } from './observer.js'; +export type { + ContextObserver, + CompileRecord, + CompressionRefusalRecord, + CalibrationSampleRecord, + CompileRefusal, + CompressionRefusal, + CalibrationSampleOutcome, +} from './observer.js'; export { CompressionWithdrawn } from './compression-withdrawn.js'; export { StoreTopologyError, type TopologyViolation } from './strategies/autobiographical.js'; export { planTopologyRepair, type RepairPlan, type RepairInputs, type RepairOptions } from './repair/topology.js'; -export type { OverBudgetDiagnostics } from './adaptive/picker.js'; +export type { OverBudgetDiagnostics, OverBudgetRawPin } from './adaptive/picker.js'; // Types export type { @@ -80,7 +92,9 @@ export type { JournalMark, RenderedLayout, LayoutUnit, + ThinkingStrip, // Strategy types + RequestOverhead, MessageStoreView, ContextLogView, StrategyContext, @@ -109,6 +123,9 @@ export type { CompressionHoldOptions, CompressionHoldInfo, RenderedSummaryInfo, + ImageCeilings, + ThinkingBinding, + RequestIdentity, } from './types/index.js'; // Accepted rounds and fold receipts diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts new file mode 100644 index 00000000..e0269b81 --- /dev/null +++ b/src/live-image-limits.ts @@ -0,0 +1,223 @@ +/** + * The live-image window's limits, kept within the image ceilings. + * + * A strategy's live window keeps the images it counts (every image, nested + * and generated ones included; see `AutobiographicalConfig.maxLiveImages`) + * newest-first up to a count (`maxLiveImages`) and a byte budget + * (`maxLiveImageBytes`). The image ceilings (`imageCeiling`, + * `imageByteCeiling`) bound those limits: an unlimited (0) limit means "up to + * the ceiling", and a limit above the ceiling keeps the ceiling's worth. They + * bound what the window counts and what a compression prompt carries, not a + * whole request: an image inside an opaque provider item isn't counted, nor + * is anything a caller injects at compile time or appends to a compiled + * request. A caller that wants a request within the ceilings counts that + * request's images itself and compares them with the strategy's + * `imageCeilings()`. + */ + +/** + * The provider refuses a request carrying more than 600 images and documents + * together ("too many images and documents: 601 + 0 > 600", measured + * 2026-10-13 and again on 2026-10-19). 500 leaves room for what the ceilings + * don't count: documents, images inside opaque provider items, and images a + * caller injects or appends. + */ +export const DEFAULT_IMAGE_CEILING = 500; + +/** + * membrane's byte cap is 28 MiB of serialized messages unless + * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise (membrane 0.5.87, 0.6.0 and 0.7.1, + * the version this package's lockfile installs). + * It checks a request when it builds it through `transformRequest`, which a + * call's first request and each native tool-loop round go through; an + * XML-mode continuation round doesn't, and of the provider adapters only the + * Anthropic adapter checks again, on every request it sends. Over the cap, + * `transformRequest` refuses the request before sending it + * (`assertWithinByteBudget`), unless the request sets `shedOversizeImages`, + * as this package's compression requests do. Then it replaces images, + * oldest first, until the request fits or no image is left, and passes it + * on either way; the Anthropic adapter's own check, which never sheds, + * refuses one still over the cap. The shed works on the messages the + * formatter built, and replaces only membrane's `image` blocks, top-level or + * in a `tool_result`: a request that sets `shedOversizeImages` and is built + * by membrane's `openai-responses` formatter, whose images are + * `input_image` parts, has none shed and isn't checked against the cap + * there, and that adapter doesn't check it again. 24 MiB of base64 images leaves 4 MiB of that + * default cap for everything else the messages carry: their text, their + * JSON, and any documents or other media. + */ +export const DEFAULT_IMAGE_BYTE_CEILING = 24 * 1024 * 1024; + +/** A strategy's image limits as its window applies them. Each figure is a + * count of the images the window counts or a sum of their inline base64 + * bytes, and 0 is no limit. `target` is what the window keeps, already + * within `ceiling` (`maxLiveImages` and `maxLiveImageBytes`, clamped); + * `ceiling` is `imageCeiling` and `imageByteCeiling`. */ +export interface ResolvedImageLimits { + target: { images: number; bytes: number }; + ceiling: { images: number; bytes: number }; +} + +/** A window limit as the window reads it: a whole number of 1 or more, or 0 + * for none. Any value that isn't a positive finite number (0, a negative, + * NaN, Infinity) is none, as the window's own checks (`limit > 0 && kept >= + * limit`) already read it. A fraction is floored, to at least 1, so the + * limit reported is the limit kept. */ +function asLimit(value: number): number { + return Number.isFinite(value) && value > 0 ? Math.max(1, Math.floor(value)) : 0; +} + +/** Whether a configured ceiling is one the window can read: 0 (none), or a + * finite number above 0. */ +function wellFormedCeiling(value: number): boolean { + return value === 0 || (Number.isFinite(value) && value > 0); +} + +/** A ceiling as configured: 0 is none, a finite number above 0 is floored to + * at least 1, and any other value (a negative, NaN, Infinity) takes the + * default, so a malformed config doesn't remove the bound (the warning names + * it). */ +function asCeiling(value: number, fallback: number): number { + return wellFormedCeiling(value) ? asLimit(value) : fallback; +} + +/** + * `target` kept within `ceiling`, where 0 means no limit for either: an + * unlimited target becomes the ceiling, and no ceiling leaves the target as + * it is. A target that isn't a positive finite number counts as unlimited, + * as the window's own checks already treat it. `ceiling` is one + * resolveLiveImageLimits gives. + */ +export function clampToCeiling(target: number, ceiling: number): number { + const c = asLimit(ceiling); + const t = asLimit(target); + if (c === 0) return t; + return t === 0 ? c : Math.min(t, c); +} + +/** The window's configured limits, the defaults applied where unset. */ +export interface ConfiguredImageLimits { + maxLiveImages: number; + maxLiveImageBytes: number; + imageCeiling: number; + imageByteCeiling: number; +} + +/** The limits a strategy's window applies, and the ceilings above them. */ +export function resolveLiveImageLimits(configured: ConfiguredImageLimits): ResolvedImageLimits { + const ceiling = { + images: asCeiling(configured.imageCeiling, DEFAULT_IMAGE_CEILING), + bytes: asCeiling(configured.imageByteCeiling, DEFAULT_IMAGE_BYTE_CEILING), + }; + return { + target: { + images: clampToCeiling(configured.maxLiveImages, ceiling.images), + bytes: clampToCeiling(configured.maxLiveImageBytes, ceiling.bytes), + }, + ceiling, + }; +} + +const MiB = 1024 * 1024; + +/** A byte figure as the strategies' lines give it: in MiB when it's a whole + * number of them, and in bytes otherwise, so two figures that differ never + * read alike. */ +function showBytes(n: number): string { + return n % MiB === 0 ? `${n / MiB} MiB` : `${n} bytes`; +} + +/** + * The " (default)" a line puts after a setting whose value is a default: one + * the host didn't give (`defaulted`), or a ceiling it gave malformed, which + * takes its default (see asCeiling). + */ +function defaultMark( + name: string, + configured: ConfiguredImageLimits, + defaulted: ReadonlySet, +): string { + const malformedCeiling = (name === 'imageCeiling' || name === 'imageByteCeiling') && + !wellFormedCeiling(configured[name]); + return defaulted.has(name) || malformedCeiling ? ' (default)' : ''; +} + +/** + * Which of the window's limits a ceiling sets: a limit the host gave as none + * (0, or anything that isn't a positive finite number) or above its ceiling, + * where that ceiling is itself a limit. The window then keeps the ceiling's + * worth. (With no ceiling the window keeps the limit as given, so the two + * never match there.) + */ +export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: boolean; bytes: boolean } { + const { target, ceiling } = resolveLiveImageLimits(configured); + return { + images: target.images === ceiling.images && asLimit(configured.maxLiveImages) !== ceiling.images, + bytes: target.bytes === ceiling.bytes && asLimit(configured.maxLiveImageBytes) !== ceiling.bytes, + }; +} + +/** + * The line a strategy says once for each ceiling, at the first render a + * select returns where images are shown as placeholders and a limit that + * ceiling sets (ceilingSetLimits) was over: at an image the render + * replaces, or, under the memory strategy's `imageStripHysteresisRatio`, at + * the start of a trim anywhere in the rule's replay. It says which ceiling, + * at what value, and the window limit of 0 or above it that the ceiling + * holds, so an operator who set no limit sees where one now applies. + */ +export function ceilingBoundLine( + strategy: string, + kind: 'images' | 'bytes', + configured: ConfiguredImageLimits, + defaulted: ReadonlySet, +): string { + const ceiling = resolveLiveImageLimits(configured).ceiling; + const [ceilingName, limitName, value, show] = kind === 'images' + ? ['imageCeiling', 'maxLiveImages', configured.maxLiveImages, String] as const + : ['imageByteCeiling', 'maxLiveImageBytes', configured.maxLiveImageBytes, showBytes] as const; + const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); + const mark = (name: string) => defaultMark(name, configured, defaulted); + return `[${strategy}] ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)} ` + + `holds ${limitName} ${limit}${mark(limitName)} to it, so some images are now shown as placeholders`; +} + +/** + * The warning a strategy gives once, when it initializes, or null when there + * is nothing to say: a ceiling configured as something other than 0 or a + * finite number above it, which takes the default, and a limit (not 0) above + * its ceiling, which is kept at the ceiling. `maxCompressionImageBytes`, when + * given, is the compression prompt's byte budget, held to the byte ceiling + * too. `defaulted` names the settings the host didn't give, which the line + * marks "(default)", so a host isn't told about a setting as if they had + * made it. A byte figure reads in MiB when it's a whole number of them. + */ +export function liveImageLimitsWarning( + strategy: string, + configured: ConfiguredImageLimits & { maxCompressionImageBytes?: number }, + defaulted: ReadonlySet, +): string | null { + const said: string[] = []; + const ceiling = resolveLiveImageLimits(configured).ceiling; + const count = (n: number) => String(n); + const bytes = showBytes; + const setting = (name: string, shown: string) => `${name} ${shown}${defaultMark(name, configured, defaulted)}`; + if (!wellFormedCeiling(configured.imageCeiling)) { + said.push(`imageCeiling ${configured.imageCeiling} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); + } + if (!wellFormedCeiling(configured.imageByteCeiling)) { + said.push(`imageByteCeiling ${configured.imageByteCeiling} isn't 0 or a finite number above it, so the default ${bytes(ceiling.bytes)} applies`); + } + const above = ( + name: string, value: number | undefined, ceilingName: string, limit: number, keeps: string, show: (n: number) => string, + ) => { + const v = value === undefined ? 0 : asLimit(value); + if (limit > 0 && v > limit) { + said.push(`${setting(name, show(v))} is above ${setting(ceilingName, show(limit))}, so ${keeps} at most ${show(limit)}`); + } + }; + above('maxLiveImages', configured.maxLiveImages, 'imageCeiling', ceiling.images, 'the live window keeps', count); + above('maxLiveImageBytes', configured.maxLiveImageBytes, 'imageByteCeiling', ceiling.bytes, 'the live window keeps', bytes); + above('maxCompressionImageBytes', configured.maxCompressionImageBytes, 'imageByteCeiling', ceiling.bytes, 'a compression prompt keeps', bytes); + return said.length === 0 ? null : `[${strategy}] ${said.join('; ')}`; +} diff --git a/src/message-store.ts b/src/message-store.ts index 6e7f5823..f8fe2f2f 100644 --- a/src/message-store.ts +++ b/src/message-store.ts @@ -23,6 +23,37 @@ import { bodyBoundsIn, bodyEnd, bodyStart, continuesBody } from './body-runs.js' import { contentForWrite, wellFormedContent } from './well-formed-content.js'; import { storedJson } from './stored-json.js'; +/** Whether content, written or stored, holds a top-level block of `type`; content that isn't an array holds none. */ +function holdsBlock(content: unknown, type: string): boolean { + return Array.isArray(content) && content.some((block) => (block as { type?: unknown } | null)?.type === type); +} + +/** + * Each call content holds, by id, and whether it's still open after the + * content's own later blocks: a result after its call in the same content, + * as in a bundled cycle (splitMixedToolMessages), answers it. + */ +function callsInOrder(content: unknown): Map { + const calls = new Map(); + if (!Array.isArray(content)) return calls; + for (const block of content as Array<{ type?: unknown; id?: unknown; toolUseId?: unknown } | null>) { + if (block?.type === 'tool_use' && typeof block.id === 'string') calls.set(block.id, true); + else if (block?.type === 'tool_result' && typeof block.toolUseId === 'string' && calls.has(block.toolUseId)) calls.set(block.toolUseId, false); + } + return calls; +} + +/** The ids of content's top-level calls (`tool_use`), or of the calls its results (`tool_result`) answer, in block order. */ +function toolIds(content: unknown, type: 'tool_use' | 'tool_result'): string[] { + if (!Array.isArray(content)) return []; + const ids: string[] = []; + for (const block of content as Array<{ type?: unknown; id?: unknown; toolUseId?: unknown } | null>) { + const id = type === 'tool_use' ? block?.id : block?.toolUseId; + if (block?.type === type && typeof id === 'string') ids.push(id); + } + return ids; +} + const DEFAULT_MESSAGE_STATE_ID = 'messages'; /** @@ -460,6 +491,7 @@ export class MessageStore { } ): StoredMessage { content = contentForWrite(content, 'Message refused'); + this.refuseToolPairSplit(content, 'Message refused'); // Extract blobs from content const storedContent = this.blobManager.extractBlobs(content); @@ -568,6 +600,115 @@ export class MessageStore { return message; } + /** + * Refuses a write that would put a system change between a tool call and + * a result that answers it: a message of changes while one of the nearest + * calls before it has no result between it and the change, a tool result + * answering a call that a change already follows, or an edit giving a + * message a `tool_use` that a change follows before the call's results. + * The API takes a system message with content only after a user turn, or an + * assistant turn ending in a server tool result, so membrane sends a change + * stored after a call after the call's result anyway, and one written after + * the result goes out the same. Stored between them, a change parts them: + * the autobiographical and Knowledge strategies keep a change raw and close + * a chunk at it, so the call and its result would be folded into different + * summaries, and a compile's pairing repair (enforceToolPairing) would drop + * the result once the call was folded. + * + * It reads back from `index` over changes and messages holding results but + * no call, to the nearest message that holds a call, reading each message + * whole, every shard of its body, since ingress sharding puts a message's + * calls and results on its body's first shard. It checks that message's + * calls, by id: those a result in a later message answers, as parallel + * calls' results in messages of their own do, and those its own later + * blocks answer, as in a bundled cycle (splitMixedToolMessages), are + * answered. The walk stops at a message of any other kind, so a call no + * result answered, further back, refuses nothing. `index` is where the + * content goes: the end for an append, the message's place for an edit. + * + * It leaves alone what a write can't see, as main does: a message of + * another kind between a call and its result, which is already off the + * API's shape; a call in an earlier message than the nearest one holding + * a call; a shape that exists only in a view, filtered or merged from + * several stores; and a removal, or an edit, that takes away a result or a + * message between. Each parts the pair once the call is folded. + */ + refuseToolPairSplit(content: ContentBlock[], what: string, index: number = this.length()): void { + const length = this.length(); + const at = (i: number): StoredMessageInternal | null => (i >= 0 && i < length ? this.getInternal(i) : null); + // A body is one step of either walk, read whole: ingress sharding puts a + // message's non-text blocks, its calls and results, on its first shard. + const bodyAt = (i: number): { from: number; to: number; shards: StoredMessageInternal[] } => { + const from = bodyStart(at, i); + const to = bodyEnd(length, at, i); + const shards: StoredMessageInternal[] = []; + for (let k = from; k <= to; k++) shards.push(at(k)!); + return { from, to, shards }; + }; + const holds = (shards: StoredMessageInternal[], type: string): boolean => shards.some((shard) => holdsBlock(shard.content, type)); + const idsIn = (shards: StoredMessageInternal[], type: 'tool_use' | 'tool_result'): string[] => + shards.flatMap((shard) => toolIds(shard.content, type)); + const change = holdsBlock(content, 'system_change'); + const answers = change ? [] : toolIds(content, 'tool_result'); + if (change || answers.length > 0) { + // Back over changes and results, to the nearest body holding a call. + const answered = new Set(); + let changeBetween: StoredMessageInternal | undefined; + const calls = new Map(); + for (let i = index - 1; i >= 0; ) { + const { from, shards } = bodyAt(i); + if (holds(shards, 'tool_use')) { + // The body's calls left open by its own blocks, read in order + // across its shards, each by the shard that holds it. + const holder = new Map(); + for (const shard of shards) for (const id of toolIds(shard.content, 'tool_use')) holder.set(id, shard.id); + for (const [id, open] of callsInOrder(shards.flatMap((shard) => shard.content))) if (open) calls.set(id, holder.get(id)!); + break; + } + if (holds(shards, 'system_change')) changeBetween ??= shards[0]; + else if (holds(shards, 'tool_result')) for (const id of idsIn(shards, 'tool_result')) answered.add(id); + else break; + i = from - 1; + } + if (calls.size > 0) { + if (change) { + const open = [...calls].filter(([id]) => !answered.has(id)); + if (open.length > 0) { + throw new Error( + `${what}: a system change can't go between a tool call and its result, and message ${open[0]![1]} holds a tool_use ` + + `(${open.map(([id]) => id).join(', ')}) with no result between it and this change. Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.`, + ); + } + } else if (changeBetween) { + const id = answers.find((answer) => calls.has(answer)); + if (id !== undefined) { + throw new Error( + `${what}: a tool_result can't follow a system change (message ${changeBetween.id}) that follows its tool call ` + + `(${id}, message ${calls.get(id)}). Remove the change and write it after the result.`, + ); + } + } + } + } + if (index < length && holdsBlock(content, 'tool_use')) { + // Forward over changes and results, while a call here is unanswered. + const open = new Set([...callsInOrder(content)].filter(([, stillOpen]) => stillOpen).map(([id]) => id)); + for (let i = index + 1; i < length && open.size > 0; ) { + const { to, shards } = bodyAt(i); + if (holds(shards, 'tool_use')) break; + if (holds(shards, 'system_change')) { + throw new Error( + `${what}: a tool_use (${[...open].join(', ')}) can't go before a system change (message ${shards[0]!.id}) that comes before its result; ` + + "a change goes after the call's result.", + ); + } + if (!holds(shards, 'tool_result')) break; + for (const id of idsIn(shards, 'tool_result')) open.delete(id); + i = to + 1; + } + } + } + /** * Edit a message's content. * @@ -594,6 +735,7 @@ export class MessageStore { `Sharded messages are immutable — remove the whole bodyGroup and re-append instead.`, ); } + this.refuseToolPairSplit(newContent, `Edit of ${messageId} refused`, index); // The old content as a read gives it, so an edit can replace a message // whose stored content doesn't match ContentBlock. @@ -1158,6 +1300,10 @@ export class MessageStore { case 'audio': case 'video': return 1000; // Default estimate for media + case 'system_change': + // Its tool definitions and system text go out whole, so it's priced + // from its JSON, as a tool call's input is. + return jsonTokenEstimator(JSON.stringify(block)); default: return MessageStore.xmlHistoryCarrierTokens(block); } diff --git a/src/observer.ts b/src/observer.ts new file mode 100644 index 00000000..2d0e8450 --- /dev/null +++ b/src/observer.ts @@ -0,0 +1,261 @@ +/** + * What a host can observe of a context manager's work as it happens: each + * compile, each refusal context-manager itself makes of compression, and each + * calibration sample. A host passes one `ContextObserver` to + * `ContextManager.open` (`observer`), and the manager hands it to its strategy + * in `StrategyContext`. + * + * The records are facts on context-manager's own terms. What a host keeps or + * sends, and in what shape, is the host's to decide: a record can hold text + * from the store, such as a refusal's message, which names pins. + */ + +import type { RenderStats } from './types/strategy.js'; + +/** + * The thrown errors that are a compile's refusals, by `Error.name`: a + * designed answer that the compile can't fit or can't cover its messages, as + * opposed to a defect. The name survives two copies of context-manager in one + * process, where `instanceof` doesn't. + */ +export const COMPILE_REFUSALS = ['OverBudgetError', 'UncoveredDropError'] as const; + +/** A name of {@link COMPILE_REFUSALS}. */ +export type CompileRefusal = typeof COMPILE_REFUSALS[number]; + +/** + * Context-manager's own refusals of compression, from a closed list. A + * provider's refusal or error on a compression call is the provider's + * decision, not one of these: + * - `'admission-rejected'`: the request didn't fit `compressionContextBudgetTokens`, + * so no call was made. + * - `'topology-violation'`: a merge's sources don't make a group it can + * mint over (fewer than two, not one level below its target, or not + * adjacent in store order), so no call was made and the group went into the + * merge quarantine. + * - `'empty'`: the response held no text to keep, such as text that was a + * literal `` preamble alone. + * - `'incomplete'`: the response stopped other than at `end_turn` and wasn't + * a refusal (a `max_tokens` cut, a tool call, a stop sequence), so its text + * isn't kept. + * - `'malformed'`: the response, or a block in it, isn't the shape a response + * has. + * - `'chunk-call-cap'`: the split rung (`compressionSplitFallback`) reached + * its cap on calls in one attempt at a chunk + * (`compressionSplitMaxCallsPerChunk`) before it could finish, so it ended + * before another call. + * - `'window-call-cap'`: the split rung's attempt alone used every call its + * ten-minute window allows (`compressionSplitMaxCallsPer10Min`), so it ended + * before another call. + * - `'overlap-guard'`: a chunk's messages are partly covered by an L1 that + * already exists, so the strategy refuses to compress it rather than mint + * a duplicate memory. In the autobiographical strategy's own chunking only + * damaged bookkeeping or store surgery leaves that; `KnowledgeStrategy` + * leaves it when a summary ends at a tool call whose result comes after + * it. A chunk with a record isn't queued again while it's refused, and + * one without is refused again each time it's queued. Either way it's + * reported once per chunk (its record id, or else its messages), as the + * guard's log line is, until the strategy loads its state again: an open, + * a branch switch, a branch's state set up again, or taking up what + * another instance sharing the store wrote. + * + * The rest are reported once per decision. A verdict on an answer the L1 ladder + * keeps in its saved progress isn't decided again when the ladder resumes, + * so it isn't reported again; an admission rejection makes no call, so + * progress keeps none, and each ladder run that reaches it decides and + * reports it anew. In the split rung, each piece's empty or incomplete answer + * is reported as it comes, a cap's trip when the rung ends, and a malformed + * piece ends the rung as `'malformed'`. + */ +export const COMPRESSION_REFUSALS = [ + 'admission-rejected', + 'topology-violation', + 'empty', + 'incomplete', + 'malformed', + 'chunk-call-cap', + 'window-call-cap', + 'overlap-guard', +] as const; + +/** A reason of {@link COMPRESSION_REFUSALS}. */ +export type CompressionRefusal = typeof COMPRESSION_REFUSALS[number]; + +/** + * What a calibration sample did, from a closed list: + * - `'out-of-band'`: rejected, since neither its ratio nor the multiplier it + * implies is within the sanity band; the multiplier stays. A sample whose + * caller declared what the request sent besides the window counts toward + * a consistent run, and the run's last sample is learned: `'replaced'`, + * or, when what it implies, clamped, is within the dead band, `'held'` + * (`'confirmed'` for a provisional multiplier), as any sample there is. + * - `'deferred'`: the first sample with its overhead declared on a + * multiplier no declared sample has marked (one learned before samples + * declared their overhead, one moved since by an undeclared sample, or + * the 1 loaded beside a newer epoch's record or an unreadable one), held; + * the multiplier stays. The next declared sample within the band decides, + * and a consistent run's last sample ends the hold instead. + * - `'held'`: within the dead band of the multiplier in force, which stays. + * - `'confirmed'`: within the dead band of a provisional multiplier, which + * stays and is no longer provisional. + * - `'replaced'`: the multiplier replaced outright, within the clamp: a + * provisional one by what the sample implies, or any by a consistent + * out-of-band run's last sample or by the mean of two declared samples + * that agree. A result within the dead band of the multiplier in force + * is `'held'` instead (`'confirmed'` for a provisional multiplier), as any + * sample there is. + * - `'stepped'`: the multiplier moved a step toward what the sample implies, + * within the clamp. + */ +export const CALIBRATION_SAMPLE_OUTCOMES = ['out-of-band', 'deferred', 'held', 'confirmed', 'replaced', 'stepped'] as const; + +/** An outcome of {@link CALIBRATION_SAMPLE_OUTCOMES}. */ +export type CalibrationSampleOutcome = typeof CALIBRATION_SAMPLE_OUTCOMES[number]; + +/** One `ContextManager.compile()`, refused or failed ones included. */ +export interface CompileRecord { + /** The strategy's `name`. */ + strategy: string; + /** Whether it was a dry compile (`SelectOptions.dryRun`). */ + dryRun: boolean; + /** When it started, on the wall clock (`Date.now()`, ms since epoch). */ + startedAt: number; + /** How long the whole `compile()` took, on the monotonic clock, in ms. */ + durationMs: number; + /** + * `'compiled'`; `'refused'`, when it threw one of {@link COMPILE_REFUSALS}; + * or `'failed'`, when it threw anything else. + */ + outcome: 'compiled' | 'refused' | 'failed'; + /** + * What a refused or failed compile threw, as the caller receives it, such as + * an `OverBudgetError` with its `stageId`, `budget`, `actual`, `clearedBy` and + * `diagnostics`. Treat it as read-only. + */ + error?: unknown; + /** + * A compiled compile's render stats, the object its `CompileResult.renderStats` + * holds (calibration in force, solver, the view's message count), when its + * strategy records them. Treat them as read-only. + */ + renderStats?: RenderStats; +} + +/** One refusal context-manager made of compression (see {@link COMPRESSION_REFUSALS}). */ +export interface CompressionRefusalRecord { + reason: CompressionRefusal; +} + +/** One calibration sample: the provider report a compile's estimate is compared with. */ +export interface CalibrationSampleRecord { + outcome: CalibrationSampleOutcome; + /** The provider's input total over the compile's estimate. */ + ratio: number; + /** The multiplier in force after the sample. */ + multiplier: number; +} + +/** + * Called synchronously as each fact happens, each callback optional. What + * it throws, or a promise it returns that rejects, never reaches the work it + * observes: the manager logs a callback's first failure through its + * diagnostics, and drops the rest. Outside that is a returned promise no + * JavaScript can give a handler, such as one whose `constructor` or species + * can't be used, or a proxy around one: its rejection goes unhandled, which + * ends a Node process by default. A record's error and render stats are the + * objects the caller gets, so an observer treats them as read-only. + */ +export interface ContextObserver { + compile?(record: CompileRecord): void; + compression?(record: CompressionRefusalRecord): void; + calibration?(record: CalibrationSampleRecord): void; +} + +/** + * `observer` with each callback it has wrapped, so that nothing it throws or + * rejects reaches the caller, but for a returned promise no JavaScript can + * give a handler (see {@link ContextObserver}). `failed` hears of a + * callback's first throw or rejection, once per callback, so a broken + * observer shows without a line per record. Each callback is read once, here, and that function is the one + * called, so replacing or removing it later changes nothing; one that can't + * even be read is treated as absent, and told of the same way. + */ +export function isolateObserver( + observer: ContextObserver, + failed: (callback: keyof ContextObserver, error: unknown) => void, +): ContextObserver { + const isolated: ContextObserver = {}; + const told = new Set(); + const fail = (callback: keyof ContextObserver, error: unknown): void => { + if (told.has(callback)) return; + told.add(callback); + try { + failed(callback, error); + } catch { + // Telling of a failure can't end the work either. + } + }; + for (const name of ['compile', 'compression', 'calibration'] as const) { + let callback: unknown; + try { + callback = observer[name]; + } catch (error) { + fail(name, error); + } + if (typeof callback !== 'function') continue; + const method = callback; + isolated[name] = (record: never) => { + try { + const returned: unknown = Reflect.apply(method, observer, [record]); + if ((typeof returned === 'object' && returned !== null) || typeof returned === 'function') { + // The fulfilment handler returns nothing, so nothing returned is + // read again. + const handlers = [() => undefined, (error: unknown) => fail(name, error)]; + let handled = false; + try { + // A native promise, a subclass's included, takes the handler on + // itself through the intrinsic `then`, so a `then` of its own + // can't skip it. The intrinsic `then` refuses anything else + // before reading it. + Reflect.apply(Promise.prototype.then, returned, handlers); + handled = true; + } catch { + // Not a native promise, or one whose constructor or species + // can't be used; the adoption below tells of the latter. + } + if (!handled) { + // Anything else may be a thenable. Promise.resolve reads its + // `then` at once and calls it in a job of its own, so a `then` + // getter that throws, or a `then` that throws before it settles, + // rejects the promise handled here. (A promise no JavaScript can + // give a handler, such as one whose constructor or species can't + // be used, or a proxy around one, keeps its own rejection + // unhandled. What it throws is told where the adoption or the + // catch below meets it, if either does: one whose species can't + // be used and whose own `then` is undefined is adopted as a plain + // value, and nothing is told.) + Reflect.apply(Promise.prototype.then, Promise.resolve(returned), handlers); + } + } + } catch (error) { + // An observer can't end a compile, a compression or a sample. + fail(name, error); + } + }; + } + return isolated; +} + +/** + * Whether `error` is one of {@link COMPILE_REFUSALS}. A value whose `name` + * can't be read, such as a revoked proxy, isn't, and reading it never throws, + * so the caller still gets what the compile threw. + */ +export function isCompileRefusal(error: unknown): boolean { + try { + const name = error !== null && typeof error === 'object' ? (error as { name?: unknown }).name : undefined; + return typeof name === 'string' && (COMPILE_REFUSALS as readonly string[]).includes(name); + } catch { + return false; + } +} diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 615138a7..d02ba5d1 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -5,6 +5,7 @@ import { registerStateIfAbsent } from '../state-registration.js'; import { phaseChannel } from '../phase-channel.js'; import type { ContextStrategy, + ImageCeilings, ResettableStrategy, StrategyContext, ReadinessState, @@ -35,6 +36,7 @@ import type { SelectOptions, PreviewResult, RenderedSummaryInfo, + ThinkingBinding, } from '../types/index.js'; import { DEFAULT_AUTOBIOGRAPHICAL_CONFIG } from '../types/index.js'; import { getSummaryParentId } from '../types/strategy.js'; @@ -54,7 +56,7 @@ import { persistMintRequestPreimage } from '../mint-preimage.js'; import { appendFileSync, mkdirSync } from 'node:fs'; import { dirname } from 'node:path'; import { createHash } from 'node:crypto'; -import { Picker, OverBudgetError, UncoveredDropError, type OverBudgetClearedBy, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; +import { Picker, OverBudgetError, UncoveredDropError, overBudgetStage, type OverBudgetClearedBy, type OverBudgetDiagnostics, type OverBudgetRawPin, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; import { FlatProfileStrategy } from '../adaptive/strategies/flat-profile.js'; import { KvStableStrategy } from '../adaptive/strategies/kv-stable.js'; import { reportOverlap } from '../adaptive/kv-control.js'; @@ -78,13 +80,27 @@ import type { import { chunkMessage, DEFAULT_CHUNKER_OPTIONS } from '../adaptive/chunker.js'; import { observeStoreBranch } from '../branch-generation.js'; import { bodyBoundsIn, bodyEnd, bodyStart } from '../body-runs.js'; +import { canonicalJson } from '../canonical-json.js'; +import { withoutThinking } from '../thinking-binding.js'; import { releaseEditedRawForms, releaseSharedRawFormsOnMove, releaseTrailingReasoning, releaseVanishedRawForms, withoutRawForms } from '../raw-forms.js'; import { isCompressionWithdrawn } from '../compression-withdrawn.js'; import { hasImage, imagePayload, imagesIn, inlineImageBytes, isImageBlock, replaceImages, strippedImageSaving } from '../inline-images.js'; import { CAP_OMISSION_TEXT, capContent, cappedPriceExtra, cappedWhole, isCapMarker, isEmptyTextBlock } from './message-cap.js'; +import { + DEFAULT_IMAGE_CEILING, + DEFAULT_IMAGE_BYTE_CEILING, + clampToCeiling, + liveImageLimitsWarning, + ceilingBoundLine, + ceilingSetLimits, + resolveLiveImageLimits, + type ResolvedImageLimits, +} from '../live-image-limits.js'; import type { MessageId } from '../types/message.js'; -import type { IngressChunkResult } from '../types/strategy.js'; +import type { CarrierPolicy, IngressChunkResult } from '../types/strategy.js'; +import type { CompressionRefusal } from '../observer.js'; import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; +import { holdsSystemChange, isSystemChange, systemChangePositions } from '../system-changes.js'; import { storedJson } from '../stored-json.js'; /** A persisted reset record is trusted only if it has the expected shape. */ @@ -355,9 +371,9 @@ function nextFailureStreak( * An L1 run's subject: the chunk's rung, by the chunk's sources (see * chunkRunSubject) and the rung's curve label, the stripped retry after a * carrier rejection being its own. Not the request's bytes: a pending - * chunk's request carries the whole unmerged frontier as recall pairs, so - * every other L1 that lands changes it, and those landings are the evidence - * a run needs. + * chunk's request carries the unmerged frontier summaries that start before + * it as recall pairs, so an L1 that lands for history before it changes it, + * and those landings are the evidence a run needs. */ function chunkRunKey(runSubject: string, curveLabel: string, stripped: boolean): string { return sha256Json({ runSubject, curveLabel, stripped }); @@ -609,6 +625,28 @@ type ToolPruningDecision = { readonly inputCap: number; }; +/** The verdict postStripEstimates gives one image, which the strip applies + * (applyImageStripping's asEstimated): the image's payload (imagePayload), + * whether it's kept, and, for the ceiling line, the count and byte limits + * it passes. */ +type ImageVerdict = { + key: unknown; + keep: boolean; + over: CeilingKinds; +}; + +/** The count limit and the byte limit, by the ceiling that can set each. */ +type CeilingKinds = { images: boolean; bytes: boolean }; + +/** imageStripHysteresisRatio's set at the window's limits + * (stripReplayAtLimits): how many of each message's images it strips, and + * which of the count and byte limits brought a trim on anywhere in the + * history it replays, for the ceiling line. */ +type StripReplay = { + out: ReadonlyMap; + over: Readonly; +}; + /** A tool call the conversation answers: its message's place in the listing, * the place of the result that first answered it, and every result that * answers it, by message and place in its content. */ @@ -1088,6 +1126,26 @@ type CompressionAttemptOutcome = */ | 'incomplete'; +/** + * The refusal an outcome is when context-manager itself made it (see + * `COMPRESSION_REFUSALS`), or nothing. A provider's refusal, and an error it + * returned or a run of them decided, are the provider's decisions, which + * context-manager doesn't report; a response's shape is context-manager's + * verdict (`assessFallbackCompressionResponse`'s `malformed_*`). + */ +function ownCompressionRefusal(outcome: CompressionAttemptOutcome, errorType?: string): CompressionRefusal | undefined { + switch (outcome) { + case 'admission_rejected': return 'admission-rejected'; + case 'topology_violation': return 'topology-violation'; + case 'unusable_empty': return 'empty'; + case 'incomplete': return 'incomplete'; + case 'provider_error': return errorType?.startsWith('malformed_') ? 'malformed' : undefined; + case 'refusal': return undefined; + // A new outcome has to be decided here before this compiles. + default: { const undecided: never = outcome; return undecided; } + } +} + /** * Typed rejection thrown by `executeMerge` when the merge response fails * the terminal-disposition gate. `tick()` catches this specifically to run @@ -1381,8 +1439,12 @@ interface CompressionInFlightResult { interface PreparationCounts { serializedMediaReplacements: number; imageReplacements: number; + /** The inline image byte budget, 0 for none. */ imageBudgetBytes: number; imageBytesKept: number; + /** The most images the prompt may carry (imageCeiling), 0 for none. */ + imageCountCap: number; + imagesKept: number; } /** A compression request after preparation, with what preparation replaced. */ @@ -1434,21 +1496,6 @@ function sha256Json(value: unknown): string { return createHash('sha256').update(JSON.stringify(value), 'utf8').digest('hex'); } -/** - * JSON with each object's keys in one order whatever order they were set in - * (sorted, though integer-like keys come first, ascending, as in any object), - * and each array's items in their own order, so two values that differ only - * in the order their keys were set give the same text. An object the store - * keeps comes back with its keys sorted, while one built here keeps the order - * it was built in. - */ -function canonicalJson(value: unknown): string { - return JSON.stringify(value, (_key, inner: unknown) => { - if (inner === null || typeof inner !== 'object' || Array.isArray(inner)) return inner; - return Object.fromEntries(Object.keys(inner).sort().map((key) => [key, (inner as Record)[key]])); - }); -} - /** * Sources as readers iterate them: a non-empty list of ids. Every index is * read, since `every` skips an empty slot of a sparse array, which the store @@ -1649,15 +1696,22 @@ export function stripEmptyTextBlocks(content: ContentBlock[]): ContentBlock[] { * `full` replay arm). Passing raw thinking through requires keeping those * turns byte-identical — a follow-up experiment, not a blanket strip removal. * - * A stripped Responses reasoning item releases the raw item it was paired - * with, so the follower isn't replayed with an id whose reasoning is gone - * (see raw-forms.ts). + * The blocks are left out as `withoutThinking` leaves them, raw forms + * included (thinking-binding.ts). */ -function stripThinkingBlocks(content: ContentBlock[]): ContentBlock[] { - return releaseEditedRawForms( - content, - content.filter((b) => b.type !== 'thinking' && b.type !== 'redacted_thinking'), - ); +const stripThinkingBlocks = withoutThinking; + +/** + * A raw message's content as it enters compression input: its thinking + * stripped (stripThinkingBlocks), and its system changes left out. A system + * change alters the request's tools and system text, not the conversation, and + * a summarizer's request would apply it, or be refused for it + * (system-changes.ts). A message of system changes alone enters empty, and + * every request path drops an empty message before sending. + */ +function compressionInputContent(content: ContentBlock[]): ContentBlock[] { + const stripped = stripThinkingBlocks(content); + return holdsSystemChange(stripped) ? stripped.filter((b) => !isSystemChange(b)) : stripped; } /** @@ -1856,8 +1910,10 @@ function imageGroup(block: ContentBlock): unknown { } /** Out-of-band calibration samples in a row it takes to learn from them, and - * how close each one's implied multiplier must be to the one before it to - * continue the run (see reportRealInputTokens). */ + * how close each one's implied multiplier must be to the run's first to + * continue the run (see reportRealInputTokens). The agreement also decides + * whether two declared samples move a store whose multiplier wasn't + * learned from declared samples. */ const OUT_OF_BAND_RUN_LENGTH = 3; const OUT_OF_BAND_RUN_AGREEMENT = 0.1; @@ -1960,7 +2016,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { */ protected chunkRecordsOrphaned = false; private _orphanWarned = false; - /** Record ids whose compression was refused by the L1 overlap guard. */ + /** Chunks the L1 overlap guard refused: a chunk's record id, or its message-id key (chunkKey) when it has none. */ private _overlapBlocked = new Set(); /** @@ -2385,7 +2441,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { log: ContextLogView, budget: TokenBudget, overrides?: AutobiographicalOptions, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ): PreviewResult { this.requireLoadedBranch('previewContext'); if (this._previewInFlight) { @@ -2420,7 +2476,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } let rendered: ContextEntry[] | undefined; try { - const out = this.select(store, log, budget, { dryRun: true }); + const out = this.select(store, log, budget, { dryRun: true, thinkingBinding: opts?.thinkingBinding }); // Only retained on request: these are full rendered entries and can be // megabytes on a large store. if (opts?.render) rendered = out; @@ -2548,7 +2604,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * lock while it stands: under the greedy solvers and kv-stable, the pinned * message renders raw, and its locked level stays stored and returns when * the pin is removed. kv-unified treats a pin and a lock as constraints and - * reports a conflict when they are incompatible. + * reports a conflict when they are incompatible. A pin on another part + * of a message stored in parts doesn't cover a locked part outside the span + * its ids name (`pinPositions`): there the lock holds, under every solver. * No-op when adaptiveResolution is false. Set-only programmatic API per * the design (no agent-facing tool in V1). Persisted to chronicle. */ @@ -2660,6 +2718,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Bind to the chronicle store + namespace for persistent strategy state. this.store = ctx.store; this.ns = ctx.namespace; + this.reportImageLimitsWarningOnce(); // Its tools getter follows the host's declarations (compressionWorkQueued). this.hostContext = ctx; observeStoreBranch(ctx.store); @@ -2784,6 +2843,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const call of this.callsOut) call.abort(); } + /** Tell the host's observer of a compression outcome that is a refusal context-manager made itself (ownCompressionRefusal). */ + private reportCompressionRefusal(outcome: CompressionAttemptOutcome, errorType?: string): void { + const reason = ownCompressionRefusal(outcome, errorType); + if (reason !== undefined) this.hostContext?.observer?.compression?.({ reason }); + } + /** * `membrane.complete()` for the current generation's work, with a * controller of its own kept in callsOut while the call is out, so that @@ -2852,6 +2917,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._lastRenderStats = null; this._lastSelectStats = null; this._lastCompileEstimate = 0; + this._lastCompileInjected = 0; this._storeView = null; // Prices memoized over the view just cleared: the next one may estimate otherwise. this._answerCostCache.clear(); @@ -2861,7 +2927,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibrationLoaded = false; this._calibrationReset = null; this._calibrationProvisional = null; + this._calibrationDeclared = false; this._outOfBandRun = null; + this._declaredCandidate = null; this._imageStripReplayMemo = null; this._lastKvStable = null; this._storeOrder = new Map(); @@ -3521,7 +3589,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel` (`normalizePinLevels`). A raw pin, with no bound or a * bound of 0, keeps the messages out of compression and renders them raw - * at their original position. A bound above 0 is read only under + * at their original position. A compile whose budget can't hold them + * beside the rest refuses with `OverBudgetError`; where the adaptive plan's + * middle is over the budget, it names each pin keeping raw tokens in that + * middle (`rawPinsDiagnostics`). Under adaptive resolution, a message + * stored in parts counts as one: a pin whose first or last id names one of its parts + * covers it from its first part to its last, less a part locked with + * `lockChunk` outside the span its ids name, for every reader of pins + * (`pinPositions`). A bound above 0 is read only under * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution` (`readsPinBounds`): there the range is still * compressed, since only a summary can supply its level, and is held at @@ -3553,6 +3628,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { * with `kind: 'document'`, and `opts` reads as `pinRange`'s: without a * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. + * Under adaptive resolution, a mark on any part of a message stored in + * parts covers the whole message, as `pinRange` does (`pinPositions`). + * Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each + * summary it reaches, so a locked part that shares a summary with a part + * the mark covers renders at the level the mark holds that summary at. */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -3770,21 +3850,50 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * Build a set of message-store positions covered by any pin. O(N pins · K range). - * Returns positions for which the message exists; orphan pins (deleted - * messages) are silently skipped. + * The store positions a pin covers, in order. Without adaptive resolution, + * the messages from its first id to its last. Under adaptive resolution, + * the only ingress that stores a message in parts, such a message counts as + * one: the span runs from the first part of the message its first id is in + * to the last part of the message its last id is in (`bodyBoundsIn`), less + * a part locked with `lockChunk` outside the span its ids name. A lock is + * said of that part, while covering the rest of its message is what a pin + * on another part implies, so the lock holds there; a pin over its own + * locked messages keeps the rule `lockChunk` describes. Every reader of + * pins takes this: the selects, compression, the summarizer's prompt and + * the replay exporter. Undefined when either id is no longer in the store. + */ + protected pinPositions( + pin: ProtectedRange, + positionOf: ReadonlyMap, + messages: readonly StoredMessage[], + ): number[] | undefined { + const first = positionOf.get(pin.firstMessageId); + const last = positionOf.get(pin.lastMessageId); + if (first === undefined || last === undefined) return undefined; + const lo = Math.min(first, last); + const hi = Math.max(first, last); + const whole = this.config.adaptiveResolution === true; + const from = whole ? bodyBoundsIn(messages, lo).from : lo; + const to = whole ? bodyBoundsIn(messages, hi).to : hi; + const out: number[] = []; + for (let i = from; i <= to; i++) { + if ((i < lo || i > hi) && this.locked.has(messages[i].id)) continue; + out.push(i); + } + return out; + } + + /** + * Build a set of message-store positions covered by any pin + * (`pinPositions`). O(N pins · K range). Returns positions for which the + * message exists; orphan pins (deleted messages) are silently skipped. */ protected pinnedPositions(messages: StoredMessage[]): Set { if (this.pins.length === 0) return new Set(); const positionOf = this.positionIndex(messages); const out = new Set(); for (const pin of this.pins) { - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) continue; - const lo = Math.min(first, last); - const hi = Math.max(first, last); - for (let i = lo; i <= hi; i++) out.add(i); + for (const i of this.pinPositions(pin, positionOf, messages) ?? []) out.add(i); } return out; } @@ -3804,14 +3913,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected resolvedPinBounds(messages: StoredMessage[]): Map { const out = new Map(); const pinned = this.pinnedPositions(messages); - if (pinned.size === 0) return out; - const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; - for (const position of pinned) { - const bound = bounds?.get(position); - if (bound?.level !== undefined) out.set(position, { level: bound.level }); - else if (bound?.maxLevel !== undefined) out.set(position, { maxLevel: bound.maxLevel }); - else out.set(position, 'raw'); + if (pinned.size > 0) { + const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; + for (const position of pinned) { + const bound = bounds?.get(position); + if (bound?.level !== undefined) out.set(position, { level: bound.level }); + else if (bound?.maxLevel !== undefined) out.set(position, { maxLevel: bound.maxLevel }); + else out.set(position, 'raw'); + } } + // A message of system changes is kept raw, as a pin keeps a message: a + // fold would take its change out of what follows it (holdsSystemChange). + for (const position of systemChangePositions(messages)) out.set(position, 'raw'); return out; } @@ -3842,8 +3955,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * The V2 dynamic-pin fold-depth bounds (`ProtectedRange.level` / * `maxLevel`) by message position. Only pins that carry a bound appear; a * classic raw pin (no bound) is absent here and handled by `pinnedPositions`. - * Where bounded pins overlap, the lowest fixed level and the lowest cap are - * kept; `resolvedPinBounds` decides between them and a classic pin. + * A pin covers what `pinPositions` says. Where bounded pins overlap, the + * lowest fixed level and the lowest cap are kept; `resolvedPinBounds` + * decides between them and a classic pin. */ protected pinLevelBounds(messages: StoredMessage[]): Map { const out = new Map(); @@ -3852,12 +3966,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const pin of this.pins) { if (pin.level === undefined && pin.maxLevel === undefined) continue; - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) continue; - const lo = Math.min(first, last); - const hi = Math.max(first, last); - for (let i = lo; i <= hi; i++) { + for (const i of this.pinPositions(pin, positionOf, messages) ?? []) { const prev = out.get(i) ?? {}; // A fixed level is the strongest constraint; when two pins fix the same // position, the shallower (lower) level wins (finest requirement). @@ -4024,12 +4133,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { * Consecutive compression/merge requests share a long, append-mostly * prefix — head window + prior recall pairs (~60% of input on synthetic * workloads, ~93% on mature stores) — that previously went to the API - * with zero `cache_control` and was re-read cold on every call. Markers + * with zero `cache_control` and was re-read cold on every call. They share + * the head window only while they summarize one topic's history: after a + * head-window reset, requests for different topics' history each carry + * that topic's head (headAsOf). Markers * land at the three boundaries whose survival probability ranks highest * (measured optimum; `analysis/2026-08-20-cm37-cache-partition-experiments.md` * in the workspace root): * - * 1. end of head window — survives everything short of identity edits + * 1. end of head window — survives everything short of identity edits, + * among requests for one topic's history * 2. last level>=2 recall pair — survives appends AND L1→L2 merges (a * merge replaces the run at this boundary and extends it) * 3. last recall pair — the append frontier @@ -4093,7 +4206,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (frontierEnd === -1) return false; // no ladder yet — nothing stable to cache // A capped ladder front-evicts on every mint, so the pair seams shift and // writing them is waste. The head seam does not: tools, system prompt and - // head window precede the ladder and are the same request after request, + // head window precede the ladder and are the same request after request + // for one topic's history (headAsOf gives each topic its own head), // unless image shedding (one byte budget, spent newest-first) replaces a // head image for one payload and not for the next. let placed = false; @@ -6663,6 +6777,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; this.mergeQuarantine.set(record.key, record); this.persistMergeQuarantine(); + this.reportCompressionRefusal('topology_violation'); this.diag.error( `[merge-topology] ⛔ refused L${targetLevel} merge over ${sourceIds.length} source(s) ` + `(${sourceIds.join(', ')}): ${reason}. The node was NOT minted; entry quarantined ` + @@ -7895,14 +8010,27 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the next compile's stable prefix and fold diff are measured against // the request before it. const prevCacheKeys = this._prevCacheKeys; + // What this select's strips record is what it reports. A dry select's + // strips, and a refused one's, record too; each live select starts by + // clearing what earlier ones left, so it reports its own render's. + this._ceilingNotes = { images: false, bytes: false }; + let entries: ContextEntry[]; + this.thinkingBound = opts?.thinkingBinding?.binds === true; try { - return this.withToolPruning(store, () => this.selectPath(store, budget, opts)); + entries = this.withToolPruning(store, () => this.selectPath(store, budget, opts)); } catch (err) { this._prevCacheKeys = prevCacheKeys; throw err; } finally { + this.thinkingBound = false; if (this._lastRenderStats !== lastRenderStats) this._lastSelectStats = this._lastRenderStats; } + // Only a render this select returns says the ceiling line: one it + // refused threw above, and a dry select never comes here. Every select + // path strips its render last (applyImageStripping, `returned`), and a + // strip with no limits records nothing, so the hits are this render's. + this.reportCeilingHits(); + return entries; } // A dry run commits nothing, and that includes what a select leaves for // the next one and for observers: the cache-marker baseline the next @@ -7932,6 +8060,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // one that re-queued its chunk (requeuePendingChunk). const restore = this.saveDryRunState(store); this._dryRunSelect = true; + this.thinkingBound = opts?.thinkingBinding?.binds === true; try { this.rebuildChunks(store); if (_diag) this.diag.error(`[cm-cache] select: rebuildChunks ${Date.now() - _t0}ms`); @@ -7940,6 +8069,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._dryRunSelect = false; if (this._lastRenderStats !== lastRenderStats) this._lastSelectStats = this._lastRenderStats; restore(); + this.thinkingBound = false; } } @@ -8505,9 +8635,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { const pos = messages.findIndex(x => x.id === id); if (pos < 0) return 'not-in-window'; // Before the LIVE head start (head-window reset moved the anchor): - // this is compressible middle history, not head — a distinct defect - // from a verbatim-head message going unrendered. - if (pos < regions.headStart) return 'before-head-start-not-rendered'; + // this is middle history, not head — a distinct defect from a + // verbatim-head message going unrendered — compressible but for a + // system change, which stays raw there, except while the head window + // holds messages, which is where a reset leaves it (headPositions). + if (pos < regions.headStart) { + return holdsSystemChange(messages[pos].content) && Math.min(regions.headEnd, messages.length) > regions.headStart + ? 'in-head-not-rendered' + : 'before-head-start-not-rendered'; + } if (pos < regions.headEnd) return 'in-head-not-rendered'; if (pos >= regions.recentStart) return 'in-tail-not-rendered'; return 'in-middle-not-rendered'; @@ -9032,6 +9168,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const key = chunk.recordId ?? chunkIdKey; if (!this._overlapBlocked.has(key)) { this._overlapBlocked.add(key); + this.hostContext?.observer?.compression?.({ reason: 'overlap-guard' }); this.diag.error( `[autobiographical] OVERLAP GUARD: refusing to compress chunk ` + `${chunk.recordId ?? `#${chunk.index}`} — ${overlapIds.length}/${chunk.messages.length} ` + @@ -9118,9 +9255,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ("Antra came to me to explore the transformation story // again…"), compounding across merges into runaway false // memories (the "68 initiations" incident). Chronological - // order is also the KV-stable order — the head never changes. - // (executeMerge always had head-first; this site was the odd - // one out.) + // order is also the KV-stable order: chunks of one topic after its + // head share that head, so their requests share its prefix. The + // head is the head as of the chunk (headAsOf): after a head-window + // reset, the head that history followed, the opening or its own + // topic's, never the later topic's. (executeMerge always had + // head-first; this site was the odd one out.) // 2. Prior summaries — narrativized as CM-asks / agent-recalls // pairs, in source order. The unmerged frontier of the // summary forest: any summary that has not yet been merged @@ -9138,10 +9278,25 @@ export class AutobiographicalStrategy implements ResettableStrategy { // future information into the model's KV state and corrupt the // as-of framing of memory formation. const allMessages = ctx.messageStore.getAll(); - const headStartIdx = this.getHeadWindowStartIndex(ctx.messageStore); - // Compression holds: provisional content never enters a prompt, not even - // as head context (a reset head can sit past the hold boundary). - const headEndIdx = Math.min(this.getHeadWindowEnd(ctx.messageStore), this.holdBoundary(ctx.messageStore)); + // Message IDs are opaque strings, not sortable sequence numbers. Chronicle's + // default IDs happen to be decimal text, where lexical order corrupts the + // timeline at the first width transition (`"103" < "15"`). Use the store's + // canonical order instead so recall pairs remain chronological for every ID + // representation and newly-created summaries extend rather than splice the + // provider-cache prefix. + const messageOrder = new Map( + allMessages.map((message, index) => [message.id, index]), + ); + const chunkFirstId = chunk.messages[0]?.id; + // The chunk's start is its first message still in the store: a host can + // remove a stored message (agent-framework's Discord module does, on a + // delete) while the chunk holding it waits in the queue. -1 only when + // none of its messages is left (startInStore). + const chunkStartIdx = this.startInStore(chunk.messages.map((m) => m.id), messageOrder); + // The head as of the chunk (headAsOf): after a head-window reset, history + // from before it is given the head it followed, not the new topic's, which + // came after it. Clipped at a hold there, as before. + const { start: headStartIdx, end: headEndIdx } = this.headAsOf(ctx.messageStore, chunkStartIdx); // ---- Prior recall set (the unmerged frontier) ---- // Computed BEFORE the head emission because the head loop needs the @@ -9154,15 +9309,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { // optimization that scales catastrophically: a 4000-message import // converged to ~500 L1s that never aged out, blowing the 200k // window around chunk 118. - // Message IDs are opaque strings, not sortable sequence numbers. Chronicle's - // default IDs happen to be decimal text, where lexical order corrupts the - // timeline at the first width transition (`"103" < "15"`). Use the store's - // canonical order instead so recall pairs remain chronological for every ID - // representation and newly-created summaries extend rather than splice the - // provider-cache prefix. - const messageOrder = new Map( - allMessages.map((message, index) => [message.id, index]), - ); // Pins in the canonical L1 prompt. The live view shows a message that a // pin keeps raw (forceRawPinPositions) raw, so this prompt does too. A // frontier summary that can't be shown as a recall pair (it covers such a @@ -9200,20 +9346,41 @@ export class AutobiographicalStrategy implements ResettableStrategy { // stay declared (reasoning_extraction guard). L1 only; merge/transition // paths do not consult this flag. const sourceOnly = this.config.compressionSourceOnly === true; - const chunkFirstId = chunk.messages[0]?.id; - const chunkStartIdx = chunkFirstId ? allMessages.findIndex((m) => m.id === chunkFirstId) : -1; // History before the chunk, laid out for the messages that pins keep raw. const layoutHistory = (forceRawMessageIds: ReadonlySet) => { // The recall frontier, the chunks pins open and the leaf coverage, as // compressionWorkQueued's count reads them too (recallCoverage). const { frontier, openedMessageIds, coveredMessageIds: priorSummaryMessageIds } = this.recallCoverage(forceRawMessageIds); + // Recall pairs only for history before the chunk: a summary whose + // source range starts before the chunk's first message, as a merge's + // prior recall set takes only what starts before its range (1a). A + // chunk compressed after later history already has summaries (the old + // head's chunk, which a head-window reset makes compressible after the + // chunks behind it; a pinned or quarantined chunk let go late) was + // given their recall pairs too, ahead of it: what came after it + // (#317). It now sees fewer pairs: only those that start before it. + // Coverage is the whole frontier's still, so a later summary's + // messages stay out of the raw middle (one-to-one). A summary that + // starts before the chunk and reaches past it (a sparse sweep + // artifact, issue #42) is still recalled, since the messages it covers + // before the chunk have no other form here; so its later part, what it + // says of history after the chunk, still reaches this request, and only + // where a summary straddles the chunk. A summary written before this + // rule may also carry what came after its own history, from the head or + // recall pairs it was written with; that stays in its text. Where a + // summary starts is summaryStartInStore's rule, the one + // a merge's prior recall set uses: a summary whose first message a host + // removed starts at its first message still there, and one with none + // left isn't recalled. A chunk with no message left (chunkStartIdx -1) + // recalls none. + const summariesById = new Map(); + for (const s of this.summaries) summariesById.set(s.id, s); + const startOf = new Map(); + for (const s of frontier) startOf.set(s.id, this.summaryStartInStore(s, messageOrder, summariesById)); const priorSummaries = frontier - .sort((a, b) => { - const aOrder = messageOrder.get(a.sourceRange.first) ?? Number.MAX_SAFE_INTEGER; - const bOrder = messageOrder.get(b.sourceRange.first) ?? Number.MAX_SAFE_INTEGER; - return aOrder - bOrder || a.sourceRange.first.localeCompare(b.sourceRange.first); - }); + .filter((s) => { const at = startOf.get(s.id)!; return at >= 0 && at < chunkStartIdx; }) + .sort((a, b) => startOf.get(a.id)! - startOf.get(b.id)! || a.sourceRange.first.localeCompare(b.sourceRange.first)); // Raw messages before the chunk outside the head window, by position. // From the head window's end on, any message no summary represents yet @@ -9257,7 +9424,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { rawMiddle.push({ position: i, message: m }); } } - return { priorSummaries, priorSummaryMessageIds, rawMiddle, pinnedRaw, pinnedRawTokens }; + return { priorSummaries, startOf, priorSummaryMessageIds, rawMiddle, pinnedRaw, pinnedRawTokens }; }; // When the chunk is a portion of a substantially larger sharded message @@ -9283,7 +9450,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Pins count only before the chunk being compressed: this prompt is the // history the chunk follows. A summary after it (a chunk compressed out of - // order, after a head-window reset or an unpin) is recalled as on main. + // order, after a head-window reset or an unpin) isn't recalled at all + // (layoutHistory), and its messages stay covered. const forceRawMessageIds = new Set(); for (const i of this.forceRawPinPositions(allMessages)) { if (i < chunkStartIdx) forceRawMessageIds.add(allMessages[i].id); @@ -9293,7 +9461,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The canonical request's messages for one layout of history, sections 1 // to 6, shaped as they are sent. Admitting the pins prices this list. const assemble = ( - { priorSummaryMessageIds, rawMiddle }: ReturnType, + { startOf, priorSummaryMessageIds, rawMiddle }: ReturnType, keptSummaries: readonly SummaryEntry[], ) => { const llmMessages: Array<{ participant: string; content: ContentBlock[] }> = []; @@ -9312,7 +9480,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the chunk." For doc-heavy chronicles, "everything before" would // be hundreds of thousands of tokens; the recall pairs below // represent that intermediate content. The head is just the - // permanent prefix that the original instance always saw. + // prefix the original instance saw ahead of this history: the head + // window as of the chunk (headAsOf), which after a head-window reset + // is the opening, or the transition and opening of the chunk's topic. // // ONE-TO-ONE INVARIANT: chunk records (and the summaries over them) // are the persistent authority over how a message is represented. @@ -9331,26 +9501,30 @@ export class AutobiographicalStrategy implements ResettableStrategy { headCoveredSkipped++; continue; } - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } // ---- 2. Prior recall pairs (budget-capped, above) and the raw middle ---- // One sequence by source position after the head window, which always - // comes first, so the summarizer never reads an older raw turn as coming - // after a later memory. Where the raw middle follows every recall pair, - // as it does when chunking runs contiguously, this is the order the - // prompt always had: recall pairs, then raw. + // comes first, so the summarizer never reads an older raw turn, or an + // older memory, as coming after a later memory. A recall pair sits at + // its summary's start as the recall filter read it (layoutHistory's + // startOf, summaryStartInStore): a summary whose first message a host + // removed sits at its first message still in the store, not last. + // Where the raw middle follows every recall pair, as it does when + // chunking runs contiguously, this is the order the prompt always had: + // recall pairs, then raw. const middle: Array<{ position: number; summary?: SummaryEntry; message?: StoredMessage }> = [ ...keptSummaries.map((summary) => ({ - position: messageOrder.get(summary.sourceRange.first) ?? Number.MAX_SAFE_INTEGER, + position: startOf.get(summary.id) ?? Number.MAX_SAFE_INTEGER, summary, })), ...rawMiddle, ].sort((a, b) => a.position - b.position); for (const { summary: s, message: m } of middle) { if (m) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); continue; } llmMessages.push({ @@ -9383,7 +9557,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ---- 5. Chunk messages raw ---- for (const m of chunk.messages) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } // ---- 6. Instruction (reading-mode aware; worked out above) ---- @@ -9801,10 +9975,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const own: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; if (readsBounds && !AutobiographicalStrategy.isRawPinBound(own)) return false; - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) return false; - for (let i = Math.min(first, last); i <= Math.max(first, last); i++) { + for (const i of this.pinPositions(pin, positionOf, allMessages) ?? []) { if (pinnedRawSet.has(i) && forceRawPositions.has(i)) return true; } return false; @@ -9816,7 +9987,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { `autobio: L1 compression of ${chunk.messages.length} message(s) from ${chunkFirstId} recalls the summaries ` + `its pins override: the pinned history it would show raw (~${pinned.pinnedRawTokens} tokens in ` + `${pinned.pinnedRaw.length} message(s); pins: ${pins.join(', ') || 'none found'}) ${limit}. ` + - `Narrow the pins or raise that budget to show them raw here.`, + `Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the ` + + `rest of that message too, less a part locked with lockChunk outside the pin's ids), or raise that budget to ` + + `show them raw here.`, ); logCompressionCall({ event: 'compression:pinned-raw-over-budget', @@ -10140,6 +10313,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { familyRecord.push(outcome); endRungRuns(outcome.curveLabel); persistProgress(); + this.reportCompressionRefusal(outcome.outcome, outcome.errorType); }; /** * A failure that is not a rejection says nothing about the request by its @@ -10470,6 +10644,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { admittedTokens: admission.admittedTokens, budgetTokens: admission.budgetTokens, }); + this.reportCompressionRefusal('admission_rejected'); logCompressionCall({ event: 'compression:curve-attempt', operation: 'compress_l1', metadata: trace }, this.ns); return; } @@ -10565,6 +10740,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { admittedTokens, budgetTokens: contextBudget, }); + this.reportCompressionRefusal('admission_rejected'); logCompressionCall({ event: 'compression:curve-attempt', operation: 'compress_l1', metadata: trace, }, this.ns); @@ -10749,7 +10925,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const buildSub = (subset: typeof chunk.messages, target: number): NormalizedRequest => { const msgs = [ { participant: 'Context Manager', content: [{ type: 'text', text: COMPRESSION_MARKER }] as ContentBlock[] }, - ...subset.map((m) => ({ participant: m.participant, content: stripThinkingBlocks(m.content) })), + ...subset.map((m) => ({ participant: m.participant, content: compressionInputContent(m.content) })), { participant: 'Context Manager', content: [{ type: 'text', text: this.applyIdentityReminder(this.getCompressionInstruction(chunk, target), this.chunkIsWitnessed(chunk)) }] as ContentBlock[] }, ]; const cleaned = stripUnpairedToolBlocks(this.collapseConsecutiveMessages(splitMixedToolMessages(msgs))); @@ -10787,6 +10963,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { let modelReportedLeaves = 0; const splitStart = Date.now(); let abortReason: string | undefined; + // Which cap ended the rung, typed as the reasons the host is told, so + // a cap spelled otherwise here fails to compile rather than going + // unreported. + let capTripped: Extract | undefined; const MAX_SPLIT_CALLS = this.config.compressionSplitMaxCallsPerChunk ?? 40; const WINDOW_MS = 10 * 60_000; const MAX_WINDOW_CALLS = this.config.compressionSplitMaxCallsPer10Min ?? 80; @@ -10801,13 +10981,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (abortReason) return false; const subset = chunk.messages.slice(a, b + 1); if (attempt) { - if (calls >= MAX_SPLIT_CALLS) { abortReason = 'chunk-call-cap'; return false; } + if (calls >= MAX_SPLIT_CALLS) { abortReason = capTripped = 'chunk-call-cap'; return false; } // This attempt alone has used the whole window: under these caps // the rung can't finish, which is its outcome, like the per-attempt // cap. Otherwise the window is shared with earlier calls (reached // only when the per-attempt cap exceeds the window), and the rest // of the attempt waits for it like a rung's start. - if (calls >= MAX_WINDOW_CALLS) { abortReason = 'window-call-cap'; return false; } + if (calls >= MAX_WINDOW_CALLS) { abortReason = capTripped = 'window-call-cap'; return false; } const windowWait = this.splitWindowWait(1, MAX_WINDOW_CALLS, WINDOW_MS); if (windowWait) throw windowWait; const partTokens = sentTokens.slice(a, b + 1).reduce((n, c) => n + c, 0); @@ -10891,6 +11071,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { } else { trace.outcome = assessment.outcome; if (assessment.outcome === 'provider_error') { abortReason = `provider-error:${assessment.errorType ?? 'unclassified'}`; return false; } + // Each piece's empty or incomplete answer is the library's verdict + // on a call, reported as it's made, whether the rung goes on to + // split the piece or ends here. A refusal reports nothing. + this.reportCompressionRefusal(assessment.outcome); } } // Refused (or empty): split at a lawful boundary, never inside a tool round. @@ -11012,6 +11196,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { ...(abortReason ? { errorType: rejectedAs ?? abortReason } : {}), }; noteOutcome(splitOutcome); + // A cap's trip ends the rung before a call, a refusal the library + // makes itself; its record stays a refusal, which reports nothing. + // Each piece's empty or incomplete answer was reported as it came, + // and a malformed piece is reported through noteOutcome. + if (capTripped !== undefined) { + this.hostContext?.observer?.compression?.({ reason: capTripped }); + } this.diag.error(`[autobiographical] split-stitch abandoned for chunk ${chunk.index}: ${reason} after ${calls} call(s), ${parts.length} piece(s) discarded (receipts only)`); logCompressionCall({ event: 'compression:split-stitch-abandoned', operation: 'compress_l1', metadata: { quarantine_key: quarantineRecord.key, leaf_hash: leafHash, reason, calls, attempted, piecesDiscarded: parts.length } }, this.ns); } @@ -11802,7 +11993,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // // No tail-after-merge: same as-of principle as L1 compression. The // consolidation is being formed at the moment the last source was - // ready, so nothing after that is visible. + // ready, so nothing after that is visible, except what a recalled + // summary's own text carries: one written before this as-of rule may + // say what came after its history. const llmMessages: Array<{ participant: string; content: ContentBlock[] }> = []; // Build lookup maps @@ -11828,9 +12021,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; for (const src of sources) collectLeaves(src); - // Find the start of the merge range in the message store. - const mergeFirstMsgId = sources[0].sourceRange.first; - const mergeStartIdx = allMessages.findIndex((m) => m.id === mergeFirstMsgId); + // The merge range's start in the store: its first leaf still there + // (startInStore), as a summary's start is read (summaryStartInStore). -1 + // only when none is: nothing in the store says when that history + // happened, so the merge gets no head (headAsOf) and recalls no summary + // (1a), as a compression chunk in that state does. + const positionOf = this.positionIndex(allMessages); + const mergeStartIdx = this.startInStore(sourceLeafIds, positionOf); // ---- 1a. PRIOR RECALL SET (chronologically before merge range) ---- // Computed BEFORE the head emission because the head loop needs the @@ -11844,20 +12041,22 @@ export class AutobiographicalStrategy implements ResettableStrategy { // (`!mergedInto`) lets a merged L1 drop out in favour of its L2/L3 // parent — the same rule used everywhere else and now in // `compressChunkHierarchical`. The cap below is the defense-in-depth. + const priorStart = new Map(); const priorSummariesAll = this.summaries // Skip empty-content summaries (see compressChunkHierarchical): an empty // text block in the merge recall header 400s and stalls compression. .filter((s) => !s.mergedInto && !!s.content && s.content.trim().length > 0) .filter((s) => { for (const lid of s.sourceIds) if (sourceLeafIds.has(lid)) return false; - const firstIdx = allMessages.findIndex((m) => m.id === s.sourceRange.first); - return firstIdx >= 0 && (mergeStartIdx < 0 || firstIdx < mergeStartIdx); + // Where it starts (summaryStartInStore): a summary whose first message + // a host removed starts at its first message still in the store, so + // it's recalled and its other messages, which its coverage holds out + // of the raw middle, keep a form here. One with none left isn't. + const firstIdx = this.summaryStartInStore(s, positionOf, summariesById); + priorStart.set(s.id, firstIdx); + return firstIdx >= 0 && firstIdx < mergeStartIdx; }) - .sort((a, b) => { - const ai = allMessages.findIndex((m) => m.id === a.sourceRange.first); - const bi = allMessages.findIndex((m) => m.id === b.sourceRange.first); - return ai - bi; - }); + .sort((a, b) => priorStart.get(a.id)! - priorStart.get(b.id)!); // Leaf coverage of every live summary — the rendering authority // (one-to-one invariant; see compressChunkHierarchical). Uses the @@ -11880,9 +12079,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ---- 1b. HEAD WINDOW (raw, FIRST, ownership-capped) ---- // - // The head window is the foundational identity anchor — the actual - // opening of the chronicle. It establishes who is speaking to whom. - // Without it, when the merge target's content is heavily first-person + // The head window is the foundational identity anchor — the opening + // the merged history followed (headAsOf: the chronicle's opening, or + // after a head-window reset the opening of its topic). It establishes + // who is speaking to whom. Without it, when the merge target's content + // is heavily first-person // from someone other than the agent, the agent loses its first-person // grounding and drifts into the content author's voice. // @@ -11890,10 +12091,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // wins over the token-derived head boundary. A head-range message // covered by a live summary renders via its recall pair; one inside // the merge tree renders in the TARGET expansion. Never raw here too. - const headStartIdx = this.getHeadWindowStartIndex(ctx.messageStore); - // Compression holds: provisional content never enters a prompt, not even - // as head context (a reset head can sit past the hold boundary). - const headEndIdx = Math.min(this.getHeadWindowEnd(ctx.messageStore), this.holdBoundary(ctx.messageStore)); + // The head as of the merge's first message (headAsOf), as for an L1: a + // merge over history from before a head-window reset is given the head + // that history followed. Clipped at a hold there, as before. + const { start: headStartIdx, end: headEndIdx } = this.headAsOf(ctx.messageStore, mergeStartIdx); let headCoveredSkipped = 0; for (let i = headStartIdx; !mergeSourceOnly && i < headEndIdx && i < allMessages.length; i++) { const m = allMessages[i]; @@ -11901,7 +12102,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { headCoveredSkipped++; continue; } - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } if (headCoveredSkipped > 0) { this.diag.warn( @@ -11985,7 +12186,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const m = allMessages[i]; if (priorSummaryMessageIds.has(m.id)) continue; if (sourceLeafIds.has(m.id)) continue; - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } } @@ -12041,7 +12242,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const messageId of src.sourceIds) { const m = messageById.get(messageId); if (m) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } } } else { @@ -12369,6 +12570,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { model: this.requireCompressionModel(), }, }, this.ns); + this.reportCompressionRefusal(assessment.outcome, errorType); throw new MergeDispositionRejection( targetLevel, assessment.outcome, requestHash, stopReason, errorType, ); @@ -12392,6 +12594,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { `[autobiographical] L${targetLevel} merge generation was all preamble — ` + `rejecting for retry (${sources.length} sources left unmerged)`, ); + this.reportCompressionRefusal('unusable_empty'); throw new MergeDispositionRejection( targetLevel, 'unusable_empty', requestHash, 'end_turn', ); @@ -12812,48 +13015,58 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the chunk's own messages, and, unless compressionSourceOnly, each message * the request carries raw ahead of the chunk, by the coverage tick's request * reads with the pins before the chunk (recallCoverage). From the lower of - * the head window's start and headEnd (headEnd clipped at a hold as tick - * clips it, and 0 with the head window off, headWindowTokens <= 0), that's - * every message the coverage leaves uncovered, up to the chunk or to - * headEnd, whichever is later. Before it, only what pins keep raw: the + * the start and end of the head that request gives (headAsOf, the head as + * of the chunk, its end clipped at a hold as tick clips it, and 0 with the + * head window off, headWindowTokens <= 0), that's every message the coverage + * leaves uncovered, up to the chunk or to that head's end, whichever is + * later. Before it, only what pins keep raw: the * messages of an L1 a pin opens, and a pinned message. Tick lays its * history out without the pins when the raw they add doesn't fit, so the * pins can only add raw here that tick leaves out; and tick strips unpaired * tool blocks first. So this can say yes where tick's request has none, * never the reverse. The test it returns takes one chunk. Chunks with the - * same pins before them read the same coverage, so each coverage is read - * once, for the first message its request carries raw with a tool block, - * and a chunk's answer is whether that message comes before the later of - * the chunk's start and headEnd. + * same pins before them, and the same head start, read the same coverage + * from the same place, so each is read once, for the first message its + * request carries raw with a tool block, and a chunk's answer is whether + * that message comes before the later of the chunk's start and its head's + * end. */ private requestCarriesToolBlocks(store: MessageStoreView): (chunk: Chunk) => boolean { const all = store.getAll(); const sourceOnly = this.config.compressionSourceOnly === true; - const headEnd = Math.min(this.getHeadWindowEnd(store), this.holdBoundary(store)); - const from = Math.min(this.getHeadWindowStartIndex(store), headEnd); const position = new Map(all.map((message, i) => [message.id, i])); const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all)].sort((a, b) => a - b); - // For each count of pins before a chunk, the index of the first message - // that request carries raw with a tool block, or all.length for none. - const firstToolBlock = new Map(); + // Built at the first head read, so a count that reads none (source-only, + // or each chunk holding a tool block itself) scans no holds. + let heads: ((pos: number) => { start: number; end: number }) | undefined; + // For each count of pins before a chunk and the start its head's raw + // reading takes, the index of the first message that request carries raw + // with a tool block, or all.length for none. + const firstToolBlock = new Map(); return (chunk) => { if (chunk.messages.some((m) => hasToolBlocks(m.content))) return true; if (sourceOnly) return false; - const start = position.get(chunk.messages[0]?.id ?? '') ?? -1; + // The chunk's start as tick's request takes it (startInStore). + const start = this.startInStore(chunk.messages.map((m) => m.id), position); + heads ??= this.headsAsOf(store); + const head = heads(start); + const headEnd = head.end; + const from = Math.min(head.start, headEnd); const pinned = pins.filter((i) => i < start); - let first = firstToolBlock.get(pinned.length); + const key = `${pinned.length}:${from}`; + let first = firstToolBlock.get(key); if (first === undefined) { const pinnedIds = new Set(pinned.map((i) => all[i].id)); const seen = this.recallCoverage(pinnedIds); // Before `from`, the request carries only what pins keep raw: an L1 a - // pin opens, and a pinned message (which lies before headEnd, since - // from is at most headEnd). + // pin opens, and a pinned message. (Either lies before headEnd there, + // since from is at most headEnd.) const carried = (message: StoredMessage, i: number): boolean => !seen.coveredMessageIds.has(message.id) && (i >= from || seen.openedMessageIds.has(message.id) || pinnedIds.has(message.id)); const found = all.findIndex((message, i) => carried(message, i) && hasToolBlocks(message.content)); first = found === -1 ? all.length : found; - firstToolBlock.set(pinned.length, first); + firstToolBlock.set(key, first); } return first < Math.max(headEnd, start); }; @@ -12906,6 +13119,59 @@ export class AutobiographicalStrategy implements ResettableStrategy { }); } + /** + * The pins that keep messages raw in the adaptive plan's middle, for its + * refusal to name (`OverBudgetDiagnostics.rawPins`): the positions the plan + * folds history in (before the tail's start, which is the recent window's + * start or the head's end where the recent window reaches back into the + * head, and outside the head window; with the head window disabled after a + * reset, the plan's tail holds the recent messages before the head) that + * pins keep raw + * (`forceRawPinPositions`), less the messages of system changes, which + * stay raw whatever the pins, each priced by `price`, the plan's own raw + * price. Every such position counts, a part locked with `lockChunk` among + * them, so the tokens are what unpinning frees at most: a pin that keeps no + * part raw alone is still named. Where bounds are read, a pin whose own + * bound isn't raw keeps nothing raw, so it isn't named even where it + * overlaps a raw pin; where they aren't, every pin renders raw. Undefined + * when no pin keeps any. + */ + private rawPinsDiagnostics(store: MessageStoreView, price: (position: number) => number): OverBudgetDiagnostics['rawPins'] { + if (this.pins.length === 0) return undefined; + const messages = store.getAll(); + const raw = this.forceRawPinPositions(messages); + // A message of system changes stays raw whatever the pins (resolvedPinBounds), so unpinning frees none of it. + const changes = new Set(systemChangePositions(messages)); + const headStart = this.getHeadWindowStartIndex(store); + const headEnd = this.getHeadWindowEnd(store); + const tailStart = Math.max(this.getRecentWindowStart(store), headEnd); + const kept = (i: number): boolean => raw.has(i) && !changes.has(i) && i < tailStart && !(i >= headStart && i < headEnd); + + let tokens = 0; + for (const i of raw) if (kept(i)) tokens += price(i); + if (tokens === 0) return undefined; + const positionOf = this.positionIndex(messages); + const readsBounds = this.readsPinBounds(); + const pins: OverBudgetRawPin[] = []; + for (const pin of this.pins) { + const bound: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } + : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; + if (readsBounds && !AutobiographicalStrategy.isRawPinBound(bound)) continue; + const at = this.pinPositions(pin, positionOf, messages); + if (!at) continue; + let own = 0; + for (const i of at) if (kept(i)) own += price(i); + if (own === 0) continue; + const named = Math.abs(positionOf.get(pin.lastMessageId)! - positionOf.get(pin.firstMessageId)!) + 1; + pins.push({ + pinId: pin.id, kind: pin.kind, ...(pin.name === undefined ? {} : { name: pin.name }), + firstMessageId: pin.firstMessageId, lastMessageId: pin.lastMessageId, + ...(at.length > named ? { parts: at.length } : {}), tokens: own, + }); + } + return { tokens, pins }; + } + // ============================================================================ // Adaptive resolution (picker-driven) path // ============================================================================ @@ -12946,6 +13212,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { // apart from the exceptions applyImageStripping's asEstimated names. const pse = this.postStripEstimates(store); if (_diag) { this.diag.error(`[cm-cache] selectAdaptive: pse ${Date.now() - _t}ms`); _t = Date.now(); } + // A middle message's raw price, as the plan prices it and its refusal + // prices the pins that keep it raw (rawPinsDiagnostics). The shard path + // in step 5 sends a run of a body's raw shards as one entry, their text + // joined and uncut, so a shard takes no extra. + const middlePrice = (i: number): number => { + const msg = messages[i]!; + return msg.bodyGroupId ? pse[i]! : pse[i]! + this.cappedForSend(store, msg, msgCap).extra; + }; // ----- 1. Build head/tail sets and reserve the tail before emitting ----- const headStart = this.getHeadWindowStartIndex(store); @@ -12960,13 +13234,24 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Compute the fixed raw windows first. They are hard reservations, not // best-effort phases: foldable history may use only the space left after // every head and tail message has been accounted for. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + // The head's positions: after a reset, the system changes stored before + // the head window, then the window (headPositions). + const head = this.headPositions(messages, headStart, headEnd); + for (const i of head) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; headMessageIds.add(msg.id); headTokens += tokens; } const effectiveRecentStart = Math.max(recentStart, headEnd); + // System changes stay raw wherever they are (resolvedPinBounds), and + // nothing a refusal can name shrinks them: the ones outside the head and + // the tail are sent raw in the middle, beside what compression folds. + let middleChangeTokens = 0; + for (const i of systemChangePositions(messages)) { + if (i >= effectiveRecentStart || headMessageIds.has(messages[i].id)) continue; + middleChangeTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; + } for (let i = effectiveRecentStart; i < messages.length; i++) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; @@ -12997,17 +13282,29 @@ export class AutobiographicalStrategy implements ResettableStrategy { // does a newest turn that starts inside the head. let floorTokens = 0; if (messages.length > 0) { - const newestBody = bodyStart((k) => messages[k], messages.length - 1); + // A system change is no conversation message, so the newest one is + // the newest that isn't one, and the changes after it stay raw with + // it (counted from floorStart below). + let newest = messages.length - 1; + while (newest > 0 && holdsSystemChange(messages[newest].content)) newest--; + const newestBody = bodyStart((k) => messages[k], newest); const newestTurn = newestBody > 0 && this.hasToolResult(messages[newestBody]) ? bodyStart((k) => messages[k], newestBody - 1) - : messages.length - 1; + : newest; const floorStart = Math.min(newestTurn, this.holdBoundary(store)); for (let i = floorStart; i < messages.length; i++) { floorTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; } + // A system change outside the head stays raw wherever the recent + // window starts, so the floor holds every one before it too, and + // when the changes alone exceed the budget nothing is named. + for (const i of systemChangePositions(messages)) { + if (i >= floorStart) break; + if (!headMessageIds.has(messages[i].id)) floorTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; + } } throw this.overBudgetError(budget, { - stage: 'Head/tail reservation exceeded budget', + ...overBudgetStage('head-tail-reservation'), budget: rejectionBudget, actual: headTokens + tailTokens, clearedBy: headTokens + floorTokens <= rejectionBudget ? ['smaller-recent-window'] : [], @@ -13029,7 +13326,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Emit the already-reserved head entries raw, each cut at the cap as any // raw message is. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + for (const i of head) { const msg = messages[i]; const { content, extra } = this.cappedForSend(store, msg, msgCap); const tokens = pse[i] + extra; @@ -13098,25 +13395,24 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Excluding that zone made pre-transition history invisible to the picker // (never planned, never rendered): the adaptive variant of the frontier // gap the coverage invariant exists to refuse. - const middleSegments: ReadonlyArray = headStart > 0 - ? [[0, headStart], [headEnd, effectiveRecentStart]] - : [[headEnd, effectiveRecentStart]]; + // A system change before a reset head is in the head (headPositions), so + // the middle walks pass over it. + const middleSegments = AutobiographicalStrategy.middleRanges(headStart, headEnd, effectiveRecentStart); const pickerChunks: PickerChunk[] = []; const salience = this.salienceByPosition(messages); for (const [segStart, segEnd] of middleSegments) for (let i = segStart; i < segEnd && i < messages.length; i++) { const msg = messages[i]; + if (headMessageIds.has(msg.id)) continue; const ch = chunkAtPosition ? chunkAtPosition[i] : chunksByMessageId.get(msg.id); - // This is the chunk's raw price. The shard path in step 5 sends a run - // of a body's raw shards as one entry, their text joined and uncut, so - // a shard takes no extra here. It prices that entry as the shards' - // text it joins, each by estimateTokens, the store's price, so a shard - // of text alone is counted there as here; a shard's other blocks are - // priced here, and the shard path leaves them out. A shard the plan - // folds is priced as its summary. The head and recent windows cut a - // shard as they emit it. - const tokens = msg.bodyGroupId ? pse[i] : pse[i] + this.cappedForSend(store, msg, msgCap).extra; + // This is the chunk's raw price (middlePrice). The shard path in step 5 + // prices its joined entry as the shards' text it joins, each by + // estimateTokens, the store's price, so a shard of text alone is + // counted there as here; a shard's other blocks are priced here, and + // the shard path leaves them out. A shard the plan folds is priced as + // its summary. The head and recent windows cut a shard as they emit it. + const tokens = middlePrice(i); const bound = pinBounds.size > 0 ? pinBounds.get(i) : undefined; const l1Id = ch?.summaryId ?? l1ByMessage.get(msg.id); pickerChunks.push({ @@ -13135,7 +13431,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Also include head and tail in PickerChunks (so token accounting matches) // — but mark them as in-head/in-tail so the picker won't fold them. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + for (const i of head) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; pickerChunks.push({ @@ -13449,8 +13745,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Taken at a refusal of this select (the plan check's, or a later // stage's), or for a dry run's preview of a plan over the budget, since a // plan that fits and renders needs no list. + // Compression shrinks neither the head and the tail nor a system change + // between them: when those alone exceed the budget, nothing the list + // names clears it. const planClearedBy = (): OverBudgetClearedBy[] => - result.produced.length > 0 && this.compressionWorkQueued(store, 'adaptive') ? ['compression'] : []; + headTokens + tailTokens + middleChangeTokens <= rejectionBudget && result.produced.length > 0 && this.compressionWorkQueued(store, 'adaptive') ? ['compression'] : []; const planFits = result.finalTokens <= rejectionBudget; // Record what a dry run needs (select() puts back the transition @@ -13489,6 +13788,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // real compile will do. If the retry at 1 is still over, it reports. const dryFallback = dryRun && this._calibrationProvisional !== null && this._calibration > 1; if (!planFits && (!dryRun || dryFallback)) { + // The plan's middle is what's over, priced as the plan priced it, so + // the pins keeping raw tokens there are named. The refusals after the + // plan count other things (an emitted prefix, a shard's text alone) + // and don't name pins. + const rawPins = this.rawPinsDiagnostics(store, middlePrice); throw this.overBudgetError(budget, { budget: rejectionBudget, actual: result.finalTokens, @@ -13501,6 +13805,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { middleTokens: Math.max(0, result.finalTokens - headTokens - tailTokens), middleChunkCount: pickerChunks.length - headMessageIds.size - tailMessageIds.size, deepestLevel, + ...(rawPins === undefined ? {} : { rawPins }), }, }); } @@ -13523,7 +13828,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const middleChunkCountDiag = middleChunkCount; const emissionOverBudget = (attempted: number): OverBudgetError => this.overBudgetError(budget, { - stage: 'Emission overran the plan (planner/emitter estimator drift)', + ...overBudgetStage('emission-overran-plan'), budget: rejectionBudget, actual: attempted + tailTokens, clearedBy: planClearedBy(), @@ -13542,6 +13847,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { let i = segStart; while (i < segEnd && i < messages.length) { const msg = messages[i]; + if (headMessageIds.has(msg.id)) { + i++; // sent in the head (headPositions) + continue; + } if (msg.bodyGroupId) { // Collect this body's consecutive shards. Another ingestion of the @@ -13728,7 +14037,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const tailStats = this.emitRecentNewestFirst(entries, store, messages, effectiveRecentStart, msgCap, rejectionBudget, totalTokens, pse); if (tailStats.messages !== messages.length - effectiveRecentStart) { throw this.overBudgetError(budget, { - stage: 'Tail emission dropped reserved recent-window messages', + ...overBudgetStage('tail-emission-dropped'), budget: rejectionBudget, actual: totalTokens + tailTokens, clearedBy: planClearedBy(), @@ -13763,7 +14072,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // post-strip context the agent actually receives. This selection budgets // by postStripEstimates, so the strip counts as they do (see // applyImageStripping's asEstimated, which names where it doesn't). - this.applyImageStripping(merged, store, { asEstimated: true }); + this.applyImageStripping(merged, store, { asEstimated: true, returned: true }); // The newest stored message is the triggering turn for this compile. A // structural repair may rewrite tool blocks, but it must never erase that @@ -13775,9 +14084,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { // newest shard alone outgrows the recent window. Only that body's shards // count: an earlier ingestion of the same text shares the group id, and its // survival proves nothing about the newest turn. - const newest = messages[messages.length - 1]; + // A system change is no turn and always stays raw, so the newest turn is + // the newest message that isn't one, as for the floor above. + let newestAt = messages.length - 1; + while (newestAt > 0 && holdsSystemChange(messages[newestAt].content)) newestAt--; + const newest = messages[newestAt]; if (newest) { - const newestBody = bodyBoundsIn(messages, messages.length - 1); + const newestBody = bodyBoundsIn(messages, newestAt); const newestBodyIds = new Set(messages.slice(newestBody.from, newestBody.to + 1).map(m => m.id)); const newestRetained = merged.some( entry => (entry.sourceMessageId !== undefined && newestBodyIds.has(entry.sourceMessageId)) || @@ -13785,13 +14098,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { ); if (!newestRetained) { throw this.overBudgetError(budget, { - stage: 'Structural repair did not retain the newest turn', + ...overBudgetStage('repair-lost-newest-turn'), budget: rejectionBudget, actual: totalTokens + tailTokens, - // The newest turn is what's missing, and only room for it brings it - // back. A newer message makes a different turn the newest, and this - // one then sits in the middle raw until its chunk closes and - // compression summarizes it, so the next compile is no smaller. + // The newest turn is what's missing: the repair trimmed it as an + // unanswered call, which no budget brings back. A newer message + // makes a different turn the newest, and this one then sits in the + // middle raw until its chunk closes and compression summarizes it, + // so the next compile is no smaller. clearedBy: [], diagnostics: { middleCountUnit: 'picker-chunks', @@ -13862,23 +14176,62 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._lastCompileEstimate = rs.head.tokens + rs.tail.tokens + rs.middleRaw.tokens + rs.summaries.l1.tokens + rs.summaries.l2.tokens + rs.summaries.l3.tokens; + // What the compile inserts among these entries after the select (its + // message-position injections) is in the request and not in the + // window: priced here, at the multiplier the window was priced at, so + // the sample counts it as the request does (#323). + let injected = 0; + for (const content of opts?.messageInjections ?? []) { + injected += store.estimateTokens({ content } as unknown as StoredMessage); + } + this._lastCompileInjected = injected; this._calibrationArmed = true; // exactly one sample per compile } return merged; } private _lastCompileEstimate = 0; + /** The armed compile's message-position injections, priced with its window. */ + private _lastCompileInjected = 0; private _storeView: MessageStoreView | null = null; + /** Thinking the context manager left out after this strategy's committing + * select priced it isn't sent, so the calibration sample's estimate leaves + * out what this strategy priced it at: each block at the store's estimate. + * The select priced each raw message's thinking blocks at just that: the + * image strip and the cap that also enter a message's price never change + * a thinking block's. */ + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { + if (blocks.length === 0) return; + const tokens = this.estimateTokens([...blocks]); + if (Number.isFinite(tokens) && tokens > 0) this._lastCompileEstimate = Math.max(0, this._lastCompileEstimate - tokens); + } + + /** Whether the select in progress is for a model that binds thinking + * (`SelectOptions.thinkingBinding`): set from the select's own option for + * the run of its select path, and false outside it. A summary's carriers + * were minted in its summarizer's request, so such a model never takes + * them in the live window, and the live window renders and prices them as + * under `carrierPolicy: 'live-strip'`. */ + private thinkingBound = false; + + /** The carrier policy the live window renders and prices under: the + * configured one, or `'live-strip'` while thinking is bound. */ + private liveCarrierPolicy(): CarrierPolicy { + return this.thinkingBound || this.config.carrierPolicy === 'live-strip' ? 'live-strip' : 'full'; + } + /** * Closed-loop estimator calibration (2026-07-12). Feed the REAL input total * for a request built from the latest compile (fresh + cache_read + * cache_creation, as the provider reports it), and what the request sent - * besides the window: its system prompt and tool definitions (#323). Those - * are priced as the store prices the window and added to the estimate, so - * the sample compares the whole request with the whole request; left out, - * the multiplier learns them as a per-token rate, which no value of it can - * be (see RequestOverhead). Maintains an EMA of real/estimated and applies + * besides the window and the compile's message-position injections: its + * system prompt and tool definitions (#323). The injections are priced at + * the compile (SelectOptions.messageInjections) and the overhead here, as + * the store prices the window, and both are added to the estimate, so the + * sample compares the whole request with the whole request; left out, the + * multiplier learns them as a per-token rate, which no value of it can be + * (see RequestOverhead). Maintains an EMA of real/estimated and applies * it as the store's global multiplier, persisted across restarts. The * per-class rates carry the shape; this carries the residual level. */ @@ -13898,21 +14251,30 @@ export class AutobiographicalStrategy implements ResettableStrategy { // exhausting the picker on every wake. One sample per compile, always. if (!this._calibrationArmed) return; this._calibrationArmed = false; + // Declared: the caller passed what the request sent besides the window + // and the compile's injections ({} included). A JS caller's null isn't. + const declared = overhead != null; // The overhead in the window's units (current calibrated), so the implied // multiplier below is real / raw(window + overhead) and doesn't depend on // the multiplier it's measured at: one sample states it. - const overheadTokens = overhead ? this.priceRequestOverhead(overhead) : 0; - const est = window + overheadTokens; + const overheadTokens = declared ? this.priceRequestOverhead(overhead!) : 0; + const injected = this._lastCompileInjected; + const est = window + injected + overheadTokens; + const parts = [ + ...(overheadTokens > 0 ? [`${Math.round(overheadTokens / 1000)}k system+tools`] : []), + ...(injected > 0 ? [`${Math.round(injected / 1000)}k injected`] : []), + ]; const counts = `(est=${Math.round(est / 1000)}k` + - `${overheadTokens > 0 ? ` incl. ${Math.round(overheadTokens / 1000)}k system+tools` : ''}` + + `${parts.length > 0 ? ` incl. ${parts.join(', ')}` : ''}` + ` real=${Math.round(realTotal / 1000)}k)`; const ratio = realTotal / est; const current = this._calibration; const observed = ratio * current; // back out the multiplier already applied // SANITY BAND: a representative sample sits near 1. Anything wilder is a // structural mismatch (a request we didn't compile, a partial compile, a - // provider quirk) — never evidence about chars-per-token. Log, don't learn. + // provider quirk) — never evidence about chars-per-token. Log, don't learn, + // except a declared consistent run (below). // The band is also checked on the IMPLIED raw multiplier: when the // multiplier sits at a clamp edge because the per-class rates were wrong // (signed thinking at a flat 600) and the rates are then fixed, every @@ -13926,8 +14288,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the same number, so a store whose true ratio is past the band (every // epoch discard restarts there) rejects every sample and never learns. // A single wild sample is still noise, but a run of them that agree with - // each other is a ruler that's off, not a request we didn't compile: the - // third in a row is learned, outright, which lands it on the clamp edge. + // the run's first, within OUT_OF_BAND_RUN_AGREEMENT, is a ruler that's + // off, not a request we didn't compile: the third in a row is learned, + // outright, which lands it on the clamp edge. Agreement is with the first, + // not the one before, so a drifting run (a growing unpriced part) isn't + // learned as a constant. // Only for samples whose caller declared what it sent besides the window: // undeclared, a consistent ratio past the band is what tool schemas and a // system prompt look like, and learning it would fold the window to the @@ -13935,20 +14300,22 @@ export class AutobiographicalStrategy implements ResettableStrategy { let fromRun = false; if (!ratioInBand && !observedInBand) { const run = this._outOfBandRun; - this._outOfBandRun = !overhead ? null - : run && Math.abs(observed / run.observed - 1) <= OUT_OF_BAND_RUN_AGREEMENT - ? { observed, count: run.count + 1 } - : { observed, count: 1 }; + this._outOfBandRun = !declared ? null + : run && Math.abs(observed / run.first - 1) <= OUT_OF_BAND_RUN_AGREEMENT + ? { first: run.first, count: run.count + 1 } + : { first: observed, count: 1 }; const count = this._outOfBandRun?.count ?? 0; if (count < OUT_OF_BAND_RUN_LENGTH) { this.diag.error( `[estimator-calibration] REJECTED out-of-band sample real/est=${ratio.toFixed(2)} ${counts} — ` + - `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}` + - (overhead - ? ` (${count} of ${OUT_OF_BAND_RUN_LENGTH} consistent samples to learn from)` - : `; if every sample reads about the same, the caller may be leaving out the system prompt ` + - `and tools it sends (reportRealInputTokens' second argument)`), + (declared + ? `multiplier stays ${this._calibration.toFixed(2)} until ${OUT_OF_BAND_RUN_LENGTH} samples in a row ` + + `agree within ${OUT_OF_BAND_RUN_AGREEMENT * 100}% (${count} of ${OUT_OF_BAND_RUN_LENGTH})` + : `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}; if every sample ` + + `reads about the same, the caller may be leaving out the system prompt and tools it sends ` + + `(reportRealInputTokens' second argument)`), ); + this.hostContext?.observer?.calibration?.({ outcome: 'out-of-band', ratio, multiplier: this._calibration }); return; } this._outOfBandRun = null; @@ -13957,16 +14324,58 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._outOfBandRun = null; } + // A multiplier learned before samples declared their overhead learned it + // as a rate (#323). Inside the band, the first declared sample on such a + // record is held, and the next declared sample decides: if the two agree + // within OUT_OF_BAND_RUN_AGREEMENT their mean replaces the multiplier, + // instead of about ten EMA steps, each a reprice of every message and + // summary; if not, the EMA steps as for any sample. Two, not one: this + // reaches every store that samples after the upgrade, those whose + // multiplier is already right included, and one wild sample shouldn't + // set their ruler alone. The held sample is in memory only, so a restart + // between them costs one more sample; an undeclared sample between them + // leaves it, since an implied multiplier doesn't depend on the multiplier + // in force. Either way the record is then marked (declaredOverhead): the + // move has run for it, and later declared samples step by the EMA; an + // undeclared sample that moves the multiplier takes the mark back. A + // provisional multiplier, or a consistent run (out of the band, three + // samples), is replaced by one sample as before. + const markDeclared = declared && !this._calibrationDeclared; + let replacement: number | null = null; + if (markDeclared && !this._calibrationProvisional && !fromRun) { + const candidate = this._declaredCandidate; + if (candidate === null) { + this._declaredCandidate = observed; + this.diag.error( + `[estimator-calibration] real/est=${ratio.toFixed(2)} implies ${observed.toFixed(2)}: the first sample ` + + `with its overhead declared, held for the next; if they agree within ` + + `${OUT_OF_BAND_RUN_AGREEMENT * 100}% their mean replaces the multiplier, and if not it steps; ` + + `multiplier stays ${current.toFixed(2)} ${counts}`, + ); + this.hostContext?.observer?.calibration?.({ outcome: 'deferred', ratio, multiplier: current }); + return; + } + this._declaredCandidate = null; + if (Math.abs(observed / candidate - 1) <= OUT_OF_BAND_RUN_AGREEMENT) replacement = (candidate + observed) / 2; + } else if (declared) { + // A declared sample that decides the multiplier another way (a + // provisional replacement, a run) or meets a marked record ends any + // held sample: what it was held for, whether two declared samples + // should replace an unmarked multiplier, has been settled. + this._declaredCandidate = null; + } + const alpha = 0.2; // slow EMA: one wild request shouldn't yank the ruler const ema = this._calibrationEma ?? current; // A provisional multiplier is a guess (see CALIBRATION_PRICING_EPOCH). This // sample is the first evidence under this release's pricing, so it replaces // the guess outright; the EMA steps from here on. A consistent run replaces - // it outright too: an EMA step from 1.00 toward a ratio past the band - // would leave the next sample just as far out of band. + // it outright too: its samples agree on the multiplier, and stepping toward + // it would take several more applied changes, each a reprice of every + // message and summary. const provisional = this._calibrationProvisional; this._calibrationProvisional = null; - const next = provisional || fromRun ? observed : ema + alpha * (observed - ema); + const next = replacement ?? (provisional || fromRun ? observed : ema + alpha * (observed - ema)); const clamped = Math.min(1.8, Math.max(0.6, next)); this._calibrationEma = clamped; // DEAD BAND. The multiplier is applied only once the EMA has moved more @@ -13985,19 +14394,28 @@ export class AutobiographicalStrategy implements ResettableStrategy { `[estimator-calibration] real/est=${ratio.toFixed(2)} confirms the provisional multiplier ` + `${current.toFixed(2)} ${counts}; stamped`, ); + if (declared) this._calibrationDeclared = true; + this.persistCalibration(); + } else if (markDeclared) { + this._calibrationDeclared = true; this.persistCalibration(); } + this.hostContext?.observer?.calibration?.({ outcome: provisional ? 'confirmed' : 'held', ratio, multiplier: current }); return; } this.diag.error( `[estimator-calibration] real/est=${ratio.toFixed(2)} ` + `multiplier ${current.toFixed(2)} -> ${clamped.toFixed(2)}` + - `${fromRun ? ` (${OUT_OF_BAND_RUN_LENGTH} consistent out-of-band samples: learned at the clamp)` : provisional ? ' (provisional: replaced, not stepped)' : ''} ` + + `${fromRun ? ` (${OUT_OF_BAND_RUN_LENGTH} consistent out-of-band samples: learned at the clamp)` : provisional ? ' (provisional: replaced, not stepped)' : replacement !== null ? ' (two samples with their overhead declared agree: replaced by their mean, not stepped)' : ''} ` + counts, ); this._calibration = clamped; + this._calibrationDeclared = declared; this.applyCalibration(); this.persistCalibration(); + // Each outright replacement (a provisional multiplier, a consistent run, + // two agreeing declared samples) is 'replaced'; an EMA step is 'stepped'. + this.hostContext?.observer?.calibration?.({ outcome: provisional || fromRun || replacement !== null ? 'replaced' : 'stepped', ratio, multiplier: clamped }); } /** The store's price of what a request sent besides the window: its system @@ -14024,6 +14442,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { at: Date.now(), pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, ...(this._calibrationReset ? { reset: this._calibrationReset } : {}), + ...(this._calibrationDeclared ? { declaredOverhead: true } : {}), }); } catch { /* persistence is best-effort */ } } @@ -14167,17 +14586,29 @@ export class AutobiographicalStrategy implements ResettableStrategy { private _calibration = 1; /** The EMA itself; `_calibration` is its last applied value (see the dead band). */ private _calibrationEma: number | null = null; - /** The out-of-band samples rejected in a row that agree with each other, - * and the last one's implied multiplier (see reportRealInputTokens). In - * memory only: a restart counts afresh. */ - private _outOfBandRun: { observed: number; count: number } | null = null; + /** The out-of-band samples rejected in a row that agree with the first, + * and the first one's implied multiplier (see reportRealInputTokens). In + * memory only: a restart, or a branch's re-initialisation + * (clearBranchMirrors), counts afresh. */ + private _outOfBandRun: { first: number; count: number } | null = null; + /** The implied multiplier of the first declared sample on a record whose + * multiplier wasn't learned from declared samples, held for the next + * declared sample (see reportRealInputTokens). In memory only. */ + private _declaredCandidate: number | null = null; + /** Whether the two-sample move has run for the multiplier in force: set + * once declared samples have set, stepped or confirmed it (one sample for a + * provisional multiplier or a run, two for a learned one). Persisted as the + * record's declaredOverhead; an undeclared sample that moves the multiplier + * clears it, and so does dropping a provisional multiplier to 1 (see + * reportRealInputTokens). */ + private _calibrationDeclared = false; private _calibrationLoaded = false; /** Set when an older-epoch multiplier was discarded; persisted with the record. */ private _calibrationReset: CalibrationReset | null = null; /** Set while the multiplier is provisional (see CALIBRATION_PRICING_EPOCH): * the first sample the band accepts replaces it outright, and one above 1 * that puts a compile over budget is dropped to 1. `fromEpoch` is the - * record's it came from, null for one without a stamp. */ + * record's it came from, null for one without a stamp or for no record. */ private _calibrationProvisional: { fromEpoch: number | null } | null = null; protected applyCalibration(): void { @@ -14236,12 +14667,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { * is persisted as such (`provisional: true`), so a restart before that * sample doesn't turn the guess into a learned value, and a later epoch's * load re-stamps it, still provisional, rather than discarding it again - * over the reset it carries. A dry run under a + * over the reset it carries. A store with no record starts from a + * provisional 1 too (#328). It writes nothing until its first sample: with + * no record, each load decides the same. A dry run under a * provisional multiplier above 1 takes the same fallback at the picker's * refusal, which it otherwise only reports. * * A binary from before a bump still learns after a rollback, and its first - * applied sample overwrites the newer epoch's record with its own stamp + * applied sample, or two declared samples that confirm and mark its 1, + * overwrite the newer epoch's record with its own stamp * (reportRealInputTokens); rolling forward again treats that as an older * record. */ @@ -14372,9 +14806,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibration = 1; this._calibrationReset = null; this._calibrationProvisional = null; + this._calibrationDeclared = false; try { const saved = this.store?.getStateJson(this.calibrationStateId) as - { multiplier?: number; pricing?: number; reset?: CalibrationReset; provisional?: boolean } | null; + { multiplier?: number; pricing?: number; reset?: CalibrationReset; provisional?: boolean; declaredOverhead?: boolean } | null; if (saved && Number.isFinite(saved.multiplier)) { const current = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; const stamped = typeof saved.pricing === 'number' && Number.isFinite(saved.pricing); @@ -14389,6 +14824,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (epoch === current) { this._calibration = Math.min(1.8, Math.max(0.6, saved.multiplier!)); this._calibrationReset = isCalibrationReset(saved.reset) ? saved.reset : null; + this._calibrationDeclared = saved.declaredOverhead === true; // A discard's 1 that no sample has confirmed yet was persisted as // provisional, and stays a guess across the restart: the first // sample still sets it, instead of stepping toward it. @@ -14409,6 +14845,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // still a guess. this._calibration = Math.min(1.8, Math.max(0.6, saved.multiplier!)); this._calibrationReset = isCalibrationReset(saved.reset) ? saved.reset : null; + this._calibrationDeclared = saved.declaredOverhead === true; if (saved.provisional === true) { this._calibrationProvisional = { fromEpoch: this._calibrationReset?.fromEpoch ?? null }; } @@ -14429,6 +14866,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { pricing: current, ...(this._calibrationReset ? { reset: this._calibrationReset } : {}), ...(this._calibrationProvisional ? { provisional: true } : {}), + ...(this._calibrationDeclared ? { declaredOverhead: true } : {}), }); } catch { /* persistence is best-effort */ } } @@ -14485,9 +14923,26 @@ export class AutobiographicalStrategy implements ResettableStrategy { // leaves it on disk and doesn't use it, and logs nothing, since // re-stamping it as the discard warning advises would destroy it. // This binary's learner still writes its own record over it on its - // first applied sample (see CALIBRATION_PRICING_EPOCH). + // first applied sample, or when two declared samples confirm and + // mark the 1 (see CALIBRATION_PRICING_EPOCH). + } else if (!(typeof saved?.pricing === 'number' && saved.pricing > AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH)) { + // No record, or none with a multiplier to use: nothing this store + // kept says what its multiplier is, so its 1 is a guess, as a + // discard's restart is, and the first sample the band accepts sets + // it (#328). Learned, it would take the slowest path for the store + // with the least evidence: for an undeclared caller nine applied + // EMA steps to reach a true 1.4, each a reprice of every message and + // summary, and for a declared one two samples, the first held. + // Nothing is + // written: with no record, the next load decides the same, and the + // first sample's write gives the stamp. A record from a newer epoch, + // with a multiplier this binary can read or not, and a read that + // throws (below) aren't this case: each may hold what a later binary + // or the store keeps, which a provisional 1's first sample would + // write over even when it only confirms the 1. + this._calibrationProvisional = { fromEpoch: null }; } - } catch { /* absent slot is fine */ } + } catch { /* a record the store can't read loads 1, not provisional: unlike no record, it may hold what the store keeps (see the no-record branch above) */ } } // A dry select changes the store's multiplier only where select() read it // first and can put it back (see MessageStoreView). @@ -14515,6 +14970,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const reset: CalibrationReset = { discardedMultiplier: discarded, fromEpoch: provisional.fromEpoch, at: Date.now() }; this._calibration = 1; this._calibrationReset = reset; + // The mark was about the multiplier dropped, and goes with it, as the + // record below is written without it. + this._calibrationDeclared = false; this.applyCalibration(); if (dryRun) return true; this.diag.warn( @@ -15137,14 +15595,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { } pending = []; }; - for (let i = window.headStart; i < window.headEnd && i < messages.length; i++) pushRaw(messages[i]!); - const segments: ReadonlyArray = window.headStart > 0 - ? [[0, window.headStart], [window.headEnd, window.recentStart]] - : [[window.headEnd, window.recentStart]]; - for (const [segStart, segEnd] of segments) { + const head = this.headPositions(messages, window.headStart, window.headEnd); + for (const i of head) pushRaw(messages[i]!); + const inHead = new Set(head); + for (const [segStart, segEnd] of AutobiographicalStrategy.middleRanges(window.headStart, window.headEnd, window.recentStart)) { let i = segStart; while (i < segEnd && i < messages.length) { const msg = messages[i]!; + if (inHead.has(i)) { + i++; + continue; + } if (msg.bodyGroupId) { const groupStart = i; i = bodyEnd(Math.min(segEnd, messages.length), (k) => messages[k]!, groupStart) + 1; @@ -15582,7 +16043,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Memoized price of a summary's recall answer as the LIVE WINDOW renders * it (carriers included under `carrierPolicy: 'full'`, omitted under - * `'live-strip'`, because this price feeds the fold planner for that same + * `'live-strip'` or while thinking is bound, as liveCarrierPolicy says, + * because this price feeds the fold planner for that same * emission): `estimate` is the estimate of that content, and `cost` is * what it's priced at, the estimate floored at the provider's measurement * where that measurement counts what's sent. Keyed by the summary identity @@ -15594,7 +16056,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected recallAnswerCost(s: SummaryEntry): { estimate: number; cost: number } { const recallEnvelope = this.config.recallEnvelope === 'xml' ? 'xml' : 'none'; - const carrierPolicy = this.config.carrierPolicy === 'live-strip' ? 'live-strip' : 'full'; + const carrierPolicy = this.liveCarrierPolicy(); const calibration = this._storeView?.getTokenCalibration?.() ?? 1; const key = JSON.stringify([s.id, s.tokens, recallEnvelope, carrierPolicy, calibration]); const cached = this._answerCostCache.get(key); @@ -15840,7 +16302,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { const maxTokens = budget.maxTokens - budget.reserveForResponse; const messages = store.getAll(); const msgCap = this.config.maxMessageTokens; - // Emission grace (coverage invariant, 76e95a0): selection below still // TARGETS maxTokens, but emission never silently drops content to fit — // it may overshoot up to the grace window, and beyond that the select @@ -15856,7 +16317,38 @@ export class AutobiographicalStrategy implements ResettableStrategy { // any raw message is (from headStart, not necessarily 0) const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); - for (let i = headStart; i < headEnd && i < messages.length; i++) { + // System changes stay raw wherever they are (resolvedPinBounds), and no + // newer message or compression shrinks them, so when they alone exceed + // the grace limit only the caller, consolidating them, clears it. That + // refusal comes first, as every strategy gives it; an eviction from the + // recent window while the changes fit stays a loss the coverage + // invariant refuses, as any there is. + let changeTokens = 0; + for (const i of systemChangePositions(messages)) { + const msg = messages[i]; + changeTokens += store.estimateTokens(this.prunedForSend(store, msg)) + this.cappedForSend(store, msg, msgCap).extra; + } + if (changeTokens > graceLimit) { + throw this.overBudgetError(budget, { + ...overBudgetStage('hierarchical-system-changes'), + budget: graceLimit, + actual: changeTokens, + clearedBy: [], + diagnostics: { + middleCountUnit: 'selection-items', + deepestLevelKind: 'attempted', + headTokens: changeTokens, + tailTokens: 0, + middleTokens: 0, + middleChunkCount: 0, + deepestLevel: 0, + }, + }); + } + // After a reset, the system changes stored before the head window go + // first, in the head (headPositions). + const head = this.headPositions(messages, headStart, headEnd); + for (const i of head) { const msg = messages[i]; const { content, extra } = this.cappedForSend(store, msg, msgCap); // The pruned form's price with its images whole: this selection's @@ -15868,17 +16360,20 @@ export class AutobiographicalStrategy implements ResettableStrategy { // messages no summary covers. Refuse honestly beyond grace. if (totalTokens + tokens > graceLimit) { throw this.overBudgetError(budget, { - stage: 'Hierarchical emission overran the budget', + ...overBudgetStage('hierarchical-emission-overran'), budget: graceLimit, actual: totalTokens + tokens, // The head is sent whole by definition, and its size is cold // configuration (headWindowTokens isn't a hot setting). Besides a - // larger budget, a head-window reset clears it, which starts a new - // topic and isn't a lever the list names (OverBudgetClearedBy). + // larger budget, a head-window reset clears the window's share of + // it, though not the system changes stored before the new head, + // which go in it; a reset starts a new topic and isn't a lever the + // list names (OverBudgetClearedBy). // Newer messages can shrink a head that holds images, since the // image strip takes the oldest images first (maxLiveImages, - // maxLiveImageBytes, imageStripDepthTokens), but whether they'd - // strip enough isn't known, so nothing is named for that either. + // maxLiveImageBytes, each within its image ceiling, and + // imageStripDepthTokens), but whether they'd strip enough isn't + // known, so nothing is named for that either. clearedBy: [], diagnostics: { middleCountUnit: 'selection-items', @@ -15930,7 +16425,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const prefix = new Set(refused.raw.map((m) => m.id)); for (const s of refused.summaries) this.expandSummaryToLeafMessageIds(s, summariesById, prefix); return this.overBudgetError(budget, { - stage: 'Hierarchical emission overran the budget', + ...overBudgetStage('hierarchical-emission-overran'), budget: graceLimit, actual: refused.actual, clearedBy: this.compressionWorkQueued(store, 'hierarchical', prefix) ? ['compression'] : [], @@ -16089,7 +16584,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The middle is everything before the recent window that is not the LIVE // head window — including [0, headStart) after a head-window reset, which // is compressible-but-often-unsummarized history, not head. - const inLiveHead = (i: number): boolean => i >= headStart && i < headEnd; + // The head, as Phase 0 sent it: the live window, and after a reset the + // system changes before it (headPositions). + const inHead = new Set(head); + const inLiveHead = (i: number): boolean => inHead.has(i); for (let i = 0; i < effectiveRecentStart && i < messages.length; i++) { if (inLiveHead(i)) continue; if (pinnedPositionsSet.has(i)) { @@ -16181,7 +16679,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const covered = new Set(); for (const s of selectedSummaries) for (const leaf of leavesOf(s)) covered.add(leaf); const rawPlanned = new Set(); - for (let i = headStart; i < headEnd && i < messages.length; i++) rawPlanned.add(messages[i].id); + for (const i of head) rawPlanned.add(messages[i].id); for (let i = effectiveRecentStart; i < messages.length; i++) rawPlanned.add(messages[i].id); for (const p of middleRaw) rawPlanned.add(p.msg.id); const selectedIdSet = new Set(selectedSummaries.map(s => s.id)); @@ -16423,7 +16921,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // applies the rule's verdicts (see applyImageStripping's asEstimated). // The render stats keep each message at the price emission // gave it, a stripped image included. - this.applyImageStripping(entries, store); + this.applyImageStripping(entries, store, { returned: true }); this.rsEnd(); return entries; } @@ -16527,13 +17025,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * A stored message's content priced as a compression request sends it: its - * thinking stripped, as a request's messages are built from raw messages, - * then priced as prepared (preparedTokensOf). Images count at the store's + * thinking stripped and its system changes left out, as a request's messages + * are built from raw messages (compressionInputContent), then priced as + * prepared (preparedTokensOf). Images count at the store's * estimate, though a request's inline image cap can replace some with * placeholders. */ protected sentTokensOf(content: ContentBlock[], ctx: StrategyContext): number { - return this.preparedTokensOf(stripThinkingBlocks(content), ctx); + return this.preparedTokensOf(compressionInputContent(content), ctx); } /** @@ -16759,6 +17258,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { let headContent = ''; let headReplacements = 0; for (const m of headMessages) { + // A message of system changes has no text: it adds no entry. + if (holdsSystemChange(m.content)) continue; // Counted as the request sends it: only the message's text blocks // reach it, with their serialized media replaced on a copy, so a // payload doesn't spend the cap. The request carries the markers, so @@ -17063,6 +17564,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // distant chronological regions and poisoned every ancestor merge. // Close even a thin run at the ownership seam. A short summary (or // raw fallback while it is pending) is cheaper than a non-tree span. + // A system change, which stays raw, is such a seam. The store refuses + // a change between a call and its result where a write can see it + // (MessageStore.refuseToolPairSplit), so this close parts a pair only + // in a shape that refusal leaves alone. closeCurrent(i); } let msgTokens = store.estimateTokens(msg); @@ -17238,13 +17743,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the count, byte and depth limits (the byte rule is greedy, so a large * newest image can be replaced while smaller older ones stay), the * hysteresis rule, which trims to a fraction of them, and the compression - * request's own image byte cap. Windowed passthrough uses the same text. */ + * request's own image byte and count caps. Windowed passthrough uses the + * same text. */ private static readonly IMAGE_PLACEHOLDER = '[image not shown: this context keeps only some images]'; /** Post-pass over compiled entries: replace image blocks with a text * placeholder once they fall outside the live-image window — either deeper * than `imageStripDepthTokens` from the newest message, or beyond the - * `maxLiveImages` most-recent images (counted newest-first). Summaries are + * `maxLiveImages` most-recent images (counted newest-first), or past + * `maxLiveImageBytes`, the count and bytes each within its image ceiling + * (resolvedImageLimits). Summaries are * already text, so they're naturally unaffected. The adjacent * "[image attachment: ]" text added at ingest preserves the filename, * so the placeholder itself stays terse. Reduces tokens, so it never pushes @@ -17299,15 +17807,27 @@ export class AutobiographicalStrategy implements ResettableStrategy { * prices by the estimates asks too. With `imageStripHysteresisRatio`, * every caller gets the verdicts: the rule's set is per stored message, * so counting what the entries send has nothing to add, and a tool result - * pairing repair moved keeps its own message's count. */ - protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView, opts: { asEstimated?: boolean } = {}): void { - const maxLive = this.config.maxLiveImages ?? 0; // 0 = unlimited count + * pairing repair moved keeps its own message's count. + * + * `returned`: select returns this render, so where it replaces an image + * the strip notes which ceiling-set limits were over, for select to + * report once it has returned it (reportCeilingHits): under the limits, + * those a replaced image is over, and with `imageStripHysteresisRatio`, + * where the rule's set replaces an image, those that brought a trim on + * anywhere in the rule's replay. + * A layout rendered only to compare against, such as kv-unified's kept + * one, and any other caller's render, leave it unset. */ + protected applyImageStripping( + entries: ContextEntry[], + store: MessageStoreView, + opts: { asEstimated?: boolean; returned?: boolean } = {}, + ): void { + // Count and bytes within the image ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; const depthTokens = this.config.imageStripDepthTokens ?? 0; // 0 = no depth strip - const maxLiveBytes = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; // Policy disabled: no limit is positive, as postStripEstimates decides it, // which then gives no verdicts. if (!(maxLive > 0) && !(depthTokens > 0) && !(maxLiveBytes > 0)) return; - const messages = store.getAll(); const posById = this.positionIndex(messages); // Image-bearing entries, newest-first by source position. Entries with no @@ -17320,19 +17840,34 @@ export class AutobiographicalStrategy implements ResettableStrategy { .filter(({ idx }) => this.hasImageBlock(entries[idx].content)) .sort((a, b) => b.pos - a.pos); + // For the ceiling line: which limits a ceiling sets were over where this + // strip replaces an image. Under the limits, those the image is over, + // whatever else strips it too; where the rule's set replaces an image, + // those that brought a trim on anywhere in its replay + // (stripReplayAtLimits). An image replaced for want of a verdict isn't + // the set's strip, and notes nothing. + const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); + const notes: CeilingKinds = { images: false, bytes: false }; + const noteCeilings = (over: Readonly): void => { + if (over.images && ceilingSet.images) notes.images = true; + if (over.bytes && ceilingSet.bytes) notes.bytes = true; + }; + if (opts.asEstimated || this.imageHysteresisRatio() !== undefined) { const verdicts = this.postStripVerdicts(store); const taken = new Map>(); /** The verdict of the next image of `group` in the message at `home`, * if that image has this image's payload. */ - const verdictOf = (image: ContentBlock, home: number, group: unknown): boolean => { + const verdictOf = (image: ContentBlock, home: number, group: unknown): ImageVerdict | undefined => { let byGroup = taken.get(home); if (!byGroup) taken.set(home, byGroup = new Map()); const n = byGroup.get(group) ?? 0; byGroup.set(group, n + 1); const judged = verdicts.get(home)?.get(group)?.[n]; - return judged !== undefined && judged.key === imagePayload(image) && judged.keep; + return judged !== undefined && judged.key === imagePayload(image) ? judged : undefined; }; + const rule = this.imageHysteresisRatio() !== undefined; + let ruleReplaced = false; for (const { idx, pos } of ordered) { const keeps: boolean[] = []; for (const block of entries[idx].content) { @@ -17341,13 +17876,25 @@ export class AutobiographicalStrategy implements ResettableStrategy { // where pairing repair moved it here, the one it was moved from. const from = this._movedFrom.get(block); const home = from === undefined ? pos : posById.get(from) ?? -1; - for (const image of imagesIn([block])) keeps.push(verdictOf(image, home, group)); + for (const image of imagesIn([block])) { + const judged = verdictOf(image, home, group); + if (judged && !judged.keep) { + if (rule) ruleReplaced = true; + else noteCeilings(judged.over); + } + keeps.push(judged?.keep ?? false); + } } // replaceImages asks about each image once, in imagesIn's order. let next = 0; entries[idx].content = replaceImages(entries[idx].content, () => !keeps[next++], AutobiographicalStrategy.IMAGE_PLACEHOLDER); } + if (ruleReplaced) { + const replay = this.stripReplayAtLimits(store); + if (replay) noteCeilings(replay.over); + } + if (opts.returned) this._ceilingNotes = notes; return; } @@ -17380,7 +17927,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { const blockBytes = inlineImageBytes(image); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; - if (tooDeep || overCount || overBytes) return false; + if (tooDeep || overCount || overBytes) { + noteCeilings({ images: overCount, bytes: overBytes }); + return false; + } keptImages++; keptImageBytes += blockBytes; return true; @@ -17402,6 +17952,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { return true; }, AutobiographicalStrategy.IMAGE_PLACEHOLDER); } + // A render select returns (`returned`) leaves which ceiling-set limits + // were over for select to report once it has returned it; the last such + // strip is the render it returns. Any other strip, such as + // renderAsNewest's, leaves nothing. + if (opts.returned) this._ceilingNotes = notes; } /** Hysteresis ratio in (0, 1), or undefined when the option is off. */ @@ -17413,8 +17968,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * imageStripHysteresisRatio's set, from the stored messages and the limits * alone: replay the messages oldest first, admitting each one's images in - * order, and whenever a limit binds (`maxLiveImages`, `maxLiveImageBytes`, - * or the oldest live image deeper than `imageStripDepthTokens`), strip the + * order, and whenever a limit binds (`maxLiveImages` and + * `maxLiveImageBytes`, each within its image ceiling, or the oldest live + * image deeper than `imageStripDepthTokens`), strip the * oldest live images one at a time until every limit fits ratio × itself. * A target that rounds to 0 is 0, but the newest image stays while it fits * the full limits, and so do the other images of its message while only @@ -17434,11 +17990,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { * prices them. */ protected imageStripReplay(store: MessageStoreView): ReadonlyMap | undefined { + return this.stripReplayAtLimits(store)?.out; + } + + /** The rule's set at the window's limits (imageStripReplay), memoized, + * with the count and byte limits that brought a trim on (applyImageStripping + * notes them for the ceiling line). */ + private stripReplayAtLimits(store: MessageStoreView): StripReplay | undefined { const ratio = this.imageHysteresisRatio(); if (ratio === undefined) return undefined; const depthCap = this.config.imageStripDepthTokens ?? 0; - const countCap = this.config.maxLiveImages ?? 0; - const bytesCap = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; + // The count and bytes within the image ceilings, as the limits apply them. + const { images: countCap, bytes: bytesCap } = this.resolvedImageLimits().target; if (!(depthCap > 0) && !(countCap > 0) && !(bytesCap > 0)) return undefined; return this.withToolPruning(store, () => { // Keyed as the post-strip memo is: the store's cached listing, its @@ -17453,7 +18016,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { memo && memo.messages === messages && memo.calibration === calibration && memo.depthCap === depthCap && memo.countCap === countCap && memo.bytesCap === bytesCap && memo.ratio === ratio && memo.pruning === pruning && memo.msgCap === msgCap - ) return memo.out; + ) return memo.replay; // tokensBefore[i]: the tokens of every message before i, so the depth of // an image in message j, with message i the newest, is // tokensBefore[i + 1] - tokensBefore[j]. @@ -17464,29 +18027,34 @@ export class AutobiographicalStrategy implements ResettableStrategy { let first = 0; let liveBytes = 0; const out = new Map(); + const brought: CeilingKinds = { images: false, bytes: false }; for (let i = 0; i < messages.length; i++) { for (const image of imagesIn(this.cappedForSend(store, messages[i]!, msgCap).content)) { const bytes = inlineImageBytes(image); live.push({ at: i, bytes }); liveBytes += bytes; } - const overCountOrBytes = (count: number, size: number): boolean => - (countCap > 0 && live.length - first > count) || (bytesCap > 0 && liveBytes > size); + const overCount = (count: number): boolean => countCap > 0 && live.length - first > count; + const overBytes = (size: number): boolean => bytesCap > 0 && liveBytes > size; + const overDepth = (depth: number): boolean => + depthCap > 0 && first < live.length && tokensBefore[i + 1]! - tokensBefore[live[first]!.at]! > depth; + const overCountOrBytes = (count: number, size: number): boolean => overCount(count) || overBytes(size); const over = (depth: number, count: number, size: number): boolean => - overCountOrBytes(count, size) || - (depthCap > 0 && first < live.length && tokensBefore[i + 1]! - tokensBefore[live[first]!.at]! > depth); + overCountOrBytes(count, size) || overDepth(depth); if (!over(depthCap, countCap, bytesCap)) continue; + if (overCount(countCap)) brought.images = true; + if (overBytes(bytesCap)) brought.bytes = true; const depth = Math.floor(depthCap * ratio); - const count = Math.floor(countCap * ratio); - const size = Math.floor(bytesCap * ratio); - while (first < live.length && over(depth, count, size)) { + const countTarget = Math.floor(countCap * ratio); + const sizeTarget = Math.floor(bytesCap * ratio); + while (first < live.length && over(depth, countTarget, sizeTarget)) { // The images left are one message's and within the full limits: the // newest of them stays, and the rest stay too while only depth is over // its target, since stripping images of one message can't lower the // depth it's at. if ( live[first]!.at === live[live.length - 1]!.at && !over(depthCap, countCap, bytesCap) && - (live.length - first === 1 || !overCountOrBytes(count, size)) + (live.length - first === 1 || !overCountOrBytes(countTarget, sizeTarget)) ) break; const oldest = live[first++]!; liveBytes -= oldest.bytes; @@ -17494,20 +18062,22 @@ export class AutobiographicalStrategy implements ResettableStrategy { out.set(id, (out.get(id) ?? 0) + 1); } } - this._imageStripReplayMemo = { messages, calibration, depthCap, countCap, bytesCap, ratio, pruning, msgCap, out }; - return out; + const replay: StripReplay = { out, over: brought }; + this._imageStripReplayMemo = { messages, calibration, depthCap, countCap, bytesCap, ratio, pruning, msgCap, replay }; + return replay; }); } private _imageStripReplayMemo: { messages: readonly StoredMessage[]; calibration: number; depthCap: number; countCap: number; bytesCap: number; ratio: number; - pruning: object; msgCap: number | undefined; out: ReadonlyMap; + pruning: object; msgCap: number | undefined; replay: StripReplay; } | null = null; /** * Post-strip token estimate per message index (2026-07-12 tail-starvation * fix). Mirrors `applyImageStripping` as `selectAdaptive` asks for it - * (`asEstimated`): an image beyond the `maxLiveImages` + * (`asEstimated`), with the same limits from resolvedImageLimits(), the + * count and bytes within their image ceilings: an image beyond the `maxLiveImages` * newest (counted newest-first) or deeper than `imageStripDepthTokens` of * the store's (calibrated) estimate from the live end renders as a placeholder — so every place * that BUDGETS messages (recent-window walk-back, head/tail sums, middle @@ -17551,8 +18121,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { ): number[] { const messages = store.getAll(); const stripDepth = this.config.imageStripDepthTokens ?? 0; - const maxLive = this.config.maxLiveImages ?? 0; - const maxLiveBytes = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; // A compile asks for this pass more than once (recent-window scan, then // the picker inputs). The listing array is the store's own cached view, // so its identity, calibration and the knobs key the memo, the @@ -17575,7 +18144,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // each block's images (a tool result's by its call's id, the top-level // images together), each with its payload (imagePayload) and whether the // strip keeps it, in order. - const verdicts = new Map>>(); + const verdicts = new Map>(); const stripActive = stripDepth > 0 || maxLive > 0 || maxLiveBytes > 0; // With imageStripHysteresisRatio, the rule's set decides in place of the // limits, which it already fits: a message's first images, as many as @@ -17613,7 +18182,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // price, through the cut's extra, rather than here. // Block by block, which walks the cut's images in imagesIn's order. const cut = this.cappedForSend(store, messages[i]!, msgCap).content; - const groups = new Map>(); + const groups = new Map(); let ruleLeft = replay !== undefined ? replay.get(messages[i]!.id) ?? 0 : undefined; for (const block of cut) { const group = imageGroup(block); @@ -17637,7 +18206,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } let judged = groups.get(group); if (!judged) groups.set(group, judged = []); - judged.push({ key: imagePayload(b), keep }); + judged.push({ key: imagePayload(b), keep, over: { images: beyondCount, bytes: beyondBytes } }); } } if (groups.size > 0) verdicts.set(i, groups); @@ -17651,12 +18220,113 @@ export class AutobiographicalStrategy implements ResettableStrategy { private _postStripMemo: { messages: readonly StoredMessage[]; calibration: number; stripDepth: number; maxLive: number; maxLiveBytes: number; ratio: number | undefined; pruning: object; msgCap: number | undefined; out: number[]; - verdicts: ReadonlyMap>>; + verdicts: ReadonlyMap>>; } | null = null; - /** Byte wall default: 20MB of base64 (API total-request cap is 32MB). */ + /** Byte wall default: 20 MiB of base64, under the default image byte + * ceiling (24 MiB), which leaves room under membrane's 28 MiB cap. */ protected static readonly DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; + /** Which ceilings have been said to strip the live window (once each per instance). */ + private readonly ceilingBoundReported = { images: false, bytes: false }; + /** Which ceiling-set limits were over where the last render select + * returned replaced an image (applyImageStripping, `returned`). */ + private _ceilingNotes: CeilingKinds = { images: false, bytes: false }; + + /** Say once for each ceiling, when a select returns a render in which + * images are shown as placeholders and a limit that ceiling sets was over + * (applyImageStripping's notes): under the limits, at an image the render + * replaces, and with imageStripHysteresisRatio, at a trim anywhere in the + * rule's replay. The depth limit or a limit the host set may be stripping + * too. While the rule is off, the line adds the cost of a window held at + * the ceiling and names the option that spaces it, except under a count + * ceiling of 1, which leaves the option no room. The rule's set is this + * class's; a subclass's override of imageStripReplay isn't consulted for + * the line. A dry select, a preview, an observer's reconstruction or a + * refused select says nothing, and the line waits for the first render + * returned. */ + private reportCeilingHits(): void { + const said = this.ceilingBoundReported; + const notes = this._ceilingNotes; + for (const kind of ['images', 'bytes'] as const) { + if (!notes[kind] || said[kind]) continue; + said[kind] = true; + const line = ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted()); + // The option can space the trims only where the ceiling leaves it room: + // under a count ceiling of 1 the newest image stays at every trim. + const spaces = kind === 'bytes' || this.resolvedImageLimits().ceiling.images > 1; + this.diag.warn(this.imageHysteresisRatio() === undefined && spaces ? line + AutobiographicalStrategy.CEILING_BATCH_CLAUSE : line); + } + } + + /** What the ceiling line adds while imageStripHysteresisRatio is off: the + * cost of a window held at a ceiling, and the option that spaces it. */ + private static readonly CEILING_BATCH_CLAUSE = + '; at the ceiling each new image turns the oldest live ones into placeholders, which rewrites the prompt ' + + 'cache from there, and imageStripHysteresisRatio does that in batches instead'; + + /** The image limits whose values are defaults: those the library's default + * layer gave or no layer gave, and those the host gave as undefined or + * null where configuredImageLimits then applies a default. For + * maxLiveImages it applies none: an explicit undefined erases the default + * 6 and reads as no limit, which is the host's doing. */ + private imageLimitsDefaulted(): ReadonlySet { + const keys = ['maxLiveImages', 'maxLiveImageBytes', 'imageCeiling', 'imageByteCeiling', 'maxCompressionImageBytes'] as const; + return new Set(keys.filter((key) => { + const source = this.configProvenance[key]; + if (source === undefined || source === 'library-default') return true; + return key !== 'maxLiveImages' && this.config[key] == null; + })); + } + + /** The live-image limits as configured, with the defaults where unset. */ + private configuredImageLimits(): { + maxLiveImages: number; maxLiveImageBytes: number; imageCeiling: number; imageByteCeiling: number; + } { + return { + maxLiveImages: this.config.maxLiveImages ?? 0, + maxLiveImageBytes: this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES, + imageCeiling: this.config.imageCeiling ?? DEFAULT_IMAGE_CEILING, + imageByteCeiling: this.config.imageByteCeiling ?? DEFAULT_IMAGE_BYTE_CEILING, + }; + } + + /** + * The live window's count and byte limits, kept within the image + * ceilings, and the ceilings themselves. The strip, the hysteresis rule's + * set, the post-strip estimates and a compression prompt all read their + * limits here, so they can't disagree about them. + */ + private resolvedImageLimits(): ResolvedImageLimits { + return resolveLiveImageLimits(this.configuredImageLimits()); + } + + /** + * The image ceilings (see ContextStrategy.imageCeilings), as the window + * applies them: a malformed one at its default, a fraction floored. + */ + imageCeilings(): ImageCeilings { + const { ceiling } = this.resolvedImageLimits(); + return { imageCeiling: ceiling.images, imageByteCeiling: ceiling.bytes }; + } + + /** Whether the image limits' warning has been said (once per instance). */ + private imageLimitsWarningReported = false; + + /** One line when a configured image limit sits above its image ceiling, + * or a ceiling is malformed and takes its default, so an operator sees why + * the window or a compression prompt keeps fewer images than they set. */ + private reportImageLimitsWarningOnce(): void { + if (this.imageLimitsWarningReported) return; + this.imageLimitsWarningReported = true; + const line = liveImageLimitsWarning(this.name, { + ...this.configuredImageLimits(), + maxCompressionImageBytes: + this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES, + }, this.imageLimitsDefaulted()); + if (line) this.diag.warn(line); + } + /** Compression prompts carry head + recall frontier + raw chunk alongside * their images, so they get a tighter image budget than the live window. */ protected static readonly DEFAULT_MAX_COMPRESSION_IMAGE_BYTES = 12 * 1024 * 1024; @@ -17693,12 +18363,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected prepareCompressionRequest(request: NormalizedRequest): CompressionPreparation { const prepared = { ...request, messages: request.messages.map(message => ({ ...message })) }; const serializedMediaReplacements = this.stripSerializedCompressionMedia(prepared.messages); - const imageBudgetBytes = - this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES; - const cap = { keptBytes: 0 }; - const imageReplacements = this.capCompressionImageBytes(prepared.messages, imageBudgetBytes, cap); + // A compression prompt carries at most the image ceilings. + const { ceiling } = this.resolvedImageLimits(); + const imageBudgetBytes = clampToCeiling( + this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES, + ceiling.bytes, + ); + const imageCountCap = ceiling.images; + const cap = { keptBytes: 0, keptImages: 0 }; + const imageReplacements = this.capCompressionImageBytes(prepared.messages, imageBudgetBytes, cap, imageCountCap); return { request: prepared, serializedMediaReplacements, imageReplacements, imageBudgetBytes, imageBytesKept: cap.keptBytes, + imageCountCap, imagesKept: cap.keptImages, }; } @@ -17740,10 +18416,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { } if (counts.imageReplacements > 0) { const mib = (bytes: number) => `${(bytes / (1024 * 1024)).toFixed(1)}MiB`; + const limits = [ + ...(counts.imageBudgetBytes > 0 ? [`the ${mib(counts.imageBudgetBytes)} inline image budget`] : []), + ...(counts.imageCountCap > 0 ? [`the ${counts.imageCountCap}-image ceiling`] : []), + ].join(' and '); this.diag.error( `[autobiographical] compression prompt: replaced ${counts.imageReplacements} image(s) with placeholders ` + - `to stay under the ${mib(counts.imageBudgetBytes)} inline image budget ` + - `(kept ${mib(counts.imageBytesKept)}, newest messages first) [${label}]`, + `to stay under ${limits} ` + + `(kept ${counts.imagesKept} image(s), ${mib(counts.imageBytesKept)}, newest messages first) [${label}]`, ); } } @@ -17964,26 +18644,30 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * Cap inline image bytes in a COMPRESSION prompt (2026-07-12). The main + * Cap inline image bytes in a COMPRESSION prompt (2026-07-12), and with + * `capImages` their number (2026-10-19, `imageCeiling`). The main * window enforces `maxLiveImageBytes` through the strip policy; the * summarizer's raw-chunk replay had no such wall and leaned on membrane's * byte shed instead — which is a transport backstop, not a policy owner * (it fired at 27MB on a live merge). Every inline image counts, whatever * its shape (see inlineImageBytes), generated images included. * Messages are walked newest-first, and the blocks within a message in - * order; each image is kept if it fits in what's left of the budget, and - * the others become the same loud placeholder the window uses, so the - * summarizer is told plainly that it is not seeing them. + * order; each image is kept if it fits in what's left of the budget and + * of the count, and the others become the same loud placeholder the window + * uses, so the summarizer is told plainly that it is not seeing them. + * A cap of 0 or less is none. * Returns the number of images replaced; `stats`, when given, receives the - * bytes kept. + * bytes and the images kept, every image's where neither cap is set. */ protected capCompressionImageBytes( messages: Array<{ content: ContentBlock[] }>, capBytes: number, - stats?: { keptBytes: number }, + stats?: { keptBytes: number; keptImages?: number }, + capImages = 0, ): number { - if (capBytes <= 0) return 0; + const countCap = capImages > 0 ? capImages : 0; let kept = 0; + let keptImages = 0; let dropped = 0; // Images nested in tool_result content count too: an agent that drives a // shell/plotter/browser carries most of its image bytes NESTED in tool @@ -17996,15 +18680,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (!Array.isArray(m.content)) continue; m.content = replaceImages(m.content, (image) => { const bytes = inlineImageBytes(image); - if (kept + bytes <= capBytes) { + if ((capBytes <= 0 || kept + bytes <= capBytes) && (countCap === 0 || keptImages < countCap)) { kept += bytes; + keptImages++; return false; } dropped++; return true; }, AutobiographicalStrategy.IMAGE_PLACEHOLDER); } - if (stats) stats.keptBytes = kept; + if (stats) { + stats.keptBytes = kept; + stats.keptImages = keptImages; + } return dropped; } @@ -18149,6 +18837,137 @@ export class AutobiographicalStrategy implements ResettableStrategy { return result; } + /** + * Where history starts in the store: the position of the first of `ids` + * still there (store order), or -1 when none is. A host can remove stored + * messages (agent-framework's Discord module does, on a delete or a history + * sync) while a summary over them, or a queued chunk holding them, lives + * on; -1 means nothing in the store says when that history happened. + */ + protected startInStore(ids: Iterable, position: ReadonlyMap): number { + let start = -1; + for (const id of ids) { + const at = position.get(id); + if (at !== undefined && (start < 0 || at < start)) start = at; + } + return start; + } + + /** + * Where a summary starts in the store: its first leaf message still there + * (expandSummaryToLeafMessageIds, startInStore), whether or not that is + * sourceRange.first; -1 when none is. The one rule for both recall sets, a + * compression request's and a merge's (1a), and for where a recall pair + * sits among them: a summary starting before the history being summarized + * is recalled, at its start. A summary with no message left isn't: its + * place is unknown, and its coverage holds no message out of the request. + */ + protected summaryStartInStore( + summary: SummaryEntry, + position: ReadonlyMap, + summariesById: ReadonlyMap, + ): number { + const leaves = new Set(); + this.expandSummaryToLeafMessageIds(summary, summariesById, leaves); + return this.startInStore(leaves, position); + } + + /** + * The head window a compression or merge request gives as context for the + * history it summarizes from `pos` on: the head in force when the message at + * `pos` was the newest, cut at `pos` and at the compression holds + * (holdBoundary). From the live head's start on, that is the live head, + * cut at `pos`, so a chunk inside the live head window (one a widened + * window grew over) no longer gets the head's messages after its start. + * A `pos` that's unknown (-1: none of the history's messages is left in + * the store, startInStore) gets no head: nothing in the store says when + * that history happened, and the live head may have come after it. Before + * the live head's start, it is the head starting at the latest topic + * transition message at or before `pos` (isTopicTransitionMessage, the test + * that restores the live head's start at a restart), or at the opening when + * there is none, ending by the walk the live head takes from its start + * (headWindowEndFrom). So history from before a head-window reset gets the + * head it followed: the opening, or the transition and opening of the topic + * it was in. There, a chunk that starts at a transition, or at the + * store's first message (the opening's own chunk, which a reset releases), + * gets none: nothing came before it. + * + * After a reset, these requests gave the new topic's opening ahead of the + * older history they summarize (#317), where section 1 of + * compressChunkHierarchical places the chronicle's opening "exactly where + * the original instance saw it" and keeps later events out. + * + * Three things it is, by choice: + * - Before the live head's start, where a head starts comes from the store + * alone, as a restart reads it. A strategy-level resetHeadWindow to a + * message that isn't a transition isn't in the store: history before that + * message gets the latest transition before it, or the opening, as after a + * restart. History from that message on gets the live head, which a + * restart moves to the last transition in the store. So a restart gives + * the same heads while the live head starts at the last transition in the + * store. + * - A past head is priced with today's headWindowTokens and calibration, as + * the live head is, and the ratchet guard (anchorHeadToCoverage) extends + * it by today's chunk ownership, not by those in force when it was the + * head. + * - Before the live head's start, any Context Manager message whose text + * starts with "[Topic Transition]" starts a head, as the restart reads + * one, whether or not resetHeadWindow wrote it. + */ + protected headAsOf(store: MessageStoreView, pos: number): { start: number; end: number } { + return this.headsAsOf(store)(pos); + } + + /** + * headAsOf for many positions in one store state, such as each queued chunk + * a count of queued work reads (requestCarriesToolBlocks): the live head's + * start, the holds and the live head's end are read once, the transitions + * before the live head's start once when a position before it is asked + * for, and each past head's end once per start. Tick's requests (through + * headAsOf) and the count both read heads here, so a subclass that changes + * heads overrides this. + */ + protected headsAsOf(store: MessageStoreView): (pos: number) => { start: number; end: number } { + const liveStart = this.getHeadWindowStartIndex(store); + const hold = this.holdBoundary(store); + let liveEnd: number | undefined; + let messages: StoredMessage[] | undefined; + let transitions: number[] | undefined; + const pastEnds = new Map(); + return (pos) => { + if (pos < 0) return { start: liveStart, end: liveStart }; + if (pos >= liveStart) { + liveEnd ??= this.getHeadWindowEnd(store); + return { start: liveStart, end: Math.min(liveEnd, hold, pos) }; + } + messages ??= store.getAll(); + if (!transitions) { + const found: number[] = []; + for (let i = 1; i < liveStart && i < messages.length; i++) { + if (this.isTopicTransitionMessage(messages[i])) found.push(i); + } + transitions = found; + } + // The latest transition at or before pos, or the opening. + let start = 0; + for (let lo = 0, hi = transitions.length - 1; lo <= hi;) { + const mid = (lo + hi) >> 1; + if (transitions[mid]! <= pos) { + start = transitions[mid]!; + lo = mid + 1; + } else { + hi = mid - 1; + } + } + let end = pastEnds.get(start); + if (end === undefined) { + end = this.headWindowEndFrom(store, messages, start); + pastEnds.set(start, end); + } + return { start, end: Math.min(end, hold, pos) }; + }; + } + /** * Index of the first message AFTER the head window. * Messages [headStart, headEnd) are sent raw, never summarized. @@ -18165,10 +18984,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { * material (issue #122). See anchorHeadToCoverage. */ protected getHeadWindowEnd(store: MessageStoreView): number { - if (this.config.headWindowTokens <= 0) return 0; + return this.headWindowEndFrom(store, store.getAll(), this.getHeadWindowStartIndex(store)); + } - const messages = store.getAll(); - const startIdx = this.getHeadWindowStartIndex(store); + /** + * Where a head window starting at `startIdx` ends, by the walk the live head + * window takes (getHeadWindowEnd): the live head's start, or a past head's + * (headAsOf). + */ + protected headWindowEndFrom(store: MessageStoreView, messages: StoredMessage[], startIdx: number): number { + if (this.config.headWindowTokens <= 0) return 0; let tokens = 0; for (let i = startIdx; i < messages.length; i++) { @@ -18244,6 +19069,46 @@ export class AutobiographicalStrategy implements ResettableStrategy { return message.content.some(block => block.type === 'tool_result'); } + /** + * The head's positions, in the order they're sent: after a head-window + * reset, the system changes stored before the head window, then the window + * [headStart, headEnd). Every selector sends the head first and the + * history before it later, with the middle, so a change left in that + * history would follow the head's own changes and apply after them: a + * removal could run before the definition it removes. In the head, every + * change keeps its order, and the solved layout, the emission, the cache + * check's rebuild and the markers read one head. The history before the + * head, which the reset already sends after the head, then reads them too, + * which the provider takes: it doesn't check a past tool call against the + * tools offered where the call sits. With nothing in the window (the head + * window disabled, or a window that ends where it starts), nothing is sent + * ahead of that history, and the changes stay where they are. + */ + protected headPositions(messages: readonly StoredMessage[], headStart: number, headEnd: number): number[] { + const end = Math.min(headEnd, messages.length); + const out: number[] = []; + if (headStart > 0 && end > headStart) { + for (const i of systemChangePositions(messages)) { + if (i >= headStart) break; + out.push(i); + } + } + for (let i = headStart; i < end; i++) out.push(i); + return out; + } + + /** + * The middle's position ranges, in order: what comes before the tail + * outside the live head window, [0, headStart) after a head-window reset + * included. With the head window disabled, headEnd is 0 under a positive + * headStart, so the live window ends at the later of the two, and no + * position is in two ranges or in a range and the tail. A walk over them + * passes over the head's other positions (headPositions). + */ + protected static middleRanges(headStart: number, headEnd: number, tailStart: number): ReadonlyArray { + return [[0, Math.min(headStart, tailStart)], [Math.max(headStart, headEnd), tailStart]]; + } + /** * Shared structural pass for both selectors: repair tool pairing over the * entries as selected. What sending does to a raw message's tool blocks @@ -18706,19 +19571,21 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Collapse consecutive messages from the same participant into single messages. * Required because Claude API rejects consecutive same-role messages. + * A message left empty, as compression input leaves one of thinking or + * system changes alone (compressionInputContent), is passed over, so its + * neighbours join as they would without it and no separator marks it. */ protected collapseConsecutiveMessages( messages: Array<{ participant: string; content: ContentBlock[] }> ): Array<{ participant: string; content: ContentBlock[] }> { - if (messages.length === 0) return []; - - const result: Array<{ participant: string; content: ContentBlock[] }> = [ - { participant: messages[0].participant, content: [...messages[0].content] }, - ]; + const result: Array<{ participant: string; content: ContentBlock[] }> = []; - for (let i = 1; i < messages.length; i++) { + for (let i = 0; i < messages.length; i++) { + if (messages[i].content.length === 0) continue; const last = result[result.length - 1]; - if (messages[i].participant === last.participant) { + if (!last) { + result.push({ participant: messages[i].participant, content: [...messages[i].content] }); + } else if (messages[i].participant === last.participant) { // Merge: add separator then content last.content.push({ type: 'text', text: '\n\n---\n\n' } as ContentBlock); last.content.push(...messages[i].content); @@ -18859,7 +19726,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * The answer body for the LIVE WINDOW — the surface the agent whose memory - * this is reads back — under `carrierPolicy`. + * this is reads back — under `carrierPolicy`, or under `'live-strip'` + * while thinking is bound (liveCarrierPolicy). * * Under `'full'` (default) this IS `summaryAnswerProse`, block for block. * Under `'live-strip'` the captured reasoning carriers are omitted from the @@ -18868,8 +19736,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * here would break the signature that the mint surface depends on — hence * omission, never rewriting. Every surviving block is passed through by * reference, except a Responses item a stripped reasoning item led to: every - * block that shares it is copied without its raw item, and so without its id - * (see raw-forms.ts). + * block that shares it, other than an opaque carrier, is copied without its + * raw item, and so without its id (see raw-forms.ts). * * A carrier-only entry (reasoning captured, no prose block — degenerate, and * guarded against at mint time) would strip to an empty assistant turn, @@ -18879,7 +19747,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { */ protected liveWindowAnswerProse(summary: SummaryEntry): ContentBlock[] { const prose = this.summaryAnswerProse(summary); - if (this.config.carrierPolicy !== 'live-strip') return prose; + if (this.liveCarrierPolicy() !== 'live-strip') return prose; const stripped = stripThinkingBlocks(prose); const carriesProse = stripped.some( (block) => block.type === 'text' && block.text.trim().length > 0, @@ -19286,11 +20154,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { transitionBlocked: this.transitionBlocked, lastRenderStats: this._lastRenderStats, lastCompileEstimate: this._lastCompileEstimate, + lastCompileInjected: this._lastCompileInjected, calibrationArmed: this._calibrationArmed, calibrationLoaded: this._calibrationLoaded, calibration: this._calibration, calibrationReset: this._calibrationReset, calibrationProvisional: this._calibrationProvisional, + calibrationDeclared: this._calibrationDeclared, storeCalibration: store.getTokenCalibration?.(), storeView: this._storeView, storeOrder: this._storeOrder, @@ -19313,11 +20183,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.transitionBlocked = saved.transitionBlocked; this._lastRenderStats = saved.lastRenderStats; this._lastCompileEstimate = saved.lastCompileEstimate; + this._lastCompileInjected = saved.lastCompileInjected; this._calibrationArmed = saved.calibrationArmed; this._calibrationLoaded = saved.calibrationLoaded; this._calibration = saved.calibration; this._calibrationReset = saved.calibrationReset; this._calibrationProvisional = saved.calibrationProvisional; + this._calibrationDeclared = saved.calibrationDeclared; if (saved.storeCalibration !== undefined) store.setTokenCalibration?.(saved.storeCalibration); this._storeView = saved.storeView; this._storeOrder = saved.storeOrder; @@ -19335,7 +20207,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * select sends each shard as its own message), then `applyImageStripping`, * so a message stored after the batch counts toward the images as in a * compile. The tool pass that comes between them (postProcessToolEntries) - * is left out: it pairs and prunes by the window around these messages. + * is left out: it pairs by the window around these messages. What sending + * prunes from a message's tool blocks reaches it through `cappedForSend`. * * Those messages are the recent window's, and the head window's too when * no message comes between the two: a compile places the middle's diff --git a/src/strategies/knowledge.ts b/src/strategies/knowledge.ts index 92bc2afe..fa4abf2c 100644 --- a/src/strategies/knowledge.ts +++ b/src/strategies/knowledge.ts @@ -9,6 +9,7 @@ import type { } from '../types/index.js'; import { AutobiographicalStrategy, type Chunk } from './autobiographical.js'; import type { ConfigLayer } from '../config-provenance.js'; +import { systemChangePositions } from '../system-changes.js'; const DEFAULT_RESEARCH_PREFIXES = ['mcpl:', 'zulip:']; const DEFAULT_SUBAGENT_PREFIXES = ['subagent:']; @@ -65,6 +66,12 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { protected rebuildChunks(store: MessageStoreView): void { const messagesToChunk = this.getCompressibleMessages(store); + const all = store.getAll(); + const position = this.positionIndex(all); + const changes = systemChangePositions(all); + // The walk below is in store order, so one cursor finds the changes + // stored between each message and the one before it. + let nextChange = 0; // Preserve existing compressed chunks const existingCompressed = new Map(); @@ -73,6 +80,20 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { existingCompressed.set(this.chunkKey(chunk), chunk); } } + // A chunk holds the messages of one summary, or of none, but for a call's + // result: it closes where the summary covering the walk's messages + // changes, except between a call and its result (below). The walk otherwise + // decides boundaries afresh, so once a message that closed a chunk is + // removed (a consolidation removes its system changes), a chunk could run + // into or across a summary's span; the overlap guard refuses to compress + // such a chunk for good, and its messages go out raw beside the summary + // that already covers some of them. Where the chunks already match the + // summaries, the walk closes there anyway. An L2's sources are summaries, + // not messages. + const coveredBy = new Map(); + for (const s of this.summaries) { + if (s.level === 1) for (const id of s.sourceIds) coveredBy.set(id, s.id); + } this.chunks = []; this.compressionQueue = []; @@ -82,9 +103,52 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { let currentPhase: PhaseType | null = null; let lastToolPhase: PhaseType = 'synthesis'; let chunkFilteredStart = 0; + const closeAt = (end: number): void => { + const chunk = this.createChunk( + this.chunks.length, + chunkFilteredStart, + end, + currentChunk, + currentTokens, + existingCompressed + ); + chunk.phaseType = currentPhase!; + this.chunks.push(chunk); + if (!chunk.compressed) this.compressionQueue.push(chunk.index); + + currentChunk = []; + currentTokens = 0; + chunkFilteredStart = end; + }; for (let i = 0; i < messagesToChunk.length; i++) { const msg = messagesToChunk[i]; + const previous = currentChunk[currentChunk.length - 1]; + let changeBefore = false; + const at = position.get(msg.id)!; + while (nextChange < changes.length && changes[nextChange]! < at) { + changeBefore = true; + nextChange++; + } + // Not between a call and its result: closing there would leave the + // result with its call folded into the summary, and pairing repair + // drops a result whose call isn't before it. So when a summary ends at + // a call answered only later, the result joins the summary's chunk, + // which the overlap guard then refuses, as on main. + const summaryChanges = previous !== undefined + && coveredBy.get(previous.id) !== coveredBy.get(msg.id) + && !(this.hasToolUse(previous) && this.hasToolResult(msg)); + if (previous && (changeBefore || summaryChanges)) { + // The compressible stream skips a system change, which stays raw. A + // chunk across it would summarize both sides as one, rendered at its + // first message, ahead of the change, so what came after the change + // would be read before it. Close even a thin run there. A store + // without changes whose chunks match its summaries chunks as before. + // The store refuses a change between a call and its result where a + // write can see it (MessageStore.refuseToolPairSplit), so this close + // parts a pair only in a shape that refusal leaves alone. + closeAt(i); + } const phase = this.classifyMessage(msg, lastToolPhase); // Track tool phase for tool_result inheritance @@ -118,23 +182,7 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { (phaseChanged || sizeExceeded) && !this.lastMessageContainsToolUse(currentChunk); - if (shouldClose) { - const chunk = this.createChunk( - this.chunks.length, - chunkFilteredStart, - i, - currentChunk, - currentTokens, - existingCompressed - ); - chunk.phaseType = currentPhase!; - this.chunks.push(chunk); - if (!chunk.compressed) this.compressionQueue.push(chunk.index); - - currentChunk = []; - currentTokens = 0; - chunkFilteredStart = i; - } + if (shouldClose) closeAt(i); currentChunk.push(msg); currentTokens += msgTokens; diff --git a/src/strategies/passthrough.ts b/src/strategies/passthrough.ts index ba8cfaf1..8ad84472 100644 --- a/src/strategies/passthrough.ts +++ b/src/strategies/passthrough.ts @@ -7,7 +7,10 @@ import type { TokenBudget, ContextEntry, StoredMessage, + SelectOptions, } from '../types/index.js'; +import { OverBudgetError } from '../adaptive/picker.js'; +import { holdsSystemChange, systemChangePositions } from '../system-changes.js'; /** * Simple passthrough strategy that copies all messages to context. @@ -16,7 +19,12 @@ import type { */ export class PassthroughStrategy implements ContextStrategy { readonly name = 'passthrough'; - /** Renders history raw; when it cannot all fit, the oldest is not rendered. */ + /** + * Renders history raw; when it cannot all fit, the oldest is not rendered, + * but for its system changes, and when those alone, or those and the + * newest conversation message, can't fit, a committing select refuses with + * OverBudgetError. + */ readonly renderedForms = ['raw', 'omitted'] as const; checkReadiness(): ReadinessState { @@ -30,7 +38,8 @@ export class PassthroughStrategy implements ContextStrategy { select( store: MessageStoreView, _log: ContextLogView, - budget: TokenBudget + budget: TokenBudget, + opts?: SelectOptions, ): ContextEntry[] { const messages = store.getAll(); const entries: ContextEntry[] = []; @@ -61,40 +70,87 @@ export class PassthroughStrategy implements ContextStrategy { // If we hit the limit before adding all messages, // keep the most recent ones that fit if (entries.length < messages.length) { - return this.selectFromEnd(store, maxTokens); + return this.selectFromEnd(store, maxTokens, opts); } return entries; } /** - * Select messages from the end when we can't fit everything. + * Select messages from the end when we can't fit everything. A system + * change before the cut is kept too, in its order ahead of the rest: it + * changes the tools and system text everything after it was read with + * (system-changes.ts). Its tokens are set aside first, and when they alone + * exceed the budget a committing select refuses: only the caller, by + * consolidating its changes, can clear that. A change is no conversation + * message, so when the newest conversation message doesn't fit beside + * them a committing select refuses too, as the windowed strategy does: a + * budget refusal, which a newer conversation message that fits clears + * while they leave room. Without changes, a select that fits no message + * sends nothing, as on main. Where a change can go in a request is + * membrane's to say (after a user turn): this strategy can't tell a user + * participant from another, and membrane refuses a request holding a + * change it can't place. */ - private selectFromEnd(store: MessageStoreView, maxTokens: number): ContextEntry[] { + private selectFromEnd(store: MessageStoreView, maxTokens: number, opts?: SelectOptions): ContextEntry[] { const messages = store.getAll(); - const entries: ContextEntry[] = []; - let totalTokens = 0; + const changes = systemChangePositions(messages); + let changeTokens = 0; + for (const i of changes) changeTokens += store.estimateTokens(messages[i]!); + let totalTokens = changeTokens; + if (totalTokens > maxTokens && changes.length > 0 && !opts?.dryRun) { + throw new OverBudgetError({ + budget: maxTokens, + actual: totalTokens, + stageId: 'passthrough-system-changes', + clearedBy: [], + diagnostics: { headTokens: totalTokens, tailTokens: 0, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, + }); + } // Work backwards from the most recent + let start = messages.length; for (let i = messages.length - 1; i >= 0; i--) { - const msg = messages[i]; + const msg = messages[i]!; + if (holdsSystemChange(msg.content)) continue; // counted above const tokens = store.estimateTokens(msg); if (totalTokens + tokens > maxTokens) { break; } - entries.unshift({ + totalTokens += tokens; + start = i; + } + // A store of changes alone gets here only when they're over the budget: + // select() sends every message when all fit. A committing select refused + // them above, and a dry one skips this refusal too. + if (start === messages.length && changes.length > 0 && !opts?.dryRun) { + let newest = messages.length - 1; + while (holdsSystemChange(messages[newest]!.content)) newest--; + const newestTokens = store.estimateTokens(messages[newest]!); + throw new OverBudgetError({ + budget: maxTokens, + actual: changeTokens + newestTokens, + stageId: 'passthrough-changes-and-newest-message', + clearedBy: changeTokens < maxTokens ? ['newer-message'] : [], + diagnostics: { headTokens: changeTokens, tailTokens: newestTokens, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, + }); + } + + const entries: ContextEntry[] = []; + const add = (i: number): void => { + const msg = messages[i]!; + entries.push({ index: i, sourceMessageId: msg.id, sourceRelation: 'copy', participant: msg.participant, content: msg.content, }); - - totalTokens += tokens; - } - + }; + for (const i of changes) if (i < start) add(i); + for (let i = start; i < messages.length; i++) add(i); return entries; } } diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 8ec09277..75de23a5 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -1,6 +1,7 @@ import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; import type { ContextStrategy, + ImageCeilings, StrategyContext, ReadinessState, MessageStoreView, @@ -20,6 +21,16 @@ import { OverBudgetError } from '../adaptive/picker.js'; import { observeStoreBranch, type StoreBranchGeneration } from '../branch-generation.js'; import { hasImage, imagesIn, inlineImageBytes, replaceImages, strippedImageSaving } from '../inline-images.js'; import { capContent, cappedPriceExtra } from './message-cap.js'; +import { + DEFAULT_IMAGE_CEILING, + DEFAULT_IMAGE_BYTE_CEILING, + liveImageLimitsWarning, + ceilingBoundLine, + ceilingSetLimits, + resolveLiveImageLimits, + type ResolvedImageLimits, +} from '../live-image-limits.js'; +import { holdsSystemChange, systemChangePositions } from '../system-changes.js'; /** * Options for {@link WindowedPassthroughStrategy}. @@ -28,12 +39,14 @@ export interface WindowedPassthroughOptions { /** * When the window no longer fits the budget, the anchor jumps forward so * the new window is roughly this fraction of the usable budget (default - * 0.5). Between jumps the anchor never moves, so compiled output between - * re-anchors is a byte-stable prefix plus pure appends — and the strategy + * 0.5), or, when the store holds system changes, which go out wherever the + * window starts, this fraction of the room they leave. Between jumps the + * anchor never moves, so compiled output between re-anchors is a + * byte-stable prefix plus pure appends — and the strategy * marks that prefix for the provider's prompt cache (see * {@link WindowedPassthroughStrategy} on cache markers), so the accepted - * cost is one cache miss per jump, amortized over ~(1 - fraction) × budget - * of appended tokens, instead of the per-turn prefix churn a naively + * cost is one cache miss per jump, amortized over ~(1 - fraction) × that + * room of appended tokens, instead of the per-turn prefix churn a naively * sliding front produces. */ reAnchorFraction?: number; @@ -69,19 +82,41 @@ export interface WindowedPassthroughOptions { * `imageStripDepthTokens` from the newest message, or past * `maxLiveImageBytes` of cumulative inline image bytes (newest-first). * Defaults follow `DEFAULT_AUTOBIOGRAPHICAL_CONFIG` (and the memory - * strategy's byte wall); 0 disables that dimension. A strip changes the + * strategy's byte wall). A count or byte limit of 0 means up to its + * image ceiling, below, and a depth of 0 means no depth strip. A strip changes the * bytes at that position, which the cache-marker placement absorbs by * measurement (the marker falls back to the deepest byte-stable entry). */ maxLiveImages?: number; imageStripDepthTokens?: number; maxLiveImageBytes?: number; + /** + * The image ceilings, mirroring `AutobiographicalConfig.imageCeiling` and + * `imageByteCeiling`: the most images, and the most of their inline image + * bytes, the window keeps, counted as + * `AutobiographicalConfig.maxLiveImages` says, whatever `maxLiveImages` and + * `maxLiveImageBytes` say (0 there means up to these). They don't bound a + * whole request: what the window doesn't count, and what a caller injects + * or appends, isn't counted. `imageCeilings()` reports them. + * Defaults 500 images and 24 MiB; 0 = no ceiling, and a value that isn't 0 + * or a finite number above it takes the default. + */ + imageCeiling?: number; + imageByteCeiling?: number; } /** Text substituted for an image block once it leaves the live-image window * (identical to the autobiographical strategy's placeholder, so a merged * timeline reads the same in both agents' windows). */ const IMAGE_PLACEHOLDER = '[image not shown: this context keeps only some images]'; + +/** The live-image policy's verdicts, by message position (see liveImagePolicy). */ +interface ImagePolicy { + strips: (boolean[] | undefined)[]; + saved: number[]; + ceilings: ({ images: boolean; bytes: boolean } | undefined)[]; +} + /** Mirrors `AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES` (protected there). */ const DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; @@ -120,21 +155,31 @@ const DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; * **Hard budget.** After per-message shaping (truncation, image policy) the * window is priced against `maxTokens - reserveForResponse`. Overflow * re-anchors; if the single newest message alone still exceeds the usable - * budget the strategy refuses with {@link OverBudgetError} (the framework's - * over-budget breaker handles it) rather than emitting a request the - * provider will reject. Previews report the oversized window instead. + * budget, or with system changes, the changes and the newest conversation + * message, or the changes alone, the strategy refuses with + * {@link OverBudgetError} (the framework's over-budget breaker handles it) + * rather than emitting a request the provider will reject. Previews report + * the oversized window instead. */ export class WindowedPassthroughStrategy implements ContextStrategy { readonly name = 'windowed-passthrough'; readonly maxMessageTokens?: number; - /** Renders the window raw; history before the anchor is not rendered. */ + /** Renders the window raw; history before the anchor is not rendered, but for its system changes. */ readonly renderedForms = ['raw', 'omitted'] as const; private readonly reAnchorFraction: number; private readonly anchorStateIdOverride?: string; - private readonly maxLiveImages: number; + /** The image limits as configured, with the defaults where unset. */ + private readonly configuredImageLimits: { + maxLiveImages: number; maxLiveImageBytes: number; imageCeiling: number; imageByteCeiling: number; + }; + /** The image limits the options didn't give, whose values are defaults. */ + private readonly imageLimitsDefaulted: ReadonlySet; private readonly imageStripDepthTokens: number; - private readonly maxLiveImageBytes: number; + /** Whether the image limits' warning has been said (once per instance). */ + private imageLimitsWarningReported = false; + /** Which ceilings have been said to strip the live window (once each per instance). */ + private readonly ceilingBoundReported = { images: false, bytes: false }; private anchor: Sequence = 0; private store: JsStore | null = null; @@ -157,15 +202,51 @@ export class WindowedPassthroughStrategy implements ContextStrategy { this.reAnchorFraction = fraction; this.anchorStateIdOverride = options.anchorStateId; this.maxMessageTokens = options.maxMessageTokens; - this.maxLiveImages = options.maxLiveImages ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.maxLiveImages ?? 0; + this.configuredImageLimits = { + maxLiveImages: options.maxLiveImages ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.maxLiveImages ?? 0, + maxLiveImageBytes: options.maxLiveImageBytes ?? DEFAULT_MAX_LIVE_IMAGE_BYTES, + imageCeiling: options.imageCeiling ?? DEFAULT_IMAGE_CEILING, + imageByteCeiling: options.imageByteCeiling ?? DEFAULT_IMAGE_BYTE_CEILING, + }; + this.imageLimitsDefaulted = new Set( + (['maxLiveImages', 'maxLiveImageBytes', 'imageCeiling', 'imageByteCeiling'] as const).filter((key) => options[key] == null), + ); this.imageStripDepthTokens = options.imageStripDepthTokens ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.imageStripDepthTokens ?? 0; - this.maxLiveImageBytes = options.maxLiveImageBytes ?? DEFAULT_MAX_LIVE_IMAGE_BYTES; + } + + /** + * The image ceilings (see ContextStrategy.imageCeilings), as the window + * applies them: a malformed one at its default, a fraction floored. + */ + imageCeilings(): ImageCeilings { + const { ceiling } = this.resolvedImageLimits(); + return { imageCeiling: ceiling.images, imageByteCeiling: ceiling.bytes }; + } + + /** The live window's count and byte limits, kept within the image + * ceilings, and the ceilings themselves. */ + private resolvedImageLimits(): ResolvedImageLimits { + return resolveLiveImageLimits(this.configuredImageLimits); + } + + /** Say once that an image ceiling holds a limit the options set as none + * or above it, once a limit it sets was over where the window strips an + * image (liveImagePolicy's `ceilings`). */ + private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { + if (this.ceilingBoundReported[kind]) return; + this.ceilingBoundReported[kind] = true; + this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits, this.imageLimitsDefaulted)); } async initialize(ctx: StrategyContext): Promise { this.store = ctx.store; this.ns = ctx.namespace; + if (!this.imageLimitsWarningReported) { + this.imageLimitsWarningReported = true; + const line = liveImageLimitsWarning(this.name, this.configuredImageLimits, this.imageLimitsDefaulted); + if (line) this.diag.warn(line); + } this.stateId = this.anchorStateIdOverride ?? `${ctx.namespace}/windowed:anchor`; registerStateIfAbsent(this.store, { id: this.stateId, strategy: 'snapshot' }); this.loadBranchState(); @@ -299,17 +380,33 @@ export class WindowedPassthroughStrategy implements ContextStrategy { return estimate + cappedPriceExtra(store, msg, content); }; + // A system change before the window is sent too, in its order ahead of + // it: it changes the tools and system text everything after it was read + // with (system-changes.ts). So every change is priced wherever the + // window starts. + const changes = systemChangePositions(messages); + const keptBefore = (from: number): number[] => changes.filter((i) => i < from); + let changeTokens = 0; + for (const i of changes) changeTokens += priced(i); let total = 0; + for (const i of keptBefore(start)) total += priced(i); for (let i = start; i < messages.length; i++) total += priced(i); + let movedAnchor: number | undefined; if (total > maxTokens) { - // Coarse re-anchor: jump forward so the window is ~fraction × budget, - // then emit from the new anchor. One prefix invalidation, then - // byte-stable prefix + appends again until the next overflow. - const target = maxTokens * this.reAnchorFraction; - let acc = 0; + // Coarse re-anchor: jump forward so the window holds the changes and + // about that fraction of the room they leave, then emit from the new + // anchor. One prefix invalidation, then + // byte-stable prefix + appends again until the next overflow. The + // system changes go out wherever the window starts, so its messages + // take that fraction of the room the changes leave, and appends get + // the rest of that room, (1 - fraction) × (budget - changes), before + // the next. + const target = changeTokens + (maxTokens - changeTokens) * this.reAnchorFraction; + let acc = changeTokens; let newStart = messages.length; for (let i = messages.length - 1; i >= start; i--) { + if (holdsSystemChange(messages[i]!.content)) continue; // priced above const tokens = priced(i); if (acc + tokens > target) break; acc += tokens; @@ -317,20 +414,18 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } // Always make progress: if even the newest message exceeds the // target, emit just that message rather than nothing — subject to the - // hard-budget check below. + // hard-budget check below. A system change is no conversation message, + // so that's the newest message that isn't one, if the window holds one. if (newStart === messages.length && messages.length > 0) { newStart = messages.length - 1; + while (newStart > start && holdsSystemChange(messages[newStart]!.content)) newStart--; } start = newStart; total = 0; + for (const i of keptBefore(start)) total += priced(i); for (let i = start; i < messages.length; i++) total += priced(i); - if (!opts?.dryRun && start < messages.length) { - const newAnchor = messages[start].sequence; - if (newAnchor !== this.anchor) { - this.anchor = newAnchor; - this.persistAnchor(); - this.pendingCause = 'window'; - } + if (start < messages.length && messages[start].sequence !== this.anchor) { + movedAnchor = messages[start].sequence; } } @@ -339,27 +434,43 @@ export class WindowedPassthroughStrategy implements ContextStrategy { // would reject the request; refuse loudly instead (the framework // recognizes OverBudgetError and runs its over-budget breaker), as the // autobiographical strategy does. Previews report rather than throw. + // The anchor moves only for a compile that sends, below: a refused one + // leaves it where the last sent window started, so the turns that fit + // once the caller consolidates its changes are still in the window. if (total > maxTokens && !opts?.dryRun) { throw new OverBudgetError({ budget: maxTokens, actual: total, - stage: 'Windowed passthrough: the newest message alone', + // Which part is over: the newest message alone, or the system + // changes sent beside it, with it or alone. + stageId: changeTokens === 0 + ? 'windowed-newest-message' + : total > changeTokens + ? 'windowed-changes-and-newest-message' + : 'windowed-system-changes', // No compression here: the window moves past this message once a - // newer one that fits arrives (the progress clause above). - clearedBy: ['newer-message'], + // newer one that fits arrives (the progress clause above). System + // changes go out wherever the window starts, so once they leave no + // room for a message only the caller, consolidating them, clears it. + clearedBy: changeTokens < maxTokens ? ['newer-message'] : [], diagnostics: { - headTokens: 0, - tailTokens: total, + headTokens: changeTokens, + tailTokens: total - changeTokens, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0, }, }); } + if (movedAnchor !== undefined && !opts?.dryRun) { + this.anchor = movedAnchor; + this.persistAnchor(); + this.pendingCause = 'window'; + } const entries: ContextEntry[] = []; - for (let i = start; i < messages.length; i++) { - const msg = messages[i]; + const add = (i: number): void => { + const msg = messages[i]!; entries.push({ index: i, sourceMessageId: msg.id, @@ -367,9 +478,14 @@ export class WindowedPassthroughStrategy implements ContextStrategy { participant: msg.participant, content: shaped.get(i) ?? msg.content, }); - } + }; + for (const i of keptBefore(start)) add(i); + for (let i = start; i < messages.length; i++) add(i); this.stripImages(entries, policy); + // Only a render select returns says a ceiling's line, for the images it + // holds. A preview's (dryRun) doesn't, nor any other caller's. + if (!opts?.dryRun) this.reportCeilings(entries, policy); if (!opts?.dryRun) this.placeCacheMarkers(entries); return entries; } @@ -387,13 +503,16 @@ export class WindowedPassthroughStrategy implements ContextStrategy { * message to its newest: each cut as select cuts it (cappedContent), then * the live-image policy's verdicts applied (liveImagePolicy, stripImages), * each decided from its message and newer ones, with depth from the view's - * newest message, so they're select's. A message before the anchor, which select - * leaves out, comes back as null. Like select, it first reloads the - * branch the store has moved to. A re-anchor the next compile's budget - * forces isn't foreseen: it moves the anchor only past a window bigger - * than the budget allows. A body sharded under the autobiographical - * strategy comes back shard by shard, since select sends each as its own - * message. + * newest message, so they're select's. A message before the anchor comes + * back as null, since select leaves it out, but for a system change, which + * comes back as select sends it ahead of the window, cut as any message + * is: a compile moves the anchor only to a message it sent, but a + * caller's setAnchor can move it past one just stored. Like select, it + * first reloads the branch the store has moved to. A re-anchor the next + * compile's budget forces isn't foreseen: it moves the anchor only past a + * window bigger than the budget allows. A body sharded under the + * autobiographical strategy comes back shard by shard, since select sends + * each as its own message. */ renderAsNewest(store: MessageStoreView, ids: readonly MessageId[]): Array { this.syncToBranch(); @@ -421,7 +540,11 @@ export class WindowedPassthroughStrategy implements ContextStrategy { this.stripImages(entries, this.liveImagePolicy(messages, store, from)); const rendered = new Map(); for (const entry of entries) rendered.set(entry.sourceMessageId!, entry.content); - return ids.map((id) => rendered.get(id) ?? null); + return ids.map((id) => { + const i = at.get(id); + if (i !== undefined && !shown(i) && holdsSystemChange(messages[i]!.content)) return this.cappedContent(messages[i]!); + return rendered.get(id) ?? null; + }); } /** 'window' when the last committed select moved the window's start. */ @@ -442,7 +565,10 @@ export class WindowedPassthroughStrategy implements ContextStrategy { * each message's estimate gives back for the images it loses (`saved`). * `select` prices the window by `saved` and applies `strips` to what it * emits (stripImages), so the verdicts it prices are the ones the strip - * applies. + * applies. `ceilings` names, for each message, the limits an image + * ceiling sets (ceilingSetLimits) that were over where one of its images + * is stripped, whatever else strips it too, for the line a returned + * render says (reportCeilings). * * That holds wherever the window starts, and the re-anchor walk reads them * as they are, as `renderAsNewest` does from the batch's oldest message: @@ -458,13 +584,14 @@ export class WindowedPassthroughStrategy implements ContextStrategy { messages: StoredMessage[], store: MessageStoreView, from: number, - ): { strips: (boolean[] | undefined)[]; saved: number[] } { + ): ImagePolicy { const strips: (boolean[] | undefined)[] = []; const saved: number[] = []; - const maxLive = this.maxLiveImages; + const ceilings: ({ images: boolean; bytes: boolean } | undefined)[] = []; + // Count and bytes within the image ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; const depthTokens = this.imageStripDepthTokens; - const maxLiveBytes = this.maxLiveImageBytes; - if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return { strips, saved }; + if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return { strips, saved, ceilings }; const calibration = store.getTokenCalibration?.() ?? 1; // The placeholder as the store prices it, calibrated, by its own text estimator. const placeholderPrice = store.estimateTokens({ content: [{ type: 'text', text: IMAGE_PLACEHOLDER }] } as unknown as StoredMessage); @@ -480,6 +607,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } + const ceilingSet = ceilingSetLimits(this.configuredImageLimits); let keptImages = 0; let keptImageBytes = 0; // Newest-first. Every image a message carries, nested in a tool_result @@ -496,6 +624,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; const strip = tooDeep || overCount || overBytes; + const decided = (ceilings[i] ??= { images: false, bytes: false }); + decided.images ||= overCount && ceilingSet.images; + decided.bytes ||= overBytes && ceilingSet.bytes; if (strip) { tokens += strippedImageSaving(image, placeholderPrice, calibration); } else { @@ -507,7 +638,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { strips[i] = verdicts; saved[i] = tokens; } - return { strips, saved }; + return { strips, saved, ceilings }; } /** The policy's verdicts applied to entries whose `index` is their @@ -523,6 +654,15 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } + /** Says a ceiling's line once, for a ceiling whose limit was over where the window strips an image of an entry the render holds. */ + private reportCeilings(entries: ContextEntry[], policy: ImagePolicy): void { + for (const entry of entries) { + const decided = policy.ceilings[entry.index]; + if (decided?.images) this.reportCeilingBoundOnce('images'); + if (decided?.bytes) this.reportCeilingBoundOnce('bytes'); + } + } + // -------------------------------------------------------------------------- // Cache markers (first-claim ≤ 3; this strategy uses ≤ 2) // -------------------------------------------------------------------------- diff --git a/src/system-changes.ts b/src/system-changes.ts new file mode 100644 index 00000000..ef36e71a --- /dev/null +++ b/src/system-changes.ts @@ -0,0 +1,69 @@ +/** + * System changes: membrane's `system_change` block (SystemChangeContent), a + * change to the tools and system text the model is offered from its place in + * the conversation on. One stands alone in its message + * (well-formed-content.ts), and membrane sends such a message as a system + * message of its own, at the first point at or after its place that follows + * a user turn, so always before the reply the request asks for. So one + * stored between a tool call and its result would go out after the result, + * and the store refuses that write where a write can see it + * (MessageStore.refuseToolPairSplit): kept raw there, it would part the call + * from its result. + * + * Every strategy compiles such a message unaltered, in its place among the + * raw messages. A fold or a window that left one out would take its change + * out of everything after it, so the autobiographical strategy keeps it raw + * as a pin keeps a message, and the passthrough strategies keep it when + * their window starts after it. Under the autobiographical strategy's + * positionedRecallPairs: false, the middle's summaries go ahead of its raw + * messages, so there a change keeps its place among raw messages only. The + * one move is after a head-window reset, when the changes stored before the + * head become part of the head, ahead of the head window, so that they keep + * their order (AutobiographicalStrategy.headPositions). + * + * None is dropped, so they're carried until the caller consolidates them. + * When they alone exceed the budget (beyond the autobiographical strategy's + * grace, and on its hierarchical path only where enforceBudget is on), every + * strategy refuses with OverBudgetError, its clearedBy empty, since + * no newer message or compression shrinks them; a dry compile reports where + * the strategy's other refusals do (the passthrough strategies refuse none + * on a dry compile). While they fit, the passthrough strategies refuse when + * the newest conversation message doesn't fit beside them (OverBudgetError, + * naming a newer message while they leave room for one) + * and otherwise drop the oldest messages as they do without changes, and + * the autobiographical strategy refuses a loss as it refuses any: the + * hierarchical path's recent window refuses an eviction with + * UncoveredDropError. What compression reads + * leaves them out: a change alters the request, not the conversation, and a + * summarizer's request would apply it, or be refused for it. + */ + +import type { ContentBlock } from '@animalabs/membrane'; + +/** Whether a block is a system change. */ +export function isSystemChange(block: ContentBlock | { type?: unknown }): boolean { + return (block as { type?: unknown }).type === 'system_change'; +} + +/** Whether a message's content holds a system change, and so, as written, nothing else. */ +export function holdsSystemChange(content: readonly ContentBlock[]): boolean { + for (let i = 0; i < content.length; i++) if (isSystemChange(content[i]!)) return true; + return false; +} + +const positionsOf = new WeakMap(); + +/** + * The positions in a listing of the messages that hold a system change, in + * order. A select reads them several times, so they're found once per + * listing: a listing is the store's, which callers never change + * (MessageStore.getAll), and a write makes a new one. + */ +export function systemChangePositions(messages: readonly { content: readonly ContentBlock[] }[]): readonly number[] { + const known = positionsOf.get(messages); + if (known) return known; + const positions: number[] = []; + for (let i = 0; i < messages.length; i++) if (holdsSystemChange(messages[i]!.content)) positions.push(i); + positionsOf.set(messages, positions); + return positions; +} diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts new file mode 100644 index 00000000..64e800c2 --- /dev/null +++ b/src/thinking-binding.ts @@ -0,0 +1,342 @@ +/** + * Thinking binding (#155): which signed thinking blocks a compile can send to + * a model that binds each block to the request it was minted in. + * + * Such a provider refuses a request that carries a thinking block, or with + * its drop control sets the block aside, once anything sent before the block + * differs from the request that minted it: a fold, an edit, a moved + * injection, a changed tool or system text. A compile can't know what minted + * a block, but it can know what its branch last sent. So the manager keeps, + * per branch, a record of what the branch last sent: the last committing + * compile's messages as sent, and the rows a host noted after them + * (`ContextManager.noteRequestSent`): what its stream's requests went on to + * carry and, once the stream ended, its last reply. A compile keeps a + * message's thinking only while nothing up to it differs from the record, + * and past the record's end nothing keeps its thinking: only the host knows + * which stored reply answered which request. On a branch with no record, a + * reply keeps its thinking only with the drop control, for the provider to + * judge. + * + * The record is a Chronicle snapshot state in the manager's namespace, so it + * is per branch, survives a restart, and a branch made from another starts + * from the parent's record as of the branch point. + */ + +import { createHash } from 'node:crypto'; +import type { JsStore } from '@animalabs/chronicle'; +import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; +import { canonicalJson } from './canonical-json.js'; +import { releaseEditedRawForms } from './raw-forms.js'; +import { registerStateIfAbsent } from './state-registration.js'; +import type { CompiledMessageSources, RawBodySource, ThinkingStrip } from './types/context.js'; +import type { MessageId, Sequence } from './types/message.js'; +import type { RequestIdentity, SelectOptions } from './types/strategy.js'; + +export function isThinkingBlock(block: ContentBlock): boolean { + return block.type === 'thinking' || (block.type as string) === 'redacted_thinking'; +} + +/** + * The content without its thinking blocks, each left out whole, with every + * raw form a left-out block carried released from the blocks that kept it, + * as `releaseEditedRawForms` releases them (see raw-forms.ts). + */ +export function withoutThinking(content: ContentBlock[]): ContentBlock[] { + return releaseEditedRawForms(content, content.filter((block) => !isThinkingBlock(block))); +} + +/** + * The content less the thinking blocks `sent` doesn't hold (the canonical + * forms `thinkingSent` takes from the compiled messages), raw forms released + * as `withoutThinking` releases them; the content itself when `sent` holds + * all of its thinking. A block is matched by its canonical form, cache + * marker aside: a signature makes each one distinct. + */ +export function thinkingAsSent(content: ContentBlock[], sent: ReadonlySet): ContentBlock[] { + const left = (block: ContentBlock) => isThinkingBlock(block) && !sent.has(thinkingKey(block)); + if (!content.some(left)) return content; + return releaseEditedRawForms(content, content.filter((block) => !left(block))); +} + +/** Each thinking block `messages` carry, by its canonical form, for `thinkingAsSent`. */ +export function thinkingSent(messages: ReadonlyArray<{ content: ContentBlock[] }>): Set { + const sent = new Set(); + for (const message of messages) for (const block of message.content) if (isThinkingBlock(block)) sent.add(thinkingKey(block)); + return sent; +} + +function thinkingKey(block: ContentBlock): string { + return canonicalJson(withoutCacheMarker(block)); +} + +/** What a branch last sent, as its record keeps it. */ +export interface BindingRecord { + /** Each message the compile that wrote the record sent, in order: a + * digest of its participant and content (`messageDigest`). `[UNVOUCHED]` + * once a note came with another head than `identity`: then the record + * vouches for nothing until a compile writes it again. */ + sent: string[]; + /** What a host noted after those messages, in order: for each stored + * row, a digest of each message the compile makes of it as the strategy + * renders it among the newest (a bundled tool cycle makes several), or + * `UNVOUCHED` for a row the next compile can't reproduce. Empty until a + * note. */ + continued: string[]; + /** The stored bodies that compile sent without their thinking. */ + without: MessageId[]; + /** The head digests that compile sent. */ + identity: RequestIdentity; + /** The branch's sequence (`JsStore.currentSequence`) when the record + * began: a message at or below it was stored before. */ + recordedAfter: Sequence; + /** The `provenance.compileId` of the committing compile that wrote it: a + * note is for that compile's stream only. */ + compileId: string; +} + +/** A row of a noted request that matches nothing a compile sends. */ +export const UNVOUCHED = ''; + +export const IDENTITY_KEYS = ['system', 'tools', 'model', 'request'] as const; + +/** + * Refuse a binding compile that can't vouch for the thinking it would keep: + * one without `thinkingBinding.participant`, which says whose replies are the + * model's own, or without `identity`, or with a digest that isn't a string. + */ +export function refuseUnboundBinding(opts: SelectOptions): void { + const participant: unknown = opts.thinkingBinding?.participant; + if (typeof participant !== 'string' || participant.length === 0) { + throw new TypeError( + '`thinkingBinding.binds` needs `thinkingBinding.participant`: the participant whose messages are ' + + "the model's own replies, such as the agent's name", + ); + } + if (typeof opts.identity !== 'object' || opts.identity === null) { + throw new TypeError("`thinkingBinding.binds` needs `identity`: the digests of what the request carries before its messages"); + } + refuseUnreadableIdentity(opts.identity); +} + +/** Refuse head digests that aren't all strings: written to a record, they + * would leave it unreadable. */ +export function refuseUnreadableIdentity(identity: unknown): void { + if (typeof identity !== 'object' || identity === null) { + throw new TypeError(`\`identity\` must be an object of four digests, got ${identity === null ? 'null' : typeof identity}`); + } + for (const key of IDENTITY_KEYS) { + const value: unknown = (identity as Record)[key]; + if (typeof value !== 'string') { + throw new TypeError(`\`identity.${key}\` must be a string, got ${value === null ? 'null' : typeof value}`); + } + } +} + +export function bindingStateId(namespace: string): string { + return `${namespace}/thinking-binding`; +} + +export function registerBindingState(store: JsStore, namespace: string): void { + registerStateIfAbsent(store, { id: bindingStateId(namespace), strategy: 'snapshot' }); +} + +/** + * The branch's record, or null when it has none. A value that isn't a + * record's shape is reported as malformed and read as no record. + */ +export function readBindingRecord(store: JsStore, namespace: string): { record: BindingRecord | null; malformed: boolean } { + const value = store.getStateJson(bindingStateId(namespace)) as unknown; + if (value === null || value === undefined) return { record: null, malformed: false }; + return isBindingRecord(value) ? { record: value, malformed: false } : { record: null, malformed: true }; +} + +export function writeBindingRecord(store: JsStore, namespace: string, record: BindingRecord): void { + store.setStateJson(bindingStateId(namespace), record); +} + +function isStringArray(value: unknown): value is string[] { + return Array.isArray(value) && value.every((item) => typeof item === 'string'); +} + +function isSequence(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value); +} + +function isBindingRecord(value: unknown): value is BindingRecord { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const v = value as Record; + if (!isStringArray(v.sent) || !isStringArray(v.continued) || !isStringArray(v.without)) return false; + if (!isSequence(v.recordedAfter) || typeof v.compileId !== 'string') return false; + if (typeof v.identity !== 'object' || v.identity === null || Array.isArray(v.identity)) return false; + const identity = v.identity as Record; + return IDENTITY_KEYS.every((key) => typeof identity[key] === 'string'); +} + +/** + * A compiled message's digest: its participant and content, keys in canonical + * order, so a block read back from the store (keys sorted) and the same block + * as built compare equal, as the provider reads them. A cache breakpoint isn't + * part of it, nor a block's own cache marker (`cache_control`, + * `cacheControl`): the binding doesn't count them, as observed in a run on + * Anthropic's API on 2026-10-23 (Haiku 5.5 with the drop control, on an + * account that enforces the binding and on one that doesn't): moving only a + * cache marker, onto or off a block before two replies, dropped neither + * reply's thinking, where changing one word dropped both. Other binding + * models weren't run for markers. 66 bits of + * SHA-256 are kept: a digest is only ever compared with the one recorded at + * the same place. + */ +export function messageDigest(message: Pick): string { + return createHash('sha256') + .update(canonicalJson({ p: message.participant, c: message.content.map(withoutCacheMarker) })) + .digest('base64url') + .slice(0, 11); +} + +function withoutCacheMarker(block: ContentBlock): unknown { + const marked = block as unknown as Record; + if (!('cache_control' in marked) && !('cacheControl' in marked)) return block; + const { cache_control: _snake, cacheControl: _camel, ...rest } = marked; + return rest; +} + +export interface BindInput { + /** The compiled messages, injections placed. */ + messages: readonly NormalizedMessage[]; + /** Parallel to `messages`. */ + sources: readonly CompiledMessageSources[]; + record: BindingRecord | null; + dropBlock: boolean; + identity: RequestIdentity; + /** The participant whose messages are the model's own replies. */ + participant: string; + /** The branch's sequence now (`JsStore.currentSequence`). */ + sequence: Sequence; + /** This compile's `provenance.compileId`. */ + compileId: string; +} + +export interface BindOutcome { + /** The messages to send. */ + messages: NormalizedMessage[]; + /** Set when a reply's thinking is newly left out. */ + strip?: ThinkingStrip; + /** What a committing compile records. Its `without` is every stored body + * whose thinking this compile leaves out. */ + record: BindingRecord; + /** The thinking blocks left out of messages with a stored body, which a + * strategy's select priced; an injection's never were. */ + leftOut: ContentBlock[]; +} + +/** + * Mark each raw body in `without` as not carried whole: a compile left its + * thinking out, a block removed (`RawBodySource.missing` gains 'content'). + */ +export function markThinkingLeftOut(sources: readonly CompiledMessageSources[], without: ReadonlySet): void { + for (const source of sources) { + if (source.kind !== 'raw') continue; + for (const body of source.bodies) { + if (!without.has(body.messageId)) continue; + body.complete = false; + body.missing = [...new Set<'shards' | 'content'>([...(body.missing ?? []), 'content'])]; + } + } +} + +function rawBodies(source: CompiledMessageSources | undefined): RawBodySource[] { + return source?.kind === 'raw' ? source.bodies : []; +} + +/** + * Decide which compiled messages keep their thinking (see + * `SelectOptions.thinkingBinding`), and the record that compile leaves. + * + * A message keeps its thinking only when it is the participant's own, from a + * stored body; none of its bodies was sent without thinking by the record's + * compile; it was stored after the record began, unless the request carries + * the drop control; and, where there is a record, nothing up to it differs + * from the record, which it is within. With no record, only the drop control + * keeps a reply's thinking. Any other message's thinking (another + * participant's, a summary's recall, an injection) was never minted in this + * agent's requests, and is always left out. + */ +export function bindThinking(input: BindInput): BindOutcome { + const { record, identity, participant } = input; + const recordedAfter = record?.recordedAfter ?? input.sequence; + const recorded = record ? [...record.sent, ...record.continued] : []; + const changed = record ? IDENTITY_KEYS.filter((key) => identity[key] !== record.identity[key]) : []; + let diverged = changed.length > 0; + const wasWithout = new Set(record?.without ?? []); + const messages: NormalizedMessage[] = []; + const sent: string[] = []; + const without = new Set(); + const newly = new Set(); + let newlyBeforeRecord = false; + const leftOut: ContentBlock[] = []; + const stripped = (message: NormalizedMessage): NormalizedMessage => ({ ...message, content: withoutThinking(message.content) }); + + for (let i = 0; i < input.messages.length; i++) { + const message = input.messages[i]!; + const bodies = rawBodies(input.sources[i]); + const thinking = message.content.some(isThinkingBlock); + // The participant's own reply, from stored bodies, not left out before. + const owned = thinking && bodies.length > 0 && message.participant === participant && + !bodies.some((body) => wasWithout.has(body.messageId)); + // Stored before the record began, and the provider won't set it aside. + const beforeRecord = !input.dropBlock && bodies.some((body) => body.sequence <= recordedAfter); + let keep = owned && !beforeRecord; + if (owned && beforeRecord) { + for (const body of bodies) newly.add(body.messageId); + newlyBeforeRecord = true; + } + let out = thinking && !keep ? stripped(message) : message; + let digest = messageDigest(out); + // Past the record's end, nothing was vouched for. + if (!diverged && record !== null) diverged = i >= recorded.length || digest !== recorded[i]; + if (diverged && keep) { + keep = false; + out = stripped(message); + digest = messageDigest(out); + for (const body of bodies) newly.add(body.messageId); + } + if (thinking && !keep && bodies.length > 0) { + for (const block of message.content) if (isThinkingBlock(block)) leftOut.push(block); + for (const body of bodies) without.add(body.messageId); + } + messages.push(out); + sent.push(digest); + } + + const cause: ThinkingStrip['cause'] = changed.length > 0 ? 'head' : newlyBeforeRecord ? 'first' : 'messages'; + return { + messages, + ...(newly.size > 0 ? { strip: { newly: [...newly], cause, ...(cause === 'head' ? { changed } : {}) } } : {}), + record: { + sent, + continued: [], + without: [...without], + identity: copyIdentity(identity), + recordedAfter, + compileId: input.compileId, + }, + leftOut, + }; +} + +function copyIdentity(identity: RequestIdentity): RequestIdentity { + return { system: identity.system, tools: identity.tools, model: identity.model, request: identity.request }; +} + +/** + * The record once a host has noted the rows it sent after the messages of + * the compile that wrote it (`continued`, each a digest or `UNVOUCHED`), + * with the head `identity`. A note with another head than the record's + * carried thinking minted under the old one: the record keeps its head and + * vouches for nothing until a compile writes it again, so the next compile + * under the new head reports the head change. + */ +export function continueRecord(record: BindingRecord, continued: string[], identity: RequestIdentity): BindingRecord { + const sameHead = IDENTITY_KEYS.every((key) => identity[key] === record.identity[key]); + return sameHead ? { ...record, continued } : { ...record, sent: [UNVOUCHED], continued: [] }; +} diff --git a/src/types/context.ts b/src/types/context.ts index e18a5792..9756bd07 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -1,6 +1,6 @@ import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; -import type { MessageId, StoredContentBlock, StoredMessage } from './message.js'; -import type { RenderStats } from './strategy.js'; +import type { MessageId, Sequence, StoredContentBlock, StoredMessage } from './message.js'; +import type { RenderStats, RequestIdentity } from './strategy.js'; /** * Describes how a context entry relates to its source message. @@ -173,6 +173,39 @@ export interface CompileResult { */ rawBodies?: ReadonlyMap; + /** + * Where this compile's messages first differ from the last committing + * compile's, for every strategy: the index of the first message, injections + * included, whose participant or content bytes differ from that compile's + * message at the same index, or that has none there. So when this compile + * only appended to that one's messages it is the first appended index, and + * it equals `messages.length` when these messages are that compile's or a + * prefix of them. An injection placed beside the newest user message moves + * with each new user message, so after one it's that injection's old index + * rather than the first appended one. Null only for the manager's first + * committing compile, since the comparison is with what this manager last + * compiled, held in memory: a manager opened again, as after a restart, + * starts with none. The comparison is by bytes, so it holds across a + * branch switch or a new strategy, whichever way the switch was made. The + * store's content is taken as the same bytes where it's the same array, so + * a block a caller changed in place after storing it, which the store's + * contract rules out, isn't seen. + * Cache markers don't count, and neither do `systemInjections`, which the + * caller adds to the system prompt itself. A dry compile leaves it out and + * doesn't move the comparison; a committing compile that returns moves it + * whether or not its request is sent, and one that throws doesn't. A + * caller can tell from it whether a compile rewrote what the last + * committing compile gave, such as a system change it wrote. It's compared + * when read, so a compile whose caller never reads it serializes only its + * injections, whose bytes a committing compile takes since a caller may + * change its own array: read it only where it's used. + * Read after later compiles, it still compares this compile with the one + * before it. Until it's read, the result keeps this compile's compiled + * messages and the last committing compile's, by reference; once read, it + * holds the number. + */ + firstChanged?: number | null; + /** * This compile's render stats, as its select tallied them while emitting * entries: the same accounting `getRenderStats()` gives a live compile. @@ -188,6 +221,55 @@ export interface CompileResult { * sending it. Absent when the strategy records none. */ renderStats?: RenderStats; + + /** + * Set on a compile with `thinkingBinding.binds` that newly leaves out a + * stored reply's thinking: what, and why (see + * `SelectOptions.thinkingBinding`). + */ + thinkingStrip?: ThinkingStrip; + + /** + * On every compile with `thinkingBinding.binds`: the branch's sequence + * (`JsStore.currentSequence`) when its binding record began, at or above + * every message stored before then. A branch that inherits its parent's + * record keeps its value, which is at most the branch point; a dry compile + * with no record reports what the first record would hold. A reply at or + * below it was stored before the context manager recorded what the branch + * sent, so the provider may refuse or set aside its thinking without any + * strip having missed. Sequences are branch-global across slots, so a + * stored message's `sequence` compares with it directly. + */ + thinkingRecordedAfter?: Sequence; +} + +/** What a compile with `thinkingBinding.binds` newly leaves out. */ +export interface ThinkingStrip { + /** + * The stored bodies (`RawBodySource.messageId`) of the participant's + * replies whose thinking this compile leaves out and the compile that + * wrote the branch's record didn't, whether or not its request went out: + * that compile sent them with their thinking, they were stored since, or + * it didn't send them as stored replies at all. A record is written by a + * committing compile with `binds`, so a compile without it is passed + * over. + * So a reply can be reported again: one whose thinking went, that a later + * compile didn't send, and that renders again. A host that tells each + * loss once keeps its own set of what it has told. + */ + newly: MessageId[]; + /** + * `'head'`: an `identity` digest changed, and every message goes without + * its thinking. `'first'`: a reply stored before the branch's record + * began, which only the drop control lets keep its thinking, goes without + * it, as every reply does at a branch's first compile without the control. + * `'messages'`: a compiled message differs from the record, and every + * message from it on goes without. When a compile has more than one, + * `'head'` is given before `'first'`, and `'first'` before `'messages'`. + */ + cause: 'messages' | 'head' | 'first'; + /** With `'head'`: which digests changed. */ + changed?: Array; } /** A branch as a compile or receipt saw it: native id, name, creation time. */ diff --git a/src/types/index.ts b/src/types/index.ts index 542205eb..3966cc27 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -40,6 +40,7 @@ export type { JournalMark, RenderedLayout, LayoutUnit, + ThinkingStrip, } from './context.js'; // Strategy types @@ -83,6 +84,9 @@ export type { RequestOverhead, RenderStatsCapableStrategy, RenderedSummaryInfo, + ImageCeilings, + ThinkingBinding, + RequestIdentity, } from './strategy.js'; export { diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 45b643a8..b6b9b8c7 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -8,6 +8,8 @@ import type { PendingWork, } from './context.js'; import type { OverBudgetClearedBy } from '../adaptive/picker.js'; +import type { ContextObserver } from '../observer.js'; +import { DEFAULT_IMAGE_BYTE_CEILING, DEFAULT_IMAGE_CEILING } from '../live-image-limits.js'; /** * Read-only view of the message store for strategies. @@ -113,6 +115,12 @@ export interface StrategyContext { * CompressionWithdrawn). */ membrane?: Membrane; + /** + * The host's observer (`ContextManagerConfig.observer`), already isolated: + * a strategy reports its compression refusals and calibration samples to it + * and needs no guard of its own. Absent when the host gave none. + */ + observer?: ContextObserver; /** Current sequence number */ currentSequence: Sequence; /** @@ -137,7 +145,8 @@ export interface StrategyContext { * Result of readiness check. */ export interface ReadinessState { - /** Whether compile() can proceed immediately */ + /** Whether the strategy has no background work pending. compile() doesn't + * wait for that work either way: it compiles from what's there. */ ready: boolean; /** Promise that resolves when ready (if not ready) */ pendingWork?: Promise; @@ -160,15 +169,16 @@ export interface ContextStrategy { readonly maxMessageTokens?: number; /** - * What the next compile sends for each of `ids`, messages just stored and - * so among the newest of the window, in the order given. Each comes back + * What the next compile renders for each of `ids`, messages just stored + * and so among the newest of the window, in the order given, before the + * manager leaves out any thinking under `SelectOptions.thinkingBinding`. Each comes back * cut by the per-message cap, and as one message where the compile joins a * body's shards into one; the strategy makes every image decision it makes * there: its count, depth and byte limits, counted newest-first over the * view from the oldest of these messages to its newest, as a compile * counts them. It's for a caller that appends messages it has just stored * to a compiled request without recompiling, such as a turn's mid-turn - * injections, so that what it appends is what the next compile sends. A + * injections, so that what it appends is what the next compile renders. A * later compile can still strip an image of theirs as newer ones arrive. * * `store` is the view a compile reads. An id comes back as null when the @@ -183,8 +193,9 @@ export interface ContextStrategy { * * What comes back is made of the stored arrays and blocks, as a compile's * messages are, so a caller that edits what comes back copies it first. - * Tool blocks come back as the cap leaves them, since how a compile pairs - * and prunes them depends on the window around them. Rendering doesn't + * Tool blocks come back as the cap leaves them, with what sending prunes + * from them, since how a compile pairs them depends on the window around + * them. Rendering doesn't * change what a later compile sends. */ renderAsNewest?(store: MessageStoreView, ids: readonly MessageId[]): Array; @@ -245,8 +256,8 @@ export interface ContextStrategy { onCompressionHoldsReleased?(ctx: StrategyContext): void; /** - * Check if strategy is ready to compile. - * Returns pending work info if not ready. + * Whether the strategy has background work pending, with its info if so. + * compile() doesn't wait for it. */ checkReadiness(): ReadinessState; @@ -311,6 +322,72 @@ export interface ContextStrategy { * (for example 'budget-fit' or 'window'). Reading clears it. */ takeSelectionCause?(): string | undefined; + + /** + * The image ceilings this strategy keeps its live window within and holds + * a compression prompt to (`imageCeiling` and `imageByteCeiling`, 0 for + * none). They're configuration, not usage: they bound the images the + * window counts (see `AutobiographicalConfig.maxLiveImages`), not every + * image a request carries, and nothing here bounds what a caller injects + * at compile time or appends to a compiled request. A caller that wants + * its request within them counts that request's images itself, every + * shape included, and compares the counts with these. Absent on a strategy with no live-image + * window. + */ + imageCeilings?(): ImageCeilings; + + /** + * The context manager left these thinking blocks of stored messages out of + * the request the last committing select priced, after the select (see + * `SelectOptions.thinkingBinding`). A strategy that calibrates its + * estimates against the provider's count of that request takes their + * price off what it expects the count to be. Not called for a dry compile. + */ + noteThinkingLeftOut?(blocks: readonly ContentBlock[]): void; +} + +/** + * A strategy's image ceilings (see ContextStrategy.imageCeilings): the most + * images, and the most inline base64 image bytes, its live window keeps and + * a compression prompt carries. 0 is no ceiling. + */ +export interface ImageCeilings { + imageCeiling: number; + imageByteCeiling: number; +} + +/** + * Whether the model a compile is for binds each signed thinking block to the + * request it was minted in, so that a block sent after anything before it + * changed is refused, or, with the provider's drop control on, set aside. + * See `SelectOptions.thinkingBinding`. + */ +export interface ThinkingBinding { + /** The model binds thinking. Absent or false: the compile is as it would be + * without the option, and neither reads nor writes the branch's record. */ + binds?: boolean; + /** The request carries the provider's drop control, so a stale block is + * set aside rather than refused. Absent means false. */ + dropBlock?: boolean; + /** The participant whose messages are the model's own replies, such as + * the agent's name. Required with `binds`: another participant's reply + * never keeps its thinking. */ + participant?: string; +} + +/** + * Digests of what a request carries before its messages, as the host sends + * it, each opaque to the context manager and compared only for equality: + * the system text (system-position injections included), the tools in the + * order sent, the model, and the rest the provider sees ahead of the + * messages (such as the tool choice, the thinking config and the output + * limit). See `SelectOptions.identity`. + */ +export interface RequestIdentity { + system: string; + tools: string; + model: string; + request: string; } /** What a rendered summary covers, for layout reporting. */ @@ -351,7 +428,10 @@ export interface SelectOptions { * throw as for a real compile, leaving nothing behind; `previewContext` * reports an over-budget throw, as the plan's own preview when the plan was * already over the budget (the live compile refuses it at the plan check), - * and passes any other throw through. + * and passes any other throw through. The passthrough strategies throw + * none of their budget refusals on a dry select: it returns what they would + * send, over the budget or without the newest message, with no verdict, + * and they have no `previewContext` report. * * A compression hold ends at its deadline, whoever looks: the manager * releases an expired hold at any compile or hold query, a dry compile @@ -360,6 +440,84 @@ export interface SelectOptions { * found after that rebuild. */ dryRun?: boolean; + /** + * The model this compile is for binds signed thinking (`binds`), so the + * compile leaves out every thinking block whose request has changed since + * it was minted, and reports it in `CompileResult.thinkingStrip`. With + * `binds`, `participant` and `identity` are required, and a compile + * without them is refused before anything is selected. Without `binds`, + * the compile is as it would be without the option: it neither reads nor + * writes the branch's record. A record an earlier binding compile left is + * compared at the next binding compile like any other, so a message keeps + * its thinking only while everything up to it matches what that compile + * sent, a prefix the provider took with that thinking, and nothing past + * the record's end keeps it, unless a host noted it after that compile. + * A record the store can't read or write is logged, and never fails a + * compile or a note. One it can't read is read as none, so a committing + * compile starts a record in its place, as for a record this version + * doesn't read. When it can't write a compile's record, the record before + * it, if any, stands, and that compile's stream can't note onto it. + * + * The manager keeps a record per branch of what the branch last sent: the + * messages its last committing compile with `binds` sent (cache + * breakpoints aside, injections included), then the rows a host noted + * after them + * (`ContextManager.noteRequestSent`): what its stream's requests went on + * to carry and, once the stream ended, its last reply. With them it keeps + * the stored bodies the compile sent without their thinking, the + * compile's `identity`, and the branch's sequence where the record began. + * A compile compares what it renders with the record in order. From the + * first message that differs, or from the first message when an + * `identity` digest changed, every message goes without its thinking, and + * so does every message past the record's end: only the host knows which + * stored reply answered which request. A reply whose stored body went + * without its thinking goes without it again, since what came after it + * was minted without it. A message whose participant isn't + * `participant`, or that has no stored body (a summary's recall answer, + * an injection), never keeps thinking: nothing it carries was minted in + * this agent's requests. A manager's record serves one participant, so + * compile each namespace for one. + * + * A reply stored at or before the sequence where the branch's record began + * (`CompileResult.thinkingRecordedAfter`), which no record vouches for, + * keeps its thinking only with `dropBlock`, for the provider to judge. So + * a branch with no record, its own or (on a branch made since) the one it + * inherits as of its branch point, leaves out every reply's thinking + * without `dropBlock`; with `dropBlock` it leaves the stored replies' + * thinking as it is. Either way it starts the record. Each committing + * compile with `binds` writes the record; a dry compile reports what it + * would leave out and writes no record. The record survives a restart. + * + * So once a branch has a record, a reply keeps its thinking at the compile + * after it's stored only if the host has noted it: each later round of a + * stream as the host sends it, and the stream's last reply once it's + * stored. A host that passes `thinkingBinding` and notes nothing loses + * each reply's thinking at the compile after it's stored, and is never + * refused for it. A note names the compile its stream was built from, so + * a committing compile with `binds` while a stream is open makes that + * stream's later notes vouch for nothing, and its replies go without + * their thinking: while one of a branch's requests is streaming, compile + * for its branch with `binds` only with `dryRun`, which writes no + * record. A preview prices a + * summary's carriers by the option it's given (`previewContext`'s + * `thinkingBinding`), as a compile does. + * + * The comparison is of compile output. A host that turns compile output + * into a request (sanitizing it, adding rows) must do so the same way at + * every request, and must note a row only where the next request will + * carry it as the stream sent or got it: context-manager can't see what + * happens to a message after it leaves the compile. + */ + thinkingBinding?: ThinkingBinding; + /** + * Digests of what this compile's request carries before its messages, for + * a compile with `thinkingBinding.binds`, which requires them: when one + * differs from the branch's record, the compile leaves out every thinking + * block. `system` is the system text as sent, with the content of this + * compile's `'system'`-position injections, which the compile returns as + * `CompileResult.systemInjections`. + */ + identity?: RequestIdentity; /** Exact normalized tools/system/context-prefix identity supplied by the * host for kv-unified cache relevance. */ kvUnifiedImmutablePrefixHash?: string; @@ -370,6 +528,14 @@ export interface SelectOptions { multiplier: number; expiresAt: number; }; + /** The content of each injection the compile inserts among the selected + * entries after the select (`beforeUser` and `afterUser`), which the + * request carries and the select's window doesn't. A strategy whose + * calibration compares the provider's input total with the compile prices + * it there (AutobiographicalStrategy.reportRealInputTokens). + * ContextManager.compile fills it from its injections; `system` ones go + * into the caller's system prompt instead. */ + messageInjections?: ReadonlyArray; } /** @@ -667,10 +833,13 @@ export function isSummaryOverviewStrategy(s: ContextStrategy): s is SummaryOverv } /** - * What a request carried besides the compiled window, as the caller sent it - * (see `AutobiographicalStrategy.reportRealInputTokens`). The provider counts - * it in the input total and the compile estimate doesn't, so a calibration - * sample that leaves it out learns it as a per-token rate. + * What a request carried besides the compiled window and the compile's + * message-position injections (which the compile prices itself), as the + * caller sent it (see `AutobiographicalStrategy.reportRealInputTokens`). The + * provider counts it in the input total and the compile estimate doesn't, so + * a calibration sample that leaves it out learns it as a per-token rate. + * Passing it says this is everything the request sent besides those; a + * caller that can't see the request's head passes nothing. */ export interface RequestOverhead { /** The system prompt the request sent. */ @@ -731,11 +900,16 @@ export interface RenderStats { * that isn't provisional was learned under that epoch's pricing, or under * an older epoch's and kept, re-stamped, because nothing this store holds * was repriced since; either way it's reported under the epoch in force. + * The exception is the 1 loaded beside a record from a newer pricing + * epoch, or a record the store can't read: neither is used, and that 1 + * is neither learned nor provisional. * `provisional` is true while the multiplier is a guess, which the first * provider sample the calibration's band accepts replaces outright, rather * than stepping toward it (see * `AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH`): one kept from a - * record without a stamp, or the 1 a discard restarts from. That 1 stays + * record without a stamp, the 1 a discard restarts from, or the 1 a store + * with no calibration record, or none with a multiplier to use other than + * a newer epoch's, starts from. A discard's 1 stays * provisional through a later epoch's load, which re-stamps it whatever * that epoch repriced. * `reset` is present once a multiplier from an older or unknown epoch has @@ -1234,6 +1408,11 @@ export interface AutobiographicalConfig { * - `'live-strip'` — carriers are OMITTED from the live window only. Mint * and merge requests still recall them verbatim. * + * A select for a model that binds thinking (`SelectOptions.thinkingBinding`) + * renders and prices the live window as under `'live-strip'` whatever this + * says: a carrier was minted in its summarizer's request, so such a model + * never takes it there. Mint and merge requests are unchanged. + * * TWO SURFACES, ONE STORE. A captured carrier is the ARCHIVIST'S cognition: * the thinking of writing the memory, not the thinking of living the span it * describes. It is replayed on two surfaces that want different things from @@ -1310,13 +1489,20 @@ export interface AutobiographicalConfig { /** Maximum number of images kept "live" (sent as real image blocks) in the * compiled context. Counted newest-first across the whole window; images - * beyond this many are replaced with a text placeholder. 0 = unlimited. - * A hard ceiling for dense bursts that pack many images into the live window. + * beyond this many are replaced with a text placeholder. Kept within + * `imageCeiling`, so 0 means up to that ceiling. A window kept at the + * ceiling leaves no room under it for images a caller appends to the + * compiled request, so a caller that keeps its requests within the + * ceilings recompiles at each image it adds; at about 1,600 tokens an + * image, the token budget binds first unless the images are small. A + * fraction is floored, to at least 1. + * A hard limit for dense bursts that pack many images into the live window. * Every `image` block counts, whatever its source (base64, a `data:` URL * or a remote URL), and so does every generated image, top-level or * nested in a tool result (see inline-images.ts). An image inside an * opaque provider item, such as a Responses `image_generation_call` - * replayed from its raw form, doesn't. */ + * replayed from its raw form, doesn't. Every live-image limit and image + * ceiling below counts the same way. */ maxLiveImages?: number; /** Token depth from the newest message beyond which images are stripped to a * text placeholder, even though the surrounding text stays verbatim. Measured @@ -1340,14 +1526,14 @@ export interface AutobiographicalConfig { /** * Byte wall for live images (2026-07-12): keep inline images newest-first * only while their cumulative inline size (a base64 source, a `data:` URL's - * payload, a generated image's `data`; 0 for a remote URL) stays - * under this budget — on top - * of `maxLiveImages` / `imageStripDepthTokens`. It keeps the images it - * counts from pushing the compiled window past the API's total request size - * cap (413 request_too_large), so that membrane's oversize check isn't what - * edits them. An image inside an opaque provider item (see `maxLiveImages`) - * isn't counted, so it's outside this budget. - * Default 20MB (of base64; the API cap is 32MB total). + * payload, a generated image's `data`; 0 for a remote URL) stays under + * this budget — on top of `maxLiveImages` / `imageStripDepthTokens`. + * Default 20 MiB of base64. The window keeps at most `imageByteCeiling` + * whatever this says, and 0 here means up to that ceiling. It counts the + * images the window counts (see `maxLiveImages`), not the request's other + * bytes, so on its own it can't keep a request under membrane's byte cap: + * see `imageByteCeiling`. An image inside an opaque provider item (see + * `maxLiveImages`) isn't counted, so it's outside this budget. */ maxLiveImageBytes?: number; @@ -1389,10 +1575,82 @@ export interface AutobiographicalConfig { * Image byte budget for COMPRESSION prompts (summarizer). Tighter than the * live window's: the prompt also carries the head, the recall frontier and * the raw chunk. Newest-first; older images become loud placeholders. - * Default 12MB of base64. + * Default 12MB of base64. A compression prompt carries at most + * `imageCeiling` images and `imageByteCeiling` bytes whatever this + * says, and 0 here, or any value that isn't a positive finite number, means + * up to the byte ceiling. */ maxCompressionImageBytes?: number; + /** + * The most images a strategy's live window keeps, whatever `maxLiveImages` + * says, and the most images a compression prompt carries (2026-10-19). A + * `maxLiveImages` of 0 means up to this many. It counts images as + * `maxLiveImages` does: every image, nested ones and generated ones + * included, but not one inside an opaque provider item. + * + * It doesn't bound a whole request. Images inside opaque provider items + * aren't counted, and neither is anything a caller injects at compile time + * or appends to a compiled request; `imageCeilings()` reports this + * ceiling, so such a caller can count its request's images itself and + * compare. The provider refuses a + * request with more than 600 images and documents together (measured + * 2026-10-13, and again on 2026-10-19), and the default, 500, leaves room + * for what this doesn't count. 0 = no ceiling. A value that isn't 0 or a + * finite number above it takes the default, and the strategy's warning at + * initialize names it. A fraction is floored, to at least 1. Under a + * window limit of 0 or above this ceiling, the strategy says once that + * this ceiling holds it, at the first render a select returns where images + * are shown as placeholders and the limit this ceiling sets was over: at + * an image the render replaces, or, with `imageStripHysteresisRatio`, + * where it brought a trim on anywhere in the rule's replay of the history. + * The depth limit or a limit the host set may be stripping images there + * too. + * + * A window held at a ceiling strips its oldest images as new ones come, + * as under any limit, but from deep in the request, so the prompt cache is + * rewritten from there at every new image. The memory strategy's + * `imageStripHysteresisRatio` strips in batches instead, so the cache is + * rewritten once a batch, and while that option is off the line names it. + */ + imageCeiling?: number; + + /** + * The most inline image bytes of the images a strategy's live window + * keeps, whatever `maxLiveImageBytes` says, and the most of its images' + * bytes a compression prompt carries (2026-10-19). A `maxLiveImageBytes` + * of 0 means up to this many. It measures an image as `maxLiveImageBytes` + * does: a base64 source's length, a `data:` URL's payload, a generated + * image's `data`, and 0 for a remote URL. + * + * It doesn't bound a request's bytes: the messages' text and JSON, + * documents, the images the window doesn't count, and anything a caller + * injects or appends aren't counted. membrane's byte cap is 28 MiB of + * serialized messages unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, + * and the default, 24 MiB, leaves 4 MiB of that cap for what this doesn't + * count. Where membrane checks a request against its cap, it refuses one + * over it before sending it; an XML-mode continuation round isn't checked + * there, and of the provider adapters only Anthropic's checks again. A + * request that asks membrane to shed images, as this strategy's compression + * requests do, first has its oldest images replaced until it fits or none + * is left, and then only the Anthropic adapter's own check refuses it if + * it's still over (`src/live-image-limits.ts` says where each check runs). + * That's for a formatter whose messages keep membrane's `image` blocks: + * such a request built with membrane's `openai-responses` formatter + * carries `input_image` parts, which aren't shed, and it isn't checked + * either. This + * setting isn't read from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes + * membrane's cap sets this too, keeping room under the cap for what this + * doesn't count. + * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A + * value that isn't 0 or a finite number above it takes the default, and a + * fraction is floored, to at least 1. Under a window limit of 0 or above + * this ceiling, the strategy says so once, and a window held at it rewrites + * the prompt cache from deep in the request at every new image, both as + * for `imageCeiling`. + */ + imageByteCeiling?: number; + /** * Merge grouping: exclude candidates whose OWN source span exceeds the * level-scaled limit `base × mergeThreshold^(max(0, level − 3))`, where @@ -2097,6 +2355,16 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * how far the span folds instead, and where pins overlap at a message, one * bound holds it (see `level`), so a raw pin there might not keep it raw. * + * A message stored in parts (a long message that ingress splits under + * `adaptiveResolution`) counts as one: under adaptive resolution, a pin + * whose first or last id names one of its parts covers it from its first + * part to its last, less a part locked with `lockChunk` outside the span + * its ids name, where the lock holds. The selects, compression and the + * summarizer's prompt all read it so. A compile whose budget can't hold the + * raw pins beside the rest refuses; where the adaptive plan's middle is over + * the budget, the refusal names each pin keeping raw tokens in that middle + * (`OverBudgetDiagnostics.rawPins`). + * * - `kind: 'pin'` — generic protected range (any first/last span). * - `kind: 'document'` — typically a single message containing a body of * information the agent wants to keep; semantically identical to a @@ -2234,6 +2502,8 @@ Write naturally, as recollection of what you experienced.`, maxMessageTokens: 0, maxLiveImages: 6, imageStripDepthTokens: 30000, + imageCeiling: DEFAULT_IMAGE_CEILING, + imageByteCeiling: DEFAULT_IMAGE_BYTE_CEILING, positionedRecallPairs: true, recallHeaderTemplate: '[Recall {id}]', compressionRefusalCurveFallbacks: 3, diff --git a/src/well-formed-content.ts b/src/well-formed-content.ts index a27848ac..25cd7e67 100644 --- a/src/well-formed-content.ts +++ b/src/well-formed-content.ts @@ -14,11 +14,21 @@ * base64, or no media type, neither a declared one nor, for an image, one * its data's signature names. The store keeps a block of the content as a * blob, not media inside a `tool_result`, which it keeps as given. - * Nothing else is checked: not ids, names or signatures, which a provider may + * Nothing else is checked: not ids, a tool call's name or signatures, which a provider may * still refuse a block without, and nothing else of a media source, such as - * a URL image's URL, which the formatters and the providers read. A block of - * a type none of these names passes as it is, since membrane adds types: - * whether a provider can take one is the provider's to say. One type is the + * a URL image's URL, which the formatters and the providers read. A + * `system_change` (membrane's SystemChangeContent) carries its content in + * `removeTools` and `addSystem`, arrays of strings, and `defineTools`, an + * array of tool definitions, each where present. A definition has the + * fields membrane's ToolDefinition types and sends to the provider with every + * later request: a non-empty string `name`, a string `description` and an + * object `inputSchema` whose `type` is `'object'`, which the API requires. + * One stands alone in its message, at + * its top level: membrane sends such a message as a system message of its + * own, and refuses one that mixes it with other blocks. A write refuses one + * that doesn't, and a read leaves it out. A + * block of a type none of these names passes as it is, since membrane adds + * types: whether a provider can take one is the provider's to say. One type is the * store's own: a `blob_ref`, which the store writes in place of inline media. * A write can't give one, and a read takes one whose `ref` is an object with * a string `hash` and an `originalType` of image, document, audio or video, @@ -92,6 +102,105 @@ function kindOf(value: unknown): string { /** The block types whose `source` carries their media. */ const MEDIA_TYPES: ReadonlySet = new Set(['image', 'document', 'audio', 'video']); +/** Whether a value is a system change (membrane's `system_change` block, SystemChangeContent). */ +function isSystemChange(value: unknown): boolean { + return isRecord(value) && value.type === 'system_change'; +} + +function isString(value: unknown): value is string { + return typeof value === 'string'; +} + +/** + * Whether a value is a tool definition as membrane reads one (ToolDefinition): + * a non-empty string `name`, a string `description` and an object + * `inputSchema` whose `type` is `'object'`. Membrane sends each to the + * provider with every request from the change on, so one without them would + * be refused on every request: the API refuses a definition whose schema has + * no `type`, or another, with a 400. + */ +function isToolDefinition(value: unknown): boolean { + return isRecord(value) && typeof value.name === 'string' && value.name !== '' && + typeof value.description === 'string' && isRecord(value.inputSchema) && value.inputSchema.type === 'object'; +} + +function stringPlaces(value: unknown, at: string, places: string[], what: string): void { + if (typeof value !== 'string') places.push(`${at} is ${kindOf(value)}, not ${what}`); +} + +function toolDefinitionPlaces(value: unknown, at: string, places: string[]): void { + if (!isRecord(value)) { + places.push(`${at} is ${kindOf(value)}, not a tool definition (an object)`); + return; + } + if (value.name === '') places.push(`${at}.name is an empty string, not a tool name`); + else stringPlaces(value.name, `${at}.name`, places, 'a tool name (a string)'); + stringPlaces(value.description, `${at}.description`, places, 'a string'); + if (!isRecord(value.inputSchema)) places.push(`${at}.inputSchema is ${kindOf(value.inputSchema)}, not a JSON Schema (an object)`); + else if (value.inputSchema.type !== 'object') places.push(`${at}.inputSchema.type is ${kindOf(value.inputSchema.type)}, not 'object'`); +} + +/** + * Whether every element of the content is a system change. An index loop, so + * a hole counts as an element that isn't one, as every walk here names a hole + * and refuses it like any other. + */ +function standsAlone(content: readonly unknown[]): boolean { + for (let i = 0; i < content.length; i++) if (!isSystemChange(content[i])) return false; + return true; +} + +/** Whether any element of the content is a system change (a hole isn't one). */ +function holdsChange(content: readonly unknown[]): boolean { + for (let i = 0; i < content.length; i++) if (isSystemChange(content[i])) return true; + return false; +} + +/** Whether a value is absent, or an array whose every element passes `each`. */ +function absentOrArrayOf(value: unknown, each: (element: unknown) => boolean): boolean { + if (value === undefined) return true; + if (!Array.isArray(value)) return false; + for (let i = 0; i < value.length; i++) if (!each(value[i])) return false; + return true; +} + +/** + * The fields of a system change that carry its content, as membrane reads + * them when it sends the change: names to withdraw and system texts as arrays + * of strings, and definitions as an array of tool definitions + * (isToolDefinition), each where present. Each with the places an element + * that fails names. + */ +const SYSTEM_CHANGE_FIELDS: ReadonlyArray boolean, + what: string, + elementPlaces: (element: unknown, at: string, places: string[]) => void, +]> = [ + ['removeTools', isString, 'an array of tool names', (v, at, places) => stringPlaces(v, at, places, 'a tool name (a string)')], + ['defineTools', isToolDefinition, 'an array of tool definitions', toolDefinitionPlaces], + ['addSystem', isString, 'an array of strings', (v, at, places) => stringPlaces(v, at, places, 'a string')], +]; + +function systemChangeWellFormed(block: Record): boolean { + for (const [field, each] of SYSTEM_CHANGE_FIELDS) if (!absentOrArrayOf(block[field], each)) return false; + return true; +} + +function systemChangePlaces(block: Record, at: string, places: string[]): void { + for (const [field, each, what, elementPlaces] of SYSTEM_CHANGE_FIELDS) { + const value = block[field]; + if (value === undefined) continue; + if (!Array.isArray(value)) { + places.push(`${at}.${field} is ${kindOf(value)}, not ${what}`); + continue; + } + for (let i = 0; i < value.length; i++) { + if (!each(value[i])) elementPlaces(value[i], `${at}.${field}[${i}]`, places); + } + } +} + /** Whether a block matches where its readers read it, without allocating: each side's common case. */ function wellFormed(block: unknown, side: Side): boolean { if (!isRecord(block)) return false; @@ -108,9 +217,13 @@ function wellFormed(block: unknown, side: Side): boolean { const inner = block.content; if (typeof inner === 'string') return true; if (!Array.isArray(inner)) return false; - for (let i = 0; i < inner.length; i++) if (!wellFormed(inner[i], side)) return false; + for (let i = 0; i < inner.length; i++) { + if (!wellFormed(inner[i], side) || isSystemChange(inner[i])) return false; + } return true; } + case 'system_change': + return systemChangeWellFormed(block); case 'generated_image': return typeof block.data === 'string' && typeof block.mimeType === 'string'; case 'blob_ref': @@ -152,10 +265,13 @@ function storeRefuses(block: unknown): boolean { } function allWellFormed(content: readonly unknown[], side: Side): boolean { + let changes = 0; for (let i = 0; i < content.length; i++) { if (!wellFormed(content[i], side) || (side === 'write' && storeRefuses(content[i]))) return false; + if (isSystemChange(content[i])) changes++; } - return true; + // A message holding a system change holds nothing else. + return changes === 0 || changes === content.length; } /** @@ -231,9 +347,18 @@ function placesIn(block: unknown, at: string, places: string[], side: Side): voi places.push(`${at}.content is ${kindOf(inner)}, not a string or an array of content blocks`); return; } - for (let i = 0; i < inner.length; i++) placesIn(inner[i], `${at}.content[${i}]`, places, side); + for (let i = 0; i < inner.length; i++) { + if (isSystemChange(inner[i])) { + places.push(`${at}.content[${i}] is a system_change inside a tool_result; a system change goes in a message of its own`); + } else { + placesIn(inner[i], `${at}.content[${i}]`, places, side); + } + } return; } + case 'system_change': + systemChangePlaces(block, at, places); + return; case 'generated_image': if (typeof block.data !== 'string') places.push(`${at}.data is ${kindOf(block.data)}, not a string`); if (typeof block.mimeType !== 'string') places.push(`${at}.mimeType is ${kindOf(block.mimeType)}, not a string`); @@ -274,9 +399,13 @@ function storePlacesIn(block: unknown, at: string, places: string[]): void { function malformedPlaces(content: readonly unknown[], side: Side, at = 'content'): string[] { const places: string[] = []; + const mixed = !standsAlone(content) && holdsChange(content); for (let i = 0; i < content.length; i++) { placesIn(content[i], `${at}[${i}]`, places, side); if (side === 'write') storePlacesIn(content[i], `${at}[${i}]`, places); + if (mixed && isSystemChange(content[i])) { + places.push(`${at}[${i}] is a system_change beside other blocks; a system change goes in a message of its own`); + } } return places; } @@ -349,7 +478,10 @@ export function blockForWrite(block: ContentBlock, what: string): ContentBlock { * so each level is reconciled on its own, and a block mended inside a * `tool_result` gives up its form like one at the top. */ -function transformed(content: readonly unknown[], side: Side): ContentBlock[] { +function transformed(content: readonly unknown[], side: Side, nested = false): ContentBlock[] { + // A system change that doesn't stand alone in its message, at its top + // level, is left out by a read, and a write has refused it already. + const changesStand = !nested && standsAlone(content); const before: ContentBlock[] = []; const after: ContentBlock[] = []; for (let i = 0; i < content.length; i++) { @@ -358,7 +490,7 @@ function transformed(content: readonly unknown[], side: Side): ContentBlock[] { // read leaves it out, and a write has refused it already. if (!isRecord(block)) continue; before.push(block as unknown as ContentBlock); - const out = side === 'write' ? writeBlock(block) : readBlock(block); + const out = isSystemChange(block) && !changesStand ? undefined : side === 'write' ? writeBlock(block) : readBlock(block); if (out) after.push(out); } return releaseEditedRawForms(before, after); @@ -369,7 +501,7 @@ function writeBlock(block: Record): ContentBlock { if (wellFormed(block, 'write')) return block as unknown as ContentBlock; if (block.type === 'tool_use') return mendedToolUse(block)!; if (block.type === 'tool_result' && Array.isArray(block.content)) { - return { ...block, content: transformed(block.content, 'write') } as unknown as ContentBlock; + return { ...block, content: transformed(block.content, 'write', true) } as unknown as ContentBlock; } return block as unknown as ContentBlock; } @@ -386,10 +518,14 @@ function writeBlock(block: Record): ContentBlock { * out (undefined): a value that isn't an object, an array, a block without a * string `type`, a text block without its string, unsigned thinking without * its string, media without a source or the data and media type to make - * one, a generated image without its data or media type, and a `blob_ref` + * one, a generated image without its data or media type, a `blob_ref` * that doesn't reference a blob, whose `originalType` isn't image, document, * audio or video, or, for a document, audio or video, without a string - * `mediaType`. + * `mediaType`, and a `system_change` whose `removeTools`, `defineTools` or + * `addSystem` isn't the array its content needs (one element of + * `defineTools` that isn't a tool definition leaves the whole change out), + * or that doesn't stand alone in its message, at its top level + * (transformed). */ function readBlock(block: Record): ContentBlock | undefined { if (wellFormed(block, 'read')) return block as unknown as ContentBlock; @@ -406,9 +542,9 @@ function readBlock(block: Record): ContentBlock | undefined { case 'tool_result': { const inner = block.content; const content = Array.isArray(inner) - ? transformed(inner, 'read') + ? transformed(inner, 'read', true) : isRecord(inner) - ? transformed([inner], 'read') + ? transformed([inner], 'read', true) : typeof inner === 'number' ? String(inner) : []; diff --git a/test/calibration-band.test.ts b/test/calibration-band.test.ts index 0c108c8c..68bf401a 100644 --- a/test/calibration-band.test.ts +++ b/test/calibration-band.test.ts @@ -14,6 +14,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { Membrane, MockAdapter, NativeFormatter, type ContentBlock } from '@animalabs/membrane'; import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; import type { MessageStoreView, StoredMessage } from '../src/types/index.js'; const dir = mkdtempSync(join(tmpdir(), 'calibration-band-')); @@ -95,11 +96,16 @@ async function openManager(name: string, opts: { signedThinking?: boolean; stamp describe('estimator calibration band', () => { it('a multiplier pinned at the ceiling comes back down once the raw estimate is honest', async () => { + // A learned 1.8: a store with no record would be provisional, and set + // outright by the first sample (#328). + const first = await openManager('pinned'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1.8, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('pinned'); - await cm.compile(BUDGET); - internals._calibration = 1.8; - internals.applyCalibration(); await cm.compile(BUDGET); // arms one sample; estimate is in calibrated units (raw × 1.8) + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1.8, null], 'learned'); const est = internals._lastCompileEstimate; assert.ok(est > 0); // The provider bills exactly the raw estimate: real/est = 1/1.8 ≈ 0.56. @@ -120,8 +126,15 @@ describe('estimator calibration band', () => { }); it('an in-band ratio is learned from as before', async () => { + // A learned 1: a store with no record would be set outright (#328). + const first = await openManager('inband'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('inband'); await cm.compile(BUDGET); + assert.equal(internals._calibrationProvisional, null, 'learned'); const est = internals._lastCompileEstimate; strategy.reportRealInputTokens(est * 1.5); assert.ok(internals._calibration > 1.0 && internals._calibration < 1.5, `got ${internals._calibration}`); @@ -204,6 +217,23 @@ describe('estimator calibration band', () => { assert.deepEqual(warned.filter((l) => l.includes('[estimator-calibration]')), []); }); + it('a record from a newer pricing epoch with no multiplier this binary can read is left alone too', async () => { + const first = await openManager('newer-epoch-unreadable'); + await first.cm.compile(BUDGET); + const future = { multipliers: { default: 1.4 }, at: 123, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH + 1 }; + first.internals.store!.setStateJson(first.internals.calibrationStateId, future); + first.cm.close(); + const { cm, strategy, internals } = await openManager('newer-epoch-unreadable'); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); + // A first sample that only confirms the 1 writes nothing over it. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + const ema = (internals as unknown as { _calibrationEma: number | null })._calibrationEma; + assert.ok(ema !== null && Math.abs(ema - 1.002) < 0.001, `premise: the sample reached the EMA, got ${ema}`); + assert.deepStrictEqual(internals.store!.getStateJson(internals.calibrationStateId), future); + cm.close(); + }); + it('a multiplier learned under the current pricing epoch survives reopen', async () => { const first = await openManager('current-epoch'); await first.cm.compile(BUDGET); @@ -401,10 +431,93 @@ describe('estimator calibration band', () => { reopened.cm.close(); }); - it('a store with no calibration history reports its multiplier and no reset', async () => { + it('a store with no calibration history reports a provisional 1 and no reset', async () => { const { cm } = await openManager('no-history'); await cm.compile(BUDGET); - assert.deepEqual(cm.getRenderStats()!.calibration, { multiplier: 1, pricingEpoch: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + assert.deepStrictEqual(cm.getRenderStats()!.calibration, + { multiplier: 1, pricingEpoch: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, provisional: true }); + cm.close(); + }); + + it('a store with no calibration record loads a provisional 1: the first sample the band accepts sets it, and stamps the record', async () => { + // The store with the least evidence: learned, its 1 would step a fifth of + // the way per sample, nine applied changes to reach a true 1.4 (#328). + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + const { cm, strategy, internals } = await openManager('fresh-provisional'); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }]); + assert.equal(internals.store!.getStateJson(internals.calibrationStateId), null, 'the load writes nothing'); + // A sample the band rejects is no evidence: the 1 stays provisional. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 3); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }]); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.4); + assert.ok(Math.abs(internals._calibration - 1.4) < 1e-9, `set by the sample, got ${internals._calibration}`); + assert.equal(internals._calibrationProvisional, null); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as + { multiplier?: number; pricing?: number; provisional?: boolean; reset?: unknown }; + assert.deepStrictEqual([saved.multiplier, saved.pricing, saved.provisional, saved.reset], [1.4, epoch, undefined, undefined]); + // From here the EMA steps as it always did. + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.2); // implies 1.68 + assert.ok(internals._calibration > 1.4 && internals._calibration < 1.6, `stepped, not replaced: ${internals._calibration}`); + cm.close(); + }); + + it("a store with no calibration record whose first sample falls inside the dead band confirms its 1, and stamps the record", async () => { + const { cm, strategy, internals } = await openManager('fresh-confirm'); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; pricing?: number; provisional?: boolean }; + assert.deepStrictEqual([saved.multiplier, saved.pricing, saved.provisional], [1, AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, undefined]); + cm.close(); + }); + + it('a store with no calibration record is still provisional after a restart before its first sample', async () => { + const first = await openManager('fresh-restart'); + await first.cm.compile(BUDGET); + first.cm.close(); + const { cm, strategy, internals } = await openManager('fresh-restart'); + await cm.compile(BUDGET); + assert.deepStrictEqual(internals._calibrationProvisional, { fromEpoch: null }); + assert.equal(internals.store!.getStateJson(internals.calibrationStateId), null); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 0.8); + assert.ok(Math.abs(internals._calibration - 0.8) < 1e-9, `set by the sample, got ${internals._calibration}`); + cm.close(); + }); + + it('a record with no multiplier to use loads as no record: a provisional 1', async () => { + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + for (const [i, record] of ([{ at: 1, pricing: epoch }, { multiplier: '1.4', at: 1, pricing: epoch }, { multiplier: null, at: 1, pricing: epoch }, 7] as unknown[]).entries()) { + const name = `fresh-unusable-${i}`; + const first = await openManager(name); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, record); + first.cm.close(); + const { cm, internals } = await openManager(name); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }], JSON.stringify(record)); + cm.close(); + } + }); + + it('a calibration read that throws loads a learned 1, as before: the record it could not read may hold one worth keeping', async () => { + const { cm, internals } = await openManager('fresh-unreadable'); + const store = internals.store as unknown as { getStateJson(id: string): unknown }; + const read = store.getStateJson; + let refused = 0; + store.getStateJson = (id: string) => { + if (id === internals.calibrationStateId) { refused++; throw new Error('unreadable'); } + return read.call(store, id); + }; + try { + await cm.compile(BUDGET); + } finally { + store.getStateJson = read; + } + assert.ok(refused > 0, 'premise: the load read the record, and the read threw'); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); cm.close(); }); @@ -636,6 +749,37 @@ describe('estimator calibration band', () => { belowReopened.cm.close(); }); + it('dropping a provisional multiplier to 1 drops its declared mark in memory as on disk', async () => { + // An operator following the unwedge advice may seed provisional over a + // record a declared caller had marked. + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + const tight = await tightBudget('drop-declared-probe', 1.2); // fits at 1.0, not at 1.45 + const first = await openManager('drop-declared'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1.45, at: 1, pricing: epoch, provisional: true, declaredOverhead: true }); + first.cm.close(); + const { cm, strategy, internals } = await openManager('drop-declared'); + const declared = () => (internals as unknown as { _calibrationDeclared: boolean })._calibrationDeclared; + await cm.compile(BUDGET); // loads the seed, which fits this budget + assert.deepStrictEqual([internals._calibration, declared()], [1.45, true], 'premise: loaded, marked'); + // A dry compile drops it too, in the dry compile only: select() puts the mark back. + const dry = await calibrationWarnings(() => cm.compile(tight, undefined, { dryRun: true })); + assert.equal(dry.result.renderStats!.calibration!.multiplier, 1, 'premise: the dry compile dropped it'); + assert.deepStrictEqual([internals._calibration, declared()], [1.45, true], 'put back after the dry compile that dropped it'); + await calibrationWarnings(() => cm.compile(tight)); + assert.equal(internals._calibration, 1); + assert.equal(cm.getRenderStats()!.calibration!.reset?.discardedMultiplier, 1.45, 'premise: the seeded 1.45 was loaded and dropped'); + assert.equal(declared(), false); + const dropped = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; provisional?: boolean; declaredOverhead?: boolean }; + assert.deepStrictEqual([dropped.multiplier, dropped.provisional, dropped.declaredOverhead], [1, true, undefined], 'the record the drop writes'); + // An undeclared sample confirming the 1 writes no mark either. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; declaredOverhead?: boolean }; + assert.deepStrictEqual([saved.multiplier, saved.declaredOverhead], [1, undefined]); + cm.close(); + }); + it('a discard\'s provisional 1 survives a restart until a sample confirms it', async () => { // Both discard writers (the load's and the drop's) persist the 1 as // provisional; a reopen before the first sample must not read it as @@ -739,7 +883,11 @@ describe('estimator calibration band', () => { control.cm.close(); const controlReopened = await openSharded('picker-control', false); assert.equal(controlReopened.cm.previewContext(tight)!.fits, false); - await assert.rejects(controlReopened.cm.compile(tight), /Adaptive picker exhausted/); + await assert.rejects(controlReopened.cm.compile(tight), (err: unknown) => { + assert.match(String((err as Error).message), /Adaptive picker exhausted/); + assertStage(err, 'picker-exhausted'); + return true; + }); controlReopened.cm.close(); const first = await openSharded('picker', true); @@ -1165,9 +1313,15 @@ describe('estimator calibration band', () => { }); it('a drift inside 2% is not applied; once the EMA passes 2% it is', async () => { + // A learned 1: a store with no record would be set by its first sample (#328). + const first = await openManager('deadband'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('deadband'); await cm.compile(BUDGET); - assert.equal(internals._calibration, 1); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null], 'learned'); // real/est = 1.01 every compile: the EMA creeps toward 1.01 and never // leaves the band, so nothing is repriced between compiles. for (let i = 0; i < 12; i++) { diff --git a/test/calibration-overhead.test.ts b/test/calibration-overhead.test.ts index 9bd493ca..9841942f 100644 --- a/test/calibration-overhead.test.ts +++ b/test/calibration-overhead.test.ts @@ -16,6 +16,8 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { Membrane, MockAdapter, NativeFormatter, type ContentBlock, type ToolDefinition } from '@animalabs/membrane'; +import type { ContextInjection } from '../src/types/index.js'; +import type { RequestOverhead } from '../src/index.js'; import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; import type { StoredMessage } from '../src/types/index.js'; @@ -82,8 +84,11 @@ describe('estimator calibration and request overhead (#323)', () => { const overhead = overheadPrice(cm, SYSTEM, tools); assert.ok(overhead > window * 0.4 && overhead < window * 0.7, `overhead ${overhead} vs window ${window}`); // Undeclared, this sample reads real/est ≈ 1.5: in band, so it would be - // learned as a per-token rate 50% above the truth. + // learned as a per-token rate 50% above the truth. The first sample sets + // a store with no record; the second steps it. strategy.reportRealInputTokens(window + overhead, { system: SYSTEM, tools }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overhead, { system: SYSTEM, tools }); assert.equal(internals._calibration, 1); cm.close(); }); @@ -194,3 +199,468 @@ describe('estimator calibration and request overhead (#323)', () => { cm.close(); }); }); + +/** The store's price of some content, at its current multiplier. */ +function priceOf(cm: ContextManager, content: ContentBlock[]): number { + return (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore.estimateTokens({ content } as unknown as StoredMessage); +} + +/** An injection at a position, of about `tokens` estimated tokens. */ +function injectionOf(position: 'beforeUser' | 'afterUser' | 'system', tokens: number): ContextInjection { + return { namespace: 'probe', position, content: [{ type: 'text', text: 'Retrieved knowledge. '.repeat(Math.ceil(tokens / 4)) }] }; +} + +type DeclaredInternals = Internals & { _calibrationDeclared: boolean; _calibrationArmed: boolean }; +const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + +async function seeded(name: string, record: Record) { + const first = await openManager(name); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, record); + first.cm.close(); + const opened = await openManager(name); + await opened.cm.compile(BUDGET); + return { ...opened, internals: opened.internals as DeclaredInternals }; +} + +/** Compile, then one sample at implied multiplier k, with system and tools declared or not. */ +async function sampleAt(cm: ContextManager, strategy: AutobiographicalStrategy, internals: Internals, k: number, declared: boolean): Promise { + await cm.compile(BUDGET); + const m = internals._calibration; + const tools = toolsOf(internals._lastCompileEstimate / m / 2); + const rawWindow = internals._lastCompileEstimate / m; + const rawOverhead = overheadPrice(cm, SYSTEM, tools) / m; + strategy.reportRealInputTokens(k * (rawWindow + (declared ? rawOverhead : 0)), declared ? { system: SYSTEM, tools } : undefined); +} + +const declaredOf = (internals: Internals) => + (internals.store!.getStateJson(internals.calibrationStateId) as { declaredOverhead?: boolean } | null)?.declaredOverhead; + +describe('estimator calibration: what the compile adds among the messages', () => { + /** Two compiles with the injection, each sampled with system, tools and the injection in the provider's count. */ + async function twoSamples(name: string, injection: (window: number) => ContextInjection) { + const { cm, strategy, internals } = await openManager(name); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const inj = injection(window); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, [inj]); + assert.equal(internals._lastCompileEstimate, window, 'the window estimate leaves the injection out'); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, inj.content), { system: SYSTEM, tools }); + } + return { cm, internals, window, inj }; + } + + it('a message-position injection is priced into the sample: in band, it is not learned as a per-token rate', async () => { + const { cm, internals, window, inj } = await twoSamples('injected-in-band', (w) => injectionOf('afterUser', w / 2)); + // Premise: priced as the window and the declared head alone, these samples read about 1.33. + assert.ok(priceOf(cm, inj.content) > window * 0.4, `injection ${priceOf(cm, inj.content)} vs window ${window}`); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('a large injection on a small window is not learned at the clamp', async () => { + const { cm, strategy, internals } = await openManager('injected-out-of-band'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('beforeUser', window * 3); + const tools = toolsOf(window / 2); + for (let i = 0; i < 4; i++) { + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken in band'); + cm.close(); + }); + + it('a dry compile between the compile and its sample leaves the compile\'s injection priced', async () => { + const { cm, strategy, internals } = await openManager('injected-dry'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, [injection]); + await cm.compile(BUDGET, undefined, { dryRun: true }); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('a compile without injections prices none, after one with', async () => { + const { cm, strategy, internals } = await openManager('injected-then-none'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + const tools = toolsOf(window / 2); + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('a system-position injection is priced once, as part of the declared system prompt', async () => { + const { cm, strategy, internals } = await openManager('injected-system'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('system', window / 2); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + const result = await cm.compile(BUDGET, [injection]); + // The caller appends systemInjections to its system prompt, and declares that. + const system = `${SYSTEM}\n${result.systemInjections.map((b) => (b as { text: string }).text).join('\n')}`; + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, tools), { system, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('a dry compile with injections leaves the committing compile\'s count', async () => { + const { cm, strategy, internals } = await openManager('dry-with-injections'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET); + await cm.compile(BUDGET, [injectionOf('afterUser', window / 2)], { dryRun: true }); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('several message-position injections are each priced', async () => { + const { cm, strategy, internals } = await openManager('two-injections'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injections = [injectionOf('beforeUser', window / 4), injectionOf('afterUser', window / 4)]; + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, injections); + const injected = injections.reduce((sum, inj) => sum + priceOf(cm, inj.content), 0); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + injected, { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); + + it('an injection is priced at the multiplier in force, as the window is', async () => { + const { cm, strategy, internals } = await seeded('injected-calibrated', { multiplier: 1.5, at: 1, pricing: epoch, declaredOverhead: true }); + assert.equal(internals._calibration, 1.5); + const tools = toolsOf(internals._lastCompileEstimate / 2); + const injection = injectionOf('afterUser', internals._lastCompileEstimate / 2); + await cm.compile(BUDGET, [injection]); + // A provider at k = 1.5: the window, the head and the injection, each at the store's price. + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + assert.equal(internals._calibration, 1.5); + assert.equal((internals as unknown as { _calibrationEma: number | null })._calibrationEma, 1.5, 'the sample was taken'); + cm.close(); + }); + + it('an undeclared sample counts the injections too', async () => { + const { cm, strategy, internals } = await openManager('injected-undeclared'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(window + priceOf(cm, injection.content)); + assert.equal(internals._calibration, 1); + assert.equal((internals as unknown as { _calibrationEma: number | null })._calibrationEma, 1, 'the sample was taken'); + cm.close(); + }); + + it('a system prompt alone is priced into the sample', async () => { + const { cm, strategy, internals } = await openManager('system-only'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const system = `You are a resident. ${'Keep to your purpose. '.repeat(Math.ceil(window / 14))}`; + // Premise: left out, the samples read 1.25 to 1.7, inside the band, where they'd be learned. + const share = overheadPrice(cm, system, []) / window; + assert.ok(share > 0.25 && share < 0.7, `the system prompt is ${share} of the window`); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, []), { system }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken'); + cm.close(); + }); +}); + +describe('estimator calibration: consistent out-of-band runs', () => { + /** Compile, then one declared sample at `ratio` of the compile's estimate. */ + async function oob(cm: ContextManager, strategy: AutobiographicalStrategy, internals: Internals, ratio: number, declared = true) { + await cm.compile(BUDGET); + const nothingBesides: RequestOverhead = {}; // the package's type, from its root + strategy.reportRealInputTokens(internals._lastCompileEstimate * ratio, declared ? nothingBesides : undefined); + } + + it('a consistent run on a learned record without the flag is learned at the clamp, not held', async () => { + // The upgrade case: a learned multiplier no declared sample has confirmed. + // A store with no record is provisional, which skips the hold anyway. + const { cm, strategy, internals } = await seeded('run-on-learned', { multiplier: 1, at: 1, pricing: epoch }); + assert.equal(internals._calibrationProvisional, null, 'learned'); + for (let i = 0; i < 3; i++) await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + + it('an undeclared out-of-band sample ends a declared run', async () => { + const { cm, strategy, internals } = await openManager('undeclared-ends-run'); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1, false); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8, 'a fresh run of three is learned'); + cm.close(); + }); + + it('a run agrees with its first sample: within 10% it is learned, and a drift past 10% of the first is not', async () => { + const drift = await openManager('run-drift'); + for (const ratio of [2.0, 2.18, 2.38]) await oob(drift.cm, drift.strategy, drift.internals, ratio); + assert.equal(drift.internals._calibration, 1, 'each within 10% of the one before, the third 19% past the first'); + drift.cm.close(); + const near = await openManager('run-near'); + for (const ratio of [2.0, 2.18, 2.08]) await oob(near.cm, near.strategy, near.internals, ratio); + assert.equal(near.internals._calibration, 1.8, 'each within 10% of the first'); + near.cm.close(); + }); + + it('a consistent run below the band is learned at the floor', async () => { + const { cm, strategy, internals } = await openManager('run-low'); + for (let i = 0; i < 3; i++) await oob(cm, strategy, internals, 0.3); + assert.equal(internals._calibration, 0.6); + cm.close(); + }); + + it('one compile counts once toward a run, however many reports follow it', async () => { + const { cm, strategy, internals } = await openManager('run-one-arm'); + await cm.compile(BUDGET); + for (let i = 0; i < 3; i++) strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, {}); + assert.equal(internals._calibration, 1, 'three reports of one compile'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1, 'the second compile is the run\'s second sample'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8, 'the third compile is its third'); + cm.close(); + }); + + it('a branch switch starts a run afresh', async () => { + const { cm, strategy, internals } = await openManager('run-branch'); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + await cm.fork('other'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1); + assert.equal((internals as unknown as { _outOfBandRun: { count: number } | null })._outOfBandRun?.count, 1, 'the sample on the new branch starts a run'); + cm.close(); + }); +}); + +describe('estimator calibration: a store that learned the overhead as a rate', () => { + it('two declared samples that agree replace a multiplier learned without them, by their mean', async () => { + const { cm, strategy, internals } = await seeded('learned-undeclared', { multiplier: 1.8, at: 1, pricing: epoch }); + assert.equal(internals._calibration, 1.8); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'the first is held'); + assert.equal(declaredOf(internals), undefined); + await sampleAt(cm, strategy, internals, 1.04, true); + assert.ok(Math.abs(internals._calibration - 1.02) <= 0.01, `replaced by the mean, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true, 'and the record says its samples declare their overhead'); + cm.close(); + }); + + it('two declared samples that disagree step by the EMA, and mark the record declared', async () => { + const { cm, strategy, internals } = await seeded('learned-disagree', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1.5, true); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `stepped toward the second, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.592) <= 0.01, `then steps by the EMA, got ${internals._calibration}`); + cm.close(); + }); + + it('a branch switch between the two loses the held sample', async () => { + const { cm, strategy, internals } = await seeded('held-branch', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await cm.fork('other'); + await cm.compile(BUDGET); + assert.equal(internals._calibration, 1.8, 'the fork has the record'); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'held, not paired with the other branch\'s sample'); + assert.notEqual((internals as unknown as { _declaredCandidate: number | null })._declaredCandidate, null, 'and held afresh'); + cm.close(); + }); + + it('an undeclared sample between the two leaves the held sample, and the pair still replaces', async () => { + const { cm, strategy, internals } = await seeded('held-undeclared', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'the first is held'); + await sampleAt(cm, strategy, internals, 1, false); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `the undeclared sample steps, got ${internals._calibration}`); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.01, `paired with the held sample and replaced, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + + it('a consistent run between the two ends the held sample', async () => { + const { cm, strategy, internals } = await seeded('held-run', { multiplier: 1, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + for (let i = 0; i < 3; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, {}); + } + assert.equal(internals._calibration, 1.8, 'the run is learned'); + await sampleAt(cm, strategy, internals, 1.5, false); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `an undeclared sample steps and unmarks, got ${internals._calibration}`); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `held, not paired with the sample from before the run, got ${internals._calibration}`); + cm.close(); + }); + + it('a restart between the two loses the held sample: it starts over, and the record is not marked', async () => { + const first = await seeded('held-restart', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + first.cm.close(); + const { cm, strategy, internals } = await openManager('held-restart'); + await cm.compile(BUDGET); + const typed = internals as DeclaredInternals; + assert.deepEqual([internals._calibration, typed._calibrationDeclared, declaredOf(internals)], [1.8, false, undefined]); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'held again'); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.02, `then replaced, got ${internals._calibration}`); + cm.close(); + }); + + it('a multiplier learned from declared samples steps by the EMA', async () => { + const { cm, strategy, internals } = await seeded('learned-declared', { multiplier: 1.8, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('after the replacement, a restart steps by the EMA', async () => { + const first = await seeded('declared-restart', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + first.cm.close(); + const { cm, strategy, internals } = await openManager('declared-restart'); + await cm.compile(BUDGET); + await sampleAt(cm, strategy, internals, 1.3, true); + assert.ok(Math.abs(internals._calibration - 1.06) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('an undeclared sample that moves the multiplier unmarks the record, and the next two declared replace it', async () => { + const { cm, strategy, internals } = await seeded('undeclared-after-declared', { multiplier: 1, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(cm, strategy, internals, 1.5, false); + assert.ok(internals._calibration > 1.05, `the undeclared sample stepped, got ${internals._calibration}`); + assert.equal(declaredOf(internals), undefined); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.02, `replaced, got ${internals._calibration}`); + cm.close(); + }); + + it('two declared samples that confirm the multiplier mark the record declared', async () => { + const { cm, strategy, internals } = await seeded('confirmed-declared', { multiplier: 1, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true); + await sampleAt(cm, strategy, internals, 1.5, true); + assert.ok(Math.abs(internals._calibration - 1.1) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('a store with no record is set by its first declared sample, with nothing held, and marked declared', async () => { + const { cm, strategy, internals } = await openManager('fresh-declared'); + await cm.compile(BUDGET); + assert.deepEqual(internals._calibrationProvisional, { fromEpoch: null }); + await sampleAt(cm, strategy, internals as DeclaredInternals, 1.4, true); + assert.ok(Math.abs(internals._calibration - 1.4) <= 0.01, `set by one sample, got ${internals._calibration}`); + assert.equal((internals as unknown as { _declaredCandidate: number | null })._declaredCandidate, null, 'nothing held'); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + + it('a provisional multiplier is replaced by one declared sample, and marked declared', async () => { + const { cm, strategy, internals } = await seeded('provisional-declared', { multiplier: 1, at: 1, pricing: epoch, provisional: true }); + await sampleAt(cm, strategy, internals, 1.4, true); + assert.ok(Math.abs(internals._calibration - 1.4) <= 0.01, `replaced, not stepped, got ${internals._calibration}`); + assert.equal(internals._calibrationProvisional, null); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + + it('a dry compile after the load keeps a record\'s declared overhead', async () => { + const { cm, strategy, internals } = await seeded('declared-dry', { multiplier: 1.8, at: 1, pricing: epoch, declaredOverhead: true }); + await cm.compile(BUDGET, undefined, { dryRun: true }); + assert.equal(internals._calibrationDeclared, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `stepped, not held, got ${internals._calibration}`); + cm.close(); + }); + + it('an undeclared sample inside the dead band leaves a record marked, and a rejected declared one leaves it unmarked', async () => { + const marked = await seeded('deadband-undeclared', { multiplier: 1, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(marked.cm, marked.strategy, marked.internals, 1.01, false); + const ema = (marked.internals as unknown as { _calibrationEma: number | null })._calibrationEma; + assert.ok(ema !== null && Math.abs(ema - 1.002) < 0.001, `premise: the sample reached the EMA, got ${ema}`); + assert.deepStrictEqual([marked.internals._calibration, marked.internals._calibrationDeclared, declaredOf(marked.internals)], [1, true, true]); + marked.cm.close(); + const unmarked = await seeded('rejected-declared', { multiplier: 1, at: 1, pricing: epoch }); + await unmarked.cm.compile(BUDGET); + unmarked.strategy.reportRealInputTokens(unmarked.internals._lastCompileEstimate * 2.1, {}); + assert.equal((unmarked.internals as unknown as { _outOfBandRun: { count: number } | null })._outOfBandRun?.count, 1, 'premise: the sample was rejected as the start of a run'); + assert.deepStrictEqual([unmarked.internals._calibration, unmarked.internals._calibrationDeclared, declaredOf(unmarked.internals)], [1, false, undefined]); + unmarked.cm.close(); + }); + + it('any overhead a JS caller passes but null or undefined counts as declared, in the run as in the mark', async () => { + const { cm, strategy, internals } = await seeded('falsy-overhead', { multiplier: 1, at: 1, pricing: epoch }); + for (let i = 0; i < 3; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, 0 as unknown as RequestOverhead); + } + assert.equal(internals._calibration, 1.8, 'learned as a declared run'); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + + it('a null overhead from a JS caller counts as undeclared', async () => { + const { cm, strategy, internals } = await seeded('null-overhead', { multiplier: 1, at: 1, pricing: epoch }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.5, null as unknown as RequestOverhead); + assert.ok(Math.abs(internals._calibration - 1.1) <= 0.01, `stepped as undeclared, not held, got ${internals._calibration}`); + assert.deepEqual([internals._calibrationDeclared, declaredOf(internals)], [false, undefined]); + cm.close(); + }); + + it('a re-stamp keeps a record\'s declared overhead', async () => { + const { cm, internals } = await seeded('restamp-declared', { multiplier: 1.2, at: 1, pricing: epoch - 1, declaredOverhead: true }); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { pricing?: number; declaredOverhead?: boolean }; + assert.equal(saved.pricing, epoch, 're-stamped'); + assert.equal(saved.declaredOverhead, true); + assert.equal(internals._calibrationDeclared, true); + cm.close(); + }); +}); diff --git a/test/compression-head-as-of.test.ts b/test/compression-head-as-of.test.ts new file mode 100644 index 00000000..90dedf96 --- /dev/null +++ b/test/compression-head-as-of.test.ts @@ -0,0 +1,766 @@ +/** + * The head a compression or merge request gives as context is the head in force when the history it summarizes + * happened: before the live head's start, the head starting at the latest topic transition at or before the + * request's first message, or the opening; from the live head's start on, the live head; cut at that message + * (#317). After resetHeadWindow, a request for history from before the reset gave the new topic's opening ahead of + * it, where the code places the chronicle's opening and keeps later events out. + */ + +import { describe, it, after } from 'node:test'; +import assert from 'node:assert'; +import { rmSync, existsSync } from 'node:fs'; +import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; +import type { Chunk } from '../src/strategies/autobiographical.js'; +import type { SummaryEntry } from '../src/types/index.js'; +import type { StrategyContext } from '../src/types/strategy.js'; +import type { ContentBlock } from '@animalabs/membrane'; + +const ROOT = './test-compression-head-as-of'; +let counter = 0; +const freshPath = (): string => `${ROOT}-${counter++}`; + +class ProbeStrategy extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { + this.pushSummary(entry); + } + runCompress(chunk: Chunk, ctx: StrategyContext): Promise { + return this.compressChunkHierarchical(chunk, ctx); + } + runMerge(targetLevel: number, sourceIds: string[], ctx: StrategyContext): Promise { + return this.executeMerge(targetLevel as never, sourceIds, ctx); + } + /** One reader of heads for many positions, as the count of queued work holds one. */ + heads(ctx: StrategyContext): (pos: number) => { start: number; end: number } { + return (this as unknown as { headsAsOf(store: unknown): (pos: number) => { start: number; end: number } }) + .headsAsOf(ctx.messageStore); + } + /** The head rule worked out for one position alone, scanning back from it, as headAsOf computed it per call + * before heads were read once per count: the oracle the shared reader is checked against. */ + headAlone(ctx: StrategyContext, pos: number): { start: number; end: number } { + const store = ctx.messageStore; + const liveStart = this.getHeadWindowStartIndex(store); + const hold = this.holdBoundary(store); + if (pos < 0) return { start: liveStart, end: liveStart }; + if (pos >= liveStart) return { start: liveStart, end: Math.min(this.getHeadWindowEnd(store), hold, pos) }; + const messages = store.getAll(); + let start = 0; + for (let i = pos; i > 0; i--) { + if (this.isTopicTransitionMessage(messages[i]!)) { + start = i; + break; + } + } + return { start, end: Math.min(this.headWindowEndFrom(store, messages, start), hold, pos) }; + } + /** Timeline scans of holdBoundary so far. */ + holdScans(): number { + return this.holdBoundaryScans; + } + /** The head a request gives for history from `pos` on. */ + asOf(ctx: StrategyContext, pos: number): { start: number; end: number } { + return (this as unknown as { headAsOf(store: unknown, pos: number): { start: number; end: number } }) + .headAsOf(ctx.messageStore, pos); + } + /** Whether the count of queued work reads tick's request for `chunk` as carrying a tool block. */ + carriesToolBlocks(ctx: StrategyContext, chunk: Chunk): boolean { + return this.toolBlockReader(ctx)(chunk); + } + /** One reader of the count's, as one count holds it across the queue (its answers memoized). */ + toolBlockReader(ctx: StrategyContext): (chunk: Chunk) => boolean { + return (this as unknown as { requestCarriesToolBlocks(store: unknown): (c: Chunk) => boolean }) + .requestCarriesToolBlocks(ctx.messageStore); + } + /** Sets the head window to `count` of this store's equal-sized messages, with half a message to spare. */ + headOfMessages(ctx: StrategyContext, count: number): void { + const messages = ctx.messageStore.getAll(); + let sum = 0; + for (let i = 0; i < count; i++) sum += ctx.messageStore.estimateTokens(messages[i]!); + this.config.headWindowTokens = sum + Math.floor(ctx.messageStore.estimateTokens(messages[count]!) / 2); + } + headWindowTokens(): number { + return this.config.headWindowTokens; + } +} + +/** Lets every merge through the topology check, so a merge whose source messages a host removed reaches its + * request and executeMerge's own start rule is what these tests pin. The real check may refuse such a merge (a + * merge whose first source message was removed returned false in a review's probe): these tests don't show that + * production reaches it. */ +class PlacingTopologyStrategy extends ProbeStrategy { + protected override mergeTopologyProblem(): string | null { + return null; + } +} + +const context = (manager: ContextManager): StrategyContext => + (manager as unknown as { createStrategyContext(): StrategyContext }).createStrategyContext(); + +const text = (t: string): ContentBlock[] => [{ type: 'text', text: t }]; +/** Equal-sized messages, so a head of `count` messages is the same count whichever message it starts at. */ +const body = (marker: string): string => `${marker} ${'word '.repeat(18)}`.slice(0, 110); + +function countIn(req: unknown, needle: string): number { + const blob = JSON.stringify(req); + let n = 0; + for (let at = blob.indexOf(needle); at >= 0; at = blob.indexOf(needle, at + needle.length)) n++; + return n; +} + +function summary(id: string, sourceIds: string[]): SummaryEntry { + return { + id, level: 1, content: `authored ${id}`, tokens: 20, sourceLevel: 0, sourceIds, + sourceRange: { first: sourceIds[0]!, last: sourceIds[sourceIds.length - 1]! }, + created: Number(id.replace(/\D/g, '')) || 1, + }; +} + +const chunkOf = (ctx: StrategyContext, from: number, to: number): Chunk => ({ + index: 900 + from, startIndex: from, endIndex: to, messages: ctx.messageStore.getAll().slice(from, to), tokens: 100, compressed: false, +}); + +/** + * A store with the opening OPEN0..OPEN9, a topic transition T1 and ONE0..ONE9, and a second transition T2 and + * TWO0..TWO5, every message the same size, the head window three messages. `resets` says how many of the two + * transitions the strategy has been reset to. + */ +async function build(resets: 0 | 1 | 2, opts: { path?: string; openingTool?: boolean; placing?: boolean } = {}) { + const captured: unknown[] = []; + const membrane = { + complete: async (req: unknown) => { + captured.push(structuredClone(req)); + return { stopReason: 'end_turn', content: [{ type: 'text', text: 'a summary' }] }; + }, + }; + const strategy = new (opts.placing ? PlacingTopologyStrategy : ProbeStrategy)({ + compressionModel: 'test-compression-model', headWindowTokens: 1, recentWindowTokens: 0, + targetChunkTokens: 1_000_000, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + }); + const path = opts.path ?? freshPath(); + const manager = await ContextManager.open({ path, strategy, membrane: membrane as never }); + const at: Record = {}; + let n = 0; + const add = (name: string, participant: string, content: ContentBlock[]): string => { + at[name] = n++; + return manager.addMessage(participant, content); + }; + for (let i = 0; i < 10; i++) { + if (opts.openingTool && i === 1) { + add('OPEN1', 'assistant', [{ type: 'text', text: body('OPEN1').slice(0, 60) }, { type: 'tool_use', id: 'call_1', name: 'fs:read', input: { p: 'a' } } as ContentBlock]); + } else if (opts.openingTool && i === 2) { + add('OPEN2', 'user', [{ type: 'tool_result', toolUseId: 'call_1', content: body('OPEN2').slice(0, 60) } as ContentBlock]); + } else { + add(`OPEN${i}`, i % 2 ? 'assistant' : 'user', text(body(`OPEN${i}`))); + } + } + const t1 = add('T1', 'Context Manager', text(`[Topic Transition]\n\n${body('TOPIC_ONE')}`.slice(0, 110))); + for (let i = 0; i < 10; i++) add(`ONE${i}`, i % 2 ? 'assistant' : 'user', text(body(`ONE${i}`))); + const t2 = add('T2', 'Context Manager', text(`[Topic Transition]\n\n${body('TOPIC_TWO')}`.slice(0, 110))); + for (let i = 0; i < 6; i++) add(`TWO${i}`, i % 2 ? 'assistant' : 'user', text(body(`TWO${i}`))); + if (resets >= 1) strategy.resetHeadWindow(t1); + if (resets >= 2) strategy.resetHeadWindow(t2); + const ctx = context(manager); + strategy.headOfMessages(ctx, 3); + return { manager, strategy, ctx, captured, at, path }; +} + +describe('the head a compression or merge request gives is the head as of the history it summarizes (#317)', () => { + after(() => { + for (let i = 0; i < counter; i++) if (existsSync(`${ROOT}-${i}`)) rmSync(`${ROOT}-${i}`, { recursive: true, force: true }); + }); + + it("a chunk from before a reset is compressed with the opening, not the new topic's head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + await strategy.runCompress(chunkOf(ctx, at.OPEN6!, at.OPEN8!), ctx); + assert.equal(captured.length, 1); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m} is the head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head's transition isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0, "nor the live head's messages"); + assert.equal(countIn(req, 'TOPIC_ONE'), 0, 'nor a later topic'); + assert.ok(JSON.stringify(req).indexOf('OPEN0 ') < JSON.stringify(req).indexOf('OPEN6 '), 'the head comes before the chunk'); + await manager.close(); + }); + + it('a chunk between two resets is compressed with the head of the topic it was in, cut at the chunk', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.ONE5!), { start: at.T1!, end: at.T1! + 3 }); + await strategy.runCompress(chunkOf(ctx, at.ONE5!, at.ONE7!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_ONE'), 1, "its topic's transition is the head"); + for (const m of ['ONE0', 'ONE1']) assert.equal(countIn(req, `${m} `), 1, `${m}, in that head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, 'not the later topic'); + assert.equal(countIn(req, 'TWO0'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor the opening, which comes before that head'); + await manager.close(); + }); + + it('a chunk that starts at a transition is compressed with no head: nothing came before it in that topic', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + await strategy.runCompress(chunkOf(ctx, at.T1!, at.ONE1!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_ONE'), 1, 'the transition is sent once, in the chunk'); + assert.equal(countIn(req, 'TOPIC_TWO'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0); + assert.deepEqual(strategy.asOf(ctx, at.T1!), { start: at.T1!, end: at.T1! }); + await manager.close(); + }); + + it('a merge across a reset gives the head as of its first message', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const a = summary('L1-501', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + const b = summary('L1-502', [ids[at.T1!]!, ids[at.ONE0!]!]); + strategy.seed(a); + strategy.seed(b); + await strategy.runMerge(2, [a.id, b.id], ctx); + assert.equal(captured.length, 1); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m} is the head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head's transition isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + await manager.close(); + }); + + it("a chunk after the live head's start keeps the live head, as before", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.TWO4!), { start: at.T2!, end: at.T2! + 3 }); + await strategy.runCompress(chunkOf(ctx, at.TWO4!, at.TWO6!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 1, "the live head's transition is the head"); + for (const m of ['TWO0', 'TWO1']) assert.equal(countIn(req, `${m} `), 1); + assert.equal(countIn(req, 'TOPIC_ONE'), 0); + await manager.close(); + }); + + it("a strategy reset to a message that isn't a transition isn't in the store: history before it gets the latest transition or the opening, as a restart reads it", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + strategy.resetHeadWindow(ids[at.OPEN6!]!); + assert.deepEqual(strategy.asOf(ctx, at.OPEN3!), { start: 0, end: 3 }, 'the opening, before the reset message'); + assert.deepEqual(strategy.asOf(ctx, at.OPEN7!), { start: at.OPEN6!, end: at.OPEN7! }, 'from the live start on, the live head, cut at the chunk'); + await strategy.runCompress(chunkOf(ctx, at.OPEN3!, at.OPEN5!), ctx); + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(captured[0], `${m} `), 1, `${m} is the head, raw once`); + await manager.close(); + }); + + it("a past head is priced with today's head window, as the live head is", async () => { + const { manager, strategy, ctx, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: 3 }); + strategy.headOfMessages(ctx, 1); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: 1 }, "today's window decides, not the one in force then"); + strategy.headOfMessages(ctx, 5); + assert.deepEqual(strategy.asOf(ctx, at.ONE7!), { start: at.T1!, end: at.T1! + 5 }); + await manager.close(); + }); + + it('before the live head\'s start, a Context Manager message whose text starts with "[Topic Transition]" starts a past head, as the restart reads one; a user\'s doesn\'t', async () => { + const { manager, strategy, ctx, at } = await build(0); + const quoted = ctx.messageStore.getAll().length; + manager.addMessage('Context Manager', text('[Topic Transition] noted by the host')); + manager.addMessage('user', text(body('LATE0'))); + const userQuoted = quoted + 2; + manager.addMessage('user', text('[Topic Transition] quoted by someone')); + for (let i = 1; i < 4; i++) manager.addMessage('user', text(body(`LATE${i}`))); + const t3 = manager.addMessage('Context Manager', text('[Topic Transition]\n\nA third topic.')); + strategy.resetHeadWindow(t3); + assert.ok(at.T2! < quoted); + assert.equal(strategy.asOf(ctx, userQuoted + 2).start, quoted, "the host's message starts a head; the user's quoting doesn't"); + await manager.close(); + }); + + it("a restart reads the same heads while the live head starts at the last transition: where a past head starts comes from the store alone", async () => { + const path = freshPath(); + const first = await build(2, { path }); + const positions = [first.at.OPEN6!, first.at.ONE5!, first.at.T1!, first.at.TWO4!]; + const before = positions.map((p) => first.strategy.asOf(first.ctx, p)); + await first.manager.close(); + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: first.strategy.headWindowTokens(), recentWindowTokens: 0, + targetChunkTokens: 1_000_000, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + }); + const manager = await ContextManager.open({ path, strategy, membrane: { complete: async () => ({ stopReason: 'end_turn', content: [] }) } as never }); + const ctx = context(manager); + assert.deepEqual(positions.map((p) => strategy.asOf(ctx, p)), before); + await manager.close(); + }); + + it("a chunk whose first message has left the store starts at its first message still there, and gets that message's head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + await strategy.runCompress({ index: 950, startIndex: at.OPEN6!, endIndex: at.OPEN8!, messages: [gone, kept], tokens: 100, compressed: false }, ctx); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + await manager.close(); + }); + + it('a chunk with no message left in the store gets no head and recalls nothing: nothing in the store says when its history happened', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, -1), { start: at.T2!, end: at.T2! }); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + const ids = ctx.messageStore.getAll().map((m) => m.id); + const earlier = summary('L1-841', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + strategy.seed(earlier); + await strategy.runCompress({ index: 950, startIndex: at.OPEN6!, endIndex: at.OPEN7!, messages: [gone], tokens: 100, compressed: false }, ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor any other head'); + assert.equal(countIn(req, `authored ${earlier.id}`), 0, 'and nothing is recalled: its place is unknown'); + await manager.close(); + }); + + it('a chunk whose first message a host removes while it waits in the queue still recalls the summaries before it', async () => { + // Twelve equal messages in chunks of four; the first message of the second chunk is removed during the first + // chunk's compression, as a delete arriving then would (agent-framework's Discord module removes stored rows). + let manager!: ContextManager; + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: 1, recentWindowTokens: 0, l1HoldbackChunks: 0, + targetChunkTokens: 60, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + } as never); + const captured: unknown[] = []; + const ids: string[] = []; + const membrane = { + complete: async (req: unknown) => { + captured.push(structuredClone(req)); + if (captured.length === 1) manager.removeMessage(ids[4]! as never); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `MEMORY_${captured.length}` }] }; + }, + }; + manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never }); + for (let i = 0; i < 12; i++) ids.push(manager.addMessage(i % 2 ? 'assistant' : 'user', text(body(`M${i}`)))); + for (let i = 0; i < 20 && captured.length < 2; i++) await manager.tick(); + assert.ok(captured.length >= 2, 'premise: the second chunk was compressed'); + const req = captured[1]!; + assert.equal(countIn(req, 'M4 '), 1, 'premise: the queued chunk still holds the removed message'); + assert.equal(countIn(req, 'MEMORY_1'), 1, "the first chunk's summary, before it, is recalled"); + await manager.close(); + }); + + it("a summary whose first message a host removed starts at its first message still there: it's recalled, so its other messages keep a form", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const s = summary('L1-801', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + strategy.seed(s); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunk, ctx); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${s.id}`), 1, 'the summary is recalled'); + assert.equal(countIn(req, 'OPEN4 '), 0, 'its messages are represented by it, not raw'); + await manager.close(); + }); + + it('a summary whose first message a host removed is recalled in its place, before a later one, so older history never follows a later memory', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN7!, at.OPEN9!); + const older = summary('L1-851', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const newer = summary('L1-852', [ids[at.OPEN5!]!, ids[at.OPEN6!]!]); + strategy.seed(older); + strategy.seed(newer); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunk, ctx); + const blob = JSON.stringify(captured[0]!); + const o = blob.indexOf(`authored ${older.id}`); + const n = blob.indexOf(`authored ${newer.id}`); + assert.ok(o >= 0 && n >= 0, 'premise: both are recalled'); + assert.ok(o < n, 'the older summary comes first'); + await manager.close(); + }); + + it("a summary with no message left in the store isn't recalled: its place is unknown", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const s = summary('L1-802', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + strategy.seed(s); + manager.removeMessage(ids[at.OPEN3!]! as never); + manager.removeMessage(ids[at.OPEN4!]! as never); + await strategy.runCompress(chunk, ctx); + assert.equal(countIn(captured[0]!, `authored ${s.id}`), 0); + await manager.close(); + }); + + it("a merge recalls a summary before it whose first message a host removed, so that summary's other messages keep a form", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const a = summary('L1-811', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + const b = summary('L1-812', [ids[at.OPEN6!]!, ids[at.OPEN7!]!]); + const c = summary('L1-813', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const x of [a, b, c]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN3!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${a.id}`), 1, 'the summary before the merge is recalled'); + assert.equal(countIn(req, 'OPEN4 '), 0, 'its messages are represented by it, not raw'); + await manager.close(); + }); + + it("a merge whose first source message a host removed starts at its first message still there: that message's head, and recall only of what starts before it", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-821', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-822', [ids[at.OPEN6!]!, ids[at.OPEN7!]!]); + const c = summary('L1-823', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + const after = summary('L1-824', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, `authored ${before.id}`), 1, 'the summary before it is recalled'); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'the summary after it is not'); + await manager.close(); + }); + + it('a merge whose sources have no message left in the store gets no head and recalls nothing, as a chunk in that state', async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-831', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-832', [ids[at.OPEN6!]!]); + const c = summary('L1-833', [ids[at.OPEN7!]!]); + const after = summary('L1-834', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + manager.removeMessage(ids[at.OPEN7!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor any other head'); + assert.equal(countIn(req, `authored ${before.id}`), 0); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'nothing is recalled: its place is unknown'); + await manager.close(); + }); + + it("a merge whose first source has no message left starts at its next source's first message still there", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-861', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-862', [ids[at.OPEN6!]!]); + const c = summary('L1-863', [ids[at.OPEN7!]!, ids[at.OPEN8!]!]); + const after = summary('L1-864', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, `authored ${before.id}`), 1, 'the summary before it is recalled'); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'the summary after it is not'); + await manager.close(); + }); + + it('a merge recalls the summaries before it in the order they start in the store, and none with no message left', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const older = summary('L1-871', [ids[at.OPEN3!]!]); + const newer = summary('L1-872', [ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + const gone = summary('L1-873', [ids[at.OPEN6!]!]); + const b = summary('L1-874', [ids[at.OPEN7!]!]); + const c = summary('L1-875', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + // Seeded newest first, so the order the request gives comes from the sort by start, not from the seeding. + for (const x of [c, b, gone, newer, older]) strategy.seed(x); + // The newer summary's first message is removed: it starts at OPEN5, its first leaf still in the store, after + // the older one's OPEN3. (Before this change the merge's prior recall set, section 1a of executeMerge, placed a + // summary by sourceRange.first, which has no place here: it left this one out. Ordering by sourceRange.first + // would put it first, at -1.) + manager.removeMessage(ids[at.OPEN4!]! as never); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const blob = JSON.stringify(captured[0]!); + const o = blob.indexOf(`authored ${older.id}`); + const n = blob.indexOf(`authored ${newer.id}`); + assert.ok(o >= 0 && n >= 0, 'premise: both are recalled'); + assert.ok(o < n, 'the older comes first'); + assert.equal(blob.indexOf(`authored ${gone.id}`), -1, 'a summary with no message left is not recalled'); + await manager.close(); + }); + + it('a merge recalls a higher-level summary before it whose first leaf a host removed, its start found through its L1s', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const x = { ...summary('L1-901', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-903' }; + const y = { ...summary('L1-902', [ids[at.OPEN5!]!]), mergedInto: 'L2-903' }; + const z: SummaryEntry = { + id: 'L2-903', level: 2, content: 'authored L2-903', tokens: 20, sourceLevel: 1, sourceIds: [x.id, y.id], + sourceRange: { first: ids[at.OPEN3!]!, last: ids[at.OPEN5!]! }, created: 903, + }; + const b = summary('L1-904', [ids[at.OPEN7!]!]); + const c = summary('L1-905', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const e of [x, y, z, b, c]) strategy.seed(e); + manager.removeMessage(ids[at.OPEN3!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + assert.equal(countIn(captured[0]!, 'authored L2-903'), 1, 'the L2 is recalled'); + await manager.close(); + }); + + it("a merge starts at the first of all its sources' leaves still in the store, whatever order its sources are listed in", async () => { + const { manager, strategy, ctx, captured, at } = await build(0, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const between = summary('L1-911', [ids[at.OPEN7!]!]); + const b = summary('L1-912', [ids[at.OPEN6!]!]); + const c = summary('L1-913', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const e of [between, b, c]) strategy.seed(e); + // Listed later source first: by its first source alone the merge would start at OPEN8, after the summary between. + assert.equal(await strategy.runMerge(2, [c.id, b.id], ctx), true, 'premise: the merge runs'); + assert.equal(countIn(captured[0]!, `authored ${between.id}`), 0, 'the summary after its first leaf, OPEN6, is not recalled'); + await manager.close(); + }); + + it('a higher-level summary whose first leaf a host removed starts at its first leaf still there', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const x = { ...summary('L1-881', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-883' }; + const y = { ...summary('L1-882', [ids[at.OPEN5!]!]), mergedInto: 'L2-883' }; + const z: SummaryEntry = { + id: 'L2-883', level: 2, content: 'authored L2-883', tokens: 20, sourceLevel: 1, sourceIds: [x.id, y.id], + sourceRange: { first: ids[at.OPEN3!]!, last: ids[at.OPEN5!]! }, created: 883, + }; + for (const e of [x, y, z]) strategy.seed(e); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunkOf(ctx, at.OPEN7!, at.OPEN9!), ctx); + assert.equal(countIn(captured[0]!, 'authored L2-883'), 1, 'the L2 is recalled, its start found through its L1s'); + await manager.close(); + }); + + it("a summary starts at its first leaf in the store, whatever its sourceRange.first says", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // An L2 whose sources came in the order [later, earlier], so its sourceRange.first names a leaf after the + // chunk while two of its leaves come before it. (An L1's range is its sourceIds' ends: pushSummary sets it.) + const earlier = { ...summary('L1-891', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-893' }; + const later = { ...summary('L1-892', [ids[at.OPEN9!]!]), mergedInto: 'L2-893' }; + const z: SummaryEntry = { + id: 'L2-893', level: 2, content: 'authored L2-893', tokens: 20, sourceLevel: 1, sourceIds: [later.id, earlier.id], + sourceRange: { first: ids[at.OPEN9!]!, last: ids[at.OPEN4!]! }, created: 893, + }; + for (const e of [earlier, later, z]) strategy.seed(e); + await strategy.runCompress(chunkOf(ctx, at.OPEN6!, at.OPEN8!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'authored L2-893'), 1, 'it is recalled, so OPEN3 and OPEN4, which it covers, keep a form'); + assert.equal(countIn(req, 'OPEN3 '), 0); + await manager.close(); + }); + + it('a compression hold inside a past head cuts it at the hold', async () => { + const { manager, strategy, ctx, at } = await build(2); + const ids = ctx.messageStore.getAll().map((m) => m.id); + manager.holdCompression([ids[at.OPEN1!]!]); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: at.OPEN1! }); + await manager.close(); + }); + + it('a chunk compressed after the history behind it was summarized recalls only the summaries before it: the old head after a reset', async () => { + const { manager, strategy, ctx, captured, at } = await build(1); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // The history after the opening's head was summarized while that head was the head; the reset released the + // head, so its chunk is compressed last. + const later = [summary('L1-601', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]), summary('L1-602', [ids[at.OPEN6!]!, ids[at.OPEN7!]!, ids[at.OPEN8!]!, ids[at.OPEN9!]!])]; + for (const s of later) strategy.seed(s); + await strategy.runCompress(chunkOf(ctx, at.OPEN0!, at.OPEN3!), ctx); + const req = captured[0]!; + for (const s of later) { + assert.equal(countIn(req, `Recall memory ${s.id}.`), 0, `${s.id}, after the chunk, isn't recalled`); + assert.equal(countIn(req, `authored ${s.id}`), 0); + } + for (const m of ['OPEN3', 'OPEN6', 'OPEN9']) assert.equal(countIn(req, `${m} `), 0, `${m} stays covered: not raw either`); + assert.equal(countIn(req, 'TOPIC_ONE'), 0, "and the live head isn't the head"); + await manager.close(); + }); + + it('a chunk compressed after the history behind it was summarized recalls only the summaries before it: a chunk let go late', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-701', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const after = summary('L1-702', [ids[at.OPEN7!]!, ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + strategy.seed(before); + strategy.seed(after); + await strategy.runCompress(chunkOf(ctx, at.OPEN5!, at.OPEN7!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, `Recall memory ${before.id}.`), 1, 'the summary before the chunk is recalled'); + assert.equal(countIn(req, `Recall memory ${after.id}.`), 0, 'the one after it is not'); + assert.equal(countIn(req, `authored ${after.id}`), 0); + assert.equal(countIn(req, 'OPEN8 '), 0, 'and its messages stay covered'); + await manager.close(); + }); + + it("one count's reader gives each chunk its own head's answer, asked in either order, and leaves out a tool cycle before that head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { openingTool: true }); + const opening = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const topicOne = chunkOf(ctx, at.ONE5!, at.ONE7!); + // The opening's head carries the cycle; topic one's head starts after it, so the cycle is left out. + const forward = strategy.toolBlockReader(ctx); + assert.deepEqual([forward(opening), forward(topicOne)], [true, false]); + const backward = strategy.toolBlockReader(ctx); + assert.deepEqual([backward(topicOne), backward(opening)], [false, true]); + // Tick agrees for the chunk whose head leaves the cycle out: it asks the model. + await strategy.runCompress(topicOne, ctx); + assert.equal(captured.length, 1, 'tick asked the model, the cycle being before its head'); + await manager.close(); + }); + + it("the count reads a chunk whose first message has left the store from its first message still there, as tick does", async () => { + const { manager, strategy, ctx, at } = await build(2, { openingTool: true }); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + const chunk: Chunk = { index: 960, startIndex: at.OPEN6!, endIndex: at.OPEN8!, messages: [gone, kept], tokens: 100, compressed: false }; + // As of OPEN7 the head is the opening, which carries the cycle raw, so tick defers this chunk before the tools. + assert.equal(strategy.carriesToolBlocks(ctx, chunk), true); + await manager.close(); + }); + + it('with merges on, through two resets, no compression request carries anything from at or after its chunk', async () => { + // From a review: the two-reset flow with merges on, run through ContextManager as a host does. + const MARKER = 'You will soon form a new memory'; + const NAME = /\b(OPEN\d|ONE\d|TWO\d|TOPIC_ONE|TOPIC_TWO)\b/g; + const namesIn = (t: string): string[] => [...new Set([...t.replace(/\\n/g, ' ').matchAll(NAME)].map((m) => m[1]!))]; + const captured: string[] = []; + const membrane = { + complete: async (req: unknown) => { + const blob = JSON.stringify(req); + captured.push(blob); + const at = blob.indexOf(MARKER); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `${at >= 0 ? 'MEM' : 'MERGE'}[${namesIn(at >= 0 ? blob.slice(at) : blob).join(',')}]` }] }; + }, + }; + const probe = await ContextManager.open({ path: freshPath(), strategy: new ProbeStrategy({ compressionModel: 'm' } as never), membrane: membrane as never }); + probe.addMessage('user', text(body('OPEN0'))); + const per = context(probe).messageStore.estimateTokens(context(probe).messageStore.getAll()[0]!); + await probe.close(); + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: 3 * per + Math.floor(per / 2), recentWindowTokens: 0, l1HoldbackChunks: 0, + targetChunkTokens: 60, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 2, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never }); + const order: string[] = []; + const add = (name: string, participant: string): void => { manager.addMessage(participant, text(body(name))); order.push(name); }; + const drain = async (): Promise => { for (let i = 0; i < 60; i++) await manager.tick(); }; + for (let i = 0; i < 10; i++) add(`OPEN${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + await manager.resetHeadWindow(body('TOPIC_ONE')); order.push('TOPIC_ONE'); + for (let i = 0; i < 10; i++) add(`ONE${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + await manager.resetHeadWindow(body('TOPIC_TWO')); order.push('TOPIC_TWO'); + for (let i = 0; i < 6; i++) add(`TWO${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + const pos = new Map(order.map((n, i) => [n, i])); + let compressions = 0; + let merges = 0; + for (const blob of captured) { + const at = blob.indexOf(MARKER); + if (at < 0) { if (blob.includes('You have just reviewed')) merges++; continue; } + compressions++; + const first = Math.min(...namesIn(blob.slice(at)).map((n) => pos.get(n)!)); + const later = namesIn(blob.slice(0, at)).filter((n) => pos.get(n)! >= first); + assert.deepEqual(later, [], `a request for history from ${order[first]} carries, before its chunk: ${later.join(',')}`); + } + assert.ok(compressions >= 5 && merges >= 1, `premise: the flow compresses and merges (${compressions} compressions, ${merges} merges)`); + await manager.close(); + }); + + it("one reader of heads gives every position the head the rule gives it alone, asked in any order, with three topics before the live head, a hold in the live head, and the live head's end overridden", async () => { + const { manager, strategy, ctx, at } = await build(2); + // Two resets more, so three transitions lie before the live head's start. + for (const topic of ['THREE', 'FOUR']) { + await manager.resetHeadWindow(`${topic} begins.`); + for (let i = 0; i < 4; i++) manager.addMessage(i % 2 ? 'assistant' : 'user', text(body(`${topic}${i}`))); + } + const messages = ctx.messageStore.getAll(); + const liveStart = strategy.asOf(ctx, messages.length - 1).start; + const transitions = messages.flatMap((m, i) => + i > 0 && i < liveStart && m.content.some((c) => c.type === 'text' && c.text.startsWith('[Topic Transition]')) ? [i] : []); + assert.equal(transitions.length, 3, 'premise: three transitions before the live head'); + const positions = Array.from({ length: messages.length + 1 }, (_, i) => i - 1); + const orders = { + ascending: positions, + descending: [...positions].reverse(), + 'evens, then odds': [...positions.filter((p) => p % 2 === 0), ...positions.filter((p) => p % 2 !== 0)], + }; + const check = (state: string): void => { + for (const [name, order] of Object.entries(orders)) { + const reader = strategy.heads(ctx); + for (const p of order) assert.deepEqual(reader(p), strategy.headAlone(ctx, p), `${state}, ${name}: position ${p}`); + } + }; + check('as built'); + assert.deepEqual(strategy.heads(ctx)(at.ONE5!), { start: at.T1!, end: at.T1! + 3 }, "topic one's head"); + const live = strategy as unknown as { getHeadWindowEnd(store: unknown): number }; + live.getHeadWindowEnd = () => liveStart + 1; + assert.deepEqual(strategy.heads(ctx)(liveStart + 2), { start: liveStart, end: liveStart + 1 }, "premise: the override shortens the live head"); + check("with the live head's end overridden"); + delete (live as { getHeadWindowEnd?: unknown }).getHeadWindowEnd; + manager.holdCompression([messages[liveStart + 2]!.id]); + assert.deepEqual(strategy.heads(ctx)(liveStart + 3), { start: liveStart, end: liveStart + 2 }, 'premise: the hold cuts the live head'); + check('with a hold in the live head'); + await manager.close(); + }); + + it('one count reads the holds once and walks a head once for the chunks that share it, however many it reads', async () => { + const { manager, strategy, ctx, at } = await build(2, { openingTool: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + manager.holdCompression([ids[at.TWO5!]!]); + const walker = strategy as unknown as { headWindowEndFrom(...args: unknown[]): number }; + const walk = walker.headWindowEndFrom.bind(strategy); + let walks = 0; + walker.headWindowEndFrom = (...args: unknown[]): number => { + walks++; + return walk(...args); + }; + const scans = strategy.holdScans(); + const reader = strategy.toolBlockReader(ctx); + const chunks = [chunkOf(ctx, at.OPEN4!, at.OPEN5!), chunkOf(ctx, at.OPEN6!, at.OPEN7!), chunkOf(ctx, at.OPEN8!, at.OPEN9!)]; + assert.deepEqual(chunks.map((c) => reader(c)), [true, true, true], "premise: the opening's head carries its tool cycle into each chunk's request"); + assert.equal(strategy.holdScans() - scans, 1, 'one scan of the holds'); + assert.equal(walks, 1, "one walk of the opening's head"); + const before = strategy.holdScans(); + const own = strategy.toolBlockReader(ctx); + assert.equal(own(chunkOf(ctx, at.OPEN1!, at.OPEN3!)), true, 'premise: a chunk holding a tool block answers from its own messages'); + assert.equal(strategy.holdScans(), before, 'so its count reads no head and scans no holds'); + await manager.close(); + }); + + it("one count's reader keys its memo by the pins before a chunk too: a pin that opens a summary over a tool cycle changes the answer at the same head start", async () => { + const { manager, strategy, ctx, at } = await build(0, { openingTool: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // The opening's tool cycle (OPEN1, OPEN2) is covered by an L1, and a pin on OPEN3 opens that L1's messages. + strategy.seed(summary('L1-921', [ids[at.OPEN1!]!, ids[at.OPEN2!]!, ids[at.OPEN3!]!])); + manager.pinRange(ids[at.OPEN3!]! as never, ids[at.OPEN3!]! as never); + const before = chunkOf(ctx, at.OPEN0!, at.OPEN1!); + const after = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + assert.equal(strategy.asOf(ctx, at.OPEN0!).start, strategy.asOf(ctx, at.OPEN6!).start, 'premise: both chunks have the same head start'); + const reader = strategy.toolBlockReader(ctx); + assert.equal(reader(before), false, 'before the pin, the L1 covers the cycle'); + assert.equal(reader(after), true, 'after it, the pin opens the L1 and its cycle is carried raw'); + await manager.close(); + }); + + it('under a recall budget for one pair, a compression request keeps the newer summary when the older one has its first message removed', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const older = summary('L1-931', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const newer = summary('L1-932', [ids[at.OPEN5!]!, ids[at.OPEN6!]!]); + strategy.seed(older); + strategy.seed(newer); + (strategy as unknown as { config: { compressionRecallBudgetTokens: number } }).config.compressionRecallBudgetTokens = 100; + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunkOf(ctx, at.OPEN7!, at.OPEN9!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${older.id}`) + countIn(req, `authored ${newer.id}`), 1, 'premise: the budget holds one pair'); + assert.equal(countIn(req, `authored ${newer.id}`), 1, 'the newer summary, which starts later, is kept'); + await manager.close(); + }); + + it("tick's tool deferral and the count of queued work agree for a chunk before a reset whose head holds a tool block", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { openingTool: true }); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + await strategy.runCompress(chunk, ctx); + assert.equal(captured.length, 0, 'tick defers it: its request would carry the opening, whose tool blocks need the host\'s tools'); + assert.equal(strategy.carriesToolBlocks(ctx, chunk), true, 'and the count reads the same request'); + await manager.close(); + }); +}); diff --git a/test/compression-request-preparation.test.ts b/test/compression-request-preparation.test.ts index 07fa6b06..e2dc36a0 100644 --- a/test/compression-request-preparation.test.ts +++ b/test/compression-request-preparation.test.ts @@ -204,7 +204,8 @@ describe('preparation counts and logging', () => { assert.deepEqual(sent.logs, [ '[autobiographical] compression prompt: replaced 1 serialized data URL(s) with markers [count-test]', '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + - '3.0MiB inline image budget (kept 2.5MiB, newest messages first) [count-test]', + '3.0MiB inline image budget and the 500-image ceiling (kept 1 image(s), 2.5MiB, newest messages first) ' + + '[count-test]', ]); assert.deepEqual(captureErrors(() => strategy.send(request([{ type: 'text', text: 'plain' }]), 'unchanged')).logs, []); }); diff --git a/test/compression-transient-ladder.test.ts b/test/compression-transient-ladder.test.ts index fff506ad..18907b85 100644 --- a/test/compression-transient-ladder.test.ts +++ b/test/compression-transient-ladder.test.ts @@ -712,8 +712,10 @@ describe('progress is a shared record, not an instance snapshot', () => { await assert.rejects(a.strategy.run(a.target(), managerContext(a.manager)), /zz network failure/); assert.equal(progressSlot(a.manager).length, 1); - const want = new Set(a.ids.slice(6, 9)); - const other: Chunk = { index: 998, startIndex: 6, endIndex: 9, messages: managerContext(b.manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; + // B's family: a chunk of its own that carries the skip_reply cycle, so its hoist rung applies. (A chunk before + // the cycle no longer does: its request's head ends at the chunk, #317.) + const want = new Set(a.ids.slice(11, 13)); + const other: Chunk = { index: 998, startIndex: 11, endIndex: 13, messages: managerContext(b.manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; await assert.rejects(b.strategy.run(other, managerContext(b.manager)), /zz network failure/); assert.equal(progressSlot(a.manager).length, 2, "B's write kept A's family"); diff --git a/test/context-observer.test.ts b/test/context-observer.test.ts new file mode 100644 index 00000000..11c5d7e7 --- /dev/null +++ b/test/context-observer.test.ts @@ -0,0 +1,1334 @@ +/** + * ContextManagerConfig.observer: what a host is told as it happens. Each + * compile, refused and failed ones included; each refusal context-manager + * itself makes of compression, and none of a provider's; each calibration + * sample. Nothing an observer throws or rejects reaches the work it observes + * (but for a returned promise no JavaScript can give a handler), so a + * compile's result is the same with one, with none, and with one that throws + * or rejects, under each strategy. + */ + +import { after, describe, it } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, NormalizedRequest, ToolDefinition } from '@animalabs/membrane'; +import { + ContextManager, + PassthroughStrategy, + WindowedPassthroughStrategy, + AutobiographicalStrategy, + KnowledgeStrategy, + OverBudgetError, + UncoveredDropError, + OVER_BUDGET_STAGES, + COMPILE_REFUSALS, + COMPRESSION_REFUSALS, + CALIBRATION_SAMPLE_OUTCOMES, + type ContextObserver, + type CompileRecord, + type CompressionRefusalRecord, + type CalibrationSampleRecord, + type ContextStrategy, +} from '../src/index.js'; +import type { Chunk } from '../src/strategies/autobiographical.js'; +import type { StrategyContext, SummaryEntry } from '../src/types/index.js'; + +const dir = mkdtempSync(join(tmpdir(), 'context-observer-')); +after(() => rmSync(dir, { recursive: true, force: true })); +let stores = 0; +const freshPath = (): string => join(dir, `store-${stores++}`); +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); + +function recorder() { + const compiles: CompileRecord[] = []; + const compressions: CompressionRefusalRecord[] = []; + const samples: CalibrationSampleRecord[] = []; + const observer: ContextObserver = { + compile: (record) => { compiles.push(record); }, + compression: (record) => { compressions.push(record); }, + calibration: (record) => { samples.push(record); }, + }; + return { observer, compiles, compressions, samples }; +} + +/** An observer each of whose callbacks throws, or returns a promise that rejects. */ +function hostile(kind: 'throws' | 'rejects', calls: string[] = []): ContextObserver { + const act = (name: string) => () => { + calls.push(name); + if (kind === 'throws') throw new Error(`observer ${name} threw`); + return Promise.reject(new Error(`observer ${name} rejected`)) as unknown as void; + }; + return { compile: act('compile'), compression: act('compression'), calibration: act('calibration') }; +} + +function strategyContext(manager: ContextManager): StrategyContext { + return (manager as unknown as { createStrategyContext(): StrategyContext }).createStrategyContext(); +} + +/** Rejections no handler took while `run` ran and for a few turns after it. */ +async function unhandledRejections(run: () => Promise): Promise { + const seen: unknown[] = []; + const listener = (reason: unknown) => { seen.push(reason); }; + process.on('unhandledRejection', listener); + try { + await run(); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + } finally { + process.off('unhandledRejection', listener); + } + return seen; +} + +describe('compile records', () => { + it('a compile is recorded once, timed whole, on the wall clock, with its dry flag, and reads the manager clock no more', async () => { + const rec = recorder(); + const reads = { observed: 0, bare: 0 }; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: rec.observer, now: () => { reads.observed++; return 1234; } }); + manager.addMessage('User', [text('hello')]); + manager.addMessage('Claude', [text('hi')]); + const bare = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), now: () => { reads.bare++; return 1234; } }); + bare.addMessage('User', [text('hello')]); + bare.addMessage('Claude', [text('hi')]); + reads.observed = reads.bare = 0; + await bare.compile(); + const before = Date.now(); + const result = await manager.compile(); + const after = Date.now(); + assert.equal(reads.observed, reads.bare, 'a compile with an observer reads the manager clock as often as one without'); + bare.close(); + assert.equal(rec.compiles.length, 1); + const [record] = rec.compiles; + assert.equal(record!.strategy, 'passthrough'); + assert.equal(record!.dryRun, false); + assert.ok(record!.startedAt >= before && record!.startedAt <= after, `${before} <= ${record!.startedAt} <= ${after}`); + assert.equal(record!.outcome, 'compiled'); + assert.ok(Number.isFinite(record!.durationMs) && record!.durationMs >= 0, String(record!.durationMs)); + assert.equal('error' in record!, false); + // Passthrough records no render stats, so none are given. + assert.equal(result.renderStats, undefined); + assert.equal('renderStats' in record!, false); + + await manager.compile(undefined, undefined, { dryRun: true }); + assert.equal(rec.compiles.length, 2); + assert.equal(rec.compiles[1]!.dryRun, true); + manager.close(); + }); + + it('an observer without a compile callback is told nothing of compiles, and the compile goes on', async () => { + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compression: () => {} } }); + manager.addMessage('User', [text('hello')]); + const result = await manager.compile(); + assert.equal(result.messages.length, 1); + manager.close(); + }); + + it('the duration covers the whole compile, the strategy\'s select included', async () => { + class Slow extends PassthroughStrategy { + override select(...args: Parameters): ReturnType { + const until = performance.now() + 40; + while (performance.now() < until) { /* a select that takes 40 ms */ } + return super.select(...args); + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Slow(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('the duration covers work the compile does after the strategy\'s select', async () => { + class SlowAfter extends PassthroughStrategy { + takeSelectionCause(): undefined { + const until = performance.now() + 40; + while (performance.now() < until) { /* 40 ms the compile spends after select */ } + return undefined; + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new SlowAfter(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('a compiled record holds the render stats its result holds, the same object', async () => { + const rec = recorder(); + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer: rec.observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const result = await manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + assert.ok(result.renderStats, 'the adaptive select records render stats'); + assert.equal(rec.compiles[0]!.strategy, 'autobiographical'); + assert.equal(rec.compiles[0]!.renderStats, result.renderStats); + manager.close(); + }); + + it('a refused compile is recorded with the error the caller gets, its stage readable', async () => { + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new WindowedPassthroughStrategy(), observer: rec.observer }); + manager.addMessage('User', [text('words '.repeat(2_000))]); + const before = Date.now(); + const thrown = await manager.compile({ maxTokens: 200, reserveForResponse: 20 }).then( + () => assert.fail('the newest message alone is over the budget'), + (error: unknown) => error, + ); + const after = Date.now(); + assert.ok(thrown instanceof OverBudgetError); + assert.equal(thrown.stageId, 'windowed-newest-message'); + assert.equal(rec.compiles.length, 1); + assert.equal(rec.compiles[0]!.outcome, 'refused'); + assert.equal(rec.compiles[0]!.strategy, 'windowed-passthrough'); + assert.equal(rec.compiles[0]!.error, thrown); + assert.equal('renderStats' in rec.compiles[0]!, false); + assert.equal(rec.compiles[0]!.dryRun, false); + assert.ok(rec.compiles[0]!.startedAt >= before && rec.compiles[0]!.startedAt <= after, `${before} <= ${rec.compiles[0]!.startedAt} <= ${after}`); + assert.ok(Number.isFinite(rec.compiles[0]!.durationMs) && rec.compiles[0]!.durationMs >= 0, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('an override of the protected overBudgetError hook is still given the stage\'s words, as 0.21.0 gave them', async () => { + const given: Array<{ stage?: string; stageId?: string }> = []; + class Decorating extends AutobiographicalStrategy { + protected override overBudgetError(...args: Parameters): OverBudgetError { + given.push({ stage: args[1].stage, stageId: args[1].stageId }); + return super.overBudgetError(args[0], { ...args[1], stage: `[custom] ${args[1].stage ?? 'Adaptive picker exhausted'}` }); + } + } + const strategy = new Decorating({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const thrown = await manager.compile({ maxTokens: 501, reserveForResponse: 500 }).then( + () => assert.fail('one token of room fits nothing'), + (error: unknown) => error, + ); + assert.deepEqual(given, [{ stage: 'Head/tail reservation exceeded budget', stageId: 'head-tail-reservation' }]); + assert.ok(thrown instanceof OverBudgetError); + assert.match(thrown.message, /^\[custom\] Head\/tail reservation exceeded budget but /); + assert.equal(thrown.stageId, 'head-tail-reservation'); + manager.close(); + }); + + it('an autobiographical refusal names its stage, and its message opens as before', async () => { + const rec = recorder(); + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer: rec.observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const thrown = await manager.compile({ maxTokens: 501, reserveForResponse: 500 }).then( + () => assert.fail('one token of room fits nothing'), + (error: unknown) => error, + ); + assert.ok(thrown instanceof OverBudgetError); + assert.equal(thrown.stageId, 'head-tail-reservation'); + assert.match(thrown.message, /^Head\/tail reservation exceeded budget but \d+ tokens still exceed hard budget 1 /); + assert.equal(rec.compiles[0]!.outcome, 'refused'); + assert.equal(rec.compiles[0]!.error, thrown); + manager.close(); + }); + + it('a thrown value whose name can\'t be read is a failed compile, and the caller still gets it', async () => { + const { proxy: thrown, revoke } = Proxy.revocable({}, {}); + revoke(); + class Throwing extends PassthroughStrategy { + override select(): never { throw thrown; } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Throwing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + // Caught by hand: assert.rejects reads what it gets, which a revoked proxy refuses. + let caught: unknown = 'nothing thrown'; + try { await manager.compile(); } catch (error) { caught = error; } + assert.ok(caught === thrown, 'the caller gets the value thrown'); + assert.equal(rec.compiles.length, 1); + assert.equal(rec.compiles[0]!.outcome, 'failed'); + assert.ok(rec.compiles[0]!.error === thrown); + manager.close(); + }); + + it('a compile that throws is timed whole too, and every record is taken before the caller awaits', async () => { + class SlowThenThrow extends PassthroughStrategy { + override select(): never { + const until = performance.now() + 40; + while (performance.now() < until) { /* 40 ms of work before the refusal */ } + throw new Error('a defect after some work'); + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new SlowThenThrow(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + const pending = manager.compile(); + assert.equal(rec.compiles.length, 1, 'told before the caller awaits'); + await pending.catch(() => {}); + assert.equal(rec.compiles[0]!.outcome, 'failed'); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + + const ok = recorder(); + const fine = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: ok.observer }); + fine.addMessage('User', [text('hello')]); + const compiled = fine.compile(); + assert.equal(ok.compiles.length, 1, 'told before the caller awaits'); + await compiled; + fine.close(); + }); + + it('anything else thrown is a failed compile, null and strings included; a refusal from another copy of the package is still a refusal', async () => { + const defect = new Error('a defect in select'); + const foreignRefusal = Object.assign(new Error('refused by another copy'), { name: 'UncoveredDropError' }); + for (const [thrown, outcome] of [[defect, 'failed'], [foreignRefusal, 'refused'], [null, 'failed'], ['a thrown string', 'failed']] as const) { + class Throwing extends PassthroughStrategy { + override select(): never { throw thrown; } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Throwing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + for (const dryRun of [false, true]) { + let caught: unknown = 'nothing thrown'; + try { await manager.compile(undefined, undefined, { dryRun }); } catch (error) { caught = error; } + assert.ok(caught === thrown, `the caller gets ${String(thrown)}`); + } + assert.equal(rec.compiles.length, 2); + assert.deepEqual(rec.compiles.map((r) => [r.outcome, r.dryRun]), [[outcome, false], [outcome, true]]); + assert.ok(rec.compiles.every((r) => r.error === thrown && 'error' in r)); + manager.close(); + } + }); + + it('an observer\'s methods are called on the observer itself', async () => { + class Recorder implements ContextObserver { + readonly records: CompileRecord[] = []; + compile(record: CompileRecord): void { this.records.push(record); } + } + const observer = new Recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.equal(observer.records.length, 1); + manager.close(); + }); + + it('a callback that throws or rejects is logged once through the manager\'s diagnostics, and one that can\'t be read is treated as absent', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + const throwing = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('throws') }); + throwing.addMessage('User', [text('hello')]); + await throwing.compile(); + await throwing.compile(); + throwing.close(); + const rejecting = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('rejects') }); + rejecting.addMessage('User', [text('hello')]); + await rejecting.compile(); + await rejecting.compile(); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + rejecting.close(); + const compressions: CompressionRefusalRecord[] = []; + const unreadable = { + get compile(): never { throw new Error('a getter that throws'); }, + compression: (r: CompressionRefusalRecord) => { compressions.push(r); }, + }; + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const opened = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: unreadable as never }); + opened.addMessage('User', [text('hello')]); + const result = await opened.compile(); + assert.equal(result.messages.length, 1, 'the manager opened and compiled'); + assert.equal(handed!.compile, undefined); + handed!.compression!({ reason: 'empty' }); + assert.deepEqual(compressions, [{ reason: 'empty' }]); + opened.close(); + } finally { + console.error = original; + } + const logged = lines.filter((line) => line.includes("the observer's compile callback threw or rejected")); + assert.equal(logged.length, 3, `once each for the throwing, the rejecting and the unreadable observer: ${logged.join(' | ')}`); + // Each line carries what failed and the manager's namespace. + assert.ok(logged[0]!.includes(' ns=') && logged[0]!.includes('observer compile threw'), logged[0]); + assert.ok(logged[1]!.includes(' ns=') && logged[1]!.includes('observer compile rejected'), logged[1]); + assert.ok(logged[2]!.includes(' ns=') && logged[2]!.includes('a getter that throws'), logged[2]); + }); + + it('a failure is logged once per callback and once per manager, under that manager\'s namespace', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + const observer = hostile('throws'); + for (const namespace of ['first-probe', 'second-probe']) { + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer, namespace }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + await manager.compile(); + handed!.compression!({ reason: 'empty' }); + handed!.compression!({ reason: 'empty' }); + manager.close(); + } + } finally { + console.error = original; + } + const count = (callback: string, namespace: string) => lines.filter((line) => line.includes(`the observer's ${callback} callback threw or rejected`) && line.includes(` ns=${namespace} `)).length; + for (const namespace of ['first-probe', 'second-probe']) { + assert.equal(count('compile', namespace), 1, `once in ${namespace}`); + assert.equal(count('compression', namespace), 1, `once in ${namespace}, apart from compile`); + } + assert.equal(lines.length, 4, lines.join(' | ')); + }); + + it('a proxy around a promise, which no JavaScript can give a handler, is told, and the compile goes on', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + let inner: Promise | undefined; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { + compile: () => { inner = Promise.reject(new Error('the proxied promise rejected')); return new Proxy(inner, {}) as never; }, + } }); + manager.addMessage('User', [text('hello')]); + const pending = manager.compile(); + // The limit the docs name: nothing reaches the proxied promise, so the + // test handles it, as only the observer could. + inner!.catch(() => {}); + assert.equal((await pending).messages.length, 1, 'the compile is unchanged'); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + manager.close(); + } finally { + console.error = original; + } + assert.equal(lines.filter((line) => line.includes("the observer's compile callback threw or rejected")).length, 1, lines.join(' | ')); + }); + + it('a log that throws as it tells of a failure changes nothing either', async () => { + const original = console.error; + let entered = 0; + console.error = () => { entered++; throw new Error('the log broke'); }; + try { + const throwing = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('throws') }); + throwing.addMessage('User', [text('hello')]); + assert.equal((await throwing.compile()).messages.length, 1); + assert.ok(entered > 0, 'the log was asked to tell of the throw'); + throwing.close(); + entered = 0; + const unreadable = { get compile(): never { throw new Error('a getter that throws'); } }; + const opened = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: unreadable as never }); + assert.ok(entered > 0, 'the log was asked to tell of the unreadable callback'); + opened.addMessage('User', [text('hello')]); + assert.equal((await opened.compile()).messages.length, 1); + opened.close(); + } finally { + console.error = original; + } + }); + + it('a returned thenable or promise that rejects, or whose then fails before it settles, is handled and told; a callback is read once', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + const fulfilling = { then(resolve: (value: unknown) => void) { setImmediate(() => resolve('fine')); } }; + const shapes: Record unknown> = { + 'a then that throws': () => ({ then() { throw new Error('then threw'); } }), + 'a then getter that throws': () => ({ get then(): never { throw new Error('then getter threw'); } }), + 'a callable thenable that rejects': () => Object.assign(() => {}, { then(_: unknown, reject: (error: unknown) => void) { reject(new Error('callable rejected')); } }), + 'a native promise with a no-op then of its own': () => Object.assign(Promise.reject(new Error('own-then promise rejected')), { then() {} }), + 'a native promise with a then of its own that isn\'t callable': () => Object.assign(Promise.reject(new Error('uncallable-then promise rejected')), { then: 42 }), + 'a resolved native promise whose constructor getter throws': () => Object.defineProperty(Promise.resolve(), 'constructor', { get(): never { throw new Error('constructor getter threw'); } }), + 'a rejecting subclass promise with a then of its own': () => { class Sub extends Promise {} return Object.assign(Sub.reject(new Error('subclass promise rejected')), { then: undefined }); }, + 'a thenable that fulfils': () => fulfilling, + 'an object whose then getter throws only when read again': () => { let reads = 0; return { get then() { if (reads++ > 0) throw new Error('read again'); return undefined; } }; }, + }; + let stray: unknown[] = []; + try { + stray = await unhandledRejections(async () => { + for (const [shape, returned] of Object.entries(shapes)) { + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compile: () => returned() as void } }); + manager.addMessage('User', [text('hello')]); + assert.equal((await manager.compile()).messages.length, 1, shape); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + manager.close(); + } + }); + } finally { + console.error = original; + } + assert.deepEqual(stray, []); + const told = lines.filter((line) => line.includes("the observer's compile callback threw or rejected")); + const expected = ['then threw', 'then getter threw', 'callable rejected', 'own-then promise rejected', 'uncallable-then promise rejected', 'constructor getter threw', 'subclass promise rejected']; + assert.equal(told.length, expected.length, `each failing shape told once, the others not: ${told.join(' | ')}`); + expected.forEach((words, i) => assert.ok(told[i]!.includes(words), `${words}: ${told[i]}`)); + + // The callback is the function read at open, read once: replacing or removing it later changes nothing. + const records: CompileRecord[] = []; + let reads = 0; + const record = (r: CompileRecord) => { records.push(r); }; + const observer = { get compile() { reads++; return record; } } as ContextObserver; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + Object.defineProperty(observer, 'compile', { value: undefined }); + await manager.compile(); + assert.equal(records.length, 2); + assert.equal(reads, 1, 'read once, when the observer was isolated'); + manager.close(); + + // A callback that isn't a function is absent: nothing called, nothing told. + const lines2: string[] = []; + console.error = (...args: unknown[]) => { lines2.push(args.map(String).join(' ')); }; + try { + const odd = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compile: 'yes' } as never }); + odd.addMessage('User', [text('hello')]); + assert.equal((await odd.compile()).messages.length, 1); + odd.close(); + } finally { + console.error = original; + } + assert.deepEqual(lines2, []); + }); +}); + +describe('what an observer throws or rejects never reaches the work', () => { + const STRATEGIES: Record ContextStrategy> = { + passthrough: () => new PassthroughStrategy(), + windowed: () => new WindowedPassthroughStrategy(), + autobiographical: () => new AutobiographicalStrategy({ + targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, hierarchical: true, adaptiveResolution: true, + l1HoldbackChunks: 0, minChunkCharsForLLM: 0, autoTickOnNewMessage: false, compressionModel: 'test-model', + }), + knowledge: () => new KnowledgeStrategy({ + targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, hierarchical: true, + l1HoldbackChunks: 0, minChunkCharsForLLM: 0, autoTickOnNewMessage: false, compressionModel: 'test-model', + } as never), + }; + + /** `value` as JSON without the fields that differ between any two compiles: timings and ids minted per compile. */ + function withoutVolatile(value: unknown): unknown { + return JSON.parse(JSON.stringify(value ?? null, (key, v) => (['compileMs', 'solveMs', 'compileId'].includes(key) ? undefined : v))); + } + + /** One store's compiled request, and what the compile left, after compression has had its turn. */ + async function compiled(strategy: ContextStrategy, observer?: ContextObserver) { + let summaries = 0; + const membrane = { + complete: async () => ({ stopReason: 'end_turn', content: [text(`Summary ${summaries++} of the chunk`)], usage: { inputTokens: 100, outputTokens: 20 } }), + }; + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, ...(observer ? { observer } : {}) }); + for (let i = 0; i < 24; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'substantive words '.repeat(20))]); + const tick = (strategy as { tick?: (ctx: StrategyContext) => Promise }).tick; + for (let i = 0; i < 12 && tick; i++) await tick.call(strategy, strategyContext(manager)); + const budget = { maxTokens: 3_000, reserveForResponse: 200 }; + const injections = [ + { namespace: 'probe', position: 'afterUser' as const, content: [text('injected')] }, + { namespace: 'probe', position: 'system' as const, content: [text('a system note')] }, + ]; + const calibration = strategy as unknown as { _calibration?: number; _calibrationArmed?: boolean; _lastCompileEstimate?: number }; + const outcome = await manager.compile(budget, injections).then( + (result) => ({ + messages: result.messages, + systemInjections: result.systemInjections, + renderStats: withoutVolatile(result.renderStats), + provenance: withoutVolatile({ messages: result.provenance?.messages, layout: result.provenance?.layout, strategy: result.provenance?.strategy }), + calibration: [calibration._calibration, calibration._calibrationArmed, calibration._lastCompileEstimate], + }), + (error: unknown) => ({ error: String(error) }), + ); + manager.close(); + return { outcome, summaries }; + } + + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`under ${name}, a compile's result is the same with an observer, without one, and with one that throws or rejects`, async () => { + const bare = await compiled(make()); + const rec = recorder(); + const observed = await compiled(make(), rec.observer); + const calls: string[] = []; + let thrown: Awaited> | undefined; + let rejected: Awaited> | undefined; + const stray = await unhandledRejections(async () => { + thrown = await compiled(make(), hostile('throws', calls)); + rejected = await compiled(make(), hostile('rejects', calls)); + }); + assert.ok(!('error' in bare.outcome), `the fixture compiles under ${name}: ${JSON.stringify(bare.outcome)}`); + assert.ok(bare.outcome.systemInjections.length > 0, 'a system injection to compare'); + assert.deepEqual(observed.outcome, bare.outcome); + assert.deepEqual(thrown!.outcome, bare.outcome); + assert.deepEqual(rejected!.outcome, bare.outcome); + assert.deepEqual(stray, [], 'an observer\'s rejection is no unhandled rejection'); + // The observers were told, so the comparison is with observed compiles. + assert.equal(rec.compiles.length, 1); + assert.ok(calls.filter((call) => call === 'compile').length === 2, calls.join(',')); + if (name === 'autobiographical' || name === 'knowledge') { + assert.ok(bare.summaries > 0, 'compression ran, so the compile folds summaries'); + } + }); + } + + it('a fork initializes the strategy again with the same observer', async () => { + const handed: Array = []; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed.push(ctx.observer); } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.fork('probe-branch'); + assert.ok(handed.length >= 2, `initialized ${handed.length} time(s)`); + assert.ok(handed.every((observer) => observer !== undefined && observer === handed[0]), 'every initialization gets the one isolated observer'); + manager.close(); + }); + + it('a strategy is handed the isolated observer: what the host\'s throws never reaches the strategy', async () => { + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const calls: string[] = []; + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: hostile('throws', calls) }); + assert.ok(handed?.compression, 'the strategy has the observer'); + handed!.compression!({ reason: 'empty' }); + handed!.calibration!({ outcome: 'held', ratio: 1, multiplier: 1 }); + assert.deepEqual(calls, ['compression', 'calibration'], 'the host\'s callbacks ran'); + manager.close(); + + let none: StrategyContext['observer'] | 'unset' = 'unset'; + class Bare extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { none = ctx.observer; } + } + const without = await ContextManager.open({ path: freshPath(), strategy: new Bare() }); + assert.equal(none, undefined, 'no observer, none handed'); + without.close(); + + let partial: StrategyContext['observer']; + class Partial extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { partial = ctx.observer; } + } + const onlyCompression = await ContextManager.open({ path: freshPath(), strategy: new Partial(), observer: { compression: () => {} } }); + assert.deepEqual(Object.keys(partial!), ['compression'], 'only the callbacks the host gave'); + onlyCompression.close(); + }); +}); + +describe('the exported lists', () => { + const diagnostics = { headTokens: 1, tailTokens: 2, middleTokens: 3, middleChunkCount: 1, deepestLevel: 1 }; + + it('words given as stage build the message 0.21.0 built from them, byte for byte, and read as strategy-defined', () => { + // 0.21.0 opened the message with `${opts.stage ?? 'Adaptive picker exhausted'}` + // and left the rest as it is; the rest is pinned by the message tests. + const rest = new OverBudgetError({ budget: 1, actual: 2, diagnostics }).message.slice('Adaptive picker exhausted'.length); + const released = (stage: unknown) => `${stage ?? 'Adaptive picker exhausted'}${rest}`; + const words = [ + // The library's own ids and words, given as a strategy's words. + ...OVER_BUDGET_STAGES, + 'Head/tail reservation exceeded budget', + 'Adaptive picker exhausted', + 'My strategy: the newest turn alone', + // Words that name a property every object has, and none. + 'constructor', 'toString', '__proto__', 'hasOwnProperty', '', + ]; + for (const stage of words) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stage }); + assert.equal(error.message, released(stage), stage); + assert.equal(error.stageId, 'strategy-defined', stage); + } + // Values a caller past the type passes build what 0.21.0 built from them. + for (const stage of [null, undefined, 42]) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stage } as never); + assert.equal(error.message, released(stage), String(stage)); + assert.equal(error.stageId, stage == null ? 'picker-exhausted' : 'strategy-defined', String(stage)); + } + }); + + it('a stageId names the library\'s stage, and words given beside it still open the message', () => { + const both = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: 'head-tail-reservation', stage: 'My words' }); + assert.equal(both.stageId, 'head-tail-reservation'); + assert.match(both.message, /^My words but 2 tokens still exceed hard budget 1 /); + // An id past the type that isn't one of the library's own keys, as a string, is no id. + for (const stageId of ['strategy-defined', 'constructor', '__proto__', 'no-such-stage', { toString: () => 'head-tail-reservation' }, ['head-tail-reservation']]) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId } as never); + assert.equal(error.stageId, 'picker-exhausted', String(stageId)); + assert.match(error.message, /^Adaptive picker exhausted but /, String(stageId)); + } + }); + + it('each of the library\'s stages opens its message with the words `main` gave it', () => { + const released: Record = { + 'picker-exhausted': 'Adaptive picker exhausted', + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', + 'passthrough-system-changes': 'Passthrough kept only its system changes', + 'passthrough-changes-and-newest-message': 'Passthrough: its system changes and the newest conversation message', + 'windowed-newest-message': 'Windowed passthrough: the newest message alone', + 'windowed-changes-and-newest-message': 'Windowed passthrough: its system changes and the newest conversation message', + 'windowed-system-changes': 'Windowed passthrough: its system changes alone', + }; + assert.deepEqual(OVER_BUDGET_STAGES.filter((s) => s !== 'strategy-defined').sort(), Object.keys(released).sort()); + for (const [stageId, words] of Object.entries(released)) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: stageId as never }); + assert.ok(error.message.startsWith(`${words} but 2 tokens still exceed hard budget 1 (`), error.message); + } + }); + + it('each stage opens its refusal\'s message with words of its own, and a refusal that names none is the picker\'s', () => { + const openings = new Set(); + const library = OVER_BUDGET_STAGES.filter((s) => s !== 'strategy-defined'); + for (const stageId of library) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: stageId as never }); + assert.equal(error.stageId, stageId); + const opening = error.message.slice(0, error.message.indexOf(' but 2 tokens still exceed hard budget 1')); + assert.ok(opening.length > 0, error.message); + openings.add(opening); + } + assert.equal(openings.size, library.length, [...openings].join(' | ')); + const unnamed = new OverBudgetError({ budget: 1, actual: 2, diagnostics }); + assert.equal(unnamed.stageId, 'picker-exhausted'); + assert.match(unnamed.message, /^Adaptive picker exhausted but 2 tokens still exceed hard budget 1 /); + }); + + it('the compile refusals are the two refusal errors by name, and nothing else', () => { + const names = [ + new OverBudgetError({ budget: 1, actual: 2, diagnostics }).name, + new UncoveredDropError({ droppedIds: [], site: 'test', diagnostics: { budget: 1, totalTokens: 2 } }).name, + ]; + assert.deepEqual([...COMPILE_REFUSALS].sort(), names.sort()); + }); + + it('the compression reasons and sample outcomes are as documented', () => { + assert.deepEqual([...COMPRESSION_REFUSALS], ['admission-rejected', 'topology-violation', 'empty', 'incomplete', 'malformed', 'chunk-call-cap', 'window-call-cap', 'overlap-guard']); + assert.deepEqual([...CALIBRATION_SAMPLE_OUTCOMES], ['out-of-band', 'deferred', 'held', 'confirmed', 'replaced', 'stepped']); + }); +}); + +// ---- Compression: an L1 chunk's ladder ---- + +class LadderProbe extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { this.pushSummary(entry); } + run(chunk: Chunk, ctx: StrategyContext): Promise { return this.compressChunkHierarchical(chunk, ctx); } +} + +const DIARY = 'I weighed the thread and decided to stay quiet. '.repeat(12); +const toolUse = (id: string, name: string, input: Record): ContentBlock => ({ type: 'tool_use', id, name, input } as ContentBlock); +const toolResult = (id: string, content: string): ContentBlock => ({ type: 'tool_result', toolUseId: id, content } as ContentBlock); +const tool = (name: string): ToolDefinition => ({ name, description: 'd', inputSchema: { type: 'object', properties: {} } } as never); + +type Answer = Record | { error: Error }; +const REFUSAL = (inputTokens = 100): Answer => ({ content: [], stopReason: 'refusal', usage: { inputTokens, outputTokens: 0 }, raw: { response: { stop_details: { category: 'reasoning_extraction' } } } }); +const ANSWER = (t: string, stopReason = 'end_turn'): Answer => ({ content: t ? [text(t)] : [], stopReason, usage: { inputTokens: 80, outputTokens: 20 } }); + +function scripted(answers: Answer[]) { + const calls: NormalizedRequest[] = []; + return { + calls, + membrane: { + complete: async (request: NormalizedRequest) => { + const answer = answers[Math.min(calls.length, answers.length - 1)]!; + calls.push(structuredClone(request)); + if ('error' in answer && answer.error instanceof Error) throw answer.error; + return answer; + }, + } as never, + }; +} + +/** A chunk whose last turn holds a long private-reasoning tool argument, after one seeded L1. */ +async function ladder(membrane: unknown, options: Record, path = freshPath(), from = 10) { + const rec = recorder(); + const strategy = new LadderProbe({ + compressionModel: 'same-model', targetChunkTokens: 100, recentWindowTokens: 0, headWindowTokens: 100_000, + autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, compressionRefusalCurveFallbacks: 0, + ...options, + } as never); + const manager = await ContextManager.open({ path, strategy, membrane: membrane as never, observer: rec.observer }); + manager.setToolDefinitions(['skip_reply', 'journal'].map(tool)); + const ids: string[] = []; + for (let i = 0; i < 10; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substantive '.repeat(12))])); + ids.push(manager.addMessage('User', [text('raw-10 ambient chatter ' + 'substantive '.repeat(12))])); + ids.push(manager.addMessage('Claude', [toolUse('skip1', 'skip_reply', { reason: DIARY, wake_in_seconds: 1500 })])); + ids.push(manager.addMessage('User', [toolResult('skip1', '{"skipped":true}')])); + strategy.seed({ id: 'L1-100', level: 1, content: 'authored L1-100', tokens: 20, sourceLevel: 0, sourceIds: [ids[0]!, ids[1]!], sourceRange: { first: ids[0]!, last: ids[1]! }, created: 100 }); + const want = new Set(ids.slice(from)); + const target = (): Chunk => ({ index: 999, startIndex: from, endIndex: 13, messages: strategyContext(manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }); + return { manager, strategy, target, rec }; +} + +const HOIST = { compressionToolProseFallback: { intoTool: 'journal', fromTools: ['skip_reply'] } }; + +describe('compression refusals: an L1 ladder', () => { + it('a provider\'s refusal is the provider\'s decision, not one of these', async () => { + const { calls, membrane } = scripted([REFUSAL()]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a provider\'s rejection of the request is the provider\'s decision too', async () => { + const rejected = Object.assign(new Error('invalid request'), { type: 'invalid_request', httpStatus: 400, retryable: false }); + const { calls, membrane } = scripted([{ error: rejected }]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a canonical answer cut short is incomplete', async () => { + const { calls, membrane } = scripted([ANSWER('a summary cut', 'max_tokens')]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }]); + fx.manager.close(); + }); + + it('a canonical answer with no text is empty', async () => { + const { membrane } = scripted([ANSWER('')]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + // The canonical answer, then the plain-prose retry's, each empty. + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }, { reason: 'empty' }]); + fx.manager.close(); + }); + + it('a later rung\'s answer that isn\'t a response\'s shape is malformed', async () => { + const { calls, membrane } = scripted([REFUSAL(), { stopReason: 'end_turn', content: 'not a list of blocks', usage: { inputTokens: 80, outputTokens: 20 } }]); + const fx = await ladder(membrane, HOIST); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 2, 'canonical, then the hoisted retry'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'malformed' }]); + fx.manager.close(); + }); + + it('a hoisted retry over the compression budget is refused before any call', async () => { + const { calls, membrane } = scripted([REFUSAL(90_000), ANSWER('must never be called')]); + const fx = await ladder(membrane, { ...HOIST, compressionContextBudgetTokens: 50_000 }); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1, 'the hoisted retry is never sent'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }]); + fx.manager.close(); + }); + + describe('the split rung, by what ended it', () => { + const SPLIT = { compressionSourceOnlyFallback: true, compressionSplitFallback: true }; + // The canonical request and source-only refuse, so the split rung runs: its + // first piece is the chunk's first message, its second the tool round, + // which it can't split further. + const cases: Array<[string, Record, Answer[], CompressionRefusalRecord[]]> = [ + ['its cap on calls in one attempt', { compressionSplitMaxCallsPerChunk: 1 }, [ANSWER('the first piece summarized ' + 'detail '.repeat(20))], [{ reason: 'chunk-call-cap' }]], + ['its window\'s calls used by the attempt alone', { compressionSplitMaxCallsPer10Min: 1 }, [ANSWER('the first piece summarized ' + 'detail '.repeat(20))], [{ reason: 'window-call-cap' }]], + ['a piece it can\'t split that answers with no text', {}, [ANSWER('')], [{ reason: 'empty' }]], + ['a piece it can\'t split that answers cut short', {}, [ANSWER('a piece summary cut', 'max_tokens')], [{ reason: 'incomplete' }]], + ['a piece it can\'t split that the provider refuses', {}, [REFUSAL()], []], + ['a piece whose answer is malformed', {}, [{ stopReason: 'end_turn', content: 'not a list', usage: { inputTokens: 80, outputTokens: 20 } }], [{ reason: 'malformed' }]], + ]; + it('a piece that answers empty and is split further is reported once, and its halves fold', async () => { + // Five messages: raw-8, raw-9, raw-10, then the tool round. The root splits + // after raw-10, so the first piece is the three plain messages, which + // answer empty and split into raw-8..9 and raw-10. + const pieces = ['raw-8 and raw-9', 'raw-10', 'the tool round'].map((p) => ANSWER(`${p} summarized ` + 'detail '.repeat(20))); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), ...pieces]); + const fx = await ladder(membrane, SPLIT, freshPath(), 8); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 6, 'canonical, source-only, the empty piece, its two halves, and the tool round'); + assert.equal(chunk.compressed, true, 'the stitch landed'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + + it('a split interrupted and run again reports its piece once per call', async () => { + const transient = new Error('a socket hang up the split resumes after'); + const tool = ANSWER('the tool round summarized ' + 'detail '.repeat(20)); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), { error: transient }, ANSWER(''), tool]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitPlaceholder: true }); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.equal(calls.length, 4); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 6, 'only the split rung runs again, asking both pieces'); + assert.equal(chunk.compressed, true); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }, { reason: 'empty' }], 'the first piece answered empty on each call'); + fx.manager.close(); + }); + + it('a split its cap ended isn\'t reported again on a later run, which makes no call', async () => { + const path = freshPath(); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER('the first piece summarized ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitMaxCallsPerChunk: 1 }, path); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3); + assert.deepEqual(fx.rec.compressions, [{ reason: 'chunk-call-cap' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'the chunk is quarantined, so nothing is asked again'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'chunk-call-cap' }]); + fx.manager.close(); + }); + + it('a chunk an existing L1 partly covers is refused by the overlap guard, reported once until the strategy loads its state again', async () => { + // The chunk has no record, so the guard keys it by its messages. + const { calls, membrane } = scripted([ANSWER('never asked')]); + const fx = await ladder(membrane, SPLIT); + const chunk = fx.target(); + assert.equal(chunk.recordId, undefined); + const first = chunk.messages[0]!.id; + fx.strategy.seed({ id: 'L1-200', level: 1, content: 'covers raw-10 alone', tokens: 20, sourceLevel: 0, sourceIds: [first], sourceRange: { first, last: first }, created: 200 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0, 'nothing is asked'); + assert.equal(chunk.compressed, false); + assert.deepEqual(fx.rec.compressions, [{ reason: 'overlap-guard' }], 'refused twice, reported once'); + (fx.strategy as unknown as { loadPersistedState(): void }).loadPersistedState(); + fx.strategy.seed({ id: 'L1-201', level: 1, content: 'covers raw-10 alone', tokens: 20, sourceLevel: 0, sourceIds: [first], sourceRange: { first, last: first }, created: 201 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.deepEqual(fx.rec.compressions, [{ reason: 'overlap-guard' }, { reason: 'overlap-guard' }], 'reported again after the load'); + fx.manager.close(); + }); + + it('a chunk an existing L1 covers whole is dropped, with no report', async () => { + const { calls, membrane } = scripted([ANSWER('never asked')]); + const fx = await ladder(membrane, SPLIT); + const chunk = fx.target(); + const ids = chunk.messages.map((m) => m.id); + fx.strategy.seed({ id: 'L1-200', level: 1, content: 'covers the whole chunk', tokens: 20, sourceLevel: 0, sourceIds: ids, sourceRange: { first: ids[0]!, last: ids.at(-1)! }, created: 200 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.equal(chunk.compressed, true, 'dropped as covered'); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('an empty piece is reported even when the stitch succeeds', async () => { + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), ANSWER('the tool round summarized ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitPlaceholder: true }); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 4, 'canonical, source-only, and both pieces'); + assert.equal(chunk.compressed, true, 'the stitch landed'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + for (const [name, options, pieces, expected] of cases) { + it(`${name}: ${expected.length ? expected[0]!.reason : 'nothing reported'}`, async () => { + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ...pieces]); + const fx = await ladder(membrane, { ...SPLIT, ...options }); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'canonical, source-only, and one piece'); + assert.deepEqual(fx.rec.compressions, expected); + fx.manager.close(); + }); + } + }); + + it('a verdict is reported once: a resumed ladder reuses it without a second report', async () => { + const transient = new Error('a socket hang up the ladder resumes after'); + const path = freshPath(); + const first = scripted([ANSWER('a summary cut', 'max_tokens'), { error: transient }, ANSWER('the source-only summary ' + 'detail '.repeat(20))]); + const fx = await ladder(first.membrane, { compressionSourceOnlyFallback: true }, path); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.equal(first.calls.length, 2, 'canonical, then source-only, which fails'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(first.calls.length, 3, 'the resumed ladder sends only source-only again, so it reached the reused canonical verdict'); + assert.equal(chunk.compressed, true, 'and the resumed ladder compressed the chunk'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }], 'the canonical verdict is reused, not reported again'); + fx.manager.close(); + }); + + it('an admission rejection is decided again by a resumed ladder, and reported again', async () => { + const transient = new Error('a socket hang up the ladder resumes after'); + const { calls, membrane } = scripted([REFUSAL(90_000), { error: transient }, ANSWER('the source-only summary ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...HOIST, compressionContextBudgetTokens: 50_000, compressionSourceOnlyFallback: true }); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'canonical and source-only, then source-only again; the hoisted retry is never sent'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }, { reason: 'admission-rejected' }]); + fx.manager.close(); + }); +}); + +// ---- Compression: the recall variants' plan ---- + +class RecallProbe extends LadderProbe { + linkChild(child: SummaryEntry, parentId: string): void { this.setMergedInto(child, parentId); } +} + +const summaryOf = (id: string, level: number, sourceIds: string[], first: string, last: string): SummaryEntry => ({ + id, level, content: `authored ${id}`, tokens: 20, sourceLevel: level - 1, sourceIds, sourceRange: { first, last }, + created: Number(id.replace(/\D/g, '')) || 1, +}); + +describe('compression refusals: the recall variants', () => { + it('each variant over the compression budget is refused before any call', async () => { + const rec = recorder(); + const { calls, membrane } = scripted([REFUSAL(90_000), ANSWER('must never be called')]); + const strategy = new RecallProbe({ + compressionModel: 'same-model', targetChunkTokens: 100, recentWindowTokens: 0, headWindowTokens: 0, + autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + compressionRefusalCurveFallbacks: 3, compressionContextBudgetTokens: 50_000, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, observer: rec.observer }); + const ids: string[] = []; + for (let i = 0; i < 14; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substantive '.repeat(18))])); + const children = [0, 2, 4, 6].map((a, k) => summaryOf(`L1-10${k}`, 1, [ids[a]!, ids[a + 1]!], ids[a]!, ids[a + 1]!)); + for (const child of children) strategy.seed(child); + const older = summaryOf('L2-200', 2, [children[1]!.id, children[0]!.id], ids[0]!, ids[3]!); + const newer = summaryOf('L2-201', 2, [children[3]!.id, children[2]!.id], ids[4]!, ids[7]!); + strategy.seed(older); + strategy.seed(newer); + for (const child of children.slice(0, 2)) strategy.linkChild(child, older.id); + for (const child of children.slice(2)) strategy.linkChild(child, newer.id); + const want = new Set(ids.slice(10, 12)); + const target: Chunk = { index: 999, startIndex: 10, endIndex: 12, messages: strategyContext(manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; + await strategy.run(target, strategyContext(manager)); + assert.equal(calls.length, 1, 'no variant is sent'); + // One for each variant the plan holds: each of the two parents expanded. + assert.deepEqual(rec.compressions, [{ reason: 'admission-rejected' }, { reason: 'admission-rejected' }]); + manager.close(); + }); +}); + +// ---- Compression: merges ---- + +class MergeProbe extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { this.pushSummary(entry); } + qMerge(level: number, sourceIds: string[]): void { this.enqueueMerge({ level, sourceIds }); } + summariesView(): SummaryEntry[] { return [...this.summaries]; } +} + +/** Two seeded L1s over real messages, and a merge of them queued at `level`. */ +async function merge(membrane: unknown, level = 2, observe: ContextObserver | 'none' | 'record' = 'record') { + const rec = recorder(); + const observer = observe === 'record' ? rec.observer : observe === 'none' ? undefined : observe; + const strategy = new MergeProbe({ + compressionModel: 'same-model', hierarchical: true, targetChunkTokens: 100_000, recentWindowTokens: 0, headWindowTokens: 0, + autoTickOnNewMessage: false, mergeThreshold: 99, quarantineAlarmIntervalMs: 0, mergeAttemptLimit: 5, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, observer }); + const ids: string[] = []; + for (let i = 0; i < 8; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substance '.repeat(10))])); + strategy.seed(summaryOf('L1-100', 1, [ids[0]!, ids[1]!], ids[0]!, ids[1]!)); + strategy.seed(summaryOf('L1-101', 1, [ids[2]!, ids[3]!], ids[2]!, ids[3]!)); + strategy.qMerge(level, ['L1-100', 'L1-101']); + return { manager, strategy, rec }; +} + +describe('compression refusals: merges', () => { + const cases: Array<[string, Answer, CompressionRefusalRecord[]]> = [ + ['a refusal, membrane\'s', REFUSAL(), []], + ['a provider error, membrane\'s', { error: Object.assign(new Error('bad request'), { type: 'invalid_request', httpStatus: 400, retryable: false }) }, []], + ['an answer cut short', ANSWER('a partial consolidation', 'max_tokens'), [{ reason: 'incomplete' }]], + ['an answer with no text', ANSWER(''), [{ reason: 'empty' }]], + ['an answer of no response\'s shape', { stopReason: 'end_turn', content: 'not a list', usage: { inputTokens: 80, outputTokens: 20 } }, [{ reason: 'malformed' }]], + ]; + for (const [name, answer, expected] of cases) { + it(`${name}: ${expected.length ? expected[0]!.reason : 'nothing reported'}`, async () => { + const { calls, membrane } = scripted([answer]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)).catch(() => {}); + assert.equal(calls.length, 1, 'the merge was asked once'); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 0, 'nothing minted'); + assert.deepEqual(fx.rec.compressions, expected); + fx.manager.close(); + }); + } + + it('a merge answer that is a thinking preamble alone is empty', async () => { + const { calls, membrane } = scripted([ANSWER('only my own scratch, nothing to keep')]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)).catch(() => {}); + assert.equal(calls.length, 1); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 0, 'nothing minted'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + + it('an observer that throws or rejects at the merge gate changes nothing the merge does', async () => { + const outcomes: unknown[] = []; + const told: string[][] = [[], []]; + const stray = await unhandledRejections(async () => { + for (const observe of ['none', hostile('throws', told[0]), hostile('rejects', told[1])] as const) { + const { calls, membrane } = scripted([ANSWER('a partial consolidation', 'max_tokens')]); + const fx = await merge(membrane, 2, observe); + await fx.strategy.tick(strategyContext(fx.manager)); + outcomes.push([calls.length, fx.strategy.summariesView().filter((s) => s.level === 2).length, fx.manager.getStore().getStateJson('default/autobio:mergeQueue')]); + fx.manager.close(); + } + }); + assert.equal((outcomes[0] as unknown[])[0], 1, 'the merge made its call'); + assert.deepEqual(outcomes[1], outcomes[0]); + assert.deepEqual(outcomes[2], outcomes[0]); + for (const calls of told) assert.ok(calls.includes('compression'), `the gate's refusal reached the hostile observer: ${calls.join(', ')}`); + assert.deepEqual(stray, []); + }); + + it('an accepted merge reports nothing', async () => { + const { membrane } = scripted([ANSWER('A real consolidation.')]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a merge over sources of the wrong level is refused before any call', async () => { + const { calls, membrane } = scripted([ANSWER('must never be called')]); + const fx = await merge(membrane, 3); + await fx.strategy.tick(strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.deepEqual(fx.rec.compressions, [{ reason: 'topology-violation' }]); + fx.manager.close(); + }); +}); + +// ---- Calibration samples ---- + +type CalibrationInternals = { + _calibration: number; + _calibrationArmed: boolean; + _lastCompileEstimate: number; + _calibrationProvisional: { fromEpoch: number | null } | null; + _calibrationDeclared: boolean; + calibrationStateId: string; + store: { setStateJson(id: string, value: unknown): void } | null; +}; + +async function calibrated(observe: ContextObserver | 'none' | 'record' = 'record', seeded: number | null = 1) { + const rec = recorder(); + const observer = observe === 'record' ? rec.observer : observe === 'none' ? undefined : observe; + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const internals = strategy as unknown as CalibrationInternals; + // A learned multiplier (1 unless asked), from a calibration record the + // first compile loads; with none, the store's multiplier is a provisional 1 + // (#328), which a sample can also ask for. + if (seeded !== null) internals.store!.setStateJson(internals.calibrationStateId, { multiplier: seeded, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + /** + * Compile, then report `ratio` times the compile's estimate as the + * provider's total, with a provisional multiplier if asked, and declaring + * what the request sent besides the window (nothing) if asked. + */ + const sample = async (ratio: number, provisional = false, declared = false) => { + await manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + if (provisional) internals._calibrationProvisional = { fromEpoch: null }; + const estimate = internals._lastCompileEstimate; + assert.ok(estimate > 0); + strategy.reportRealInputTokens(estimate * ratio, declared ? {} : undefined); + }; + return { manager, strategy, internals, rec, sample }; +} + +describe('calibration samples', () => { + it('one record per sample, with its outcome, ratio and the multiplier after', async () => { + const fx = await calibrated(); + await fx.sample(3); + await fx.sample(1.01); + await fx.sample(1.5); + // A second report after one compile is no sample. + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 1.5); + const outcomes = fx.rec.samples.map((s) => s.outcome); + assert.deepEqual(outcomes, ['out-of-band', 'held', 'stepped']); + for (const [record, ratio] of fx.rec.samples.map((r, i) => [r, [3, 1.01, 1.5][i]!] as const)) { + assert.ok(Math.abs(record.ratio - ratio) < 1e-9, `${record.outcome}: ratio ${record.ratio}, given ${ratio}`); + } + assert.equal(fx.rec.samples[0]!.multiplier, 1); + assert.equal(fx.rec.samples[1]!.multiplier, 1); + // The held sample moved the average to 1.002, and the step goes a fifth of the way from there to 1.5. + assert.ok(Math.abs(fx.rec.samples[2]!.multiplier - (1.002 + 0.2 * (1.5 - 1.002))) < 1e-9, String(fx.rec.samples[2]!.multiplier)); + assert.equal(fx.rec.samples[2]!.multiplier, fx.internals._calibration); + fx.manager.close(); + }); + + it('a provisional multiplier is confirmed in the dead band, and replaced outside it', async () => { + const confirm = await calibrated(); + await confirm.sample(1.01, true); + assert.deepEqual(confirm.rec.samples.map((s) => [s.outcome, s.multiplier]), [['confirmed', 1]]); + assert.ok(Math.abs(confirm.rec.samples[0]!.ratio - 1.01) < 1e-9); + confirm.manager.close(); + + const replace = await calibrated(); + await replace.sample(1.3, true); + assert.equal(replace.rec.samples.length, 1); + assert.equal(replace.rec.samples[0]!.outcome, 'replaced'); + assert.ok(Math.abs(replace.rec.samples[0]!.multiplier - 1.3) < 1e-9, String(replace.rec.samples[0]!.multiplier)); + assert.ok(Math.abs(replace.rec.samples[0]!.ratio - 1.3) < 1e-9); + replace.manager.close(); + }); + + it('a declared out-of-band run is out-of-band until its last sample, learned at the clamp as replaced; an undeclared one stays out-of-band', async () => { + const declared = await calibrated(); + for (let i = 0; i < 3; i++) await declared.sample(3, false, true); + assert.deepEqual(declared.rec.samples.map((s) => [s.outcome, s.multiplier]), [['out-of-band', 1], ['out-of-band', 1], ['replaced', 1.8]]); + assert.equal(declared.internals._calibration, 1.8); + declared.manager.close(); + + const undeclared = await calibrated(); + for (let i = 0; i < 3; i++) await undeclared.sample(3); + assert.deepEqual(undeclared.rec.samples.map((s) => [s.outcome, s.multiplier]), [['out-of-band', 1], ['out-of-band', 1], ['out-of-band', 1]]); + undeclared.manager.close(); + + // At the 1.8 clamp, a declared run implying 3.6 is learned at the clamp, within the dead band: held. + // Each record's ratio is the provider's total over the estimate, 2, not the 3.6 it implies. + const clamped = await calibrated('record', 1.8); + for (let i = 0; i < 3; i++) await clamped.sample(2, false, true); + assert.deepEqual(clamped.rec.samples.map((s) => [s.outcome, s.multiplier]), [['out-of-band', 1.8], ['out-of-band', 1.8], ['held', 1.8]]); + for (const record of clamped.rec.samples) assert.ok(Math.abs(record.ratio - 2) < 1e-9, `${record.outcome}: ratio ${record.ratio}`); + clamped.manager.close(); + + // The same run on a provisional multiplier: confirmed, as any sample in the dead band of one is. + const provisional = await calibrated('record', 1.8); + for (let i = 0; i < 3; i++) await provisional.sample(2, true, true); + assert.deepEqual(provisional.rec.samples.map((s) => [s.outcome, s.multiplier]), [['out-of-band', 1.8], ['out-of-band', 1.8], ['confirmed', 1.8]]); + provisional.manager.close(); + }); + + it('the first declared sample on a multiplier learned before declarations is deferred, and the next decides', async () => { + const agree = await calibrated(); + await agree.sample(1.3, false, true); + assert.equal(agree.internals._calibrationDeclared, false, 'the loaded record is unmarked, and the held sample leaves it so'); + await agree.sample(1.32, false, true); + assert.deepEqual(agree.rec.samples.map((s) => s.outcome), ['deferred', 'replaced']); + assert.equal(agree.rec.samples[0]!.multiplier, 1, 'the multiplier stays while the sample is held'); + assert.ok(Math.abs(agree.rec.samples[0]!.ratio - 1.3) < 1e-9); + assert.ok(Math.abs(agree.rec.samples[1]!.multiplier - 1.31) < 1e-9, `the mean of the two: ${agree.rec.samples[1]!.multiplier}`); + agree.manager.close(); + + const disagree = await calibrated(); + await disagree.sample(1.3, false, true); + await disagree.sample(1.6, false, true); + assert.deepEqual(disagree.rec.samples.map((s) => s.outcome), ['deferred', 'stepped']); + assert.ok(Math.abs(disagree.rec.samples[1]!.multiplier - (1 + 0.2 * (1.6 - 1))) < 1e-9, String(disagree.rec.samples[1]!.multiplier)); + disagree.manager.close(); + + // A store whose multiplier was already right: the pair's mean is within the dead band, so it's held, and the record is marked. + const right = await calibrated(); + await right.sample(1.0, false, true); + await right.sample(1.01, false, true); + assert.deepEqual(right.rec.samples.map((s) => [s.outcome, s.multiplier]), [['deferred', 1], ['held', 1]]); + assert.equal(right.internals._calibrationDeclared, true); + // An undeclared sample that moves the multiplier takes the mark back, so the next declared sample is held again. + await right.sample(1.5); + assert.equal(right.internals._calibrationDeclared, false); + await right.sample(1.1, false, true); + assert.deepEqual(right.rec.samples.slice(2).map((s) => s.outcome), ['stepped', 'deferred']); + right.manager.close(); + + // On a learned 1.2, the held sample's record carries its ratio, 1.1, not the 1.32 it implies, and the 1.2 in force. + const scaled = await calibrated('record', 1.2); + await scaled.sample(1.1, false, true); + assert.equal(scaled.rec.samples.length, 1); + assert.equal(scaled.rec.samples[0]!.outcome, 'deferred'); + assert.ok(Math.abs(scaled.rec.samples[0]!.ratio - 1.1) < 1e-9, String(scaled.rec.samples[0]!.ratio)); + assert.equal(scaled.rec.samples[0]!.multiplier, 1.2); + scaled.manager.close(); + + // A declared sample rejected out of band between the two leaves the held one, and the next in band decides. + const between = await calibrated(); + await between.sample(1.3, false, true); + await between.sample(3, false, true); + await between.sample(1.32, false, true); + assert.deepEqual(between.rec.samples.map((s) => s.outcome), ['deferred', 'out-of-band', 'replaced']); + assert.ok(Math.abs(between.rec.samples[2]!.multiplier - 1.31) < 1e-9, String(between.rec.samples[2]!.multiplier)); + between.manager.close(); + }); + + it('a store with no calibration record starts provisional: its first sample is confirmed or replaced', async () => { + const confirm = await calibrated('record', null); + await confirm.sample(1.01); + assert.deepEqual(confirm.rec.samples.map((s) => [s.outcome, s.multiplier]), [['confirmed', 1]]); + confirm.manager.close(); + + const replace = await calibrated('record', null); + await replace.sample(1.3); + assert.deepEqual(replace.rec.samples.map((s) => s.outcome), ['replaced']); + assert.ok(Math.abs(replace.rec.samples[0]!.multiplier - 1.3) < 1e-9, String(replace.rec.samples[0]!.multiplier)); + replace.manager.close(); + + // Declared or not: the first sample replaces the provisional 1 outright, without the two-sample hold. + const declared = await calibrated('record', null); + await declared.sample(1.3, false, true); + assert.deepEqual(declared.rec.samples.map((s) => s.outcome), ['replaced']); + assert.ok(Math.abs(declared.rec.samples[0]!.multiplier - 1.3) < 1e-9, String(declared.rec.samples[0]!.multiplier)); + declared.manager.close(); + }); + + it('a ratio out of band whose implied multiplier is in band is learned, not out-of-band', async () => { + // At the 1.8 clamp, an honest sample reads about 0.56 of the estimate: out of band as a ratio, but it implies 1.008. + const fx = await calibrated(); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + fx.internals._calibration = 1.8; + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 0.56); + assert.equal(fx.rec.samples.length, 1); + assert.equal(fx.rec.samples[0]!.outcome, 'stepped'); + assert.ok(Math.abs(fx.rec.samples[0]!.ratio - 0.56) < 1e-9); + assert.ok(Math.abs(fx.rec.samples[0]!.multiplier - (1.8 + 0.2 * (0.56 * 1.8 - 1.8))) < 1e-9, String(fx.rec.samples[0]!.multiplier)); + fx.manager.close(); + }); + + it('an observed preview leaves the strategy as an unobserved one does: its options reach the work', async () => { + for (const observe of ['none', 'record'] as const) { + const fx = await calibrated(observe); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }, undefined, { dryRun: true }); + assert.equal(fx.internals._calibrationArmed, false, `${observe}: a preview arms no sample`); + fx.strategy.reportRealInputTokens(5_000); + assert.deepEqual(fx.rec.samples, [], `${observe}: so a report after it is no sample`); + if (observe === 'record') assert.equal(fx.rec.compiles.at(-1)!.dryRun, true); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + assert.equal(fx.internals._calibrationArmed, true, `${observe}: a live compile arms one`); + fx.manager.close(); + } + }); + + it('a report that is no sample makes no record and leaves the sample armed', async () => { + const fresh = await calibrated(); + fresh.strategy.reportRealInputTokens(5_000); + assert.deepEqual(fresh.rec.samples, [], 'no compile, so no estimate to compare'); + fresh.manager.close(); + + const fx = await calibrated(); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + for (const total of [0, -5, Number.NaN, Number.POSITIVE_INFINITY]) fx.strategy.reportRealInputTokens(total); + assert.equal(fx.rec.samples.length, 0); + assert.equal(fx.internals._calibrationArmed, true, 'the compile\'s sample is still to come'); + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 1.01); + assert.deepEqual(fx.rec.samples.map((r) => r.outcome), ['held']); + fx.manager.close(); + }); + + it('an observer that throws or rejects changes nothing a sample does', async () => { + const multipliers: number[] = []; + const told: string[][] = [[], []]; + const stray = await unhandledRejections(async () => { + for (const observe of ['none', hostile('throws', told[0]), hostile('rejects', told[1])] as const) { + const fx = await calibrated(observe); + await fx.sample(3); + await fx.sample(1.5); + multipliers.push(fx.internals._calibration); + fx.manager.close(); + } + }); + assert.equal(multipliers[1], multipliers[0]); + assert.equal(multipliers[2], multipliers[0]); + assert.notEqual(multipliers[0], 1, 'the stepped sample moved it'); + for (const calls of told) assert.equal(calls.filter((c) => c === 'calibration').length, 2, `both samples reached the hostile observer: ${calls.join(', ')}`); + assert.deepEqual(stray, []); + }); +}); diff --git a/test/fixtures/over-budget-hook.ts b/test/fixtures/over-budget-hook.ts new file mode 100644 index 00000000..774c2033 --- /dev/null +++ b/test/fixtures/over-budget-hook.ts @@ -0,0 +1,53 @@ +/** + * Records what each refusal through `AutobiographicalStrategy`'s protected + * `overBudgetError` hook was given, so a test can check that an override of + * the hook still gets a library stage's words as `stage`, as it did before + * `stageId`. Importing this wraps the hook for the importing test file. + */ +import assert from 'node:assert/strict'; +import { AutobiographicalStrategy, OverBudgetError } from '../../src/index.js'; + +type HookOptions = { stage?: string; stageId?: string }; + +/** The words each of AutobiographicalStrategy's stages gave the hook before stageId, as `main` gave them. */ +const HOOK_WORDS: Record = { + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', +}; + +const given = new WeakMap(); +const proto = AutobiographicalStrategy.prototype as unknown as { + overBudgetError(budget: unknown, opts: HookOptions): OverBudgetError; +}; +const original = proto.overBudgetError; +proto.overBudgetError = function (this: unknown, budget: unknown, opts: HookOptions): OverBudgetError { + const error = original.call(this, budget, opts); + given.set(error, { stage: opts.stage, stageId: opts.stageId }); + return error; +}; + +/** + * `error` is an `OverBudgetError` naming `stageId`. A refusal of + * `AutobiographicalStrategy`'s (any but passthrough's and windowed + * passthrough's) came through the hook, which was given that id and exactly + * the words `main` gave it, which open its message, or, for the picker's own + * refusal, neither, as before. + */ +export function assertStage(error: unknown, stageId: string): void { + assert.ok(error instanceof OverBudgetError, String(error)); + assert.equal(error.stageId, stageId); + const hook = given.get(error); + if (stageId.startsWith('windowed-') || stageId.startsWith('passthrough-')) return; + assert.ok(hook !== undefined, `${stageId}: the refusal came through the hook`); + if (stageId === 'picker-exhausted') { + assert.deepEqual(hook, { stage: undefined, stageId: undefined }, 'the picker\'s refusal gives the hook no stage'); + return; + } + assert.equal(hook.stageId, stageId); + assert.equal(hook.stage, HOOK_WORDS[stageId], `${stageId}: the hook was given the words main gave it`); + assert.ok(error.message.startsWith(`${hook.stage} but `), `the words open the message: ${error.message.slice(0, 80)}`); +} diff --git a/test/fixtures/recall-envelope-fixture.ts b/test/fixtures/recall-envelope-fixture.ts index cdc75d95..2eb632f5 100644 --- a/test/fixtures/recall-envelope-fixture.ts +++ b/test/fixtures/recall-envelope-fixture.ts @@ -25,7 +25,7 @@ import { join } from 'node:path'; import type { ContentBlock } from '@animalabs/membrane'; import { ContextManager, AutobiographicalStrategy } from '../../src/index.js'; -import type { AutobiographicalOptions, SummaryEntry } from '../../src/types/index.js'; +import type { AutobiographicalOptions, SelectOptions, SummaryEntry } from '../../src/types/index.js'; export interface RenderedMessage { participant: string; @@ -51,6 +51,9 @@ export interface FixtureRender { * Empty for fixtures that seed summaries instead of minting them. */ compressionRequests: RenderedMessage[][]; + /** The strategy's calibration estimate of the compiled request, where the + * fixture reads it (renderAdaptiveFixture). */ + estimate?: number; } /** Strategy subclass that can seed summaries without an LLM round trip. */ @@ -238,6 +241,11 @@ export async function renderAdaptiveFixture( options: AutobiographicalOptions = {}, responseContentForCall: (callCount: number) => ContentBlock[] = adaptiveTextResponseContent, maxTokens = 900, + selectOptions?: SelectOptions, + /** Options for a committing compile before the one rendered, if any. */ + firstSelectOptions?: SelectOptions, + /** Called with the manager after the rendered compile. */ + inspect?: (manager: ContextManager) => void | Promise, ): Promise { return withTempStore(async (path) => { const strategy = new AutobiographicalStrategy({ @@ -262,9 +270,12 @@ export async function renderAdaptiveFixture( // Drain compression to a fixed point: eight ticks take this workload // past L1 into merged L2s, so the adaptive render carries both levels. for (let tick = 0; tick < 8; tick++) await manager.tick(); - const compiled = await manager.compile({ maxTokens, reserveForResponse: 0 }); + if (firstSelectOptions) await manager.compile({ maxTokens, reserveForResponse: 0 }, undefined, firstSelectOptions); + const compiled = await manager.compile({ maxTokens, reserveForResponse: 0 }, undefined, selectOptions); const summaries = summariesOf(strategy); + const estimate = (strategy as unknown as { _lastCompileEstimate: number })._lastCompileEstimate; + await inspect?.(manager); manager.close(); - return { messages: project(compiled.messages), summaries, compressionRequests: mock.requests }; + return { messages: project(compiled.messages), summaries, compressionRequests: mock.requests, estimate }; }); } diff --git a/test/image-stripping.test.ts b/test/image-stripping.test.ts index a590da3e..760640f3 100644 --- a/test/image-stripping.test.ts +++ b/test/image-stripping.test.ts @@ -108,15 +108,20 @@ describe('Live image policy', () => { hierarchical: true, maxLiveImages: 0, imageStripDepthTokens: 0, + maxLiveImageBytes: 0, + // The image ceilings bound an unlimited window too. + imageCeiling: 0, + imageByteCeiling: 0, }); const manager = await ContextManager.open({ path: TEST_STORE_PATH, strategy }); - for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-0${i}`)); + // Eight, more than the default window of 6. + for (let i = 1; i <= 8; i++) manager.addMessage('user', imageMessage(`img-0${i}`)); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - for (const label of ['img-01', 'img-02', 'img-03']) { - assert.equal(imageState(compiled.messages, label), 'image', `${label} kept when policy disabled`); + for (let i = 1; i <= 8; i++) { + assert.equal(imageState(compiled.messages, `img-0${i}`), 'image', `img-0${i} kept when policy disabled`); } manager.close(); diff --git a/test/kv-unified-render-state.test.ts b/test/kv-unified-render-state.test.ts index 71aeae96..05cc1182 100644 --- a/test/kv-unified-render-state.test.ts +++ b/test/kv-unified-render-state.test.ts @@ -1218,3 +1218,33 @@ describe('a reference loaded from the store', () => { }); } }); + +describe('system changes before a reset head', () => { + it("they lead the head, and the check's rebuild puts them there, as the emission sends them", async () => { + const { dir, strategy, manager } = await open({ headWindowTokens: 4_000, recentWindowTokens: 4_000 }); + try { + const change = { type: 'system_change', addSystem: ['Answer in metric units.'] } as unknown as ContentBlock; + add(manager, [['u', [text('a')]], ['Harness', [change]], ['a', [text('b')]], ...MIDDLE, ...CLOSE]); + strategy.resetHeadWindow(ids(manager)[3]!); + const one = await compile(manager); + assert.deepEqual(one[0]!.content, [change], 'the change before the head leads'); + assert.deepEqual(one[1]!.content, manager.getMessage(ids(manager)[3]!)!.content, 'then the head window'); + accept(strategy, one); + const ref = reference(strategy)!; + assert.ok(ref.markers.length > 0, 'premise: the accepted request has markers'); + const hooks = strategy as unknown as { kvUnifiedKeepEntries(...args: unknown[]): unknown }; + const keep = hooks.kvUnifiedKeepEntries.bind(strategy); + let rebuilt = 0; + hooks.kvUnifiedKeepEntries = (...args) => { rebuilt++; return keep(...args); }; + add(manager, SMALL); + const two = await compile(manager); + assert.ok(rebuilt > 0, 'premise: the check rebuilt'); + assert.equal(firstChange(one, two), one.length, 'premise: nothing the accepted request sent changes'); + assert.equal(solveMarkers(strategy), ref.markers.length, 'every accepted marker survives the check'); + // afterEach: the self-check found the rebuild matching what was sent. + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/test/l1-recall-raw-pins-frontier.test.ts b/test/l1-recall-raw-pins-frontier.test.ts index d1f7de68..e8920588 100644 --- a/test/l1-recall-raw-pins-frontier.test.ts +++ b/test/l1-recall-raw-pins-frontier.test.ts @@ -225,7 +225,7 @@ describe('raw pins open only their own branch of the L1 recall frontier', () => cm.close(); }); - it('a pin after the chunk being compressed opens nothing: its summary is recalled, as on main', async () => { + it('a pin after the chunk being compressed opens nothing, and its summary, which comes after the chunk, is not recalled (#317)', async () => { // A message the chunker skipped while it was pinned raw, then unpinned, is // compressed out of order, after the history that follows it. let skippedPin = ''; @@ -247,7 +247,9 @@ describe('raw pins open only their own branch of the L1 recall frontier', () => const request = requests.find(compressing); assert.ok(request, 'setup: the skipped message was compressed'); const texts = textsOf(request); - assert.ok(texts.includes(`[CM] Recall memory ${root.id}.`), 'the summary holding the later pin is recalled'); + const rootFirst = ids.indexOf((root as unknown as { sourceRange: { first: string } }).sourceRange.first); + assert.ok(rootFirst > 30, `setup: the summary holding the later pin starts after the chunk (at ${rootFirst})`); + assert.ok(!texts.includes(`[CM] Recall memory ${root.id}.`), 'the summary holding the later pin, after the chunk, is not recalled: the request holds only what came before'); assert.ok(!texts.includes(textOf.get(ids[100])!), 'the later pin adds no raw'); cm.close(); }); @@ -835,7 +837,7 @@ describe('the raw pins add is spent from the recall budget', () => { }); describe('a pin resolves once, for compression and every selector', () => { - const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}` })); + const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}`, content: [] })); const pins: Array & { at: number }> = [ { at: 1 }, // classic raw pin { at: 2, level: 0 }, diff --git a/test/overbudget-diagnostics.test.ts b/test/overbudget-diagnostics.test.ts index 81b36e04..b8c7c0b1 100644 --- a/test/overbudget-diagnostics.test.ts +++ b/test/overbudget-diagnostics.test.ts @@ -4,6 +4,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { AutobiographicalStrategy, ContextManager, OverBudgetError } from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; import type { StoredMessage, SummaryEntry, SummaryLevel } from '../src/types/index.js'; import { Picker, type PickerInputs } from '../src/adaptive/picker.js'; import type { FoldingBudget } from '../src/adaptive/folding-strategy.js'; @@ -503,6 +504,7 @@ async function refusalOfWholeMiddle(manager: ContextManager): Promise { assert.doesNotMatch(error.message, /deepest planned level/); } else { assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.match(error.message, /attempted representation level=L0/); } }); @@ -650,6 +653,7 @@ describe('OverBudgetError diagnostics', () => { strategy.seedPyramid([id], 5); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); + assertStage(error, 'emission-overran-plan'); checkBudget(error); assert.deepEqual(error.clearedBy, [], 'the plan has no summaries left to produce'); assert.equal(error.diagnostics.deepestLevel, 5, 'the first raw emission fails, but the complete plan reaches L5'); @@ -1022,6 +1026,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); + assertStage(error, 'emission-overran-plan'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -1043,10 +1048,12 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Tail emission dropped reserved recent-window messages/); + assertStage(error, 'tail-emission-dropped'); assert.deepEqual(error.clearedBy, ['compression']); strategy.produce = false; const without = await refusal(manager); assert.match(without.message, /^Tail emission dropped reserved recent-window messages/); + assertStage(without, 'tail-emission-dropped'); assert.deepEqual(without.clearedBy, [], 'a plan with nothing to produce names nothing'); }); }); @@ -1112,6 +1119,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Structural repair did not retain the newest turn/); + assertStage(dry, 'repair-lost-newest-turn'); assert.deepEqual(dry.clearedBy, []); // The live compile refuses the plan at its gate first, so the preview // names what that refusal names. @@ -1223,9 +1231,11 @@ describe('OverBudgetError.clearedBy', () => { for (let i = 0; i < 6; i++) manager.addMessage('user', [{ type: 'text', text: `message ${i} ` + 'word '.repeat(30) }]); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Emission overran the plan/); + assertStage(dry, 'emission-overran-plan'); assert.deepEqual(dry.clearedBy, []); const live = await refusal(manager); assert.match(live.message, /^Emission overran the plan/); + assertStage(live, 'emission-overran-plan'); assert.deepEqual(live.clearedBy, ['compression'], 'the live compile demands the chunk first'); }); }); @@ -1246,6 +1256,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are waiting for compression'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -1684,7 +1695,7 @@ describe('OverBudgetError.clearedBy', () => { }); } - it('before the host declares its tools, counts no queued chunk that starts inside the head window when the head carries a tool cycle after the chunk\'s start, which tick\'s head carries raw', async () => { + it('before the host declares its tools, counts no queued chunk that lies inside the head window, and tick asks the model for the first: its request carries the head only up to the chunk, not the tool cycle after it (#317)', async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1712,19 +1723,22 @@ describe('OverBudgetError.clearedBy', () => { const queued = await settledQueue(manager, strategy); assert.ok(!queued[0].some((id) => cycle.includes(id)), 'the first queued chunk is text ahead of the cycle'); strategy.setHeadWindowTokens(100_000); - assert.equal(strategy.countsCompressionWork(), false, 'tick defers every queued chunk until the tools come'); - // The premise: tick's head carries the cycle raw, so it defers each queued chunk, with no model call. - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); - assert.equal(calls.n, 0, 'tick asked no model'); - assert.deepEqual(strategy.summaryIds(), [], 'and compressed none'); + assert.equal(strategy.countsCompressionWork(), false, 'chunks inside the head fold nothing, so none counts'); + // The head a request gives is cut at its chunk's start (headAsOf), so the cycle after the first chunk isn't in + // its request: tick asks the model rather than deferring, and the counting membrane throws. Before #317 the + // head went whole, messages after the chunk included. + await assert.rejects(async () => { + for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); + }, /no model in this test/); + assert.equal(calls.n, 1, 'tick asked the model'); }, { membrane: countingMembrane(calls) }); }); // The chunks the test above widens the head over lie wholly inside it, so they fold nothing and don't count - // whatever the tools say. A chunk that lies before a reset head start folds, and the head after it, which - // tick's request carries raw, is what holds it back: a tool message in the head lies past the chunk's start. - it('before the host declares its tools, counts no queued chunk that lies before a reset head start when the head carries a tool cycle, which tick\'s request carries raw though it lies after the chunk', async () => { + // whatever the tools say. A chunk that lies before a reset head start folds. Tick's request for it gives the + // head that history followed, the opening here, and not the reset head after it (#317), so a tool cycle in the + // reset head doesn't hold it back. + it('before the host declares its tools, names compression for a queued chunk that lies before a reset head start when that head carries a tool cycle, which lies after the chunk and tick\'s request leaves out (#317)', async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1756,14 +1770,13 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(chunkAt.every((i) => i >= 0 && i < headStart), 'the premise: the first queued chunk lies before the head window\'s start, so it folds'); assert.ok(!queued[0]!.some((id) => cycle.includes(id)), 'the premise: and holds no tool block of its own'); assert.ok(cycleAt.every((i) => i >= headStart && i < headEnd), 'the premise: the cycle sits in the head window, past the chunk\'s start'); - assert.deepEqual(error.clearedBy, [], 'tick defers every queued chunk until the tools come'); - // The premise: tick's request carries the head raw, cycle included, so it defers each queued chunk, with no model call. - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); - assert.equal(calls.n, 0, 'tick asked no model'); - assert.deepEqual(strategy.summaryIds(), [], 'and compressed none'); - manager.setToolDefinitions([{ name: 'zz_tool', description: 'zz', inputSchema: { type: 'object' } }]); - assert.deepEqual((await refusal(manager)).clearedBy, ['compression'], 'with the tools declared, they count'); + assert.deepEqual(error.clearedBy, ['compression'], 'the chunk folds and tick will take it without the tools'); + // The premise: tick's request gives the opening as its head, without the cycle, so tick asks the model, and the + // counting membrane throws. + await assert.rejects(async () => { + for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); + }, /no model in this test/); + assert.equal(calls.n, 1, 'tick asked the model'); }, { membrane: countingMembrane(calls) }); }); @@ -1814,7 +1827,7 @@ describe('OverBudgetError.clearedBy', () => { } for (const pinned of [false, true]) { - it(`names ${pinned ? 'nothing' : 'compression'} before the host declares its tools for text chunks after a tool cycle no L1 covers, before the head window's start, ${pinned ? 'pinned, which tick\'s request then carries raw' : 'which tick\'s request leaves out'}`, async () => { + it(`names nothing before the host declares its tools for text chunks after a tool cycle no L1 covers, before the head window's start, ${pinned ? 'pinned' : 'unpinned'}: tick's request carries the cycle raw in the opening, the head of that history (#317)`, async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1841,17 +1854,10 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(indices.every((i) => i >= 0 && i < headStart), 'the premise: the cycle sits before the head window\'s start'); const queued = strategy.queuedChunkIds(); assert.ok(queued.some((ids) => !ids.some((id) => cycle.includes(id))), 'text chunks are queued'); - if (!pinned) { - assert.deepEqual(error.clearedBy, ['compression']); - // The premise: tick's request leaves the cycle out, so tick asks the model, and the counting membrane throws. - await assert.rejects(async () => { - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - }, /no model in this test/); - assert.equal(calls.n, 1, 'tick asked the model'); - return; - } assert.deepEqual(error.clearedBy, [], 'nothing tick will do clears it'); - // The premise: tick's request carries the pinned cycle raw, so tick defers every queued chunk, with no model call. + // The premise: tick's request for a chunk before the reset gives the opening as its head, cycle included, raw + // (pinned or not), so tick defers every queued chunk, with no model call. Before #317 an unpinned cycle there + // was left out, since the head was the reset one. for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); assert.equal(calls.n, 0, 'tick asked no model'); @@ -2039,6 +2045,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.seedL1('L1-covering', ids); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // Tick drops each one without compressing it: the stub membrane has no // model to call, and no L1 is added. @@ -2094,6 +2101,7 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(!strategy.summaryIds().some((id) => id.startsWith('L1-stored-')), 'no exact L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2113,6 +2121,7 @@ describe('OverBudgetError.clearedBy', () => { }); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, [], 'adoption doesn\'t persist, so the stored L1s shrink nothing'); }, { membrane: true }); }); @@ -2127,6 +2136,7 @@ describe('OverBudgetError.clearedBy', () => { assert.deepEqual(strategy.summaryIds(), [], 'no L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2158,6 +2168,7 @@ describe('OverBudgetError.clearedBy', () => { queued.forEach((ids, i) => strategy.seedL1(`L1-exact-${i}`, ids)); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(strategy.queuedChunkIds(), queued, 'the compile closed no new chunk'); assert.deepEqual(error.clearedBy, persistChunks ? ['compression'] : []); if (!persistChunks) return; @@ -2454,6 +2465,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'the merge is queued'); const error = await refusal(manager, undefined, positionedRecallPairs ? undefined : PAIR_FITS); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.equal(Reflect.get(error.diagnostics, 'middleCountUnit'), positionedRecallPairs ? 'selection-items' : 'raw-messages', 'the pair, or with the combined pair fitting, the raw message after it, refuses'); assert.deepEqual(error.clearedBy, [], 'tick stops at a blocked merge until the hold is released'); @@ -2498,6 +2510,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are queued'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, merge === 'blocked' ? [] : ['compression']); }); }); @@ -2641,6 +2654,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: tick still takes them, adopting each stored L1. await adoptAll(manager, strategy); @@ -2658,6 +2672,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: adopting the exact L1s leaves the render as it was, where unpinned chunks' adoption shrinks it. const before = (await manager.compile({ maxTokens: 100_000, reserveForResponse: 20 })).messages.length; @@ -2782,6 +2797,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2796,6 +2812,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2809,6 +2826,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts new file mode 100644 index 00000000..e0715636 --- /dev/null +++ b/test/request-image-ceilings.test.ts @@ -0,0 +1,1268 @@ +/** + * The image ceilings (imageCeiling, imageByteCeiling): the most images, and + * their bytes, a strategy's live window keeps, and the most + * images and bytes a compression prompt carries. The live window keeps its + * own limits within them, compression prompts carry no more than them, and + * imageCeilings() reports them. + */ +import { describe, it } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, WindowedPassthroughStrategy } from '../src/index.js'; +import type { ContextStrategy, ImageCeilings } from '../src/index.js'; +import { ceilingBoundLine, resolveLiveImageLimits } from '../src/live-image-limits.js'; + +const MiB = 1024 * 1024; +const PLACEHOLDER = '[image not shown: this context keeps only some images]'; +/** What every ceiling line says, and no other line. */ +const CEILING_LINE = 'so some images are now shown as placeholders'; + +/** A message holding one image of `bytes` base64 characters and a label. */ +const imageMessage = (label: string, bytes = 4): ContentBlock[] => [ + { type: 'image', source: { type: 'base64', data: 'A'.repeat(bytes), mediaType: 'image/png' }, tokenEstimate: 100 } as ContentBlock, + { type: 'text', text: label }, +]; + +/** Which of img-1..n kept its image ('I'), was stripped ('.') or is missing ('-'). */ +function pattern(messages: Array<{ content: ContentBlock[] }>, n: number): string { + let out = ''; + for (let i = 1; i <= n; i++) { + const m = messages.find((x) => x.content.some((b) => b.type === 'text' && b.text === `img-${i}`)); + out += !m ? '-' : m.content.some((b) => b.type === 'image') ? 'I' : '.'; + } + return out; +} + +/** Compile a fresh store holding `n` image messages of `bytes` each. */ +async function compiled(strategy: ContextStrategy, n: number, bytes = 4): Promise { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + for (let i = 1; i <= n; i++) manager.addMessage('user', imageMessage(`img-${i}`, bytes)); + const result = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + return pattern(result.messages, n); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +const memory = (options: Record) => new AutobiographicalStrategy({ + headWindowTokens: 0, recentWindowTokens: 1_000_000, hierarchical: true, imageStripDepthTokens: 0, ...options, +} as ConstructorParameters[0]); +const windowed = (options: Record) => new WindowedPassthroughStrategy({ + imageStripDepthTokens: 0, ...options, +} as ConstructorParameters[0]); +const STRATEGIES = { autobiographical: memory, 'windowed passthrough': windowed }; +const DEFAULTS = { + autobiographical: () => new AutobiographicalStrategy(), + 'windowed passthrough': () => new WindowedPassthroughStrategy(), +}; + +describe('imageCeilings()', () => { + it('is reachable through the ContextStrategy interface, typed by the exported ImageCeilings', () => { + const strategy: ContextStrategy = new WindowedPassthroughStrategy(); + const ceilings: ImageCeilings | undefined = strategy.imageCeilings?.(); + assert.deepEqual(ceilings, { imageCeiling: 500, imageByteCeiling: 24 * MiB }); + }); + + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: the defaults are 500 images and 24 MiB`, () => { + assert.deepEqual(DEFAULTS[name as keyof typeof DEFAULTS]().imageCeilings(), { imageCeiling: 500, imageByteCeiling: 24 * MiB }); + }); + + it(`${name}: reports the ceilings as the window applies them`, () => { + const ceilings = (options: Record) => make(options).imageCeilings(); + assert.deepEqual(ceilings({ imageCeiling: 10, imageByteCeiling: 2 * MiB }), { imageCeiling: 10, imageByteCeiling: 2 * MiB }); + assert.deepEqual(ceilings({ imageCeiling: 0, imageByteCeiling: 0 }), { imageCeiling: 0, imageByteCeiling: 0 }, '0 is none'); + for (const bad of [-3, Number.NaN, Infinity]) { + assert.deepEqual(ceilings({ imageCeiling: bad, imageByteCeiling: bad }), { imageCeiling: 500, imageByteCeiling: 24 * MiB }, + `a ceiling of ${bad} takes the default`); + } + assert.deepEqual(ceilings({ imageCeiling: 0.5, imageByteCeiling: 100.7 }), { imageCeiling: 1, imageByteCeiling: 100 }, + 'a fraction is floored, to at least 1'); + assert.deepEqual(ceilings({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }), { imageCeiling: 500, imageByteCeiling: 24 * MiB }, + "the window's own limits leave the ceilings as they are"); + }); + } +}); + +describe("the memory strategy's effective config", () => { + it('reports both ceilings, as the library defaults when the host sets neither, and as the host set them otherwise', async () => { + const report = async (options: Record) => { + const lines: string[] = []; + const original = console.error; + console.error = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: new AutobiographicalStrategy({ logEffectiveConfig: true, ...options }) }); + await manager.close(); + } finally { + console.error = original; + rmSync(dir, { recursive: true, force: true }); + } + const line = lines.map((l) => { try { return JSON.parse(l); } catch { return null; } }).find((l) => l?.event === 'config:effective'); + assert.ok(line, JSON.stringify(lines)); + return line as { effective: Record; provenance: Record }; + }; + const defaults = await report({}); + assert.equal(defaults.effective.imageCeiling, 500); + assert.equal(defaults.effective.imageByteCeiling, 24 * MiB); + assert.equal(defaults.provenance.imageCeiling, 'library-default'); + assert.equal(defaults.provenance.imageByteCeiling, 'library-default'); + const set = await report({ imageCeiling: 40, imageByteCeiling: 0 }); + assert.equal(set.effective.imageCeiling, 40); + assert.equal(set.effective.imageByteCeiling, 0); + assert.equal(set.provenance.imageCeiling, 'caller'); + }); +}); + +describe('the window limits resolved within the ceilings', () => { + const resolve = (configured: Partial>) => + resolveLiveImageLimits({ maxLiveImages: 6, maxLiveImageBytes: 20 * MiB, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...configured }); + + it('the defaults are a window of 6 images and 20 MiB under ceilings of 500 images and 24 MiB', () => { + assert.deepEqual(resolve({}), { target: { images: 6, bytes: 20 * MiB }, ceiling: { images: 500, bytes: 24 * MiB } }); + }); + + it('the window is kept within the ceilings, and a ceiling of 0 is none', () => { + assert.deepEqual(resolve({ maxLiveImages: 0, maxLiveImageBytes: 0 }), { + target: { images: 500, bytes: 24 * MiB }, ceiling: { images: 500, bytes: 24 * MiB }, + }, 'an unlimited window means up to the ceilings'); + assert.deepEqual(resolve({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }).target, + { images: 500, bytes: 24 * MiB }, 'a window above the ceilings keeps the ceilings'); + assert.deepEqual(resolve({ maxLiveImages: 4, maxLiveImageBytes: MiB, imageCeiling: 10, imageByteCeiling: 2 * MiB }), { + target: { images: 4, bytes: MiB }, ceiling: { images: 10, bytes: 2 * MiB }, + }, 'a window under configured ceilings keeps its own limits'); + assert.deepEqual(resolve({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0 }), { + target: { images: 0, bytes: 0 }, ceiling: { images: 0, bytes: 0 }, + }, 'no ceiling leaves an unlimited window unlimited'); + assert.equal(resolve({ maxLiveImages: 4, imageCeiling: 0 }).target.images, 4, 'no ceiling leaves a window as set'); + }); + + it("a window limit that isn't a positive finite number is none, and a malformed ceiling takes its default", () => { + for (const bad of [-3, Number.NaN, Infinity]) { + assert.equal(resolve({ maxLiveImages: bad }).target.images, 500, `maxLiveImages ${bad} is unlimited, so up to the ceiling`); + assert.equal(resolve({ maxLiveImageBytes: bad }).target.bytes, 24 * MiB, `maxLiveImageBytes ${bad} is unlimited, so up to the ceiling`); + assert.equal(resolve({ imageCeiling: bad }).ceiling.images, 500, `imageCeiling ${bad} takes the default`); + assert.equal(resolve({ imageByteCeiling: bad }).ceiling.bytes, 24 * MiB, `imageByteCeiling ${bad} takes the default`); + } + assert.equal(resolve({ maxLiveImages: 0, imageCeiling: -1 }).target.images, 500, 'the window is clamped to the default the malformed ceiling took'); + assert.equal(resolve({ maxLiveImages: Infinity, imageCeiling: 0 }).target.images, 0, + 'an infinite limit with no ceiling is reported as none, 0'); + }); + + it('a fractional limit or ceiling is floored, to at least 1', () => { + assert.equal(resolve({ maxLiveImages: 2.5 }).target.images, 2); + assert.deepEqual(resolve({ maxLiveImages: 0, imageCeiling: 1.5 }), { + target: { images: 1, bytes: 20 * MiB }, ceiling: { images: 1, bytes: 24 * MiB }, + }); + assert.equal(resolve({ imageCeiling: 0.5 }).ceiling.images, 1, 'a fraction under 1 is 1, not none'); + assert.equal(resolve({ imageByteCeiling: 100.7 }).ceiling.bytes, 100); + }); +}); + +describe('the live window within the image ceilings', () => { + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: an unlimited count keeps the newest imageCeiling`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, imageCeiling: 2 }), 4), '..II'); + }); + it(`${name}: a count above the ceiling keeps the ceiling's worth`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 3, imageCeiling: 2 }), 4), '..II'); + }); + it(`${name}: an unlimited byte budget keeps the newest imageByteCeiling`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 2000 }), 4, 1000), '..II'); + }); + it(`${name}: with no window and no ceilings, every image stays, more than the default window's 6`, async () => { + assert.equal(await compiled(make({ + maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0, + }), 8), 'IIIIIIII'); + }); + it(`${name}: the count ceiling binds the images the window counts, nested and generated ones included`, async () => { + // What the ceilings bind is what the window counts: every image, an + // image nested in a tool result and a generated image among them. + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, imageCeiling: 1 }) }); + try { + const image = (data: string) => + ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; + manager.addMessage('Claude', [{ type: 'tool_use', id: 'tu-1', name: 'shot', input: {} }] as ContentBlock[]); + manager.addMessage('user', [{ type: 'tool_result', toolUseId: 'tu-1', content: [image('AAAA'), image('BBBB')] }] as ContentBlock[]); + for (let i = 1; i <= 2; i++) { + manager.addMessage('user', [ + { type: 'generated_image', data: 'A'.repeat(4), mimeType: 'image/png' } as ContentBlock, + { type: 'text', text: `gen-${i}` }, + ]); + } + const result = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const blocks = result.messages.flatMap((m) => m.content); + const nested = blocks + .filter((b) => b.type === 'tool_result') + .flatMap((b) => (b as { content: ContentBlock[] }).content) + .filter((b) => b.type === 'image').length; + assert.equal(nested, 0, 'the nested images are past a count ceiling of 1'); + const kept = blocks.filter((b) => b.type === 'generated_image').length; + assert.equal(kept, 1, 'the newest image, a generated one, takes the one slot'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + it('the post-strip estimates price an image the ceiling strips as the placeholder', () => { + const image = (): ContentBlock => + ({ type: 'image', source: { type: 'base64', data: 'AAAA', mediaType: 'image/png' } }) as ContentBlock; + const messages = [{ id: 'old', content: [image()] }, { id: 'new', content: [image()] }]; + const store = { + getAll: () => messages as never[], + get: (id: string) => (messages.find((m) => m.id === id) as never) ?? null, + getFrom: (i: number) => messages.slice(i) as never[], + getTail: (n: number) => messages.slice(-n) as never[], + length: () => messages.length, + estimateTokens: (m: { content: ContentBlock[] }) => + m.content.reduce((sum, b) => sum + (b.type === 'image' ? 1600 : b.type === 'text' ? Math.ceil(b.text.length / 4) : 0), 0), + getTokenCalibration: () => 1, + }; + const estimate = (strategy: AutobiographicalStrategy) => + (strategy as unknown as { postStripEstimates(s: typeof store): number[] }).postStripEstimates(store); + const stripped = [Math.ceil(PLACEHOLDER.length / 4), 1600]; + assert.deepEqual(estimate(memory({ maxLiveImages: 0, imageCeiling: 1 })), stripped, 'by the count ceiling'); + assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 5 })), stripped, + 'by the byte ceiling: 4 bytes fit, 8 do not'); + // The memo is keyed by the clamped limits, so a changed ceiling prices anew. + const strategy = memory({ maxLiveImages: 0, imageCeiling: 1 }); + assert.deepEqual(estimate(strategy), stripped); + (strategy as unknown as { config: { imageCeiling: number } }).config.imageCeiling = 2; + assert.deepEqual(estimate(strategy), [1600, 1600], 'with a ceiling of 2, both are live'); + }); + + it('the hysteresis rule trims against the ceiling when the count is unlimited', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + }); + try { + const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + assert.equal(await strip(4), 'IIII'); + manager.addMessage('user', imageMessage('img-5')); + assert.equal(await strip(5), '...II', 'the ceiling binds: one trim to half of it, not a slide of one'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('the hysteresis rule against the byte ceiling', () => { + it('trims against the byte ceiling when the byte budget is unlimited', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), + }); + try { + const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`, 4)); + assert.equal(await strip(4), 'IIII', 'four 4-byte images fill the 16-byte ceiling'); + manager.addMessage('user', imageMessage('img-5', 4)); + assert.equal(await strip(5), '...II', 'the ceiling binds: one trim to half of it'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('the image limits warning', () => { + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: says once, at initialize, when a configured window limit is above its ceiling`, async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const cases = [ + [{ maxLiveImages: 8, imageCeiling: 4 }, ['maxLiveImages 8 is above imageCeiling 4, so the live window keeps at most 4']], + [{ maxLiveImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }, + ['maxLiveImageBytes 3 MiB is above imageByteCeiling 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 8, imageCeiling: 4, maxLiveImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }, + ['maxLiveImages 8 is above imageCeiling 4', 'maxLiveImageBytes 3 MiB is above imageByteCeiling 2 MiB']], + // A byte figure that isn't a whole number of MiB reads in bytes. + [{ maxLiveImageBytes: 3 * MiB + 1, imageByteCeiling: 2 * MiB }, [`maxLiveImageBytes ${3 * MiB + 1} bytes is above imageByteCeiling 2 MiB`]], + [{ imageCeiling: Number.NaN }, ["imageCeiling NaN isn't 0 or a finite number above it, so the default 500 applies"]], + [{ imageByteCeiling: -1 }, ["imageByteCeiling -1 isn't 0 or a finite number above it, so the default 24 MiB applies"]], + [{ maxLiveImages: 0, maxLiveImageBytes: 0 }, null], + [{ maxLiveImages: 4, imageCeiling: 4 }, null], + // At or above the memory strategy's 12 MiB compression budget, so + // that budget isn't above the ceiling either. + [{ maxLiveImageBytes: 16 * MiB, imageByteCeiling: 16 * MiB }, null], + // A default above a lowered ceiling is named too, marked as the + // default, and so is a default ceiling under a limit the host set. + [{ imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 600 }, ['maxLiveImages 600 is above imageCeiling 500 (default), so the live window keeps at most 500']], + // The default window of 6 comes from the strategy's default config, + // not from the host, so it's marked too. + [{ imageCeiling: 4 }, ['maxLiveImages 6 (default) is above imageCeiling 4, so the live window keeps at most 4']], + // An infinite limit is none, like 0, so nothing is said. + [{ maxLiveImages: Infinity }, null], + // A malformed ceiling takes its default, which the later clause marks. + [{ maxLiveImages: 600, imageCeiling: Number.NaN }, + ["imageCeiling NaN isn't 0 or a finite number above it, so the default 500 applies", + 'maxLiveImages 600 is above imageCeiling 500 (default)']], + // A byte budget given as null takes its default, and is marked so. + [{ maxLiveImageBytes: null, imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB']], + [{ maxLiveImageBytes: undefined, imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB']], + [{ maxLiveImageBytes: 30 * MiB, imageByteCeiling: -1 }, + ["imageByteCeiling -1 isn't 0 or a finite number above it, so the default 24 MiB applies", + 'maxLiveImageBytes 30 MiB is above imageByteCeiling 24 MiB (default)']], + [{ maxLiveImages: 8, imageCeiling: 0 }, null], + [{ maxLiveImageBytes: 30 * MiB, imageByteCeiling: 0 }, null], + ] as const; + for (const [index, [options, said]] of cases.entries()) { + lines.length = 0; + const strategy = make(options); + // Two stores opened in turn with the one strategy initialize it twice. + for (const store of ['first', 'second']) { + const manager = await ContextManager.open({ path: join(dir, `${index}-${store}`), strategy }); + await manager.close(); + } + const ours = lines.filter((line) => line.includes('above') || line.includes('the default')); + if (said === null) assert.deepEqual(ours, [], `nothing said for ${JSON.stringify(options)}`); + else { + assert.equal(ours.length, 1, `said once, in one line, for ${JSON.stringify(options)}: ${JSON.stringify(ours)}`); + for (const part of said) assert.ok(ours[0]!.includes(part), `${ours[0]} names ${part}`); + } + } + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); + } +}); + +class Prepared extends AutobiographicalStrategy { + prepare(request: NormalizedRequest) { return this.prepareCompressionRequest(request); } + send(request: NormalizedRequest, label: string): NormalizedRequest { + return (this as unknown as { prepareCompressionCandidate(r: NormalizedRequest, l: string): NormalizedRequest }) + .prepareCompressionCandidate(request, label); + } +} +function captureErrors(run: () => T): string[] { + const logs: string[] = []; + const original = console.error; + console.error = (...parts: unknown[]) => { logs.push(parts.map(String).join(' ')); }; + try { run(); return logs; } finally { console.error = original; } +} +const imageBlock = (data: string) => ({ type: 'image', source: { type: 'base64', mediaType: 'image/png', data } }) as ContentBlock; +const request = (...messages: ContentBlock[][]): NormalizedRequest => ({ + messages: messages.map((content) => ({ participant: 'User', content })), config: { model: 'mock', maxTokens: 1000 }, +}); + +describe("the memory strategy's warning names a compression budget above the byte ceiling", () => { + it('says it once, at initialize', async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: memory({ maxCompressionImageBytes: 30 * MiB }) }); + await manager.close(); + const ours = lines.filter((line) => line.includes('maxCompressionImageBytes')); + assert.equal(ours.length, 1, JSON.stringify(lines)); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 30 MiB is above imageByteCeiling 24 MiB (default), so a compression prompt keeps at most 24 MiB'), ours[0]); + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("the memory strategy's warning names its default compression budget under a lowered byte ceiling", () => { + it('says it, marked as the default, with no clause for a window limit under the ceiling', async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ + path: join(dir, 'store'), strategy: memory({ maxLiveImageBytes: 1 * MiB, imageByteCeiling: 2 * MiB }), + }); + await manager.close(); + const ours = lines.filter((line) => line.includes('is above')); + assert.equal(ours.length, 1, JSON.stringify(lines)); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 12 MiB (default) is above imageByteCeiling 2 MiB, so a compression prompt keeps at most 2 MiB'), ours[0]); + assert.ok(!ours[0]!.includes('maxLiveImageBytes'), ours[0]); + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('the line when an image ceiling first strips', () => { + /** Run `steps` on a fresh store: each adds images (`add`, of `bytes` each, + * 4 when not given) and compiles. Returns the ceiling lines each step said, + * in order, the pattern each compile kept, and the last pattern. Byte + * sizes are multiples of 4: the store keeps an + * image's data as the bytes its base64 decodes to, so other lengths + * come back changed. */ + async function ceilingLines( + strategy: ContextStrategy, + steps: Array, + ): Promise<{ lines: string[]; byStep: string[][]; patterns: string[]; last: string }> { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + let n = 0; + const byStep: string[][] = []; + const patterns: string[] = []; + try { + for (const step of steps) { + const { add, bytes } = typeof step === 'number' ? { add: step, bytes: 4 } : step; + const before = said.length; + for (let i = 0; i < add; i++) manager.addMessage('user', imageMessage(`img-${++n}`, bytes)); + patterns.push(pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n)); + byStep.push(said.slice(before).filter((line) => line.includes(CEILING_LINE))); + } + return { lines: byStep.flat(), byStep, patterns, last: patterns[patterns.length - 1] ?? '' }; + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: says once which ceiling strips, and what it holds`, async () => { + const cases = [ + [{ maxLiveImages: 0, imageCeiling: 2 }, 'imageCeiling 2 holds maxLiveImages 0 (no limit) to it'], + [{ maxLiveImages: 5, imageCeiling: 2 }, 'imageCeiling 2 holds maxLiveImages 5 to it'], + [{ imageCeiling: 2 }, 'imageCeiling 2 holds maxLiveImages 6 (default) to it'], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 8 }, + 'imageByteCeiling 8 bytes holds maxLiveImageBytes 0 (no limit) to it'], + ] as const; + for (const [options, said] of cases) { + // Three images, then two more: the ceiling strips at the first compile + // and again at the second, and it's said once, at the first. + const strategy = make(options); + const { lines, byStep, patterns } = await ceilingLines(strategy, [3, 2]); + assert.deepEqual(patterns, ['.II', '...II'], `the ceiling of 2 strips at each compile for ${JSON.stringify(options)}`); + assert.deepEqual(byStep.map((step) => step.length), [1, 0], `said once, at the first step, for ${JSON.stringify(options)}`); + assert.ok(lines[0]!.startsWith(`[${strategy.name}] ${said}, so some images are now shown as placeholders`), lines[0]); + assert.match(lines[0]!, / ns=/, 'through the namespaced route the warning takes'); + // The memory strategy names the option that strips in batches, which + // is off here; windowed passthrough has none to name. + assert.equal(lines[0]!.includes('; at the ceiling each new image turns the oldest live ones into placeholders, which rewrites the prompt cache from there, and imageStripHysteresisRatio does that in batches instead'), + name === 'autobiographical', lines[0]); + } + }); + + it(`${name}: says nothing when the host's own limit strips, or nothing falls past the ceiling`, async () => { + for (const [options, steps, kept] of [ + [{ maxLiveImages: 2 }, [3, 2], '...II'], + [{ maxLiveImages: 2, imageCeiling: 2 }, [3, 2], '...II'], + [{ maxLiveImages: 0, imageCeiling: 2 }, [1, 1], 'II'], + // The host's own byte limit strips, under the default byte ceiling. + [{ maxLiveImageBytes: 8 }, [3, 2], '...II'], + [{ maxLiveImageBytes: 8, imageByteCeiling: 8 }, [3, 2], '...II'], + ] as const) { + const { lines, last } = await ceilingLines(make(options), [...steps]); + assert.equal(last, kept, `the strip ran for ${JSON.stringify(options)}`); + assert.deepEqual(lines, [], `nothing said for ${JSON.stringify(options)}`); + } + }); + + it(`${name}: says it once for each ceiling, when each first strips`, async () => { + // Four-byte images fill the count ceiling of 3 first, at 12 of the 16 + // bytes; then a 16-byte image fills the byte ceiling, and the next one + // falls past it. + const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 3, imageByteCeiling: 16 }); + const { byStep, patterns } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); + // At the third step the byte walk drops the 16-byte image and, being + // greedy, keeps two older 4-byte ones that still fit (as on main). + assert.deepEqual(patterns, ['.III', '....I', '..II.I'], 'the count, then the bytes, strip'); + assert.deepEqual(byStep.map((step) => step.length), [1, 1, 0], JSON.stringify(byStep)); + assert.ok(byStep[0]![0]!.includes('imageCeiling 3 holds'), byStep[0]![0]); + assert.ok(byStep[1]![0]!.includes('imageByteCeiling 16 bytes holds'), byStep[1]![0]); + }); + + it(`${name}: says it when a message's first image passes the byte ceiling and its second fits`, async () => { + // One message: a 12-byte image, then a 4-byte one, under a byte + // ceiling of 8. The first is past it on its own and the second fits, + // so the message holds one image past the ceiling and one it kept. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }) }); + try { + manager.addMessage('user', [ + { type: 'image', source: { type: 'base64', data: 'A'.repeat(12), mediaType: 'image/png' } }, + { type: 'image', source: { type: 'base64', data: 'B'.repeat(4), mediaType: 'image/png' } }, + { type: 'text', text: 'pair' }, + ] as ContentBlock[]); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const pair = compiled.messages.find((m) => m.content.some((b) => b.type === 'text' && b.text === 'pair'))!; + assert.deepEqual(pair.content.filter((b) => b.type === 'image').map((b) => (b as { source: { data: string } }).source.data), ['B'.repeat(4)], + 'the first image is replaced, the second kept'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageByteCeiling 8 bytes holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it(`${name}: says both lines when both ceilings' limits are over at the same image`, async () => { + // Three 4-byte images under ceilings of 2 images and 8 bytes: the + // oldest is past both, so one strip notes both ceilings. + const { byStep, last } = await ceilingLines( + make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 8 }), [3], + ); + assert.equal(last, '.II'); + assert.equal(byStep[0]!.length, 2, JSON.stringify(byStep)); + assert.ok(byStep[0]!.some((line) => line.includes('imageCeiling 2 holds')), JSON.stringify(byStep)); + assert.ok(byStep[0]!.some((line) => line.includes('imageByteCeiling 8 bytes holds')), JSON.stringify(byStep)); + }); + + it(`${name}: says it for a single image bigger than the byte ceiling`, async () => { + const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }), [{ add: 1, bytes: 12 }]); + assert.deepEqual(patterns, ['.'], 'the one image is past the ceiling on its own'); + assert.equal(byStep[0]!.length, 1, JSON.stringify(byStep)); + assert.ok(byStep[0]![0]!.includes('imageByteCeiling 8 bytes holds maxLiveImageBytes 0 (no limit) to it'), byStep[0]![0]); + }); + + it(`${name}: says it when the ceiling's limit is over at an image the render replaces, whether or not the depth limit strips it too`, async () => { + // Two 8-byte images under a 12-byte ceiling: the older one is past the + // ceiling. With a depth limit that the older one is also past, the + // depth strips it with or without the ceiling; the line still says the + // ceiling holds the setting, since its limit is over there. + const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500 }; + const alone = await ceilingLines(make({ ...depth, imageByteCeiling: 0 }), [{ add: 2, bytes: 8 }]); + assert.equal(alone.last, '.I', 'the depth limit alone strips the older image'); + assert.deepEqual(alone.lines, [], 'no ceiling, no line'); + const both = await ceilingLines(make({ ...depth, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(both.last, '.I', 'the ceiling changes nothing the window keeps'); + assert.equal(both.lines.length, 1, JSON.stringify(both.lines)); + const ceilingOnly = await ceilingLines(make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); + assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); + }); + + it(`${name}: says it for the byte ceiling when its limit is over at an image the host's own count limit strips too`, async () => { + // The older image is past the host's maxLiveImages of 1 and past the + // 12-byte ceiling: the host's limit strips it whatever the ceiling, and + // the byte ceiling's limit is over there, so its line is said. + const { lines, last } = await ceilingLines(make({ maxLiveImages: 1, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(last, '.I'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageByteCeiling 12 bytes holds'), lines[0]); + }); + + it(`${name}: says nothing from renderAsNewest, whose render a ceiling strips, and leaves the line for the next compile`, async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, imageCeiling: 1 }) }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + const ids = [1, 2].map((i) => manager.addMessage('user', imageMessage(`img-${i}`))); + const rendered = manager.renderAsNewest(ids); + assert.ok(rendered, 'the strategy renders'); + const images = rendered.flatMap((content) => content ?? []).filter((b) => b.type === 'image').length; + assert.equal(images, 1, 'the ceiling of 1 stripped the older image in the render'); + assert.deepEqual(ours(), [], 'the render says nothing'); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it(`${name}: says nothing from a dry compile, and leaves the line for the first compile select returns`, async () => { + for (const [options, said0] of [ + [{ maxLiveImages: 0, imageCeiling: 2 }, 'imageCeiling 2 holds'], + [{ maxLiveImageBytes: 0, imageByteCeiling: 8 }, 'imageByteCeiling 8 bytes holds'], + ] as const) { + const strategy = make(options); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const dry = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }, undefined, { dryRun: true }); + assert.equal(pattern(dry.messages, 3), '.II', `the dry compile strips for ${JSON.stringify(options)}`); + assert.deepEqual(ours(), [], `a dry compile says nothing for ${JSON.stringify(options)}`); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, `the compile returned says it for ${JSON.stringify(options)}`); + assert.ok(ours()[0]!.includes(said0), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + }); + } + + it('the memory strategy says it when its hysteresis rule trims against a ceiling', async () => { + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + [5], + ); + assert.equal(last, '...II', 'one trim to half the ceiling'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 4 holds maxLiveImages 0 (no limit) to it'), lines[0]); + assert.ok(!lines[0]!.includes('imageStripHysteresisRatio'), `the option is on, so the line doesn't name it: ${lines[0]}`); + }); + + it('the memory strategy says it from its adaptive selection too, and nothing from a preview', async () => { + const adaptive = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 2, + }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: adaptive }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 1 }, { render: true })?.entries ?? []; + assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); + assert.deepEqual(ours(), [], 'a preview, with its own ceiling, says nothing'); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), '.II'); + assert.equal(ours().length, 1, JSON.stringify(said)); + assert.ok(ours()[0]!.includes('imageCeiling 2 holds maxLiveImages 0 (no limit) to it'), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's strip leaves the line only from a render select returns, not a layout rendered to compare against", async () => { + // Both ways the strip decides: counting what the entries send, and the + // verdicts of the post-strip estimates (asEstimated), as the adaptive + // selection and the kv-unified cache check ask for them. + for (const [options, asEstimated] of [ + [{ maxLiveImages: 0, imageCeiling: 1 }, false], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, false], + [{ maxLiveImages: 0, imageCeiling: 1 }, true], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, true], + ] as const) { + const strategy = memory(options); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 2; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const view = (manager as unknown as { messageStore: { createView(): unknown } }).messageStore.createView() as { + getAll(): Array<{ id: string; participant: string; content: ContentBlock[] }>; + }; + const entries = () => view.getAll().map((m, index) => ({ + index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content, + })); + const strip = strategy as unknown as { + applyImageStripping(entries: unknown[], store: unknown, opts?: { returned?: boolean; asEstimated?: boolean }): void; + reportCeilingHits(store: unknown): void; + }; + const kept = entries(); + strip.applyImageStripping(kept, view, { asEstimated }); + assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling replaced one'); + strip.reportCeilingHits(view); + assert.deepEqual(ours(), [], `a layout rendered to compare against leaves nothing for ${JSON.stringify(options)}, asEstimated ${asEstimated}`); + strip.applyImageStripping(entries(), view, { returned: true, asEstimated }); + assert.deepEqual(ours(), [], 'the strip itself says nothing: select reports once it returns the render'); + strip.reportCeilingHits(view); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + }); + + it('the memory strategy says nothing for a select it refuses, and says it for the next one it returns', async () => { + const adaptive = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 2, + }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: adaptive }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + // The newest turn is a call with no result yet: structural repair drops + // it, and the check after the strip refuses the select. + manager.addMessage('Claude', [{ type: 'tool_use', id: 'tu-1', name: 'shot', input: {} }] as ContentBlock[]); + await assert.rejects(manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }), + { message: /^Structural repair did not retain the newest turn/ }); + assert.deepEqual(ours(), [], 'a refused select says nothing'); + manager.addMessage('user', [{ type: 'tool_result', toolUseId: 'tu-1', content: 'done' }] as ContentBlock[]); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's adaptive selection says it when the ceiling's limit is over at an image it replaces, whether or not the depth limit strips it too", async () => { + // The adaptive selection's strip applies the post-strip estimates' + // verdicts, which carry the depth limit's own: two 8-byte images under a + // 12-byte ceiling, the older one also past a 2,500-token depth limit. + const adaptive = (options: Record) => memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 12, ...options, + }); + const depth = await ceilingLines(adaptive({ imageStripDepthTokens: 2500 }), [{ add: 2, bytes: 8 }]); + assert.equal(depth.last, '.I', 'the depth limit strips the older image'); + assert.equal(depth.lines.length, 1, JSON.stringify(depth.lines)); + const ceilingOnly = await ceilingLines(adaptive({}), [{ add: 2, bytes: 8 }]); + assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); + assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); + }); + + it("the memory strategy says nothing for the rule's set under a preview's own ceiling", async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 10, imageStripHysteresisRatio: 0.5, + }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + // A preview with a ceiling of 2 strips by the rule, for itself only: its + // trim to half keeps one image when the third comes, and the fourth fits. + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 2 }, { render: true })?.entries ?? []; + assert.equal(pattern(entries, 4), '..II', "the preview's own set trims to half its ceiling"); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 4), 'IIII', 'the real ceiling of 10 strips nothing'); + assert.deepEqual(ours(), [], "the preview's hypothetical ceiling left no line"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('the memory strategy says nothing for a host with no limits, after a preview with a ceiling of its own', async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + // Every limit off (the helper sets no depth strip), so the compile's strip has nothing to record. + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0, + }), + }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 1 }, { render: true })?.entries ?? []; + assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), 'III', 'with no limits, the compile keeps every image'); + assert.deepEqual(ours(), [], "the preview's strip isn't the compile's"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's rule reads its set from the stored messages alone, so a reopened instance says it at its first compile", async () => { + // The rule keeps no state: a second instance on the same store replays + // the same set, with the ceiling's trim in it, so it says the line at the + // first compile it returns, as the first instance did. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const options = { maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }; + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + try { + const first = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + for (let i = 1; i <= 5; i++) first.addMessage('user', imageMessage(`img-${i}`)); + assert.equal(pattern((await first.compile(budget)).messages, 5), '...II', 'the ceiling trims to half of 4'); + assert.equal(ours().length, 1, JSON.stringify(said)); + await first.close(); + said.length = 0; + const second = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + try { + assert.equal(pattern((await second.compile(budget)).messages, 5), '...II', 'the same set from the same messages'); + assert.equal(ours().length, 1, JSON.stringify(said)); + assert.ok(ours()[0]!.includes('imageCeiling 4 holds'), ours()[0]); + } finally { + await second.close(); + } + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); + + for (const [kind, options, named] of [ + ['count', { maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }, 'imageCeiling 4 holds'], + ['byte', { maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }, 'imageByteCeiling 16 bytes holds'], + ] as const) { + it(`the memory strategy's rule reads the set of the branch it compiles, under the ${kind} ceiling`, async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + const ours = () => said.filter((line) => line.includes(CEILING_LINE)); + const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + try { + manager.addMessage('user', imageMessage('img-1')); + const main = manager.currentBranch().name; + const busy = await manager.fork('busy'); + // On the fork, four more images make five, past a count of 4 and past + // 16 bytes. + for (let i = 2; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + await manager.switchBranch(main); + assert.equal(pattern((await manager.compile(budget)).messages, 1), 'I', 'main holds one image, under its ceiling'); + assert.deepEqual(ours(), [], "main's compile says nothing for the fork's messages"); + await manager.switchBranch(busy); + assert.equal(pattern((await manager.compile(budget)).messages, 5), '...II', "the fork's set, from its own messages"); + assert.equal(ours().length, 1, JSON.stringify(said)); + assert.ok(ours()[0]!.includes(named), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + it("the memory strategy's rule says nothing when the host's own limit binds", async () => { + const { lines, last } = await ceilingLines(memory({ maxLiveImages: 4, imageStripHysteresisRatio: 0.5 }), [5]); + assert.equal(last, '...II', 'one trim to half the host limit'); + assert.deepEqual(lines, []); + }); + + it("the memory strategy's rule says nothing when the depth limit's trims strip what the ceiling's would", async () => { + // Three 8-byte images under a 20-byte ceiling and a 2,500-token depth + // limit, each image message about 1,600 tokens: the depth limit brings + // a trim on at the second image and again at the third, each stripping + // the older one, so the bytes never pass the ceiling where a trim begins. + const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500, imageStripHysteresisRatio: 0.5 }; + const alone = await ceilingLines(memory({ ...depth, imageByteCeiling: 0 }), [{ add: 3, bytes: 8 }]); + const both = await ceilingLines(memory({ ...depth, imageByteCeiling: 20 }), [{ add: 3, bytes: 8 }]); + assert.equal(alone.last, '..I', 'the depth limit alone strips the two older images'); + assert.equal(both.last, alone.last, 'the ceiling changes nothing the window keeps'); + assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); + }); + + /** The memory strategy over `messages`, compiled once: the pattern its + * image messages kept, and the ceiling lines it said. */ + async function ruleRun(options: Record, messages: unknown[][]) { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + try { + for (const content of messages) manager.addMessage('user', content as never); + const images = messages.filter((content) => content.some((b) => (b as { type?: string }).type === 'image')).length; + const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, images); + return { kept, lines: said.filter((line) => line.includes(CEILING_LINE)) }; + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + + it("the memory strategy's rule says it when the ceiling's limit brings a trim on, though the host's own byte limit or the depth limit is over there too", async () => { + // A 32-byte image, then three 4-byte ones, under a count ceiling of 3 and + // a host byte limit of 40, in force: at the fourth image the count passes + // the ceiling and the bytes (44) pass the host's limit, so one trim has + // both over. The line is said, since the ceiling's limit brought it on. + const steps = [{ add: 1, bytes: 32 }, { add: 3, bytes: 4 }]; + const host = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); + assert.ok(host.last.includes('.'), `premise: the trim strips (${host.last})`); + assert.equal(host.lines.length, 1, JSON.stringify(host.lines)); + assert.ok(host.lines[0]!.includes('imageCeiling 3 holds'), host.lines[0]); + + // A count ceiling of 2 and a 4,000-token depth limit: an image, then a + // message of two images with about 5,000 tokens of text, where the count + // passes the ceiling and the first image is past the depth, both at once. + const image = (data: string): ContentBlock => + ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' }, tokenEstimate: 100 }) as ContentBlock; + const depth = await ruleRun({ maxLiveImages: 0, imageCeiling: 2, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }, [ + imageMessage('img-1'), + [image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'img-2' }, { type: 'text', text: 'word '.repeat(2900) }], + ]); + assert.ok(depth.kept.startsWith('.'), `premise: the trim strips the first image (${depth.kept})`); + assert.equal(depth.lines.length, 1, JSON.stringify(depth.lines)); + assert.ok(depth.lines[0]!.includes('imageCeiling 2 holds'), depth.lines[0]); + }); + + it("the memory strategy's rule says it from its own replay, not from the limits' account of the images it strips", async () => { + // A host byte limit of 40, in force, and a count ceiling of 4. A 36-byte + // image and an 8-byte one pass the host's limit (44 bytes), and that + // trim, to 20 bytes, strips the 36-byte image. Three 4-byte images then + // make four live, 20 bytes, so no trim's start ever has the count over + // its ceiling. By the limits' own account, newest first, the 36-byte + // image is past the count ceiling (four kept before it), but the set + // decides it, and the replay's trims were the host's limit's alone. + const { lines, patterns } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), + [{ add: 1, bytes: 36 }, { add: 1, bytes: 8 }, { add: 3, bytes: 4 }], + ); + assert.deepEqual(patterns, ['I', '.I', '.IIII']); + assert.deepEqual(lines, [], JSON.stringify(lines)); + }); + + it("the memory strategy's rule says nothing for a trim the host's own byte limit brings on, though the ceiling-set count is over its target there", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force. Three + // 4-byte images, then a 32-byte one: the bytes (44) pass the host's + // limit, and the count, 4, isn't over its ceiling, though it's over the + // trim's target of 2. The trim is the host's, so no line. + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), + [3, { add: 1, bytes: 32 }], + ); + assert.ok(last.includes('.'), `premise: the trim strips (${last})`); + assert.deepEqual(lines, [], JSON.stringify(lines)); + }); + + it("the memory strategy's rule says nothing at a render that replaces no image, though its replay has a ceiling's trim", async () => { + // Three images under a count ceiling of 2: the third brings a trim on + // that strips the first two. Then enough text that compression folds the + // image messages into summaries before the first compile: the replay + // still has the ceiling's trim, but the render shows no placeholder, so + // the line waits. + const membrane = { + complete: async () => ({ stopReason: 'end_turn', content: [{ type: 'text', text: '[mock summary] ' + 'x '.repeat(40) }], usage: { input_tokens: 100, output_tokens: 25 } }), + } as never; + const prose = (label: string, tokens: number): ContentBlock[] => [{ type: 'text', text: `${label} ` + 'abcd'.repeat(Math.round(tokens * 2.9 / 4)) }]; + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ + compressionModel: 'mock', targetChunkTokens: 1500, headWindowTokens: 0, recentWindowTokens: 9000, hierarchical: true, + maxLiveImages: 0, imageCeiling: 2, imageStripHysteresisRatio: 0.5, + }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane }); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + for (let k = 0; k < 20; k++) manager.addMessage('user', prose(`t${k}`, 800)); + for (let k = 0; k < 500 && !manager.isReady(); k++) await manager.tick(); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), '---', 'premise: every image message is folded into a summary'); + assert.ok(!compiled.messages.some((m) => m.content.some((b) => b.type === 'text' && b.text === PLACEHOLDER)), 'premise: no placeholder'); + assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), []); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's rule says both lines when both ceilings' limits bring the same trim on", async () => { + // Three 4-byte images under ceilings of 2 images and 8 bytes: at the + // third, the count (3) and the bytes (12) are both over, at one trim. + const { lines } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 8, imageStripHysteresisRatio: 0.5 }), [3], + ); + assert.equal(lines.length, 2, JSON.stringify(lines)); + assert.ok(lines.some((line) => line.includes('imageCeiling 2 holds')), JSON.stringify(lines)); + assert.ok(lines.some((line) => line.includes('imageByteCeiling 8 bytes holds')), JSON.stringify(lines)); + }); + + it("the memory strategy's rule keeps a ceiling's trim in its record past a later trim only the host's limit brings on", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force, compiled + // once, at the end. Five 4-byte images bring the count ceiling's trim + // on; a 36-byte image then brings a trim on that only the host's byte + // limit is over at. The replay's record of the first trim stands. + const { kept, lines } = await ruleRun( + { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }, + [1, 2, 3, 4, 5].map((i) => imageMessage(`img-${i}`)).concat([imageMessage('img-6', 36)]), + ); + assert.equal(kept, '.....I', 'premise: both trims strip'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 4 holds'), lines[0]); + }); + + it("the memory strategy's line doesn't name imageStripHysteresisRatio under a count ceiling of 1, which leaves it no room", async () => { + // Under a count ceiling of 1 the rule keeps the newest image at every + // trim, so it strips at every image as the limits do. + const { lines } = await ceilingLines(memory({ maxLiveImages: 0, imageCeiling: 1 }), [2]); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 1 holds maxLiveImages 0 (no limit) to it, so some images are now shown as placeholders'), lines[0]); + assert.ok(!lines[0]!.includes('imageStripHysteresisRatio'), lines[0]); + }); + + it("the memory strategy's line isn't read from a preview's set under ceilings that set no limit", async () => { + // A preview with maxLiveImages 4 under a count ceiling of 10 limits the + // count to 4, as the configured 0 under a ceiling of 4 does, but there + // the host sets the limit, so its trims say nothing for a ceiling. + // The compile after it, under the configured ceilings, still names the + // count ceiling its own trim brought on. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ hierarchical: false, adaptiveResolution: true, maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + }); + try { + for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + assert.ok(manager.previewContext(budget, { maxLiveImages: 4, imageCeiling: 10 }), 'premise: the strategy previews'); + const compiled = await manager.compile(budget); + assert.equal(pattern(compiled.messages, 5), '...II'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageCeiling 4 holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("windowed passthrough says nothing for a ceiling's verdict on a message its window drops", async () => { + // Two image messages under a count ceiling of 1, about 2,000 tokens of + // text between them, past a 3,000-token budget: the window re-anchors to + // the newest message, whose image the ceiling keeps. The verdicts are + // made from the old anchor, where the ceiling strips the older image, + // which the render no longer holds. With room for both, the same images + // say the line. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: windowed({ maxLiveImages: 0, imageCeiling: 1 }) }); + try { + manager.addMessage('user', imageMessage('img-1')); + manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(1200) }]); + manager.addMessage('user', imageMessage('img-2')); + const compiled = await manager.compile({ maxTokens: 3000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 2), '-I', 'the window holds the newest message alone'); + assert.equal(compiled.messages.length, 1, 'premise: the text went with the older image'); + assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), [], JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + const both = await ceilingLines(windowed({ maxLiveImages: 0, imageCeiling: 1 }), [2]); + assert.equal(both.last, '.I'); + assert.equal(both.lines.length, 1, JSON.stringify(both.lines)); + }); + + for (const hierarchical of [true, false]) { + it(`the memory strategy's rule strips a moved tool result's image as its own message's, and says the line (${hierarchical ? 'hierarchical' : 'adaptive'})`, async () => { + // Result a holds image A, ahead of image B in its message. Under a count + // ceiling of 1 the rule strips that message's first image, A, and keeps + // B. Pairing repair moves result a up beside result b, into another + // message's entry; A is still its own message's first image. + const image = (data: string): ContentBlock => ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ hierarchical, adaptiveResolution: !hierarchical, maxLiveImages: 0, imageCeiling: 1, imageStripHysteresisRatio: 0.5 }), + }); + try { + manager.addMessage('Claude', [{ type: 'tool_use', id: 'a', name: 'shot', input: {} }, { type: 'tool_use', id: 'b', name: 'shot', input: {} }] as ContentBlock[]); + manager.addMessage('User', [{ type: 'tool_result', toolUseId: 'b', content: [{ type: 'text', text: 'result b' }] }] as unknown as ContentBlock[]); + manager.addMessage('User', [{ type: 'tool_result', toolUseId: 'a', content: [image('AAAA'), { type: 'text', text: 'result a' }] }, image('BBBB')] as unknown as ContentBlock[]); + manager.addMessage('Claude', [{ type: 'text', text: 'done' }]); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const results = (content: ContentBlock[]) => content.filter((b) => b.type === 'tool_result').map((b) => (b as unknown as { toolUseId: string }).toolUseId).sort().join(','); + assert.ok(compiled.messages.some((x) => results(x.content) === 'a,b'), 'premise: result a moved beside result b'); + const data = (content: ContentBlock[]): string[] => content.flatMap((b) => + b.type === 'image' ? [(b as { source: { data: string } }).source.data] + : b.type === 'tool_result' && Array.isArray((b as { content?: unknown }).content) ? data((b as unknown as { content: ContentBlock[] }).content) : []); + assert.deepEqual(compiled.messages.flatMap((x) => data(x.content)), ['BBBB'], 'A stripped, B live'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageCeiling 1 holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { + const auto = await ceilingLines(memory({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); + assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); + assert.ok(auto.lines[0]!.includes('imageCeiling 2 holds maxLiveImages 0 (no limit) to it,'), auto.lines[0]); + const win = await ceilingLines(windowed({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); + assert.equal(win.lines.length, 1, JSON.stringify(win.lines)); + assert.ok(win.lines[0]!.includes('imageCeiling 2 holds maxLiveImages 6 (default) to it,'), win.lines[0]); + }); + + it("doesn't mark a value a subclass's enforced layer gives as a default", async () => { + class Enforced extends AutobiographicalStrategy { + protected override enforcedConfigLayer() { + return { source: 'enforced', values: { imageCeiling: 2 } }; + } + } + const { lines } = await ceilingLines( + new Enforced({ headWindowTokens: 0, recentWindowTokens: 1_000_000, hierarchical: true, imageStripDepthTokens: 0, maxLiveImages: 0 } as ConstructorParameters[0]), + [3], + ); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); + }); + + it('ceilingBoundLine marks a default or malformed ceiling, and gives bytes in MiB', () => { + const configured = (o: Record) => + ({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...o }); + assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['imageCeiling'])), + '[s] imageCeiling 500 (default) holds maxLiveImages 0 (no limit) to it, so some images are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'images', configured({ imageCeiling: Number.NaN, maxLiveImages: 800 }), new Set()), + '[s] imageCeiling 500 (default) holds maxLiveImages 800 to it, so some images are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB }), new Set()), + '[s] imageByteCeiling 24 MiB holds maxLiveImageBytes 30 MiB to it, so some images are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['imageByteCeiling'])), + `[s] imageByteCeiling 24 MiB (default) holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so some images are now shown as placeholders`); + }); + + it('the memory strategy says it when its hysteresis rule trims against the byte ceiling', async () => { + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), + [5], + ); + assert.equal(last, '...II', 'one trim to half the ceiling'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageByteCeiling 16 bytes holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); + }); +}); + +describe('compression prompts within the image ceilings', () => { + it('carry at most imageCeiling, the newest messages first', () => { + const prepared = new Prepared({ imageCeiling: 2 }).prepare( + request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), + ); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imageCountCap, 2); + assert.equal(prepared.imagesKept, 2); + assert.deepEqual(prepared.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image']); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER); + }); + + it('count an image nested in a tool result toward the ceiling', () => { + const prepared = new Prepared({ imageCeiling: 1 }).prepare(request( + [imageBlock('AAAA')], + [{ type: 'tool_result', toolUseId: 't', content: [imageBlock('BBBB')] } as ContentBlock], + )); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imagesKept, 1); + const nested = (prepared.request.messages[1]!.content[0] as { content: ContentBlock[] }).content[0]!; + assert.equal(nested.type, 'image', 'the newer, nested image takes the one slot'); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER, 'and the older one is replaced'); + }); + + it('count a generated image toward the ceiling, so a newer one takes the last slot from an older image', () => { + const generated = { type: 'generated_image', data: 'BBBB', mimeType: 'image/png' } as ContentBlock; + const prepared = new Prepared({ imageCeiling: 1 }).prepare(request([imageBlock('AAAA')], [generated])); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imagesKept, 1); + assert.equal(prepared.request.messages[1]!.content[0]!.type, 'generated_image', 'the newer, generated image takes the one slot'); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER, 'and the older one is replaced'); + }); + + it('keep their byte budget within imageByteCeiling, and the images kept are what it lets through', () => { + const prepared = new Prepared({ maxCompressionImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }).prepare(request([])); + assert.equal(prepared.imageBudgetBytes, 2 * MiB); + for (const none of [0, Infinity, -3, Number.NaN]) { + assert.equal(new Prepared({ maxCompressionImageBytes: none, imageByteCeiling: 8 }).prepare(request([])).imageBudgetBytes, 8, + `a budget of ${none} is none, so up to the byte ceiling`); + } + assert.equal(new Prepared({ maxCompressionImageBytes: 8.5, imageByteCeiling: 100 }).prepare(request([])).imageBudgetBytes, 8, + 'a fractional budget is floored'); + const walked = new Prepared({ maxCompressionImageBytes: 12, imageByteCeiling: 8 }).prepare( + request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), + ); + assert.equal(walked.imageBudgetBytes, 8); + assert.equal(walked.imageReplacements, 1, 'under the clamped 8 bytes, two 4-byte images fit and the oldest does not'); + assert.equal(walked.imageBytesKept, 8); + assert.deepEqual(walked.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image'], 'the oldest is the one replaced'); + assert.equal(new Prepared({ maxCompressionImageBytes: 0, imageByteCeiling: 2 * MiB }).prepare(request([])).imageBudgetBytes, + 2 * MiB, 'an unlimited budget means up to the ceiling'); + assert.equal(new Prepared().prepare(request([])).imageBudgetBytes, 12 * MiB, 'the default budget is under the default ceiling'); + }); + + it('carry at most imageCeiling with no byte budget, and say only the ceiling', () => { + const strategy = new Prepared({ maxCompressionImageBytes: 0, imageByteCeiling: 0, imageCeiling: 1 }); + const input = request([imageBlock('AAAA')], [imageBlock('BBBB')]); + assert.equal(strategy.prepare(input).imageReplacements, 1); + assert.deepEqual(captureErrors(() => strategy.send(input, 'no-budget')), [ + '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + + '1-image ceiling (kept 1 image(s), 0.0MiB, newest messages first) [no-budget]', + ]); + }); + + it('with no count ceiling, keep to the byte budget, and the log says only the budget', () => { + const strategy = new Prepared({ maxCompressionImageBytes: 8, imageCeiling: 0 }); + const input = request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]); + const prepared = strategy.prepare(input); + assert.equal(prepared.imageCountCap, 0); + assert.equal(prepared.imageReplacements, 1, 'two 4-byte images fit the 8-byte budget, the oldest does not'); + assert.deepEqual(prepared.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image']); + const logs = captureErrors(() => strategy.send(input, 'no-count')); + assert.equal(logs.length, 1); + assert.ok(!logs[0]!.includes('image ceiling'), logs[0]); + assert.ok(logs[0]!.includes('inline image budget (kept 2 image(s)'), logs[0]); + }); + + it('with no budget and no ceilings, keep every image, more than the default ceiling of 500', () => { + const images = Array.from({ length: 501 }, () => [imageBlock('AAAA')]); + const prepared = new Prepared({ maxCompressionImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0 }).prepare( + request(...images), + ); + assert.equal(prepared.imageReplacements, 0); + assert.equal(prepared.request.messages.filter((m) => m.content[0]!.type === 'image').length, 501, 'all 501 are still images'); + assert.equal(prepared.imagesKept, 501, 'the counts say what was kept'); + assert.equal(prepared.imageBytesKept, 501 * 4); + }); +}); diff --git a/test/system-changes.test.ts b/test/system-changes.test.ts new file mode 100644 index 00000000..2ee82577 --- /dev/null +++ b/test/system-changes.test.ts @@ -0,0 +1,1294 @@ +/** + * System changes (membrane's `system_change` block, SystemChangeContent): a + * message of them changes the tools and system text the model is offered from + * its place on (src/system-changes.ts). context-manager prices one from its + * JSON, keeps one alone in its message (a write refuses one that isn't, and + * a read leaves it out), compiles one in place and unaltered under every + * strategy (the autobiographical strategy keeps it raw as a pin keeps a + * message, and puts the ones before a reset head in the head, ahead of the + * head window; the passthrough strategies keep it when their window starts + * after it), refuses over budget when the changes alone don't fit (beyond + * grace, where enforcement is on), or, under the passthrough strategies, + * when the newest conversation message doesn't fit beside them, + * and leaves it out of what compression reads. A committing compile reports + * `firstChanged`, where its messages first differ from the last committing + * compile's. + */ +import { describe, it, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock } from '@animalabs/membrane'; +import { + AutobiographicalStrategy, + ContextManager, + KnowledgeStrategy, + MalformedContentError, + OverBudgetError, + PassthroughStrategy, + UncoveredDropError, + WindowedPassthroughStrategy, + malformedContentPlaces, +} from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; +import { jsonTokenEstimator } from '../src/message-store.js'; +import { wellFormedContent } from '../src/well-formed-content.js'; +import { systemChangePositions } from '../src/system-changes.js'; +import type { CompileResult, ContextStrategy, StoredMessage, TokenBudget } from '../src/types/index.js'; + +const dir = mkdtempSync(join(tmpdir(), 'cm-system-changes-')); +after(() => rmSync(dir, { recursive: true, force: true })); +let opened = 0; + +const tool = { name: 'get_weather', description: 'Weather.', inputSchema: { type: 'object', properties: { city: { type: 'string' } } } }; +const change = (extra: Record = {}): ContentBlock => + ({ type: 'system_change', defineTools: [tool], addSystem: ['Use metric units.'], ...extra }) as unknown as ContentBlock; +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); +const toolUse = (id: string): ContentBlock => ({ type: 'tool_use', id, name: 'get_weather', input: { city: 'Oslo' } }) as ContentBlock; +const toolResult = (id: string, t = 'Sunny.'): ContentBlock => ({ type: 'tool_result', toolUseId: id, content: t }) as ContentBlock; +const BIG = { maxTokens: 1_000_000, reserveForResponse: 0 }; + +async function open(strategy: ContextStrategy, membrane?: unknown): Promise { + return ContextManager.open({ path: join(dir, `store-${opened++}`), strategy, ...(membrane ? { membrane: membrane as never } : {}) }); +} + +const holdsChange = (content: unknown[]): boolean => content.some((b) => (b as { type?: string }).type === 'system_change'); +/** The content of each compiled message that holds a change, in order. */ +const changesSent = (compiled: CompileResult): unknown[] => compiled.messages.filter((m) => holdsChange(m.content)).map((m) => m.content); +/** The tools a compile's changes leave offered, each applied in compiled order as membrane applies one: its removals, then its definitions. */ +const toolsAfter = (compiled: CompileResult): string[] => { + const offered = new Set(); + for (const m of compiled.messages) { + for (const b of m.content as Array<{ type?: string; removeTools?: string[]; defineTools?: Array<{ name: string }> }>) { + if (b.type !== 'system_change') continue; + for (const name of b.removeTools ?? []) offered.delete(name); + for (const t of b.defineTools ?? []) offered.add(t.name); + } + } + return [...offered]; +}; +const estimateOf = (cm: ContextManager, id: string): number => + (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore.estimateTokens(cm.getMessage(id)!); +/** The stored message ids a compile sends raw, in its order (a summary's recall pair adds none). */ +const rawIds = (compiled: CompileResult): string[] => + compiled.provenance!.messages.flatMap((s) => (s.kind === 'raw' ? s.bodies.map((b) => b.messageId) : [])); +/** What a compile sends of the store, priced by the store: every raw message's estimate. */ +const sentTokens = (cm: ContextManager, compiled: CompileResult): number => rawIds(compiled).reduce((n, id) => n + estimateOf(cm, id), 0); +const KV_UNIFIED = { + policy: { + alpha: 0.7, budgetLowRatio: 0.5, budgetHighRatio: 0.9, budgetUnderLambda: 10, budgetOverLambda: 10, + cacheLambda: 1, cacheScale: 1000, cacheReadPrice: 0.1, cacheWritePrice: 1.25, + continuityLambda: 1, continuityScale: 1000, continuityRecencyHalfLifeTokens: 1000, continuityRecencyFloor: 0.2, + continuityStableHalfLife: 10, continuityStableFloor: 0.25, + }, + tokenBucketSize: 100, continuityBucketSize: 100, fidelityBucketSize: 100, labelCeiling: 10_000, adoptEpsilon: 0, + treeifyNonContiguousSummaries: false, preserveGapBearingSummaries: false, +}; +/** A summarizer stand-in that records every request it gets. */ +function recordingMembrane(): { requests: unknown[]; membrane: unknown } { + const requests: unknown[] = []; + return { requests, membrane: { complete: async (req: unknown) => { + requests.push(JSON.parse(JSON.stringify(req))); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `Summary #${requests.length}.` }], usage: { inputTokens: 500, outputTokens: 100 } }; + } } }; +} +async function settle(cm: ContextManager, requests: unknown[]): Promise { + let before = -1; + for (let r = 0; r < 200 && requests.length !== before; r++) { before = requests.length; await cm.compile(); await cm.tick(); await cm.tick(); } +} + +describe('a system change, written and priced', () => { + it('is priced from its JSON, where it used to cost nothing', async () => { + const cm = await open(new PassthroughStrategy()); + const block = change(); + const id = cm.addMessage('Harness', [block]); + const store = (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore; + const estimate = store.estimateTokens(cm.getMessage(id)!); + assert.equal(estimate, jsonTokenEstimator(JSON.stringify(block))); + assert.ok(estimate > 0); + // The context log prices it by the same rule, with its own estimator. + const log = (cm as unknown as { contextLog: { estimateTokens(entry: { content: ContentBlock[] }): number; tokenEstimator(t: string): number } }).contextLog; + assert.equal(log.estimateTokens({ content: [block] }), log.tokenEstimator(JSON.stringify(block))); + }); + + it('is refused beside other blocks, or inside a tool result, by name', async () => { + assert.deepEqual(malformedContentPlaces([text('hi'), change()]), [ + 'content[1] is a system_change beside other blocks; a system change goes in a message of its own', + ]); + assert.deepEqual(malformedContentPlaces([{ type: 'tool_result', toolUseId: 't1', content: [change()] }]), [ + 'content[0].content[0] is a system_change inside a tool_result; a system change goes in a message of its own', + ]); + const cm = await open(new PassthroughStrategy()); + assert.throws(() => cm.addMessage('Harness', [text('hi'), change()]), MalformedContentError); + // Beside a hole, which counts as another entry. + const sparse: unknown[] = [change()]; + sparse.length = 2; + assert.ok(malformedContentPlaces(sparse).includes('content[0] is a system_change beside other blocks; a system change goes in a message of its own'), JSON.stringify(malformedContentPlaces(sparse))); + assert.throws(() => cm.addMessage('Harness', sparse as ContentBlock[]), MalformedContentError); + assert.equal(cm.getAllMessages().length, 0); + }); + + it('is refused while the nearest call before it has no result between them, by the manager before an expired hold is released and by the store, and taken after the result', async () => { + const clock = { t: 0 }; + let released = 0; + const strategy = new (class extends PassthroughStrategy { + onCompressionHoldsReleased(): void { released++; } + })(); + const cm = await ContextManager.open({ path: join(dir, `store-${opened++}`), strategy, now: () => clock.t }); + cm.addMessage('User', [text('Weather in Oslo?')], undefined, undefined, { holdCompression: { timeoutMs: 100 } }); + const call = cm.addMessage('Claude', [toolUse('t1')]); + clock.t = 150; + assert.throws(() => cm.addMessage('Harness', [change()]), (e: Error) => e.message === + `Message refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (t1) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + assert.equal(released, 0, 'refused before the expired hold was released'); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[]): unknown } }).messageStore; + assert.throws(() => store.append('Harness', [change()]), /a system change can't go between a tool call and its result/); + assert.equal(cm.getAllMessages().length, 2); + cm.addMessage('User', [toolResult('t1')]); + assert.equal(released, 1); + // After the result, and after another change there. + cm.addMessage('Harness', [change()]); + cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]); + assert.equal(cm.getAllMessages().length, 5); + }); + + it('is refused while any of parallel calls is unanswered, their results in messages of their own, and taken once the last lands', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather in Oslo and Bergen?')]); + const call = cm.addMessage('Claude', [toolUse('a'), toolUse('b')]); + cm.addMessage('User', [toolResult('a')]); + assert.throws(() => cm.addMessage('Harness', [change()]), (e: Error) => e.message === + `Message refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (b) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + cm.addMessage('User', [toolResult('b')]); + cm.addMessage('Harness', [change()]); + assert.equal(cm.getAllMessages().length, 5); + }); + + it("is refused after a call on a sharded body's first shard, whichever shard is last, and the refusal names the shard holding the call", async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[], metadata?: unknown, causedBy?: unknown, extra?: unknown): StoredMessage } }).messageStore; + cm.addMessage('User', [text('Summarize this and look it up.')]); + // Ingress sharding puts a message's non-text blocks on its body's first shard. + const first = store.append('Claude', [text('part one '), toolUse('t1')], undefined, undefined, { bodyGroupId: 'g', shardIndex: 0, shardCount: 2 }); + store.append('Claude', [text('part two')], undefined, undefined, { bodyGroupId: 'g', shardIndex: 1, shardCount: 2 }); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${first.id} holds a tool_use \\(t1\\) with no result between it and this change`)); + cm.addMessage('User', [toolResult('t1')]); + cm.addMessage('Harness', [change()]); + }); + + it("reads a sharded results message whole, from its first shard, walking back from a change and forward from an edited call", async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[], metadata?: unknown, causedBy?: unknown, extra?: unknown): StoredMessage } }).messageStore; + cm.addMessage('User', [text('Weather in Oslo and Bergen?')]); + const call = cm.addMessage('Claude', [toolUse('a'), toolUse('b')]); + // Ingress sharding puts a message's results on its first shard, its long text after. + store.append('User', [toolResult('a')], undefined, undefined, { bodyGroupId: 'r', shardIndex: 0, shardCount: 2 }); + store.append('User', [text('a long note beside the result')], undefined, undefined, { bodyGroupId: 'r', shardIndex: 1, shardCount: 2 }); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${call} holds a tool_use \\(b\\) with no result between it and this change`)); + cm.addMessage('User', [toolResult('b')]); + const later = cm.addMessage('Harness', [change()]); + // Forward: an edit giving a message calls that the sharded results answer only in part, before the change. + cm.editMessage(call, [text('Checking.')]); + assert.throws(() => cm.editMessage(call, [toolUse('a'), toolUse('c')]), (e: Error) => e.message === + `Edit of ${call} refused: a tool_use (c) can't go before a system change (message ${later}) that comes before its result; a change goes after the call's result.`); + cm.editMessage(call, [toolUse('a'), toolUse('b')]); + assert.equal(cm.getAllMessages().length, 6); + }); + + it('takes a change after a bundled cycle, whose own later blocks answer its calls, an edit giving one to a message right before a change, and an edit giving a call to a message a plain one follows', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + // A bundled cycle, as claude.ai exports store one: the compile splits it (splitMixedToolMessages). + cm.addMessage('Claude', [text('Checking.'), toolUse('t1'), toolResult('t1'), text('Sunny.')]); + cm.addMessage('Harness', [change()]); + const reply = cm.addMessage('Claude', [text('Anything else?')]); + cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]); + cm.editMessage(reply, [toolUse('t2'), toolResult('t2')]); + // A plain message between a call and the next change ends the forward walk, as it ends the back walk. + const asked = cm.addMessage('Claude', [text('Let me see.')]); + cm.addMessage('User', [text('ok')]); + cm.addMessage('Harness', [change({ addSystem: ['Be kind.'] })]); + cm.editMessage(asked, [toolUse('t3')]); + // A call given to the last message: the forward walk ends with the store. + const last = cm.addMessage('Claude', [text('Bye.')]); + cm.editMessage(last, [toolUse('t4')]); + assert.deepEqual(cm.getAllMessages().map((m) => m.content.map((b) => b.type).join('+')), + ['text', 'text+tool_use+tool_result+text', 'system_change', 'tool_use+tool_result', 'system_change', 'tool_use', 'text', 'system_change', 'tool_use']); + }); + + it("leaves open a bundled message's call that none of its later blocks answers: a parallel call beside an answered one, and a call after its result", async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + const parallel = cm.addMessage('Claude', [toolUse('a'), toolUse('b'), toolResult('a')]); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${parallel} holds a tool_use \\(b\\) with no result`)); + cm.addMessage('User', [toolResult('b')]); + cm.addMessage('Harness', [change()]); + // A result before its call in the same message answers nothing: the compile splits it ahead of the call. + const reversed = cm.addMessage('Claude', [toolResult('c'), toolUse('c')]); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${reversed} holds a tool_use \\(c\\) with no result`)); + // And an edit giving a message right before a change such a call is refused. + assert.throws(() => cm.editMessage(cm.getAllMessages()[2]!.id, [toolUse('d'), toolUse('e'), toolResult('d')]), /a tool_use \(e\) can't go before a system change/); + }); + + it('refuses nothing for a call further back than a message of another kind, whose result never came', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + cm.addMessage('Claude', [toolUse('t1')]); + // An interrupted round: the call's result never came, and the conversation went on. + cm.addMessage('User', [text('Never mind.')]); + cm.addMessage('Harness', [change()]); + assert.equal(cm.getAllMessages().length, 4); + }); + + it('refuses a tool result answering a call that a change follows, as a removal can leave them, and another change there', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + const call = cm.addMessage('Claude', [toolUse('t1'), toolUse('t2')]); + const result = cm.addMessage('User', [toolResult('t1')]); + cm.addMessage('User', [toolResult('t2')]); + const first = cm.addMessage('Harness', [change()]); + // Removing a result leaves its call unanswered before the change. + cm.removeMessage(result); + assert.throws(() => cm.addMessage('User', [toolResult('t1')]), (e: Error) => e.message === + `Message refused: a tool_result can't follow a system change (message ${first}) that follows its tool call (t1, message ${call}). ` + + 'Remove the change and write it after the result.'); + assert.throws(() => cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]), new RegExp(`message ${call} holds a tool_use \\(t1\\)`)); + // A result answering no call there isn't refused. + cm.addMessage('User', [toolResult('other')]); + cm.addMessage('User', [text('Never mind.')]); + assert.equal(cm.getAllMessages().length, 6); + }); + + it('refuses an edit that puts a change between a call and its result, from either side, through the manager and the store, and takes one that keeps them together', async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { edit(id: string, content: ContentBlock[]): void } }).messageStore; + cm.addMessage('User', [text('Weather?')]); + const call = cm.addMessage('Claude', [toolUse('t1')]); + const result = cm.addMessage('User', [toolResult('t1')]); + const reply = cm.addMessage('Claude', [text('Sunny.')]); + const partial = cm.addMessage('User', [toolResult('t2')]); + const later = cm.addMessage('Harness', [change()]); + assert.throws(() => cm.editMessage(result, [change()]), (e: Error) => e.message === + `Edit of ${result} refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (t1) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + assert.throws(() => store.edit(result, [change()]), /a system change can't go between a tool call and its result/); + // A call whose results, in the messages after it, run into a change first. + assert.throws(() => cm.editMessage(reply, [toolUse('t2'), toolUse('t3')]), (e: Error) => e.message === + `Edit of ${reply} refused: a tool_use (t3) can't go before a system change (message ${later}) that comes before its result; a change goes after the call's result.`); + cm.editMessage(reply, [toolUse('t2')]); + cm.editMessage(result, [toolResult('t1', 'Rainy.')]); + assert.deepEqual(cm.getAllMessages().map((m) => m.content[0]!.type), ['text', 'tool_use', 'tool_result', 'tool_use', 'tool_result', 'system_change']); + assert.equal(cm.getAllMessages()[4]!.id, partial); + }); + + it('is refused when a field that carries its content has the wrong type, each field and element by name, and taken with several changes alone', () => { + for (const [fields, place] of [ + [{ removeTools: 'get_time' }, 'content[0].removeTools is a string, not an array of tool names'], + [{ removeTools: ['get_time', 1] }, 'content[0].removeTools[1] is a number, not a tool name (a string)'], + [{ defineTools: tool }, 'content[0].defineTools is an object, not an array of tool definitions'], + [{ defineTools: [tool, [tool]] }, 'content[0].defineTools[1] is an array, not a tool definition (an object)'], + [{ defineTools: [{ ...tool, name: '' }] }, 'content[0].defineTools[0].name is an empty string, not a tool name'], + [{ defineTools: [{ ...tool, name: 7 }] }, 'content[0].defineTools[0].name is a number, not a tool name (a string)'], + [{ defineTools: [{ name: 'x', inputSchema: tool.inputSchema }] }, 'content[0].defineTools[0].description is undefined, not a string'], + [{ defineTools: [{ ...tool, inputSchema: 'object' }] }, 'content[0].defineTools[0].inputSchema is a string, not a JSON Schema (an object)'], + [{ defineTools: [{ ...tool, inputSchema: {} }] }, "content[0].defineTools[0].inputSchema.type is undefined, not 'object'"], + [{ defineTools: [{ ...tool, inputSchema: { type: 'string' } }] }, "content[0].defineTools[0].inputSchema.type is a string, not 'object'"], + [{ addSystem: 'Use metric units.' }, 'content[0].addSystem is a string, not an array of strings'], + [{ addSystem: [null] }, 'content[0].addSystem[0] is null, not a string'], + ] as const) { + assert.deepEqual(malformedContentPlaces([change(fields)]), [place], JSON.stringify(fields)); + } + assert.deepEqual(malformedContentPlaces([change(), change({ removeTools: ['get_time'], defineTools: undefined, addSystem: undefined })]), []); + }); + + it('is left out by a read where a write would refuse it, the rest kept, with one line naming where', () => { + const lines: string[] = []; + const log = { error: (line: string) => { lines.push(line); }, warn: () => {}, info: () => {}, debug: () => {} } as never; + const read = (content: unknown[]): unknown[] => wellFormedContent(content as never, 'message m-1', new Set(), log); + // Alone and well formed: as stored, and nothing logged. + const alone = [change(), change({ removeTools: ['get_time'] })]; + assert.equal(read(alone), alone); + assert.equal(lines.length, 0); + // Beside other blocks: the change goes, the conversation stays. + assert.deepEqual(read([text('hi'), change()]), [text('hi')]); + assert.match(lines.pop()!, /message m-1: .*content\[1\] is a system_change beside other blocks/); + // Inside a tool result: the result keeps what else it holds. + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't1', content: [text('42'), change()] }]), [{ type: 'tool_result', toolUseId: 't1', content: [text('42')] }]); + assert.match(lines.pop()!, /content\[0\]\.content\[1\] is a system_change inside a tool_result/); + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't2', content: [change()] }]), [{ type: 'tool_result', toolUseId: 't2', content: [] }]); + assert.match(lines.pop()!, /content\[0\]\.content\[0\] is a system_change inside a tool_result/); + // A field of the wrong type: the change goes. + assert.deepEqual(read([change({ addSystem: [2] })]), []); + assert.match(lines.pop()!, /content\[0\]\.addSystem\[0\] is a number, not a string/); + // A definition without the fields the provider needs: the change goes. + assert.deepEqual(read([change({ defineTools: [{ name: 'x' }] })]), []); + assert.match(lines.pop()!, /content\[0\]\.defineTools\[0\]\.description is undefined, not a string/); + // A schema that isn't of type 'object', which the API refuses: the change goes. + assert.deepEqual(read([change({ defineTools: [{ ...tool, inputSchema: {} }] })]), []); + assert.match(lines.pop()!, /content\[0\]\.defineTools\[0\]\.inputSchema\.type is undefined, not 'object'/); + // Beside a hole, which counts as another entry: the change goes. + const sparse: unknown[] = [change()]; + sparse.length = 2; + assert.deepEqual(read(sparse), []); + assert.match(lines.pop()!, /content\[0\] is a system_change beside other blocks/); + // A damaged tool result whose content is one change object, not an array: left out, and named. + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't3', content: change() }]), [{ type: 'tool_result', toolUseId: 't3', content: [] }]); + assert.match(lines.pop()!, /message m-1: .*content\[0\]\.content/); + }); +}); + +describe('the autobiographical strategy and a system change', () => { + /** Every stored position a compiled message stands for: its raw bodies, or its summaries' source ranges. */ + const spansOf = (cm: ContextManager, strategy: AutobiographicalStrategy, compiled: CompileResult): Array<[number, number] | null> => { + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + return compiled.provenance!.messages.map((s) => { + const at: number[] = s.kind === 'raw' ? s.bodies.map((b) => position.get(b.messageId)!) + : s.kind === 'summary' ? s.summaries.flatMap((x) => { + const range = strategy.getSummary(x.id)!.sourceRange; + return [position.get(range.first)!, position.get(range.last)!]; + }) : []; + return at.length ? [Math.min(...at), Math.max(...at)] : null; + }); + }; + + for (const [label, overrides, headWindowTokens] of [ + ['hierarchical', {}, 0], + ['adaptive', { adaptiveResolution: true }, 0], + ['hierarchical, the head window holding a change', {}, 300], + ['adaptive, the head window holding a change', { adaptiveResolution: true }, 300], + ] as const) { + it(`${label}: no fold takes it, it compiles exactly in place as stored, and no summarizer request reads it`, async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new AutobiographicalStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', hierarchical: true, mergeThreshold: 6, ...overrides, + }); + const cm = await open(strategy, membrane); + const block = change(); + const headBlock = change({ addSystem: ['Answer briefly.'] }); + let changeId = ''; + for (let i = 0; i < 160; i++) { + if (i === 41) changeId = cm.addMessage('Harness', [block]); + else if (i === 1 && headWindowTokens > 0) cm.addMessage('Harness', [headBlock]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await settle(cm, requests); + assert.ok(requests.length > 5, `setup: the strategy compressed (${requests.length} calls)`); + const compiled = await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const changes = compiled.messages.filter((m) => holdsChange(m.content)); + assert.equal(changes.length, headWindowTokens > 0 ? 2 : 1, 'the changes are in the compile'); + const at = compiled.messages.indexOf(changes.at(-1)!); + assert.deepEqual(compiled.messages[at]!.content, [block]); + assert.ok(compiled.messages.length < 160, 'setup: the history folded'); + // Everything before it stands for history before it, and everything after for history after. + const spans = spansOf(cm, strategy, compiled); + const position = cm.getAllMessages().findIndex((m) => m.id === changeId); + spans.forEach((span, i) => { + if (!span || i === at) return; + if (i < at) assert.ok(span[1] < position, `compiled ${i} (${span}) comes before the change at ${position}`); + else assert.ok(span[0] > position, `compiled ${i} (${span}) comes after the change at ${position}`); + }); + assert.ok(!JSON.stringify(requests).includes('system_change'), 'no summarizer request carries one'); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.length > 0 && chunks.every((c) => c.messages.every((m) => m.id !== changeId)), 'no chunk holds it'); + }); + } + + for (const [label, overrides, opts] of [ + ['hierarchical', {}, undefined], + ['adaptive', { adaptiveResolution: true }, undefined], + ['kv-unified', { adaptiveResolution: true, foldingStrategy: 'kv-unified', kvUnified: KV_UNIFIED }, { kvUnifiedImmutablePrefixHash: 'v1' }], + ] as const) { + it(`${label}: after a head-window reset, the changes before the head are part of it, ahead of the head window, in order, once`, async () => { + const strategy = new AutobiographicalStrategy({ headWindowTokens: 400, recentWindowTokens: 100, autoTickOnNewMessage: false, ...overrides } as never); + const cm = await open(strategy); + const define = change(); + const remove = change({ defineTools: undefined, addSystem: undefined, removeTools: ['get_weather'] }); + const inHead = change({ defineTools: undefined, addSystem: ['Answer briefly.'] }); + const ids: string[] = []; + for (let i = 0; i < 30; i++) { + if (i === 5) ids.push(cm.addMessage('Harness', [define])); + else if (i === 12) ids.push(cm.addMessage('Harness', [remove])); + else if (i === 22) ids.push(cm.addMessage('Harness', [inHead])); + else ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(10)}`)])); + } + strategy.resetHeadWindow(ids[20]!); + // No render stats before a live compile (#311). + assert.equal(cm.getRenderStats(), null, 'no render stats before a compile'); + const compiled = await cm.compile(BIG, undefined, opts as never); + const order = rawIds(compiled); + const where = (i: number): number => order.indexOf(ids[i]!); + assert.ok(where(21) === where(20) + 1 && where(0) > where(20), 'setup: the head runs from message 20, and the history before it comes after it'); + assert.deepEqual([where(5), where(12), where(20), where(21), where(22)], [0, 1, 2, 3, 4], 'the changes lead, in their order, then the head window, a change inside it in its place'); + assert.equal(order.filter((id) => id === ids[5] || id === ids[12] || id === ids[22]).length, 3, 'each once'); + assert.deepEqual(toolsAfter(compiled), [], 'the removal applies after the definition it removes'); + // The head costs what it sends: the changes before it and the window, each once. + const head = order.slice(0, where(0)); + const headTokens = head.reduce((n, id) => n + estimateOf(cm, id), 0); + assert.deepEqual(cm.getRenderStats()!.head, { messages: head.length, tokens: headTokens }, "the compile's render stats"); + // The adaptive paths reserve the head at that price: below it they refuse the head and tail, and say what the head cost. + if (label !== 'hierarchical') { + await assert.rejects(cm.compile({ maxTokens: headTokens - 1, reserveForResponse: 0 }, undefined, opts as never), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.equal(err.diagnostics.headTokens, headTokens); + return true; + }); + } + }); + } + + for (const [label, overrides, opts] of [ + ['hierarchical', {}, undefined], + ['adaptive', { adaptiveResolution: true }, undefined], + ['kv-unified', { adaptiveResolution: true, foldingStrategy: 'kv-unified', kvUnified: KV_UNIFIED }, { kvUnifiedImmutablePrefixHash: 'v1' }], + ] as const) { + it(`${label}: with the head window disabled, a reset sends the history before it once, in order, a change in it in its place`, async () => { + const strategy = new AutobiographicalStrategy({ headWindowTokens: 0, recentWindowTokens: 100, autoTickOnNewMessage: false, ...overrides } as never); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 30; i++) ids.push(i === 15 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(10)}`)])); + // A reset before the recent window starts, and one inside it. + for (const at of [20, 27]) { + strategy.resetHeadWindow(ids[at]!); + assert.deepEqual(rawIds(await cm.compile(BIG, undefined, opts as never)), ids, `reset at ${at}`); + } + }); + } + + it('adaptive: a change after the newest turn is no turn, so a structural repair that drops that turn still refuses', async () => { + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, headWindowTokens: 400, recentWindowTokens: 0, autoTickOnNewMessage: false }); + const cm = await open(strategy); + cm.addMessage('User', [text('message ' + 'word '.repeat(10))]); + cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + const result = cm.addMessage('User', [{ type: 'tool_result', toolUseId: 't1', content: 'read.' } as ContentBlock]); + const trailing = cm.addMessage('Harness', [change()]); + // Removing the result leaves an unanswered call as the newest turn, right + // before the change (a write can't put one there). A reset at the change + // puts it in the head, so the call is the last compiled message, and the + // structural repair trims it. + cm.removeMessage(result); + strategy.resetHeadWindow(trailing); + await assert.rejects(cm.compile(BIG), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Structural repair did not retain the newest turn/); + return true; + }); + }); + + it('a transition summary reads no change: a head message of changes alone adds no entry to its request', async () => { + const { requests, membrane } = recordingMembrane(); + const cm = await open(new AutobiographicalStrategy({ + compressionModel: 'm', headWindowTokens: 100_000, recentWindowTokens: 100_000, autoTickOnNewMessage: false, + }), membrane); + cm.addMessage('User', [text('first')]); + cm.addMessage('Harness', [change()]); + cm.addMessage('Claude', [text('second')]); + await cm.resetHeadWindow(); + assert.equal(requests.length, 1); + const sent = JSON.stringify(requests[0]); + assert.ok(sent.includes('User: first') && sent.includes('Claude: second'), 'setup: the head is in the request'); + assert.ok(!sent.includes('Harness'), 'no entry for the message of changes'); + }); + + it('the coverage check names an unrendered change before the head\'s start as the head\'s only while the head window holds messages', async () => { + const strategy = new AutobiographicalStrategy({ autoTickOnNewMessage: false }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 10; i++) ids.push(i === 3 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i}`)])); + const messages = cm.getAllMessages() as StoredMessage[]; + // Every message but the change, as a compile that lost it would hand the check. + const entries = messages.filter((m) => m.id !== ids[3]).map((m, index) => ({ index, sourceMessageId: m.id, sourceRelation: 'raw' as const, participant: m.participant, content: m.content })); + const check = strategy as unknown as { assertFullCoverage(e: unknown[], m: StoredMessage[], r: { headStart: number; headEnd: number; recentStart: number }): void }; + const label = (headStart: number, headEnd: number): string => { + try { check.assertFullCoverage(entries, messages, { headStart, headEnd, recentStart: 8 }); } catch (err) { + assert.ok(err instanceof UncoveredDropError, String(err)); + return err.site; + } + return assert.fail('no refusal'); + }; + // After a reset at 5 with the head window holding messages, the change at 3 is the head's (headPositions). + assert.match(label(5, 8), /in-head-not-rendered=1/); + // With the head window empty or disabled, no head holds it: it's history before the head's start. + assert.match(label(5, 5), /before-head-start-not-rendered=1/); + assert.match(label(5, 0), /before-head-start-not-rendered=1/); + }); + + it('KnowledgeStrategy: no chunk holds or spans a change, and a chunk closes there', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + let changeId = ''; + for (let i = 0; i < 80; i++) { + if (i === 21) changeId = cm.addMessage('Harness', [change()]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.some((c) => c.messages.some((m) => position.get(m.id)! < 21)), 'setup: chunks before the change'); + assert.ok(chunks.every((c) => c.messages.every((m) => m.id !== changeId)), 'no chunk holds it'); + for (const chunk of chunks) { + const at = chunk.messages.map((m) => position.get(m.id)!); + assert.ok(!(at[0]! < 21 && at.at(-1)! > 21), `chunk ${at[0]}..${at.at(-1)} doesn't span the change`); + } + // It closes only the chunk it ends: the chunks after it hold several messages, as the rest do. + const later = chunks.filter((c) => position.get(c.messages[0]!.id)! > 21); + assert.ok(later.length > 0 && later.every((c) => c.messages.length > 1), `chunks after it: ${later.map((c) => c.messages.length)}`); + }); + + it('KnowledgeStrategy: no chunk holds or spans a change first, side by side, beside a pin or last, none is empty, and a pair of changes closes one chunk', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + const at = [0, 30, 31, 51, 79]; + const ids: string[] = []; + for (let i = 0; i < 80; i++) { + ids.push(at.includes(i) ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + cm.pinRange(ids[50]!, ids[50]!); + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.length > 4, `setup: ${chunks.length} chunks`); + assert.ok(chunks.every((c) => c.messages.length > 0), 'no chunk is empty'); + for (const chunk of chunks) { + const span = chunk.messages.map((m) => position.get(m.id)!); + for (const c of at) { + assert.ok(!span.includes(c), `chunk ${span[0]}..${span.at(-1)} doesn't hold the change at ${c}`); + assert.ok(!(span[0]! < c && span.at(-1)! > c), `chunk ${span[0]}..${span.at(-1)} doesn't span the change at ${c}`); + } + } + // The two changes side by side close one chunk between them, not two. + const after = chunks.find((c) => position.get(c.messages[0]!.id) === 32); + assert.ok(after && after.messages.length > 1, `the chunk after the pair holds ${after?.messages.length} messages`); + }); + + it('KnowledgeStrategy: once a change between two compressed chunks is removed, each keeps its span, so compression goes on past them', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 30; i++) { + ids.push(i === 22 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + const span = (c: { messages: Array<{ id: string }> }): string => c.messages.map((m) => m.id).join(','); + await settle(cm, requests); + const compressed = chunks().filter((c) => c.compressed).map(span); + assert.ok(compressed.some((s) => s.endsWith(ids[21]!)) && compressed.some((s) => s.startsWith(ids[23]!)), 'setup: the chunks either side of the change are compressed'); + cm.removeMessage(ids[22]!); + for (let i = 30; i < 45; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await settle(cm, requests); + for (const s of compressed) assert.ok(chunks().map(span).includes(s), 'each compressed chunk keeps its span'); + assert.ok(chunks().slice(0, -1).every((c) => c.compressed), `every chunk but the newest is compressed: ${chunks().map((c) => (c.compressed ? 'C' : 'u')).join('')}`); + }); + + it('KnowledgeStrategy: a summary ending at an unanswered call doesn\'t part the call from the result that comes after it', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + for (let i = 0; i < 9; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const use = cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + await settle(cm, requests); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'setup: a summary ends at the call'); + const result = cm.addMessage('User', [{ type: 'tool_result', toolUseId: 't1', content: 'read it' } as ContentBlock]); + await cm.compile(BIG); + assert.ok(!chunks().some((c) => c.messages.at(-1)!.id === use), 'no chunk ends at the call now that its result follows it'); + assert.ok(chunks().some((c) => c.messages.some((m) => m.id === use) && c.messages.some((m) => m.id === result)), 'the call and its result share a chunk'); + // As on main, that chunk overlaps the summary, which refuses it, so both go out raw, the result with its call. + await settle(cm, requests); + const compiled = await cm.compile(BIG); + const sent = rawIds(compiled); + assert.ok(sent.includes(use) && sent.includes(result), 'the call and its result are sent raw'); + const blocks = compiled.messages.flatMap((m) => m.content as Array<{ type: string; id?: string; toolUseId?: string; content?: unknown }>); + assert.ok(blocks.some((b) => b.type === 'tool_use' && b.id === 't1'), "the call's tool_use is sent"); + assert.ok(blocks.some((b) => b.type === 'tool_result' && b.toolUseId === 't1' && b.content === 'read it'), 'the result is sent with its content'); + }); + + it('KnowledgeStrategy: a summary ending at a call that a plain message follows still closes there', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + for (let i = 0; i < 9; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const use = cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + await settle(cm, requests); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'setup: a summary ends at the call'); + // No result comes; the conversation goes on. + const after = cm.addMessage('User', [text(`never mind ${'word '.repeat(30)}`)]); + await cm.compile(BIG); + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'the summary keeps its chunk'); + assert.ok(!chunks().some((c) => c.messages.some((m) => m.id === use) && c.messages.some((m) => m.id === after)), 'the message after it starts a chunk of its own'); + }); + + it('KnowledgeStrategy: a chunk closes where a summary covers its messages, even once the summary\'s last message is pinned, and where an unpinned message precedes a summary', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 30; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + // A message pinned before anything compresses is in no summary. + const early = cm.pinRange(ids[12]!, ids[12]!); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + const chunkOf = (id: string): Array<{ id: string }> | undefined => chunks().find((c) => c.messages.some((m) => m.id === id))?.messages; + await settle(cm, requests); + // The last message of the summary holding M-20, pinned after it compressed. + const covered = chunks().find((c) => c.compressed && c.messages.some((m) => m.id === ids[20]))!; + const last = covered.messages.at(-1)!.id; + const next = ids[ids.indexOf(last) + 1]!; + assert.ok(chunks().filter((c) => c.compressed).length >= 3 && next !== undefined, 'setup: compressed chunks, and a message after the one pinned'); + cm.pinRange(last, last); + cm.unpin(early); + for (let i = 30; i < 45; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await settle(cm, requests); + const beforeLast = covered.messages.at(-2)!.id; + assert.ok(chunkOf(beforeLast), 'setup: the message before the pinned one is in a chunk'); + assert.ok(!chunkOf(beforeLast)!.some((m) => m.id === next), 'no chunk runs on past a summary whose last message is pinned'); + // Summaries made before it was unpinned, which don't cover it. + const others = new Set((strategy as unknown as { summaries: Array<{ level: number; sourceIds: string[] }> }).summaries + .filter((x) => x.level === 1 && !x.sourceIds.includes(ids[12]!)).flatMap((x) => x.sourceIds)); + assert.ok(chunkOf(ids[12]!)!.every((m) => !others.has(m.id)), 'the unpinned message shares no chunk with another summary\'s messages'); + }); + + it('KnowledgeStrategy: once a change is removed, the chunks rebuilt around where it was run across that place', async () => { + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 40; i++) { + ids.push(i === 22 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + const chunks = (): Array<{ messages: Array<{ id: string }> }> => (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + const spans = (): string[][] => chunks().map((c) => c.messages.map((m) => m.id)); + await cm.compile(BIG); + assert.ok(spans().some((s) => s.at(-1) === ids[21]) && spans().some((s) => s[0] === ids[23]), 'setup: a chunk ends at the message before the change, and one starts after it'); + cm.removeMessage(ids[22]!); + await cm.compile(BIG); + assert.ok(spans().some((s) => s.includes(ids[21]!) && s.includes(ids[23]!)), `a chunk now holds both neighbours: ${JSON.stringify(spans().map((s) => s.length))}`); + }); + + it('KnowledgeStrategy: a store without changes chunks by its other rules, so a chunk can span a pinned message', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 80; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + cm.pinRange(ids[21]!, ids[21]!); + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.some((c) => { + const at = c.messages.map((m) => position.get(m.id)!); + return at[0]! < 21 && at.at(-1)! > 21; + }), 'a chunk runs across the pin'); + }); + + it('stays raw under a level pin over it, where pin bounds are read', async () => { + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, foldingStrategy: 'kv-stable', autoTickOnNewMessage: false } as never); + const cm = await open(strategy); + const ids = [cm.addMessage('User', [text('a')]), cm.addMessage('Harness', [change()]), cm.addMessage('User', [text('b')])]; + cm.pinRange(ids[0]!, ids[2]!, { name: 'level-1', level: 1 }); + const bounds = (strategy as unknown as { resolvedPinBounds(m: StoredMessage[]): Map }).resolvedPinBounds(cm.getAllMessages()); + assert.deepEqual([bounds.get(0), bounds.get(1), bounds.get(2)], [{ level: 1 }, 'raw', { level: 1 }]); + }); + + it('compression input passes over a message it left empty: no separator where it sat, and its neighbours join', () => { + const collapse = (AutobiographicalStrategy.prototype as unknown as { + collapseConsecutiveMessages(m: Array<{ participant: string; content: ContentBlock[] }>): Array<{ participant: string; content: ContentBlock[] }>; + }).collapseConsecutiveMessages; + const sep = text('\n\n---\n\n'); + assert.deepEqual(collapse([{ participant: 'User', content: [text('a')] }, { participant: 'User', content: [] }, { participant: 'User', content: [text('b')] }]), + [{ participant: 'User', content: [text('a'), sep, text('b')] }]); + assert.deepEqual(collapse([{ participant: 'User', content: [text('a')] }, { participant: 'Harness', content: [] }, { participant: 'User', content: [text('b')] }]), + [{ participant: 'User', content: [text('a'), sep, text('b')] }]); + // An empty first message, and nothing but empty ones. + assert.deepEqual(collapse([{ participant: 'Harness', content: [] }, { participant: 'User', content: [text('a')] }]), + [{ participant: 'User', content: [text('a')] }]); + assert.deepEqual(collapse([{ participant: 'User', content: [] }, { participant: 'Claude', content: [] }]), []); + }); + + it("finds a listing's changes once, and a new listing's afresh", () => { + const m = (...content: ContentBlock[]): { content: ContentBlock[] } => ({ content }); + const listing = [m(text('a')), m(change()), m(text('b'))]; + assert.deepEqual(systemChangePositions(listing), [1]); + assert.equal(systemChangePositions(listing), systemChangePositions(listing), 'the same listing: the same answer'); + assert.notEqual(systemChangePositions([...listing]), systemChangePositions(listing), 'a new listing: found again'); + }); + + it('a large change is never sharded at ingress', async () => { + const cm = await open(new AutobiographicalStrategy({ compressionModel: 'm', targetChunkTokens: 200, adaptiveResolution: true, autoTickOnNewMessage: false })); + cm.addMessage('Harness', [change({ addSystem: ['x '.repeat(5000)] })]); + assert.equal(cm.getAllMessages().length, 1); + }); +}); + +describe('the passthrough strategies and a system change before their window', () => { + /** + * `first`, a change A, M-00..M-15, a change B, M-16..M-19: B sits among the + * newest five messages, and the budget fits both changes and those five. + */ + async function seeded(strategy: ContextStrategy): Promise<{ cm: ContextManager; ids: Map; T: number; A: number; B: number }> { + const cm = await open(strategy); + const ids = new Map(); + ids.set('first', cm.addMessage('User', [text('first ' + 'word '.repeat(50))])); + ids.set('A', cm.addMessage('Harness', [change()])); + for (let i = 0; i < 20; i++) { + if (i === 16) ids.set('B', cm.addMessage('Harness', [change({ addSystem: ['Use metric units, always.'] })])); + const label = `M-${String(i).padStart(2, '0')}`; + ids.set(label, cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`${label} ${'word '.repeat(30)}`)])); + } + const T = estimateOf(cm, ids.get('M-00')!); + const A = estimateOf(cm, ids.get('A')!); + const B = estimateOf(cm, ids.get('B')!); + assert.ok(B > T / 2, `setup: a change counted twice would cost a message (B ${B}, T ${T})`); + return { cm, ids, T, A, B }; + } + const expected = ['A', 'M-15', 'B', 'M-16', 'M-17', 'M-18', 'M-19']; + const labelsOf = (ids: Map, compiled: CompileResult): string[] => { + const label = new Map([...ids].map(([k, v]) => [v, k] as const)); + return rawIds(compiled).map((id) => label.get(id)!); + }; + + it('passthrough: when the oldest no longer fit, the changes are sent ahead of the rest, counted once, within the budget', async () => { + const { cm, ids, T, A, B } = await seeded(new PassthroughStrategy()); + const budget = A + B + 5 * T + Math.floor(T / 2); + const compiled = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + assert.deepEqual(labelsOf(ids, compiled), expected); + assert.ok(sentTokens(cm, compiled) <= budget); + assert.deepEqual(changesSent(compiled), [cm.getMessage(ids.get('A')!)!.content, cm.getMessage(ids.get('B')!)!.content], 'each as stored'); + }); + + it('windowed-passthrough: a re-anchor past a change keeps it ahead of the window, sizes the window as that fraction of the room the changes leave, and no compile exceeds the budget', async () => { + const strategy = new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 }); + const { cm, ids, T, A, B } = await seeded(strategy); + const budget = 2 * (A + B + 5 * T) + T; + const compiled = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + assert.ok(strategy.getAnchor() > cm.getMessage(ids.get('A')!)!.sequence, 'setup: the anchor moved past the change'); + const labels = labelsOf(ids, compiled); + assert.equal(labels[0], 'A', 'the change before the window leads'); + assert.deepEqual(changesSent(compiled), [cm.getMessage(ids.get('A')!)!.content, cm.getMessage(ids.get('B')!)!.content], 'each once, as stored'); + // The window's turns are the newest that fit in half the room the changes leave, and no more. + const room = (budget - A - B) * 0.5; + const turns = labels.filter((l) => l.startsWith('M-')); + const W = turns.reduce((n, l) => n + estimateOf(cm, ids.get(l)!), 0); + const older = `M-${String(Number(turns[0]!.slice(2)) - 1).padStart(2, '0')}`; + assert.ok(W <= room && W + estimateOf(cm, ids.get(older)!) > room, `window ${W}, room ${room}, next older ${older}`); + // Appends grow the window until the next re-anchor, the change before it priced in all along. + let anchors = 0; + for (let i = 20; i < 60; i++) { + const anchor = strategy.getAnchor(); + cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const next = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + if (strategy.getAnchor() !== anchor) anchors++; + assert.ok(sentTokens(cm, next) <= budget, `compile after M-${i}: ${sentTokens(cm, next)} within ${budget}`); + assert.ok(holdsChange(next.messages[0]!.content), 'a change still leads'); + } + assert.ok(anchors >= 2, `setup: the window moved again (${anchors})`); + }); + + it('windowed-passthrough: when the changes fill the target, the window still starts at the newest turn, not at a trailing change', async () => { + const cm = await open(new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 })); + cm.addMessage('User', [text('old ' + 'word '.repeat(40))]); + const big = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + const user = cm.addMessage('User', [text('the newest turn ' + 'word '.repeat(10))]); + const last = cm.addMessage('Harness', [change({ addSystem: ['Use metric units.'] })]); + const C = estimateOf(cm, big) + estimateOf(cm, last); + const U = estimateOf(cm, user); + assert.ok(U > (U + 5) * 0.5, 'setup: the turn alone passes the window the changes leave room for, so the progress clause picks it'); + const compiled = await cm.compile({ maxTokens: C + U + 5, reserveForResponse: 0 }); + assert.deepEqual(rawIds(compiled), [big, user, last]); + }); + + it('every strategy refuses with OverBudgetError, rather than drop one, when the changes alone exceed the budget, and names nothing that would clear it', async () => { + const huge = change({ addSystem: ['Rule. '.repeat(400)] }); + const budget: TokenBudget = { maxTokens: 300, reserveForResponse: 0 }; + for (const [label, strategy] of [ + ['passthrough', new PassthroughStrategy()], + ['windowed-passthrough', new WindowedPassthroughStrategy()], + ['hierarchical', new AutobiographicalStrategy({ autoTickOnNewMessage: false })], + ['adaptive', new AutobiographicalStrategy({ adaptiveResolution: true, autoTickOnNewMessage: false })], + ] as const) { + const cm = await open(strategy); + cm.addMessage('User', [text('hello')]); + cm.addMessage('Harness', [huge]); + cm.addMessage('User', [text('again')]); + await assert.rejects(cm.compile(budget), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); + assert.deepEqual(err.clearedBy, [], `${label}: only consolidating clears it`); + if (label === 'passthrough') assert.match(err.message, /^Passthrough kept only its system changes/, label); + if (label === 'passthrough') assertStage(err, 'passthrough-system-changes'); + // Its window still holds the newest conversation message, which its numbers count. + if (label === 'windowed-passthrough') assert.match(err.message, /^Windowed passthrough: its system changes and the newest conversation message/, label); + if (label === 'windowed-passthrough') assertStage(err, 'windowed-changes-and-newest-message'); + if (label === 'hierarchical') assert.match(err.message, /system changes/, label); + if (label === 'hierarchical') assertStage(err, 'hierarchical-system-changes'); + return true; + }); + } + // A store of changes alone: over the budget, each passthrough strategy names them alone; + // filling it exactly, each sends them. + for (const [label, make, stage, stageId] of [ + ['passthrough', () => new PassthroughStrategy(), /^Passthrough kept only its system changes/, 'passthrough-system-changes'], + ['windowed-passthrough', () => new WindowedPassthroughStrategy(), /^Windowed passthrough: its system changes alone/, 'windowed-system-changes'], + ] as const) { + const alone = await open(make()); + const id = alone.addMessage('Harness', [huge]); + await assert.rejects(alone.compile(budget), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); + assert.match(err.message, stage, label); + assertStage(err, stageId); + return true; + }); + assert.deepEqual(rawIds(await alone.compile({ maxTokens: estimateOf(alone, id), reserveForResponse: 0 })), [id], label); + } + // A preview reports rather than refuses: passthrough sends the change alone. + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('hello')]); + const id = cm.addMessage('Harness', [huge]); + assert.deepEqual(rawIds(await cm.compile(budget, undefined, { dryRun: true })), [id]); + }); + + it('the hierarchical path refuses an eviction from its recent window while the changes fit, as it refuses any', async () => { + const cm = await open(new AutobiographicalStrategy({ autoTickOnNewMessage: false })); + cm.addMessage('Harness', [change()]); + for (let i = 0; i < 12; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await assert.rejects(cm.compile({ maxTokens: 300, reserveForResponse: 0 }), UncoveredDropError); + }); + + it('adaptive: a plan that can\'t fit beside a change in the middle names nothing, though compression is queued', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, recentWindowTokens: 50, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + for (let i = 0; i < 30; i++) { + if (i === 10) cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(400)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await assert.rejects(cm.compile({ maxTokens: 300, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, []); + return true; + }); + }); + + it('adaptive: a plan refusal names compression only while the head, the tail and the changes between them fit', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 200, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + let c = ''; + for (let i = 0; i < 30; i++) { + if (i === 10) c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + const C = estimateOf(cm, c); + const refusal = async (maxTokens: number): Promise => { + try { await cm.compile({ maxTokens, reserveForResponse: 0 }); } catch (err) { assert.ok(err instanceof OverBudgetError, String(err)); return err; } + return assert.fail(`no refusal at ${maxTokens}`); + }; + // The change alone fits, beside the tail it doesn't: compression can't clear that. + const tight = await refusal(C + 10); + assert.ok(tight.diagnostics.tailTokens > 10, `setup: tail ${tight.diagnostics.tailTokens}, change ${C}`); + assert.deepEqual(tight.clearedBy, []); + // Room for the tail and the change, but not the unfolded middle: compression can. + const roomy = await refusal(tight.diagnostics.tailTokens + C + 20); + assert.deepEqual(roomy.clearedBy, ['compression']); + }); + + it('adaptive: a change in the recent window counts once toward what compression can\'t shrink', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + let c = ''; + for (let i = 0; i < 30; i++) { + if (i === 28) c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + const C = estimateOf(cm, c); + let first: OverBudgetError | undefined; + try { await cm.compile({ maxTokens: C + 1, reserveForResponse: 0 }); } catch (err) { first = err as OverBudgetError; } + // The tail holds the change; a budget between the tail and the tail with the change again fits it once, not the middle. + const tail = first!.diagnostics.tailTokens; + assert.ok(tail > C, `setup: the tail (${tail}) holds the change (${C})`); + await assert.rejects(cm.compile({ maxTokens: tail + Math.floor(C / 2), reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, ['compression']); + return true; + }); + }); + + it('adaptive: a change a reset left in the head counts once toward what compression can\'t shrink', async () => { + const strategy = new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 200, recentWindowTokens: 200, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + }); + const cm = await open(strategy); + let c = ''; + const ids: string[] = []; + for (let i = 0; i < 40; i++) { + if (i === 10) ids.push(c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })])); + else ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + strategy.resetHeadWindow(ids[20]!); + const C = estimateOf(cm, c); + let first: OverBudgetError | undefined; + try { await cm.compile({ maxTokens: 1, reserveForResponse: 0 }); } catch (err) { first = err as OverBudgetError; } + assert.ok(first instanceof OverBudgetError, String(first)); + const { headTokens, tailTokens } = first.diagnostics; + assert.ok(headTokens > C, `setup: the head (${headTokens}) holds the change (${C})`); + // Room for the head and the tail, the change among them once, but not the unfolded middle: compression can. + await assert.rejects(cm.compile({ maxTokens: headTokens + tailTokens + Math.floor(C / 2), reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.doesNotMatch(err.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(err.clearedBy, ['compression']); + return true; + }); + }); + + it('adaptive: a refusal of the head and tail finds the newest conversation message past the changes after it, so it names a smaller recent window only when that message fits beside them', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, headWindowTokens: 0, recentWindowTokens: 100_000, overBudgetGraceRatio: 0, autoTickOnNewMessage: false, + })); + for (let i = 0; i < 6; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const turn = cm.addMessage('User', [text('the newest turn ' + 'word '.repeat(60))]); + const c = cm.addMessage('Harness', [change()]); + const U = estimateOf(cm, turn); + const C = estimateOf(cm, c); + const refusal = async (maxTokens: number): Promise => { + try { await cm.compile({ maxTokens, reserveForResponse: 0 }); } catch (err) { assert.ok(err instanceof OverBudgetError, String(err)); return err; } + return assert.fail(`no refusal at ${maxTokens}`); + }; + const tight = await refusal(U + C - 1); + assert.match(tight.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(tight.clearedBy, [], 'the newest message and the change after it don\'t fit together'); + const roomy = await refusal(U + C + 5); + assert.match(roomy.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(roomy.clearedBy, ['smaller-recent-window']); + }); + + for (const [label, make] of [ + ['passthrough', () => new PassthroughStrategy()], + ['windowed-passthrough', () => new WindowedPassthroughStrategy()], + ] as const) { + it(`${label}: refuses when the newest conversation message doesn't fit beside its changes, naming a newer one, which clears it, while they leave room for one`, async () => { + const cm = await open(make()); + const c = cm.addMessage('Harness', [change()]); + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(400))]); + const C = estimateOf(cm, c); + const U = estimateOf(cm, big); + await assert.rejects(cm.compile({ maxTokens: C + 10 + 50, reserveForResponse: 50 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, ['newer-message']); + assert.match(err.message, /its system changes and the newest conversation message/); + assert.deepEqual([err.budget, err.actual], [C + 10, C + U], 'the budget less the reserve, and the changes with the newest message'); + assert.deepEqual([err.diagnostics.headTokens, err.diagnostics.tailTokens], [C, U], 'the changes, then the newest message'); + return true; + }); + // Changes that fill the budget leave no room for any message. + await assert.rejects(cm.compile({ maxTokens: C, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, []); + return true; + }); + // A change after the newest message: that message is still the one priced as the newest. + const trailing = cm.addMessage('Harness', [change({ addSystem: ['Answer briefly.'] })]); + const C2 = estimateOf(cm, trailing); + await assert.rejects(cm.compile({ maxTokens: C + C2 + 10, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual([err.diagnostics.headTokens, err.diagnostics.tailTokens], [C + C2, U], 'the changes, then the newest conversation message'); + return true; + }); + // A newer conversation message that fits beside the changes clears it. + const small = cm.addMessage('User', [text('ok')]); + const compiled = await cm.compile({ maxTokens: C + C2 + estimateOf(cm, small), reserveForResponse: 0 }); + assert.deepEqual(rawIds(compiled), [c, trailing, small]); + }); + } + + it('passthrough: without changes, a newest message over the budget still sends nothing, as on main', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('hello')]); + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(400))]); + const compiled = await cm.compile({ maxTokens: estimateOf(cm, big) - 1, reserveForResponse: 0 }); + assert.equal(compiled.messages.length, 0); + // Nor does a response reserve larger than the budget make it refuse. + assert.equal((await cm.compile({ maxTokens: 10, reserveForResponse: 20 })).messages.length, 0); + // The windowed strategy refuses it, naming the newest message alone. + const windowed = await open(new WindowedPassthroughStrategy()); + windowed.addMessage('User', [text('hello')]); + const big2 = windowed.addMessage('User', [text('big ' + 'word '.repeat(400))]); + await assert.rejects(windowed.compile({ maxTokens: estimateOf(windowed, big2) - 1, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Windowed passthrough: the newest message alone/); + assertStage(err, 'windowed-newest-message'); + return true; + }); + // Passthrough's changes fit, but no conversation message fits beside them: + // it names both, and the newest message's tokens. + const both = await open(new PassthroughStrategy()); + const rule = both.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(40)] })]); + const newest = both.addMessage('User', [text('big ' + 'word '.repeat(400))]); + await assert.rejects(both.compile({ maxTokens: estimateOf(both, rule) + 10, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Passthrough: its system changes and the newest conversation message/); + assertStage(err, 'passthrough-changes-and-newest-message'); + assert.equal(err.diagnostics.tailTokens, estimateOf(both, newest)); + return true; + }); + }); + + it('windowed-passthrough: renderAsNewest gives a change before the anchor as select sends it, ahead of the window', async () => { + const strategy = new WindowedPassthroughStrategy(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + const id = cm.addMessage('Harness', [change()]); + const after = cm.addMessage('User', [text('b')]); + strategy.setAnchor(cm.getMessage(after)!.sequence); + const compiled = await cm.compile(BIG); + assert.deepEqual(changesSent(compiled), [[change()]], 'setup: select sends the change ahead of the window'); + assert.deepEqual(cm.renderAsNewest([id]), changesSent(compiled)); + }); + + it('windowed-passthrough: a refused compile leaves the anchor where the last sent window started', async () => { + const strategy = new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 6; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + const budget: TokenBudget = { maxTokens: 20 * estimateOf(cm, ids[0]!), reserveForResponse: 0 }; + await cm.compile(budget); + const anchor = strategy.getAnchor(); + const c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(2000)] })]); + cm.addMessage('User', [text('again')]); + assert.ok(estimateOf(cm, c) > budget.maxTokens, 'setup: the change alone is over the budget'); + await assert.rejects(cm.compile(budget), OverBudgetError); + assert.equal(strategy.getAnchor(), anchor, 'the refusal moved nothing'); + // Consolidated: the turns that fit are still in the window. + cm.removeMessage(c); + assert.equal(rawIds(await cm.compile(budget))[0], ids[0]); + // Without any change too: a newest message alone over the budget is refused, and a retry with a larger budget sends the older turns. + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(3000))]); + assert.ok(estimateOf(cm, big) > budget.maxTokens, 'setup: the newest alone is over the budget'); + await assert.rejects(cm.compile(budget), OverBudgetError); + assert.equal(strategy.getAnchor(), anchor, 'that refusal moved nothing either'); + assert.equal(rawIds(await cm.compile(BIG))[0], ids[0]); + }); +}); + +describe('firstChanged', () => { + it('is null at first, the count when nothing changed, the first new index after an append, and a dry compile neither reads nor moves it', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + assert.equal((await cm.compile(BIG)).firstChanged, null); + assert.equal((await cm.compile(BIG)).firstChanged, 2); + cm.addMessage('User', [text('c')]); + const dry = await cm.compile(BIG, undefined, { dryRun: true }); + assert.equal(dry.firstChanged, undefined); + assert.equal((await cm.compile(BIG)).firstChanged, 2); + // An injection placed before the last user message changes from there. + const injected = await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + assert.equal(injected.firstChanged, 2); + // Taking it out again changes from there too. + assert.equal((await cm.compile(BIG)).firstChanged, 2); + // An injection rebuilt as a new array of the same bytes is the same message. + await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + const same = await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + assert.equal(same.firstChanged, same.messages.length); + }); + + it('tells a participant apart from content, and sees an edit, a removal, and an injection changed in place', async () => { + const cm = await open(new PassthroughStrategy()); + const ids = [cm.addMessage('User', [text('a')]), cm.addMessage('Claude', [text('b')]), cm.addMessage('User', [text('c')])]; + await cm.compile(BIG); + // The same content under another participant: the injection's namespace names it. + await cm.compile(BIG, [{ namespace: 'one', position: 'afterUser', content: [text('note')] }]); + assert.equal((await cm.compile(BIG, [{ namespace: 'two', position: 'afterUser', content: [text('note')] }])).firstChanged, 3); + // An injection's own array, changed in place between compiles. + const note = [text('note v1')]; + await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }]); + note[0] = text('note v2'); + assert.equal((await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }])).firstChanged, 3); + await cm.compile(BIG); + cm.editMessage(ids[1]!, [text('b, edited')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + cm.removeMessage(ids[0]!); + assert.equal((await cm.compile(BIG)).firstChanged, 0); + }); + + it('takes an injection\'s bytes as it compiles, also when a binding compile sends it without its thinking', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + cm.addMessage('User', [text('c')]); + const binds = { + thinkingBinding: { binds: true, participant: 'Claude' }, + identity: { system: 'zz-system', tools: 'zz-tools', model: 'zz-model', request: 'zz-request' }, + }; + const said = text('note v1'); + const note = [{ type: 'thinking', thinking: 'zz-reasoning', signature: 'zz-sig' } as ContentBlock, said]; + const first = await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }], binds); + assert.deepEqual(first.messages[3]!.content, [text('note v1')], 'setup: the injection goes without its thinking, as a new array'); + // A block of the caller's, changed in place between compiles. + (said as { text: string }).text = 'note v2'; + assert.equal((await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }], binds)).firstChanged, 3); + }); + + it('is not compared with a compile that throws before it returns', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + await cm.compile(BIG); + cm.addMessage('User', [text('c')]); + const internals = cm as unknown as { renderedLayoutFor: (...args: unknown[]) => unknown }; + const layout = internals.renderedLayoutFor; + internals.renderedLayoutFor = () => { internals.renderedLayoutFor = layout; throw new Error('layout failed'); }; + await assert.rejects(cm.compile(BIG), /layout failed/); + // The same three messages again: the last compile that returned sent two. + assert.equal((await cm.compile(BIG)).firstChanged, 2); + }); + + it('is compared when read, serializing nothing again on a second read, and read after a later compile still compares with the compile before it', async () => { + // A strategy sending new arrays each compile, as caps and recall pairs + // do, whose blocks count each serialization. + let serialized = 0; + const counted = (block: ContentBlock): ContentBlock => ({ ...block, toJSON: () => { serialized++; return block; } }) as unknown as ContentBlock; + const strategy = new (class extends PassthroughStrategy { + select(...args: Parameters): ReturnType { + return super.select(...args).map((entry) => ({ ...entry, content: entry.content.map(counted) })); + } + })(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + const first = await cm.compile(BIG); + const second = await cm.compile(BIG); + const before = serialized; + assert.equal(second.firstChanged, 2, 'the same bytes in new arrays'); + assert.ok(serialized > before, 'the comparison serialized them when read'); + const once = serialized; + assert.equal(second.firstChanged, 2); + assert.equal(serialized, once, 'and not again'); + cm.editMessage(cm.getAllMessages()[1]!.id, [text('b, edited')]); + const third = await cm.compile(BIG); + cm.addMessage('User', [text('c')]); + await cm.compile(BIG); + assert.equal(first.firstChanged, null); + assert.equal(third.firstChanged, 1, 'read first after a later compile, the third against the second'); + assert.equal(second.firstChanged, 2, 'still the second compile against the first'); + }); + + it('serializes nothing on a read where the compiled arrays are the last compile\'s', async () => { + let serialized = 0; + const counted = (block: ContentBlock): ContentBlock => ({ ...block, toJSON: () => { serialized++; return block; } }) as unknown as ContentBlock; + const sent = new Map(); + const strategy = new (class extends PassthroughStrategy { + select(...args: Parameters): ReturnType { + return super.select(...args).map((entry) => { + const id = entry.sourceMessageId!; + if (!sent.has(id)) sent.set(id, entry.content.map(counted)); + return { ...entry, content: sent.get(id)! }; + }); + } + })(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + await cm.compile(BIG); + const second = await cm.compile(BIG); + const before = serialized; + assert.equal(second.firstChanged, 2); + assert.equal(serialized, before, 'the same arrays are compared by identity'); + }); + + it('is carried by a copy and JSON, read on a frozen result, takes an assignment, and leaves out system injections', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + await cm.compile(BIG); + const copied = await cm.compile(BIG); + assert.equal({ ...copied }.firstChanged, 1); + assert.equal(JSON.parse(JSON.stringify(copied)).firstChanged, 1); + const frozen = Object.freeze(await cm.compile(BIG)); + assert.equal(frozen.firstChanged, 1, 'a frozen result reads it'); + const assigned = await cm.compile(BIG); + assigned.firstChanged = 7; + assert.equal(assigned.firstChanged, 7); + const withSystem = await cm.compile(BIG, [{ namespace: 'n', position: 'system', content: [text('system note')] }]); + assert.equal(withSystem.firstChanged, withSystem.messages.length, 'a system injection goes in the system prompt, not the messages'); + }); + + it('does not count cache markers', async () => { + const cm = await open(new WindowedPassthroughStrategy()); + cm.addMessage('User', [text('a')]); + let previous = await cm.compile(BIG); + let moved = false; + for (let i = 0; i < 6 && !moved; i++) { + cm.addMessage(i % 2 ? 'User' : 'Claude', [text(`m${i}`)]); + const next = await cm.compile(BIG); + moved = previous.messages.some((m, k) => !!m.cacheBreakpoint !== !!next.messages[k]!.cacheBreakpoint); + if (moved) assert.equal(next.firstChanged, previous.messages.length, 'a marker moved, and nothing else changed'); + previous = next; + } + assert.ok(moved, 'setup: a marker moved on a message both compiles hold'); + }); + + it('compares bytes across a fork, a branch switch, a switch made on the store itself, and a new strategy, and is null only at the first compile', async () => { + const cm = await open(new PassthroughStrategy()); + const a = cm.addMessage('User', [text('a')]); + assert.equal((await cm.compile(BIG)).firstChanged, null); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + const main = cm.currentBranch().name; + await cm.fork(); + const forked = cm.currentBranch().name; + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'a fork holds the same messages'); + cm.addMessage('Claude', [text('on the fork')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + await cm.switchBranch(main); + assert.equal((await cm.compile(BIG)).firstChanged, 1, "back on main, a prefix of the fork's"); + const alt = cm.branchAt(a, 'alt'); + cm.addMessage('User', [text('b')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + cm.getStore().switchBranch(alt); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'after the host switched the store'); + await cm.setStrategy(new PassthroughStrategy()); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'a new strategy compiling the same messages'); + // Two branches of the same length and participants, whose second messages differ. + cm.addMessage('Claude', [text('on alt')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + await cm.switchBranch(forked); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'the fork differs from the second message on'); + }); +}); diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts new file mode 100644 index 00000000..ccd7038d --- /dev/null +++ b/test/thinking-binding.test.ts @@ -0,0 +1,1402 @@ +/** + * Thinking binding (#155): on a model that binds each signed thinking block + * to the request it was minted in, a compile leaves out the thinking whose + * request has changed since, against a per-branch record of what the branch + * last sent (src/thinking-binding.ts, SelectOptions.thinkingBinding). + */ + +import { describe, it, before, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { rmSync, existsSync } from 'node:fs'; +import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, PassthroughStrategy } from '../src/index.js'; +import type { + CompileResult, + CompiledMessageSources, + ContextEntry, + ContextInjection, + ContextLogView, + ContextStrategy, + MessageId, + MessageStoreView, + RawBodySource, + ReadinessState, + RequestIdentity, + ThinkingBinding, + ThinkingStrip, + TokenBudget, +} from '../src/index.js'; +import type { RequestIdentity as RequestIdentityFromTypes, SelectOptions, ThinkingStrip as ThinkingStripFromTypes } from '../src/types/index.js'; +import { UNVOUCHED, bindThinking, markThinkingLeftOut, messageDigest, type BindingRecord } from '../src/thinking-binding.js'; +import { renderAdaptiveFixture } from './fixtures/recall-envelope-fixture.js'; + +const STORE = './test-thinking-binding'; +const STORE_B = './test-thinking-binding-b'; + +function cleanup() { + for (const path of [STORE, STORE_B]) if (existsSync(path)) rmSync(path, { recursive: true, force: true }); +} + +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); +const thought = (n: number): ContentBlock => + ({ type: 'thinking', thinking: `zz-reasoning-${n} `.repeat(20), signature: `zz-sig-${n}-0123456789abcdef` }) as ContentBlock; +const redacted = (n: number): ContentBlock => ({ type: 'redacted_thinking', data: `zz-enc-${n}-payload==` }) as unknown as ContentBlock; +const isThinking = (b: ContentBlock) => b.type === 'thinking' || (b.type as string) === 'redacted_thinking'; +const toolUse = (id: string): ContentBlock => ({ type: 'tool_use', id, name: 'zz_search', input: { q: id } }) as ContentBlock; +const toolResult = (id: string, content: string): ContentBlock => ({ type: 'tool_result', toolUseId: id, content }) as ContentBlock; + +const identity = (over: Partial = {}): RequestIdentity => + ({ system: 'zz-system-1', tools: 'zz-tools-1', model: 'zz-model-1', request: 'zz-request-1', ...over }); +/** The options a binding compile takes, as agent-framework passes them. */ +const bind = (over: { dropBlock?: boolean; identity?: Partial; dryRun?: boolean } = {}): SelectOptions => { + const binding: ThinkingBinding = { binds: true, dropBlock: over.dropBlock === true, participant: 'Claude' }; + return { thinkingBinding: binding, identity: identity(over.identity), ...(over.dryRun ? { dryRun: true } : {}) }; +}; +const BIND = bind(); +const BIND_DROP = bind({ dropBlock: true }); +const BIG = { maxTokens: 1_000_000, reserveForResponse: 0 }; + +/** Which compiled messages carry thinking, by their text. */ +function thinkingByText(result: CompileResult): Record { + const out: Record = {}; + for (const m of result.messages) { + const label = m.content.filter((b) => b.type === 'text').map((b) => (b as { text: string }).text).join('|'); + out[label || m.content.map((b) => b.type).join('+')] = m.content.some(isThinking); + } + return out; +} + +async function open(path = STORE, strategy: ContextStrategy = new PassthroughStrategy(), namespace?: string) { + return ContextManager.open({ path, strategy, ...(namespace ? { namespace } : {}) }); +} + +/** U1, A1 (thinking), U2, A2 (thinking). */ +function seed(manager: ContextManager): { a1: MessageId; a2: MessageId } { + manager.addMessage('User', [text('U1')]); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + manager.addMessage('User', [text('U2')]); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + return { a1, a2 }; +} + +function recordOf(manager: ContextManager, namespace = 'default'): unknown { + return manager.getStore().getStateJson(`${namespace}/thinking-binding`); +} + +/** The compile id the branch's record holds, in the manager's namespace, if any. */ +function recordCompileId(manager: ContextManager): string { + const namespace = (manager as unknown as { strategyNamespace: string }).strategyNamespace; + const id = (recordOf(manager, namespace) as { compileId?: unknown } | null)?.compileId; + return typeof id === 'string' ? id : 'zz-no-record'; +} + +/** Note rows for the stream of the compile that wrote the branch's record, + * as a host does with that compile's provenance.compileId. */ +function note( + manager: ContextManager, + rows: ReadonlyArray, + identity: RequestIdentity, + compileId = recordCompileId(manager), +): boolean { + return manager.noteRequestSent(rows, identity, compileId); +} + +/** The budget the newest `n` messages take, as the passthrough strategy prices them. */ +function tokensOfNewest(manager: ContextManager, n: number): number { + const store = (manager as unknown as { messageStore: { estimateTokens(m: unknown): number } }).messageStore; + return manager.getAllMessages().slice(-n).reduce((sum, m) => sum + store.estimateTokens(m), 0); +} + +/** Passthrough that counts its selects, and can render a header ahead of + * each tool result message (`header`). */ +class Watching extends PassthroughStrategy { + selects = 0; + header = false; + select(store: MessageStoreView, log: ContextLogView, budget: TokenBudget): ContextEntry[] { + this.selects++; + return super.select(store, log, budget).map((entry) => { + const headed = this.header && entry.content.some((b) => b.type === 'tool_result'); + return headed ? { ...entry, content: [text('zz-header'), ...entry.content] } : entry; + }); + } +} + +/** U1, a committing compile, then a stream's first round: A1 calling a + * tool, and its result. */ +async function streamTurn(manager: ContextManager): Promise<{ a1: MessageId; r1: MessageId }> { + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + const r1 = manager.addMessage('User', [toolResult('t1', 'zz-result')]); + return { a1, r1 }; +} + +describe('thinking binding: compile', () => { + before(() => cleanup()); + after(() => cleanup()); + + it('without binds, a compile is unchanged and records nothing', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const plain = await manager.compile(BIG); + const off = await manager.compile(BIG, undefined, { thinkingBinding: { binds: false } }); + assert.deepEqual(off.messages, plain.messages); + assert.equal(off.thinkingStrip, undefined); + assert.equal(off.thinkingRecordedAfter, undefined); + assert.ok(plain.messages.filter((m) => m.content.some(isThinking)).length === 2, 'setup: both replies carry thinking'); + assert.equal(recordOf(manager), null); + await manager.close(); + }); + + it('with no record and no drop control, leaves out every reply\u2019s thinking once; a reply the host notes keeps its own', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const u3 = manager.addMessage('User', [text('U3')]); + const first = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(first), { U1: false, A1: false, U2: false, A2: false, U3: false }); + assert.deepEqual(first.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + assert.ok(first.thinkingRecordedAfter! >= manager.getMessage(u3)!.sequence, 'at or above every message stored before'); + + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + manager.addMessage('User', [text('U4')]); + const next = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(next), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: true, U4: false }); + assert.equal(next.thinkingStrip, undefined, 'nothing newly left out'); + assert.equal(next.thinkingRecordedAfter, first.thinkingRecordedAfter, 'where the record began stays put'); + await manager.close(); + }); + + it('with no record and the drop control, leaves the replies\u2019 thinking and starts the record', async () => { + cleanup(); + const manager = await open(); + const { a2 } = seed(manager); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(first), { U1: false, A1: true, U2: false, A2: true }); + assert.equal(first.thinkingStrip, undefined); + assert.ok(first.thinkingRecordedAfter! >= manager.getMessage(a2)!.sequence); + assert.notEqual(recordOf(manager), null); + await manager.close(); + }); + + it('a record begun under the drop control hands its earlier replies to no compile without it', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + manager.addMessage('User', [text('U3')]); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + const without = await manager.compile(BIG, undefined, BIND); + // A3 was minted after A1 and A2 with their thinking, so it goes too. + assert.deepEqual(thinkingByText(without), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: false }); + assert.deepEqual(without.thinkingStrip, { newly: [a1, a2, a3], cause: 'first' }); + const next = await manager.compile(BIG, undefined, BIND); + assert.equal(next.thinkingStrip, undefined, 'control: nothing newly left out after'); + await manager.close(); + }); + + it('a moved injection: replies after its old place lose their thinking, earlier ones keep it, and a lost one stays lost', async () => { + cleanup(); + const manager = await open(); + const { a2 } = seed(manager); + const lesson: ContextInjection = { namespace: 'zz-lessons', position: 'afterUser', content: [text('zz-lesson')] }; + const first = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(first.messages.map((m) => m.participant), ['User', 'Claude', 'User', 'system_context:zz-lessons', 'Claude']); + + manager.addMessage('User', [text('U3')]); + const moved = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(thinkingByText(moved), { U1: false, A1: true, U2: false, A2: false, U3: false, 'zz-lesson': false }); + assert.deepEqual(moved.thinkingStrip, { newly: [a2], cause: 'messages' }); + + const again = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(thinkingByText(again), thinkingByText(moved), 'A2 goes without again, and A1 keeps'); + assert.equal(again.thinkingStrip, undefined, 'A2 was left out before, so not newly'); + await manager.close(); + }); + + it('oldest messages leaving the window change the first message, so later replies lose their thinking, without the drop control too', async () => { + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + const kept = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(kept).A3, true, 'setup: A3 keeps its thinking'); + // A budget that fits only the newest two messages: the rest fall out. + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, BIND); + assert.deepEqual(thinkingByText(tight), { U3: false, A3: false }); + assert.deepEqual(tight.thinkingStrip, { newly: [a3], cause: 'messages' }); + await manager.close(); + }); + + it('a stream\u2019s replies keep their thinking when the host notes each round and the last reply, and lose it, never refused, when it doesn\u2019t', async () => { + for (const noted of ['every note', 'rounds only', 'none'] as const) { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + if (noted !== 'none') assert.equal(note(manager, [run.a1, run.r1], identity()), true); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + // The stream's end: the last reply is stored, then noted. + if (noted === 'every note') note(manager, [run.a1, run.r1, a2], identity()); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + const expected = { 'every note': [true, true], 'rounds only': [true, false], none: [false, false] }[noted]; + assert.deepEqual(thinkingByText(next), { U1: false, A1: expected[0], tool_result: false, A2: expected[1], U2: false }, noted); + const newly = [...(expected[0] ? [] : [run.a1]), ...(expected[1] ? [] : [a2])]; + assert.deepEqual(next.thinkingStrip, newly.length > 0 ? { newly, cause: 'messages' } : undefined, noted); + await manager.close(); + } + }); + + it('a write between a stream\u2019s rounds that the stream didn\u2019t send leaves out the reply after it', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + manager.addMessage('Other', [text('zz-interleaved')]); + const r1 = manager.addMessage('User', [toolResult('t1', 'zz-result')]); + note(manager, [a1, r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [a1, r1, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, 'zz-interleaved': false, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a write in an auxiliary slot between a stream\u2019s rounds leaves out the reply after it', async () => { + cleanup(); + const main = await open(); + const side = await ContextManager.open({ + store: main.getStore(), namespace: 'zz-side', isolate: true, strategy: new PassthroughStrategy(), auxiliaryMessageViews: [{}], + }); + side.addMessage('User', [text('U1')]); + await side.compile(BIG, undefined, BIND_DROP); + const a1 = side.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + main.addMessage('Other', [text('zz-auxiliary')]); + const r1 = side.addMessage('User', [toolResult('t1', 'zz-result')]); + note(side, [a1, r1], identity()); + const a2 = side.addMessage('Claude', [thought(2), text('A2')]); + note(side, [a1, r1, a2], identity()); + const next = await side.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, 'zz-auxiliary': false, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await side.close(); + await main.close(); + }); + + it('an edit to a noted row, which keeps its sequence, leaves out the reply after it', async () => { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const sequence = manager.getMessage(run.r1)!.sequence; + manager.editMessage(run.r1, [toolResult('t1', 'zz-edited')]); + assert.equal(manager.getMessage(run.r1)!.sequence, sequence, 'setup: the edit keeps the sequence'); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a noted row removed, or one the view filter leaves out, leaves out the reply after it', async () => { + for (const how of ['removed', 'filtered'] as const) { + cleanup(); + const manager = await ContextManager.open({ + path: STORE, strategy: new PassthroughStrategy(), + viewFilter: (message) => !(message.metadata as { hidden?: unknown } | undefined)?.hidden, + }); + const run = await streamTurn(manager); + const injected = manager.addMessage('User', [text('zz-injected')], how === 'filtered' ? { hidden: true } : undefined); + note(manager, [run.a1, run.r1, injected], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, injected, a2], identity()); + if (how === 'removed') manager.removeMessage(injected); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }, how); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }, how); + await manager.close(); + } + }); + + it('a row the compile renders with an addition leaves out the reply after it', async () => { + cleanup(); + const strategy = new Watching(); + const manager = await open(STORE, strategy); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const control = await manager.compile(BIG, undefined, { ...BIND_DROP, dryRun: true }); + assert.equal(thinkingByText(control).A2, true, 'control: rendered as stored, A2 keeps its thinking'); + strategy.header = true; + const decorated = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(thinkingByText(decorated).A2, false); + await manager.close(); + }); + + it('past a row noted as unstored, nothing keeps its thinking', async () => { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, { unstored: true }], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, { unstored: true }, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a noted row is digested as renderAsNewest renders it, and one it renders as null matches nothing', async () => { + for (const render of ['as-stored', 'null', 'otherwise'] as const) { + cleanup(); + class Rendering extends PassthroughStrategy { + renderAsNewest(store: MessageStoreView, ids: readonly MessageId[]): Array { + return ids.map((id) => { + const content = store.get(id)!.content; + const result = content.some((b) => b.type === 'tool_result'); + if (!result || render === 'as-stored') return content; + return render === 'null' ? null : [text('zz-other'), ...content]; + }); + } + } + const manager = await open(STORE, new Rendering()); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(thinkingByText(next).A2, render === 'as-stored', render); + await manager.close(); + } + }); + + it('with no renderAsNewest, a noted body stored as shards matches nothing', async () => { + cleanup(); + class Sharding extends PassthroughStrategy { + chunkIngressMessage(_participant: string, content: ContentBlock[]) { + const body = content.length === 1 && content[0]!.type === 'text' ? (content[0] as { text: string }).text : ''; + if (!body.startsWith('zz-shard')) return null; + const half = Math.floor(body.length / 2); + return { + bodyGroupId: 'zz-group', + shards: [ + { content: [text(body.slice(0, half))], shardIndex: 0 }, + { content: [text(body.slice(half))], shardIndex: 1 }, + ], + }; + } + } + const manager = await open(STORE, new Sharding()); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const before = manager.getAllMessages().length; + const shard = manager.addMessage('User', [text('zz-shard ' + 'y'.repeat(100))]); + assert.equal(manager.getAllMessages().length, before + 2, 'setup: stored as two shards'); + const plain = manager.addMessage('User', [text('zz-plain')]); + note(manager, [shard, plain], identity()); + const noted = recordOf(manager) as BindingRecord; + assert.equal(noted.continued[0], UNVOUCHED); + assert.notEqual(noted.continued[1], UNVOUCHED, 'control: an unsharded row is digested'); + await manager.close(); + }); + + it('noteRequestSent writes nothing on a branch with no record, and refuses what it can\u2019t record before writing', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + assert.equal(note(manager, [a1], identity()), false, 'no record yet'); + assert.equal(note(manager, ['zz-missing'], identity()), false, 'an unknown id, with no record'); + assert.equal(recordOf(manager), null); + await manager.compile(BIG, undefined, BIND); + const before = recordOf(manager) as BindingRecord; + assert.deepEqual(before.without, [a1], 'setup: the compile left A1 out'); + const recorded = JSON.stringify(recordOf(manager)); + const refused: Array<[string, () => unknown, RegExp]> = [ + ['rows not a list', () => note(manager, 'zz' as unknown as MessageId[], identity()), /`rows` must be an array/], + ['a row of neither kind', () => note(manager, [a1, { unstored: false } as unknown as { unstored: true }], identity()), /row 1 must be a message id or \{ unstored: true \}/], + ['no identity', () => note(manager, [a1], undefined as unknown as RequestIdentity), /`identity` must be an object/], + ['a digest not a string', () => note(manager, [a1], { ...identity(), model: 3 as unknown as string }), /`identity.model` must be a string, got number/], + ['a compileId not a string', () => manager.noteRequestSent([a1], identity(), undefined as unknown as string), /`compileId` must be a string, the provenance.compileId of the compile the stream was built from; got undefined/], + ]; + for (const [label, call, message] of refused) { + assert.throws(call, (error: unknown) => error instanceof Error && message.test(error.message), label); + } + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as it was'); + // An id the store doesn't hold, as after a removal, vouches for nothing, as the marker does. + assert.equal(note(manager, [a1, 'zz-missing', { unstored: true }], identity()), true); + const noted = recordOf(manager) as BindingRecord; + assert.equal(noted.continued.length, 3); + assert.notEqual(noted.continued[0], UNVOUCHED, 'control: a stored row is digested'); + assert.equal(noted.continued[1], UNVOUCHED); + assert.equal(noted.continued[2], UNVOUCHED); + assert.deepEqual(noted.sent, before.sent, 'the compiled part stays'); + assert.deepEqual(noted.without, before.without, 'and what it left out'); + assert.equal(noted.recordedAfter, before.recordedAfter, 'and where the record began'); + // A compile starts the request over. + await manager.compile(BIG, undefined, BIND); + assert.deepEqual((recordOf(manager) as BindingRecord).continued, []); + await manager.close(); + }); + + it('a note is for the stream of the compile that wrote the record: after another committing compile with binds it writes nothing, and a dry compile or one without binds changes nothing', async () => { + // previewContext calls the strategy alone and never reaches the record. + for (const between of ['a committing compile', 'a committing compile without binds', 'a dry compile', 'nothing'] as const) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND); + // Stored while C1's stream is open, so C1's reply was minted without it. + manager.addMessage('User', [text('zz-X')]); + if (between === 'a committing compile') await manager.compile(BIG, undefined, BIND); + if (between === 'a committing compile without binds') await manager.compile(BIG); + if (between === 'a dry compile') await manager.compile(BIG, undefined, bind({ dryRun: true })); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + const accepted = manager.noteRequestSent([a1], identity(), c1.provenance!.compileId); + assert.equal(accepted, between !== 'a committing compile', between); + const next = await manager.compile(BIG, undefined, BIND); + // Where C1's record still stands, X sits where its note put A1, so A1 + // goes without its thinking as well; only the committing compile's + // record would have lined A1 up with the note, and it refused the note. + assert.equal(thinkingByText(next).A1, false, between); + await manager.close(); + } + // The control: C1's own stream, noted under C1's id, keeps its reply's thinking. + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND); + await manager.compile(BIG, undefined, bind({ dryRun: true })); + // A compile without binds leaves the record, so C1's stream still notes. + await manager.compile(BIG); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + assert.equal(manager.noteRequestSent([a1], identity(), c1.provenance!.compileId), true); + assert.equal(manager.noteRequestSent([a1], identity(), 'zz-another-compile'), false, 'another compile\u2019s id writes nothing'); + assert.equal(manager.noteRequestSent([a1], identity(), c1.provenance!.compileId), true, 'a later note of the same stream, under the same id'); + const next = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(next).A1, true, 'control'); + await manager.close(); + }); + + it('each note carries every round so far; a note of the latest round alone leaves out the rounds before', async () => { + for (const cumulative of [true, false]) { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2'), toolUse('t2')]); + const r2 = manager.addMessage('User', [toolResult('t2', 'zz-result-2')]); + note(manager, cumulative ? [run.a1, run.r1, a2, r2] : [a2, r2], identity()); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, cumulative ? [run.a1, run.r1, a2, r2, a3] : [a3], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: cumulative, tool_result: false, A2: cumulative, A3: cumulative }, `cumulative: ${cumulative}`); + if (!cumulative) assert.deepEqual(next.thinkingStrip, { newly: [run.a1, a2, a3], cause: 'messages' }); + await manager.close(); + } + }); + + it('a note sent with another head leaves a record that vouches for no thinking, until the next committing compile', async () => { + for (const later of [false, true]) { + for (const next of ['new head', 'old head'] as const) { + const label = `${next}${later ? ', after a later note under the record\u2019s head' : ''}`; + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity({ tools: 'zz-tools-2' })); + // A later note under the record's own head doesn't take the mark away. + if (later) note(manager, [a3], identity()); + const head = next === 'new head' ? { tools: 'zz-tools-2' } : {}; + const after = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: head })); + assert.deepEqual(thinkingByText(after), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: false }, label); + // The record kept its own head, so only a compile under the new one reports a head change. + assert.equal(after.thinkingStrip?.cause, next === 'new head' ? 'head' : 'messages', label); + if (next === 'new head') assert.deepEqual(after.thinkingStrip?.changed, ['tools'], label); + // The compile starts a record again, which vouches as usual. + const again = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: head })); + assert.equal(again.thinkingStrip, undefined, `${label}: nothing newly left out once the compile records again`); + await manager.close(); + } + } + // A later note's rows can't line up with a compile's first messages either: + // here the window keeps only the noted reply and what follows it. + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity({ tools: 'zz-tools-2' })); + note(manager, [a3], identity()); + manager.addMessage('User', [text('U4')]); + const windowed = await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, { ...BIND_DROP, dryRun: true }); + assert.deepEqual(thinkingByText(windowed), { A3: false, U4: false }, 'the window starts at the noted reply, which goes without its thinking'); + await manager.close(); + }); + + it('a noted row removed, by this manager or another on the shared slot, mid-stream or after, leaves out the replies after it', async () => { + for (const how of ['this manager', 'another manager', 'mid-stream'] as const) { + cleanup(); + const main = await open(); + const side = await ContextManager.open({ store: main.getStore(), namespace: 'zz-side', strategy: new PassthroughStrategy() }); + side.addMessage('User', [text('U1')]); + await side.compile(BIG, undefined, BIND_DROP); + const a1 = side.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + const r1 = side.addMessage('User', [toolResult('t1', 'zz-result')]); + const extra = side.addMessage('User', [text('zz-extra')]); + note(side, [a1, r1, extra], identity()); + if (how === 'mid-stream') side.removeMessage(extra); + const a2 = side.addMessage('Claude', [thought(2), text('A2')]); + // The end note names the removed row by its id, which now vouches for nothing. + assert.equal(note(side, [a1, r1, extra, a2], identity()), true, how); + if (how === 'this manager') side.removeMessage(extra); + if (how === 'another manager') main.removeMessage(extra); + const next = await side.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }, how); + await side.close(); + await main.close(); + } + }); + + it('an edit to the noted last reply, ahead of its thinking or after it, leaves out its thinking', async () => { + for (const where of ['ahead', 'after'] as const) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + note(manager, [a1], identity()); + const control = await manager.compile(BIG, undefined, { ...BIND_DROP, dryRun: true }); + assert.equal(thinkingByText(control).A1, true, `control, ${where}`); + manager.editMessage(a1, where === 'ahead' ? [text('zz-edit'), thought(1), text('A1')] : [thought(1), text('A1 edited')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(next.thinkingStrip, { newly: [a1], cause: 'messages' }, where); + await manager.close(); + } + }); + + it('a single-round turn keeps its reply\u2019s thinking when the host notes it, and loses only that reply\u2019s when it doesn\u2019t, as when the host stops before its note', async () => { + for (const noted of [true, false]) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + note(manager, [a1], identity()); + manager.addMessage('User', [text('U2')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + if (noted) note(manager, [a2], identity()); + manager.addMessage('User', [text('U3')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, U2: false, A2: noted, U3: false }, `noted: ${noted}`); + assert.deepEqual(next.thinkingStrip, noted ? undefined : { newly: [a2], cause: 'messages' }, `noted: ${noted}`); + await manager.close(); + } + }); + + it('a noted row is digested as the compile sends it: a result under another participant as the user\u2019s, a bundled tool cycle as each part', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), toolUse('t1')]); + const r1 = manager.addMessage('Tool', [toolResult('t1', 'zz-result')]); + const b1 = manager.addMessage('Claude', [thought(2), toolUse('t2'), toolResult('t2', 'zz-bundled'), thought(3), text('B1')]); + assert.equal(manager.noteRequestSent([a1, r1, b1], identity(), c1.provenance!.compileId), true); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.ok(next.messages.some((m) => m.participant === 'user' && m.content.some((b) => b.type === 'tool_result')), 'setup: the compile sends results as the user\u2019s'); + assert.equal(next.messages.length, 7, 'setup: the bundled cycle goes as three messages'); + assert.equal(next.thinkingStrip, undefined); + assert.equal(next.messages.reduce((n, m) => n + m.content.filter(isThinking).length, 0), 3, 'every noted reply keeps its thinking'); + await manager.close(); + }); + + it('the layout prices a split body\u2019s thinking as each part sent it', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND_DROP); + // The first part's thinking includes a redacted block; the second part's + // costs more than the first's, so the price says which part kept its own. + const redacted = { type: 'redacted_thinking', data: 'zz-redacted-payload==' } as ContentBlock; + const long = { type: 'thinking', thinking: 'zz-long '.repeat(80), signature: 'zz-sig-long-0123456789' } as ContentBlock; + const b1 = manager.addMessage('Claude', [thought(1), redacted, toolUse('t1'), toolResult('t1', 'zz-bundled'), long, text('B1')]); + manager.noteRequestSent([b1], identity(), c1.provenance!.compileId); + // Placed after the cycle's result, the injection changes what precedes + // the second part, so its thinking goes and the first part's stays. + const late: ContextInjection = { namespace: 'zz-late', position: 'afterUser', content: [text('zz-late')] }; + const plain = await manager.compile(BIG, [late], { dryRun: true }); + const bound = await manager.compile(BIG, [late], { ...BIND_DROP, dryRun: true }); + const kept = bound.messages.flatMap((m) => m.content.filter(isThinking)); + assert.deepEqual(kept, [thought(1), redacted], 'setup: the first part keeps its thinking, the second goes without'); + const price = (blocks: ContentBlock[]) => (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens(blocks); + assert.ok(price([long]) > price([thought(1), redacted]), 'setup: the parts\u2019 thinking prices differ'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([long])); + await manager.close(); + }); + + it('a strategy that throws taking note of the thinking left out is logged, and the compile stands', async () => { + cleanup(); + class Throwing extends PassthroughStrategy { + noteThinkingLeftOut(): void { throw new Error('zz-note-failed'); } + } + const manager = await open(STORE, new Throwing()); + seed(manager); + const logged: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + let result: CompileResult; + try { + result = await manager.compile(BIG, undefined, BIND); + } finally { + console.error = error; + } + assert.equal(result.thinkingStrip?.cause, 'first'); + assert.notEqual(recordOf(manager), null, 'the record is written'); + assert.ok(logged.some((line) => /failed to take note of thinking left out/.test(line) && /zz-note-failed/.test(line)), logged.join('\n')); + await manager.close(); + }); + + it('a compile without binds on a branch with no record writes none', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; + const set = store.setStateJson.bind(store); + let writes = 0; + store.setStateJson = (id: string, value: unknown) => { + if (id === 'default/thinking-binding') writes++; + set(id, value); + }; + await manager.compile(BIG); + store.setStateJson = set; + assert.equal(writes, 0); + await manager.close(); + }); + + it('a reply long enough to shard and not noted goes without its thinking, every shard read as partial', async () => { + cleanup(); + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, targetChunkTokens: 50, autoTickOnNewMessage: false, headWindowTokens: 0 }); + const manager = await ContextManager.open({ path: STORE, strategy }); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const before = manager.getAllMessages().length; + const a1 = manager.addMessage('Claude', [thought(1), text('zz-long ' + 'w '.repeat(1_500))]); + const shards = manager.getAllMessages().length - before; + assert.ok(shards > 1, 'setup: the reply was stored as shards'); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + const reply = next.messages.find((m) => m.participant === 'Claude')!; + assert.ok(!reply.content.some(isThinking), 'the sharded reply goes without its thinking'); + assert.deepEqual(next.thinkingStrip, { newly: [a1], cause: 'messages' }); + const partial = next.provenance!.layout!.units.filter((unit) => (unit as { p?: number }).p === 1).length; + assert.equal(partial, shards, `every shard's unit is partial (${partial} of ${shards})`); + await manager.close(); + }); + + it('a shorter or empty record doesn\u2019t make older messages look stored since', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const empty = await manager.compile({ maxTokens: 1, reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(empty.messages.length, 0, 'setup: nothing fits'); + const back = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(back), { U1: false, A1: false, U2: false, A2: false }); + assert.deepEqual(back.thinkingStrip, { newly: [a1, a2], cause: 'messages' }); + await manager.close(); + }); + + for (const key of ['system', 'tools', 'model', 'request'] as const) { + it(`a changed ${key} digest leaves out every reply\u2019s thinking`, async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const same = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(same.thinkingStrip, undefined, 'control: the same digests change nothing'); + const changed = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: { [key]: 'zz-changed' } })); + assert.deepEqual(changed.thinkingStrip, { newly: [a1, a2], cause: 'head', changed: [key] }); + assert.deepEqual(thinkingByText(changed), { U1: false, A1: false, U2: false, A2: false }); + await manager.close(); + }); + } + + it('refuses a binding compile without participant or identity, or with a digest that isn\u2019t a string, before anything is selected or recorded', async () => { + cleanup(); + const strategy = new Watching(); + const manager = await open(STORE, strategy); + seed(manager); + await manager.compile(BIG, undefined, BIND); + const recorded = JSON.stringify(recordOf(manager)); + const selects = strategy.selects; + const refused: Array<[string, SelectOptions, RegExp]> = [ + ['no participant', { thinkingBinding: { binds: true }, identity: identity() }, /needs `thinkingBinding.participant`/], + ['an empty participant', { thinkingBinding: { binds: true, participant: '' }, identity: identity() }, /needs `thinkingBinding.participant`/], + ['no identity', { thinkingBinding: { binds: true, participant: 'Claude' } }, /needs `identity`: the digests of what the request carries before its messages/], + ...(['system', 'tools', 'model', 'request'] as const).map((key): [string, SelectOptions, RegExp] => [ + `a ${key} digest that isn't a string`, + { ...BIND, identity: { ...identity(), [key]: 7 as unknown as string } }, + new RegExp(`identity\\.${key}\` must be a string, got number`), + ]), + ]; + for (const [label, opts, message] of refused) { + await assert.rejects(manager.compile(BIG, undefined, opts), (error: unknown) => error instanceof TypeError && message.test(error.message), label); + } + assert.equal(strategy.selects, selects, 'no select ran'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as it was'); + await manager.close(); + }); + + it('a compile without binds neither reads nor writes the record; one an earlier binding compile left is compared like any other', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const recorded = JSON.stringify(recordOf(manager)); + // A store that can't give or take the record state doesn't touch a + // compile without the option, committing or dry. + const store = manager.getStore() as unknown as { getStateJson(id: string): unknown; setStateJson(id: string, value: unknown): void }; + const get = store.getStateJson.bind(store); + const set = store.setStateJson.bind(store); + let touched = 0; + store.getStateJson = (id: string) => { if (id === 'default/thinking-binding') { touched++; throw new Error('zz-store-unreadable'); } return get(id); }; + store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') { touched++; throw new Error('zz-store-unwritable'); } set(id, value); }; + await manager.compile(BIG); + await manager.compile(BIG, undefined, { dryRun: true }); + manager.addMessage('User', [text('U3')]); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + await manager.compile(BIG); + store.getStateJson = get; + store.setStateJson = set; + assert.equal(touched, 0, 'no read or write of the record'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as the binding compile left it'); + // Back on a binding model, the record is compared as it stands: the + // prefix that matches keeps its thinking, and what came after goes without. + const back = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(back), { U1: false, A1: true, U2: false, A2: true, U3: false, A3: false }); + assert.deepEqual(back.thinkingStrip, { newly: [a3], cause: 'messages' }); + void a1; void a2; + await manager.close(); + }); + + it('a reply a compile didn\u2019t send is reported again when it renders again', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const first = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(first.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 1), reserveForResponse: 0 }, undefined, BIND); + assert.deepEqual(thinkingByText(tight), { A2: false }, 'setup: only A2 is sent'); + assert.equal(tight.thinkingStrip, undefined, 'A2 was left out by the last compile'); + const back = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(back.thinkingStrip, { newly: [a1], cause: 'first' }, 'A1 again, since the last compile didn\u2019t send it'); + await manager.close(); + }); + + it('another participant\u2019s reply never keeps its thinking', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + manager.addMessage('Other', [thought(8), text('O1')]); + manager.addMessage('Claude', [thought(1), text('A1')]); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(first), { U1: false, O1: false, A1: true }); + assert.equal(first.thinkingStrip, undefined, 'another participant\u2019s thinking is never newly left out'); + await manager.close(); + }); + + it('the record survives a restart', async () => { + cleanup(); + let manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + const before = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(before).A3, true, 'setup: A3 keeps its thinking'); + await manager.close(); + + manager = await open(); + const after = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(after), thinkingByText(before), 'with no record this would leave out A3 too'); + assert.equal(after.thinkingStrip, undefined); + assert.equal(after.thinkingRecordedAfter, before.thinkingRecordedAfter); + await manager.close(); + }); + + it('a branch starts from the record as of its branch point, and its writes stay its own', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const first = await manager.compile(BIG, undefined, BIND_DROP); + const u3 = manager.addMessage('User', [text('U3')]); + manager.addMessage('Claude', [thought(3), text('A3')]); + // On main, U1 falls out of the window: A1, A2 and A3 lose their thinking. + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 4), reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(thinkingByText(tight).A2, false, 'setup: main\u2019s latest record leaves A2 out'); + const mainRecord = JSON.stringify(recordOf(manager)); + + // A branch at U3 sees the record the first compile wrote, before U3 was + // stored, and not main's latest. + const atU3 = manager.branchAt(u3, 'zz-at-u3'); + await manager.switchBranch(atU3); + const there = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(there), { U1: false, A1: true, U2: false, A2: true, U3: false }); + assert.equal(there.thinkingStrip, undefined); + assert.equal(there.thinkingRecordedAfter, first.thinkingRecordedAfter); + assert.ok(there.thinkingRecordedAfter! <= manager.currentBranch().branchPoint!, 'at most the branch point'); + await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, BIND_DROP); + + await manager.switchBranch('main'); + assert.equal(JSON.stringify(recordOf(manager)), mainRecord, 'the branch\u2019s writes left main\u2019s record'); + + const forked = await manager.fork('zz-fork'); + assert.equal(forked, 'zz-fork'); + assert.equal(JSON.stringify(recordOf(manager)), mainRecord, 'the fork holds main\u2019s record'); + const onFork = await manager.compile({ maxTokens: tokensOfNewest(manager, 4), reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(onFork.thinkingStrip, undefined, 'the fork starts from main\u2019s record'); + await manager.close(); + }); + + it('a dry compile reports what it would leave out and records nothing', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const dry = await manager.compile(BIG, undefined, bind({ dryRun: true })); + assert.deepEqual(dry.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + assert.deepEqual(thinkingByText(dry), { U1: false, A1: false, U2: false, A2: false }); + assert.equal(recordOf(manager), null); + const real = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(real.thinkingStrip, { newly: [a1, a2], cause: 'first' }, 'the dry compile left no record behind'); + const recorded = JSON.stringify(recordOf(manager)); + await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, bind({ dryRun: true })); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'a dry compile leaves a record as it was'); + await manager.close(); + }); + + it('a record it cannot read is warned of and read as none', async () => { + const good = { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }; + const malformed: Array<[string, unknown]> = [ + ['sent not a list', { ...good, sent: 'zz-not-a-list' }], + ['sent holding a number', { ...good, sent: [7] }], + ['without missing', { sent: [], continued: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }], + ['continued missing', { sent: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }], + ['compileId missing', { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity() }], + ['compileId not a string', { ...good, compileId: 7 }], + ['continued holding a number', { ...good, continued: [7] }], + ['without holding a number', { ...good, without: [7] }], + ['recordedAfter not a number', { ...good, recordedAfter: '1' }], + ['identity missing', { sent: [], continued: [], without: [], recordedAfter: 1, compileId: 'zz-compile' }], + ['identity null', { ...good, identity: null }], + ['an identity digest not a string', { ...good, identity: { ...identity(), model: 5 } }], + ['not an object', ['zz']], + ]; + for (const [label, value] of malformed) { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + manager.getStore().setStateJson('default/thinking-binding', value); + const warned: string[] = []; + const warn = console.warn; + console.warn = (message: string) => { warned.push(message); }; + let result: CompileResult; + try { + assert.equal(note(manager, [a2], identity()), false, `${label}: a note writes nothing`); + assert.deepEqual(recordOf(manager), value, `${label}: the value is as it was`); + assert.equal(warned.length, 1, label); + assert.match(warned[0]!, /thinking-binding record of branch main isn't one this version reads; this note writes nothing/, label); + warned.length = 0; + result = await manager.compile(BIG, undefined, BIND); + } finally { + console.warn = warn; + } + assert.deepEqual(result.thinkingStrip, { newly: [a1, a2], cause: 'first' }, label); + assert.equal(warned.length, 1, label); + assert.match(warned[0]!, /thinking-binding record of branch main isn't one this version reads; this compile goes on as if the branch had no record/, label); + assert.ok(Array.isArray((recordOf(manager) as { sent: unknown }).sent), `${label}: the compile wrote a record it can read`); + await manager.close(); + } + cleanup(); + const control = await open(); + seed(control); + control.getStore().setStateJson('default/thinking-binding', good); + const read = await control.compile(BIG, undefined, BIND_DROP); + assert.equal(read.thinkingRecordedAfter, 1, 'control: a record of the right shape is read'); + await control.close(); + }); + + it('a stripped body reads as incomplete, and the rendered layout prices what was sent', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const plain = await manager.compile(BIG, undefined, { dryRun: true }); + const bound = await manager.compile(BIG, undefined, BIND); + const bodies = (result: CompileResult) => { + const out = new Map(); + for (const source of result.provenance!.messages) if (source.kind === 'raw') for (const body of source.bodies) out.set(body.messageId, body); + return out; + }; + assert.equal(bodies(plain).get(a1)!.complete, true, 'setup: carried whole without binding'); + assert.equal(bodies(bound).get(a1)!.complete, false); + assert.deepEqual(bodies(bound).get(a2)!.missing, ['content']); + assert.equal([...bodies(bound).values()].filter((body) => body.complete).length, 2, 'the user messages are still whole'); + const thoughts = (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens([thought(1), thought(2)]); + assert.ok(thoughts > 0, 'setup: the thinking has a price'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - thoughts); + const partial = (result: CompileResult) => result.provenance!.layout!.units.filter((unit) => (unit as { p?: number }).p === 1).length; + assert.equal(partial(plain), 0, 'setup: every raw unit whole without binding'); + assert.equal(partial(bound), 2, 'the two replies\u2019 units are partial'); + await manager.close(); + }); + + it('the rendered layout prices a summary\u2019s and an injection\u2019s thinking as never sent', async () => { + cleanup(); + class Summarizing extends PassthroughStrategy { + select(store: MessageStoreView, log: ContextLogView, budget: TokenBudget): ContextEntry[] { + const entries = super.select(store, log, budget); + return [...entries, { index: entries.length, sourceRelation: 'derived', participant: 'Claude', content: [thought(7), text('zz-summary')], summaries: [{ id: 'zz-s1', level: 1 }] }]; + } + describeRenderedSummaries(ids: readonly string[]) { + return new Map(ids.map((id) => [id, { level: 1, leaves: [] as MessageId[], method: 'authored' }])); + } + } + const manager = await open(STORE, new Summarizing()); + seed(manager); + const odd: ContextInjection = { namespace: 'zz-odd', position: 'afterUser', content: [thought(9), text('zz-odd')] }; + const plain = await manager.compile(BIG, [odd], { dryRun: true }); + const bound = await manager.compile(BIG, [odd], BIND_DROP); + assert.deepEqual(thinkingByText(bound), { U1: false, A1: true, U2: false, 'zz-odd': false, A2: true, 'zz-summary': false }, 'setup: the replies keep their thinking'); + const price = (blocks: ContentBlock[]) => (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens(blocks); + assert.ok(plain.provenance!.layout, 'setup: the strategy reports a layout'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([thought(7), thought(9)]), + 'the kept replies are priced as sent, the summary and the injection without theirs'); + await manager.close(); + }); + + it('a record the store can\u2019t read or write is logged, and never fails a compile or a note', async () => { + for (const fails of ['write', 'read'] as const) { + cleanup(); + const told: ContentBlock[][] = []; + class Listening extends PassthroughStrategy { + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + } + const manager = await open(STORE, new Listening()); + const { a1, a2 } = seed(manager); + const store = manager.getStore() as unknown as { getStateJson(id: string): unknown; setStateJson(id: string, value: unknown): void }; + const get = store.getStateJson.bind(store); + const set = store.setStateJson.bind(store); + const failing = () => { + if (fails === 'read') store.getStateJson = (id: string) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unreadable'); return get(id); }; + else store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unwritable'); set(id, value); }; + }; + const warned: string[] = []; + const warn = console.warn; + console.warn = (message: string) => { warned.push(message); }; + try { + failing(); + // Read as no record, the compile strips as a branch's first does. + const result = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(result.thinkingStrip, { newly: [a1, a2], cause: 'first' }, fails); + assert.equal(told.length, 1, `${fails}: the strategy is told what was left out of what was sent`); + assert.equal(warned.length, 1, fails); + assert.match(warned[0]!, fails === 'read' ? /thinking-binding record of branch main couldn't be read \(.*zz-store-unreadable/ : /thinking-binding record of branch main couldn't be written \(.*zz-store-unwritable/, fails); + store.getStateJson = get; + store.setStateJson = set; + // A note can't read or write it either: it writes nothing and says so. + const compiled = await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + const before = JSON.stringify(recordOf(manager)); + failing(); + warned.length = 0; + assert.equal(manager.noteRequestSent([a3], identity(), compiled.provenance!.compileId), false, fails); + assert.equal(warned.length, 1, `${fails}: the note is logged`); + store.getStateJson = get; + store.setStateJson = set; + assert.equal(JSON.stringify(recordOf(manager)), before, `${fails}: the record is as it was`); + } finally { + console.warn = warn; + store.getStateJson = get; + store.setStateJson = set; + } + await manager.close(); + } + }); + + it('a compile whose record the store can\u2019t write leaves the record before it, which that compile\u2019s stream can\u2019t note onto', async () => { + cleanup(); + const manager = await open(); + seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const recorded = JSON.stringify(recordOf(manager)); + manager.addMessage('User', [text('U3')]); + const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; + const set = store.setStateJson.bind(store); + store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unwritable'); set(id, value); }; + const warn = console.warn; + const warned: string[] = []; + console.warn = (message: string) => { warned.push(message); }; + let c2: CompileResult; + try { + c2 = await manager.compile(BIG, undefined, BIND_DROP); + } finally { + console.warn = warn; + store.setStateJson = set; + } + assert.equal(warned.length, 1, 'the failed write is logged'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record before it stands'); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + assert.equal(manager.noteRequestSent([a3], identity(), c2.provenance!.compileId), false, 'the failed compile\u2019s stream can\u2019t note onto it'); + // Compared against the first compile's record: U3 and A3 are past its end. + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, U2: false, A2: true, U3: false, A3: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a3], cause: 'messages' }); + await manager.close(); + }); + + it('a compile that fails after selecting writes no record and tells the strategy nothing', async () => { + cleanup(); + const told: ContentBlock[][] = []; + class Failing implements ContextStrategy { + readonly name = 'zz-failing'; + readonly renderedForms = ['raw', 'summary', 'omitted'] as const; + checkReadiness(): ReadinessState { return { ready: true }; } + select(store: MessageStoreView): ContextEntry[] { + const entries: ContextEntry[] = store.getAll().map((m, index) => ({ index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content })); + return [...entries, { index: entries.length, sourceRelation: 'derived', participant: 'Claude', content: [text('zz-summary')], summaries: [{ id: 'zz-s1', level: 1 }] }]; + } + describeRenderedSummaries(): ReadonlyMap { throw new Error('zz-layout-failed'); } + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + } + const manager = await open(STORE, new Failing()); + seed(manager); + await assert.rejects(manager.compile(BIG, undefined, BIND), /zz-layout-failed/); + assert.equal(recordOf(manager), null); + assert.deepEqual(told, []); + // Nor does one without binds clear a record when it fails. + const planted = { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }; + manager.getStore().setStateJson('default/thinking-binding', planted); + await assert.rejects(manager.compile(BIG), /zz-layout-failed/); + assert.deepEqual(recordOf(manager), planted); + await manager.close(); + }); + + it('the strategy is told the stored thinking left out, at each committing compile and only then', async () => { + cleanup(); + const told: ContentBlock[][] = []; + class Listening extends PassthroughStrategy { + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { + told.push([...blocks]); + } + } + const manager = await open(STORE, new Listening()); + seed(manager); + manager.addMessage('User', [text('U3')]); + const r3 = manager.addMessage('Claude', [redacted(3)]); + const odd: ContextInjection = { namespace: 'zz-odd', position: 'afterUser', content: [thought(9), text('zz-odd')] }; + const dry = await manager.compile(BIG, [odd], bind({ dryRun: true })); + assert.equal(told.length, 0, 'a dry compile tells nothing'); + assert.equal(thinkingByText(dry)['zz-odd'], false, 'an injection never keeps thinking'); + + const first = await manager.compile(BIG, [odd], BIND); + assert.ok(first.thinkingStrip!.newly.includes(r3), 'a reply of redacted thinking alone is left out too'); + assert.deepEqual(told, [[thought(1), thought(2), redacted(3)]], 'the replies\u2019 thinking, not the injection\u2019s'); + await manager.compile(BIG, [odd], BIND); + assert.deepEqual(told[1], [thought(1), thought(2), redacted(3)], 'told again: each compile\u2019s estimate prices them again'); + await manager.close(); + }); + + it('calibration: the strategy\u2019s estimate leaves out the price of the thinking left out', async () => { + const estimateAfter = async (path: string, opts?: SelectOptions) => { + if (existsSync(path)) rmSync(path, { recursive: true, force: true }); + const strategy = new AutobiographicalStrategy({ autoTickOnNewMessage: false, headWindowTokens: 0, adaptiveResolution: true }); + const manager = await ContextManager.open({ path, strategy }); + seed(manager); + await manager.compile(BIG, undefined, opts); + const reach = strategy as unknown as { _lastCompileEstimate: number; estimateTokens(c: ContentBlock[]): number }; + const result = { estimate: reach._lastCompileEstimate, thoughts: reach.estimateTokens([thought(1), thought(2)]) }; + await manager.close(); + return result; + }; + const plain = await estimateAfter(STORE); + const bound = await estimateAfter(STORE_B, BIND); + assert.ok(plain.thoughts > 0, 'setup: the thinking has a price'); + assert.ok(plain.estimate > plain.thoughts, 'setup: the estimate covers the replies'); + assert.equal(bound.estimate, plain.estimate - plain.thoughts); + cleanup(); + }); + + it('thinkingRecordedAfter is the branch\u2019s sequence where its record began, at or above every message stored before, one the view filter hides included', async () => { + cleanup(); + const manager = await ContextManager.open({ + path: STORE, + strategy: new PassthroughStrategy(), + viewFilter: (message) => !(message.metadata as { hidden?: unknown } | undefined)?.hidden, + }); + seed(manager); + const hidden = manager.addMessage('User', [text('zz-hidden')], { hidden: true }); + // A state write after the newest message moves the branch's sequence past it. + manager.getStore().setStateJson('default/thinking-binding', null); + const sequence = manager.getStore().currentSequence(); + assert.ok(sequence > manager.getMessage(hidden)!.sequence, 'setup: the branch is past its newest message'); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(first.messages.length, 4, 'setup: the hidden message isn\u2019t sent'); + assert.ok(first.thinkingRecordedAfter! >= sequence); + assert.ok(first.thinkingRecordedAfter! <= manager.getStore().currentSequence()); + await manager.close(); + }); + + it('each namespace on a store keeps its own record', async () => { + cleanup(); + const a = await open(STORE, new PassthroughStrategy(), 'zz-a'); + seed(a); + await a.compile(BIG, undefined, BIND_DROP); + const recorded = JSON.stringify(recordOf(a, 'zz-a')); + const b = await ContextManager.open({ store: a.getStore(), strategy: new PassthroughStrategy(), namespace: 'zz-b' }); + assert.equal(recordOf(b, 'zz-b'), null, 'zz-b starts with none'); + await b.compile({ maxTokens: tokensOfNewest(a, 2), reserveForResponse: 0 }, undefined, BIND); + assert.notEqual(recordOf(b, 'zz-b'), null); + assert.equal(JSON.stringify(recordOf(a, 'zz-a')), recorded, 'zz-b\u2019s write left zz-a\u2019s record'); + await b.close(); + await a.close(); + }); +}); + +describe('thinking binding: a summary\u2019s carriers', () => { + it('a binding select renders and prices the live window as under live-strip', async () => { + // Carriers big enough that pricing them changes what fits in 900 tokens. + const carriers = (call: number): ContentBlock[] => [ + { type: 'thinking', thinking: `zz-archivist-${call} `.repeat(50), signature: `zz-asig-${call}-fedcba9876543210` } as ContentBlock, + { type: 'redacted_thinking', data: `zz-aenc-${call}-payload==` } as ContentBlock, + { type: 'text', text: `zz-adaptive-memory-${call} recollection` }, + ]; + await assert.rejects( + renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900), + /Adaptive picker exhausted/, + 'setup: rendered and priced with their carriers, these summaries don\u2019t fit', + ); + const stripped = await renderAdaptiveFixture({ carrierPolicy: 'live-strip' }, carriers, 900); + const bound = await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, BIND_DROP); + assert.ok(stripped.messages.length > 0, 'setup: the live-strip render fits'); + assert.equal(typeof stripped.estimate, 'number', 'setup: the fixture reads the estimate'); + assert.equal(typeof bound.estimate, 'number'); + assert.deepEqual(bound.messages, stripped.messages); + assert.equal(bound.estimate, stripped.estimate, 'priced as rendered'); + // A dry select prices by its own option, whatever compiled before it. + const preview = await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, { ...BIND_DROP, dryRun: true }); + assert.deepEqual(preview.messages, stripped.messages, 'a binding dry compile, with no committing compile before it'); + await assert.rejects( + renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, { dryRun: true }, BIND_DROP), + /Emission overran the plan/, + 'a dry compile without the option, after a binding compile, prices as configured: it reports an infeasible plan rather than refusing it, so it fails where its render overruns', + ); + // previewContext takes the option too. + const previewed: Record = {}; + await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, BIND_DROP, undefined, (manager) => { + const budget = { maxTokens: 900, reserveForResponse: 0 }; + previewed.bound = manager.previewContext(budget, undefined, { thinkingBinding: { binds: true } })!.fits; + previewed.unbound = manager.previewContext(budget)!.fits; + }); + assert.deepEqual(previewed, { bound: true, unbound: false }, 'priced in full, they don\u2019t fit'); + }); + + it('a recall pair is priced as its own select\u2019s binding says', async () => { + cleanup(); + const entry = { + id: 'zz-sum-bound', level: 1, content: 'zz-memory prose.', tokens: 4_000, + sourceIds: ['zz-m1'], sourceRange: { first: 'zz-m1', last: 'zz-m2' }, sourceLevel: 0, timestamp: 0, + responseContent: [ + { type: 'thinking', thinking: 'zz-archivist '.repeat(300), signature: 'zz-sig-bound-0123' }, + { type: 'text', text: 'zz-memory prose.' }, + ], + }; + const strategy = new AutobiographicalStrategy({ carrierPolicy: 'full', autoTickOnNewMessage: false }); + const stripStrategy = new AutobiographicalStrategy({ carrierPolicy: 'live-strip', autoTickOnNewMessage: false }); + const price = (s: AutobiographicalStrategy) => (s as unknown as { recallPairCost(e: unknown): number }).recallPairCost(entry); + const manager = await ContextManager.open({ path: STORE, strategy }); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG); + // The reference: the same pricing, on a store, under live-strip. + const reference = await ContextManager.open({ path: STORE_B, strategy: stripStrategy }); + reference.addMessage('User', [text('U1')]); + await reference.compile(BIG); + const full = price(strategy); + const live = price(stripStrategy); + assert.ok(full > live + 1_000, 'setup: the measured tokens, carriers included, set the full price'); + + // The price as each select sees it, from inside its select path. + // The price as each select sees it, from inside its select path, and as + // it stands once the select has returned. + const seen: number[] = []; + const after: number[] = []; + const hooked = strategy as unknown as { selectPath: (...args: unknown[]) => unknown }; + const selectPath = hooked.selectPath.bind(strategy); + hooked.selectPath = (...args: unknown[]) => { + seen.push(price(strategy)); + return selectPath(...args); + }; + for (const opts of [BIND, { dryRun: true }, { ...BIND, dryRun: true }, { thinkingBinding: { binds: false }, dryRun: true }, undefined]) { + await manager.compile(BIG, undefined, opts); + after.push(price(strategy)); + } + assert.deepEqual(seen, [live, full, live, full, full], 'each select prices by its own option'); + assert.deepEqual(after, [full, full, full, full, full], 'outside a select, as configured, after a binding one too'); + // And after a binding select that throws, committing or dry. + const refusing = { maxTokens: 1, reserveForResponse: 0 }; + manager.addMessage('User', [text('zz-over ' + 'w '.repeat(400))]); + await assert.rejects(manager.compile(refusing, undefined, BIND), 'setup: the binding compile is refused'); + assert.equal(price(strategy), full, 'after a refused binding compile'); + await assert.rejects(manager.compile(refusing, undefined, { ...BIND, dryRun: true }), 'setup: the binding dry compile is refused'); + assert.equal(price(strategy), full, 'after a refused binding dry compile'); + await manager.close(); + await reference.close(); + cleanup(); + }); +}); + +describe('thinking binding: bindThinking', () => { + const user = (t: string, participant = 'User'): NormalizedMessage => ({ participant, content: [text(t)] }); + const reply = (n: number, t: string): NormalizedMessage => ({ participant: 'Claude', content: [thought(n), text(t)] }); + const body = (id: string, sequence: number): RawBodySource => ({ messageId: id, sequence, complete: true }); + const raw = (...bodies: RawBodySource[]): CompiledMessageSources => ({ kind: 'raw', bodies }); + const recordFor = (messages: NormalizedMessage[], over: Partial = {}): BindingRecord => ({ + sent: messages.map((m) => messageDigest(m)), + continued: [], + without: [], + identity: identity(), + recordedAfter: 0, + compileId: 'zz-compile', + ...over, + }); + const input = (messages: NormalizedMessage[], sources: CompiledMessageSources[], record: BindingRecord | null) => ({ + messages, sources, record, dropBlock: true, identity: identity(), participant: 'Claude', sequence: 9, compileId: 'zz-compile-2', + }); + + it('compares content as the provider reads it: key order and cache breakpoints aside', () => { + const sent = [user('U1'), reply(1, 'A1')]; + const record = recordFor(sent); + const reordered: NormalizedMessage[] = [ + { participant: 'User', content: [{ text: 'U1', type: 'text' } as ContentBlock], cacheBreakpoint: true }, + reply(1, 'A1'), + ]; + const outcome = bindThinking(input(reordered, [raw(body('u1', 1)), raw(body('a1', 2))], record)); + assert.equal(outcome.strip, undefined); + assert.ok(outcome.messages[1]!.content.some(isThinking)); + assert.equal(outcome.record.recordedAfter, 0, 'kept from the record'); + assert.deepEqual(outcome.record.continued, [], 'a compile starts the request over'); + }); + + it('a message whose participant alone changed is a difference', () => { + const sent = [user('U1'), reply(1, 'A1')]; + const now = [user('U1', 'Someone'), reply(1, 'A1')]; + const outcome = bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 2))], recordFor(sent))); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'messages' }); + }); + + it('after the first difference, a reply left out before is not newly left out', () => { + const sent = [user('U1'), reply(1, 'A1'), user('U2'), { participant: 'Claude', content: [text('A2')] }]; + const record = recordFor(sent, { without: ['a2'] }); + const now = [user('U1-edited'), reply(1, 'A1'), user('U2'), reply(2, 'A2')]; + const outcome = bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 2)), raw(body('u2', 3)), raw(body('a2', 4))], record)); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'messages' }); + assert.deepEqual(outcome.record.without, ['a1', 'a2']); + assert.ok(outcome.messages.every((m) => !m.content.some(isThinking))); + }); + + it('a message of several bodies goes without when any of them did, and records them all', () => { + const merged: NormalizedMessage = { participant: 'Claude', content: [thought(1), text('A1'), text('A1b')] }; + const sent = [user('U1'), { participant: 'Claude', content: [text('A1'), text('A1b')] }]; + const record = recordFor(sent, { without: ['a1b'] }); + const outcome = bindThinking(input([user('U1'), merged], [raw(body('u1', 1)), raw(body('a1', 2), body('a1b', 3))], record)); + assert.equal(outcome.strip, undefined, 'left out before, so not newly'); + assert.ok(!outcome.messages[1]!.content.some(isThinking)); + assert.deepEqual(outcome.record.without, ['a1', 'a1b']); + }); + + it('the participant\u2019s message with no stored body never keeps its thinking', () => { + const sources: CompiledMessageSources[] = [ + raw(body('u1', 1)), + { kind: 'summary', summaries: [{ id: 'zz-s1', level: 1 }] }, + { kind: 'other' }, + raw(body('a1', 2)), + ]; + const now = [user('U1'), reply(8, 'S1'), reply(9, 'O1'), reply(1, 'A1')]; + const outcome = bindThinking(input(now, sources, null)); + assert.deepEqual(outcome.messages.map((m) => m.content.some(isThinking)), [false, false, false, true]); + assert.equal(outcome.strip, undefined, 'no stored body, so nothing is newly left out'); + assert.deepEqual(outcome.record.without, [], 'and nothing recorded as left out'); + assert.deepEqual(outcome.leftOut, [], 'nor handed to the strategy: only stored bodies\u2019 thinking is'); + }); + + it('past the record\u2019s end nothing keeps its thinking; a noted row extends the record, and an unvouched one matches nothing', () => { + const sent = [user('U1')]; + const now = [user('U1'), reply(1, 'A1'), reply(2, 'A2')]; + const keeps = (record: Partial) => + bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 5)), raw(body('a2', 6))], recordFor(sent, record))).messages.map((m) => m.content.some(isThinking)); + assert.deepEqual(keeps({}), [false, false, false]); + assert.deepEqual(keeps({ continued: [messageDigest(reply(1, 'A1'))] }), [false, true, false]); + assert.deepEqual(keeps({ continued: [messageDigest(reply(1, 'A1')), messageDigest(reply(2, 'A2'))] }), [false, true, true]); + assert.deepEqual(keeps({ continued: [UNVOUCHED, messageDigest(reply(2, 'A2'))] }), [false, false, false]); + assert.deepEqual(keeps({ sent: [UNVOUCHED], continued: [messageDigest(reply(1, 'A1'))] }), [false, false, false], 'a record that vouches for nothing'); + }); + + it('a head change is the cause given, when replies from before the record go too', () => { + const sent = [user('U1')]; + const now = [user('U1'), reply(1, 'A1')]; + const outcome = bindThinking({ + ...input(now, [raw(body('u1', 1)), raw(body('a1', 2))], recordFor(sent, { recordedAfter: 5, identity: identity({ tools: 'zz-old' }) })), + dropBlock: false, + }); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'head', changed: ['tools'] }); + }); + + it('a body marked as left out gains \u2018content\u2019 once, and keeps what it was missing', () => { + const sources: CompiledMessageSources[] = [ + raw({ messageId: 'a1', sequence: 1, complete: false, missing: ['shards'] }), + raw(body('a2', 2)), + raw(body('u1', 3)), + ]; + markThinkingLeftOut(sources, new Set(['a1', 'a2'])); + markThinkingLeftOut(sources, new Set(['a2'])); + const bodiesOf = (i: number) => (sources[i] as { bodies: RawBodySource[] }).bodies[0]!; + assert.deepEqual(bodiesOf(0).missing, ['shards', 'content']); + assert.deepEqual(bodiesOf(1).missing, ['content']); + assert.equal(bodiesOf(1).complete, false); + assert.equal(bodiesOf(2).complete, true, 'a body not named is left as it was'); + }); + + it('a digest is 11 base64url characters, tells an array from an object, and keeps array order', () => { + const digest = messageDigest(user('U1')); + assert.match(digest, /^[A-Za-z0-9_-]{11}$/); + const withInput = (input: unknown): NormalizedMessage => ({ participant: 'Claude', content: [{ type: 'tool_use', id: 't', name: 'n', input } as ContentBlock] }); + assert.notEqual(messageDigest(withInput({ value: ['x'] })), messageDigest(withInput({ value: { 0: 'x' } }))); + assert.notEqual(messageDigest(withInput({ value: ['x', 'y'] })), messageDigest(withInput({ value: ['y', 'x'] }))); + assert.equal(messageDigest(withInput({ a: 1, b: 2 })), messageDigest(withInput({ b: 2, a: 1 })), 'control: key order aside'); + const marked = (key: string): NormalizedMessage => ({ participant: 'User', content: [{ type: 'text', text: 'U1', [key]: { type: 'ephemeral' } } as unknown as ContentBlock] }); + assert.equal(messageDigest(marked('cache_control')), digest, 'a block\u2019s cache marker aside'); + assert.equal(messageDigest(marked('cacheControl')), digest); + }); +}); + +// The public types, through both barrels: a removed export fails the build. +const exported: { binding: ThinkingBinding; identity: RequestIdentity | RequestIdentityFromTypes; strip: ThinkingStrip | ThinkingStripFromTypes } = { + binding: { binds: true, participant: 'Claude' }, + identity: identity(), + strip: { newly: [], cause: 'messages' }, +}; +void exported; diff --git a/test/tool-prose-hoist.test.ts b/test/tool-prose-hoist.test.ts index 558c431f..a0d8d6e4 100644 --- a/test/tool-prose-hoist.test.ts +++ b/test/tool-prose-hoist.test.ts @@ -125,8 +125,9 @@ function scripted(responses: Array>) { return { calls, membrane: { complete: async (request: NormalizedRequest) => { calls.push(structuredClone(request)); return responses[Math.min(n++, responses.length - 1)]; } } as never }; } -// The fixture's large head window replays the chunk's messages in the head as -// well as in the target slot, so canonical requests carry each call twice. +// The fixture's large head window holds every message, but a request's head is +// cut at its chunk's start (headAsOf, #317), so canonical requests carry each of +// the chunk's calls once, in the target slot. const names = (req: NormalizedRequest): string => [...new Set(toolUses(req).map((u) => u.name))].join('+'); describe('tool-prose hoist rung', () => { diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts new file mode 100644 index 00000000..1c92b5c2 --- /dev/null +++ b/test/whole-body-pins.test.ts @@ -0,0 +1,760 @@ +/** + * A pin covers whole messages: under adaptive resolution, a pin or document + * mark whose first or last id names a part of a message stored in parts + * covers that message from its first part to its last, for every reader of + * pins, the selects, compression and the summarizer's prompt alike, as a + * pinRange over all its parts does where no part is locked (`pinPositions`): + * a part locked with `lockChunk` outside the span the pin's ids name keeps + * its lock. A compile that can't hold a whole message refuses, as for any + * pin the budget can't hold, and the adaptive plan's refusal names the pin. + * + * The store: six history messages, then long messages that ingress stores + * in parts (adaptive resolution with 1,500-token chunks), then 360 messages + * of traffic. A head window of 500 tokens and a recent window of 8,000 leave + * the long messages in the middle. With one 6,000-word message marked, its + * first part alone fits at 22,000 tokens and the whole message needs about + * 38,000; the tests use 30,000 for "can't be whole" and 42,000 or 60,000 for + * "fits". + */ + +import { describe, it } from 'node:test'; +import assert from 'node:assert'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, OverBudgetError } from '../src/index.js'; +import type { AutobiographicalConfig } from '../src/index.js'; + +const words = (label: string, n: number): ContentBlock[] => + [{ type: 'text', text: `${label} ${Array.from({ length: n }, (_, i) => `word${i}`).join(' ')}` }]; + +const KV_UNIFIED = { + policy: { + alpha: 0.7, budgetLowRatio: 0.5, budgetHighRatio: 0.9, budgetUnderLambda: 10, budgetOverLambda: 10, + cacheLambda: 1, cacheScale: 1000, cacheReadPrice: 0.1, cacheWritePrice: 1.25, continuityLambda: 1, + continuityScale: 1000, continuityRecencyHalfLifeTokens: 1000, continuityRecencyFloor: 0.2, + continuityStableHalfLife: 10, continuityStableFloor: 0.25, + }, + tokenBucketSize: 100, continuityBucketSize: 100, fidelityBucketSize: 100, labelCeiling: 10_000, adoptEpsilon: 0, + treeifyNonContiguousSummaries: false, preserveGapBearingSummaries: false, +}; + +/** Reaches the strategy's pins and windows. */ +class PinStore extends AutobiographicalStrategy { + /** The fold bound each pinned position resolves to, as every reader of pins takes it. */ + boundsAt(messages: unknown[]): Map { + return this.resolvedPinBounds(messages as never); + } + /** The level bounds pins give, as the replay exporter reads them. */ + levelBounds(messages: unknown[]): Map { + return this.pinLevelBounds(messages as never); + } + /** The ids of the messages in compressed chunks. */ + summarizedIds(): Set { + const out = new Set(); + for (const chunk of this.chunks) if (chunk.compressed) for (const m of chunk.messages) out.add(m.id); + return out; + } + /** The ids of the messages in any chunk. */ + chunkedIds(): Set { + const out = new Set(); + for (const chunk of this.chunks) for (const m of chunk.messages) out.add(m.id); + return out; + } + /** Where the head window ends. */ + headEnd(store: never): number { + return this.getHeadWindowEnd(store); + } + /** The stored resolution of a message (0 is raw). */ + resolutionOf(id: string): number { + return this.resolutions.get(id) ?? 0; + } +} + +type Kind = 'adaptive' | 'kv-stable' | 'kv-unified'; +const config = (kind: Kind, recentWindowTokens: number): AutobiographicalConfig => ({ + headWindowTokens: 500, recentWindowTokens, imageStripDepthTokens: 0, compressionModel: 'test-model', + hierarchical: false, adaptiveResolution: true, targetChunkTokens: 1500, + ...(kind === 'kv-stable' ? { foldingStrategy: 'kv-stable' } : {}), + ...(kind === 'kv-unified' ? { foldingStrategy: 'kv-unified', kvUnified: { ...KV_UNIFIED, policy: { ...KV_UNIFIED.policy } } } : {}), +} as AutobiographicalConfig); + +function summarizer(words = 30): { membrane: never; calls: () => number; requests: string[] } { + let n = 0; + const requests: string[] = []; + const membrane = { + complete: async (request: unknown) => (requests.push(JSON.stringify(request)), { + stopReason: 'end_turn', content: [{ type: 'text', text: `[S${++n}] ` + 'recalled '.repeat(words) }], + usage: { input_tokens: 100, output_tokens: 40 }, + }), + } as never; + return { membrane, calls: () => n, requests }; +} + +async function open(dir: string, kind: Kind = 'adaptive', recentWindowTokens = 8000, extra: Partial = {}) { + const strategy = new PinStore({ ...config(kind, recentWindowTokens), ...extra } as AutobiographicalConfig); + const sum = summarizer(); + const cm = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: sum.membrane }); + return { cm, strategy, calls: sum.calls, requests: sum.requests }; +} + +const budget = (maxTokens: number) => ({ maxTokens, reserveForResponse: 0 }); +type Sent = { participant: string; content: ContentBlock[] }; +const sent = (r: { messages: unknown[] }): Sent[] => r.messages as Sent[]; +const textOf = (m: Sent): string => m.content.map((b) => (b.type === 'text' ? b.text : '')).join(''); +/** The raw words of a marked message the request carries, by their numbers. */ +function rawWords(messages: Sent[], label: string): number[] { + return messages.map(textOf) + .filter((t) => t.startsWith(label) || /^\s*word\d+/.test(t)) + .flatMap((t) => [...t.matchAll(/\bword(\d+)\b/g)].map((m) => Number(m[1]))) + .sort((a, b) => a - b); +} +const allOf = (n: number): number[] => Array.from({ length: n }, (_, i) => i); +const ready = async (cm: ContextManager) => { + for (let i = 0; i < 800 && !cm.isReady(); i++) await cm.tick(); + assert.ok(cm.isReady(), 'compression caught up'); +}; + +async function history(cm: ContextManager): Promise { + for (let i = 0; i < 6; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`hist${i}`, 200)); + await cm.compile(budget(42_000)); +} +async function traffic(cm: ContextManager, n: number, label = 'event'): Promise { + for (let i = 0; i < n; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`${label}${i}`, 200)); +} + +function warnings(t: { mock: { method: (o: object, m: string, f: (...a: unknown[]) => void) => unknown } }): string[] { + const out: string[] = []; + t.mock.method(console, 'warn', (...args: unknown[]) => { out.push(String(args[0])); }); + return out; +} + +type View = { getAll(): Array<{ id: string; bodyGroupId?: string }>; estimateTokens(message: unknown): number }; +const viewOf = (cm: ContextManager): View => (cm as unknown as { strategyMessageView(): View }).strategyMessageView(); +/** The ids of the parts of the message `id` is in, in store order. */ +const partsOf = (cm: ContextManager, id: string): string[] => { + const all = viewOf(cm).getAll(); + const group = all.find((m) => m.id === id)!.bodyGroupId; + return all.filter((m) => m.bodyGroupId !== undefined && m.bodyGroupId === group).map((m) => m.id); +}; +/** The word numbers of one stored part's own text. */ +const wordsOfPart = (cm: ContextManager, id: string): number[] => { + const part = (viewOf(cm).getAll() as unknown as Array<{ id: string; content: ContentBlock[] }>).find((m) => m.id === id)!; + return part.content.flatMap((b) => (b.type === 'text' ? [...b.text.matchAll(/\bword(\d+)\b/g)].map((m) => Number(m[1])) : [])); +}; +async function withDir(body: (dir: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), 'cm-whole-pins-')); + try { + await body(dir); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} + +/** A store where a long message arrives, is chunked and summarized, and then is marked, as a store an earlier version marked reads now. */ +async function markedAfterSummaries(dir: string, kind: Kind = 'adaptive') { + const s = await open(dir, kind); + await history(s.cm); + const b = s.cm.addMessage('user', words('Bdoc', 6000)); + await traffic(s.cm, 360); + await ready(s.cm); + await s.cm.compile(budget(30_000)); + const parts = partsOf(s.cm, b); + assert.ok(parts.slice(1).every((id) => s.strategy.summarizedIds().has(id)), 'the later parts are summarized'); + return { ...s, b, parts }; +} + +/** The compile's refusal, or a failure when it compiles. */ +async function refusalOf(cm: ContextManager, maxTokens: number): Promise { + try { + await cm.compile(budget(maxTokens)); + } catch (err) { + if (err instanceof OverBudgetError) return err; + throw err; + } + assert.fail(`the compile at ${maxTokens} went ahead`); +} + +describe('a pin covers whole messages, for every reader', () => { + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: a document mark on a long message's first part is read as a range pin over all its parts: the same request, chunks and summarizer input`, async () => { + const run = async (how: 'mark' | 'range') => { + let out: { request: string[]; raw: number[]; chunked: boolean[]; prompts: Array<[boolean, boolean]> } | undefined; + await withDir(async (dir) => { + // A recent window of 20,000 holds the whole message as it arrives, so the pin is there before + // compression chunks any part of it. + const { cm, strategy, requests } = await open(dir, kind, 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + assert.ok(parts.length > 2, 'the message is stored in parts'); + assert.deepStrictEqual(parts.filter((id) => strategy.chunkedIds().has(id)), [], 'no part is chunked yet'); + if (how === 'mark') cm.markDocument(b); + else cm.pinRange(parts[0]!, parts.at(-1)!); + await traffic(cm, 360); + await ready(cm); + const request = sent(await cm.compile(budget(60_000))); + // The summarizer's requests for chunks long after the message: does each carry its first and last words raw? + const later = requests.filter((r) => r.includes('"event300 word0')); + assert.ok(later.length > 0, 'the chunk was summarized'); + out = { + request: request.map(textOf), raw: rawWords(request, 'Bdoc'), + chunked: parts.map((id) => strategy.chunkedIds().has(id)), + prompts: later.map((r) => [r.includes('Bdoc word0 word1'), r.includes('word5999')]), + }; + await cm.close(); + }); + return out!; + }; + const mark = await run('mark'); + const range = await run('range'); + assert.deepStrictEqual(mark.raw, allOf(6000), 'the whole message goes out raw'); + assert.deepStrictEqual(mark.chunked, mark.chunked.map(() => false), 'compression leaves every part out'); + assert.ok(mark.prompts.every(([first, last]) => first && last), 'the summarizer\'s prompt carries the whole message raw'); + assert.deepStrictEqual(mark, range); + }); + } + + it('a mark made after its message\'s early parts were chunked as it arrived sends the whole message raw', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + assert.ok(strategy.chunkedIds().has(parts[1]!), 'the second part was chunked before the mark'); + cm.markDocument(b); + await traffic(cm, 360); + await ready(cm); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a mark on a long message whose other parts were summarized before it, as an earlier version left one, sends the whole message where it fits', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir); + assert.ok(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc').length < 6000, 'before the mark, the later parts fold'); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a range pin given its ends in reverse order covers the same whole message', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const parts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); + cm.pinRange(parts[3]!, parts[1]!); + const all = viewOf(cm).getAll(); + const bounds = strategy.boundsAt(all); + const of = new Set(parts); + assert.deepStrictEqual(all.flatMap((m, i) => (bounds.has(i) ? [m.id] : [])), all.filter((m) => of.has(m.id)).map((m) => m.id)); + await cm.close(); + }); + }); + + it('a range pin over a part locked at a fold between its ids renders it raw over the lock, as before', async () => { + await withDir(async (dir) => { + const { cm, strategy, parts } = await markedAfterSummaries(dir); + strategy.lockChunk(parts[3]!); + assert.ok(strategy.resolutionOf(parts[3]!) > 0, 'locked at a fold'); + cm.pinRange(parts[1]!, parts[5]!); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a range pin whose ends fall inside long messages covers both of them whole', async () => { + await withDir(async (dir) => { + const { cm } = await open(dir); + await history(cm); + const bParts = partsOf(cm, cm.addMessage('user', words('Bdoc', 2500))); + const cParts = partsOf(cm, cm.addMessage('user', words('Cdoc', 2500))); + assert.ok(bParts.length > 2 && cParts.length > 2, 'both messages are stored in parts'); + // From B's second part to C's second part. + cm.pinRange(bParts[1]!, cParts[1]!); + await traffic(cm, 360); + await ready(cm); + const request = sent(await cm.compile(budget(42_000))); + assert.deepStrictEqual(rawWords(request, 'Bdoc'), allOf(2500)); + assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(2500)); + await cm.close(); + }); + }); + + it('a mark\'s level bound holds every part of its message', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + cm.markDocument(b, { maxLevel: 1 }); + const all = viewOf(cm).getAll(); + const parts = new Set(partsOf(cm, b)); + const at = all.flatMap((m, i) => (parts.has(m.id) ? [i] : [])); + assert.ok(at.length > 2, 'the message is stored in parts'); + const bounds = strategy.boundsAt(all); + assert.deepStrictEqual(at.map((i) => bounds.get(i)), at.map(() => ({ maxLevel: 1 }))); + await cm.close(); + }); + }); + + it('kv-unified: a mark\'s own part that another pin\'s whole reading holds at a level is summarized, so the compile is feasible', async () => { + await withDir(async (dir) => { + // A recent window of 20,000 holds the message as it arrives, so both pins are there before any part is chunked. + const { cm, strategy } = await open(dir, 'kv-unified', 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + // A raw mark on the first part; a range at level 1 from the second part to the third, which read whole covers + // the first part too, where a bound takes a raw pin's place. + cm.markDocument(b); + cm.pinRange(parts[1]!, parts[2]!, { level: 1 }); + await traffic(cm, 360); + await ready(cm); + assert.ok(strategy.summarizedIds().has(parts[0]!), 'the first part is summarized'); + await cm.compile(budget(60_000)); + await cm.close(); + }); + }); + + it('the summarizer\'s over-budget warning counts a mark\'s whole message and names the mark', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // A head window of 3,000 holds the message's first part (about 2,070 tokens), where the mark's own id is. + // Its later parts are raw history ahead of the traffic's chunks, more than the summarizer's recall budget of 2,000. + const { cm, strategy, calls } = await open(dir, 'adaptive', 8000, { headWindowTokens: 3000, compressionRecallBudgetTokens: 2000 }); + cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { name: 'P' }); + await traffic(cm, 120); + await ready(cm); + assert.strictEqual(strategy.headEnd(viewOf(cm) as never), 1, 'the head holds the first part alone'); + assert.ok(calls() > 0, 'the summarizer was called'); + const over = out.filter((w) => w.includes('recalls the summaries its pins override')); + assert.ok(over.length > 0, 'the warning was given'); + for (const w of over) { + assert.ok(w.includes('pins: P)'), w); + assert.ok(w.includes('Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the rest of that message too, less a part locked with lockChunk outside the pin\'s ids)'), w); + } + await cm.close(); + }); + }); + + it('a store opened without adaptive resolution reads a mark by its ids, as before', async () => { + await withDir(async (dir) => { + const first = await open(dir); + await history(first.cm); + const b = first.cm.addMessage('user', words('Bdoc', 6000)); + await traffic(first.cm, 360); + await ready(first.cm); + first.cm.markDocument(b); + const summarized = first.strategy.summarizedIds(); + assert.ok(partsOf(first.cm, b).slice(1).every((id) => summarized.has(id)), 'the later parts are summarized'); + await first.cm.close(); + const { cm } = await open(dir, 'adaptive', 8000, { adaptiveResolution: false }); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(763)); + await cm.close(); + }); + }); +}); + +describe('a lock holds against a pin\'s whole reading', () => { + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: parts locked at a fold after the part a mark names keep their lock`, async () => { + await withDir(async (dir) => { + const { cm, strategy, b, parts } = await markedAfterSummaries(dir, kind); + const later = parts.slice(1); + for (const id of later) strategy.lockChunk(id); + assert.ok(later.every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), wordsOfPart(cm, b), 'only the part the mark names'); + await cm.close(); + }); + }); + + it(`${kind}: parts locked at a fold before the part a mark names keep their lock`, async () => { + await withDir(async (dir) => { + const { cm, strategy, parts } = await markedAfterSummaries(dir, kind); + const earlier = parts.slice(0, -1); + for (const id of earlier) strategy.lockChunk(id); + assert.ok(earlier.every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); + cm.markDocument(parts.at(-1)!); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), wordsOfPart(cm, parts.at(-1)!), 'only the part the mark names'); + await cm.close(); + }); + }); + } + + it('a level pin\'s bounds leave out a part locked with lockChunk that its ids don\'t name, as the replay exporter reads them', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + strategy.lockChunk(parts[3]!); + cm.pinAtLevel(b, b, 1); + const messages = viewOf(cm).getAll(); + const at = (id: string) => messages.findIndex((m) => m.id === id); + const bounds = strategy.levelBounds(messages); + assert.deepStrictEqual(parts.filter((id) => bounds.has(at(id))), parts.filter((_, i) => i !== 3)); + assert.deepStrictEqual(bounds.get(at(parts[0]!)), { level: 1 }); + await cm.close(); + }); + }); + + it('a mark over its own part locked at a fold renders it raw over the lock, as before', async () => { + await withDir(async (dir) => { + const { cm, strategy, b } = await markedAfterSummaries(dir); + assert.ok(strategy.resolutionOf(b) > 0, 'the first part folded'); + strategy.lockChunk(b); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); +}); + +describe('a compile that can\'t hold a whole message refuses, naming the pins that keep raw tokens in the middle', () => { + for (const kind of ['adaptive', 'kv-stable', 'kv-unified'] as const) { + it(`${kind}: names the mark, its whole message and its tokens; unpinned, the compile goes ahead`, async () => { + await withDir(async (dir) => { + const { cm, b, parts } = await markedAfterSummaries(dir, kind); + const mark = cm.markDocument(b, { name: 'the spec' }); + const refusal = await refusalOf(cm, 30_000); + const raw = refusal.diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map(({ tokens, ...pin }) => pin), [{ + pinId: mark, kind: 'document', name: 'the spec', firstMessageId: b, lastMessageId: b, parts: parts.length, + }]); + assert.ok(raw.tokens > 10_000 && raw.pins[0]!.tokens === raw.tokens, `the whole message's tokens: ${raw.tokens}`); + assert.ok(raw.tokens <= refusal.diagnostics.middleTokens, `no more than the middle holds: ${raw.tokens} of ${refusal.diagnostics.middleTokens}`); + assert.match(refusal.message, /exceed hard budget/, 'the phrase agent-framework matches stays'); + assert.ok(refusal.message.endsWith( + `). A pin keeps ~${raw.tokens} tokens raw in the middle: document mark "the spec" (${mark}), message ${b}, ` + + `covering ${parts.length} stored parts of its message. Whoever manages pins can unpin it, freeing up to those tokens; ` + + `whoever sets the hard budget can raise it if the model's window allows.`), refusal.message); + assert.ok(cm.unpin(mark)); + await cm.compile(budget(30_000)); + await cm.close(); + }); + }); + } + + it('counts the parts a mark covers, less a part locked with lockChunk that it doesn\'t name', async () => { + await withDir(async (dir) => { + const { cm, strategy, b, parts } = await markedAfterSummaries(dir); + strategy.lockChunk(parts[3]!); + const mark = cm.markDocument(b); + const refusal = await refusalOf(cm, 30_000); + assert.deepStrictEqual(refusal.diagnostics.rawPins!.pins.map((p) => [p.pinId, p.parts]), [[mark, parts.length - 1]]); + assert.match(refusal.message, new RegExp(`covering ${parts.length - 1} stored parts of its message`)); + await cm.close(); + }); + }); + + it('adaptive: prices a pinned message the cap cuts as the select sends it, cut', async () => { + await withDir(async (dir) => { + // A message of about 1,000 tokens, under the size at which ingress stores a message in parts, cut at 300. + const { cm } = await open(dir, 'adaptive', 8000, { maxMessageTokens: 300 }); + await history(cm); + const m = cm.addMessage('user', words('Mlong', 280)); + await traffic(cm, 360); + await ready(cm); + const pin = cm.pinRange(m, m); + let raw: NonNullable | undefined; + for (const maxTokens of [12_000, 9_000, 7_000, 5_000, 4_000, 3_000, 2_000, 1_500, 1_000, 700]) { + try { + await cm.compile(budget(maxTokens)); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + raw = err.diagnostics.rawPins; + if (raw) break; + } + } + assert.ok(raw, 'a refusal named the pin'); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [pin]); + assert.ok(raw.pins[0]!.tokens > 0 && raw.pins[0]!.tokens < 450, `priced cut: ${raw.pins[0]!.tokens}`); + await cm.close(); + }); + }); + + it('prices a marked message\'s parts uncut, as the plan sends a run of them joined', async () => { + await withDir(async (dir) => { + // Parts of about 2,000 tokens, each over a cap of 300 that the plan doesn't apply to them. + const { cm } = await open(dir, 'adaptive', 8000, { maxMessageTokens: 300 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const mark = cm.markDocument(b); + let raw: NonNullable | undefined; + for (const maxTokens of [30_000, 25_000, 20_000, 15_000]) { + try { + await cm.compile(budget(maxTokens)); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + raw = err.diagnostics.rawPins; + if (raw) break; + } + } + assert.ok(raw, 'a refusal named the mark'); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + assert.ok(raw.tokens > 10_000, `priced uncut: ${raw.tokens}`); + await cm.close(); + }); + }); + + it('names every pin keeping raw tokens in the middle, counting a message two pins cover once, and none in the recent window', async () => { + await withDir(async (dir) => { + const { cm, b, parts } = await markedAfterSummaries(dir); + const all = viewOf(cm).getAll(); + const at = all.findIndex((m) => m.id === parts.at(-1)); + const mark = cm.markDocument(b); + // A range over B's third and fourth parts, which read whole covers all of B, as the mark does. + const inside = cm.pinRange(parts[2]!, parts[3]!); + // A range over two traffic messages in the middle, and a pin on the newest message, in the recent window. + const traffic2 = cm.pinRange(all[at + 10]!.id, all[at + 11]!.id, { name: 'two events' }); + cm.pinRange(all.at(-1)!.id, all.at(-1)!.id); + const refusal = await refusalOf(cm, 30_000); + const raw = refusal.diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => [p.pinId, p.parts]), [[mark, parts.length], [inside, parts.length], [traffic2, undefined]]); + const [m, i, t] = raw.pins; + assert.strictEqual(m!.tokens, i!.tokens, 'both cover the whole message'); + assert.strictEqual(raw.tokens, m!.tokens + t!.tokens, 'the message both cover counts once'); + assert.ok(refusal.message.endsWith('Whoever manages pins can unpin them, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), refusal.message); + await cm.close(); + }); + }); + + it('a pin in the head window isn\'t named, since the head is sent raw either way', async () => { + await withDir(async (dir) => { + // A head window of 3,000 holds the first history messages. + const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 3000 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const all = viewOf(cm).getAll(); + assert.ok(strategy.headEnd(viewOf(cm) as never) > 0, 'the head window holds the first message'); + cm.pinRange(all[0]!.id, all[0]!.id); + const mark = cm.markDocument(b); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + await cm.close(); + }); + }); + + it('after a head reset, names a pin in the history before the new head, which the select folds as middle', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await markedAfterSummaries(dir); + const all = viewOf(cm).getAll(); + // The head now starts at the newest message, so the recent window reaches back before it: the history before + // the head, recent messages included, is the select's middle. + strategy.resetHeadWindow(all.at(-1)!.id); + const pin = cm.pinRange(all.at(-3)!.id, all.at(-3)!.id); + const raw = (await refusalOf(cm, 3_000)).diagnostics.rawPins; + assert.deepStrictEqual(raw?.pins.map((p) => p.pinId), [pin]); + await cm.close(); + }); + }); + + it('with the head window disabled, a head reset leaves the recent messages before the head in the tail, so a pin there isn\'t named', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 0 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const mark = cm.markDocument(b); + const all = viewOf(cm).getAll(); + strategy.resetHeadWindow(all.at(-1)!.id); + cm.pinRange(all.at(-3)!.id, all.at(-3)!.id); + const refusal = await refusalOf(cm, 30_000); + assert.match(refusal.message, /^Adaptive picker exhausted/); + assert.deepStrictEqual(refusal.diagnostics.rawPins?.pins.map((p) => p.pinId), [mark]); + await cm.close(); + }); + }); + + it('kv-stable: names a raw pin but not a pin with a level or a cap that overlaps it, which keeps nothing raw', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir, 'kv-stable'); + const mark = cm.markDocument(b, { level: 0 }); + cm.pinAtLevel(b, b, 2); + cm.pinRange(b, b, { maxLevel: 2 }); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + await cm.close(); + }); + }); + + it('kv-unified: doesn\'t count toward a raw range the message a level pin holds inside it', async () => { + await withDir(async (dir) => { + const { cm } = await open(dir, 'kv-unified'); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 3000)); + const c = cm.addMessage('user', words('Cdoc', 3000)); + await traffic(cm, 360); + await ready(cm); + // The range keeps B and C raw; a level, which takes a classic pin's place, holds C at level 1. + const range = cm.pinRange(b, c); + cm.pinAtLevel(c, c, 1); + let raw: NonNullable | undefined; + const seen: string[] = []; + for (const maxTokens of [30_000, 25_000, 20_000, 15_000, 12_000]) { + try { + await cm.compile(budget(maxTokens)); + seen.push(`${maxTokens}: compiled`); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + seen.push(`${maxTokens}: ${err.message.slice(0, 120)}`); + raw = err.diagnostics.rawPins; + if (raw) break; + } + } + assert.ok(raw, `a refusal named the range: ${seen.join(' | ')}`); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [range]); + assert.strictEqual(raw.pins[0]!.tokens, raw.tokens, 'the range keeps raw only what the total counts'); + // B and C are the same size, so B alone is about half of both. + const ids = new Set([...partsOf(cm, b), ...partsOf(cm, c)]); + const both = viewOf(cm).getAll().filter((m) => ids.has(m.id)).reduce((n, m) => n + viewOf(cm).estimateTokens(m), 0); + assert.ok(raw.tokens < both * 0.6, `${raw.tokens} of ${both}: C, held at a level, isn't counted`); + await cm.close(); + }); + }); + + it('under a solver that reads no bounds, names a mark with a level, which renders raw there', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir); + const mark = cm.markDocument(b, { level: 2 }); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + await cm.close(); + }); + }); + + it('counts a part locked raw inside a pin\'s span, as every raw position in the middle', async () => { + const named = async (lock: boolean) => { + let tokens = 0; + await withDir(async (dir) => { + const { cm, strategy } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const parts = partsOf(cm, b); + // No compile has folded it yet, so the lock holds it raw. + if (lock) strategy.lockChunk(parts[4]!); + assert.strictEqual(strategy.resolutionOf(parts[4]!), 0); + // A range over every part, so the locked one is inside the span its ids name and the pin covers it. + cm.pinRange(parts[0]!, parts.at(-1)!); + tokens = (await refusalOf(cm, 30_000)).diagnostics.rawPins!.tokens; + await cm.close(); + }); + return tokens; + }; + assert.strictEqual(await named(true), await named(false)); + }); + + it('a mark held at a level keeps nothing raw, so a refusal in the middle names no pin, as before', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir, 'kv-stable'); + cm.markDocument(b, { level: 1 }); + const refusal = await refusalOf(cm, 10_000); + assert.match(refusal.message, /^Adaptive picker exhausted/, 'the plan\'s refusal, the one that names pins'); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); + assert.ok(refusal.message.endsWith(')'), refusal.message); + await cm.close(); + }); + }); + + it('a refusal of the head and recent windows alone names no pin, since unpinning wouldn\'t clear it', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir); + cm.markDocument(b); + const refusal = await refusalOf(cm, 5_000); + assert.match(refusal.message, /^Head\/tail reservation exceeded budget/); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); + await cm.close(); + }); + }); + + it('a hierarchical refusal names no pin, as before: only the adaptive plan\'s refusal does', async () => { + await withDir(async (dir) => { + const strategy = new AutobiographicalStrategy({ + compressionModel: 'test-model', adaptiveResolution: false, targetChunkTokens: 40, headWindowTokens: 0, + recentWindowTokens: 0, overBudgetGraceRatio: 0.25, autoTickOnNewMessage: false, speculativeProduction: false, + } as AutobiographicalConfig); + const cm = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: summarizer().membrane }); + const ids: string[] = []; + for (let i = 0; i < 12; i++) ids.push(cm.addMessage('user', [{ type: 'text', text: `message ${i} ` + 'word '.repeat(30) }])); + cm.pinRange(ids[2]!, ids[3]!); + const refusal = await refusalOf(cm, 100); + assert.match(refusal.message, /^Hierarchical emission overran the budget/); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); + assert.ok(refusal.message.endsWith(')'), refusal.message); + cm.close(); + }); + }); + + it('counts no message of system changes toward the pins\' tokens: it stays raw whatever the pins', async () => { + await withDir(async (dir) => { + const { cm } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const rules = (words('rule', 1500)[0] as { text: string }).text; + const changeId = cm.addMessage('Harness', [{ type: 'system_change', addSystem: [rules] } as unknown as ContentBlock]); + await traffic(cm, 360); + await ready(cm); + const mark = cm.markDocument(b); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + assert.strictEqual(raw.tokens, raw.pins[0]!.tokens, 'the total is the mark\'s alone, not the change beside it'); + // A pin over the mark's message and the change counts the change for nothing, too. + assert.ok(cm.unpin(mark)); + const range = cm.pinRange(b, changeId); + const ranged = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(ranged.pins.map((p) => [p.pinId, p.tokens]), [[range, raw.tokens]]); + assert.strictEqual(ranged.tokens, raw.tokens); + await cm.close(); + }); + }); +}); + +describe('the refusal\'s words for the pins it names', () => { + const diagnostics = { headTokens: 0, tailTokens: 10, middleTokens: 90, middleChunkCount: 3, deepestLevel: 1 }; + it('names one pin, a range read whole, and a pin with no name', () => { + const error = new OverBudgetError({ + budget: 50, actual: 100, + diagnostics: { ...diagnostics, rawPins: { tokens: 70, pins: [{ pinId: 'pin-1', kind: 'pin', firstMessageId: 'a', lastMessageId: 'c', parts: 5, tokens: 70 }] } }, + }); + assert.ok(error.message.endsWith( + '). A pin keeps ~70 tokens raw in the middle: pin pin-1, messages a to c, covering 5 stored messages. ' + + 'Whoever manages pins can unpin it, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), error.message); + }); + it('names several pins, and says nothing of pins without any', () => { + const error = new OverBudgetError({ + budget: 50, actual: 100, + diagnostics: { + ...diagnostics, + rawPins: { tokens: 80, pins: [ + { pinId: 'pin-0', kind: 'document', name: 'notes', firstMessageId: 'm', lastMessageId: 'm', tokens: 50 }, + { pinId: 'pin-2', kind: 'pin', firstMessageId: 'x', lastMessageId: 'x', tokens: 30 }, + ] }, + }, + }); + assert.ok(error.message.endsWith( + '). Pins keep ~80 tokens raw in the middle: document mark "notes" (pin-0), message m, ~50 tokens; pin pin-2, message x, ~30 tokens. ' + + 'Whoever manages pins can unpin them, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), error.message); + assert.ok(new OverBudgetError({ budget: 50, actual: 100, diagnostics }).message.endsWith(')')); + }); + it('ends with a remedy short enough to stay whole in the last quarter of a 600-character cut', () => { + for (const n of [1, 2]) { + const pins = Array.from({ length: n }, (_, i) => ({ pinId: `pin-${i}`, kind: 'pin' as const, name: 'x'.repeat(400), firstMessageId: 'm', lastMessageId: 'm', tokens: 10 })); + const message = new OverBudgetError({ budget: 50, actual: 100, diagnostics: { ...diagnostics, rawPins: { tokens: 10 * n, pins } } }).message; + // agent-framework quotes a compile error cut at 600 characters, keeping the head and the last 140. + const remedy = message.lastIndexOf(' Whoever manages pins'); + assert.ok(message.length > 600 && remedy >= 0, message); + assert.ok(message.length - remedy <= 140, message); + } + }); +});