From a0efb9ffdf387b5be5e4379dbffaa6a956ae1230 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 13:02:54 -0400 Subject: [PATCH 01/67] fix(images): request ceilings for images and image bytes, with the live window kept within them A request carries at most so many images and so many image bytes. The provider refuses one with more than 600 images and documents together, and membrane refuses, before sending it, one whose serialized messages pass its byte cap (28 MiB unless MEMBRANE_MAX_REQUEST_BYTES says otherwise). The live window's limits are targets a compile keeps to, and nothing bounded what a request could reach: a window configured as unlimited (0), or above those limits, compiled requests the provider or membrane would refuse, and a compression prompt had no count limit at all. A caller that appends to a compiled request without recompiling, such as a turn's continuation rounds, had no figure to stop at. - maxRequestImages (default 500) and maxRequestImageBytes (default 24 MiB) on AutobiographicalConfig and WindowedPassthroughOptions. 0 is no ceiling, and a value that isn't 0 or a finite number above it takes the default. - The window's count and byte limits are clamped to them in one place, resolveLiveImageLimits, which both strategies' strip, the hysteresis watermark and the post-strip estimates read. An unlimited limit means up to the ceiling, a limit above it keeps the ceiling's worth, and a fractional limit or ceiling is floored, to at least 1. - In the window, the ceilings bind what the window counts: top-level image blocks, by the length of their base64 source.data. An image nested in a tool result, a generated image and a data: URL's bytes aren't counted there, and a caller's compile-time injections and its appends are its own. maxLiveImages' doc says so once, and the other limits' docs refer to it. - ContextStrategy.liveImageLimits() reports the clamped limits and the ceilings. Both strategies implement it. - Compression prompts count every image, nested and generated ones included, against maxRequestImages, in the walk that keeps their byte budget: newest message first, and in block order within one. That budget stays within maxRequestImageBytes, and a budget that isn't a positive finite number means up to it. The log line names the count ceiling and the images kept. - A strategy warns once, when it initializes, when a limit sits above its ceiling (the compression budget included) or a ceiling is malformed. It marks a setting the host didn't give as the default, and gives a byte figure in MiB when it's a whole number of them. - Text: maxLiveImages' and windowed passthrough's docs no longer say 0 is unlimited or off, and maxLiveImages' doc says a window kept at the ceiling leaves an appending caller no room once it's full. maxLiveImageBytes' doc and the byte default's comment no longer promise membrane's whole-request cap or cite the API's 32 MB. Tests: request-image-ceilings covers the accessor's defaults, clamps, malformed and fractional values; each strategy's strip under the ceilings, and the window counting top-level images only (a nested and a generated image pass the count ceiling); the post-strip estimates under both ceilings and after the clamped limit changes; the watermark under both ceilings; the warning's cases; and the compression prompt's count and budget. The image-stripping test of a disabled policy now disables the ceilings too and carries more images than the default window, and the compression log line's test takes its new wording. Co-Authored-By: Claude Opus 5.5 --- changelog.d/request-image-ceilings.added.md | 2 + changelog.d/request-image-ceilings.changed.md | 4 + changelog.d/request-image-ceilings.fixed.md | 1 + src/index.ts | 1 + src/live-image-limits.ts | 135 ++++++ src/strategies/autobiographical.ts | 130 ++++-- src/strategies/windowed-passthrough.ts | 60 ++- src/types/index.ts | 1 + src/types/strategy.ts | 90 +++- test/compression-request-preparation.test.ts | 3 +- test/image-stripping.test.ts | 11 +- test/request-image-ceilings.test.ts | 393 ++++++++++++++++++ 12 files changed, 788 insertions(+), 43 deletions(-) create mode 100644 changelog.d/request-image-ceilings.added.md create mode 100644 changelog.d/request-image-ceilings.changed.md create mode 100644 changelog.d/request-image-ceilings.fixed.md create mode 100644 src/live-image-limits.ts create mode 100644 test/request-image-ceilings.test.ts diff --git a/changelog.d/request-image-ceilings.added.md b/changelog.d/request-image-ceilings.added.md new file mode 100644 index 00000000..92e2e51f --- /dev/null +++ b/changelog.d/request-image-ceilings.added.md @@ -0,0 +1,2 @@ +- `maxRequestImages` and `maxRequestImageBytes`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most inline image bytes, one request may carry. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `ContextStrategy.liveImageLimits()`, which both strategies implement, reports the live window's count and byte limits, kept within the ceilings, and the ceilings themselves. It's for a caller that appends to a compiled request without recompiling, such as a turn's continuation rounds, which can then keep its appends below the ceilings. It only reports them. `LiveImageLimits` is exported. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md new file mode 100644 index 00000000..7b6ac7d0 --- /dev/null +++ b/changelog.d/request-image-ceilings.changed.md @@ -0,0 +1,4 @@ +- The live window's image limits stay within the request ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A `maxLiveImages` or `maxLiveImageBytes` of 0, which meant unlimited, now means up to `maxRequestImages` or `maxRequestImageBytes`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- A `maxCompressionImageBytes` that isn't a positive finite number (0, a negative, `NaN` or `Infinity`) now means up to `maxRequestImageBytes`. It had meant no budget, or for `NaN`, that every image was replaced. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. +- `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/changelog.d/request-image-ceilings.fixed.md b/changelog.d/request-image-ceilings.fixed.md new file mode 100644 index 00000000..98b5a304 --- /dev/null +++ b/changelog.d/request-image-ceilings.fixed.md @@ -0,0 +1 @@ +- A compression prompt carries at most `maxRequestImages` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `maxRequestImageBytes`, and the line it logs when it replaces images names the count ceiling and the images kept. diff --git a/src/index.ts b/src/index.ts index 0df30c54..53b42883 100644 --- a/src/index.ts +++ b/src/index.ts @@ -106,6 +106,7 @@ export type { CompressionHoldOptions, CompressionHoldInfo, RenderedSummaryInfo, + LiveImageLimits, } from './types/index.js'; // Accepted rounds and fold receipts diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts new file mode 100644 index 00000000..f12b269d --- /dev/null +++ b/src/live-image-limits.ts @@ -0,0 +1,135 @@ +/** + * The live-image window's limits, clamped to the request ceilings. + * + * A strategy's live window keeps images newest-first up to a count + * (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The request + * ceilings (`maxRequestImages`, `maxRequestImageBytes`) are the most one + * request may carry at all, so the window's own limits never exceed them: + * an unlimited (0) target means "up to the ceiling", and a target above the + * ceiling keeps the ceiling's worth. A caller that appends to a compiled + * request without recompiling, such as a turn's continuation rounds, can + * read both through the strategy's `liveImageLimits()` and stop below the + * ceiling; nothing here bounds what it appends. + */ + +import type { LiveImageLimits } from './types/strategy.js'; + +/** + * The provider refuses a request carrying more than 600 images and documents + * together ("too many images and documents: 601 + 0 > 600", measured + * 2026-10-13 and again on 2026-10-19). 500 leaves room for documents, which + * nothing here counts, and for image blocks a host adds outside the compile. + */ +export const DEFAULT_MAX_REQUEST_IMAGES = 500; + +/** + * membrane refuses, before sending it, a request whose serialized messages + * exceed its byte cap: 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says + * otherwise (membrane 0.5.87 and 0.6.0, `assertWithinByteBudget`, checked at + * every round of a tool loop). 24 MiB of base64 images leaves 4 MiB of that + * default cap for everything else the messages carry: their text, their JSON, + * and any documents or other media. + */ +export const DEFAULT_MAX_REQUEST_IMAGE_BYTES = 24 * 1024 * 1024; + +/** A window limit as the window reads it: a whole number of 1 or more, or 0 + * for none. Any value that isn't a positive finite number (0, a negative, + * NaN, Infinity) is none, as the window's own checks (`limit > 0 && kept >= + * limit`) already read it. A fraction is floored, to at least 1, so the + * limit reported is the limit kept. */ +function asLimit(value: number): number { + return Number.isFinite(value) && value > 0 ? Math.max(1, Math.floor(value)) : 0; +} + +/** Whether a configured ceiling is one the window can read: 0 (none), or a + * finite number above 0. */ +function wellFormedCeiling(value: number): boolean { + return value === 0 || (Number.isFinite(value) && value > 0); +} + +/** A ceiling as configured: 0 is none, a finite number above 0 is floored to + * at least 1, and any other value (a negative, NaN, Infinity) takes the + * default, so a malformed config doesn't remove the bound (the warning names + * it). */ +function asCeiling(value: number, fallback: number): number { + return wellFormedCeiling(value) ? asLimit(value) : fallback; +} + +/** + * `target` kept within `ceiling`, where 0 means no limit for either: an + * unlimited target becomes the ceiling, and no ceiling leaves the target as + * it is. A target that isn't a positive finite number counts as unlimited, + * as the window's own checks already treat it. `ceiling` is one + * resolveLiveImageLimits gives. + */ +export function clampToCeiling(target: number, ceiling: number): number { + const c = asLimit(ceiling); + const t = asLimit(target); + if (c === 0) return t; + return t === 0 ? c : Math.min(t, c); +} + +/** The window's configured limits, the defaults applied where unset. */ +export interface ConfiguredImageLimits { + maxLiveImages: number; + maxLiveImageBytes: number; + maxRequestImages: number; + maxRequestImageBytes: number; +} + +/** The limits a strategy's window applies, and the ceilings above them. */ +export function resolveLiveImageLimits(configured: ConfiguredImageLimits): LiveImageLimits { + const ceiling = { + images: asCeiling(configured.maxRequestImages, DEFAULT_MAX_REQUEST_IMAGES), + bytes: asCeiling(configured.maxRequestImageBytes, DEFAULT_MAX_REQUEST_IMAGE_BYTES), + }; + return { + target: { + images: clampToCeiling(configured.maxLiveImages, ceiling.images), + bytes: clampToCeiling(configured.maxLiveImageBytes, ceiling.bytes), + }, + ceiling, + }; +} + +const MiB = 1024 * 1024; + +/** + * The warning a strategy gives once, when it initializes, or null when there + * is nothing to say: a ceiling configured as something other than 0 or a + * finite number above it, which takes the default, and a limit (not 0) above + * its ceiling, which is kept at the ceiling. `maxCompressionImageBytes`, when + * given, is the compression prompt's byte budget, held to the byte ceiling + * too. `defaulted` names the settings the host didn't give, which the line + * marks "(default)", so a host isn't told about a setting as if they had + * made it. A byte figure reads in MiB when it's a whole number of them. + */ +export function liveImageLimitsWarning( + strategy: string, + configured: ConfiguredImageLimits & { maxCompressionImageBytes?: number }, + defaulted: ReadonlySet, +): string | null { + const said: string[] = []; + const ceiling = resolveLiveImageLimits(configured).ceiling; + const count = (n: number) => String(n); + const bytes = (n: number) => (n % MiB === 0 ? `${n / MiB} MiB` : `${n} bytes`); + const setting = (name: string, shown: string) => `${name} ${shown}${defaulted.has(name) ? ' (default)' : ''}`; + if (!wellFormedCeiling(configured.maxRequestImages)) { + said.push(`maxRequestImages ${configured.maxRequestImages} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); + } + if (!wellFormedCeiling(configured.maxRequestImageBytes)) { + said.push(`maxRequestImageBytes ${configured.maxRequestImageBytes} isn't 0 or a finite number above it, so the default ${bytes(ceiling.bytes)} applies`); + } + const above = ( + name: string, value: number | undefined, ceilingName: string, limit: number, keeps: string, show: (n: number) => string, + ) => { + const v = value === undefined ? 0 : asLimit(value); + if (limit > 0 && v > limit) { + said.push(`${setting(name, show(v))} is above ${setting(ceilingName, show(limit))}, so ${keeps} at most ${show(limit)}`); + } + }; + above('maxLiveImages', configured.maxLiveImages, 'maxRequestImages', ceiling.images, 'the live window keeps', count); + above('maxLiveImageBytes', configured.maxLiveImageBytes, 'maxRequestImageBytes', ceiling.bytes, 'the live window keeps', bytes); + above('maxCompressionImageBytes', configured.maxCompressionImageBytes, 'maxRequestImageBytes', ceiling.bytes, 'a compression prompt keeps', bytes); + return said.length === 0 ? null : `[${strategy}] ${said.join('; ')}`; +} diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index d24e484c..3d1ca229 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -5,6 +5,7 @@ import { registerStateIfAbsent } from '../state-registration.js'; import { phaseChannel } from '../phase-channel.js'; import type { ContextStrategy, + LiveImageLimits, ResettableStrategy, StrategyContext, ReadinessState, @@ -77,6 +78,13 @@ import { bodyBoundsIn, bodyEnd, bodyStart } from '../body-runs.js'; import { releaseEditedRawForms, releaseSharedRawFormsOnMove, releaseTrailingReasoning, releaseVanishedRawForms, withoutRawForms } from '../raw-forms.js'; import { isCompressionWithdrawn } from '../compression-withdrawn.js'; import { CAP_OMISSION_TEXT, capContent, cappedPriceExtra, cappedWhole, isCapMarker, isEmptyTextBlock } from './message-cap.js'; +import { + DEFAULT_MAX_REQUEST_IMAGES, + DEFAULT_MAX_REQUEST_IMAGE_BYTES, + clampToCeiling, + liveImageLimitsWarning, + resolveLiveImageLimits, +} from '../live-image-limits.js'; import type { MessageId } from '../types/message.js'; import type { IngressChunkResult } from '../types/strategy.js'; import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; @@ -1156,8 +1164,12 @@ interface CompressionInFlightResult { interface PreparationCounts { serializedMediaReplacements: number; imageReplacements: number; + /** The inline image byte budget, 0 for none. */ imageBudgetBytes: number; imageBytesKept: number; + /** The most images the prompt may carry (maxRequestImages), 0 for none. */ + imageCountCap: number; + imagesKept: number; } /** A compression request after preparation, with what preparation replaced. */ @@ -2409,6 +2421,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Bind to the chronicle store + namespace for persistent strategy state. this.store = ctx.store; this.ns = ctx.namespace; + this.reportImageLimitsWarningOnce(); // Its tools getter follows the host's declarations (compressionWorkQueued). this.hostContext = ctx; observeStoreBranch(ctx.store); @@ -16016,7 +16029,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Post-pass over compiled entries: replace image blocks with a text * placeholder once they fall outside the live-image window — either deeper * than `imageStripDepthTokens` from the newest message, or beyond the - * `maxLiveImages` most-recent images (counted newest-first). Summaries are + * `maxLiveImages` most-recent images (counted newest-first), or past + * `maxLiveImageBytes`, the count and bytes each within its request ceiling + * (liveImageLimits). Summaries are * already text, so they're naturally unaffected. The adjacent * "[image attachment: ]" text added at ingest preserves the filename, * so the placeholder itself stays terse. Reduces tokens, so it never pushes @@ -16028,10 +16043,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the in-progress render stats by the reclaimed tokens, keeping * `RenderStats.total` equal to the real rendered size. */ protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView): void { - const maxLive = this.config.maxLiveImages ?? 0; // 0 = unlimited count + // Count and bytes within the request ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; const depthTokens = this.config.imageStripDepthTokens ?? 0; // 0 = no depth strip - const maxLiveBytes = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; - if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return; // policy disabled + if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return; // no window and no ceilings const messages = store.getAll(); const posById = this.positionIndex(messages); @@ -16122,8 +16137,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const ratio = this.imageHysteresisRatio(); if (ratio === undefined) return; const stripDepth = this.config.imageStripDepthTokens ?? 0; - const maxLive = this.config.maxLiveImages ?? 0; - const maxLiveBytes = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; + const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; if (stripDepth === 0 && maxLive === 0 && maxLiveBytes === 0) return; const messages = store.getAll(); const w = this.imageWatermark(); @@ -16186,7 +16200,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Post-strip token estimate per message index (2026-07-12 tail-starvation - * fix). Mirrors `applyImageStripping`: an image beyond the `maxLiveImages` + * fix). Mirrors `applyImageStripping`, with the same limits from + * liveImageLimits(), the count and bytes within their request ceilings: + * an image beyond the `maxLiveImages` * newest (counted newest-first) or deeper than `imageStripDepthTokens` of * raw estimate from the live end renders as a placeholder — so every place * that BUDGETS messages (recent-window walk-back, head/tail sums, middle @@ -16199,8 +16215,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected postStripEstimates(store: MessageStoreView): number[] { const messages = store.getAll(); const stripDepth = this.config.imageStripDepthTokens ?? 0; - const maxLive = this.config.maxLiveImages ?? 0; - const maxLiveBytes = this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES; + const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; // A compile asks for this pass more than once (recent-window scan, then // the picker inputs). The listing array is the store's own cached view, // so its identity, calibration and the knobs key the memo, the @@ -16264,9 +16279,55 @@ export class AutobiographicalStrategy implements ResettableStrategy { messages: readonly StoredMessage[]; calibration: number; stripDepth: number; maxLive: number; maxLiveBytes: number; ratio: number | undefined; out: number[]; } | null = null; - /** Byte wall default: 20MB of base64 (API total-request cap is 32MB). */ + /** Byte wall default: 20 MiB of base64, under the default request byte + * ceiling (24 MiB), which leaves room under membrane's 28 MiB cap. */ protected static readonly DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; + /** The image limits the host didn't give, whose values are defaults. */ + private imageLimitsDefaulted(): ReadonlySet { + const keys = ['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes', 'maxCompressionImageBytes'] as const; + return new Set(keys.filter((key) => this.configProvenance[key] !== 'caller' || this.config[key] == null)); + } + + /** The live-image limits as configured, with the defaults where unset. */ + private configuredImageLimits(): { + maxLiveImages: number; maxLiveImageBytes: number; maxRequestImages: number; maxRequestImageBytes: number; + } { + return { + maxLiveImages: this.config.maxLiveImages ?? 0, + maxLiveImageBytes: this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES, + maxRequestImages: this.config.maxRequestImages ?? DEFAULT_MAX_REQUEST_IMAGES, + maxRequestImageBytes: this.config.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES, + }; + } + + /** + * The live window's count and byte limits, clamped to the request + * ceilings, and the ceilings themselves (see ContextStrategy.liveImageLimits). + * The strip, the watermark and the post-strip estimates all read their + * limits here, so they can't disagree about them. + */ + liveImageLimits(): LiveImageLimits { + return resolveLiveImageLimits(this.configuredImageLimits()); + } + + /** Whether the image limits' warning has been said (once per instance). */ + private imageLimitsWarningReported = false; + + /** One line when a configured image limit sits above its request ceiling, + * or a ceiling is malformed and takes its default, so an operator sees why + * the window or a compression prompt keeps fewer images than they set. */ + private reportImageLimitsWarningOnce(): void { + if (this.imageLimitsWarningReported) return; + this.imageLimitsWarningReported = true; + const line = liveImageLimitsWarning(this.name, { + ...this.configuredImageLimits(), + maxCompressionImageBytes: + this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES, + }, this.imageLimitsDefaulted()); + if (line) this.diag.warn(line); + } + /** Compression prompts carry head + recall frontier + raw chunk alongside * their images, so they get a tighter image budget than the live window. */ protected static readonly DEFAULT_MAX_COMPRESSION_IMAGE_BYTES = 12 * 1024 * 1024; @@ -16307,12 +16368,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected prepareCompressionRequest(request: NormalizedRequest): CompressionPreparation { const prepared = { ...request, messages: request.messages.map(message => ({ ...message })) }; const serializedMediaReplacements = this.stripSerializedCompressionMedia(prepared.messages); - const imageBudgetBytes = - this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES; - const cap = { keptBytes: 0 }; - const imageReplacements = this.capCompressionImageBytes(prepared.messages, imageBudgetBytes, cap); + // A compression prompt is a request too: at most the request ceilings. + const { ceiling } = this.liveImageLimits(); + const imageBudgetBytes = clampToCeiling( + this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES, + ceiling.bytes, + ); + const imageCountCap = ceiling.images; + const cap = { keptBytes: 0, keptImages: 0 }; + const imageReplacements = this.capCompressionImageBytes(prepared.messages, imageBudgetBytes, cap, imageCountCap); return { request: prepared, serializedMediaReplacements, imageReplacements, imageBudgetBytes, imageBytesKept: cap.keptBytes, + imageCountCap, imagesKept: cap.keptImages, }; } @@ -16354,10 +16421,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { } if (counts.imageReplacements > 0) { const mib = (bytes: number) => `${(bytes / (1024 * 1024)).toFixed(1)}MiB`; + const limits = [ + ...(counts.imageBudgetBytes > 0 ? [`the ${mib(counts.imageBudgetBytes)} inline image budget`] : []), + ...(counts.imageCountCap > 0 ? [`the ${counts.imageCountCap}-image request ceiling`] : []), + ].join(' and '); this.diag.error( `[autobiographical] compression prompt: replaced ${counts.imageReplacements} image(s) with placeholders ` + - `to stay under the ${mib(counts.imageBudgetBytes)} inline image budget ` + - `(kept ${mib(counts.imageBytesKept)}, newest messages first) [${label}]`, + `to stay under ${limits} ` + + `(kept ${counts.imagesKept} image(s), ${mib(counts.imageBytesKept)}, newest messages first) [${label}]`, ); } } @@ -16578,26 +16649,31 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * Cap inline image bytes in a COMPRESSION prompt (2026-07-12). The main + * Cap inline image bytes in a COMPRESSION prompt (2026-07-12), and with + * `capImages` their number (2026-10-19, `maxRequestImages`). The main * window enforces `maxLiveImageBytes` through the strip policy; the * summarizer's raw-chunk replay had no such wall and leaned on membrane's * byte shed instead — which is a transport backstop, not a policy owner * (it fired at 27MB on a live merge). Every inline image counts, whatever * its shape (see compressionImageBytes), generated images included. * Messages are walked newest-first, and the blocks within a message in - * order; each image is kept if it fits in what's left of the budget, and - * the others become the same loud placeholder the window uses, so the - * summarizer is told plainly that it is not seeing them. + * order; each image is kept if it fits in what's left of the budget and + * of the count, and the others become the same loud placeholder the window + * uses, so the summarizer is told plainly that it is not seeing them. + * A cap of 0 or less is none. * Returns the number of images replaced; `stats`, when given, receives the - * bytes kept. + * bytes and the images kept. */ protected capCompressionImageBytes( messages: Array<{ content: ContentBlock[] }>, capBytes: number, - stats?: { keptBytes: number }, + stats?: { keptBytes: number; keptImages?: number }, + capImages = 0, ): number { - if (capBytes <= 0) return 0; + const countCap = capImages > 0 ? capImages : 0; + if (capBytes <= 0 && countCap === 0) return 0; let kept = 0; + let keptImages = 0; let dropped = 0; // Recurse into tool_result content: an agent that drives a shell/plotter/ // browser carries most of its image bytes NESTED in tool results, not as @@ -16610,8 +16686,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const capped = blocks.map((b) => { if (b.type === 'image' || b.type === 'generated_image') { const bytes = AutobiographicalStrategy.compressionImageBytes(b); - if (kept + bytes <= capBytes) { + if ((capBytes <= 0 || kept + bytes <= capBytes) && (countCap === 0 || keptImages < countCap)) { kept += bytes; + keptImages++; return b; } dropped++; @@ -16634,7 +16711,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (!Array.isArray(m.content)) continue; m.content = capBlocks(m.content); } - if (stats) stats.keptBytes = kept; + if (stats) { + stats.keptBytes = kept; + stats.keptImages = keptImages; + } return dropped; } diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 60f707bd..9bf67486 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -1,6 +1,7 @@ import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; import type { ContextStrategy, + LiveImageLimits, StrategyContext, ReadinessState, MessageStoreView, @@ -18,6 +19,12 @@ import type { JsStore } from '@animalabs/chronicle'; import { OverBudgetError } from '../adaptive/picker.js'; import { observeStoreBranch, type StoreBranchGeneration } from '../branch-generation.js'; import { capContent, cappedPriceExtra } from './message-cap.js'; +import { + DEFAULT_MAX_REQUEST_IMAGES, + DEFAULT_MAX_REQUEST_IMAGE_BYTES, + liveImageLimitsWarning, + resolveLiveImageLimits, +} from '../live-image-limits.js'; /** * Options for {@link WindowedPassthroughStrategy}. @@ -66,13 +73,26 @@ export interface WindowedPassthroughOptions { * `imageStripDepthTokens` from the newest message, or past * `maxLiveImageBytes` of cumulative inline image bytes (newest-first). * Defaults follow `DEFAULT_AUTOBIOGRAPHICAL_CONFIG` (and the memory - * strategy's byte wall); 0 disables that dimension. A strip changes the + * strategy's byte wall). A count or byte limit of 0 means up to its + * request ceiling, below, and a depth of 0 means no depth strip. A strip changes the * bytes at that position, which the cache-marker placement absorbs by * measurement (the marker falls back to the deepest byte-stable entry). */ maxLiveImages?: number; imageStripDepthTokens?: number; maxLiveImageBytes?: number; + /** + * The request ceilings, mirroring `AutobiographicalConfig`: the most images, + * and the most inline image bytes, one request may carry. The window keeps + * at most these of the images it counts, counted as + * `AutobiographicalConfig.maxLiveImages` says, whatever `maxLiveImages` and + * `maxLiveImageBytes` say (0 there means up to these), and + * `liveImageLimits()` reports them. + * Defaults 500 images and 24 MiB; 0 = no ceiling, and a value that isn't 0 + * or a finite number above it takes the default. + */ + maxRequestImages?: number; + maxRequestImageBytes?: number; } /** Text substituted for an image block once it leaves the live-image window @@ -129,9 +149,15 @@ export class WindowedPassthroughStrategy implements ContextStrategy { private readonly reAnchorFraction: number; private readonly anchorStateIdOverride?: string; - private readonly maxLiveImages: number; + /** The image limits as configured, with the defaults where unset. */ + private readonly configuredImageLimits: { + maxLiveImages: number; maxLiveImageBytes: number; maxRequestImages: number; maxRequestImageBytes: number; + }; + /** The image limits the options didn't give, whose values are defaults. */ + private readonly imageLimitsDefaulted: ReadonlySet; private readonly imageStripDepthTokens: number; - private readonly maxLiveImageBytes: number; + /** Whether the image limits' warning has been said (once per instance). */ + private imageLimitsWarningReported = false; private anchor: Sequence = 0; private store: JsStore | null = null; @@ -154,15 +180,35 @@ export class WindowedPassthroughStrategy implements ContextStrategy { this.reAnchorFraction = fraction; this.anchorStateIdOverride = options.anchorStateId; this.maxMessageTokens = options.maxMessageTokens; - this.maxLiveImages = options.maxLiveImages ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.maxLiveImages ?? 0; + this.configuredImageLimits = { + maxLiveImages: options.maxLiveImages ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.maxLiveImages ?? 0, + maxLiveImageBytes: options.maxLiveImageBytes ?? DEFAULT_MAX_LIVE_IMAGE_BYTES, + maxRequestImages: options.maxRequestImages ?? DEFAULT_MAX_REQUEST_IMAGES, + maxRequestImageBytes: options.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES, + }; + this.imageLimitsDefaulted = new Set( + (['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes'] as const).filter((key) => options[key] == null), + ); this.imageStripDepthTokens = options.imageStripDepthTokens ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.imageStripDepthTokens ?? 0; - this.maxLiveImageBytes = options.maxLiveImageBytes ?? DEFAULT_MAX_LIVE_IMAGE_BYTES; + } + + /** + * The live window's count and byte limits, clamped to the request + * ceilings, and the ceilings themselves (see ContextStrategy.liveImageLimits). + */ + liveImageLimits(): LiveImageLimits { + return resolveLiveImageLimits(this.configuredImageLimits); } async initialize(ctx: StrategyContext): Promise { this.store = ctx.store; this.ns = ctx.namespace; + if (!this.imageLimitsWarningReported) { + this.imageLimitsWarningReported = true; + const line = liveImageLimitsWarning(this.name, this.configuredImageLimits, this.imageLimitsDefaulted); + if (line) this.diag.warn(line); + } this.stateId = this.anchorStateIdOverride ?? `${ctx.namespace}/windowed:anchor`; registerStateIfAbsent(this.store, { id: this.stateId, strategy: 'snapshot' }); this.loadBranchState(); @@ -384,9 +430,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { messages: StoredMessage[], store: MessageStoreView, ): void { - const maxLive = this.maxLiveImages; + // Count and bytes within the request ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; const depthTokens = this.imageStripDepthTokens; - const maxLiveBytes = this.maxLiveImageBytes; if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return; // Depth boundary: walk newest→oldest over the whole view (the policy is diff --git a/src/types/index.ts b/src/types/index.ts index d30af70d..60e53133 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -79,6 +79,7 @@ export type { CalibrationReset, RenderStatsCapableStrategy, RenderedSummaryInfo, + LiveImageLimits, } from './strategy.js'; export { diff --git a/src/types/strategy.ts b/src/types/strategy.ts index e90f57fa..5462c318 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -235,6 +235,29 @@ export interface ContextStrategy { * (for example 'budget-fit' or 'window'). Reading clears it. */ takeSelectionCause?(): string | undefined; + + /** + * The live-image window's limits and the request ceilings above them, for + * a caller that appends to a compiled request without recompiling, such + * as a turn's continuation rounds: the strategy's compile keeps the images + * its window counts within `target`, and the caller can let the request + * grow to `ceiling` before compiling it again. It only reports the figures: + * a caller bounds what it appends, and what it injects at compile time. + * Absent on a strategy with no live-image window. + */ + liveImageLimits?(): LiveImageLimits; +} + +/** + * A strategy's live-image limits. Each figure is a count of image blocks or + * a sum of their inline base64 bytes, and 0 means no limit. `target` is + * what the window keeps, already within `ceiling` (`maxLiveImages` and + * `maxLiveImageBytes`, clamped); `ceiling` is the most one request may carry + * (`maxRequestImages` and `maxRequestImageBytes`). + */ +export interface LiveImageLimits { + target: { images: number; bytes: number }; + ceiling: { images: number; bytes: number }; } /** What a rendered summary covers, for layout reporting. */ @@ -1055,8 +1078,20 @@ export interface AutobiographicalConfig { /** Maximum number of images kept "live" (sent as real image blocks) in the * compiled context. Counted newest-first across the whole window; images - * beyond this many are replaced with a text placeholder. 0 = unlimited. - * A hard ceiling for dense bursts that pack many images into the live window. */ + * beyond this many are replaced with a text placeholder. Kept within + * `maxRequestImages`, so 0 means up to that ceiling. A window kept at the + * ceiling leaves a caller that appends to the compiled request no room + * below it once it's full, so that caller recompiles at each image it + * adds; at about 1,600 tokens an image, the token budget binds first + * unless the images are small. A fraction is floored. + * A hard ceiling for dense bursts that pack many images into the live window. + * The window counts top-level `image` blocks, and measures their bytes by + * the length of their base64 `source.data`. So an image nested in a tool + * result and a `generated_image` are neither counted nor replaced, and an + * image with a `data:` URL counts but its bytes don't. In the window, + * every live-image limit and request ceiling below counts the same way; a + * compression prompt counts every image, nested and generated ones + * included (see `maxRequestImages`). */ maxLiveImages?: number; /** Token depth from the newest message beyond which images are stripped to a * text placeholder, even though the surrounding text stays verbatim. Measured @@ -1076,10 +1111,12 @@ export interface AutobiographicalConfig { /** * Byte wall for live images (2026-07-12): keep inline images newest-first * only while their cumulative base64 size stays under this budget — on top - * of `maxLiveImages` / `imageStripDepthTokens`. Guarantees the compiled - * window fits the API's total request size cap (413 request_too_large), so - * membrane's oversize check is a true invariant, never a silent editor. - * Default 20MB (of base64; the API cap is 32MB total). + * of `maxLiveImages` / `imageStripDepthTokens`. Default 20 MiB of base64. + * The window keeps at most `maxRequestImageBytes` whatever this says, and 0 + * here means up to that ceiling. It counts the bytes of the images the + * window counts (see `maxLiveImages`), not the request's other bytes, so on + * its own it can't keep a request under membrane's byte cap: see + * `maxRequestImageBytes`. */ maxLiveImageBytes?: number; @@ -1100,10 +1137,49 @@ export interface AutobiographicalConfig { * Image byte budget for COMPRESSION prompts (summarizer). Tighter than the * live window's: the prompt also carries the head, the recall frontier and * the raw chunk. Newest-first; older images become loud placeholders. - * Default 12MB of base64. + * Default 12MB of base64. A compression prompt carries at most + * `maxRequestImages` images and `maxRequestImageBytes` bytes whatever this + * says, and 0 here, or any value that isn't a positive finite number, means + * up to the byte ceiling. */ maxCompressionImageBytes?: number; + /** + * The most images one request may carry (2026-10-19). The live window keeps + * at most this many of the images it counts (see `maxLiveImages`), + * whatever `maxLiveImages` says, so a `maxLiveImages` of 0 means up to this + * many. A compression prompt carries at most this many images, nested ones + * and generated ones included. `liveImageLimits()` reports it, for a + * caller that appends to a compiled request without recompiling, such as a + * turn's continuation rounds, to keep its own appends below; nothing here + * bounds what a caller appends or injects. The provider refuses a request + * with more than 600 images and documents together (measured 2026-10-13, + * and again on 2026-10-19). Default 500, which leaves room for documents, + * which nothing here counts, and for image blocks a host adds outside the + * compile. 0 = no ceiling. A value that isn't 0 or a finite number above it + * takes the default, and the strategy's warning at initialize names it. A + * fraction is floored. + */ + maxRequestImages?: number; + + /** + * The most inline image bytes (base64) one request may carry (2026-10-19), + * as `maxRequestImages` is for their number. The live window keeps at most + * this many bytes of the images it counts (see `maxLiveImages`), whatever + * `maxLiveImageBytes` says, and a + * compression prompt carries at most this many bytes of its images. + * `liveImageLimits()` reports it, as for `maxRequestImages`. membrane + * refuses, before sending it, a request whose serialized messages exceed + * its byte cap, 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise. + * Default 24 MiB, which leaves 4 MiB of that cap for everything else the + * messages carry: their text, their JSON, and any documents or other media, + * which nothing here counts. It doesn't follow `MEMBRANE_MAX_REQUEST_BYTES`, + * so a host that changes membrane's cap sets this to match. 0 = no ceiling. + * A value that isn't 0 or a finite number above it takes the default, as + * for `maxRequestImages`. + */ + maxRequestImageBytes?: number; + /** * Merge grouping: exclude candidates whose OWN source span exceeds the * level-scaled limit `base × mergeThreshold^(max(0, level − 3))`, where diff --git a/test/compression-request-preparation.test.ts b/test/compression-request-preparation.test.ts index 28a52c73..20f4b686 100644 --- a/test/compression-request-preparation.test.ts +++ b/test/compression-request-preparation.test.ts @@ -203,7 +203,8 @@ describe('preparation counts and logging', () => { assert.deepEqual(sent.logs, [ '[autobiographical] compression prompt: replaced 1 serialized data URL(s) with markers [count-test]', '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + - '3.0MiB inline image budget (kept 2.5MiB, newest messages first) [count-test]', + '3.0MiB inline image budget and the 500-image request ceiling (kept 1 image(s), 2.5MiB, newest messages first) ' + + '[count-test]', ]); assert.deepEqual(captureErrors(() => strategy.send(request([{ type: 'text', text: 'plain' }]), 'unchanged')).logs, []); }); diff --git a/test/image-stripping.test.ts b/test/image-stripping.test.ts index a590da3e..3427262d 100644 --- a/test/image-stripping.test.ts +++ b/test/image-stripping.test.ts @@ -108,15 +108,20 @@ describe('Live image policy', () => { hierarchical: true, maxLiveImages: 0, imageStripDepthTokens: 0, + maxLiveImageBytes: 0, + // The request ceilings bound an unlimited window too. + maxRequestImages: 0, + maxRequestImageBytes: 0, }); const manager = await ContextManager.open({ path: TEST_STORE_PATH, strategy }); - for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-0${i}`)); + // Eight, more than the default window of 6. + for (let i = 1; i <= 8; i++) manager.addMessage('user', imageMessage(`img-0${i}`)); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - for (const label of ['img-01', 'img-02', 'img-03']) { - assert.equal(imageState(compiled.messages, label), 'image', `${label} kept when policy disabled`); + for (let i = 1; i <= 8; i++) { + assert.equal(imageState(compiled.messages, `img-0${i}`), 'image', `img-0${i} kept when policy disabled`); } manager.close(); diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts new file mode 100644 index 00000000..df611367 --- /dev/null +++ b/test/request-image-ceilings.test.ts @@ -0,0 +1,393 @@ +/** + * The request ceilings (maxRequestImages, maxRequestImageBytes): the most + * images, and image bytes, one request may carry. The live window keeps its + * own limits within them, compression prompts carry no more than them, and + * liveImageLimits() reports both, for a caller that appends to a compiled + * request without recompiling. + */ +import { describe, it } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, WindowedPassthroughStrategy } from '../src/index.js'; +import type { ContextStrategy } from '../src/index.js'; + +const MiB = 1024 * 1024; +const PLACEHOLDER = '[image not shown: this context keeps only some images]'; + +/** A message holding one image of `bytes` base64 characters and a label. */ +const imageMessage = (label: string, bytes = 4): ContentBlock[] => [ + { type: 'image', source: { type: 'base64', data: 'A'.repeat(bytes), mediaType: 'image/png' }, tokenEstimate: 100 } as ContentBlock, + { type: 'text', text: label }, +]; + +/** Which of img-1..n kept its image ('I'), was stripped ('.') or is missing ('-'). */ +function pattern(messages: Array<{ content: ContentBlock[] }>, n: number): string { + let out = ''; + for (let i = 1; i <= n; i++) { + const m = messages.find((x) => x.content.some((b) => b.type === 'text' && b.text === `img-${i}`)); + out += !m ? '-' : m.content.some((b) => b.type === 'image') ? 'I' : '.'; + } + return out; +} + +/** Compile a fresh store holding `n` image messages of `bytes` each. */ +async function compiled(strategy: ContextStrategy, n: number, bytes = 4): Promise { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + for (let i = 1; i <= n; i++) manager.addMessage('user', imageMessage(`img-${i}`, bytes)); + const result = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + return pattern(result.messages, n); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +const memory = (options: Record) => new AutobiographicalStrategy({ + headWindowTokens: 0, recentWindowTokens: 1_000_000, hierarchical: true, imageStripDepthTokens: 0, ...options, +} as ConstructorParameters[0]); +const windowed = (options: Record) => new WindowedPassthroughStrategy({ + imageStripDepthTokens: 0, ...options, +} as ConstructorParameters[0]); +const STRATEGIES = { autobiographical: memory, 'windowed passthrough': windowed }; +const DEFAULTS = { + autobiographical: () => new AutobiographicalStrategy(), + 'windowed passthrough': () => new WindowedPassthroughStrategy(), +}; + +describe('liveImageLimits()', () => { + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: the defaults are a window of 6 images and 20 MiB under ceilings of 500 images and 24 MiB`, () => { + assert.deepEqual(DEFAULTS[name as keyof typeof DEFAULTS]().liveImageLimits(), { + target: { images: 6, bytes: 20 * MiB }, + ceiling: { images: 500, bytes: 24 * MiB }, + }); + }); + + it(`${name}: the window is kept within the ceilings, and a ceiling of 0 is none`, () => { + const limits = (options: Record) => make(options).liveImageLimits(); + assert.deepEqual(limits({ maxLiveImages: 0, maxLiveImageBytes: 0 }), { + target: { images: 500, bytes: 24 * MiB }, ceiling: { images: 500, bytes: 24 * MiB }, + }, 'an unlimited window means up to the ceilings'); + assert.deepEqual(limits({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }).target, + { images: 500, bytes: 24 * MiB }, 'a window above the ceilings keeps the ceilings'); + assert.deepEqual(limits({ maxLiveImages: 4, maxLiveImageBytes: MiB, maxRequestImages: 10, maxRequestImageBytes: 2 * MiB }), { + target: { images: 4, bytes: MiB }, ceiling: { images: 10, bytes: 2 * MiB }, + }, 'a window under configured ceilings keeps its own limits'); + assert.deepEqual(limits({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0 }), { + target: { images: 0, bytes: 0 }, ceiling: { images: 0, bytes: 0 }, + }, 'no ceiling leaves an unlimited window unlimited'); + assert.equal(limits({ maxLiveImages: 4, maxRequestImages: 0 }).target.images, 4, 'no ceiling leaves a window as set'); + }); + + it(`${name}: a window limit that isn't a positive finite number is none, and a malformed ceiling takes its default`, () => { + const limits = (options: Record) => make(options).liveImageLimits(); + for (const bad of [-3, Number.NaN, Infinity]) { + assert.equal(limits({ maxLiveImages: bad }).target.images, 500, `maxLiveImages ${bad} is unlimited, so up to the ceiling`); + assert.equal(limits({ maxLiveImageBytes: bad }).target.bytes, 24 * MiB, `maxLiveImageBytes ${bad} is unlimited, so up to the ceiling`); + assert.equal(limits({ maxRequestImages: bad }).ceiling.images, 500, `maxRequestImages ${bad} takes the default`); + assert.equal(limits({ maxRequestImageBytes: bad }).ceiling.bytes, 24 * MiB, `maxRequestImageBytes ${bad} takes the default`); + } + assert.equal(limits({ maxLiveImages: 0, maxRequestImages: -1 }).target.images, 500, 'the window is clamped to the default the malformed ceiling took'); + assert.equal(limits({ maxLiveImages: Infinity, maxRequestImages: 0 }).target.images, 0, + 'an infinite limit with no ceiling is reported as none, 0'); + }); + + it(`${name}: a fractional limit or ceiling is floored, to at least 1`, () => { + const limits = (options: Record) => make(options).liveImageLimits(); + assert.equal(limits({ maxLiveImages: 2.5 }).target.images, 2); + assert.deepEqual(limits({ maxLiveImages: 0, maxRequestImages: 1.5 }), { + target: { images: 1, bytes: 20 * MiB }, ceiling: { images: 1, bytes: 24 * MiB }, + }); + assert.equal(limits({ maxRequestImages: 0.5 }).ceiling.images, 1, 'a fraction under 1 is 1, not none'); + assert.equal(limits({ maxRequestImageBytes: 100.7 }).ceiling.bytes, 100); + }); + } +}); + +describe('the live window within the request ceilings', () => { + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: an unlimited count keeps the newest maxRequestImages`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, maxRequestImages: 2 }), 4), '..II'); + }); + it(`${name}: a count above the ceiling keeps the ceiling's worth`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 3, maxRequestImages: 2 }), 4), '..II'); + }); + it(`${name}: an unlimited byte budget keeps the newest maxRequestImageBytes`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 2000 }), 4, 1000), '..II'); + }); + it(`${name}: with no window and no ceilings, every image stays, more than the default window's 6`, async () => { + assert.equal(await compiled(make({ + maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0, + }), 8), 'IIIIIIII'); + }); + it(`${name}: the window counts top-level image blocks only, so a nested or a generated image passes the count ceiling`, async () => { + // What the ceilings bind is what the window counts. An image nested in + // a tool result, or a generated image, isn't one of those (a change + // that counts them would change this). + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, maxRequestImages: 1 }) }); + try { + const image = (data: string) => + ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; + manager.addMessage('Claude', [{ type: 'tool_use', id: 'tu-1', name: 'shot', input: {} }] as ContentBlock[]); + manager.addMessage('user', [{ type: 'tool_result', toolUseId: 'tu-1', content: [image('AAAA'), image('BBBB')] }] as ContentBlock[]); + for (let i = 1; i <= 2; i++) { + manager.addMessage('user', [ + { type: 'generated_image', data: 'A'.repeat(4), mimeType: 'image/png' } as ContentBlock, + { type: 'text', text: `gen-${i}` }, + ]); + } + const result = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const blocks = result.messages.flatMap((m) => m.content); + const nested = blocks + .filter((b) => b.type === 'tool_result') + .flatMap((b) => (b as { content: ContentBlock[] }).content) + .filter((b) => b.type === 'image').length; + assert.equal(nested, 2, 'both nested images stay, past a count ceiling of 1'); + const kept = blocks.filter((b) => b.type === 'generated_image').length; + assert.equal(kept, 2, 'both generated images stay, past a count ceiling of 1'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + it('the post-strip estimates price an image the ceiling strips as the placeholder', () => { + const image = (): ContentBlock => + ({ type: 'image', source: { type: 'base64', data: 'AAAA', mediaType: 'image/png' } }) as ContentBlock; + const messages = [{ id: 'old', content: [image()] }, { id: 'new', content: [image()] }]; + const store = { + getAll: () => messages as never[], + get: (id: string) => (messages.find((m) => m.id === id) as never) ?? null, + getFrom: (i: number) => messages.slice(i) as never[], + getTail: (n: number) => messages.slice(-n) as never[], + length: () => messages.length, + estimateTokens: (m: { content: ContentBlock[] }) => + m.content.reduce((sum, b) => sum + (b.type === 'image' ? 1600 : b.type === 'text' ? Math.ceil(b.text.length / 4) : 0), 0), + getTokenCalibration: () => 1, + }; + const estimate = (strategy: AutobiographicalStrategy) => + (strategy as unknown as { postStripEstimates(s: typeof store): number[] }).postStripEstimates(store); + const stripped = [Math.ceil(PLACEHOLDER.length / 4), 1600]; + assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxRequestImages: 1 })), stripped, 'by the count ceiling'); + assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 5 })), stripped, + 'by the byte ceiling: 4 bytes fit, 8 do not'); + // The memo is keyed by the clamped limits, so a changed ceiling prices anew. + const strategy = memory({ maxLiveImages: 0, maxRequestImages: 1 }); + assert.deepEqual(estimate(strategy), stripped); + (strategy as unknown as { config: { maxRequestImages: number } }).config.maxRequestImages = 2; + assert.deepEqual(estimate(strategy), [1600, 1600], 'with a ceiling of 2, both are live'); + }); + + it('the hysteresis watermark trims against the ceiling when the count is unlimited', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + }); + try { + const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + assert.equal(await strip(4), 'IIII'); + manager.addMessage('user', imageMessage('img-5')); + assert.equal(await strip(5), '...II', 'the ceiling binds: one trim to half of it, not a slide of one'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('the hysteresis watermark against the byte ceiling', () => { + it('trims against the byte ceiling when the byte budget is unlimited', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 16, imageStripHysteresisRatio: 0.5 }), + }); + try { + const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`, 4)); + assert.equal(await strip(4), 'IIII', 'four 4-byte images fill the 16-byte ceiling'); + manager.addMessage('user', imageMessage('img-5', 4)); + assert.equal(await strip(5), '...II', 'the ceiling binds: one trim to half of it'); + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('the image limits warning', () => { + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: says once, at initialize, when a configured window limit is above its ceiling`, async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const cases = [ + [{ maxLiveImages: 8, maxRequestImages: 4 }, ['maxLiveImages 8 is above maxRequestImages 4, so the live window keeps at most 4']], + [{ maxLiveImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }, + ['maxLiveImageBytes 3 MiB is above maxRequestImageBytes 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 8, maxRequestImages: 4, maxLiveImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }, + ['maxLiveImages 8 is above maxRequestImages 4', 'maxLiveImageBytes 3 MiB is above maxRequestImageBytes 2 MiB']], + // A byte figure that isn't a whole number of MiB reads in bytes. + [{ maxLiveImageBytes: 3 * MiB + 1, maxRequestImageBytes: 2 * MiB }, [`maxLiveImageBytes ${3 * MiB + 1} bytes is above maxRequestImageBytes 2 MiB`]], + [{ maxRequestImages: Number.NaN }, ["maxRequestImages NaN isn't 0 or a finite number above it, so the default 500 applies"]], + [{ maxRequestImageBytes: -1 }, ["maxRequestImageBytes -1 isn't 0 or a finite number above it, so the default 24 MiB applies"]], + [{ maxLiveImages: 0, maxLiveImageBytes: 0 }, null], + [{ maxLiveImages: 4, maxRequestImages: 4 }, null], + // At or above the memory strategy's 12 MiB compression budget, so + // that budget isn't above the ceiling either. + [{ maxLiveImageBytes: 16 * MiB, maxRequestImageBytes: 16 * MiB }, null], + // A default above a lowered ceiling is named too, marked as the + // default, and so is a default ceiling under a limit the host set. + [{ maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 600 }, ['maxLiveImages 600 is above maxRequestImages 500 (default), so the live window keeps at most 500']], + // The default window of 6 comes from the strategy's default config, + // not from the host, so it's marked too. + [{ maxRequestImages: 4 }, ['maxLiveImages 6 (default) is above maxRequestImages 4, so the live window keeps at most 4']], + // An infinite limit is none, like 0, so nothing is said. + [{ maxLiveImages: Infinity }, null], + [{ maxLiveImages: 8, maxRequestImages: 0 }, null], + [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: 0 }, null], + ] as const; + for (const [index, [options, said]] of cases.entries()) { + lines.length = 0; + const strategy = make(options); + // Two stores opened in turn with the one strategy initialize it twice. + for (const store of ['first', 'second']) { + const manager = await ContextManager.open({ path: join(dir, `${index}-${store}`), strategy }); + await manager.close(); + } + const ours = lines.filter((line) => line.includes('above') || line.includes('the default')); + if (said === null) assert.deepEqual(ours, [], `nothing said for ${JSON.stringify(options)}`); + else { + assert.equal(ours.length, 1, `said once, in one line, for ${JSON.stringify(options)}: ${JSON.stringify(ours)}`); + for (const part of said) assert.ok(ours[0]!.includes(part), `${ours[0]} names ${part}`); + } + } + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); + } +}); + +class Prepared extends AutobiographicalStrategy { + prepare(request: NormalizedRequest) { return this.prepareCompressionRequest(request); } + send(request: NormalizedRequest, label: string): NormalizedRequest { + return (this as unknown as { prepareCompressionCandidate(r: NormalizedRequest, l: string): NormalizedRequest }) + .prepareCompressionCandidate(request, label); + } +} +function captureErrors(run: () => T): string[] { + const logs: string[] = []; + const original = console.error; + console.error = (...parts: unknown[]) => { logs.push(parts.map(String).join(' ')); }; + try { run(); return logs; } finally { console.error = original; } +} +const imageBlock = (data: string) => ({ type: 'image', source: { type: 'base64', mediaType: 'image/png', data } }) as ContentBlock; +const request = (...messages: ContentBlock[][]): NormalizedRequest => ({ + messages: messages.map((content) => ({ participant: 'User', content })), config: { model: 'mock', maxTokens: 1000 }, +}); + +describe("the memory strategy's warning names a compression budget above the byte ceiling", () => { + it('says it once, at initialize', async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: memory({ maxCompressionImageBytes: 30 * MiB }) }); + await manager.close(); + const ours = lines.filter((line) => line.includes('maxCompressionImageBytes')); + assert.equal(ours.length, 1, JSON.stringify(lines)); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 30 MiB is above maxRequestImageBytes 24 MiB (default), so a compression prompt keeps at most 24 MiB'), ours[0]); + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('compression prompts within the request ceilings', () => { + it('carry at most maxRequestImages, the newest messages first', () => { + const prepared = new Prepared({ maxRequestImages: 2 }).prepare( + request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), + ); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imageCountCap, 2); + assert.equal(prepared.imagesKept, 2); + assert.deepEqual(prepared.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image']); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER); + }); + + it('count an image nested in a tool result toward the ceiling', () => { + const prepared = new Prepared({ maxRequestImages: 1 }).prepare(request( + [imageBlock('AAAA')], + [{ type: 'tool_result', toolUseId: 't', content: [imageBlock('BBBB')] } as ContentBlock], + )); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imagesKept, 1); + const nested = (prepared.request.messages[1]!.content[0] as { content: ContentBlock[] }).content[0]!; + assert.equal(nested.type, 'image', 'the newer, nested image takes the one slot'); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER, 'and the older one is replaced'); + }); + + it('keep their byte budget within maxRequestImageBytes, and the images kept are what it lets through', () => { + const prepared = new Prepared({ maxCompressionImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }).prepare(request([])); + assert.equal(prepared.imageBudgetBytes, 2 * MiB); + for (const none of [0, Infinity]) { + assert.equal(new Prepared({ maxCompressionImageBytes: none, maxRequestImageBytes: 8 }).prepare(request([])).imageBudgetBytes, 8, + `a budget of ${none} is none, so up to the byte ceiling`); + } + const walked = new Prepared({ maxCompressionImageBytes: 12, maxRequestImageBytes: 8 }).prepare( + request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), + ); + assert.equal(walked.imageBudgetBytes, 8); + assert.equal(walked.imageReplacements, 1, 'under the clamped 8 bytes, two 4-byte images fit and the oldest does not'); + assert.equal(walked.imageBytesKept, 8); + assert.equal(new Prepared({ maxCompressionImageBytes: 0, maxRequestImageBytes: 2 * MiB }).prepare(request([])).imageBudgetBytes, + 2 * MiB, 'an unlimited budget means up to the ceiling'); + assert.equal(new Prepared().prepare(request([])).imageBudgetBytes, 12 * MiB, 'the default budget is under the default ceiling'); + }); + + it('carry at most maxRequestImages with no byte budget, and say only the ceiling', () => { + const strategy = new Prepared({ maxCompressionImageBytes: 0, maxRequestImageBytes: 0, maxRequestImages: 1 }); + const input = request([imageBlock('AAAA')], [imageBlock('BBBB')]); + assert.equal(strategy.prepare(input).imageReplacements, 1); + assert.deepEqual(captureErrors(() => strategy.send(input, 'no-budget')), [ + '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + + '1-image request ceiling (kept 1 image(s), 0.0MiB, newest messages first) [no-budget]', + ]); + }); + + it('with no count ceiling, keep to the byte budget, and the log says only the budget', () => { + const strategy = new Prepared({ maxCompressionImageBytes: 8, maxRequestImages: 0 }); + const input = request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]); + const prepared = strategy.prepare(input); + assert.equal(prepared.imageCountCap, 0); + assert.equal(prepared.imageReplacements, 1, 'two 4-byte images fit the 8-byte budget, the oldest does not'); + const logs = captureErrors(() => strategy.send(input, 'no-count')); + assert.equal(logs.length, 1); + assert.ok(!logs[0]!.includes('request ceiling'), logs[0]); + assert.ok(logs[0]!.includes('inline image budget (kept 2 image(s)'), logs[0]); + }); + + it('with no budget and no ceilings, keep every image, more than the default ceiling of 500', () => { + const images = Array.from({ length: 501 }, () => [imageBlock('AAAA')]); + const prepared = new Prepared({ maxCompressionImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0 }).prepare( + request(...images), + ); + assert.equal(prepared.imageReplacements, 0); + }); +}); From 10984186a2e9fd80b422f2cb898df15bca3e729b Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 13:40:32 -0400 Subject: [PATCH 02/67] fix(images): say once when an image first falls past a request ceiling Under a window limit of 0, which meant no limit, or one above a ceiling, the window keeps the ceiling's worth, and an operator who set no limit had no line saying where one now applies: the initialize warning says nothing for 0, since "as many as fit" is what 0 means. - ceilingSetLimits says which of the window's limits a ceiling sets: a limit given as none or above its ceiling, where the ceiling is a limit. - Each strategy says once for each ceiling, the first time an image falls past it under such a limit, through the same route as the initialize warning: which ceiling, its value, and the window limit it held (ceilingBoundLine), marking defaults as the warning does. The memory strategy says it from its strip and from its hysteresis watermark, whose trims strip below the ceiling. - It changes no request: the strip keeps and replaces what it did. - maxRequestImages' and maxRequestImageBytes' docs and the changed fragment say so. Tests: for both strategies, the line once for a count of 0, one above the ceiling, the default window under a lowered ceiling and a byte budget of 0; nothing when the host's own limit strips, equals the ceiling, or nothing falls past it; and for the memory strategy, the line from its watermark under each ceiling. Co-Authored-By: Claude Opus 5.5 --- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 47 ++++++++++- src/strategies/autobiographical.ts | 31 +++++++- src/strategies/windowed-passthrough.ts | 15 ++++ src/types/strategy.ts | 8 +- test/request-image-ceilings.test.ts | 78 +++++++++++++++++++ 6 files changed, 172 insertions(+), 9 deletions(-) diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 7b6ac7d0..d40e2934 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,4 +1,4 @@ - The live window's image limits stay within the request ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A `maxLiveImages` or `maxLiveImageBytes` of 0, which meant unlimited, now means up to `maxRequestImages` or `maxRequestImageBytes`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. - A `maxCompressionImageBytes` that isn't a positive finite number (0, a negative, `NaN` or `Infinity`) now means up to `maxRequestImageBytes`. It had meant no budget, or for `NaN`, that every image was replaced. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time an image falls past that ceiling, which ceiling it was and what it held. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index f12b269d..4767cf23 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -94,6 +94,51 @@ export function resolveLiveImageLimits(configured: ConfiguredImageLimits): LiveI const MiB = 1024 * 1024; +/** A byte figure as the strategies' lines give it: in MiB when it's a whole + * number of them, and in bytes otherwise, so two figures that differ never + * read alike. */ +function showBytes(n: number): string { + return n % MiB === 0 ? `${n / MiB} MiB` : `${n} bytes`; +} + +/** + * Which of the window's limits a request ceiling sets: a limit the host gave + * as none (0, or anything that isn't a positive finite number) or above its + * ceiling, where that ceiling is itself a limit. The window then keeps the + * ceiling's worth. (With no ceiling the window keeps the limit as given, so + * the two never match there.) + */ +export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: boolean; bytes: boolean } { + const { target, ceiling } = resolveLiveImageLimits(configured); + return { + images: target.images === ceiling.images && asLimit(configured.maxLiveImages) !== ceiling.images, + bytes: target.bytes === ceiling.bytes && asLimit(configured.maxLiveImageBytes) !== ceiling.bytes, + }; +} + +/** + * The line a strategy says once for each ceiling, the first time an image + * meets the live window full to a ceiling that sets its limit (see + * ceilingSetLimits): which ceiling, at what value, and the window limit of 0 + * or above it that the ceiling held, so an operator who set no limit sees + * where one now applies. + */ +export function ceilingBoundLine( + strategy: string, + kind: 'images' | 'bytes', + configured: ConfiguredImageLimits, + defaulted: ReadonlySet, +): string { + const ceiling = resolveLiveImageLimits(configured).ceiling; + const [ceilingName, limitName, value, show] = kind === 'images' + ? ['maxRequestImages', 'maxLiveImages', configured.maxLiveImages, String] as const + : ['maxRequestImageBytes', 'maxLiveImageBytes', configured.maxLiveImageBytes, showBytes] as const; + const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); + const mark = (name: string) => (defaulted.has(name) ? ' (default)' : ''); + return `[${strategy}] the live window reached ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + + `which holds ${limitName} ${limit}${mark(limitName)} to it, so older images are now shown as placeholders`; +} + /** * The warning a strategy gives once, when it initializes, or null when there * is nothing to say: a ceiling configured as something other than 0 or a @@ -112,7 +157,7 @@ export function liveImageLimitsWarning( const said: string[] = []; const ceiling = resolveLiveImageLimits(configured).ceiling; const count = (n: number) => String(n); - const bytes = (n: number) => (n % MiB === 0 ? `${n / MiB} MiB` : `${n} bytes`); + const bytes = showBytes; const setting = (name: string, shown: string) => `${name} ${shown}${defaulted.has(name) ? ' (default)' : ''}`; if (!wellFormedCeiling(configured.maxRequestImages)) { said.push(`maxRequestImages ${configured.maxRequestImages} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 3d1ca229..a27eaee8 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -83,6 +83,8 @@ import { DEFAULT_MAX_REQUEST_IMAGE_BYTES, clampToCeiling, liveImageLimitsWarning, + ceilingBoundLine, + ceilingSetLimits, resolveLiveImageLimits, } from '../live-image-limits.js'; import type { MessageId } from '../types/message.js'; @@ -16075,6 +16077,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { .filter(({ idx }) => this.hasImageBlock(entries[idx].content)) .sort((a, b) => b.pos - a.pos); + const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); let keptImages = 0; let keptImageBytes = 0; for (const { idx, pos } of ordered) { @@ -16087,6 +16090,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const blockBytes = AutobiographicalStrategy.imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; + if (overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); + if (overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); if (belowMark || tooDeep || overCount || overBytes) { // Stats-neutral (2026-07-12): every budgeting site now tallies at // POST-STRIP prices (see postStripEstimates), so the bucket never @@ -16142,7 +16147,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const messages = store.getAll(); const w = this.imageWatermark(); // Walk newest-first with the given limits; return the sequence of the - // first message holding an image that does not fit (or undefined). + // first message holding an image that does not fit (or undefined), and + // which limits it passes. + let misfitBy = { images: false, bytes: false }; const firstMisfit = (depthCap: number, countCap: number, bytesCap: number): number | undefined => { let depth = 0, count = 0, bytes = 0; for (let i = messages.length - 1; i >= 0; i--) { @@ -16151,9 +16158,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const b of m.content) { if (b.type !== 'image') continue; const size = AutobiographicalStrategy.imageBlockBytes(b); - const misfit = (depthCap > 0 && depth > depthCap) || (countCap > 0 && count >= countCap) || - (bytesCap > 0 && bytes + size > bytesCap); - if (misfit) return m.sequence; + misfitBy = { + images: countCap > 0 && count >= countCap, + bytes: bytesCap > 0 && bytes + size > bytesCap, + }; + if ((depthCap > 0 && depth > depthCap) || misfitBy.images || misfitBy.bytes) return m.sequence; count++; bytes += size; } @@ -16162,6 +16171,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { return undefined; }; if (firstMisfit(stripDepth, maxLive, maxLiveBytes) === undefined) return; // no limit binds + const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); + if (misfitBy.images && ceilingSet.images) this.reportCeilingBoundOnce('images'); + if (misfitBy.bytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); const next = firstMisfit( stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, maxLive > 0 ? Math.max(0, Math.floor(maxLive * ratio)) || 1 : 0, @@ -16283,6 +16295,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { * ceiling (24 MiB), which leaves room under membrane's 28 MiB cap. */ protected static readonly DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; + /** Which ceilings have been said to strip the live window (once each per instance). */ + private readonly ceilingBoundReported = { images: false, bytes: false }; + + /** Say once that a request ceiling, rather than a limit the host set, has + * stripped an image from the live window (see ceilingSetLimits). */ + private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { + if (this.ceilingBoundReported[kind]) return; + this.ceilingBoundReported[kind] = true; + this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); + } + /** The image limits the host didn't give, whose values are defaults. */ private imageLimitsDefaulted(): ReadonlySet { const keys = ['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes', 'maxCompressionImageBytes'] as const; diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 9bf67486..acd71384 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -23,6 +23,8 @@ import { DEFAULT_MAX_REQUEST_IMAGES, DEFAULT_MAX_REQUEST_IMAGE_BYTES, liveImageLimitsWarning, + ceilingBoundLine, + ceilingSetLimits, resolveLiveImageLimits, } from '../live-image-limits.js'; @@ -158,6 +160,8 @@ export class WindowedPassthroughStrategy implements ContextStrategy { private readonly imageStripDepthTokens: number; /** Whether the image limits' warning has been said (once per instance). */ private imageLimitsWarningReported = false; + /** Which ceilings have been said to strip the live window (once each per instance). */ + private readonly ceilingBoundReported = { images: false, bytes: false }; private anchor: Sequence = 0; private store: JsStore | null = null; @@ -446,6 +450,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } + const ceilingSet = ceilingSetLimits(this.configuredImageLimits); let keptImages = 0; let keptImageBytes = 0; // Entries are in timeline order; the policy counts newest-first. @@ -458,6 +463,8 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const bytes = imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; + if (overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); + if (overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); if (tooDeep || overCount || overBytes) { return { type: 'text', text: IMAGE_PLACEHOLDER } as ContentBlock; } @@ -468,6 +475,14 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } + /** Say once that a request ceiling, rather than a limit the options set, has + * stripped an image from the live window (see ceilingSetLimits). */ + private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { + if (this.ceilingBoundReported[kind]) return; + this.ceilingBoundReported[kind] = true; + this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits, this.imageLimitsDefaulted)); + } + // -------------------------------------------------------------------------- // Cache markers (first-claim ≤ 3; this strategy uses ≤ 2) // -------------------------------------------------------------------------- diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 5462c318..c59e8c21 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1158,7 +1158,8 @@ export interface AutobiographicalConfig { * which nothing here counts, and for image blocks a host adds outside the * compile. 0 = no ceiling. A value that isn't 0 or a finite number above it * takes the default, and the strategy's warning at initialize names it. A - * fraction is floored. + * fraction is floored. Under a window limit of 0 or above this ceiling, the + * strategy says once, the first time an image falls past it. */ maxRequestImages?: number; @@ -1175,8 +1176,9 @@ export interface AutobiographicalConfig { * messages carry: their text, their JSON, and any documents or other media, * which nothing here counts. It doesn't follow `MEMBRANE_MAX_REQUEST_BYTES`, * so a host that changes membrane's cap sets this to match. 0 = no ceiling. - * A value that isn't 0 or a finite number above it takes the default, as - * for `maxRequestImages`. + * A value that isn't 0 or a finite number above it takes the default, and + * the strategy says once when an image first falls past it, as for + * `maxRequestImages`. */ maxRequestImageBytes?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index df611367..cd815042 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -319,6 +319,84 @@ describe("the memory strategy's warning names a compression budget above the byt }); }); +describe('the line when an image first falls past a request ceiling', () => { + /** Run `steps` (each adds images and compiles) on a fresh store, and return + * the ceiling lines said and the last compile's image pattern. */ + async function ceilingLines( + strategy: ContextStrategy, + steps: number[], + ): Promise<{ lines: string[]; last: string }> { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + let n = 0; + let last = ''; + try { + for (const add of steps) { + for (let i = 0; i < add; i++) manager.addMessage('user', imageMessage(`img-${++n}`)); + last = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + } + return { lines: lines.filter((line) => line.includes('the live window reached')), last }; + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`${name}: says once which ceiling an image fell past, and what it held`, async () => { + const cases = [ + [{ maxLiveImages: 0, maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 0 (no limit) to it'], + [{ maxLiveImages: 5, maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 5 to it'], + [{ maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 6 (default) to it'], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 8 }, + 'maxRequestImageBytes 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'], + ] as const; + for (const [options, said] of cases) { + // Three images, then two more: the ceiling strips at the first compile + // and again at the second, and it's said once. + const { lines } = await ceilingLines(make(options), [3, 2]); + assert.equal(lines.length, 1, `said once for ${JSON.stringify(options)}: ${JSON.stringify(lines)}`); + assert.ok(lines[0]!.includes(`the live window reached ${said}, so older images are now shown as placeholders`), lines[0]); + } + }); + + it(`${name}: says nothing when the host's own limit strips, or nothing falls past the ceiling`, async () => { + for (const [options, steps] of [ + [{ maxLiveImages: 2 }, [3, 2]], + [{ maxLiveImages: 2, maxRequestImages: 2 }, [3, 2]], + [{ maxLiveImages: 0, maxRequestImages: 2 }, [1, 1]], + ] as const) { + const { lines } = await ceilingLines(make(options), [...steps]); + assert.deepEqual(lines, [], `nothing said for ${JSON.stringify(options)}`); + } + }); + } + + it('the memory strategy says it when its hysteresis watermark trims against a ceiling', async () => { + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + [5], + ); + assert.equal(last, '...II', 'one trim to half the ceiling'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('the live window reached maxRequestImages 4, which holds maxLiveImages 0 (no limit) to it'), lines[0]); + }); + + it('the memory strategy says it when its hysteresis watermark trims against the byte ceiling', async () => { + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 16, imageStripHysteresisRatio: 0.5 }), + [5], + ); + assert.equal(last, '...II', 'one trim to half the ceiling'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('the live window reached maxRequestImageBytes 16 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); + }); +}); + describe('compression prompts within the request ceilings', () => { it('carry at most maxRequestImages, the newest messages first', () => { const prepared = new Prepared({ maxRequestImages: 2 }).prepare( From 52f0a186f43d27ed0f1c192cf311b2418b9e0ae0 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 14:08:18 -0400 Subject: [PATCH 03/67] fix(images): the ceiling line from sent renders only, its words and marks exact From a review of the round before it: - The ceiling line comes only from a render that's sent: a dry compile, a preview, an observer's reconstruction say nothing (_dryRunSelect), and the memory strategy's strip says it only when its selection asks (applyImageStripping's opts.sent, from selectAdaptive and selectHierarchical), so kv-unified's kept layout, rendered to compare against, doesn't. Windowed passthrough's dry render says nothing either. A preview's own ceiling no longer reads as the window's, or uses up the line. - Its words hold for a single image bigger than the byte ceiling: "an image fell past maxRequestImageBytes ..., so images past it are now shown as placeholders". - "(default)" marks only defaults in force: in the memory strategy, a value the library's default layer gave or no layer gave, and a value given as undefined or null where a default then applies, but not maxLiveImages, whose undefined reads as no limit; a value a subclass's enforced layer gives isn't a default. A malformed ceiling, which takes its default, is marked in both lines. - Docs: the byte ceiling's margin names images the window doesn't count, and maxRequestImageBytes' doc qualifies when the line is said. Tests: the line said at the first compile that strips, with the strategy's name and through the warning's route; once for each ceiling when both strip; nothing when a host's byte limit strips, from a dry compile, from a preview with its own ceiling, or from the watermark when the host's own limit binds; the adaptive selection's line; nullish and enforced settings' marks; the line's words for default and malformed ceilings and byte figures; compression budgets that are negative, NaN or fractional; the default compression budget's warning clause alone; and the returned requests of the compression walks. Co-Authored-By: Claude Opus 5.5 --- src/live-image-limits.ts | 33 +++- src/strategies/autobiographical.ts | 35 +++- src/strategies/windowed-passthrough.ts | 24 +-- src/types/strategy.ts | 13 +- test/request-image-ceilings.test.ts | 229 ++++++++++++++++++++++--- 5 files changed, 277 insertions(+), 57 deletions(-) diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 4767cf23..af316fca 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -101,6 +101,21 @@ function showBytes(n: number): string { return n % MiB === 0 ? `${n / MiB} MiB` : `${n} bytes`; } +/** + * The " (default)" a line puts after a setting whose value is a default: one + * the host didn't give (`defaulted`), or a ceiling it gave malformed, which + * takes its default (see asCeiling). + */ +function defaultMark( + name: string, + configured: ConfiguredImageLimits, + defaulted: ReadonlySet, +): string { + const malformedCeiling = (name === 'maxRequestImages' || name === 'maxRequestImageBytes') && + !wellFormedCeiling(configured[name]); + return defaulted.has(name) || malformedCeiling ? ' (default)' : ''; +} + /** * Which of the window's limits a request ceiling sets: a limit the host gave * as none (0, or anything that isn't a positive finite number) or above its @@ -118,10 +133,12 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b /** * The line a strategy says once for each ceiling, the first time an image - * meets the live window full to a ceiling that sets its limit (see - * ceilingSetLimits): which ceiling, at what value, and the window limit of 0 - * or above it that the ceiling held, so an operator who set no limit sees - * where one now applies. + * falls past a ceiling that sets the window's limit (see ceilingSetLimits): + * which ceiling, at what value, and the window limit of 0 or above it that + * the ceiling held, so an operator who set no limit sees where one now + * applies. An image falls past the count ceiling when the window already + * keeps the ceiling's worth, and past the byte ceiling when it doesn't fit + * with what's kept, a single image bigger than the ceiling included. */ export function ceilingBoundLine( strategy: string, @@ -134,9 +151,9 @@ export function ceilingBoundLine( ? ['maxRequestImages', 'maxLiveImages', configured.maxLiveImages, String] as const : ['maxRequestImageBytes', 'maxLiveImageBytes', configured.maxLiveImageBytes, showBytes] as const; const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); - const mark = (name: string) => (defaulted.has(name) ? ' (default)' : ''); - return `[${strategy}] the live window reached ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + - `which holds ${limitName} ${limit}${mark(limitName)} to it, so older images are now shown as placeholders`; + const mark = (name: string) => defaultMark(name, configured, defaulted); + return `[${strategy}] an image fell past ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + + `which holds ${limitName} ${limit}${mark(limitName)} to it, so images past it are now shown as placeholders`; } /** @@ -158,7 +175,7 @@ export function liveImageLimitsWarning( const ceiling = resolveLiveImageLimits(configured).ceiling; const count = (n: number) => String(n); const bytes = showBytes; - const setting = (name: string, shown: string) => `${name} ${shown}${defaulted.has(name) ? ' (default)' : ''}`; + const setting = (name: string, shown: string) => `${name} ${shown}${defaultMark(name, configured, defaulted)}`; if (!wellFormedCeiling(configured.maxRequestImages)) { said.push(`maxRequestImages ${configured.maxRequestImages} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); } diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index a27eaee8..c345a585 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -12839,7 +12839,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.postProcessToolEntries(merged); // Strip stale images BEFORE placing markers and committing stats, so both // describe the post-strip context the agent actually receives. - this.applyImageStripping(merged, store); + this.applyImageStripping(merged, store, { sent: true }); // The newest stored message is the triggering turn for this compile. A // structural repair may rewrite tool blocks, but it must never erase that @@ -15245,7 +15245,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.postProcessToolEntries(entries); // Strip stale images before committing stats so RenderStats.total reflects // the post-strip context (this path places no cache markers). - this.applyImageStripping(entries, store); + this.applyImageStripping(entries, store, { sent: true }); this.rsEnd(); return entries; } @@ -16043,8 +16043,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { * so the committed render stats (and the cache breakpoints) describe the * post-strip context. As it strips, it decrements the matching raw bucket of * the in-progress render stats by the reclaimed tokens, keeping - * `RenderStats.total` equal to the real rendered size. */ - protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView): void { + * `RenderStats.total` equal to the real rendered size. + * + * `opts.sent`: the render is a selection's own, so a ceiling it strips + * against says its line (reportCeilingBoundOnce). A layout rendered only to + * compare against, such as kv-unified's kept one, leaves it unset. */ + protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView, opts: { sent?: boolean } = {}): void { // Count and bytes within the request ceilings; 0 = unlimited. const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; const depthTokens = this.config.imageStripDepthTokens ?? 0; // 0 = no depth strip @@ -16090,8 +16094,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { const blockBytes = AutobiographicalStrategy.imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; - if (overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); - if (overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); + // Only the selections' own renders say the ceiling line (`sent`), not + // a layout rendered to compare against, such as kv-unified's kept one. + if (opts.sent && overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); + if (opts.sent && overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); if (belowMark || tooDeep || overCount || overBytes) { // Stats-neutral (2026-07-12): every budgeting site now tallies at // POST-STRIP prices (see postStripEstimates), so the bucket never @@ -16299,17 +16305,28 @@ export class AutobiographicalStrategy implements ResettableStrategy { private readonly ceilingBoundReported = { images: false, bytes: false }; /** Say once that a request ceiling, rather than a limit the host set, has - * stripped an image from the live window (see ceilingSetLimits). */ + * stripped an image from the live window (see ceilingSetLimits). A dry + * select, a preview or an observer's reconstruction says nothing and + * leaves the line for the first render that's sent. */ private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { + if (this._dryRunSelect) return; if (this.ceilingBoundReported[kind]) return; this.ceilingBoundReported[kind] = true; this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); } - /** The image limits the host didn't give, whose values are defaults. */ + /** The image limits whose values are defaults: those the library's default + * layer gave or no layer gave, and those the host gave as undefined or + * null where configuredImageLimits then applies a default. For + * maxLiveImages it applies none: an explicit undefined erases the default + * 6 and reads as no limit, which is the host's doing. */ private imageLimitsDefaulted(): ReadonlySet { const keys = ['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes', 'maxCompressionImageBytes'] as const; - return new Set(keys.filter((key) => this.configProvenance[key] !== 'caller' || this.config[key] == null)); + return new Set(keys.filter((key) => { + const source = this.configProvenance[key]; + if (source === undefined || source === 'library-default') return true; + return key !== 'maxLiveImages' && this.config[key] == null; + })); } /** The live-image limits as configured, with the defaults where unset. */ diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index acd71384..90e2d0a9 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -205,6 +205,14 @@ export class WindowedPassthroughStrategy implements ContextStrategy { return resolveLiveImageLimits(this.configuredImageLimits); } + /** Say once that a request ceiling, rather than a limit the options set, has + * stripped an image from the live window (see ceilingSetLimits). */ + private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { + if (this.ceilingBoundReported[kind]) return; + this.ceilingBoundReported[kind] = true; + this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits, this.imageLimitsDefaulted)); + } + async initialize(ctx: StrategyContext): Promise { this.store = ctx.store; this.ns = ctx.namespace; @@ -413,7 +421,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { }); } - this.applyImageStripping(entries, messages, store); + this.applyImageStripping(entries, messages, store, !opts?.dryRun); if (!opts?.dryRun) this.placeCacheMarkers(entries); return entries; } @@ -433,6 +441,8 @@ export class WindowedPassthroughStrategy implements ContextStrategy { entries: ContextEntry[], messages: StoredMessage[], store: MessageStoreView, + // Whether this render is sent: a preview's (dryRun) says no ceiling line. + sent: boolean, ): void { // Count and bytes within the request ceilings; 0 = unlimited. const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; @@ -463,8 +473,8 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const bytes = imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; - if (overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); - if (overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); + if (sent && overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); + if (sent && overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); if (tooDeep || overCount || overBytes) { return { type: 'text', text: IMAGE_PLACEHOLDER } as ContentBlock; } @@ -475,14 +485,6 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } - /** Say once that a request ceiling, rather than a limit the options set, has - * stripped an image from the live window (see ceilingSetLimits). */ - private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { - if (this.ceilingBoundReported[kind]) return; - this.ceilingBoundReported[kind] = true; - this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits, this.imageLimitsDefaulted)); - } - // -------------------------------------------------------------------------- // Cache markers (first-claim ≤ 3; this strategy uses ≤ 2) // -------------------------------------------------------------------------- diff --git a/src/types/strategy.ts b/src/types/strategy.ts index c59e8c21..d5361d6e 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1173,12 +1173,13 @@ export interface AutobiographicalConfig { * refuses, before sending it, a request whose serialized messages exceed * its byte cap, 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise. * Default 24 MiB, which leaves 4 MiB of that cap for everything else the - * messages carry: their text, their JSON, and any documents or other media, - * which nothing here counts. It doesn't follow `MEMBRANE_MAX_REQUEST_BYTES`, - * so a host that changes membrane's cap sets this to match. 0 = no ceiling. - * A value that isn't 0 or a finite number above it takes the default, and - * the strategy says once when an image first falls past it, as for - * `maxRequestImages`. + * messages carry: their text, their JSON, any documents or other media, and + * in the window any images it doesn't count, none of which this counts. It + * doesn't follow `MEMBRANE_MAX_REQUEST_BYTES`, so a host that changes + * membrane's cap sets this to match. 0 = no ceiling. + * A value that isn't 0 or a finite number above it takes the default. Under + * a window limit of 0 or above this ceiling, the strategy says once when an + * image first falls past it, as for `maxRequestImages`. */ maxRequestImageBytes?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index cd815042..faef9b32 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -13,6 +13,7 @@ import { join } from 'node:path'; import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; import { AutobiographicalStrategy, ContextManager, WindowedPassthroughStrategy } from '../src/index.js'; import type { ContextStrategy } from '../src/index.js'; +import { ceilingBoundLine } from '../src/live-image-limits.js'; const MiB = 1024 * 1024; const PLACEHOLDER = '[image not shown: this context keeps only some images]'; @@ -256,6 +257,12 @@ describe('the image limits warning', () => { [{ maxRequestImages: 4 }, ['maxLiveImages 6 (default) is above maxRequestImages 4, so the live window keeps at most 4']], // An infinite limit is none, like 0, so nothing is said. [{ maxLiveImages: Infinity }, null], + // A malformed ceiling takes its default, which the later clause marks. + [{ maxLiveImages: 600, maxRequestImages: Number.NaN }, + ["maxRequestImages NaN isn't 0 or a finite number above it, so the default 500 applies", + 'maxLiveImages 600 is above maxRequestImages 500 (default)']], + // A byte budget given as null takes its default, and is marked so. + [{ maxLiveImageBytes: null, maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB']], [{ maxLiveImages: 8, maxRequestImages: 0 }, null], [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: 0 }, null], ] as const; @@ -319,26 +326,57 @@ describe("the memory strategy's warning names a compression budget above the byt }); }); +describe("the memory strategy's warning names its default compression budget under a lowered byte ceiling", () => { + it('says it, marked as the default, with no clause for a window limit under the ceiling', async () => { + const lines: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ + path: join(dir, 'store'), strategy: memory({ maxLiveImageBytes: 1 * MiB, maxRequestImageBytes: 2 * MiB }), + }); + await manager.close(); + const ours = lines.filter((line) => line.includes('is above')); + assert.equal(ours.length, 1, JSON.stringify(lines)); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 12 MiB (default) is above maxRequestImageBytes 2 MiB, so a compression prompt keeps at most 2 MiB'), ours[0]); + assert.ok(!ours[0]!.includes('maxLiveImageBytes'), ours[0]); + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + describe('the line when an image first falls past a request ceiling', () => { - /** Run `steps` (each adds images and compiles) on a fresh store, and return - * the ceiling lines said and the last compile's image pattern. */ + /** Run `steps` on a fresh store: each adds images (`add`, of `bytes` each, + * 4 when not given) and compiles. Returns the ceiling lines each step said, + * in order (a new message can move the memory strategy's watermark, so a + * step's adds count with its compile), the pattern each compile kept, and + * the last pattern. Byte sizes are multiples of 4: the store keeps an + * image's data as the bytes its base64 decodes to, so other lengths + * come back changed. */ async function ceilingLines( strategy: ContextStrategy, - steps: number[], - ): Promise<{ lines: string[]; last: string }> { - const lines: string[] = []; + steps: Array, + ): Promise<{ lines: string[]; byStep: string[][]; patterns: string[]; last: string }> { + const said: string[] = []; const original = console.warn; - console.warn = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); let n = 0; - let last = ''; + const byStep: string[][] = []; + const patterns: string[] = []; try { - for (const add of steps) { - for (let i = 0; i < add; i++) manager.addMessage('user', imageMessage(`img-${++n}`)); - last = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); + for (const step of steps) { + const { add, bytes } = typeof step === 'number' ? { add: step, bytes: 4 } : step; + const before = said.length; + for (let i = 0; i < add; i++) manager.addMessage('user', imageMessage(`img-${++n}`, bytes)); + patterns.push(pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n)); + byStep.push(said.slice(before).filter((line) => line.includes('an image fell past'))); } - return { lines: lines.filter((line) => line.includes('the live window reached')), last }; + return { lines: byStep.flat(), byStep, patterns, last: patterns[patterns.length - 1] ?? '' }; } finally { console.warn = original; await manager.close(); @@ -357,23 +395,62 @@ describe('the line when an image first falls past a request ceiling', () => { ] as const; for (const [options, said] of cases) { // Three images, then two more: the ceiling strips at the first compile - // and again at the second, and it's said once. - const { lines } = await ceilingLines(make(options), [3, 2]); - assert.equal(lines.length, 1, `said once for ${JSON.stringify(options)}: ${JSON.stringify(lines)}`); - assert.ok(lines[0]!.includes(`the live window reached ${said}, so older images are now shown as placeholders`), lines[0]); + // and again at the second, and it's said once, at the first. + const strategy = make(options); + const { lines, byStep, patterns } = await ceilingLines(strategy, [3, 2]); + assert.deepEqual(patterns, ['.II', '...II'], `the ceiling of 2 strips at each compile for ${JSON.stringify(options)}`); + assert.deepEqual(byStep.map((step) => step.length), [1, 0], `said once, at the first step, for ${JSON.stringify(options)}`); + assert.ok(lines[0]!.startsWith(`[${strategy.name}] an image fell past ${said}, so images past it are now shown as placeholders`), lines[0]); + assert.match(lines[0]!, / ns=/, 'through the namespaced route the warning takes'); } }); it(`${name}: says nothing when the host's own limit strips, or nothing falls past the ceiling`, async () => { - for (const [options, steps] of [ - [{ maxLiveImages: 2 }, [3, 2]], - [{ maxLiveImages: 2, maxRequestImages: 2 }, [3, 2]], - [{ maxLiveImages: 0, maxRequestImages: 2 }, [1, 1]], + for (const [options, steps, kept] of [ + [{ maxLiveImages: 2 }, [3, 2], '...II'], + [{ maxLiveImages: 2, maxRequestImages: 2 }, [3, 2], '...II'], + [{ maxLiveImages: 0, maxRequestImages: 2 }, [1, 1], 'II'], + // The host's own byte limit strips, under the default byte ceiling. + [{ maxLiveImageBytes: 8 }, [3, 2], '...II'], + [{ maxLiveImageBytes: 8, maxRequestImageBytes: 8 }, [3, 2], '...II'], ] as const) { - const { lines } = await ceilingLines(make(options), [...steps]); + const { lines, last } = await ceilingLines(make(options), [...steps]); + assert.equal(last, kept, `the strip ran for ${JSON.stringify(options)}`); assert.deepEqual(lines, [], `nothing said for ${JSON.stringify(options)}`); } }); + + it(`${name}: says it once for each ceiling, when each first strips`, async () => { + // Four-byte images fill the count ceiling of 3 first, at 12 of the 16 + // bytes; then a 16-byte image fills the byte ceiling, and the next one + // falls past it. + const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 3, maxRequestImageBytes: 16 }); + const { byStep } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); + assert.equal(byStep[0]!.length, 1); + assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImages 3'), byStep[0]![0]); + assert.equal(byStep.flat().filter((line) => line.includes('maxRequestImageBytes')).length, 1, JSON.stringify(byStep)); + assert.equal(byStep.flat().length, 2, JSON.stringify(byStep)); + }); + + it(`${name}: says nothing from a preview, and leaves the line for the first compile that's sent`, async () => { + const strategy = make({ maxLiveImages: 0, maxRequestImages: 2 }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }, undefined, { dryRun: true }); + assert.deepEqual(said.filter((line) => line.includes('an image fell past')), [], 'a dry compile says nothing'); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(said.filter((line) => line.includes('an image fell past')).length, 1, 'the sent compile says it'); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); } it('the memory strategy says it when its hysteresis watermark trims against a ceiling', async () => { @@ -383,7 +460,108 @@ describe('the line when an image first falls past a request ceiling', () => { ); assert.equal(last, '...II', 'one trim to half the ceiling'); assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('the live window reached maxRequestImages 4, which holds maxLiveImages 0 (no limit) to it'), lines[0]); + assert.ok(lines[0]!.includes('an image fell past maxRequestImages 4, which holds maxLiveImages 0 (no limit) to it'), lines[0]); + }); + + it('the memory strategy says it from its adaptive selection too, and nothing from a preview', async () => { + const adaptive = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 2, + }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: adaptive }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; + assert.ok(entries.length > 0, 'the preview rendered its entries'); + assert.deepEqual(ours(), [], 'a preview, with its own ceiling, says nothing'); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), '.II'); + assert.equal(ours().length, 1, JSON.stringify(said)); + assert.ok(ours()[0]!.includes('an image fell past maxRequestImages 2, which holds maxLiveImages 0 (no limit) to it'), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's strip says the line only for a render that's sent, not a layout rendered to compare against", async () => { + const strategy = memory({ maxLiveImages: 0, maxRequestImages: 1 }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 2; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const view = (manager as unknown as { messageStore: { createView(): unknown } }).messageStore.createView() as { + getAll(): Array<{ id: string; participant: string; content: ContentBlock[] }>; + }; + const entries = () => view.getAll().map((m, index) => ({ + index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content, + })); + const strip = strategy as unknown as { + applyImageStripping(entries: unknown[], store: unknown, opts?: { sent?: boolean }): void; + }; + const kept = entries(); + strip.applyImageStripping(kept, view); + assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling of 1 replaced one'); + assert.deepEqual(ours(), [], 'a layout rendered to compare against says nothing'); + strip.applyImageStripping(entries(), view, { sent: true }); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's watermark says nothing when the host's own limit binds", async () => { + const { lines, last } = await ceilingLines(memory({ maxLiveImages: 4, imageStripHysteresisRatio: 0.5 }), [5]); + assert.equal(last, '...II', 'one trim to half the host limit'); + assert.deepEqual(lines, []); + }); + + it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { + const auto = await ceilingLines(memory({ maxLiveImages: undefined, maxRequestImages: 2 }), [3]); + assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); + assert.ok(auto.lines[0]!.includes('which holds maxLiveImages 0 (no limit) to it,'), auto.lines[0]); + const win = await ceilingLines(windowed({ maxLiveImages: undefined, maxRequestImages: 2 }), [3]); + assert.equal(win.lines.length, 1, JSON.stringify(win.lines)); + assert.ok(win.lines[0]!.includes('which holds maxLiveImages 6 (default) to it,'), win.lines[0]); + }); + + it("doesn't mark a value a subclass's enforced layer gives as a default", async () => { + class Enforced extends AutobiographicalStrategy { + protected override enforcedConfigLayer() { + return { source: 'enforced', values: { maxRequestImages: 2 } }; + } + } + const { lines } = await ceilingLines( + new Enforced({ headWindowTokens: 0, recentWindowTokens: 1_000_000, hierarchical: true, imageStripDepthTokens: 0, maxLiveImages: 0 } as ConstructorParameters[0]), + [3], + ); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('an image fell past maxRequestImages 2, which holds'), lines[0]); + }); + + it('ceilingBoundLine marks a default or malformed ceiling, and gives bytes in MiB', () => { + const configured = (o: Record) => + ({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 500, maxRequestImageBytes: 24 * MiB, ...o }); + assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['maxRequestImages'])), + '[s] an image fell past maxRequestImages 500 (default), which holds maxLiveImages 0 (no limit) to it, so images past it are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'images', configured({ maxRequestImages: Number.NaN, maxLiveImages: 800 }), new Set()), + '[s] an image fell past maxRequestImages 500 (default), which holds maxLiveImages 800 to it, so images past it are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB }), new Set()), + '[s] an image fell past maxRequestImageBytes 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so images past it are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['maxRequestImageBytes'])), + `[s] an image fell past maxRequestImageBytes 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so images past it are now shown as placeholders`); }); it('the memory strategy says it when its hysteresis watermark trims against the byte ceiling', async () => { @@ -393,7 +571,7 @@ describe('the line when an image first falls past a request ceiling', () => { ); assert.equal(last, '...II', 'one trim to half the ceiling'); assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('the live window reached maxRequestImageBytes 16 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); + assert.ok(lines[0]!.includes('an image fell past maxRequestImageBytes 16 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); }); }); @@ -424,16 +602,19 @@ describe('compression prompts within the request ceilings', () => { it('keep their byte budget within maxRequestImageBytes, and the images kept are what it lets through', () => { const prepared = new Prepared({ maxCompressionImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }).prepare(request([])); assert.equal(prepared.imageBudgetBytes, 2 * MiB); - for (const none of [0, Infinity]) { + for (const none of [0, Infinity, -3, Number.NaN]) { assert.equal(new Prepared({ maxCompressionImageBytes: none, maxRequestImageBytes: 8 }).prepare(request([])).imageBudgetBytes, 8, `a budget of ${none} is none, so up to the byte ceiling`); } + assert.equal(new Prepared({ maxCompressionImageBytes: 8.5, maxRequestImageBytes: 100 }).prepare(request([])).imageBudgetBytes, 8, + 'a fractional budget is floored'); const walked = new Prepared({ maxCompressionImageBytes: 12, maxRequestImageBytes: 8 }).prepare( request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), ); assert.equal(walked.imageBudgetBytes, 8); assert.equal(walked.imageReplacements, 1, 'under the clamped 8 bytes, two 4-byte images fit and the oldest does not'); assert.equal(walked.imageBytesKept, 8); + assert.deepEqual(walked.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image'], 'the oldest is the one replaced'); assert.equal(new Prepared({ maxCompressionImageBytes: 0, maxRequestImageBytes: 2 * MiB }).prepare(request([])).imageBudgetBytes, 2 * MiB, 'an unlimited budget means up to the ceiling'); assert.equal(new Prepared().prepare(request([])).imageBudgetBytes, 12 * MiB, 'the default budget is under the default ceiling'); @@ -455,6 +636,7 @@ describe('compression prompts within the request ceilings', () => { const prepared = strategy.prepare(input); assert.equal(prepared.imageCountCap, 0); assert.equal(prepared.imageReplacements, 1, 'two 4-byte images fit the 8-byte budget, the oldest does not'); + assert.deepEqual(prepared.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image']); const logs = captureErrors(() => strategy.send(input, 'no-count')); assert.equal(logs.length, 1); assert.ok(!logs[0]!.includes('request ceiling'), logs[0]); @@ -467,5 +649,6 @@ describe('compression prompts within the request ceilings', () => { request(...images), ); assert.equal(prepared.imageReplacements, 0); + assert.equal(prepared.request.messages.filter((m) => m.content[0]!.type === 'image').length, 501, 'all 501 are still images'); }); }); From 6287092e3392b89a95525247ab203ec4e0d2546f Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 15:05:56 -0400 Subject: [PATCH 04/67] fix(images): the memory strategy's ceiling line once a select returns its render From a review of the round before it: - The memory strategy reports the ceiling line once a live select returns its render (reportCeilingHits), not from inside the strip: a select refused after its strip, such as one whose structural repair drops the newest turn, says nothing, and the next select it returns says it. The selection's strip records what its ceilings stripped for that render (`sent`), and a hysteresis watermark moved by a ceiling outside a dry run, as on a new message, leaves it for the next render returned. - The accessor's doc says its figures count the images the window counts. Tests: a refused select says nothing and the next one says it; a preview's own ceiling moving the watermark leaves no line; a single image bigger than the byte ceiling; the dry compile, with what it kept, and the compare-only strip for the byte ceiling too; both ceilings each at its own step; and the warning's marks for an undefined byte budget and a malformed byte ceiling under a limit above it. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 64 ++++++++----- src/types/strategy.ts | 8 +- test/request-image-ceilings.test.ts | 135 ++++++++++++++++++++++------ 3 files changed, 158 insertions(+), 49 deletions(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index c345a585..c6d234a5 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -7115,12 +7115,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the next compile's stable prefix and fold diff are measured against // the request before it. const prevCacheKeys = this._prevCacheKeys; + let entries: ContextEntry[]; try { - return this.selectPath(store, budget, opts); + entries = this.selectPath(store, budget, opts); } catch (err) { this._prevCacheKeys = prevCacheKeys; throw err; } + // Only a render this select returns says the ceiling line: one it + // refused threw above, and a dry select never comes here. Every select + // path strips its render last (applyImageStripping, `sent`), so the + // hits are this render's. + this.reportCeilingHits(); + return entries; } // A dry run commits nothing, and that includes what a select leaves for // the next one and for observers: the cache-marker baseline the next @@ -16046,8 +16053,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `RenderStats.total` equal to the real rendered size. * * `opts.sent`: the render is a selection's own, so a ceiling it strips - * against says its line (reportCeilingBoundOnce). A layout rendered only to - * compare against, such as kv-unified's kept one, leaves it unset. */ + * against says its line once select returns it (reportCeilingHits). A + * layout rendered only to compare against, such as kv-unified's kept one, + * leaves it unset. */ protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView, opts: { sent?: boolean } = {}): void { // Count and bytes within the request ceilings; 0 = unlimited. const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; @@ -16082,6 +16090,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { .sort((a, b) => b.pos - a.pos); const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); + const hits = { images: false, bytes: false }; let keptImages = 0; let keptImageBytes = 0; for (const { idx, pos } of ordered) { @@ -16094,10 +16103,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const blockBytes = AutobiographicalStrategy.imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; - // Only the selections' own renders say the ceiling line (`sent`), not - // a layout rendered to compare against, such as kv-unified's kept one. - if (opts.sent && overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); - if (opts.sent && overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); + if (overCount && ceilingSet.images) hits.images = true; + if (overBytes && ceilingSet.bytes) hits.bytes = true; if (belowMark || tooDeep || overCount || overBytes) { // Stats-neutral (2026-07-12): every budgeting site now tallies at // POST-STRIP prices (see postStripEstimates), so the bucket never @@ -16112,6 +16119,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { return block; }); } + // A selection's own render (`sent`) leaves what its ceilings stripped for + // select to report once it returns that render; the last such strip is + // the render it returns. A layout rendered only to compare against, such + // as kv-unified's kept one, leaves nothing. + if (opts.sent) this._ceilingHitsThisRender = hits; } /** imageStripHysteresisRatio: images in messages at or before this @@ -16177,9 +16189,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { return undefined; }; if (firstMisfit(stripDepth, maxLive, maxLiveBytes) === undefined) return; // no limit binds - const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); - if (misfitBy.images && ceilingSet.images) this.reportCeilingBoundOnce('images'); - if (misfitBy.bytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); + // A ceiling that moves the watermark strips every later render by it, so + // the next render a select returns reports it (reportCeilingHits). A dry + // advance is undone, so it leaves nothing. + if (!this._dryRunSelect) { + const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); + if (misfitBy.images && ceilingSet.images) this._ceilingMovedWatermark.images = true; + if (misfitBy.bytes && ceilingSet.bytes) this._ceilingMovedWatermark.bytes = true; + } const next = firstMisfit( stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, maxLive > 0 ? Math.max(0, Math.floor(maxLive * ratio)) || 1 : 0, @@ -16303,16 +16320,23 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Which ceilings have been said to strip the live window (once each per instance). */ private readonly ceilingBoundReported = { images: false, bytes: false }; - - /** Say once that a request ceiling, rather than a limit the host set, has - * stripped an image from the live window (see ceilingSetLimits). A dry - * select, a preview or an observer's reconstruction says nothing and - * leaves the line for the first render that's sent. */ - private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { - if (this._dryRunSelect) return; - if (this.ceilingBoundReported[kind]) return; - this.ceilingBoundReported[kind] = true; - this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); + /** The ceilings the last selection's render stripped against (applyImageStripping, `sent`). */ + private _ceilingHitsThisRender = { images: false, bytes: false }; + /** The ceilings that have moved the hysteresis watermark, outside a dry run. */ + private readonly _ceilingMovedWatermark = { images: false, bytes: false }; + + /** Say once for each ceiling, when a select returns a render that a + * request ceiling, rather than a limit the host set, stripped (see + * ceilingSetLimits), directly or through the watermark it moved. A dry + * select, a preview, an observer's reconstruction or a refused select + * says nothing, and the line waits for the first render returned. */ + private reportCeilingHits(): void { + for (const kind of ['images', 'bytes'] as const) { + const hit = this._ceilingHitsThisRender[kind] || this._ceilingMovedWatermark[kind]; + if (!hit || this.ceilingBoundReported[kind]) continue; + this.ceilingBoundReported[kind] = true; + this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); + } } /** The image limits whose values are defaults: those the library's default diff --git a/src/types/strategy.ts b/src/types/strategy.ts index d5361d6e..4eee1a1e 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -241,9 +241,11 @@ export interface ContextStrategy { * a caller that appends to a compiled request without recompiling, such * as a turn's continuation rounds: the strategy's compile keeps the images * its window counts within `target`, and the caller can let the request - * grow to `ceiling` before compiling it again. It only reports the figures: - * a caller bounds what it appends, and what it injects at compile time. - * Absent on a strategy with no live-image window. + * grow to `ceiling` before compiling it again. Both count the images the + * window counts (see `AutobiographicalConfig.maxLiveImages`), not every + * image a request can carry. It only reports the figures: a caller bounds + * what it appends, and what it injects at compile time. Absent on a + * strategy with no live-image window. */ liveImageLimits?(): LiveImageLimits; } diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index faef9b32..ea94ed74 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -263,6 +263,10 @@ describe('the image limits warning', () => { 'maxLiveImages 600 is above maxRequestImages 500 (default)']], // A byte budget given as null takes its default, and is marked so. [{ maxLiveImageBytes: null, maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB']], + [{ maxLiveImageBytes: undefined, maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB']], + [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: -1 }, + ["maxRequestImageBytes -1 isn't 0 or a finite number above it, so the default 24 MiB applies", + 'maxLiveImageBytes 30 MiB is above maxRequestImageBytes 24 MiB (default)']], [{ maxLiveImages: 8, maxRequestImages: 0 }, null], [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: 0 }, null], ] as const; @@ -425,30 +429,45 @@ describe('the line when an image first falls past a request ceiling', () => { // bytes; then a 16-byte image fills the byte ceiling, and the next one // falls past it. const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 3, maxRequestImageBytes: 16 }); - const { byStep } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); - assert.equal(byStep[0]!.length, 1); + const { byStep, patterns } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); + assert.deepEqual(patterns, ['.III', '....I', '.....I'], 'the count, then the bytes, strip'); + assert.deepEqual(byStep.map((step) => step.length), [1, 1, 0], JSON.stringify(byStep)); assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImages 3'), byStep[0]![0]); - assert.equal(byStep.flat().filter((line) => line.includes('maxRequestImageBytes')).length, 1, JSON.stringify(byStep)); - assert.equal(byStep.flat().length, 2, JSON.stringify(byStep)); + assert.ok(byStep[1]![0]!.includes('an image fell past maxRequestImageBytes 16 bytes'), byStep[1]![0]); }); - it(`${name}: says nothing from a preview, and leaves the line for the first compile that's sent`, async () => { - const strategy = make({ maxLiveImages: 0, maxRequestImages: 2 }); - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - try { - for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }, undefined, { dryRun: true }); - assert.deepEqual(said.filter((line) => line.includes('an image fell past')), [], 'a dry compile says nothing'); - await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - assert.equal(said.filter((line) => line.includes('an image fell past')).length, 1, 'the sent compile says it'); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); + it(`${name}: says it for a single image bigger than the byte ceiling`, async () => { + const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, maxRequestImageBytes: 8 }), [{ add: 1, bytes: 12 }]); + assert.deepEqual(patterns, ['.'], 'the one image is past the ceiling on its own'); + assert.equal(byStep[0]!.length, 1, JSON.stringify(byStep)); + assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImageBytes 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), byStep[0]![0]); + }); + + it(`${name}: says nothing from a dry compile, and leaves the line for the first compile that's sent`, async () => { + for (const [options, said0] of [ + [{ maxLiveImages: 0, maxRequestImages: 2 }, 'an image fell past maxRequestImages 2'], + [{ maxLiveImageBytes: 0, maxRequestImageBytes: 8 }, 'an image fell past maxRequestImageBytes 8 bytes'], + ] as const) { + const strategy = make(options); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const dry = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }, undefined, { dryRun: true }); + assert.equal(pattern(dry.messages, 3), '.II', `the dry compile strips for ${JSON.stringify(options)}`); + assert.deepEqual(ours(), [], `a dry compile says nothing for ${JSON.stringify(options)}`); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, `the sent compile says it for ${JSON.stringify(options)}`); + assert.ok(ours()[0]!.includes(said0), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } } }); } @@ -477,7 +496,7 @@ describe('the line when an image first falls past a request ceiling', () => { try { for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; - assert.ok(entries.length > 0, 'the preview rendered its entries'); + assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); assert.deepEqual(ours(), [], 'a preview, with its own ceiling, says nothing'); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 3), '.II'); @@ -490,8 +509,12 @@ describe('the line when an image first falls past a request ceiling', () => { } }); - it("the memory strategy's strip says the line only for a render that's sent, not a layout rendered to compare against", async () => { - const strategy = memory({ maxLiveImages: 0, maxRequestImages: 1 }); + it("the memory strategy's strip leaves the line only from a render that's sent, not a layout rendered to compare against", async () => { + for (const options of [ + { maxLiveImages: 0, maxRequestImages: 1 }, + { maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 4 }, + ]) { + const strategy = memory(options); const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -508,18 +531,78 @@ describe('the line when an image first falls past a request ceiling', () => { })); const strip = strategy as unknown as { applyImageStripping(entries: unknown[], store: unknown, opts?: { sent?: boolean }): void; + reportCeilingHits(): void; }; const kept = entries(); strip.applyImageStripping(kept, view); - assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling of 1 replaced one'); - assert.deepEqual(ours(), [], 'a layout rendered to compare against says nothing'); + assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling replaced one'); + strip.reportCeilingHits(); + assert.deepEqual(ours(), [], `a layout rendered to compare against leaves nothing for ${JSON.stringify(options)}`); strip.applyImageStripping(entries(), view, { sent: true }); + assert.deepEqual(ours(), [], 'the strip itself says nothing: select reports once it returns the render'); + strip.reportCeilingHits(); assert.equal(ours().length, 1, JSON.stringify(said)); } finally { console.warn = original; await manager.close(); rmSync(dir, { recursive: true, force: true }); } + } + }); + + it('the memory strategy says nothing for a select it refuses, and says it for the next one it returns', async () => { + const adaptive = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 2, + }); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: adaptive }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + // The newest turn is a call with no result yet: structural repair drops + // it, after the strip, and the select refuses. + manager.addMessage('Claude', [{ type: 'tool_use', id: 'tu-1', name: 'shot', input: {} }] as ContentBlock[]); + await assert.rejects(manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })); + assert.deepEqual(ours(), [], 'a refused select says nothing'); + manager.addMessage('user', [{ type: 'tool_result', toolUseId: 'tu-1', content: 'done' }] as ContentBlock[]); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's watermark leaves the line for the next compile, and a preview's watermark leaves nothing", async () => { + // A new message can move the watermark; the line waits for a compile. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 10, imageStripHysteresisRatio: 0.5, + }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + // A preview with a ceiling of 1 moves the watermark only for itself. + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; + assert.ok(entries.length > 0, 'the preview rendered its entries'); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), 'III', 'the real ceiling of 10 strips nothing'); + assert.deepEqual(ours(), [], "the preview's hypothetical ceiling left no line"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } }); it("the memory strategy's watermark says nothing when the host's own limit binds", async () => { From b28bddeaa2ab8d9ce9bc4dda6d88d3e45932b1c4 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 15:25:52 -0400 Subject: [PATCH 05/67] fix(images): each select's own ceiling hits, and a watermark's move kept with its branch From a review of the round before it: - The memory strategy's live select clears what earlier strips recorded before its own strips run. A dry select's strips record too, and a strip with no limits returns before recording, so a host with every limit off heard the line at its next compile after a preview with a ceiling of its own. - A ceiling's move of the hysteresis watermark is cleared with the watermark itself when a branch loads (clearBranchMirrors), so a move on one branch isn't reported at the first render returned on another. reportCeilingHits' doc says the move it reports is this instance's, on this branch: a watermark loaded from the store carries no record of what moved it. - The ceilings' docs, LiveImageLimits' and the changelog fragment say the window keeps the images it counts within the ceilings, and the 500 default's margin names what the window doesn't count. Tests: a preview at a ceiling of its own, then a compile with no limits, says nothing; a watermark a new message moves says nothing until a compile returns, which says it once; a watermark moved on a fork says nothing back on main; the preview's own watermark trim is checked; the refused select is matched by its error; and the step where both ceilings strip in turn expects the byte walk's greedy keep, as on main. Co-Authored-By: Claude Opus 5.5 --- changelog.d/request-image-ceilings.added.md | 2 +- src/live-image-limits.ts | 20 ++-- src/strategies/autobiographical.ts | 21 ++-- src/types/strategy.ts | 17 ++-- test/request-image-ceilings.test.ts | 101 ++++++++++++++++++-- 5 files changed, 129 insertions(+), 32 deletions(-) diff --git a/changelog.d/request-image-ceilings.added.md b/changelog.d/request-image-ceilings.added.md index 92e2e51f..33581345 100644 --- a/changelog.d/request-image-ceilings.added.md +++ b/changelog.d/request-image-ceilings.added.md @@ -1,2 +1,2 @@ -- `maxRequestImages` and `maxRequestImageBytes`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most inline image bytes, one request may carry. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `maxRequestImages` and `maxRequestImageBytes`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most inline image bytes, one request may carry. The live window keeps the images it counts within them: top-level image blocks, as `maxLiveImages` counts them, so images nested in tool results and generated images aren't bound there. A compression prompt counts those too. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. - `ContextStrategy.liveImageLimits()`, which both strategies implement, reports the live window's count and byte limits, kept within the ceilings, and the ceilings themselves. It's for a caller that appends to a compiled request without recompiling, such as a turn's continuation rounds, which can then keep its appends below the ceilings. It only reports them. `LiveImageLimits` is exported. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index af316fca..fa8ede5a 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -1,15 +1,17 @@ /** * The live-image window's limits, clamped to the request ceilings. * - * A strategy's live window keeps images newest-first up to a count - * (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The request - * ceilings (`maxRequestImages`, `maxRequestImageBytes`) are the most one - * request may carry at all, so the window's own limits never exceed them: - * an unlimited (0) target means "up to the ceiling", and a target above the - * ceiling keeps the ceiling's worth. A caller that appends to a compiled - * request without recompiling, such as a turn's continuation rounds, can - * read both through the strategy's `liveImageLimits()` and stop below the - * ceiling; nothing here bounds what it appends. + * A strategy's live window keeps the images it counts (top-level image + * blocks; see `AutobiographicalConfig.maxLiveImages`) newest-first up to a + * count (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The + * request ceilings (`maxRequestImages`, `maxRequestImageBytes`) are the most + * one request may carry, so the window's own limits never exceed them: an + * unlimited (0) target means "up to the ceiling", and a target above the + * ceiling keeps the ceiling's worth. What the window doesn't count, it + * doesn't bound. A caller that appends to a compiled request without + * recompiling, such as a turn's continuation rounds, can read both through + * the strategy's `liveImageLimits()` and stop below the ceiling; nothing + * here bounds what it appends. */ import type { LiveImageLimits } from './types/strategy.js'; diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index c6d234a5..b61435f3 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2560,6 +2560,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibrationProvisional = null; this._imageWatermark = undefined; this._imageWatermarkLoaded = false; + // A ceiling's move of the watermark belongs to the branch it moved on. + this._ceilingMovedWatermark.images = false; + this._ceilingMovedWatermark.bytes = false; this._lastKvStable = null; this._storeOrder = new Map(); this._storeOrderSource = []; @@ -7115,6 +7118,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the next compile's stable prefix and fold diff are measured against // the request before it. const prevCacheKeys = this._prevCacheKeys; + // What this select's strips record is what it reports. A dry select's + // strips, or a refused one's, record too, and are dropped here. + this._ceilingHitsThisRender = { images: false, bytes: false }; let entries: ContextEntry[]; try { entries = this.selectPath(store, budget, opts); @@ -7124,8 +7130,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { } // Only a render this select returns says the ceiling line: one it // refused threw above, and a dry select never comes here. Every select - // path strips its render last (applyImageStripping, `sent`), so the - // hits are this render's. + // path strips its render last (applyImageStripping, `sent`), and a + // strip with no limits records nothing, so the hits are this render's. this.reportCeilingHits(); return entries; } @@ -16322,14 +16328,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { private readonly ceilingBoundReported = { images: false, bytes: false }; /** The ceilings the last selection's render stripped against (applyImageStripping, `sent`). */ private _ceilingHitsThisRender = { images: false, bytes: false }; - /** The ceilings that have moved the hysteresis watermark, outside a dry run. */ + /** The ceilings that have moved the hysteresis watermark on this branch, outside a dry run. */ private readonly _ceilingMovedWatermark = { images: false, bytes: false }; /** Say once for each ceiling, when a select returns a render that a * request ceiling, rather than a limit the host set, stripped (see - * ceilingSetLimits), directly or through the watermark it moved. A dry - * select, a preview, an observer's reconstruction or a refused select - * says nothing, and the line waits for the first render returned. */ + * ceilingSetLimits), directly or through a watermark it moved on this + * branch while this instance ran. A watermark loaded from the store + * carries no record of what moved it, so a later instance says it when + * its own walk next meets the ceiling. A dry select, a preview, an + * observer's reconstruction or a refused select says nothing, and the + * line waits for the first render returned. */ private reportCeilingHits(): void { for (const kind of ['images', 'bytes'] as const) { const hit = this._ceilingHitsThisRender[kind] || this._ceilingMovedWatermark[kind]; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 4eee1a1e..ab2093f3 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -251,11 +251,13 @@ export interface ContextStrategy { } /** - * A strategy's live-image limits. Each figure is a count of image blocks or - * a sum of their inline base64 bytes, and 0 means no limit. `target` is - * what the window keeps, already within `ceiling` (`maxLiveImages` and + * A strategy's live-image limits. Each figure is a count of the image blocks + * the window counts (see `AutobiographicalConfig.maxLiveImages`) or a sum of + * their inline base64 bytes, and 0 means no limit. `target` is what the + * window keeps, already within `ceiling` (`maxLiveImages` and * `maxLiveImageBytes`, clamped); `ceiling` is the most one request may carry - * (`maxRequestImages` and `maxRequestImageBytes`). + * (`maxRequestImages` and `maxRequestImageBytes`), which the window keeps + * its own images within. */ export interface LiveImageLimits { target: { images: number; bytes: number }; @@ -1156,9 +1158,10 @@ export interface AutobiographicalConfig { * turn's continuation rounds, to keep its own appends below; nothing here * bounds what a caller appends or injects. The provider refuses a request * with more than 600 images and documents together (measured 2026-10-13, - * and again on 2026-10-19). Default 500, which leaves room for documents, - * which nothing here counts, and for image blocks a host adds outside the - * compile. 0 = no ceiling. A value that isn't 0 or a finite number above it + * and again on 2026-10-19). Default 500, which leaves room for what the + * window doesn't count: documents, images nested in tool results and + * generated images (see `maxLiveImages`), and image blocks a host adds + * outside the compile. 0 = no ceiling. A value that isn't 0 or a finite number above it * takes the default, and the strategy's warning at initialize names it. A * fraction is floored. Under a window limit of 0 or above this ceiling, the * strategy says once, the first time an image falls past it. diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index ea94ed74..3fd4f658 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -430,7 +430,9 @@ describe('the line when an image first falls past a request ceiling', () => { // falls past it. const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 3, maxRequestImageBytes: 16 }); const { byStep, patterns } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); - assert.deepEqual(patterns, ['.III', '....I', '.....I'], 'the count, then the bytes, strip'); + // At the third step the byte walk drops the 16-byte image and, being + // greedy, keeps two older 4-byte ones that still fit (as on main). + assert.deepEqual(patterns, ['.III', '....I', '..II.I'], 'the count, then the bytes, strip'); assert.deepEqual(byStep.map((step) => step.length), [1, 1, 0], JSON.stringify(byStep)); assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImages 3'), byStep[0]![0]); assert.ok(byStep[1]![0]!.includes('an image fell past maxRequestImageBytes 16 bytes'), byStep[1]![0]); @@ -564,9 +566,10 @@ describe('the line when an image first falls past a request ceiling', () => { try { for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); // The newest turn is a call with no result yet: structural repair drops - // it, after the strip, and the select refuses. + // it, and the check after the strip refuses the select. manager.addMessage('Claude', [{ type: 'tool_use', id: 'tu-1', name: 'shot', input: {} }] as ContentBlock[]); - await assert.rejects(manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })); + await assert.rejects(manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }), + { message: /^Structural repair did not retain the newest turn/ }); assert.deepEqual(ours(), [], 'a refused select says nothing'); manager.addMessage('user', [{ type: 'tool_result', toolUseId: 'tu-1', content: 'done' }] as ContentBlock[]); await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); @@ -578,8 +581,32 @@ describe('the line when an image first falls past a request ceiling', () => { } }); - it("the memory strategy's watermark leaves the line for the next compile, and a preview's watermark leaves nothing", async () => { - // A new message can move the watermark; the line waits for a compile. + it("the memory strategy's watermark, moved by a new message, says nothing until a compile returns", async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + // The fifth image's message moves the watermark past the ceiling of 4. + for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + assert.deepEqual(ours(), [], 'the new messages say nothing'); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 5), '...II', 'one trim to half the ceiling'); + assert.equal(ours().length, 1, JSON.stringify(said)); + assert.ok(ours()[0]!.includes('an image fell past maxRequestImages 4'), ours()[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy says nothing for a watermark a preview's own ceiling moved", async () => { const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -590,14 +617,70 @@ describe('the line when an image first falls past a request ceiling', () => { }); const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + // A preview with a ceiling of 2 moves the watermark, for itself only: + // its trim to half keeps one image, where the count alone keeps two. + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 2 }, { render: true })?.entries ?? []; + assert.equal(pattern(entries, 4), '...I', "the preview's own watermark trimmed to half its ceiling"); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 4), 'IIII', 'the real ceiling of 10 strips nothing'); + assert.deepEqual(ours(), [], "the preview's hypothetical ceiling left no line"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('the memory strategy says nothing for a host with no limits, after a preview with a ceiling of its own', async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + // Every limit off (the helper sets no depth strip), so the compile's strip has nothing to record. + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0, + }), + }); + const ours = () => said.filter((line) => line.includes('an image fell past')); try { for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - // A preview with a ceiling of 1 moves the watermark only for itself. const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; - assert.ok(entries.length > 0, 'the preview rendered its entries'); + assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - assert.equal(pattern(compiled.messages, 3), 'III', 'the real ceiling of 10 strips nothing'); - assert.deepEqual(ours(), [], "the preview's hypothetical ceiling left no line"); + assert.equal(pattern(compiled.messages, 3), 'III', 'with no limits, the compile keeps every image'); + assert.deepEqual(ours(), [], "the preview's strip isn't the compile's"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy leaves a watermark's line with the branch it moved on", async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + manager.addMessage('user', imageMessage('img-1')); + const main = manager.currentBranch().name; + await manager.fork('busy'); + // On the fork, new messages move the watermark past the ceiling of 4. + for (let i = 2; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + await manager.switchBranch(main); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 1), 'I', 'main holds one image, under its ceiling'); + assert.deepEqual(ours(), [], "main's compile says nothing for the fork's watermark"); } finally { console.warn = original; await manager.close(); From 87013f316707da8c2e6b8f9a6c7acb087013057b Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Mon, 19 Oct 2026 23:29:59 -0400 Subject: [PATCH 06/67] fix(images): name the ceilings for what they bound, and say the line only when a ceiling decides Answers review round 1 on #268: R1-1, R1-2 and R1-4 to R1-9. R1-1: maxRequestImages and maxRequestImageBytes become imageCeiling and imageByteCeiling. Each doc opens with what it bounds: the top-level image blocks a strategy's live window keeps, and what a compression prompt carries. Each then says what it doesn't count: the window's nested and generated images, and whatever a caller injects or appends. The module's constants follow (DEFAULT_IMAGE_CEILING, DEFAULT_IMAGE_BYTE_CEILING). R1-2: ContextStrategy.liveImageLimits() becomes imageCeilings(), which reports the two ceilings alone. The window's own target leaves the public API (ResolvedImageLimits, inside the module), and the doc says the ceilings are configuration, not usage: a caller that wants a request within them counts that request's images itself. R1-4: the once-only line is said only when a ceiling decides that an image is stripped (ceilingsDeciding), not when the depth limit, the hysteresis watermark, or a count or byte limit the host set strips it anyway. That holds in both strategies' strips and in the watermark's move. "sent" becomes "returned", since a strategy sees the render select returns, not whether it was sent. R1-5: a breaking fragment names the change in what 0 means for maxLiveImages, maxLiveImageBytes and maxCompressionImageBytes, with the migration, the siblings' ranges on npm, and what doesn't change. R1-6: both ceilings are in DEFAULT_AUTOBIOGRAPHICAL_CONFIG, so config:effective reports them. R1-7: windowed passthrough's applyImageStripping takes `returned` with a default of false, so a caller passing three arguments, as renderAsNewest does on main, stays correct and says nothing. R1-8: a compression test where a newer generated_image takes the last slot from an older image. R1-9: the note on membrane's byte cap says where it's checked: transformRequest, which a call's first request and each native tool-loop round go through but an XML-mode continuation round doesn't, and the Anthropic adapter, on every request it sends (membrane 0.5.87 and 0.6.0). Co-Authored-By: Claude Opus 5.5 --- changelog.d/request-image-ceilings.added.md | 4 +- .../request-image-ceilings.breaking.md | 4 + changelog.d/request-image-ceilings.changed.md | 5 +- changelog.d/request-image-ceilings.fixed.md | 2 +- src/index.ts | 2 +- src/live-image-limits.ts | 124 +++-- src/strategies/autobiographical.ts | 120 ++--- src/strategies/windowed-passthrough.ts | 67 +-- src/types/index.ts | 2 +- src/types/strategy.ts | 142 +++--- test/compression-request-preparation.test.ts | 2 +- test/image-stripping.test.ts | 6 +- test/request-image-ceilings.test.ts | 431 ++++++++++++------ 13 files changed, 554 insertions(+), 357 deletions(-) create mode 100644 changelog.d/request-image-ceilings.breaking.md diff --git a/changelog.d/request-image-ceilings.added.md b/changelog.d/request-image-ceilings.added.md index 33581345..5c7608ce 100644 --- a/changelog.d/request-image-ceilings.added.md +++ b/changelog.d/request-image-ceilings.added.md @@ -1,2 +1,2 @@ -- `maxRequestImages` and `maxRequestImageBytes`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most inline image bytes, one request may carry. The live window keeps the images it counts within them: top-level image blocks, as `maxLiveImages` counts them, so images nested in tool results and generated images aren't bound there. A compression prompt counts those too. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. -- `ContextStrategy.liveImageLimits()`, which both strategies implement, reports the live window's count and byte limits, kept within the ceilings, and the ceilings themselves. It's for a caller that appends to a compiled request without recompiling, such as a turn's continuation rounds, which can then keep its appends below the ceilings. It only reports them. `LiveImageLimits` is exported. +- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most top-level image blocks, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They don't bound a whole request: the window's images nested in tool results and its generated images, and anything a caller injects at compile time or appends to a compiled request, aren't counted; a compression prompt counts every image. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `ContextStrategy.imageCeilings()`, which both strategies implement, reports the two ceilings as the strategy applies them, a malformed one at its default. It reports configuration, not usage: a caller that wants a request within the ceilings counts that request's images itself. `ImageCeilings` is exported. diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md new file mode 100644 index 00000000..90ba4b76 --- /dev/null +++ b/changelog.d/request-image-ceilings.breaking.md @@ -0,0 +1,4 @@ +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative, `Infinity`, or `NaN`, which had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps the ceiling's worth, and its strategy says once, the first time a ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. + - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. + - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.1 depends on `^0.14.1`, and connectome-host 0.10.2 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. + - **Unchanged:** for a host that sets none of these, the default window (6 images and 20 MiB) and the default compression budget (12 MiB) are under the default ceilings, so what it keeps doesn't change. A limit above 0 and under its ceiling works as before. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index d40e2934..6c8e50dd 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,4 +1,3 @@ -- The live window's image limits stay within the request ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A `maxLiveImages` or `maxLiveImageBytes` of 0, which meant unlimited, now means up to `maxRequestImages` or `maxRequestImageBytes`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- A `maxCompressionImageBytes` that isn't a positive finite number (0, a negative, `NaN` or `Infinity`) now means up to `maxRequestImageBytes`. It had meant no budget, or for `NaN`, that every image was replaced. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time an image falls past that ceiling, which ceiling it was and what it held. +- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time that ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. Only a render a select returns says it, and not when the depth limit, the hysteresis watermark or a limit the host set strips the image anyway. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/changelog.d/request-image-ceilings.fixed.md b/changelog.d/request-image-ceilings.fixed.md index 98b5a304..83838a43 100644 --- a/changelog.d/request-image-ceilings.fixed.md +++ b/changelog.d/request-image-ceilings.fixed.md @@ -1 +1 @@ -- A compression prompt carries at most `maxRequestImages` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `maxRequestImageBytes`, and the line it logs when it replaces images names the count ceiling and the images kept. +- A compression prompt carries at most `imageCeiling` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `imageByteCeiling`, and the line it logs when it replaces images names the count ceiling and the images kept. diff --git a/src/index.ts b/src/index.ts index 53b42883..2472094a 100644 --- a/src/index.ts +++ b/src/index.ts @@ -106,7 +106,7 @@ export type { CompressionHoldOptions, CompressionHoldInfo, RenderedSummaryInfo, - LiveImageLimits, + ImageCeilings, } from './types/index.js'; // Accepted rounds and fold receipts diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index fa8ede5a..048836f6 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -1,38 +1,50 @@ /** - * The live-image window's limits, clamped to the request ceilings. + * The live-image window's limits, kept within the image ceilings. * * A strategy's live window keeps the images it counts (top-level image * blocks; see `AutobiographicalConfig.maxLiveImages`) newest-first up to a - * count (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The - * request ceilings (`maxRequestImages`, `maxRequestImageBytes`) are the most - * one request may carry, so the window's own limits never exceed them: an - * unlimited (0) target means "up to the ceiling", and a target above the - * ceiling keeps the ceiling's worth. What the window doesn't count, it - * doesn't bound. A caller that appends to a compiled request without - * recompiling, such as a turn's continuation rounds, can read both through - * the strategy's `liveImageLimits()` and stop below the ceiling; nothing - * here bounds what it appends. + * count (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The image + * ceilings (`imageCeiling`, `imageByteCeiling`) bound those limits: an + * unlimited (0) limit means "up to the ceiling", and a limit above the + * ceiling keeps the ceiling's worth. They bound what the window counts and + * what a compression prompt carries, not a whole request: images nested in + * tool results, generated images, and whatever a caller injects at compile + * time or appends to a compiled request aren't counted here. A caller that + * wants a request within the ceilings counts that request's images itself + * and compares them with the strategy's `imageCeilings()`. */ -import type { LiveImageLimits } from './types/strategy.js'; - /** * The provider refuses a request carrying more than 600 images and documents * together ("too many images and documents: 601 + 0 > 600", measured - * 2026-10-13 and again on 2026-10-19). 500 leaves room for documents, which - * nothing here counts, and for image blocks a host adds outside the compile. + * 2026-10-13 and again on 2026-10-19). 500 leaves room for what the ceilings + * don't count: documents, images nested in tool results, generated images, + * and images a caller injects or appends. */ -export const DEFAULT_MAX_REQUEST_IMAGES = 500; +export const DEFAULT_IMAGE_CEILING = 500; /** * membrane refuses, before sending it, a request whose serialized messages * exceed its byte cap: 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says - * otherwise (membrane 0.5.87 and 0.6.0, `assertWithinByteBudget`, checked at - * every round of a tool loop). 24 MiB of base64 images leaves 4 MiB of that - * default cap for everything else the messages carry: their text, their JSON, - * and any documents or other media. + * otherwise (`assertWithinByteBudget`, membrane 0.5.87 and 0.6.0). It checks + * when it builds a request through `transformRequest`, which a call's first + * request and each native tool-loop round go through; an XML-mode + * continuation round doesn't, and of the provider adapters only the + * Anthropic adapter checks again, on every request it sends. 24 MiB of base64 + * images leaves 4 MiB of that default cap for everything else the messages + * carry: their text, their JSON, and any documents or other media. */ -export const DEFAULT_MAX_REQUEST_IMAGE_BYTES = 24 * 1024 * 1024; +export const DEFAULT_IMAGE_BYTE_CEILING = 24 * 1024 * 1024; + +/** A strategy's image limits as its window applies them. Each figure is a + * count of the images the window counts or a sum of their inline base64 + * bytes, and 0 is no limit. `target` is what the window keeps, already + * within `ceiling` (`maxLiveImages` and `maxLiveImageBytes`, clamped); + * `ceiling` is `imageCeiling` and `imageByteCeiling`. */ +export interface ResolvedImageLimits { + target: { images: number; bytes: number }; + ceiling: { images: number; bytes: number }; +} /** A window limit as the window reads it: a whole number of 1 or more, or 0 * for none. Any value that isn't a positive finite number (0, a negative, @@ -75,15 +87,15 @@ export function clampToCeiling(target: number, ceiling: number): number { export interface ConfiguredImageLimits { maxLiveImages: number; maxLiveImageBytes: number; - maxRequestImages: number; - maxRequestImageBytes: number; + imageCeiling: number; + imageByteCeiling: number; } /** The limits a strategy's window applies, and the ceilings above them. */ -export function resolveLiveImageLimits(configured: ConfiguredImageLimits): LiveImageLimits { +export function resolveLiveImageLimits(configured: ConfiguredImageLimits): ResolvedImageLimits { const ceiling = { - images: asCeiling(configured.maxRequestImages, DEFAULT_MAX_REQUEST_IMAGES), - bytes: asCeiling(configured.maxRequestImageBytes, DEFAULT_MAX_REQUEST_IMAGE_BYTES), + images: asCeiling(configured.imageCeiling, DEFAULT_IMAGE_CEILING), + bytes: asCeiling(configured.imageByteCeiling, DEFAULT_IMAGE_BYTE_CEILING), }; return { target: { @@ -113,17 +125,17 @@ function defaultMark( configured: ConfiguredImageLimits, defaulted: ReadonlySet, ): string { - const malformedCeiling = (name === 'maxRequestImages' || name === 'maxRequestImageBytes') && + const malformedCeiling = (name === 'imageCeiling' || name === 'imageByteCeiling') && !wellFormedCeiling(configured[name]); return defaulted.has(name) || malformedCeiling ? ' (default)' : ''; } /** - * Which of the window's limits a request ceiling sets: a limit the host gave - * as none (0, or anything that isn't a positive finite number) or above its - * ceiling, where that ceiling is itself a limit. The window then keeps the - * ceiling's worth. (With no ceiling the window keeps the limit as given, so - * the two never match there.) + * Which of the window's limits a ceiling sets: a limit the host gave as none + * (0, or anything that isn't a positive finite number) or above its ceiling, + * where that ceiling is itself a limit. The window then keeps the ceiling's + * worth. (With no ceiling the window keeps the limit as given, so the two + * never match there.) */ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: boolean; bytes: boolean } { const { target, ceiling } = resolveLiveImageLimits(configured); @@ -134,13 +146,31 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b } /** - * The line a strategy says once for each ceiling, the first time an image - * falls past a ceiling that sets the window's limit (see ceilingSetLimits): - * which ceiling, at what value, and the window limit of 0 or above it that - * the ceiling held, so an operator who set no limit sees where one now - * applies. An image falls past the count ceiling when the window already - * keeps the ceiling's worth, and past the byte ceiling when it doesn't fit - * with what's kept, a single image bigger than the ceiling included. + * Which ceilings decide that an image is stripped: those whose limit it + * passes (`over`) where the ceiling sets the window's limit (`ceilingSet`, + * see ceilingSetLimits), when nothing else strips it. Something else strips + * it when `strippedOtherwise` says so, as the host's depth limit or the + * memory strategy's hysteresis watermark does, or when it also passes a + * count or byte limit the host set itself. Only a ceiling that decides + * changes what the window keeps, so only that one says its line. + */ +export function ceilingsDeciding( + over: { images: boolean; bytes: boolean }, + strippedOtherwise: boolean, + ceilingSet: { images: boolean; bytes: boolean }, +): { images: boolean; bytes: boolean } { + const otherwise = strippedOtherwise || (over.images && !ceilingSet.images) || (over.bytes && !ceilingSet.bytes); + return otherwise ? { images: false, bytes: false } : { images: over.images, bytes: over.bytes }; +} + +/** + * The line a strategy says once for each ceiling, the first time that + * ceiling decides that an image is stripped (see ceilingsDeciding): which + * ceiling, at what value, and the window limit of 0 or above it that the + * ceiling held, so an operator who set no limit sees where one now applies. + * An image passes the count ceiling when the window already keeps the + * ceiling's worth, and the byte ceiling when it doesn't fit with what's + * kept, a single image bigger than the ceiling included. */ export function ceilingBoundLine( strategy: string, @@ -150,8 +180,8 @@ export function ceilingBoundLine( ): string { const ceiling = resolveLiveImageLimits(configured).ceiling; const [ceilingName, limitName, value, show] = kind === 'images' - ? ['maxRequestImages', 'maxLiveImages', configured.maxLiveImages, String] as const - : ['maxRequestImageBytes', 'maxLiveImageBytes', configured.maxLiveImageBytes, showBytes] as const; + ? ['imageCeiling', 'maxLiveImages', configured.maxLiveImages, String] as const + : ['imageByteCeiling', 'maxLiveImageBytes', configured.maxLiveImageBytes, showBytes] as const; const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); const mark = (name: string) => defaultMark(name, configured, defaulted); return `[${strategy}] an image fell past ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + @@ -178,11 +208,11 @@ export function liveImageLimitsWarning( const count = (n: number) => String(n); const bytes = showBytes; const setting = (name: string, shown: string) => `${name} ${shown}${defaultMark(name, configured, defaulted)}`; - if (!wellFormedCeiling(configured.maxRequestImages)) { - said.push(`maxRequestImages ${configured.maxRequestImages} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); + if (!wellFormedCeiling(configured.imageCeiling)) { + said.push(`imageCeiling ${configured.imageCeiling} isn't 0 or a finite number above it, so the default ${count(ceiling.images)} applies`); } - if (!wellFormedCeiling(configured.maxRequestImageBytes)) { - said.push(`maxRequestImageBytes ${configured.maxRequestImageBytes} isn't 0 or a finite number above it, so the default ${bytes(ceiling.bytes)} applies`); + if (!wellFormedCeiling(configured.imageByteCeiling)) { + said.push(`imageByteCeiling ${configured.imageByteCeiling} isn't 0 or a finite number above it, so the default ${bytes(ceiling.bytes)} applies`); } const above = ( name: string, value: number | undefined, ceilingName: string, limit: number, keeps: string, show: (n: number) => string, @@ -192,8 +222,8 @@ export function liveImageLimitsWarning( said.push(`${setting(name, show(v))} is above ${setting(ceilingName, show(limit))}, so ${keeps} at most ${show(limit)}`); } }; - above('maxLiveImages', configured.maxLiveImages, 'maxRequestImages', ceiling.images, 'the live window keeps', count); - above('maxLiveImageBytes', configured.maxLiveImageBytes, 'maxRequestImageBytes', ceiling.bytes, 'the live window keeps', bytes); - above('maxCompressionImageBytes', configured.maxCompressionImageBytes, 'maxRequestImageBytes', ceiling.bytes, 'a compression prompt keeps', bytes); + above('maxLiveImages', configured.maxLiveImages, 'imageCeiling', ceiling.images, 'the live window keeps', count); + above('maxLiveImageBytes', configured.maxLiveImageBytes, 'imageByteCeiling', ceiling.bytes, 'the live window keeps', bytes); + above('maxCompressionImageBytes', configured.maxCompressionImageBytes, 'imageByteCeiling', ceiling.bytes, 'a compression prompt keeps', bytes); return said.length === 0 ? null : `[${strategy}] ${said.join('; ')}`; } diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b61435f3..631c44e1 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -5,7 +5,7 @@ import { registerStateIfAbsent } from '../state-registration.js'; import { phaseChannel } from '../phase-channel.js'; import type { ContextStrategy, - LiveImageLimits, + ImageCeilings, ResettableStrategy, StrategyContext, ReadinessState, @@ -79,13 +79,15 @@ import { releaseEditedRawForms, releaseSharedRawFormsOnMove, releaseTrailingReas import { isCompressionWithdrawn } from '../compression-withdrawn.js'; import { CAP_OMISSION_TEXT, capContent, cappedPriceExtra, cappedWhole, isCapMarker, isEmptyTextBlock } from './message-cap.js'; import { - DEFAULT_MAX_REQUEST_IMAGES, - DEFAULT_MAX_REQUEST_IMAGE_BYTES, + DEFAULT_IMAGE_CEILING, + DEFAULT_IMAGE_BYTE_CEILING, clampToCeiling, liveImageLimitsWarning, ceilingBoundLine, ceilingSetLimits, + ceilingsDeciding, resolveLiveImageLimits, + type ResolvedImageLimits, } from '../live-image-limits.js'; import type { MessageId } from '../types/message.js'; import type { IngressChunkResult } from '../types/strategy.js'; @@ -1169,7 +1171,7 @@ interface PreparationCounts { /** The inline image byte budget, 0 for none. */ imageBudgetBytes: number; imageBytesKept: number; - /** The most images the prompt may carry (maxRequestImages), 0 for none. */ + /** The most images the prompt may carry (imageCeiling), 0 for none. */ imageCountCap: number; imagesKept: number; } @@ -7130,7 +7132,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } // Only a render this select returns says the ceiling line: one it // refused threw above, and a dry select never comes here. Every select - // path strips its render last (applyImageStripping, `sent`), and a + // path strips its render last (applyImageStripping, `returned`), and a // strip with no limits records nothing, so the hits are this render's. this.reportCeilingHits(); return entries; @@ -12852,7 +12854,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.postProcessToolEntries(merged); // Strip stale images BEFORE placing markers and committing stats, so both // describe the post-strip context the agent actually receives. - this.applyImageStripping(merged, store, { sent: true }); + this.applyImageStripping(merged, store, { returned: true }); // The newest stored message is the triggering turn for this compile. A // structural repair may rewrite tool blocks, but it must never erase that @@ -15258,7 +15260,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.postProcessToolEntries(entries); // Strip stale images before committing stats so RenderStats.total reflects // the post-strip context (this path places no cache markers). - this.applyImageStripping(entries, store, { sent: true }); + this.applyImageStripping(entries, store, { returned: true }); this.rsEnd(); return entries; } @@ -16045,8 +16047,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * placeholder once they fall outside the live-image window — either deeper * than `imageStripDepthTokens` from the newest message, or beyond the * `maxLiveImages` most-recent images (counted newest-first), or past - * `maxLiveImageBytes`, the count and bytes each within its request ceiling - * (liveImageLimits). Summaries are + * `maxLiveImageBytes`, the count and bytes each within its image ceiling + * (resolvedImageLimits). Summaries are * already text, so they're naturally unaffected. The adjacent * "[image attachment: ]" text added at ingest preserves the filename, * so the placeholder itself stays terse. Reduces tokens, so it never pushes @@ -16058,13 +16060,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the in-progress render stats by the reclaimed tokens, keeping * `RenderStats.total` equal to the real rendered size. * - * `opts.sent`: the render is a selection's own, so a ceiling it strips - * against says its line once select returns it (reportCeilingHits). A - * layout rendered only to compare against, such as kv-unified's kept one, - * leaves it unset. */ - protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView, opts: { sent?: boolean } = {}): void { - // Count and bytes within the request ceilings; 0 = unlimited. - const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; + * `opts.returned`: select returns this render, so a ceiling that decides + * an image here says its line once select has returned it + * (reportCeilingHits). A layout rendered only to compare against, such as + * kv-unified's kept one, and any other caller's render, leave it unset. */ + protected applyImageStripping(entries: ContextEntry[], store: MessageStoreView, opts: { returned?: boolean } = {}): void { + // Count and bytes within the image ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; const depthTokens = this.config.imageStripDepthTokens ?? 0; // 0 = no depth strip if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return; // no window and no ceilings @@ -16109,8 +16111,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const blockBytes = AutobiographicalStrategy.imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; - if (overCount && ceilingSet.images) hits.images = true; - if (overBytes && ceilingSet.bytes) hits.bytes = true; + const deciding = ceilingsDeciding({ images: overCount, bytes: overBytes }, belowMark || tooDeep, ceilingSet); + if (deciding.images) hits.images = true; + if (deciding.bytes) hits.bytes = true; if (belowMark || tooDeep || overCount || overBytes) { // Stats-neutral (2026-07-12): every budgeting site now tallies at // POST-STRIP prices (see postStripEstimates), so the bucket never @@ -16125,11 +16128,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { return block; }); } - // A selection's own render (`sent`) leaves what its ceilings stripped for - // select to report once it returns that render; the last such strip is + // A render select returns (`returned`) leaves what its ceilings decided + // for select to report once it has returned it; the last such strip is // the render it returns. A layout rendered only to compare against, such // as kv-unified's kept one, leaves nothing. - if (opts.sent) this._ceilingHitsThisRender = hits; + if (opts.returned) this._ceilingHitsThisRender = hits; } /** imageStripHysteresisRatio: images in messages at or before this @@ -16166,14 +16169,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { const ratio = this.imageHysteresisRatio(); if (ratio === undefined) return; const stripDepth = this.config.imageStripDepthTokens ?? 0; - const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; if (stripDepth === 0 && maxLive === 0 && maxLiveBytes === 0) return; const messages = store.getAll(); const w = this.imageWatermark(); // Walk newest-first with the given limits; return the sequence of the // first message holding an image that does not fit (or undefined), and // which limits it passes. - let misfitBy = { images: false, bytes: false }; + let misfitBy = { depth: false, images: false, bytes: false }; const firstMisfit = (depthCap: number, countCap: number, bytesCap: number): number | undefined => { let depth = 0, count = 0, bytes = 0; for (let i = messages.length - 1; i >= 0; i--) { @@ -16183,10 +16186,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (b.type !== 'image') continue; const size = AutobiographicalStrategy.imageBlockBytes(b); misfitBy = { + depth: depthCap > 0 && depth > depthCap, images: countCap > 0 && count >= countCap, bytes: bytesCap > 0 && bytes + size > bytesCap, }; - if ((depthCap > 0 && depth > depthCap) || misfitBy.images || misfitBy.bytes) return m.sequence; + if (misfitBy.depth || misfitBy.images || misfitBy.bytes) return m.sequence; count++; bytes += size; } @@ -16195,13 +16199,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { return undefined; }; if (firstMisfit(stripDepth, maxLive, maxLiveBytes) === undefined) return; // no limit binds - // A ceiling that moves the watermark strips every later render by it, so - // the next render a select returns reports it (reportCeilingHits). A dry - // advance is undone, so it leaves nothing. + // A ceiling that decides the watermark's move strips every later render + // by it, so the next render a select returns reports it + // (reportCeilingHits). One the depth limit or a limit the host set also + // forces isn't the ceiling's. A dry advance is undone, so it leaves nothing. if (!this._dryRunSelect) { - const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); - if (misfitBy.images && ceilingSet.images) this._ceilingMovedWatermark.images = true; - if (misfitBy.bytes && ceilingSet.bytes) this._ceilingMovedWatermark.bytes = true; + const deciding = ceilingsDeciding(misfitBy, misfitBy.depth, ceilingSetLimits(this.configuredImageLimits())); + if (deciding.images) this._ceilingMovedWatermark.images = true; + if (deciding.bytes) this._ceilingMovedWatermark.bytes = true; } const next = firstMisfit( stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, @@ -16242,7 +16247,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Post-strip token estimate per message index (2026-07-12 tail-starvation * fix). Mirrors `applyImageStripping`, with the same limits from - * liveImageLimits(), the count and bytes within their request ceilings: + * resolvedImageLimits(), the count and bytes within their image ceilings: * an image beyond the `maxLiveImages` * newest (counted newest-first) or deeper than `imageStripDepthTokens` of * raw estimate from the live end renders as a placeholder — so every place @@ -16256,7 +16261,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected postStripEstimates(store: MessageStoreView): number[] { const messages = store.getAll(); const stripDepth = this.config.imageStripDepthTokens ?? 0; - const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; // A compile asks for this pass more than once (recent-window scan, then // the picker inputs). The listing array is the store's own cached view, // so its identity, calibration and the knobs key the memo, the @@ -16320,21 +16325,21 @@ export class AutobiographicalStrategy implements ResettableStrategy { messages: readonly StoredMessage[]; calibration: number; stripDepth: number; maxLive: number; maxLiveBytes: number; ratio: number | undefined; out: number[]; } | null = null; - /** Byte wall default: 20 MiB of base64, under the default request byte + /** Byte wall default: 20 MiB of base64, under the default image byte * ceiling (24 MiB), which leaves room under membrane's 28 MiB cap. */ protected static readonly DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; /** Which ceilings have been said to strip the live window (once each per instance). */ private readonly ceilingBoundReported = { images: false, bytes: false }; - /** The ceilings the last selection's render stripped against (applyImageStripping, `sent`). */ + /** The ceilings that decided an image in the last render select returned (applyImageStripping, `returned`). */ private _ceilingHitsThisRender = { images: false, bytes: false }; - /** The ceilings that have moved the hysteresis watermark on this branch, outside a dry run. */ + /** The ceilings that have decided a move of the hysteresis watermark on this branch, outside a dry run. */ private readonly _ceilingMovedWatermark = { images: false, bytes: false }; - /** Say once for each ceiling, when a select returns a render that a - * request ceiling, rather than a limit the host set, stripped (see - * ceilingSetLimits), directly or through a watermark it moved on this - * branch while this instance ran. A watermark loaded from the store + /** Say once for each ceiling, when a select returns a render in which an + * image ceiling, rather than a limit the host set, decided that an image + * is stripped (see ceilingsDeciding), directly or through a watermark it + * moved on this branch while this instance ran. A watermark loaded from the store * carries no record of what moved it, so a later instance says it when * its own walk next meets the ceiling. A dry select, a preview, an * observer's reconstruction or a refused select says nothing, and the @@ -16354,7 +16359,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { * maxLiveImages it applies none: an explicit undefined erases the default * 6 and reads as no limit, which is the host's doing. */ private imageLimitsDefaulted(): ReadonlySet { - const keys = ['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes', 'maxCompressionImageBytes'] as const; + const keys = ['maxLiveImages', 'maxLiveImageBytes', 'imageCeiling', 'imageByteCeiling', 'maxCompressionImageBytes'] as const; return new Set(keys.filter((key) => { const source = this.configProvenance[key]; if (source === undefined || source === 'library-default') return true; @@ -16364,30 +16369,39 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** The live-image limits as configured, with the defaults where unset. */ private configuredImageLimits(): { - maxLiveImages: number; maxLiveImageBytes: number; maxRequestImages: number; maxRequestImageBytes: number; + maxLiveImages: number; maxLiveImageBytes: number; imageCeiling: number; imageByteCeiling: number; } { return { maxLiveImages: this.config.maxLiveImages ?? 0, maxLiveImageBytes: this.config.maxLiveImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_LIVE_IMAGE_BYTES, - maxRequestImages: this.config.maxRequestImages ?? DEFAULT_MAX_REQUEST_IMAGES, - maxRequestImageBytes: this.config.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES, + imageCeiling: this.config.imageCeiling ?? DEFAULT_IMAGE_CEILING, + imageByteCeiling: this.config.imageByteCeiling ?? DEFAULT_IMAGE_BYTE_CEILING, }; } /** - * The live window's count and byte limits, clamped to the request - * ceilings, and the ceilings themselves (see ContextStrategy.liveImageLimits). - * The strip, the watermark and the post-strip estimates all read their - * limits here, so they can't disagree about them. + * The live window's count and byte limits, kept within the image + * ceilings, and the ceilings themselves. The strip, the watermark, the + * post-strip estimates and a compression prompt all read their limits + * here, so they can't disagree about them. */ - liveImageLimits(): LiveImageLimits { + private resolvedImageLimits(): ResolvedImageLimits { return resolveLiveImageLimits(this.configuredImageLimits()); } + /** + * The image ceilings (see ContextStrategy.imageCeilings), as the window + * applies them: a malformed one at its default, a fraction floored. + */ + imageCeilings(): ImageCeilings { + const { ceiling } = this.resolvedImageLimits(); + return { imageCeiling: ceiling.images, imageByteCeiling: ceiling.bytes }; + } + /** Whether the image limits' warning has been said (once per instance). */ private imageLimitsWarningReported = false; - /** One line when a configured image limit sits above its request ceiling, + /** One line when a configured image limit sits above its image ceiling, * or a ceiling is malformed and takes its default, so an operator sees why * the window or a compression prompt keeps fewer images than they set. */ private reportImageLimitsWarningOnce(): void { @@ -16441,8 +16455,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected prepareCompressionRequest(request: NormalizedRequest): CompressionPreparation { const prepared = { ...request, messages: request.messages.map(message => ({ ...message })) }; const serializedMediaReplacements = this.stripSerializedCompressionMedia(prepared.messages); - // A compression prompt is a request too: at most the request ceilings. - const { ceiling } = this.liveImageLimits(); + // A compression prompt carries at most the image ceilings. + const { ceiling } = this.resolvedImageLimits(); const imageBudgetBytes = clampToCeiling( this.config.maxCompressionImageBytes ?? AutobiographicalStrategy.DEFAULT_MAX_COMPRESSION_IMAGE_BYTES, ceiling.bytes, @@ -16496,7 +16510,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const mib = (bytes: number) => `${(bytes / (1024 * 1024)).toFixed(1)}MiB`; const limits = [ ...(counts.imageBudgetBytes > 0 ? [`the ${mib(counts.imageBudgetBytes)} inline image budget`] : []), - ...(counts.imageCountCap > 0 ? [`the ${counts.imageCountCap}-image request ceiling`] : []), + ...(counts.imageCountCap > 0 ? [`the ${counts.imageCountCap}-image ceiling`] : []), ].join(' and '); this.diag.error( `[autobiographical] compression prompt: replaced ${counts.imageReplacements} image(s) with placeholders ` + @@ -16723,7 +16737,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Cap inline image bytes in a COMPRESSION prompt (2026-07-12), and with - * `capImages` their number (2026-10-19, `maxRequestImages`). The main + * `capImages` their number (2026-10-19, `imageCeiling`). The main * window enforces `maxLiveImageBytes` through the strip policy; the * summarizer's raw-chunk replay had no such wall and leaned on membrane's * byte shed instead — which is a transport backstop, not a policy owner diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 90e2d0a9..eba33dfa 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -1,7 +1,7 @@ import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; import type { ContextStrategy, - LiveImageLimits, + ImageCeilings, StrategyContext, ReadinessState, MessageStoreView, @@ -20,12 +20,14 @@ import { OverBudgetError } from '../adaptive/picker.js'; import { observeStoreBranch, type StoreBranchGeneration } from '../branch-generation.js'; import { capContent, cappedPriceExtra } from './message-cap.js'; import { - DEFAULT_MAX_REQUEST_IMAGES, - DEFAULT_MAX_REQUEST_IMAGE_BYTES, + DEFAULT_IMAGE_CEILING, + DEFAULT_IMAGE_BYTE_CEILING, liveImageLimitsWarning, ceilingBoundLine, ceilingSetLimits, + ceilingsDeciding, resolveLiveImageLimits, + type ResolvedImageLimits, } from '../live-image-limits.js'; /** @@ -76,7 +78,7 @@ export interface WindowedPassthroughOptions { * `maxLiveImageBytes` of cumulative inline image bytes (newest-first). * Defaults follow `DEFAULT_AUTOBIOGRAPHICAL_CONFIG` (and the memory * strategy's byte wall). A count or byte limit of 0 means up to its - * request ceiling, below, and a depth of 0 means no depth strip. A strip changes the + * image ceiling, below, and a depth of 0 means no depth strip. A strip changes the * bytes at that position, which the cache-marker placement absorbs by * measurement (the marker falls back to the deepest byte-stable entry). */ @@ -84,17 +86,18 @@ export interface WindowedPassthroughOptions { imageStripDepthTokens?: number; maxLiveImageBytes?: number; /** - * The request ceilings, mirroring `AutobiographicalConfig`: the most images, - * and the most inline image bytes, one request may carry. The window keeps - * at most these of the images it counts, counted as + * The image ceilings, mirroring `AutobiographicalConfig.imageCeiling` and + * `imageByteCeiling`: the most top-level image blocks, and the most of + * their inline image bytes, the window keeps, counted as * `AutobiographicalConfig.maxLiveImages` says, whatever `maxLiveImages` and - * `maxLiveImageBytes` say (0 there means up to these), and - * `liveImageLimits()` reports them. + * `maxLiveImageBytes` say (0 there means up to these). They don't bound a + * whole request: what the window doesn't count, and what a caller injects + * or appends, isn't counted. `imageCeilings()` reports them. * Defaults 500 images and 24 MiB; 0 = no ceiling, and a value that isn't 0 * or a finite number above it takes the default. */ - maxRequestImages?: number; - maxRequestImageBytes?: number; + imageCeiling?: number; + imageByteCeiling?: number; } /** Text substituted for an image block once it leaves the live-image window @@ -153,7 +156,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { private readonly anchorStateIdOverride?: string; /** The image limits as configured, with the defaults where unset. */ private readonly configuredImageLimits: { - maxLiveImages: number; maxLiveImageBytes: number; maxRequestImages: number; maxRequestImageBytes: number; + maxLiveImages: number; maxLiveImageBytes: number; imageCeiling: number; imageByteCeiling: number; }; /** The image limits the options didn't give, whose values are defaults. */ private readonly imageLimitsDefaulted: ReadonlySet; @@ -187,26 +190,34 @@ export class WindowedPassthroughStrategy implements ContextStrategy { this.configuredImageLimits = { maxLiveImages: options.maxLiveImages ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.maxLiveImages ?? 0, maxLiveImageBytes: options.maxLiveImageBytes ?? DEFAULT_MAX_LIVE_IMAGE_BYTES, - maxRequestImages: options.maxRequestImages ?? DEFAULT_MAX_REQUEST_IMAGES, - maxRequestImageBytes: options.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES, + imageCeiling: options.imageCeiling ?? DEFAULT_IMAGE_CEILING, + imageByteCeiling: options.imageByteCeiling ?? DEFAULT_IMAGE_BYTE_CEILING, }; this.imageLimitsDefaulted = new Set( - (['maxLiveImages', 'maxLiveImageBytes', 'maxRequestImages', 'maxRequestImageBytes'] as const).filter((key) => options[key] == null), + (['maxLiveImages', 'maxLiveImageBytes', 'imageCeiling', 'imageByteCeiling'] as const).filter((key) => options[key] == null), ); this.imageStripDepthTokens = options.imageStripDepthTokens ?? DEFAULT_AUTOBIOGRAPHICAL_CONFIG.imageStripDepthTokens ?? 0; } /** - * The live window's count and byte limits, clamped to the request - * ceilings, and the ceilings themselves (see ContextStrategy.liveImageLimits). + * The image ceilings (see ContextStrategy.imageCeilings), as the window + * applies them: a malformed one at its default, a fraction floored. */ - liveImageLimits(): LiveImageLimits { + imageCeilings(): ImageCeilings { + const { ceiling } = this.resolvedImageLimits(); + return { imageCeiling: ceiling.images, imageByteCeiling: ceiling.bytes }; + } + + /** The live window's count and byte limits, kept within the image + * ceilings, and the ceilings themselves. */ + private resolvedImageLimits(): ResolvedImageLimits { return resolveLiveImageLimits(this.configuredImageLimits); } - /** Say once that a request ceiling, rather than a limit the options set, has - * stripped an image from the live window (see ceilingSetLimits). */ + /** Say once that an image ceiling, rather than a limit the options set, + * has decided that an image is stripped from the live window (see + * ceilingsDeciding). */ private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { if (this.ceilingBoundReported[kind]) return; this.ceilingBoundReported[kind] = true; @@ -441,11 +452,12 @@ export class WindowedPassthroughStrategy implements ContextStrategy { entries: ContextEntry[], messages: StoredMessage[], store: MessageStoreView, - // Whether this render is sent: a preview's (dryRun) says no ceiling line. - sent: boolean, + // Whether select returns this render: only such a render says a + // ceiling's line. A preview's (dryRun) doesn't, nor any other caller's. + returned = false, ): void { - // Count and bytes within the request ceilings; 0 = unlimited. - const { images: maxLive, bytes: maxLiveBytes } = this.liveImageLimits().target; + // Count and bytes within the image ceilings; 0 = unlimited. + const { images: maxLive, bytes: maxLiveBytes } = this.resolvedImageLimits().target; const depthTokens = this.imageStripDepthTokens; if (maxLive === 0 && depthTokens === 0 && maxLiveBytes === 0) return; @@ -473,8 +485,11 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const bytes = imageBlockBytes(block); const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; - if (sent && overCount && ceilingSet.images) this.reportCeilingBoundOnce('images'); - if (sent && overBytes && ceilingSet.bytes) this.reportCeilingBoundOnce('bytes'); + if (returned) { + const deciding = ceilingsDeciding({ images: overCount, bytes: overBytes }, tooDeep, ceilingSet); + if (deciding.images) this.reportCeilingBoundOnce('images'); + if (deciding.bytes) this.reportCeilingBoundOnce('bytes'); + } if (tooDeep || overCount || overBytes) { return { type: 'text', text: IMAGE_PLACEHOLDER } as ContentBlock; } diff --git a/src/types/index.ts b/src/types/index.ts index 60e53133..3a1095fb 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -79,7 +79,7 @@ export type { CalibrationReset, RenderStatsCapableStrategy, RenderedSummaryInfo, - LiveImageLimits, + ImageCeilings, } from './strategy.js'; export { diff --git a/src/types/strategy.ts b/src/types/strategy.ts index ab2093f3..c1be9833 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -8,6 +8,7 @@ import type { PendingWork, } from './context.js'; import type { OverBudgetClearedBy } from '../adaptive/picker.js'; +import { DEFAULT_IMAGE_BYTE_CEILING, DEFAULT_IMAGE_CEILING } from '../live-image-limits.js'; /** * Read-only view of the message store for strategies. @@ -237,31 +238,28 @@ export interface ContextStrategy { takeSelectionCause?(): string | undefined; /** - * The live-image window's limits and the request ceilings above them, for - * a caller that appends to a compiled request without recompiling, such - * as a turn's continuation rounds: the strategy's compile keeps the images - * its window counts within `target`, and the caller can let the request - * grow to `ceiling` before compiling it again. Both count the images the - * window counts (see `AutobiographicalConfig.maxLiveImages`), not every - * image a request can carry. It only reports the figures: a caller bounds - * what it appends, and what it injects at compile time. Absent on a - * strategy with no live-image window. + * The image ceilings this strategy keeps its live window within and holds + * a compression prompt to (`imageCeiling` and `imageByteCeiling`, 0 for + * none). They're configuration, not usage: they bound the images the + * window counts (top-level image blocks; see + * `AutobiographicalConfig.maxLiveImages`), not every image a request + * carries, and nothing here bounds what a caller injects at compile time + * or appends to a compiled request. A caller that wants its request within + * them counts that request's images itself, every shape included, and + * compares the counts with these. Absent on a strategy with no live-image + * window. */ - liveImageLimits?(): LiveImageLimits; + imageCeilings?(): ImageCeilings; } /** - * A strategy's live-image limits. Each figure is a count of the image blocks - * the window counts (see `AutobiographicalConfig.maxLiveImages`) or a sum of - * their inline base64 bytes, and 0 means no limit. `target` is what the - * window keeps, already within `ceiling` (`maxLiveImages` and - * `maxLiveImageBytes`, clamped); `ceiling` is the most one request may carry - * (`maxRequestImages` and `maxRequestImageBytes`), which the window keeps - * its own images within. + * A strategy's image ceilings (see ContextStrategy.imageCeilings): the most + * images, and the most inline base64 image bytes, its live window keeps and + * a compression prompt carries. 0 is no ceiling. */ -export interface LiveImageLimits { - target: { images: number; bytes: number }; - ceiling: { images: number; bytes: number }; +export interface ImageCeilings { + imageCeiling: number; + imageByteCeiling: number; } /** What a rendered summary covers, for layout reporting. */ @@ -1083,19 +1081,20 @@ export interface AutobiographicalConfig { /** Maximum number of images kept "live" (sent as real image blocks) in the * compiled context. Counted newest-first across the whole window; images * beyond this many are replaced with a text placeholder. Kept within - * `maxRequestImages`, so 0 means up to that ceiling. A window kept at the - * ceiling leaves a caller that appends to the compiled request no room - * below it once it's full, so that caller recompiles at each image it - * adds; at about 1,600 tokens an image, the token budget binds first - * unless the images are small. A fraction is floored. + * `imageCeiling`, so 0 means up to that ceiling. A window kept at the + * ceiling leaves no room under it for images a caller appends to the + * compiled request, so a caller that keeps its requests within the + * ceilings recompiles at each image it adds; at about 1,600 tokens an + * image, the token budget binds first unless the images are small. A + * fraction is floored. * A hard ceiling for dense bursts that pack many images into the live window. * The window counts top-level `image` blocks, and measures their bytes by * the length of their base64 `source.data`. So an image nested in a tool * result and a `generated_image` are neither counted nor replaced, and an * image with a `data:` URL counts but its bytes don't. In the window, - * every live-image limit and request ceiling below counts the same way; a + * every live-image limit and image ceiling below counts the same way; a * compression prompt counts every image, nested and generated ones - * included (see `maxRequestImages`). */ + * included (see `imageCeiling`). */ maxLiveImages?: number; /** Token depth from the newest message beyond which images are stripped to a * text placeholder, even though the surrounding text stays verbatim. Measured @@ -1116,11 +1115,11 @@ export interface AutobiographicalConfig { * Byte wall for live images (2026-07-12): keep inline images newest-first * only while their cumulative base64 size stays under this budget — on top * of `maxLiveImages` / `imageStripDepthTokens`. Default 20 MiB of base64. - * The window keeps at most `maxRequestImageBytes` whatever this says, and 0 + * The window keeps at most `imageByteCeiling` whatever this says, and 0 * here means up to that ceiling. It counts the bytes of the images the * window counts (see `maxLiveImages`), not the request's other bytes, so on * its own it can't keep a request under membrane's byte cap: see - * `maxRequestImageBytes`. + * `imageByteCeiling`. */ maxLiveImageBytes?: number; @@ -1142,51 +1141,56 @@ export interface AutobiographicalConfig { * live window's: the prompt also carries the head, the recall frontier and * the raw chunk. Newest-first; older images become loud placeholders. * Default 12MB of base64. A compression prompt carries at most - * `maxRequestImages` images and `maxRequestImageBytes` bytes whatever this + * `imageCeiling` images and `imageByteCeiling` bytes whatever this * says, and 0 here, or any value that isn't a positive finite number, means * up to the byte ceiling. */ maxCompressionImageBytes?: number; /** - * The most images one request may carry (2026-10-19). The live window keeps - * at most this many of the images it counts (see `maxLiveImages`), - * whatever `maxLiveImages` says, so a `maxLiveImages` of 0 means up to this - * many. A compression prompt carries at most this many images, nested ones - * and generated ones included. `liveImageLimits()` reports it, for a - * caller that appends to a compiled request without recompiling, such as a - * turn's continuation rounds, to keep its own appends below; nothing here - * bounds what a caller appends or injects. The provider refuses a request - * with more than 600 images and documents together (measured 2026-10-13, - * and again on 2026-10-19). Default 500, which leaves room for what the - * window doesn't count: documents, images nested in tool results and - * generated images (see `maxLiveImages`), and image blocks a host adds - * outside the compile. 0 = no ceiling. A value that isn't 0 or a finite number above it - * takes the default, and the strategy's warning at initialize names it. A - * fraction is floored. Under a window limit of 0 or above this ceiling, the - * strategy says once, the first time an image falls past it. - */ - maxRequestImages?: number; - - /** - * The most inline image bytes (base64) one request may carry (2026-10-19), - * as `maxRequestImages` is for their number. The live window keeps at most - * this many bytes of the images it counts (see `maxLiveImages`), whatever - * `maxLiveImageBytes` says, and a - * compression prompt carries at most this many bytes of its images. - * `liveImageLimits()` reports it, as for `maxRequestImages`. membrane - * refuses, before sending it, a request whose serialized messages exceed - * its byte cap, 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise. - * Default 24 MiB, which leaves 4 MiB of that cap for everything else the - * messages carry: their text, their JSON, any documents or other media, and - * in the window any images it doesn't count, none of which this counts. It - * doesn't follow `MEMBRANE_MAX_REQUEST_BYTES`, so a host that changes - * membrane's cap sets this to match. 0 = no ceiling. - * A value that isn't 0 or a finite number above it takes the default. Under - * a window limit of 0 or above this ceiling, the strategy says once when an - * image first falls past it, as for `maxRequestImages`. - */ - maxRequestImageBytes?: number; + * The most top-level image blocks a strategy's live window keeps, whatever + * `maxLiveImages` says, and the most images a compression prompt carries + * (2026-10-19). A `maxLiveImages` of 0 means up to this many. The window + * counts top-level `image` blocks (see `maxLiveImages`); a compression + * prompt counts every image, nested ones and generated ones included. + * + * It doesn't bound a whole request. The window's images nested in tool + * results and its generated images aren't counted, and neither is + * anything a caller injects at compile time or appends to a compiled + * request; `imageCeilings()` reports this ceiling, so such a caller can + * count its request's images itself and compare. The provider refuses a + * request with more than 600 images and documents together (measured + * 2026-10-13, and again on 2026-10-19), and the default, 500, leaves room + * for what this doesn't count. 0 = no ceiling. A value that isn't 0 or a + * finite number above it takes the default, and the strategy's warning at + * initialize names it. A fraction is floored. Under a window limit of 0 or + * above this ceiling, the strategy says once, the first time this ceiling + * decides that an image is shown as a placeholder. + */ + imageCeiling?: number; + + /** + * The most inline image bytes (base64) of the top-level image blocks a + * strategy's live window keeps, whatever `maxLiveImageBytes` says, and the + * most of its images' bytes a compression prompt carries (2026-10-19). A + * `maxLiveImageBytes` of 0 means up to this many. The window measures an + * image by the length of its base64 `source.data` (see `maxLiveImages`). + * + * It doesn't bound a request's bytes: the messages' text and JSON, + * documents, the images the window doesn't count, and anything a caller + * injects or appends aren't counted. membrane refuses, before sending it, a + * request whose serialized messages exceed its byte cap, 28 MiB unless + * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, and the default, 24 MiB, + * leaves 4 MiB of that cap for what this doesn't count. It doesn't follow + * `MEMBRANE_MAX_REQUEST_BYTES`, so a host that changes membrane's cap + * changes this with it, keeping room under the cap for what this doesn't + * count. `imageCeilings()` reports it, as for `imageCeiling`. 0 = no + * ceiling. A value that isn't 0 or a finite number above it takes the + * default. Under a window limit of 0 or above this ceiling, the strategy + * says once when this ceiling first decides that an image is shown as a + * placeholder, as for `imageCeiling`. + */ + imageByteCeiling?: number; /** * Merge grouping: exclude candidates whose OWN source span exceeds the @@ -1998,6 +2002,8 @@ Write naturally, as recollection of what you experienced.`, maxMessageTokens: 0, maxLiveImages: 6, imageStripDepthTokens: 30000, + imageCeiling: DEFAULT_IMAGE_CEILING, + imageByteCeiling: DEFAULT_IMAGE_BYTE_CEILING, positionedRecallPairs: true, recallHeaderTemplate: '[Recall {id}]', compressionRefusalCurveFallbacks: 3, diff --git a/test/compression-request-preparation.test.ts b/test/compression-request-preparation.test.ts index 20f4b686..6ac54a6e 100644 --- a/test/compression-request-preparation.test.ts +++ b/test/compression-request-preparation.test.ts @@ -203,7 +203,7 @@ describe('preparation counts and logging', () => { assert.deepEqual(sent.logs, [ '[autobiographical] compression prompt: replaced 1 serialized data URL(s) with markers [count-test]', '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + - '3.0MiB inline image budget and the 500-image request ceiling (kept 1 image(s), 2.5MiB, newest messages first) ' + + '3.0MiB inline image budget and the 500-image ceiling (kept 1 image(s), 2.5MiB, newest messages first) ' + '[count-test]', ]); assert.deepEqual(captureErrors(() => strategy.send(request([{ type: 'text', text: 'plain' }]), 'unchanged')).logs, []); diff --git a/test/image-stripping.test.ts b/test/image-stripping.test.ts index 3427262d..760640f3 100644 --- a/test/image-stripping.test.ts +++ b/test/image-stripping.test.ts @@ -109,9 +109,9 @@ describe('Live image policy', () => { maxLiveImages: 0, imageStripDepthTokens: 0, maxLiveImageBytes: 0, - // The request ceilings bound an unlimited window too. - maxRequestImages: 0, - maxRequestImageBytes: 0, + // The image ceilings bound an unlimited window too. + imageCeiling: 0, + imageByteCeiling: 0, }); const manager = await ContextManager.open({ path: TEST_STORE_PATH, strategy }); diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 3fd4f658..5cc952e2 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -1,9 +1,9 @@ /** - * The request ceilings (maxRequestImages, maxRequestImageBytes): the most - * images, and image bytes, one request may carry. The live window keeps its + * The image ceilings (imageCeiling, imageByteCeiling): the most top-level + * image blocks, and their bytes, a strategy's live window keeps, and the most + * images and bytes a compression prompt carries. The live window keeps its * own limits within them, compression prompts carry no more than them, and - * liveImageLimits() reports both, for a caller that appends to a compiled - * request without recompiling. + * imageCeilings() reports them. */ import { describe, it } from 'node:test'; import assert from 'node:assert/strict'; @@ -13,7 +13,7 @@ import { join } from 'node:path'; import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; import { AutobiographicalStrategy, ContextManager, WindowedPassthroughStrategy } from '../src/index.js'; import type { ContextStrategy } from '../src/index.js'; -import { ceilingBoundLine } from '../src/live-image-limits.js'; +import { ceilingBoundLine, resolveLiveImageLimits } from '../src/live-image-limits.js'; const MiB = 1024 * 1024; const PLACEHOLDER = '[image not shown: this context keeps only some images]'; @@ -60,70 +60,117 @@ const DEFAULTS = { 'windowed passthrough': () => new WindowedPassthroughStrategy(), }; -describe('liveImageLimits()', () => { +describe('imageCeilings()', () => { for (const [name, make] of Object.entries(STRATEGIES)) { - it(`${name}: the defaults are a window of 6 images and 20 MiB under ceilings of 500 images and 24 MiB`, () => { - assert.deepEqual(DEFAULTS[name as keyof typeof DEFAULTS]().liveImageLimits(), { - target: { images: 6, bytes: 20 * MiB }, - ceiling: { images: 500, bytes: 24 * MiB }, - }); - }); - - it(`${name}: the window is kept within the ceilings, and a ceiling of 0 is none`, () => { - const limits = (options: Record) => make(options).liveImageLimits(); - assert.deepEqual(limits({ maxLiveImages: 0, maxLiveImageBytes: 0 }), { - target: { images: 500, bytes: 24 * MiB }, ceiling: { images: 500, bytes: 24 * MiB }, - }, 'an unlimited window means up to the ceilings'); - assert.deepEqual(limits({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }).target, - { images: 500, bytes: 24 * MiB }, 'a window above the ceilings keeps the ceilings'); - assert.deepEqual(limits({ maxLiveImages: 4, maxLiveImageBytes: MiB, maxRequestImages: 10, maxRequestImageBytes: 2 * MiB }), { - target: { images: 4, bytes: MiB }, ceiling: { images: 10, bytes: 2 * MiB }, - }, 'a window under configured ceilings keeps its own limits'); - assert.deepEqual(limits({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0 }), { - target: { images: 0, bytes: 0 }, ceiling: { images: 0, bytes: 0 }, - }, 'no ceiling leaves an unlimited window unlimited'); - assert.equal(limits({ maxLiveImages: 4, maxRequestImages: 0 }).target.images, 4, 'no ceiling leaves a window as set'); + it(`${name}: the defaults are 500 images and 24 MiB`, () => { + assert.deepEqual(DEFAULTS[name as keyof typeof DEFAULTS]().imageCeilings(), { imageCeiling: 500, imageByteCeiling: 24 * MiB }); }); - it(`${name}: a window limit that isn't a positive finite number is none, and a malformed ceiling takes its default`, () => { - const limits = (options: Record) => make(options).liveImageLimits(); + it(`${name}: reports the ceilings as the window applies them`, () => { + const ceilings = (options: Record) => make(options).imageCeilings(); + assert.deepEqual(ceilings({ imageCeiling: 10, imageByteCeiling: 2 * MiB }), { imageCeiling: 10, imageByteCeiling: 2 * MiB }); + assert.deepEqual(ceilings({ imageCeiling: 0, imageByteCeiling: 0 }), { imageCeiling: 0, imageByteCeiling: 0 }, '0 is none'); for (const bad of [-3, Number.NaN, Infinity]) { - assert.equal(limits({ maxLiveImages: bad }).target.images, 500, `maxLiveImages ${bad} is unlimited, so up to the ceiling`); - assert.equal(limits({ maxLiveImageBytes: bad }).target.bytes, 24 * MiB, `maxLiveImageBytes ${bad} is unlimited, so up to the ceiling`); - assert.equal(limits({ maxRequestImages: bad }).ceiling.images, 500, `maxRequestImages ${bad} takes the default`); - assert.equal(limits({ maxRequestImageBytes: bad }).ceiling.bytes, 24 * MiB, `maxRequestImageBytes ${bad} takes the default`); + assert.deepEqual(ceilings({ imageCeiling: bad, imageByteCeiling: bad }), { imageCeiling: 500, imageByteCeiling: 24 * MiB }, + `a ceiling of ${bad} takes the default`); } - assert.equal(limits({ maxLiveImages: 0, maxRequestImages: -1 }).target.images, 500, 'the window is clamped to the default the malformed ceiling took'); - assert.equal(limits({ maxLiveImages: Infinity, maxRequestImages: 0 }).target.images, 0, - 'an infinite limit with no ceiling is reported as none, 0'); + assert.deepEqual(ceilings({ imageCeiling: 0.5, imageByteCeiling: 100.7 }), { imageCeiling: 1, imageByteCeiling: 100 }, + 'a fraction is floored, to at least 1'); + assert.deepEqual(ceilings({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }), { imageCeiling: 500, imageByteCeiling: 24 * MiB }, + "the window's own limits leave the ceilings as they are"); }); + } +}); - it(`${name}: a fractional limit or ceiling is floored, to at least 1`, () => { - const limits = (options: Record) => make(options).liveImageLimits(); - assert.equal(limits({ maxLiveImages: 2.5 }).target.images, 2); - assert.deepEqual(limits({ maxLiveImages: 0, maxRequestImages: 1.5 }), { - target: { images: 1, bytes: 20 * MiB }, ceiling: { images: 1, bytes: 24 * MiB }, - }); - assert.equal(limits({ maxRequestImages: 0.5 }).ceiling.images, 1, 'a fraction under 1 is 1, not none'); - assert.equal(limits({ maxRequestImageBytes: 100.7 }).ceiling.bytes, 100); +describe("the memory strategy's effective config", () => { + it('reports both ceilings, as the library defaults when the host sets neither, and as the host set them otherwise', async () => { + const report = async (options: Record) => { + const lines: string[] = []; + const original = console.error; + console.error = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + try { + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: new AutobiographicalStrategy({ logEffectiveConfig: true, ...options }) }); + await manager.close(); + } finally { + console.error = original; + rmSync(dir, { recursive: true, force: true }); + } + const line = lines.map((l) => { try { return JSON.parse(l); } catch { return null; } }).find((l) => l?.event === 'config:effective'); + assert.ok(line, JSON.stringify(lines)); + return line as { effective: Record; provenance: Record }; + }; + const defaults = await report({}); + assert.equal(defaults.effective.imageCeiling, 500); + assert.equal(defaults.effective.imageByteCeiling, 24 * MiB); + assert.equal(defaults.provenance.imageCeiling, 'library-default'); + assert.equal(defaults.provenance.imageByteCeiling, 'library-default'); + const set = await report({ imageCeiling: 40, imageByteCeiling: 0 }); + assert.equal(set.effective.imageCeiling, 40); + assert.equal(set.effective.imageByteCeiling, 0); + assert.equal(set.provenance.imageCeiling, 'caller'); + }); +}); + +describe('the window limits resolved within the ceilings', () => { + const resolve = (configured: Partial>) => + resolveLiveImageLimits({ maxLiveImages: 6, maxLiveImageBytes: 20 * MiB, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...configured }); + + it('the defaults are a window of 6 images and 20 MiB under ceilings of 500 images and 24 MiB', () => { + assert.deepEqual(resolve({}), { target: { images: 6, bytes: 20 * MiB }, ceiling: { images: 500, bytes: 24 * MiB } }); + }); + + it('the window is kept within the ceilings, and a ceiling of 0 is none', () => { + assert.deepEqual(resolve({ maxLiveImages: 0, maxLiveImageBytes: 0 }), { + target: { images: 500, bytes: 24 * MiB }, ceiling: { images: 500, bytes: 24 * MiB }, + }, 'an unlimited window means up to the ceilings'); + assert.deepEqual(resolve({ maxLiveImages: 800, maxLiveImageBytes: 30 * MiB }).target, + { images: 500, bytes: 24 * MiB }, 'a window above the ceilings keeps the ceilings'); + assert.deepEqual(resolve({ maxLiveImages: 4, maxLiveImageBytes: MiB, imageCeiling: 10, imageByteCeiling: 2 * MiB }), { + target: { images: 4, bytes: MiB }, ceiling: { images: 10, bytes: 2 * MiB }, + }, 'a window under configured ceilings keeps its own limits'); + assert.deepEqual(resolve({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0 }), { + target: { images: 0, bytes: 0 }, ceiling: { images: 0, bytes: 0 }, + }, 'no ceiling leaves an unlimited window unlimited'); + assert.equal(resolve({ maxLiveImages: 4, imageCeiling: 0 }).target.images, 4, 'no ceiling leaves a window as set'); + }); + + it("a window limit that isn't a positive finite number is none, and a malformed ceiling takes its default", () => { + for (const bad of [-3, Number.NaN, Infinity]) { + assert.equal(resolve({ maxLiveImages: bad }).target.images, 500, `maxLiveImages ${bad} is unlimited, so up to the ceiling`); + assert.equal(resolve({ maxLiveImageBytes: bad }).target.bytes, 24 * MiB, `maxLiveImageBytes ${bad} is unlimited, so up to the ceiling`); + assert.equal(resolve({ imageCeiling: bad }).ceiling.images, 500, `imageCeiling ${bad} takes the default`); + assert.equal(resolve({ imageByteCeiling: bad }).ceiling.bytes, 24 * MiB, `imageByteCeiling ${bad} takes the default`); + } + assert.equal(resolve({ maxLiveImages: 0, imageCeiling: -1 }).target.images, 500, 'the window is clamped to the default the malformed ceiling took'); + assert.equal(resolve({ maxLiveImages: Infinity, imageCeiling: 0 }).target.images, 0, + 'an infinite limit with no ceiling is reported as none, 0'); + }); + + it('a fractional limit or ceiling is floored, to at least 1', () => { + assert.equal(resolve({ maxLiveImages: 2.5 }).target.images, 2); + assert.deepEqual(resolve({ maxLiveImages: 0, imageCeiling: 1.5 }), { + target: { images: 1, bytes: 20 * MiB }, ceiling: { images: 1, bytes: 24 * MiB }, }); - } + assert.equal(resolve({ imageCeiling: 0.5 }).ceiling.images, 1, 'a fraction under 1 is 1, not none'); + assert.equal(resolve({ imageByteCeiling: 100.7 }).ceiling.bytes, 100); + }); }); -describe('the live window within the request ceilings', () => { +describe('the live window within the image ceilings', () => { for (const [name, make] of Object.entries(STRATEGIES)) { - it(`${name}: an unlimited count keeps the newest maxRequestImages`, async () => { - assert.equal(await compiled(make({ maxLiveImages: 0, maxRequestImages: 2 }), 4), '..II'); + it(`${name}: an unlimited count keeps the newest imageCeiling`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, imageCeiling: 2 }), 4), '..II'); }); it(`${name}: a count above the ceiling keeps the ceiling's worth`, async () => { - assert.equal(await compiled(make({ maxLiveImages: 3, maxRequestImages: 2 }), 4), '..II'); + assert.equal(await compiled(make({ maxLiveImages: 3, imageCeiling: 2 }), 4), '..II'); }); - it(`${name}: an unlimited byte budget keeps the newest maxRequestImageBytes`, async () => { - assert.equal(await compiled(make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 2000 }), 4, 1000), '..II'); + it(`${name}: an unlimited byte budget keeps the newest imageByteCeiling`, async () => { + assert.equal(await compiled(make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 2000 }), 4, 1000), '..II'); }); it(`${name}: with no window and no ceilings, every image stays, more than the default window's 6`, async () => { assert.equal(await compiled(make({ - maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0, + maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0, }), 8), 'IIIIIIII'); }); it(`${name}: the window counts top-level image blocks only, so a nested or a generated image passes the count ceiling`, async () => { @@ -131,7 +178,7 @@ describe('the live window within the request ceilings', () => { // a tool result, or a generated image, isn't one of those (a change // that counts them would change this). const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, maxRequestImages: 1 }) }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, imageCeiling: 1 }) }); try { const image = (data: string) => ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; @@ -176,13 +223,13 @@ describe('the live window within the request ceilings', () => { const estimate = (strategy: AutobiographicalStrategy) => (strategy as unknown as { postStripEstimates(s: typeof store): number[] }).postStripEstimates(store); const stripped = [Math.ceil(PLACEHOLDER.length / 4), 1600]; - assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxRequestImages: 1 })), stripped, 'by the count ceiling'); - assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 5 })), stripped, + assert.deepEqual(estimate(memory({ maxLiveImages: 0, imageCeiling: 1 })), stripped, 'by the count ceiling'); + assert.deepEqual(estimate(memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 5 })), stripped, 'by the byte ceiling: 4 bytes fit, 8 do not'); // The memo is keyed by the clamped limits, so a changed ceiling prices anew. - const strategy = memory({ maxLiveImages: 0, maxRequestImages: 1 }); + const strategy = memory({ maxLiveImages: 0, imageCeiling: 1 }); assert.deepEqual(estimate(strategy), stripped); - (strategy as unknown as { config: { maxRequestImages: number } }).config.maxRequestImages = 2; + (strategy as unknown as { config: { imageCeiling: number } }).config.imageCeiling = 2; assert.deepEqual(estimate(strategy), [1600, 1600], 'with a ceiling of 2, both are live'); }); @@ -190,7 +237,7 @@ describe('the live window within the request ceilings', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), }); try { const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); @@ -210,7 +257,7 @@ describe('the hysteresis watermark against the byte ceiling', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 16, imageStripHysteresisRatio: 0.5 }), + strategy: memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), }); try { const strip = async (n: number) => pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n); @@ -234,41 +281,41 @@ describe('the image limits warning', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); try { const cases = [ - [{ maxLiveImages: 8, maxRequestImages: 4 }, ['maxLiveImages 8 is above maxRequestImages 4, so the live window keeps at most 4']], - [{ maxLiveImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }, - ['maxLiveImageBytes 3 MiB is above maxRequestImageBytes 2 MiB, so the live window keeps at most 2 MiB']], - [{ maxLiveImages: 8, maxRequestImages: 4, maxLiveImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }, - ['maxLiveImages 8 is above maxRequestImages 4', 'maxLiveImageBytes 3 MiB is above maxRequestImageBytes 2 MiB']], + [{ maxLiveImages: 8, imageCeiling: 4 }, ['maxLiveImages 8 is above imageCeiling 4, so the live window keeps at most 4']], + [{ maxLiveImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }, + ['maxLiveImageBytes 3 MiB is above imageByteCeiling 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 8, imageCeiling: 4, maxLiveImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }, + ['maxLiveImages 8 is above imageCeiling 4', 'maxLiveImageBytes 3 MiB is above imageByteCeiling 2 MiB']], // A byte figure that isn't a whole number of MiB reads in bytes. - [{ maxLiveImageBytes: 3 * MiB + 1, maxRequestImageBytes: 2 * MiB }, [`maxLiveImageBytes ${3 * MiB + 1} bytes is above maxRequestImageBytes 2 MiB`]], - [{ maxRequestImages: Number.NaN }, ["maxRequestImages NaN isn't 0 or a finite number above it, so the default 500 applies"]], - [{ maxRequestImageBytes: -1 }, ["maxRequestImageBytes -1 isn't 0 or a finite number above it, so the default 24 MiB applies"]], + [{ maxLiveImageBytes: 3 * MiB + 1, imageByteCeiling: 2 * MiB }, [`maxLiveImageBytes ${3 * MiB + 1} bytes is above imageByteCeiling 2 MiB`]], + [{ imageCeiling: Number.NaN }, ["imageCeiling NaN isn't 0 or a finite number above it, so the default 500 applies"]], + [{ imageByteCeiling: -1 }, ["imageByteCeiling -1 isn't 0 or a finite number above it, so the default 24 MiB applies"]], [{ maxLiveImages: 0, maxLiveImageBytes: 0 }, null], - [{ maxLiveImages: 4, maxRequestImages: 4 }, null], + [{ maxLiveImages: 4, imageCeiling: 4 }, null], // At or above the memory strategy's 12 MiB compression budget, so // that budget isn't above the ceiling either. - [{ maxLiveImageBytes: 16 * MiB, maxRequestImageBytes: 16 * MiB }, null], + [{ maxLiveImageBytes: 16 * MiB, imageByteCeiling: 16 * MiB }, null], // A default above a lowered ceiling is named too, marked as the // default, and so is a default ceiling under a limit the host set. - [{ maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB, so the live window keeps at most 2 MiB']], - [{ maxLiveImages: 600 }, ['maxLiveImages 600 is above maxRequestImages 500 (default), so the live window keeps at most 500']], + [{ imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB, so the live window keeps at most 2 MiB']], + [{ maxLiveImages: 600 }, ['maxLiveImages 600 is above imageCeiling 500 (default), so the live window keeps at most 500']], // The default window of 6 comes from the strategy's default config, // not from the host, so it's marked too. - [{ maxRequestImages: 4 }, ['maxLiveImages 6 (default) is above maxRequestImages 4, so the live window keeps at most 4']], + [{ imageCeiling: 4 }, ['maxLiveImages 6 (default) is above imageCeiling 4, so the live window keeps at most 4']], // An infinite limit is none, like 0, so nothing is said. [{ maxLiveImages: Infinity }, null], // A malformed ceiling takes its default, which the later clause marks. - [{ maxLiveImages: 600, maxRequestImages: Number.NaN }, - ["maxRequestImages NaN isn't 0 or a finite number above it, so the default 500 applies", - 'maxLiveImages 600 is above maxRequestImages 500 (default)']], + [{ maxLiveImages: 600, imageCeiling: Number.NaN }, + ["imageCeiling NaN isn't 0 or a finite number above it, so the default 500 applies", + 'maxLiveImages 600 is above imageCeiling 500 (default)']], // A byte budget given as null takes its default, and is marked so. - [{ maxLiveImageBytes: null, maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB']], - [{ maxLiveImageBytes: undefined, maxRequestImageBytes: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above maxRequestImageBytes 2 MiB']], - [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: -1 }, - ["maxRequestImageBytes -1 isn't 0 or a finite number above it, so the default 24 MiB applies", - 'maxLiveImageBytes 30 MiB is above maxRequestImageBytes 24 MiB (default)']], - [{ maxLiveImages: 8, maxRequestImages: 0 }, null], - [{ maxLiveImageBytes: 30 * MiB, maxRequestImageBytes: 0 }, null], + [{ maxLiveImageBytes: null, imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB']], + [{ maxLiveImageBytes: undefined, imageByteCeiling: 2 * MiB }, ['maxLiveImageBytes 20 MiB (default) is above imageByteCeiling 2 MiB']], + [{ maxLiveImageBytes: 30 * MiB, imageByteCeiling: -1 }, + ["imageByteCeiling -1 isn't 0 or a finite number above it, so the default 24 MiB applies", + 'maxLiveImageBytes 30 MiB is above imageByteCeiling 24 MiB (default)']], + [{ maxLiveImages: 8, imageCeiling: 0 }, null], + [{ maxLiveImageBytes: 30 * MiB, imageByteCeiling: 0 }, null], ] as const; for (const [index, [options, said]] of cases.entries()) { lines.length = 0; @@ -322,7 +369,7 @@ describe("the memory strategy's warning names a compression budget above the byt await manager.close(); const ours = lines.filter((line) => line.includes('maxCompressionImageBytes')); assert.equal(ours.length, 1, JSON.stringify(lines)); - assert.ok(ours[0]!.includes('maxCompressionImageBytes 30 MiB is above maxRequestImageBytes 24 MiB (default), so a compression prompt keeps at most 24 MiB'), ours[0]); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 30 MiB is above imageByteCeiling 24 MiB (default), so a compression prompt keeps at most 24 MiB'), ours[0]); } finally { console.warn = original; rmSync(dir, { recursive: true, force: true }); @@ -338,12 +385,12 @@ describe("the memory strategy's warning names its default compression budget und const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); try { const manager = await ContextManager.open({ - path: join(dir, 'store'), strategy: memory({ maxLiveImageBytes: 1 * MiB, maxRequestImageBytes: 2 * MiB }), + path: join(dir, 'store'), strategy: memory({ maxLiveImageBytes: 1 * MiB, imageByteCeiling: 2 * MiB }), }); await manager.close(); const ours = lines.filter((line) => line.includes('is above')); assert.equal(ours.length, 1, JSON.stringify(lines)); - assert.ok(ours[0]!.includes('maxCompressionImageBytes 12 MiB (default) is above maxRequestImageBytes 2 MiB, so a compression prompt keeps at most 2 MiB'), ours[0]); + assert.ok(ours[0]!.includes('maxCompressionImageBytes 12 MiB (default) is above imageByteCeiling 2 MiB, so a compression prompt keeps at most 2 MiB'), ours[0]); assert.ok(!ours[0]!.includes('maxLiveImageBytes'), ours[0]); } finally { console.warn = original; @@ -352,7 +399,7 @@ describe("the memory strategy's warning names its default compression budget und }); }); -describe('the line when an image first falls past a request ceiling', () => { +describe('the line when an image first falls past an image ceiling', () => { /** Run `steps` on a fresh store: each adds images (`add`, of `bytes` each, * 4 when not given) and compiles. Returns the ceiling lines each step said, * in order (a new message can move the memory strategy's watermark, so a @@ -391,11 +438,11 @@ describe('the line when an image first falls past a request ceiling', () => { for (const [name, make] of Object.entries(STRATEGIES)) { it(`${name}: says once which ceiling an image fell past, and what it held`, async () => { const cases = [ - [{ maxLiveImages: 0, maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 0 (no limit) to it'], - [{ maxLiveImages: 5, maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 5 to it'], - [{ maxRequestImages: 2 }, 'maxRequestImages 2, which holds maxLiveImages 6 (default) to it'], - [{ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 8 }, - 'maxRequestImageBytes 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'], + [{ maxLiveImages: 0, imageCeiling: 2 }, 'imageCeiling 2, which holds maxLiveImages 0 (no limit) to it'], + [{ maxLiveImages: 5, imageCeiling: 2 }, 'imageCeiling 2, which holds maxLiveImages 5 to it'], + [{ imageCeiling: 2 }, 'imageCeiling 2, which holds maxLiveImages 6 (default) to it'], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 8 }, + 'imageByteCeiling 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'], ] as const; for (const [options, said] of cases) { // Three images, then two more: the ceiling strips at the first compile @@ -412,11 +459,11 @@ describe('the line when an image first falls past a request ceiling', () => { it(`${name}: says nothing when the host's own limit strips, or nothing falls past the ceiling`, async () => { for (const [options, steps, kept] of [ [{ maxLiveImages: 2 }, [3, 2], '...II'], - [{ maxLiveImages: 2, maxRequestImages: 2 }, [3, 2], '...II'], - [{ maxLiveImages: 0, maxRequestImages: 2 }, [1, 1], 'II'], + [{ maxLiveImages: 2, imageCeiling: 2 }, [3, 2], '...II'], + [{ maxLiveImages: 0, imageCeiling: 2 }, [1, 1], 'II'], // The host's own byte limit strips, under the default byte ceiling. [{ maxLiveImageBytes: 8 }, [3, 2], '...II'], - [{ maxLiveImageBytes: 8, maxRequestImageBytes: 8 }, [3, 2], '...II'], + [{ maxLiveImageBytes: 8, imageByteCeiling: 8 }, [3, 2], '...II'], ] as const) { const { lines, last } = await ceilingLines(make(options), [...steps]); assert.equal(last, kept, `the strip ran for ${JSON.stringify(options)}`); @@ -428,27 +475,51 @@ describe('the line when an image first falls past a request ceiling', () => { // Four-byte images fill the count ceiling of 3 first, at 12 of the 16 // bytes; then a 16-byte image fills the byte ceiling, and the next one // falls past it. - const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 3, maxRequestImageBytes: 16 }); + const strategy = make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 3, imageByteCeiling: 16 }); const { byStep, patterns } = await ceilingLines(strategy, [{ add: 4, bytes: 4 }, { add: 1, bytes: 16 }, { add: 1, bytes: 4 }]); // At the third step the byte walk drops the 16-byte image and, being // greedy, keeps two older 4-byte ones that still fit (as on main). assert.deepEqual(patterns, ['.III', '....I', '..II.I'], 'the count, then the bytes, strip'); assert.deepEqual(byStep.map((step) => step.length), [1, 1, 0], JSON.stringify(byStep)); - assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImages 3'), byStep[0]![0]); - assert.ok(byStep[1]![0]!.includes('an image fell past maxRequestImageBytes 16 bytes'), byStep[1]![0]); + assert.ok(byStep[0]![0]!.includes('an image fell past imageCeiling 3'), byStep[0]![0]); + assert.ok(byStep[1]![0]!.includes('an image fell past imageByteCeiling 16 bytes'), byStep[1]![0]); }); it(`${name}: says it for a single image bigger than the byte ceiling`, async () => { - const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, maxRequestImageBytes: 8 }), [{ add: 1, bytes: 12 }]); + const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }), [{ add: 1, bytes: 12 }]); assert.deepEqual(patterns, ['.'], 'the one image is past the ceiling on its own'); assert.equal(byStep[0]!.length, 1, JSON.stringify(byStep)); - assert.ok(byStep[0]![0]!.includes('an image fell past maxRequestImageBytes 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), byStep[0]![0]); + assert.ok(byStep[0]![0]!.includes('an image fell past imageByteCeiling 8 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), byStep[0]![0]); }); - it(`${name}: says nothing from a dry compile, and leaves the line for the first compile that's sent`, async () => { + it(`${name}: says nothing when the depth limit strips the image the ceiling would, and says it when only the ceiling does`, async () => { + // Two 8-byte images under a 12-byte ceiling: the older one is past the + // ceiling. With a depth limit that the older one is also past, the + // depth strips it with or without the ceiling, so the ceiling decided + // nothing (Anarchid's probe on #268, R1-4). + const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500 }; + const alone = await ceilingLines(make({ ...depth, imageByteCeiling: 0 }), [{ add: 2, bytes: 8 }]); + assert.equal(alone.last, '.I', 'the depth limit alone strips the older image'); + const both = await ceilingLines(make({ ...depth, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(both.last, '.I', 'the ceiling changes nothing the window keeps'); + assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); + const ceilingOnly = await ceilingLines(make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); + assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); + }); + + it(`${name}: says nothing when the host's own count limit strips the image the byte ceiling would`, async () => { + // The older image is past the host's maxLiveImages of 1 and past the + // 12-byte ceiling: the host's limit strips it whatever the ceiling. + const { lines, last } = await ceilingLines(make({ maxLiveImages: 1, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); + assert.equal(last, '.I'); + assert.deepEqual(lines, [], JSON.stringify(lines)); + }); + + it(`${name}: says nothing from a dry compile, and leaves the line for the first compile select returns`, async () => { for (const [options, said0] of [ - [{ maxLiveImages: 0, maxRequestImages: 2 }, 'an image fell past maxRequestImages 2'], - [{ maxLiveImageBytes: 0, maxRequestImageBytes: 8 }, 'an image fell past maxRequestImageBytes 8 bytes'], + [{ maxLiveImages: 0, imageCeiling: 2 }, 'an image fell past imageCeiling 2'], + [{ maxLiveImageBytes: 0, imageByteCeiling: 8 }, 'an image fell past imageByteCeiling 8 bytes'], ] as const) { const strategy = make(options); const said: string[] = []; @@ -463,7 +534,7 @@ describe('the line when an image first falls past a request ceiling', () => { assert.equal(pattern(dry.messages, 3), '.II', `the dry compile strips for ${JSON.stringify(options)}`); assert.deepEqual(ours(), [], `a dry compile says nothing for ${JSON.stringify(options)}`); await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - assert.equal(ours().length, 1, `the sent compile says it for ${JSON.stringify(options)}`); + assert.equal(ours().length, 1, `the compile returned says it for ${JSON.stringify(options)}`); assert.ok(ours()[0]!.includes(said0), ours()[0]); } finally { console.warn = original; @@ -476,18 +547,18 @@ describe('the line when an image first falls past a request ceiling', () => { it('the memory strategy says it when its hysteresis watermark trims against a ceiling', async () => { const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), [5], ); assert.equal(last, '...II', 'one trim to half the ceiling'); assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('an image fell past maxRequestImages 4, which holds maxLiveImages 0 (no limit) to it'), lines[0]); + assert.ok(lines[0]!.includes('an image fell past imageCeiling 4, which holds maxLiveImages 0 (no limit) to it'), lines[0]); }); it('the memory strategy says it from its adaptive selection too, and nothing from a preview', async () => { const adaptive = memory({ hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, - autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 2, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 2, }); const said: string[] = []; const original = console.warn; @@ -497,13 +568,13 @@ describe('the line when an image first falls past a request ceiling', () => { const ours = () => said.filter((line) => line.includes('an image fell past')); try { for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 1 }, { render: true })?.entries ?? []; assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); assert.deepEqual(ours(), [], 'a preview, with its own ceiling, says nothing'); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 3), '.II'); assert.equal(ours().length, 1, JSON.stringify(said)); - assert.ok(ours()[0]!.includes('an image fell past maxRequestImages 2, which holds maxLiveImages 0 (no limit) to it'), ours()[0]); + assert.ok(ours()[0]!.includes('an image fell past imageCeiling 2, which holds maxLiveImages 0 (no limit) to it'), ours()[0]); } finally { console.warn = original; await manager.close(); @@ -511,10 +582,10 @@ describe('the line when an image first falls past a request ceiling', () => { } }); - it("the memory strategy's strip leaves the line only from a render that's sent, not a layout rendered to compare against", async () => { + it("the memory strategy's strip leaves the line only from a render select returns, not a layout rendered to compare against", async () => { for (const options of [ - { maxLiveImages: 0, maxRequestImages: 1 }, - { maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 4 }, + { maxLiveImages: 0, imageCeiling: 1 }, + { maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, ]) { const strategy = memory(options); const said: string[] = []; @@ -532,7 +603,7 @@ describe('the line when an image first falls past a request ceiling', () => { index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content, })); const strip = strategy as unknown as { - applyImageStripping(entries: unknown[], store: unknown, opts?: { sent?: boolean }): void; + applyImageStripping(entries: unknown[], store: unknown, opts?: { returned?: boolean }): void; reportCeilingHits(): void; }; const kept = entries(); @@ -540,7 +611,7 @@ describe('the line when an image first falls past a request ceiling', () => { assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling replaced one'); strip.reportCeilingHits(); assert.deepEqual(ours(), [], `a layout rendered to compare against leaves nothing for ${JSON.stringify(options)}`); - strip.applyImageStripping(entries(), view, { sent: true }); + strip.applyImageStripping(entries(), view, { returned: true }); assert.deepEqual(ours(), [], 'the strip itself says nothing: select reports once it returns the render'); strip.reportCeilingHits(); assert.equal(ours().length, 1, JSON.stringify(said)); @@ -555,7 +626,7 @@ describe('the line when an image first falls past a request ceiling', () => { it('the memory strategy says nothing for a select it refuses, and says it for the next one it returns', async () => { const adaptive = memory({ hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, - autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 2, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 2, }); const said: string[] = []; const original = console.warn; @@ -588,7 +659,7 @@ describe('the line when an image first falls past a request ceiling', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), }); const ours = () => said.filter((line) => line.includes('an image fell past')); try { @@ -598,7 +669,7 @@ describe('the line when an image first falls past a request ceiling', () => { const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 5), '...II', 'one trim to half the ceiling'); assert.equal(ours().length, 1, JSON.stringify(said)); - assert.ok(ours()[0]!.includes('an image fell past maxRequestImages 4'), ours()[0]); + assert.ok(ours()[0]!.includes('an image fell past imageCeiling 4'), ours()[0]); } finally { console.warn = original; await manager.close(); @@ -613,7 +684,7 @@ describe('the line when an image first falls past a request ceiling', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const strategy = memory({ hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, - autoTickOnNewMessage: false, maxLiveImages: 0, maxRequestImages: 10, imageStripHysteresisRatio: 0.5, + autoTickOnNewMessage: false, maxLiveImages: 0, imageCeiling: 10, imageStripHysteresisRatio: 0.5, }); const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); const ours = () => said.filter((line) => line.includes('an image fell past')); @@ -621,7 +692,7 @@ describe('the line when an image first falls past a request ceiling', () => { for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); // A preview with a ceiling of 2 moves the watermark, for itself only: // its trim to half keeps one image, where the count alone keeps two. - const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 2 }, { render: true })?.entries ?? []; + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 2 }, { render: true })?.entries ?? []; assert.equal(pattern(entries, 4), '...I', "the preview's own watermark trimmed to half its ceiling"); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 4), 'IIII', 'the real ceiling of 10 strips nothing'); @@ -643,13 +714,13 @@ describe('the line when an image first falls past a request ceiling', () => { path: join(dir, 'store'), strategy: memory({ hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, - autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0, + autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0, }), }); const ours = () => said.filter((line) => line.includes('an image fell past')); try { for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { maxRequestImages: 1 }, { render: true })?.entries ?? []; + const entries = manager.previewContext({ maxTokens: 1_000_000, reserveForResponse: 0 }, { imageCeiling: 1 }, { render: true })?.entries ?? []; assert.equal(pattern(entries, 3), '..I', 'the preview strips to its own ceiling'); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 3), 'III', 'with no limits, the compile keeps every image'); @@ -661,6 +732,43 @@ describe('the line when an image first falls past a request ceiling', () => { } }); + it('the memory strategy leaves an image below a watermark it loaded to the watermark, and says it when its own walk meets the ceiling', async () => { + // One instance's ceiling moves the watermark and says so. A second + // instance loads that watermark, which carries no record of what moved it. + // When the images above it exactly fill the ceiling, an image below it is + // past the ceiling too, but the watermark is what strips it, so the second + // instance says nothing until its own walk meets the ceiling. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const options = { maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }; + const ours = () => said.filter((line) => line.includes('an image fell past')); + const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + try { + const first = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + for (let i = 1; i <= 5; i++) first.addMessage('user', imageMessage(`img-${i}`)); + assert.equal(pattern((await first.compile(budget)).messages, 5), '...II', 'the ceiling trims to half of 4'); + assert.equal(ours().length, 1, JSON.stringify(said)); + await first.close(); + said.length = 0; + const second = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); + try { + for (let i = 6; i <= 7; i++) second.addMessage('user', imageMessage(`img-${i}`)); + assert.equal(pattern((await second.compile(budget)).messages, 7), '...IIII', 'four above the loaded watermark fill the ceiling'); + assert.deepEqual(ours(), [], JSON.stringify(said)); + second.addMessage('user', imageMessage('img-8')); + assert.equal(pattern((await second.compile(budget)).messages, 8), '......II', "the second instance's own walk meets the ceiling"); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + await second.close(); + } + } finally { + console.warn = original; + rmSync(dir, { recursive: true, force: true }); + } + }); + it("the memory strategy leaves a watermark's line with the branch it moved on", async () => { const said: string[] = []; const original = console.warn; @@ -668,7 +776,7 @@ describe('the line when an image first falls past a request ceiling', () => { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, maxRequestImages: 4, imageStripHysteresisRatio: 0.5 }), + strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), }); const ours = () => said.filter((line) => line.includes('an image fell past')); try { @@ -694,11 +802,23 @@ describe('the line when an image first falls past a request ceiling', () => { assert.deepEqual(lines, []); }); + it("the memory strategy's watermark says nothing when the depth limit moves it where the ceiling also would", async () => { + // Three 8-byte images under a 20-byte ceiling: the oldest is past the + // ceiling (24 bytes) and, two image messages deep, past the depth limit + // too, so the depth moves the watermark with or without the ceiling. + const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500, imageStripHysteresisRatio: 0.5 }; + const alone = await ceilingLines(memory({ ...depth, imageByteCeiling: 0 }), [{ add: 3, bytes: 8 }]); + const both = await ceilingLines(memory({ ...depth, imageByteCeiling: 20 }), [{ add: 3, bytes: 8 }]); + assert.equal(both.last, alone.last, 'the ceiling changes nothing the window keeps'); + assert.equal(both.last, '..I'); + assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); + }); + it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { - const auto = await ceilingLines(memory({ maxLiveImages: undefined, maxRequestImages: 2 }), [3]); + const auto = await ceilingLines(memory({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); assert.ok(auto.lines[0]!.includes('which holds maxLiveImages 0 (no limit) to it,'), auto.lines[0]); - const win = await ceilingLines(windowed({ maxLiveImages: undefined, maxRequestImages: 2 }), [3]); + const win = await ceilingLines(windowed({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); assert.equal(win.lines.length, 1, JSON.stringify(win.lines)); assert.ok(win.lines[0]!.includes('which holds maxLiveImages 6 (default) to it,'), win.lines[0]); }); @@ -706,7 +826,7 @@ describe('the line when an image first falls past a request ceiling', () => { it("doesn't mark a value a subclass's enforced layer gives as a default", async () => { class Enforced extends AutobiographicalStrategy { protected override enforcedConfigLayer() { - return { source: 'enforced', values: { maxRequestImages: 2 } }; + return { source: 'enforced', values: { imageCeiling: 2 } }; } } const { lines } = await ceilingLines( @@ -714,36 +834,36 @@ describe('the line when an image first falls past a request ceiling', () => { [3], ); assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('an image fell past maxRequestImages 2, which holds'), lines[0]); + assert.ok(lines[0]!.includes('an image fell past imageCeiling 2, which holds'), lines[0]); }); it('ceilingBoundLine marks a default or malformed ceiling, and gives bytes in MiB', () => { const configured = (o: Record) => - ({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImages: 500, maxRequestImageBytes: 24 * MiB, ...o }); - assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['maxRequestImages'])), - '[s] an image fell past maxRequestImages 500 (default), which holds maxLiveImages 0 (no limit) to it, so images past it are now shown as placeholders'); - assert.equal(ceilingBoundLine('s', 'images', configured({ maxRequestImages: Number.NaN, maxLiveImages: 800 }), new Set()), - '[s] an image fell past maxRequestImages 500 (default), which holds maxLiveImages 800 to it, so images past it are now shown as placeholders'); + ({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...o }); + assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['imageCeiling'])), + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 0 (no limit) to it, so images past it are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'images', configured({ imageCeiling: Number.NaN, maxLiveImages: 800 }), new Set()), + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 800 to it, so images past it are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB }), new Set()), - '[s] an image fell past maxRequestImageBytes 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so images past it are now shown as placeholders'); - assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['maxRequestImageBytes'])), - `[s] an image fell past maxRequestImageBytes 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so images past it are now shown as placeholders`); + '[s] an image fell past imageByteCeiling 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so images past it are now shown as placeholders'); + assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['imageByteCeiling'])), + `[s] an image fell past imageByteCeiling 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so images past it are now shown as placeholders`); }); it('the memory strategy says it when its hysteresis watermark trims against the byte ceiling', async () => { const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxLiveImageBytes: 0, maxRequestImageBytes: 16, imageStripHysteresisRatio: 0.5 }), + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), [5], ); assert.equal(last, '...II', 'one trim to half the ceiling'); assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('an image fell past maxRequestImageBytes 16 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); + assert.ok(lines[0]!.includes('an image fell past imageByteCeiling 16 bytes, which holds maxLiveImageBytes 0 (no limit) to it'), lines[0]); }); }); -describe('compression prompts within the request ceilings', () => { - it('carry at most maxRequestImages, the newest messages first', () => { - const prepared = new Prepared({ maxRequestImages: 2 }).prepare( +describe('compression prompts within the image ceilings', () => { + it('carry at most imageCeiling, the newest messages first', () => { + const prepared = new Prepared({ imageCeiling: 2 }).prepare( request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), ); assert.equal(prepared.imageReplacements, 1); @@ -754,7 +874,7 @@ describe('compression prompts within the request ceilings', () => { }); it('count an image nested in a tool result toward the ceiling', () => { - const prepared = new Prepared({ maxRequestImages: 1 }).prepare(request( + const prepared = new Prepared({ imageCeiling: 1 }).prepare(request( [imageBlock('AAAA')], [{ type: 'tool_result', toolUseId: 't', content: [imageBlock('BBBB')] } as ContentBlock], )); @@ -765,39 +885,48 @@ describe('compression prompts within the request ceilings', () => { assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER, 'and the older one is replaced'); }); - it('keep their byte budget within maxRequestImageBytes, and the images kept are what it lets through', () => { - const prepared = new Prepared({ maxCompressionImageBytes: 3 * MiB, maxRequestImageBytes: 2 * MiB }).prepare(request([])); + it('count a generated image toward the ceiling, so a newer one takes the last slot from an older image', () => { + const generated = { type: 'generated_image', data: 'BBBB', mimeType: 'image/png' } as ContentBlock; + const prepared = new Prepared({ imageCeiling: 1 }).prepare(request([imageBlock('AAAA')], [generated])); + assert.equal(prepared.imageReplacements, 1); + assert.equal(prepared.imagesKept, 1); + assert.equal(prepared.request.messages[1]!.content[0]!.type, 'generated_image', 'the newer, generated image takes the one slot'); + assert.equal((prepared.request.messages[0]!.content[0] as { text: string }).text, PLACEHOLDER, 'and the older one is replaced'); + }); + + it('keep their byte budget within imageByteCeiling, and the images kept are what it lets through', () => { + const prepared = new Prepared({ maxCompressionImageBytes: 3 * MiB, imageByteCeiling: 2 * MiB }).prepare(request([])); assert.equal(prepared.imageBudgetBytes, 2 * MiB); for (const none of [0, Infinity, -3, Number.NaN]) { - assert.equal(new Prepared({ maxCompressionImageBytes: none, maxRequestImageBytes: 8 }).prepare(request([])).imageBudgetBytes, 8, + assert.equal(new Prepared({ maxCompressionImageBytes: none, imageByteCeiling: 8 }).prepare(request([])).imageBudgetBytes, 8, `a budget of ${none} is none, so up to the byte ceiling`); } - assert.equal(new Prepared({ maxCompressionImageBytes: 8.5, maxRequestImageBytes: 100 }).prepare(request([])).imageBudgetBytes, 8, + assert.equal(new Prepared({ maxCompressionImageBytes: 8.5, imageByteCeiling: 100 }).prepare(request([])).imageBudgetBytes, 8, 'a fractional budget is floored'); - const walked = new Prepared({ maxCompressionImageBytes: 12, maxRequestImageBytes: 8 }).prepare( + const walked = new Prepared({ maxCompressionImageBytes: 12, imageByteCeiling: 8 }).prepare( request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]), ); assert.equal(walked.imageBudgetBytes, 8); assert.equal(walked.imageReplacements, 1, 'under the clamped 8 bytes, two 4-byte images fit and the oldest does not'); assert.equal(walked.imageBytesKept, 8); assert.deepEqual(walked.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image'], 'the oldest is the one replaced'); - assert.equal(new Prepared({ maxCompressionImageBytes: 0, maxRequestImageBytes: 2 * MiB }).prepare(request([])).imageBudgetBytes, + assert.equal(new Prepared({ maxCompressionImageBytes: 0, imageByteCeiling: 2 * MiB }).prepare(request([])).imageBudgetBytes, 2 * MiB, 'an unlimited budget means up to the ceiling'); assert.equal(new Prepared().prepare(request([])).imageBudgetBytes, 12 * MiB, 'the default budget is under the default ceiling'); }); - it('carry at most maxRequestImages with no byte budget, and say only the ceiling', () => { - const strategy = new Prepared({ maxCompressionImageBytes: 0, maxRequestImageBytes: 0, maxRequestImages: 1 }); + it('carry at most imageCeiling with no byte budget, and say only the ceiling', () => { + const strategy = new Prepared({ maxCompressionImageBytes: 0, imageByteCeiling: 0, imageCeiling: 1 }); const input = request([imageBlock('AAAA')], [imageBlock('BBBB')]); assert.equal(strategy.prepare(input).imageReplacements, 1); assert.deepEqual(captureErrors(() => strategy.send(input, 'no-budget')), [ '[autobiographical] compression prompt: replaced 1 image(s) with placeholders to stay under the ' + - '1-image request ceiling (kept 1 image(s), 0.0MiB, newest messages first) [no-budget]', + '1-image ceiling (kept 1 image(s), 0.0MiB, newest messages first) [no-budget]', ]); }); it('with no count ceiling, keep to the byte budget, and the log says only the budget', () => { - const strategy = new Prepared({ maxCompressionImageBytes: 8, maxRequestImages: 0 }); + const strategy = new Prepared({ maxCompressionImageBytes: 8, imageCeiling: 0 }); const input = request([imageBlock('AAAA')], [imageBlock('BBBB')], [imageBlock('CCCC')]); const prepared = strategy.prepare(input); assert.equal(prepared.imageCountCap, 0); @@ -805,13 +934,13 @@ describe('compression prompts within the request ceilings', () => { assert.deepEqual(prepared.request.messages.map((m) => m.content[0]!.type), ['text', 'image', 'image']); const logs = captureErrors(() => strategy.send(input, 'no-count')); assert.equal(logs.length, 1); - assert.ok(!logs[0]!.includes('request ceiling'), logs[0]); + assert.ok(!logs[0]!.includes('image ceiling'), logs[0]); assert.ok(logs[0]!.includes('inline image budget (kept 2 image(s)'), logs[0]); }); it('with no budget and no ceilings, keep every image, more than the default ceiling of 500', () => { const images = Array.from({ length: 501 }, () => [imageBlock('AAAA')]); - const prepared = new Prepared({ maxCompressionImageBytes: 0, maxRequestImages: 0, maxRequestImageBytes: 0 }).prepare( + const prepared = new Prepared({ maxCompressionImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0 }).prepare( request(...images), ); assert.equal(prepared.imageReplacements, 0); From ae432a341427822dad5283ed6007e1ce7257cf70 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 00:03:02 -0400 Subject: [PATCH 07/67] docs(strategy): renderAsNewest's tool pass only pairs; pruning reaches it through cappedForSend Since tool pruning is priced as sent, the tool pass a compile runs between the cut and the image pass (postProcessToolEntries) only repairs pairing, and what sending prunes from a message's tool blocks is applied in cappedForSend, which renderAsNewest runs for each stored copy. Its doc said the pass it leaves out also prunes. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 532a8e15..f699fb5f 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -18659,7 +18659,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * select sends each shard as its own message), then `applyImageStripping`, * so a message stored after the batch counts toward the images as in a * compile. The tool pass that comes between them (postProcessToolEntries) - * is left out: it pairs and prunes by the window around these messages. + * is left out: it pairs by the window around these messages. What sending + * prunes from a message's tool blocks reaches it through `cappedForSend`. * * Those messages are the recent window's, and the head window's too when * no message comes between the two: a compile places the middle's From ec108e8df61d34b57f731b104ac60704ff2b93f9 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 00:54:38 -0400 Subject: [PATCH 08/67] fix(images): a ceiling decides a watermark move only where it lands it newer; the docs say where membrane checks The memory strategy's hysteresis watermark credited a ceiling with a move whenever the first image past a limit passed that ceiling and wasn't past the depth by the watermark's own measure, which counts only newer messages. The strips count the image's own message too, so a move the depth limit would make anyway, to the same place, could say the line, and a ceiling whose trim landed the watermark newer than the depth's could be missed. The watermark now works out where it would land with only the limits the host has in force, and credits each ceiling whose limit its walks passed when its own landing is newer. The docs say where membrane checks its byte cap, and that a compression request, which sets shedOversizeImages, has its oldest images shed rather than being refused. They say the line comes from a render a select returns, that a fraction is floored to at least 1, and what the window and a compression prompt each count. The breaking fragment's Unchanged bullet names the compression count cap and the fractional floor, and its account of a negative, Infinity or NaN limit is per value. Tests: the watermark at the depth boundary, with and without a move the depth would make anyway; a byte ceiling whose trim lands it newer; the kind credited when the host's own count limit misfits first; a trim that stops at the depth; the branch test under the byte ceiling; renderAsNewest under a binding ceiling, for both strategies; the watermark read where a test says it moved; and imageCeilings typed through ContextStrategy. Co-Authored-By: Claude Opus 5.5 --- .../request-image-ceilings.breaking.md | 4 +- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 39 ++-- src/strategies/autobiographical.ts | 76 +++--- src/types/strategy.ts | 35 +-- test/request-image-ceilings.test.ts | 218 +++++++++++++++--- 6 files changed, 280 insertions(+), 94 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index 90ba4b76..f9dafd30 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative, `Infinity`, or `NaN`, which had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps the ceiling's worth, and its strategy says once, the first time a ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once, the first time a ceiling decides that an image in a render a select returns is shown as a placeholder, which ceiling it was and what it held. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.1 depends on `^0.14.1`, and connectome-host 0.10.2 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. - - **Unchanged:** for a host that sets none of these, the default window (6 images and 20 MiB) and the default compression budget (12 MiB) are under the default ceilings, so what it keeps doesn't change. A limit above 0 and under its ceiling works as before. + - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 6c8e50dd..8ef7811e 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time that ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. Only a render a select returns says it, and not when the depth limit, the hysteresis watermark or a limit the host set strips the image anyway. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time that ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. Only a render a select returns says it, and not when the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips the image anyway. A move of the watermark counts as a ceiling's only when the host's own limits wouldn't move it. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 048836f6..eea60a91 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -7,11 +7,12 @@ * ceilings (`imageCeiling`, `imageByteCeiling`) bound those limits: an * unlimited (0) limit means "up to the ceiling", and a limit above the * ceiling keeps the ceiling's worth. They bound what the window counts and - * what a compression prompt carries, not a whole request: images nested in - * tool results, generated images, and whatever a caller injects at compile - * time or appends to a compiled request aren't counted here. A caller that - * wants a request within the ceilings counts that request's images itself - * and compares them with the strategy's `imageCeilings()`. + * what a compression prompt carries, not a whole request. In the window, + * images nested in tool results and generated images aren't counted, nor is + * anything a caller injects at compile time or appends to a compiled + * request; a compression prompt counts every image it carries. A caller + * that wants a request within the ceilings counts that request's images + * itself and compares them with the strategy's `imageCeilings()`. */ /** @@ -24,15 +25,18 @@ export const DEFAULT_IMAGE_CEILING = 500; /** - * membrane refuses, before sending it, a request whose serialized messages - * exceed its byte cap: 28 MiB unless `MEMBRANE_MAX_REQUEST_BYTES` says - * otherwise (`assertWithinByteBudget`, membrane 0.5.87 and 0.6.0). It checks - * when it builds a request through `transformRequest`, which a call's first - * request and each native tool-loop round go through; an XML-mode - * continuation round doesn't, and of the provider adapters only the - * Anthropic adapter checks again, on every request it sends. 24 MiB of base64 - * images leaves 4 MiB of that default cap for everything else the messages - * carry: their text, their JSON, and any documents or other media. + * membrane's byte cap is 28 MiB of serialized messages unless + * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise (membrane 0.5.87 and 0.6.0). + * It checks a request when it builds it through `transformRequest`, which a + * call's first request and each native tool-loop round go through; an + * XML-mode continuation round doesn't, and of the provider adapters only the + * Anthropic adapter checks again, on every request it sends. Over the cap, + * `transformRequest` refuses the request before sending it + * (`assertWithinByteBudget`), unless the request sets `shedOversizeImages`, + * as this package's compression requests do, and then it replaces the oldest + * images until it fits. 24 MiB of base64 images leaves 4 MiB of that + * default cap for everything else the messages carry: their text, their + * JSON, and any documents or other media. */ export const DEFAULT_IMAGE_BYTE_CEILING = 24 * 1024 * 1024; @@ -151,8 +155,11 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b * see ceilingSetLimits), when nothing else strips it. Something else strips * it when `strippedOtherwise` says so, as the host's depth limit or the * memory strategy's hysteresis watermark does, or when it also passes a - * count or byte limit the host set itself. Only a ceiling that decides - * changes what the window keeps, so only that one says its line. + * count or byte limit the host set itself and has in force, under its + * ceiling. A limit the host set above its ceiling isn't in force: the + * ceiling holds it, and the warning at initialize says so. Only a ceiling + * that decides changes what the window keeps, so only that one says its + * line. */ export function ceilingsDeciding( over: { images: boolean; bytes: boolean }, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index f699fb5f..63bd7fc9 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -7519,7 +7519,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the request before it. const prevCacheKeys = this._prevCacheKeys; // What this select's strips record is what it reports. A dry select's - // strips, or a refused one's, record too, and are dropped here. + // strips, and a refused one's, record too; each live select starts by + // clearing what earlier ones left, so it reports its own render's. this._ceilingHitsThisRender = { images: false, bytes: false }; let entries: ContextEntry[]; try { @@ -15279,8 +15280,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // topic and isn't a lever the list names (OverBudgetClearedBy). // Newer messages can shrink a head that holds images, since the // image strip takes the oldest images first (maxLiveImages, - // maxLiveImageBytes, imageStripDepthTokens), but whether they'd - // strip enough isn't known, so nothing is named for that either. + // maxLiveImageBytes, each within its image ceiling, and + // imageStripDepthTokens), but whether they'd strip enough isn't + // known, so nothing is named for that either. clearedBy: [], diagnostics: { middleCountUnit: 'selection-items', @@ -16559,7 +16561,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the count, byte and depth limits (the byte rule is greedy, so a large * newest image can be replaced while smaller older ones stay), the * hysteresis watermark, which strips below them, and the compression - * request's own image byte cap. Windowed passthrough uses the same text. */ + * request's own image byte and count caps. Windowed passthrough uses the + * same text. */ private static readonly IMAGE_PLACEHOLDER = '[image not shown: this context keeps only some images]'; /** Post-pass over compiled entries: replace image blocks with a text @@ -16695,7 +16698,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Walk newest-first with the given limits; return the sequence of the // first message holding an image that does not fit (or undefined), and // which limits it passes. - let misfitBy = { depth: false, images: false, bytes: false }; + let misfitBy = { images: false, bytes: false }; const firstMisfit = (depthCap: number, countCap: number, bytesCap: number): number | undefined => { let depth = 0, count = 0, bytes = 0; for (let i = messages.length - 1; i >= 0; i--) { @@ -16705,11 +16708,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (b.type !== 'image') continue; const size = AutobiographicalStrategy.imageBlockBytes(b); misfitBy = { - depth: depthCap > 0 && depth > depthCap, images: countCap > 0 && count >= countCap, bytes: bytesCap > 0 && bytes + size > bytesCap, }; - if (misfitBy.depth || misfitBy.images || misfitBy.bytes) return m.sequence; + if ((depthCap > 0 && depth > depthCap) || misfitBy.images || misfitBy.bytes) return m.sequence; count++; bytes += size; } @@ -16717,22 +16719,37 @@ export class AutobiographicalStrategy implements ResettableStrategy { } return undefined; }; - if (firstMisfit(stripDepth, maxLive, maxLiveBytes) === undefined) return; // no limit binds - // A ceiling that decides the watermark's move strips every later render - // by it, so the next render a select returns reports it - // (reportCeilingHits). One the depth limit or a limit the host set also - // forces isn't the ceiling's. A dry advance is undone, so it leaves nothing. + // Where the watermark lands for given limits: past the first misfit, at + // the first one of the trim to ratio × each limit, or undefined if no + // limit binds. `passed` collects the limits the two walks' misfits pass. + const passed = { images: false, bytes: false }; + const landing = (countCap: number, bytesCap: number, note: boolean): number | undefined => { + if (firstMisfit(stripDepth, countCap, bytesCap) === undefined) return undefined; + if (note) { passed.images ||= misfitBy.images; passed.bytes ||= misfitBy.bytes; } + const at = firstMisfit( + stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, + countCap > 0 ? Math.max(0, Math.floor(countCap * ratio)) || 1 : 0, + bytesCap > 0 ? Math.floor(bytesCap * ratio) : 0, + ); + if (note) { passed.images ||= misfitBy.images; passed.bytes ||= misfitBy.bytes; } + return at; + }; + const next = landing(maxLive, maxLiveBytes, true); + if (next === undefined || (w !== undefined && next <= w)) return; // no limit binds, or no advance + // The ceilings decide this move when it lands newer, so strips more, than + // the limits the host has in force would land it: its depth limit, and a + // count or byte limit no ceiling sets (ceilingSetLimits). The move then + // strips every later render by it, so the next render a select returns + // reports each ceiling whose limit the walks passed (reportCeilingHits). + // A dry advance is undone, so it leaves nothing. if (!this._dryRunSelect) { - const deciding = ceilingsDeciding(misfitBy, misfitBy.depth, ceilingSetLimits(this.configuredImageLimits())); - if (deciding.images) this._ceilingMovedWatermark.images = true; - if (deciding.bytes) this._ceilingMovedWatermark.bytes = true; - } - const next = firstMisfit( - stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, - maxLive > 0 ? Math.max(0, Math.floor(maxLive * ratio)) || 1 : 0, - maxLiveBytes > 0 ? Math.floor(maxLiveBytes * ratio) : 0, - ); - if (next === undefined || (w !== undefined && next <= w)) return; + const set = ceilingSetLimits(this.configuredImageLimits()); + const own = landing(set.images ? 0 : maxLive, set.bytes ? 0 : maxLiveBytes, false); + if (own === undefined || next > own) { + if (set.images && passed.images) this._ceilingMovedWatermark.images = true; + if (set.bytes && passed.bytes) this._ceilingMovedWatermark.bytes = true; + } + } this._imageWatermark = next; // Dry selections and observer reconstructions advance it locally only; // their finally blocks restore it together with the post-strip memo. @@ -16870,12 +16887,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Say once for each ceiling, when a select returns a render in which an * image ceiling, rather than a limit the host set, decided that an image - * is stripped (see ceilingsDeciding), directly or through a watermark it - * moved on this branch while this instance ran. A watermark loaded from the store - * carries no record of what moved it, so a later instance says it when - * its own walk next meets the ceiling. A dry select, a preview, an - * observer's reconstruction or a refused select says nothing, and the - * line waits for the first render returned. */ + * is stripped (see ceilingsDeciding), or the first render a select + * returns after a ceiling decided a move of the watermark on this branch + * while this instance held it (maybeAdvanceImageWatermark). That render + * needn't hold an image the move stripped: the line says that the ceiling + * acted, and when. A watermark loaded from the store, by a later instance + * or a reload of the branch, carries no record of what moved it, so the + * line waits until this instance's own walk meets the ceiling. A dry + * select, a preview, an observer's reconstruction or a refused select says + * nothing, and the line waits for the first render returned. */ private reportCeilingHits(): void { for (const kind of ['images', 'bytes'] as const) { const hit = this._ceilingHitsThisRender[kind] || this._ceilingMovedWatermark[kind]; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index d8e1d98a..e2eb2f3e 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1229,7 +1229,7 @@ export interface AutobiographicalConfig { * ceilings recompiles at each image it adds; at about 1,600 tokens an * image, the token budget binds first unless the images are small. A * fraction is floored. - * A hard ceiling for dense bursts that pack many images into the live window. + * A hard limit for dense bursts that pack many images into the live window. * The window counts top-level `image` blocks, and measures their bytes by * the length of their base64 `source.data`. So an image nested in a tool * result and a `generated_image` are neither counted nor replaced, and an @@ -1306,9 +1306,10 @@ export interface AutobiographicalConfig { * 2026-10-13, and again on 2026-10-19), and the default, 500, leaves room * for what this doesn't count. 0 = no ceiling. A value that isn't 0 or a * finite number above it takes the default, and the strategy's warning at - * initialize names it. A fraction is floored. Under a window limit of 0 or - * above this ceiling, the strategy says once, the first time this ceiling - * decides that an image is shown as a placeholder. + * initialize names it. A fraction is floored, to at least 1. Under a + * window limit of 0 or above this ceiling, the strategy says once, the + * first time this ceiling decides that an image in a render a select + * returns is shown as a placeholder. */ imageCeiling?: number; @@ -1321,17 +1322,21 @@ export interface AutobiographicalConfig { * * It doesn't bound a request's bytes: the messages' text and JSON, * documents, the images the window doesn't count, and anything a caller - * injects or appends aren't counted. membrane refuses, before sending it, a - * request whose serialized messages exceed its byte cap, 28 MiB unless - * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, and the default, 24 MiB, - * leaves 4 MiB of that cap for what this doesn't count. It doesn't follow - * `MEMBRANE_MAX_REQUEST_BYTES`, so a host that changes membrane's cap - * changes this with it, keeping room under the cap for what this doesn't - * count. `imageCeilings()` reports it, as for `imageCeiling`. 0 = no - * ceiling. A value that isn't 0 or a finite number above it takes the - * default. Under a window limit of 0 or above this ceiling, the strategy - * says once when this ceiling first decides that an image is shown as a - * placeholder, as for `imageCeiling`. + * injects or appends aren't counted. membrane's byte cap is 28 MiB of + * serialized messages unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, + * and the default, 24 MiB, leaves 4 MiB of that cap for what this doesn't + * count. Where membrane checks a request against its cap, it refuses one + * over it before sending, or sheds its oldest images when the request asks + * it to, as this strategy's compression requests do (see + * `src/live-image-limits.ts` for where it checks). This setting isn't read + * from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes membrane's cap + * sets this too, keeping room under the cap for what this doesn't count. + * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A + * value that isn't 0 or a finite number above it takes the default, and a + * fraction is floored, to at least 1. Under a window limit of 0 or above + * this ceiling, the strategy says once when this ceiling first decides + * that an image in a render a select returns is shown as a placeholder, as + * for `imageCeiling`. */ imageByteCeiling?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 5cc952e2..2f4fb356 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -12,7 +12,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import type { ContentBlock, NormalizedRequest } from '@animalabs/membrane'; import { AutobiographicalStrategy, ContextManager, WindowedPassthroughStrategy } from '../src/index.js'; -import type { ContextStrategy } from '../src/index.js'; +import type { ContextStrategy, ImageCeilings } from '../src/index.js'; import { ceilingBoundLine, resolveLiveImageLimits } from '../src/live-image-limits.js'; const MiB = 1024 * 1024; @@ -34,6 +34,15 @@ function pattern(messages: Array<{ content: ContentBlock[] }>, n: number): strin return out; } +/** The memory strategy's persisted hysteresis watermark, or null where it has + * none (or the strategy keeps none). */ +function watermarkOf(manager: ContextManager, strategy: ContextStrategy): number | null { + const id = (strategy as unknown as { imageWatermarkStateId?: string }).imageWatermarkStateId; + if (!id) return null; + const state = manager.getStore().getStateJson(id) as { sequence?: number } | null | undefined; + return typeof state?.sequence === 'number' ? state.sequence : null; +} + /** Compile a fresh store holding `n` image messages of `bytes` each. */ async function compiled(strategy: ContextStrategy, n: number, bytes = 4): Promise { const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); @@ -61,6 +70,12 @@ const DEFAULTS = { }; describe('imageCeilings()', () => { + it('is reachable through the ContextStrategy interface, typed by the exported ImageCeilings', () => { + const strategy: ContextStrategy = new WindowedPassthroughStrategy(); + const ceilings: ImageCeilings | undefined = strategy.imageCeilings?.(); + assert.deepEqual(ceilings, { imageCeiling: 500, imageByteCeiling: 24 * MiB }); + }); + for (const [name, make] of Object.entries(STRATEGIES)) { it(`${name}: the defaults are 500 images and 24 MiB`, () => { assert.deepEqual(DEFAULTS[name as keyof typeof DEFAULTS]().imageCeilings(), { imageCeiling: 500, imageByteCeiling: 24 * MiB }); @@ -410,7 +425,7 @@ describe('the line when an image first falls past an image ceiling', () => { async function ceilingLines( strategy: ContextStrategy, steps: Array, - ): Promise<{ lines: string[]; byStep: string[][]; patterns: string[]; last: string }> { + ): Promise<{ lines: string[]; byStep: string[][]; patterns: string[]; last: string; watermarks: Array }> { const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -419,6 +434,7 @@ describe('the line when an image first falls past an image ceiling', () => { let n = 0; const byStep: string[][] = []; const patterns: string[] = []; + const watermarks: Array = []; try { for (const step of steps) { const { add, bytes } = typeof step === 'number' ? { add: step, bytes: 4 } : step; @@ -426,8 +442,9 @@ describe('the line when an image first falls past an image ceiling', () => { for (let i = 0; i < add; i++) manager.addMessage('user', imageMessage(`img-${++n}`, bytes)); patterns.push(pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, n)); byStep.push(said.slice(before).filter((line) => line.includes('an image fell past'))); + watermarks.push(watermarkOf(manager, strategy)); } - return { lines: byStep.flat(), byStep, patterns, last: patterns[patterns.length - 1] ?? '' }; + return { lines: byStep.flat(), byStep, patterns, last: patterns[patterns.length - 1] ?? '', watermarks }; } finally { console.warn = original; await manager.close(); @@ -516,6 +533,29 @@ describe('the line when an image first falls past an image ceiling', () => { assert.deepEqual(lines, [], JSON.stringify(lines)); }); + it(`${name}: says nothing from renderAsNewest, whose render a ceiling strips, and leaves the line for the next compile`, async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, imageCeiling: 1 }) }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + const ids = [1, 2].map((i) => manager.addMessage('user', imageMessage(`img-${i}`))); + const rendered = manager.renderAsNewest(ids); + assert.ok(rendered, 'the strategy renders'); + const images = rendered.flatMap((content) => content ?? []).filter((b) => b.type === 'image').length; + assert.equal(images, 1, 'the ceiling of 1 stripped the older image in the render'); + assert.deepEqual(ours(), [], 'the render says nothing'); + await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(ours().length, 1, JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + it(`${name}: says nothing from a dry compile, and leaves the line for the first compile select returns`, async () => { for (const [options, said0] of [ [{ maxLiveImages: 0, imageCeiling: 2 }, 'an image fell past imageCeiling 2'], @@ -657,14 +697,13 @@ describe('the line when an image first falls past an image ceiling', () => { const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ - path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), - }); + const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); const ours = () => said.filter((line) => line.includes('an image fell past')); try { // The fifth image's message moves the watermark past the ceiling of 4. for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + assert.notEqual(watermarkOf(manager, strategy), null, 'the new messages moved the watermark'); assert.deepEqual(ours(), [], 'the new messages say nothing'); const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); assert.equal(pattern(compiled.messages, 5), '...II', 'one trim to half the ceiling'); @@ -769,32 +808,37 @@ describe('the line when an image first falls past an image ceiling', () => { } }); - it("the memory strategy leaves a watermark's line with the branch it moved on", async () => { - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ - path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + for (const [kind, options] of [ + ['count', { maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }], + ['byte', { maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }], + ] as const) { + it(`the memory strategy leaves a watermark's line with the branch it moved on, under the ${kind} ceiling`, async () => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory(options); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const ours = () => said.filter((line) => line.includes('an image fell past')); + try { + manager.addMessage('user', imageMessage('img-1')); + const main = manager.currentBranch().name; + await manager.fork('busy'); + // On the fork, new messages move the watermark past the ceiling: four + // more images make five, past a count of 4 and past 16 bytes. + for (let i = 2; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + assert.notEqual(watermarkOf(manager, strategy), null, "the fork's new messages moved its watermark"); + await manager.switchBranch(main); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 1), 'I', 'main holds one image, under its ceiling'); + assert.deepEqual(ours(), [], "main's compile says nothing for the fork's watermark"); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } }); - const ours = () => said.filter((line) => line.includes('an image fell past')); - try { - manager.addMessage('user', imageMessage('img-1')); - const main = manager.currentBranch().name; - await manager.fork('busy'); - // On the fork, new messages move the watermark past the ceiling of 4. - for (let i = 2; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - await manager.switchBranch(main); - const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - assert.equal(pattern(compiled.messages, 1), 'I', 'main holds one image, under its ceiling'); - assert.deepEqual(ours(), [], "main's compile says nothing for the fork's watermark"); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } - }); + } it("the memory strategy's watermark says nothing when the host's own limit binds", async () => { const { lines, last } = await ceilingLines(memory({ maxLiveImages: 4, imageStripHysteresisRatio: 0.5 }), [5]); @@ -809,11 +853,121 @@ describe('the line when an image first falls past an image ceiling', () => { const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500, imageStripHysteresisRatio: 0.5 }; const alone = await ceilingLines(memory({ ...depth, imageByteCeiling: 0 }), [{ add: 3, bytes: 8 }]); const both = await ceilingLines(memory({ ...depth, imageByteCeiling: 20 }), [{ add: 3, bytes: 8 }]); + assert.notEqual(alone.watermarks[0], null, 'the depth limit alone moves the watermark'); + assert.equal(both.watermarks[0], alone.watermarks[0], 'to the same place with the ceiling'); assert.equal(both.last, alone.last, 'the ceiling changes nothing the window keeps'); assert.equal(both.last, '..I'); assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); }); + it("the memory strategy's watermark says it when only the ceiling moves it, and nothing when the depth limit would move it anyway", async () => { + // Image messages price at about 1,600 tokens. Under a 4,000-token depth + // limit, the oldest of three is past the depth by the strips' measure + // (its own message counted) but not by the watermark's (newer messages + // only), so without a ceiling the watermark doesn't move. A count ceiling + // of 2 moves it, and its trim to half strips the middle image too, which + // the depth limit alone keeps: the ceiling decided. + const options = { maxLiveImages: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; + const alone = await ceilingLines(memory({ ...options, imageCeiling: 0 }), [3]); + assert.equal(alone.watermarks[0], null, 'the depth limit alone moves no watermark'); + assert.equal(alone.last, '.II'); + const ceiling = await ceilingLines(memory({ ...options, imageCeiling: 2 }), [3]); + assert.notEqual(ceiling.watermarks[0], null); + assert.equal(ceiling.last, '..I', 'the ceiling strips what the depth limit alone keeps'); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('an image fell past imageCeiling 2'), ceiling.lines[0]); + // Under a 3,000-token depth limit, the oldest of the three is past the + // depth by the watermark's measure too, from the message that brings the + // ceiling's first misfit on, so the depth limit moves the watermark with + // or without the ceiling, to the same place. + const deeper = { maxLiveImages: 0, imageStripDepthTokens: 3000, imageStripHysteresisRatio: 0.5 }; + const deepAlone = await ceilingLines(memory({ ...deeper, imageCeiling: 0 }), [3]); + assert.notEqual(deepAlone.watermarks[0], null, 'the depth limit moves the watermark'); + const deepCeiling = await ceilingLines(memory({ ...deeper, imageCeiling: 2 }), [3]); + assert.equal(deepCeiling.watermarks[0], deepAlone.watermarks[0]); + assert.equal(deepCeiling.last, deepAlone.last, 'the ceiling changes nothing the window keeps'); + assert.deepEqual(deepCeiling.lines, [], JSON.stringify(deepCeiling.lines)); + }); + + it("the memory strategy's watermark says it when the depth limit would move it but the byte ceiling's trim lands it newer", async () => { + // A 4-byte image, about 3,100 tokens of text, then a 16-byte image. When + // the second image comes, the first is past a 4,000-token depth limit, so + // the watermark moves with or without a ceiling. Without one, its trim to + // half the depth lands on the first image. The trim to half of a 20-byte + // ceiling, 10 bytes, can't keep the 16-byte image either, so it lands + // there and strips it. + const run = async (imageByteCeiling: number) => { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + manager.addMessage('user', imageMessage('img-1', 4)); + manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(1800) }]); + manager.addMessage('user', imageMessage('img-2', 16)); + const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, 2); + return { kept, watermark: watermarkOf(manager, strategy), lines: said.filter((line) => line.includes('an image fell past')) }; + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }; + const alone = await run(0); + assert.notEqual(alone.watermark, null, 'the depth limit alone moves the watermark'); + assert.equal(alone.kept, '.I', 'and keeps the 16-byte image'); + const ceiling = await run(20); + assert.equal(ceiling.kept, '..', "the ceiling's trim strips the 16-byte image too"); + assert.ok(ceiling.watermark! > alone.watermark!, 'it lands newer'); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), ceiling.lines[0]); + }); + + it("the memory strategy's watermark names only the ceiling that decided its move, not the host's own count limit", async () => { + // A host count limit of 2, in force, and a byte ceiling of 20. When the + // third image (12 bytes) comes, the host's count misfits the oldest, so + // the watermark moves anyway; its own trim, to a count of 1, lands on the + // middle image. The byte ceiling's trim, to 10 bytes, can't keep the + // 12-byte image, so it lands newer, on it: the byte ceiling decided. + const { lines, patterns } = await ceilingLines( + memory({ maxLiveImages: 2, maxLiveImageBytes: 0, imageByteCeiling: 20, imageStripHysteresisRatio: 0.5 }), + [{ add: 2, bytes: 4 }, { add: 1, bytes: 12 }], + ); + assert.deepEqual(patterns, ['II', '...'], 'the byte trim strips the newest image too'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), lines[0]); + }); + + it("the memory strategy's watermark credits the ceiling that brought the move, where the trim stops at the depth", async () => { + // A count ceiling of 4 and a 20,000-token depth limit. The fifth image's + // message brings the ceiling's misfit on the oldest, with nothing past + // the depth. About 9,000 tokens of text before the newest image put the + // second-newest past half the depth, so the trim lands there by the + // depth, not the count: the move is still the ceiling's. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, imageStripDepthTokens: 20000, imageStripHysteresisRatio: 0.5 }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(5200) }]); + manager.addMessage('user', imageMessage('img-5')); + const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, 5); + assert.equal(kept, '....I', 'the trim lands on the second-newest image'); + const ours = said.filter((line) => line.includes('an image fell past')); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('an image fell past imageCeiling 4'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { const auto = await ceilingLines(memory({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); From 5279d2d45e2f8692f23c88453a5380a5af847788 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 02:08:41 -0400 Subject: [PATCH 09/67] test(images): the watermark names the count ceiling, not the host's own byte limit The byte guard's mirror of the count guard's test: a host byte limit of 40 in force misfits the oldest image first, and the count ceiling of 4 shows up only in the trim, which lands newer. Without the byte guard, the line names the default byte ceiling as well, which held nothing. Co-Authored-By: Claude Opus 5.5 --- test/request-image-ceilings.test.ts | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 2f4fb356..27343c4f 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -940,6 +940,22 @@ describe('the line when an image first falls past an image ceiling', () => { assert.ok(lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), lines[0]); }); + it("the memory strategy's watermark names only the count ceiling that decided its move, not the host's own byte limit", async () => { + // The byte guard's mirror: a host byte limit of 40, in force, and a count + // ceiling of 4. When three 4-byte images come after a 32-byte one, the + // host's byte limit misfits the oldest (44 bytes), so the watermark moves + // anyway; its own trim, to 20 bytes, lands on that oldest image too. The + // count ceiling's trim, to 2, lands newer, on the second-oldest: the + // count ceiling decided, and the byte limit is the host's. + const { lines, patterns } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), + [{ add: 1, bytes: 32 }, { add: 3, bytes: 4 }], + ); + assert.deepEqual(patterns, ['I', '..II'], 'the count trim strips the second-oldest image too'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('an image fell past imageCeiling 4'), lines[0]); + }); + it("the memory strategy's watermark credits the ceiling that brought the move, where the trim stops at the depth", async () => { // A count ceiling of 4 and a 20,000-token depth limit. The fifth image's // message brings the ceiling's misfit on the oldest, with nothing past From 043130f56f0ea22661f3b4f21c6053d976eaba8a Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 02:33:42 -0400 Subject: [PATCH 10/67] fix(images): say a watermark's ceiling line only for an image the ceiling alone stripped, in the render returned The memory strategy's hysteresis watermark said the ceiling line at the next render a select returned after a ceiling decided its move, whether or not that render showed an image the move stripped. It credited a ceiling whose trim, at ratio x the ceiling, decided the landing though no image passed the ceiling itself, and it measured the host's own depth limit the watermark's way, newer messages only, while the strips count an image's own message too. A live move now records the sequences it stripped that the limits the host has in force wouldn't have: after where those limits would land it, through where it lands, with the ceilings the first image past a limit passed, which brought the move on. A later live move cuts earlier records at its own host-only landing, since the host's limits strip those images anyway. The strips credit a record's ceilings for an image in it that the depth limit doesn't strip, and the select reports them for the render it returns, then drops the records. So a move a host limit brings on says nothing even where a ceiling's trim lands it newer, and the line's words, that an image fell past the ceiling, hold. The guard that a landing is above the watermark is gone: both walks stop at it, and the trim's caps are no larger than the first walk's. The docs say the line comes at the first render a select returns where images are placeholders because an image passed the ceiling, that a fraction of maxLiveImages is floored to at least 1, and that membrane's shedding stops when the request fits or no image is left, after which only the Anthropic adapter's check refuses one still over its cap. Tests: a move the depth limit, or the host's own count or byte limit, brings on says nothing where a ceiling's trim lands it newer; an image the move strips that the strip's depth also strips says nothing; the host's own limit landing the move where a ceiling does says nothing, for each kind; a later host move overtaking a ceiling's before a render says nothing; and only the ceiling an image passed is named when both limits are the ceilings'. Co-Authored-By: Claude Opus 5.5 --- .../request-image-ceilings.breaking.md | 2 +- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 6 +- src/strategies/autobiographical.ts | 94 +++++++----- src/types/strategy.ts | 28 ++-- test/request-image-ceilings.test.ts | 142 +++++++++++++----- 6 files changed, 181 insertions(+), 93 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index f9dafd30..9e094753 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once, the first time a ceiling decides that an image in a render a select returns is shown as a placeholder, which ceiling it was and what it held. +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.1 depends on `^0.14.1`, and connectome-host 0.10.2 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 8ef7811e..2f0184b2 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once, the first time that ceiling decides that an image is shown as a placeholder, which ceiling it was and what it held. Only a render a select returns says it, and not when the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips the image anyway. A move of the watermark counts as a ceiling's only when the host's own limits wouldn't move it. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling, rather than because of a limit the host set. That's the image past the ceiling, unless the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips it anyway, or the images the hysteresis watermark strips when an image past the ceiling moves it, unless the host's own limits, the depth limit included, would strip them too. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index eea60a91..795b46ee 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -33,8 +33,10 @@ export const DEFAULT_IMAGE_CEILING = 500; * Anthropic adapter checks again, on every request it sends. Over the cap, * `transformRequest` refuses the request before sending it * (`assertWithinByteBudget`), unless the request sets `shedOversizeImages`, - * as this package's compression requests do, and then it replaces the oldest - * images until it fits. 24 MiB of base64 images leaves 4 MiB of that + * as this package's compression requests do. Then it replaces images, + * oldest first, until the request fits or no image is left, and passes it + * on either way; the Anthropic adapter's own check, which never sheds, + * refuses one still over the cap. 24 MiB of base64 images leaves 4 MiB of that * default cap for everything else the messages carry: their text, their * JSON, and any documents or other media. */ diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 63bd7fc9..1aa83966 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2703,8 +2703,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._imageWatermark = undefined; this._imageWatermarkLoaded = false; // A ceiling's move of the watermark belongs to the branch it moved on. - this._ceilingMovedWatermark.images = false; - this._ceilingMovedWatermark.bytes = false; + this._ceilingWatermarkRanges = []; this._lastKvStable = null; this._storeOrder = new Map(); this._storeOrderSource = []; @@ -16627,6 +16626,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { const entry = entries[idx]; const tooDeep = depthTokens > 0 && (pos < 0 || pos < stripStart); const belowMark = pos >= 0 && this.belowImageWatermark(messages[pos]!.sequence); + // Below the watermark only because a ceiling moved it there, and not + // past the depth limit, which strips it anyway: the ceiling decided. + if (belowMark && !tooDeep) { + const sequence = messages[pos]!.sequence; + for (const range of this._ceilingWatermarkRanges) { + if (sequence <= range.after || sequence > range.through) continue; + if (range.images) hits.images = true; + if (range.bytes) hits.bytes = true; + } + } const bucket = bucketAt(pos); entry.content = entry.content.map((block) => { if (block.type !== 'image') return block; @@ -16721,34 +16730,37 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; // Where the watermark lands for given limits: past the first misfit, at // the first one of the trim to ratio × each limit, or undefined if no - // limit binds. `passed` collects the limits the two walks' misfits pass. - const passed = { images: false, bytes: false }; - const landing = (countCap: number, bytesCap: number, note: boolean): number | undefined => { + // limit binds. Both walks stop at the watermark, and the trim's caps are + // each no larger than the first walk's, so a landing is always above it. + // `passed` is the limits the first misfit passes: what brought the move on. + let passed = { images: false, bytes: false }; + const landing = (countCap: number, bytesCap: number): number | undefined => { if (firstMisfit(stripDepth, countCap, bytesCap) === undefined) return undefined; - if (note) { passed.images ||= misfitBy.images; passed.bytes ||= misfitBy.bytes; } - const at = firstMisfit( + passed = misfitBy; + return firstMisfit( stripDepth > 0 ? Math.floor(stripDepth * ratio) : 0, countCap > 0 ? Math.max(0, Math.floor(countCap * ratio)) || 1 : 0, bytesCap > 0 ? Math.floor(bytesCap * ratio) : 0, ); - if (note) { passed.images ||= misfitBy.images; passed.bytes ||= misfitBy.bytes; } - return at; }; - const next = landing(maxLive, maxLiveBytes, true); - if (next === undefined || (w !== undefined && next <= w)) return; // no limit binds, or no advance - // The ceilings decide this move when it lands newer, so strips more, than - // the limits the host has in force would land it: its depth limit, and a - // count or byte limit no ceiling sets (ceilingSetLimits). The move then - // strips every later render by it, so the next render a select returns - // reports each ceiling whose limit the walks passed (reportCeilingHits). - // A dry advance is undone, so it leaves nothing. + const next = landing(maxLive, maxLiveBytes); + if (next === undefined) return; // no limit binds + // `own` is where the limits the host has in force would land this move: + // its depth limit, and a count or byte limit no ceiling sets (per + // ceilingSetLimits). They strip every image at or before it anyway, so + // what earlier moves left for a ceiling stops there. When an image past a + // ceiling brought this move on, the images after `own` and through this + // landing are stripped only because of that ceiling, and the strips + // report it for such an image in the next render a select returns + // (reportCeilingHits). A dry advance is undone, so it leaves nothing. if (!this._dryRunSelect) { + const brought = passed; const set = ceilingSetLimits(this.configuredImageLimits()); - const own = landing(set.images ? 0 : maxLive, set.bytes ? 0 : maxLiveBytes, false); - if (own === undefined || next > own) { - if (set.images && passed.images) this._ceilingMovedWatermark.images = true; - if (set.bytes && passed.bytes) this._ceilingMovedWatermark.bytes = true; - } + const own = landing(set.images ? 0 : maxLive, set.bytes ? 0 : maxLiveBytes); + if (own !== undefined) for (const range of this._ceilingWatermarkRanges) range.after = Math.max(range.after, own); + const images = set.images && brought.images; + const bytes = set.bytes && brought.bytes; + if (images || bytes) this._ceilingWatermarkRanges.push({ after: own ?? w ?? -Infinity, through: next, images, bytes }); } this._imageWatermark = next; // Dry selections and observer reconstructions advance it locally only; @@ -16882,27 +16894,33 @@ export class AutobiographicalStrategy implements ResettableStrategy { private readonly ceilingBoundReported = { images: false, bytes: false }; /** The ceilings that decided an image in the last render select returned (applyImageStripping, `returned`). */ private _ceilingHitsThisRender = { images: false, bytes: false }; - /** The ceilings that have decided a move of the hysteresis watermark on this branch, outside a dry run. */ - private readonly _ceilingMovedWatermark = { images: false, bytes: false }; - - /** Say once for each ceiling, when a select returns a render in which an - * image ceiling, rather than a limit the host set, decided that an image - * is stripped (see ceilingsDeciding), or the first render a select - * returns after a ceiling decided a move of the watermark on this branch - * while this instance held it (maybeAdvanceImageWatermark). That render - * needn't hold an image the move stripped: the line says that the ceiling - * acted, and when. A watermark loaded from the store, by a later instance - * or a reload of the branch, carries no record of what moved it, so the - * line waits until this instance's own walk meets the ceiling. A dry - * select, a preview, an observer's reconstruction or a refused select says - * nothing, and the line waits for the first render returned. */ + /** The sequences a ceiling's move of the hysteresis watermark stripped + * that the host's own limits wouldn't have, after `after` and through + * `through`, with the ceilings an image passed to bring the move on. Set + * by a live move on this branch (maybeAdvanceImageWatermark), read by the + * strips, and kept until the next render a select returns. */ + private _ceilingWatermarkRanges: Array<{ after: number; through: number; images: boolean; bytes: boolean }> = []; + + /** Say once for each ceiling, when a select returns a render in which + * images are shown as placeholders because an image passed that ceiling, + * rather than because of a limit the host set: an image past the ceiling + * itself (see ceilingsDeciding), or one stripped only by a watermark move + * an image past the ceiling brought on, while this instance held the + * watermark on this branch (maybeAdvanceImageWatermark). A watermark + * loaded from the store, by a later instance or a reload of the branch, + * carries no record of what moved it, so the line waits until this + * instance's own walk meets the ceiling. A dry select, a preview, an + * observer's reconstruction or a refused select says nothing, and the line + * waits for the first render returned. That render ends what a move left + * to report: an image it stripped that the render doesn't hold, such as + * one compressed since, says nothing later. */ private reportCeilingHits(): void { for (const kind of ['images', 'bytes'] as const) { - const hit = this._ceilingHitsThisRender[kind] || this._ceilingMovedWatermark[kind]; - if (!hit || this.ceilingBoundReported[kind]) continue; + if (!this._ceilingHitsThisRender[kind] || this.ceilingBoundReported[kind]) continue; this.ceilingBoundReported[kind] = true; this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); } + this._ceilingWatermarkRanges = []; } /** The image limits whose values are defaults: those the library's default diff --git a/src/types/strategy.ts b/src/types/strategy.ts index e2eb2f3e..0906d90d 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1228,7 +1228,7 @@ export interface AutobiographicalConfig { * compiled request, so a caller that keeps its requests within the * ceilings recompiles at each image it adds; at about 1,600 tokens an * image, the token budget binds first unless the images are small. A - * fraction is floored. + * fraction is floored, to at least 1. * A hard limit for dense bursts that pack many images into the live window. * The window counts top-level `image` blocks, and measures their bytes by * the length of their base64 `source.data`. So an image nested in a tool @@ -1307,9 +1307,11 @@ export interface AutobiographicalConfig { * for what this doesn't count. 0 = no ceiling. A value that isn't 0 or a * finite number above it takes the default, and the strategy's warning at * initialize names it. A fraction is floored, to at least 1. Under a - * window limit of 0 or above this ceiling, the strategy says once, the - * first time this ceiling decides that an image in a render a select - * returns is shown as a placeholder. + * window limit of 0 or above this ceiling, the strategy says so once, at + * the first render a select returns where images are shown as + * placeholders because an image passed this ceiling, rather than because + * of a limit the host set: the image past it, or those the hysteresis + * watermark strips with it. */ imageCeiling?: number; @@ -1325,18 +1327,18 @@ export interface AutobiographicalConfig { * injects or appends aren't counted. membrane's byte cap is 28 MiB of * serialized messages unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, * and the default, 24 MiB, leaves 4 MiB of that cap for what this doesn't - * count. Where membrane checks a request against its cap, it refuses one - * over it before sending, or sheds its oldest images when the request asks - * it to, as this strategy's compression requests do (see - * `src/live-image-limits.ts` for where it checks). This setting isn't read - * from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes membrane's cap - * sets this too, keeping room under the cap for what this doesn't count. + * count. membrane refuses a request over its cap before sending it. One + * that asks it to shed images, as this strategy's compression requests do, + * first has its oldest images replaced until it fits or none is left, and + * then only the Anthropic adapter's own check refuses it if it's still + * over (`src/live-image-limits.ts` says where each check runs). This + * setting isn't read from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes + * membrane's cap sets this too, keeping room under the cap for what this + * doesn't count. * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A * value that isn't 0 or a finite number above it takes the default, and a * fraction is floored, to at least 1. Under a window limit of 0 or above - * this ceiling, the strategy says once when this ceiling first decides - * that an image in a render a select returns is shown as a placeholder, as - * for `imageCeiling`. + * this ceiling, the strategy says so once, as for `imageCeiling`. */ imageByteCeiling?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 27343c4f..820dc825 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -889,73 +889,139 @@ describe('the line when an image first falls past an image ceiling', () => { assert.deepEqual(deepCeiling.lines, [], JSON.stringify(deepCeiling.lines)); }); - it("the memory strategy's watermark says it when the depth limit would move it but the byte ceiling's trim lands it newer", async () => { + /** Compile, after adding each message in turn, with a strategy built from + * `options`; the window kept, the watermark, and the ceiling lines said. */ + async function watermarkRun(options: Record, messages: unknown[][]) { + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory(options); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + try { + for (const content of messages) manager.addMessage('user', content as never); + const images = messages.filter((content) => content.some((b) => (b as { type?: string }).type === 'image')).length; + const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, images); + return { kept, watermark: watermarkOf(manager, strategy), lines: said.filter((line) => line.includes('an image fell past')) }; + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + } + + it("the memory strategy's watermark says nothing when the depth limit brings the move on, though the byte ceiling's trim lands it newer", async () => { // A 4-byte image, about 3,100 tokens of text, then a 16-byte image. When // the second image comes, the first is past a 4,000-token depth limit, so // the watermark moves with or without a ceiling. Without one, its trim to // half the depth lands on the first image. The trim to half of a 20-byte // ceiling, 10 bytes, can't keep the 16-byte image either, so it lands - // there and strips it. - const run = async (imageByteCeiling: number) => { - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const strategy = memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - try { - manager.addMessage('user', imageMessage('img-1', 4)); - manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(1800) }]); - manager.addMessage('user', imageMessage('img-2', 16)); - const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, 2); - return { kept, watermark: watermarkOf(manager, strategy), lines: said.filter((line) => line.includes('an image fell past')) }; - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } - }; - const alone = await run(0); + // there and strips it. No image passed the ceiling itself (20 bytes in + // all), so the line, which says one did, stays quiet. + const text = [{ type: 'text', text: 'word '.repeat(1800) }]; + const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; + const alone = await watermarkRun({ ...options, imageByteCeiling: 0 }, [imageMessage('img-1', 4), text, imageMessage('img-2', 16)]); assert.notEqual(alone.watermark, null, 'the depth limit alone moves the watermark'); assert.equal(alone.kept, '.I', 'and keeps the 16-byte image'); - const ceiling = await run(20); + const ceiling = await watermarkRun({ ...options, imageByteCeiling: 20 }, [imageMessage('img-1', 4), text, imageMessage('img-2', 16)]); assert.equal(ceiling.kept, '..', "the ceiling's trim strips the 16-byte image too"); assert.ok(ceiling.watermark! > alone.watermark!, 'it lands newer'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), ceiling.lines[0]); + assert.deepEqual(ceiling.lines, []); }); - it("the memory strategy's watermark names only the ceiling that decided its move, not the host's own count limit", async () => { + it("the memory strategy's watermark says nothing for an image its move strips that the depth limit strips anyway", async () => { + // A 16-byte image, about 1,500 tokens of text, then an 8-byte image. The + // first image passes a 20-byte ceiling (24 bytes), so the watermark moves + // onto it. By the watermark's measure, the messages newer than it, it's + // within a 4,000-token depth limit, so the host's own limits wouldn't + // move it. But the strip counts the image's own message too, and by that + // measure it's past the depth: the render strips it with or without the + // ceiling, so the ceiling decided nothing. + const text = [{ type: 'text', text: 'word '.repeat(870) }]; + const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; + const messages = [imageMessage('img-1', 16), text, imageMessage('img-2', 8)]; + const alone = await watermarkRun({ ...options, imageByteCeiling: 0 }, messages); + assert.equal(alone.watermark, null, 'without the ceiling the watermark stays'); + assert.equal(alone.kept, '.I', 'the depth limit strips the first image'); + const ceiling = await watermarkRun({ ...options, imageByteCeiling: 20 }, messages); + assert.notEqual(ceiling.watermark, null, 'the ceiling moves the watermark'); + assert.equal(ceiling.kept, '.I', 'the render is the same'); + assert.deepEqual(ceiling.lines, []); + }); + + it("the memory strategy's watermark says nothing when the host's own count limit brings the move on, though the byte ceiling's trim lands it newer", async () => { // A host count limit of 2, in force, and a byte ceiling of 20. When the // third image (12 bytes) comes, the host's count misfits the oldest, so - // the watermark moves anyway; its own trim, to a count of 1, lands on the - // middle image. The byte ceiling's trim, to 10 bytes, can't keep the - // 12-byte image, so it lands newer, on it: the byte ceiling decided. + // the watermark moves anyway; its own trim, to a count of 1, would land on + // the middle image. The byte ceiling's trim, to 10 bytes, can't keep the + // 12-byte image, so it lands newer, on it. No image passed the byte + // ceiling itself (20 bytes in all), and the count limit is the host's. const { lines, patterns } = await ceilingLines( memory({ maxLiveImages: 2, maxLiveImageBytes: 0, imageByteCeiling: 20, imageStripHysteresisRatio: 0.5 }), [{ add: 2, bytes: 4 }, { add: 1, bytes: 12 }], ); assert.deepEqual(patterns, ['II', '...'], 'the byte trim strips the newest image too'); - assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), lines[0]); + assert.deepEqual(lines, []); }); - it("the memory strategy's watermark names only the count ceiling that decided its move, not the host's own byte limit", async () => { - // The byte guard's mirror: a host byte limit of 40, in force, and a count - // ceiling of 4. When three 4-byte images come after a 32-byte one, the - // host's byte limit misfits the oldest (44 bytes), so the watermark moves - // anyway; its own trim, to 20 bytes, lands on that oldest image too. The - // count ceiling's trim, to 2, lands newer, on the second-oldest: the - // count ceiling decided, and the byte limit is the host's. + it("the memory strategy's watermark says nothing when the host's own byte limit brings the move on, though the count ceiling's trim lands it newer", async () => { + // A host byte limit of 40, in force, and a count ceiling of 4. When three + // 4-byte images come after a 32-byte one, the host's byte limit misfits + // the oldest (44 bytes), so the watermark moves anyway; its own trim, to + // 20 bytes, would land on that oldest image. The count ceiling's trim, to + // 2, lands newer, on the second-oldest. No image passed the count + // ceiling itself (4 images in all), and the byte limit is the host's. const { lines, patterns } = await ceilingLines( memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), [{ add: 1, bytes: 32 }, { add: 3, bytes: 4 }], ); assert.deepEqual(patterns, ['I', '..II'], 'the count trim strips the second-oldest image too'); + assert.deepEqual(lines, []); + }); + + it("the memory strategy's watermark names the count ceiling an image passed, not a byte ceiling none did", async () => { + // Both window limits are set by their ceilings. The fifth 4-byte image + // passes the count ceiling of 4, and 20 bytes are far under the byte + // ceiling, so only the count ceiling brought the move on. + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + [5], + ); + assert.equal(last, '...II'); assert.equal(lines.length, 1, JSON.stringify(lines)); assert.ok(lines[0]!.includes('an image fell past imageCeiling 4'), lines[0]); }); + it("the memory strategy's watermark says nothing for a ceiling's move that the host's own limit later overtakes, before a render", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force. The fifth + // image passes the count ceiling and moves the watermark onto the + // third-newest. Before any compile, a 44-byte image passes the host's + // byte limit on its own and moves the watermark onto itself: the host's + // limit strips every image now, those the ceiling's move stripped too, so + // the render is the same with or without the ceiling. + const messages = [imageMessage('img-1', 20), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`)), imageMessage('img-6', 44)]; + const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; + const alone = await watermarkRun({ ...options, imageCeiling: 0 }, messages); + const ceiling = await watermarkRun({ ...options, imageCeiling: 4 }, messages); + assert.equal(ceiling.kept, '......'); + assert.equal(ceiling.kept, alone.kept, 'the render is the same without the ceiling'); + assert.deepEqual(ceiling.lines, []); + }); + + for (const [kind, options, why] of [ + ['byte', { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 16 }, 'the host byte limit of 16 lands it where the count ceiling of 4 does'], + ['count', { maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 16 }, 'the host count limit of 4 lands it where the byte ceiling of 16 does'], + ] as const) { + it(`the memory strategy's watermark says nothing when the host's own ${kind} limit would land it where a ceiling does`, async () => { + // The fifth 4-byte image passes both the host's limit and the ceiling, + // and both trims, to 2 images and to 8 bytes, land on the third-newest: + // ${why}, so the host's limits strip the same images anyway. + const { lines, last } = await ceilingLines(memory({ ...options, imageStripHysteresisRatio: 0.5 }), [5]); + assert.equal(last, '...II'); + assert.deepEqual(lines, [], why); + }); + } + it("the memory strategy's watermark credits the ceiling that brought the move, where the trim stops at the depth", async () => { // A count ceiling of 4 and a 20,000-token depth limit. The fifth image's // message brings the ceiling's misfit on the oldest, with nothing past From 5aaf37d733a4b596e12226b0325abb3a9a861806 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 03:00:51 -0400 Subject: [PATCH 11/67] fix(images): keep a watermark move's record until each ceiling it holds is said Dropping the records at every reported render lost the line when that render left the move's images out, such as an adaptive selection at a smaller budget, and a later render showed them. A record now stays until each ceiling it holds has been said, and goes once a later move's cut empties it, which bounds them as the clear did. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 1aa83966..657c159d 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -16898,7 +16898,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * that the host's own limits wouldn't have, after `after` and through * `through`, with the ceilings an image passed to bring the move on. Set * by a live move on this branch (maybeAdvanceImageWatermark), read by the - * strips, and kept until the next render a select returns. */ + * strips, and kept until each ceiling it holds has been said + * (reportCeilingHits). */ private _ceilingWatermarkRanges: Array<{ after: number; through: number; images: boolean; bytes: boolean }> = []; /** Say once for each ceiling, when a select returns a render in which @@ -16911,16 +16912,20 @@ export class AutobiographicalStrategy implements ResettableStrategy { * carries no record of what moved it, so the line waits until this * instance's own walk meets the ceiling. A dry select, a preview, an * observer's reconstruction or a refused select says nothing, and the line - * waits for the first render returned. That render ends what a move left - * to report: an image it stripped that the render doesn't hold, such as - * one compressed since, says nothing later. */ + * waits for the first render returned. A move's record stays until each + * ceiling it holds has been said, so a render that leaves its images out, + * such as an adaptive selection at a smaller budget, leaves the line for + * a later one that holds them; a record a later move's cut has emptied + * goes. */ private reportCeilingHits(): void { for (const kind of ['images', 'bytes'] as const) { if (!this._ceilingHitsThisRender[kind] || this.ceilingBoundReported[kind]) continue; this.ceilingBoundReported[kind] = true; this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); } - this._ceilingWatermarkRanges = []; + const said = this.ceilingBoundReported; + this._ceilingWatermarkRanges = this._ceilingWatermarkRanges.filter((range) => + range.after < range.through && ((range.images && !said.images) || (range.bytes && !said.bytes))); } /** The image limits whose values are defaults: those the library's default From 1cf163e138dca0002e6c6d871f89157b961d5f58 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 03:29:29 -0400 Subject: [PATCH 12/67] fix(images): a ceiling's watermark move says nothing for an image the host's own limit strips anyway The strips credited a ceiling's move for any image in its record that the depth limit kept, without the check of the host's in-force count and byte limits that the strip applies to an image past a ceiling itself. So an image a later message pushed past the host's own limit, below the watermark where the next move's walk can't see it, still brought the line. The strips now credit a move's record for an image only when it passes no limit the host has in force, counted as the strip counts it. The line ends "so older images are now shown as placeholders": a watermark's trim strips images inside the ceiling too, down to ratio x the ceiling. ceilingsDeciding's doc says a ceiling's move is credited from its record, and the landing's comment no longer claims the trim's caps are always tighter, since a floored cap of 0 is no limit; a landing is above the watermark because both walks stop there. Tests: a ceiling-brought move whose host-only landing is defined and older keeps the line for the images between, for each kind; and an image the host's own byte limit strips by render time says nothing. Co-Authored-By: Claude Opus 5.5 --- src/live-image-limits.ts | 6 ++- src/strategies/autobiographical.ts | 22 +++++++---- test/request-image-ceilings.test.ts | 58 ++++++++++++++++++++++++++--- 3 files changed, 72 insertions(+), 14 deletions(-) diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 795b46ee..2d13ce06 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -161,7 +161,9 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b * ceiling. A limit the host set above its ceiling isn't in force: the * ceiling holds it, and the warning at initialize says so. Only a ceiling * that decides changes what the window keeps, so only that one says its - * line. + * line. An image the watermark strips is the ceiling's when a ceiling's + * move alone put it there; the memory strategy credits that from its record + * of the move (applyImageStripping), not here. */ export function ceilingsDeciding( over: { images: boolean; bytes: boolean }, @@ -194,7 +196,7 @@ export function ceilingBoundLine( const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); const mark = (name: string) => defaultMark(name, configured, defaulted); return `[${strategy}] an image fell past ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + - `which holds ${limitName} ${limit}${mark(limitName)} to it, so images past it are now shown as placeholders`; + `which holds ${limitName} ${limit}${mark(limitName)} to it, so older images are now shown as placeholders`; } /** diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 657c159d..a2782092 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -16626,14 +16626,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { const entry = entries[idx]; const tooDeep = depthTokens > 0 && (pos < 0 || pos < stripStart); const belowMark = pos >= 0 && this.belowImageWatermark(messages[pos]!.sequence); - // Below the watermark only because a ceiling moved it there, and not - // past the depth limit, which strips it anyway: the ceiling decided. + // The ceilings whose move alone put this message below the watermark, + // when the depth limit doesn't strip it anyway. + let moved: { images: boolean; bytes: boolean } | undefined; if (belowMark && !tooDeep) { const sequence = messages[pos]!.sequence; for (const range of this._ceilingWatermarkRanges) { if (sequence <= range.after || sequence > range.through) continue; - if (range.images) hits.images = true; - if (range.bytes) hits.bytes = true; + moved ??= { images: false, bytes: false }; + moved.images ||= range.images; + moved.bytes ||= range.bytes; } } const bucket = bucketAt(pos); @@ -16645,6 +16647,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { const deciding = ceilingsDeciding({ images: overCount, bytes: overBytes }, belowMark || tooDeep, ceilingSet); if (deciding.images) hits.images = true; if (deciding.bytes) hits.bytes = true; + // A limit the host has in force that this image passes strips it + // anyway, so a ceiling's move decided nothing for it. + if (moved && !((overCount && !ceilingSet.images) || (overBytes && !ceilingSet.bytes))) { + if (moved.images) hits.images = true; + if (moved.bytes) hits.bytes = true; + } if (belowMark || tooDeep || overCount || overBytes) { // Stats-neutral (2026-07-12): every budgeting site now tallies at // POST-STRIP prices (see postStripEstimates), so the bucket never @@ -16730,9 +16738,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; // Where the watermark lands for given limits: past the first misfit, at // the first one of the trim to ratio × each limit, or undefined if no - // limit binds. Both walks stop at the watermark, and the trim's caps are - // each no larger than the first walk's, so a landing is always above it. - // `passed` is the limits the first misfit passes: what brought the move on. + // limit binds. Both walks stop at the watermark, so a landing is always + // above it. `passed` is the limits the first misfit passes: what brought + // the move on. let passed = { images: false, bytes: false }; const landing = (countCap: number, bytesCap: number): number | undefined => { if (firstMisfit(stripDepth, countCap, bytesCap) === undefined) return undefined; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 820dc825..fc62cf0a 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -468,7 +468,7 @@ describe('the line when an image first falls past an image ceiling', () => { const { lines, byStep, patterns } = await ceilingLines(strategy, [3, 2]); assert.deepEqual(patterns, ['.II', '...II'], `the ceiling of 2 strips at each compile for ${JSON.stringify(options)}`); assert.deepEqual(byStep.map((step) => step.length), [1, 0], `said once, at the first step, for ${JSON.stringify(options)}`); - assert.ok(lines[0]!.startsWith(`[${strategy.name}] an image fell past ${said}, so images past it are now shown as placeholders`), lines[0]); + assert.ok(lines[0]!.startsWith(`[${strategy.name}] an image fell past ${said}, so older images are now shown as placeholders`), lines[0]); assert.match(lines[0]!, / ns=/, 'through the namespaced route the warning takes'); } }); @@ -1008,6 +1008,54 @@ describe('the line when an image first falls past an image ceiling', () => { assert.deepEqual(ceiling.lines, []); }); + it("the memory strategy's watermark names the count ceiling for an image its move stripped that the host's own limit, landing older, keeps", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force. When the + // fifth image comes (8 bytes, after a 24-byte one and three 4-byte ones), + // it passes both: the count ceiling and 44 bytes. The host's limit alone + // would trim to 20 bytes, landing on the oldest; the count ceiling's trim, + // to 2, lands on the middle image. So the two between are stripped only + // because an image passed the count ceiling. + const messages = [imageMessage('img-1', 24), ...[2, 3, 4].map((i) => imageMessage(`img-${i}`)), imageMessage('img-5', 8)]; + const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; + const alone = await watermarkRun({ ...options, imageCeiling: 0 }, messages); + assert.equal(alone.kept, '.IIII', "the host's limit alone strips the oldest"); + const ceiling = await watermarkRun({ ...options, imageCeiling: 4 }, messages); + assert.equal(ceiling.kept, '...II'); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('an image fell past imageCeiling 4'), ceiling.lines[0]); + }); + + it("the memory strategy's watermark names the byte ceiling for an image its move stripped that the host's own limit, landing older, keeps", async () => { + // The mirror, with a byte ceiling of 20 and a host count limit of 4, in + // force. When the fifth image comes (8 bytes, after three 4-byte ones and + // an 8-byte one), the second-oldest passes the byte ceiling (24 bytes) + // while the count is 3. The host's count alone would move the watermark + // only at the oldest and trim to 2, landing on the middle image; the byte + // ceiling's trim, to 10 bytes, lands on the 8-byte fourth image. So that + // one is stripped only because an image passed the byte ceiling. + const messages = [...[1, 2, 3].map((i) => imageMessage(`img-${i}`)), imageMessage('img-4', 8), imageMessage('img-5', 8)]; + const options = { maxLiveImages: 4, maxLiveImageBytes: 0, imageStripHysteresisRatio: 0.5 }; + const alone = await watermarkRun({ ...options, imageByteCeiling: 0 }, messages); + assert.equal(alone.kept, '...II', "the host's count alone strips the three oldest"); + const ceiling = await watermarkRun({ ...options, imageByteCeiling: 20 }, messages); + assert.equal(ceiling.kept, '....I'); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('an image fell past imageByteCeiling 20 bytes'), ceiling.lines[0]); + }); + + it("the memory strategy's watermark says nothing for an image its move stripped that the host's own limit strips anyway by render time", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force. The fifth + // image passes the count ceiling and moves the watermark onto the middle + // image, which the host's limit alone would keep then. A 24-byte image + // comes before any compile: above the watermark it fits, but in the + // render the middle image no longer fits the host's 40 bytes with the + // images kept after it, so the host's limit strips it anyway. + const messages = [...[1, 2, 3].map((i) => imageMessage(`img-${i}`, 12)), imageMessage('img-4'), imageMessage('img-5'), imageMessage('img-6', 24)]; + const ceiling = await watermarkRun({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }, messages); + assert.equal(ceiling.kept, '...III'); + assert.deepEqual(ceiling.lines, []); + }); + for (const [kind, options, why] of [ ['byte', { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 16 }, 'the host byte limit of 16 lands it where the count ceiling of 4 does'], ['count', { maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 16 }, 'the host count limit of 4 lands it where the byte ceiling of 16 does'], @@ -1077,13 +1125,13 @@ describe('the line when an image first falls past an image ceiling', () => { const configured = (o: Record) => ({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...o }); assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['imageCeiling'])), - '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 0 (no limit) to it, so images past it are now shown as placeholders'); + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 0 (no limit) to it, so older images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'images', configured({ imageCeiling: Number.NaN, maxLiveImages: 800 }), new Set()), - '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 800 to it, so images past it are now shown as placeholders'); + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 800 to it, so older images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB }), new Set()), - '[s] an image fell past imageByteCeiling 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so images past it are now shown as placeholders'); + '[s] an image fell past imageByteCeiling 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so older images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['imageByteCeiling'])), - `[s] an image fell past imageByteCeiling 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so images past it are now shown as placeholders`); + `[s] an image fell past imageByteCeiling 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so older images are now shown as placeholders`); }); it('the memory strategy says it when its hysteresis watermark trims against the byte ceiling', async () => { From 91df717795b576060975c6eb5942b87c1312bcf9 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Tue, 20 Oct 2026 04:23:39 -0400 Subject: [PATCH 13/67] test(images): the host's own count limit, by render time, quiets a ceiling's watermark move too The check of the host's in-force limits before crediting a ceiling's move had a render-time test for its byte half only. A byte ceiling's move lands on the oldest image; two more images come before the compile, and in the render the oldest is the fifth, past the host's count of 4, so nothing is said. The comment on the advance now says the strips report such an image in a render a select returns, not the next one. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 2 +- test/request-image-ceilings.test.ts | 14 ++++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index a2782092..1365a1b9 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -16759,7 +16759,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // what earlier moves left for a ceiling stops there. When an image past a // ceiling brought this move on, the images after `own` and through this // landing are stripped only because of that ceiling, and the strips - // report it for such an image in the next render a select returns + // report it for such an image in a render a select returns // (reportCeilingHits). A dry advance is undone, so it leaves nothing. if (!this._dryRunSelect) { const brought = passed; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index fc62cf0a..36adc72e 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -1056,6 +1056,20 @@ describe('the line when an image first falls past an image ceiling', () => { assert.deepEqual(ceiling.lines, []); }); + it("the memory strategy's watermark says nothing for an image its move stripped that the host's own count limit strips anyway by render time", async () => { + // A byte ceiling of 20 and a host count limit of 4, in force. A 16-byte + // image and two 4-byte ones: the oldest passes the byte ceiling (24 + // bytes) at a count of 2, so the move, which lands on it, is the + // ceiling's. Two more 4-byte images come before any compile, and fit above + // the mark. In the render the oldest is the fifth image, past the host's + // count of 4, so the host's limit strips it anyway. (Without the ceiling, + // the host's own move at the fifth image strips three.) + const messages = [imageMessage('img-1', 16), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`))]; + const ceiling = await watermarkRun({ maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 20, imageStripHysteresisRatio: 0.5 }, messages); + assert.equal(ceiling.kept, '.IIII'); + assert.deepEqual(ceiling.lines, []); + }); + for (const [kind, options, why] of [ ['byte', { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 16 }, 'the host byte limit of 16 lands it where the count ceiling of 4 does'], ['count', { maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 16 }, 'the host count limit of 4 lands it where the byte ceiling of 16 does'], From 706198c8bf013f7cdbe1470d85cc6e432ae3e23d Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Wed, 21 Oct 2026 11:10:33 -0400 Subject: [PATCH 14/67] docs(images): the ceiling line's ending, membrane's checks, and a loaded watermark's silence From our internal review, text only: - The line ends "so some images are now shown as placeholders". The greedy byte strip can replace the newest image and keep older ones, so "older images" named the wrong ones. Its five test strings follow. - imageByteCeiling's doc says where membrane checks a request against its cap: an XML-mode continuation round isn't checked there, and of the provider adapters only Anthropic's checks again, as src/live-image-limits.ts already says. - imageCeiling's doc, imageByteCeiling's doc and the changed and breaking fragments say the line isn't said for an image the host's own limits would strip, or for what a watermark loaded from the store strips, which carries no record of what moved it. - The breaking fragment's Siblings line names agent-framework 0.23.2 and connectome-host 0.11.0, the versions now released, whose ranges are still ^0.14.1 and ^0.14.0 (npm view, 2026-10-21). Co-Authored-By: Claude Opus 5.5 --- .../request-image-ceilings.breaking.md | 4 ++-- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 2 +- src/types/strategy.ts | 20 ++++++++++++------- test/request-image-ceilings.test.ts | 10 +++++----- 5 files changed, 22 insertions(+), 16 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index 9e094753..144bea57 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling. +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling. It doesn't say so for an image the host's own limits would strip anyway, or for what a watermark loaded from the store strips, which carries no record of what moved it. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.1 depends on `^0.14.1`, and connectome-host 0.10.2 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. + - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.2 depends on `^0.14.1`, and connectome-host 0.11.0 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 2f0184b2..0e919014 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling, rather than because of a limit the host set. That's the image past the ceiling, unless the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips it anyway, or the images the hysteresis watermark strips when an image past the ceiling moves it, unless the host's own limits, the depth limit included, would strip them too. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling, rather than because of a limit the host set. That's the image past the ceiling, unless the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips it anyway, or the images the hysteresis watermark strips when an image past the ceiling moves it, unless the host's own limits, the depth limit included, would strip them too. A watermark the strategy loads from the store carries no record of what moved it, so the instance that loads it says nothing for what it strips. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 2d13ce06..48d9f96b 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -196,7 +196,7 @@ export function ceilingBoundLine( const limit = asLimit(value) === 0 ? `${value} (no limit)` : show(asLimit(value)); const mark = (name: string) => defaultMark(name, configured, defaulted); return `[${strategy}] an image fell past ${ceilingName} ${show(ceiling[kind])}${mark(ceilingName)}, ` + - `which holds ${limitName} ${limit}${mark(limitName)} to it, so older images are now shown as placeholders`; + `which holds ${limitName} ${limit}${mark(limitName)} to it, so some images are now shown as placeholders`; } /** diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 0906d90d..b7e969ac 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1311,7 +1311,9 @@ export interface AutobiographicalConfig { * the first render a select returns where images are shown as * placeholders because an image passed this ceiling, rather than because * of a limit the host set: the image past it, or those the hysteresis - * watermark strips with it. + * watermark strips with it that the host's own limits wouldn't strip. A + * watermark the strategy loads from the store carries no record of what + * moved it, so the instance that loads it says nothing for what it strips. */ imageCeiling?: number; @@ -1327,18 +1329,22 @@ export interface AutobiographicalConfig { * injects or appends aren't counted. membrane's byte cap is 28 MiB of * serialized messages unless `MEMBRANE_MAX_REQUEST_BYTES` says otherwise, * and the default, 24 MiB, leaves 4 MiB of that cap for what this doesn't - * count. membrane refuses a request over its cap before sending it. One - * that asks it to shed images, as this strategy's compression requests do, - * first has its oldest images replaced until it fits or none is left, and - * then only the Anthropic adapter's own check refuses it if it's still - * over (`src/live-image-limits.ts` says where each check runs). This + * count. Where membrane checks a request against its cap, it refuses one + * over it before sending it; an XML-mode continuation round isn't checked + * there, and of the provider adapters only Anthropic's checks again. A + * request that asks membrane to shed images, as this strategy's compression + * requests do, first has its oldest images replaced until it fits or none + * is left, and then only the Anthropic adapter's own check refuses it if + * it's still over (`src/live-image-limits.ts` says where each check runs). This * setting isn't read from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes * membrane's cap sets this too, keeping room under the cap for what this * doesn't count. * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A * value that isn't 0 or a finite number above it takes the default, and a * fraction is floored, to at least 1. Under a window limit of 0 or above - * this ceiling, the strategy says so once, as for `imageCeiling`. + * this ceiling, the strategy says so once, as for `imageCeiling`, with the + * same exceptions: not for an image the host's own limits would strip, and + * not for what a watermark loaded from the store strips. */ imageByteCeiling?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 36adc72e..fdc4a66a 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -468,7 +468,7 @@ describe('the line when an image first falls past an image ceiling', () => { const { lines, byStep, patterns } = await ceilingLines(strategy, [3, 2]); assert.deepEqual(patterns, ['.II', '...II'], `the ceiling of 2 strips at each compile for ${JSON.stringify(options)}`); assert.deepEqual(byStep.map((step) => step.length), [1, 0], `said once, at the first step, for ${JSON.stringify(options)}`); - assert.ok(lines[0]!.startsWith(`[${strategy.name}] an image fell past ${said}, so older images are now shown as placeholders`), lines[0]); + assert.ok(lines[0]!.startsWith(`[${strategy.name}] an image fell past ${said}, so some images are now shown as placeholders`), lines[0]); assert.match(lines[0]!, / ns=/, 'through the namespaced route the warning takes'); } }); @@ -1139,13 +1139,13 @@ describe('the line when an image first falls past an image ceiling', () => { const configured = (o: Record) => ({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 500, imageByteCeiling: 24 * MiB, ...o }); assert.equal(ceilingBoundLine('s', 'images', configured({}), new Set(['imageCeiling'])), - '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 0 (no limit) to it, so older images are now shown as placeholders'); + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 0 (no limit) to it, so some images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'images', configured({ imageCeiling: Number.NaN, maxLiveImages: 800 }), new Set()), - '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 800 to it, so older images are now shown as placeholders'); + '[s] an image fell past imageCeiling 500 (default), which holds maxLiveImages 800 to it, so some images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB }), new Set()), - '[s] an image fell past imageByteCeiling 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so older images are now shown as placeholders'); + '[s] an image fell past imageByteCeiling 24 MiB, which holds maxLiveImageBytes 30 MiB to it, so some images are now shown as placeholders'); assert.equal(ceilingBoundLine('s', 'bytes', configured({ maxLiveImageBytes: 30 * MiB + 1 }), new Set(['imageByteCeiling'])), - `[s] an image fell past imageByteCeiling 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so older images are now shown as placeholders`); + `[s] an image fell past imageByteCeiling 24 MiB (default), which holds maxLiveImageBytes ${30 * MiB + 1} bytes to it, so some images are now shown as placeholders`); }); it('the memory strategy says it when its hysteresis watermark trims against the byte ceiling', async () => { From b41c43695bbd4a6a1cb3b058b179bc56abab845a Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 07:35:44 -0400 Subject: [PATCH 15/67] docs(images): the ceilings count images as #219's limits do The live window now counts every image, nested and generated ones included, and an image's own message toward its depth (#219). So: - live-image-limits.ts' account and the added and changed fragments say the ceilings bound the images the window counts, as its limits count them, and that an image inside an opaque provider item isn't counted. - The breaking fragment's sibling line names agent-framework 0.24.0 and connectome-host 0.12.0, which depend on ^0.18.0. - The test that pinned a nested and a generated image passing the count ceiling now pins the ceiling binding them. - Three watermark tests had rested on the watermark measuring depth from newer messages only. Each is rebuilt on the same rule with the depth counted as #219 counts it: the ceiling's line when only the ceiling moves the watermark, the quiet when the depth limit brings the move on, and the quiet for an image a ceiling's move stripped that later messages took past the depth. Co-Authored-By: Claude Opus 5.5 --- changelog.d/request-image-ceilings.added.md | 2 +- .../request-image-ceilings.breaking.md | 2 +- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 28 +++---- test/request-image-ceilings.test.ts | 78 +++++++++---------- 5 files changed, 56 insertions(+), 56 deletions(-) diff --git a/changelog.d/request-image-ceilings.added.md b/changelog.d/request-image-ceilings.added.md index 5c7608ce..968a414a 100644 --- a/changelog.d/request-image-ceilings.added.md +++ b/changelog.d/request-image-ceilings.added.md @@ -1,2 +1,2 @@ -- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most top-level image blocks, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They don't bound a whole request: the window's images nested in tool results and its generated images, and anything a caller injects at compile time or appends to a compiled request, aren't counted; a compression prompt counts every image. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They count images as the window's limits do. They don't bound a whole request: an image inside an opaque provider item, and anything a caller injects at compile time or appends to a compiled request, aren't counted. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. - `ContextStrategy.imageCeilings()`, which both strategies implement, reports the two ceilings as the strategy applies them, a malformed one at its default. It reports configuration, not usage: a caller that wants a request within the ceilings counts that request's images itself. `ImageCeilings` is exported. diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index 144bea57..e1cd6638 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ - **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling. It doesn't say so for an image the host's own limits would strip anyway, or for what a watermark loaded from the store strips, which carries no record of what moved it. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** no released sibling's range admits this release: agent-framework 0.23.2 depends on `^0.14.1`, and connectome-host 0.11.0 on `^0.14.0`, so their installs keep getting 0.14.x. A sibling takes these changes on when it raises its range. + - **Siblings:** agent-framework 0.24.0 and connectome-host 0.12.0 depend on `^0.18.0` (npm, 2026-10-22), which admits a 0.18.x release and not a 0.19.0 one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 0e919014..a84b0e1a 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ -- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts: top-level `image` blocks, by the length of their base64 `source.data`. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. - Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling, rather than because of a limit the host set. That's the image past the ceiling, unless the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips it anyway, or the images the hysteresis watermark strips when an image past the ceiling moves it, unless the host's own limits, the depth limit included, would strip them too. A watermark the strategy loads from the store carries no record of what moved it, so the instance that loads it says nothing for what it strips. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 48d9f96b..518b236b 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -1,26 +1,26 @@ /** * The live-image window's limits, kept within the image ceilings. * - * A strategy's live window keeps the images it counts (top-level image - * blocks; see `AutobiographicalConfig.maxLiveImages`) newest-first up to a - * count (`maxLiveImages`) and a byte budget (`maxLiveImageBytes`). The image - * ceilings (`imageCeiling`, `imageByteCeiling`) bound those limits: an - * unlimited (0) limit means "up to the ceiling", and a limit above the - * ceiling keeps the ceiling's worth. They bound what the window counts and - * what a compression prompt carries, not a whole request. In the window, - * images nested in tool results and generated images aren't counted, nor is - * anything a caller injects at compile time or appends to a compiled - * request; a compression prompt counts every image it carries. A caller - * that wants a request within the ceilings counts that request's images - * itself and compares them with the strategy's `imageCeilings()`. + * A strategy's live window keeps the images it counts (every image, nested + * and generated ones included; see `AutobiographicalConfig.maxLiveImages`) + * newest-first up to a count (`maxLiveImages`) and a byte budget + * (`maxLiveImageBytes`). The image ceilings (`imageCeiling`, + * `imageByteCeiling`) bound those limits: an unlimited (0) limit means "up to + * the ceiling", and a limit above the ceiling keeps the ceiling's worth. They + * bound what the window counts and what a compression prompt carries, not a + * whole request: an image inside an opaque provider item isn't counted, nor + * is anything a caller injects at compile time or appends to a compiled + * request. A caller that wants a request within the ceilings counts that + * request's images itself and compares them with the strategy's + * `imageCeilings()`. */ /** * The provider refuses a request carrying more than 600 images and documents * together ("too many images and documents: 601 + 0 > 600", measured * 2026-10-13 and again on 2026-10-19). 500 leaves room for what the ceilings - * don't count: documents, images nested in tool results, generated images, - * and images a caller injects or appends. + * don't count: documents, images inside opaque provider items, and images a + * caller injects or appends. */ export const DEFAULT_IMAGE_CEILING = 500; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index fdc4a66a..6c28370d 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -1,6 +1,6 @@ /** - * The image ceilings (imageCeiling, imageByteCeiling): the most top-level - * image blocks, and their bytes, a strategy's live window keeps, and the most + * The image ceilings (imageCeiling, imageByteCeiling): the most images, and + * their bytes, a strategy's live window keeps, and the most * images and bytes a compression prompt carries. The live window keeps its * own limits within them, compression prompts carry no more than them, and * imageCeilings() reports them. @@ -188,10 +188,9 @@ describe('the live window within the image ceilings', () => { maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 0, imageByteCeiling: 0, }), 8), 'IIIIIIII'); }); - it(`${name}: the window counts top-level image blocks only, so a nested or a generated image passes the count ceiling`, async () => { - // What the ceilings bind is what the window counts. An image nested in - // a tool result, or a generated image, isn't one of those (a change - // that counts them would change this). + it(`${name}: the count ceiling binds the images the window counts, nested and generated ones included`, async () => { + // What the ceilings bind is what the window counts: every image, an + // image nested in a tool result and a generated image among them. const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImages: 0, imageCeiling: 1 }) }); try { @@ -211,9 +210,9 @@ describe('the live window within the image ceilings', () => { .filter((b) => b.type === 'tool_result') .flatMap((b) => (b as { content: ContentBlock[] }).content) .filter((b) => b.type === 'image').length; - assert.equal(nested, 2, 'both nested images stay, past a count ceiling of 1'); + assert.equal(nested, 0, 'the nested images are past a count ceiling of 1'); const kept = blocks.filter((b) => b.type === 'generated_image').length; - assert.equal(kept, 2, 'both generated images stay, past a count ceiling of 1'); + assert.equal(kept, 1, 'the newest image, a generated one, takes the one slot'); } finally { await manager.close(); rmSync(dir, { recursive: true, force: true }); @@ -861,25 +860,23 @@ describe('the line when an image first falls past an image ceiling', () => { }); it("the memory strategy's watermark says it when only the ceiling moves it, and nothing when the depth limit would move it anyway", async () => { - // Image messages price at about 1,600 tokens. Under a 4,000-token depth - // limit, the oldest of three is past the depth by the strips' measure - // (its own message counted) but not by the watermark's (newer messages - // only), so without a ceiling the watermark doesn't move. A count ceiling - // of 2 moves it, and its trim to half strips the middle image too, which - // the depth limit alone keeps: the ceiling decided. - const options = { maxLiveImages: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; + // Image messages price at about 1,600 tokens, so three of them are within + // a 6,000-token depth limit: without a ceiling the watermark doesn't move + // and nothing is stripped. A count ceiling of 2 moves it, and its trim to + // half strips the two older images, which the depth limit alone keeps: + // the ceiling decided. + const options = { maxLiveImages: 0, imageStripDepthTokens: 6000, imageStripHysteresisRatio: 0.5 }; const alone = await ceilingLines(memory({ ...options, imageCeiling: 0 }), [3]); assert.equal(alone.watermarks[0], null, 'the depth limit alone moves no watermark'); - assert.equal(alone.last, '.II'); + assert.equal(alone.last, 'III'); const ceiling = await ceilingLines(memory({ ...options, imageCeiling: 2 }), [3]); assert.notEqual(ceiling.watermarks[0], null); assert.equal(ceiling.last, '..I', 'the ceiling strips what the depth limit alone keeps'); assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); assert.ok(ceiling.lines[0]!.includes('an image fell past imageCeiling 2'), ceiling.lines[0]); - // Under a 3,000-token depth limit, the oldest of the three is past the - // depth by the watermark's measure too, from the message that brings the - // ceiling's first misfit on, so the depth limit moves the watermark with - // or without the ceiling, to the same place. + // Under a 3,000-token depth limit, the first image is past the depth once + // the second comes, before the count ceiling is passed, so the depth limit + // moves the watermark with or without the ceiling, to the same place. const deeper = { maxLiveImages: 0, imageStripDepthTokens: 3000, imageStripHysteresisRatio: 0.5 }; const deepAlone = await ceilingLines(memory({ ...deeper, imageCeiling: 0 }), [3]); assert.notEqual(deepAlone.watermarks[0], null, 'the depth limit moves the watermark'); @@ -911,14 +908,15 @@ describe('the line when an image first falls past an image ceiling', () => { } it("the memory strategy's watermark says nothing when the depth limit brings the move on, though the byte ceiling's trim lands it newer", async () => { - // A 4-byte image, about 3,100 tokens of text, then a 16-byte image. When - // the second image comes, the first is past a 4,000-token depth limit, so - // the watermark moves with or without a ceiling. Without one, its trim to - // half the depth lands on the first image. The trim to half of a 20-byte - // ceiling, 10 bytes, can't keep the 16-byte image either, so it lands - // there and strips it. No image passed the ceiling itself (20 bytes in - // all), so the line, which says one did, stays quiet. - const text = [{ type: 'text', text: 'word '.repeat(1800) }]; + // A 4-byte image, about 1,500 tokens of text, then a 16-byte image. The + // first image is within a 4,000-token depth limit until the second comes + // (about 1,600 + 1,500 + 1,600 tokens), so the watermark moves then, with + // or without a ceiling. Without one, its trim to half the depth lands on + // the first image. The trim to half of a 20-byte ceiling, 10 bytes, can't + // keep the 16-byte image either, so it lands there and strips it. No image + // passed the ceiling itself (20 bytes in all), so the line, which says one + // did, stays quiet. + const text = [{ type: 'text', text: 'word '.repeat(870) }]; const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; const alone = await watermarkRun({ ...options, imageByteCeiling: 0 }, [imageMessage('img-1', 4), text, imageMessage('img-2', 16)]); assert.notEqual(alone.watermark, null, 'the depth limit alone moves the watermark'); @@ -930,21 +928,23 @@ describe('the line when an image first falls past an image ceiling', () => { }); it("the memory strategy's watermark says nothing for an image its move strips that the depth limit strips anyway", async () => { - // A 16-byte image, about 1,500 tokens of text, then an 8-byte image. The - // first image passes a 20-byte ceiling (24 bytes), so the watermark moves - // onto it. By the watermark's measure, the messages newer than it, it's - // within a 4,000-token depth limit, so the host's own limits wouldn't - // move it. But the strip counts the image's own message too, and by that - // measure it's past the depth: the render strips it with or without the - // ceiling, so the ceiling decided nothing. + // A 16-byte image, an 8-byte image, then about 1,500 tokens of text. The + // watermark moves as each message is stored. When the second image comes, + // the first passes a 20-byte ceiling (24 bytes), so the watermark moves + // onto it, while it's still within a 4,000-token depth limit (about 3,200 + // tokens deep), where the host's own limits wouldn't move it. The text + // then takes it past the depth (about 4,700), so the render that says the + // line strips it with or without the ceiling: the ceiling decided nothing + // there. Without the ceiling, the depth limit moves the watermark onto it + // when the text comes. const text = [{ type: 'text', text: 'word '.repeat(870) }]; - const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }; - const messages = [imageMessage('img-1', 16), text, imageMessage('img-2', 8)]; + const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.9 }; + const messages = [imageMessage('img-1', 16), imageMessage('img-2', 8), text]; const alone = await watermarkRun({ ...options, imageByteCeiling: 0 }, messages); - assert.equal(alone.watermark, null, 'without the ceiling the watermark stays'); + assert.notEqual(alone.watermark, null, 'without the ceiling, the depth limit moves the watermark'); assert.equal(alone.kept, '.I', 'the depth limit strips the first image'); const ceiling = await watermarkRun({ ...options, imageByteCeiling: 20 }, messages); - assert.notEqual(ceiling.watermark, null, 'the ceiling moves the watermark'); + assert.equal(ceiling.watermark, alone.watermark, 'the ceiling moved it to the same place, earlier'); assert.equal(ceiling.kept, '.I', 'the render is the same'); assert.deepEqual(ceiling.lines, []); }); From 2f9e7f31b90d14f99ee794d1da791a5c6f4c0635 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 07:37:13 -0400 Subject: [PATCH 16/67] refactor(images): windowed passthrough notes every image's deciding ceilings The policy's guard around noting them only saved an object per image message, and no test could tell it was there. Co-Authored-By: Claude Opus 5.5 --- src/strategies/windowed-passthrough.ts | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 646c4caa..200d6565 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -581,11 +581,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; const strip = tooDeep || overCount || overBytes; const deciding = ceilingsDeciding({ images: overCount, bytes: overBytes }, tooDeep, ceilingSet); - if (deciding.images || deciding.bytes) { - const decided = (ceilings[i] ??= { images: false, bytes: false }); - decided.images ||= deciding.images; - decided.bytes ||= deciding.bytes; - } + const decided = (ceilings[i] ??= { images: false, bytes: false }); + decided.images ||= deciding.images; + decided.bytes ||= deciding.bytes; if (strip) { tokens += strippedImageSaving(image, placeholderPrice, calibration); } else { From bbca9e3334e5ad25a86b56cbe335fe6ed3d78e77 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Thu, 22 Oct 2026 07:44:47 -0400 Subject: [PATCH 17/67] feat: carry membrane's system changes in place under every strategy; firstChanged on compile results membrane 0.7.0's system_change block (SystemChangeContent) changes the tools the model is offered, and its system text, from a place in the conversation on. context-manager now carries a message of such blocks: priced, alone in its message, compiled unaltered in its place under every strategy, never dropped, and kept out of compression. The membrane floor rises to ^0.7.0, the release that carries the block. - Kept in place. The autobiographical strategy (and KnowledgeStrategy) keeps a message of changes raw at resolvedPinBounds, the one site the chunker, the adaptive picker and its solvers, and the hierarchical emission read. After a head-window reset, the changes stored before the head are part of the head, ahead of the head window, in order (headPositions), on the hierarchical and adaptive paths and in kv-unified's cache-check rebuild, so a removal never runs before the definition it removes. The middle's ranges come from one function (middleRanges), which also ends main's overlap with the head window disabled after a reset. - KnowledgeStrategy's chunker closes a chunk at a change between two of its compressible messages, and where the L1 summary covering its messages changes, except between a call and the result right after it, where closing would let pairing repair drop the result; that chunk is refused by the overlap guard and both are sent raw, as on main. - The passthrough strategies send the changes before their window ahead of it, in order, and count them first; windowed's re-anchor target is the changes plus reAnchorFraction of the room they leave, and its fallback anchors at the newest message that isn't a change. Windowed's refusal names one of three stages by what its numbers hold: the newest message alone, its system changes and the newest conversation message, or its system changes alone. Its anchor moves only for a compile that sends. - None is dropped. Changes alone over the budget are refused by every strategy with OverBudgetError and clearedBy empty, since only the caller consolidating them clears it. The adaptive plan refusal names compression only while the head, the tail and the changes between them fit, and its head/tail floor takes the newest conversation message past any trailing changes. PassthroughStrategy refuses when the newest conversation message doesn't fit beside the changes, naming newer-message while they leave room for one. - Kept out of compression: every summarizer request leaves changes out, as it strips thinking, and merging messages for the request passes over a message left empty, so no empty turn or separator marks where it sat. A head-window reset's transition summary leaves out a message of changes alone. - Priced from its JSON in MessageStore and ContextLog; refused at a write when beside other blocks, inside a tool result, or malformed (each field, element and definition field named by its place; a definition needs an object inputSchema of type 'object'); left out at a read, with a log line. - Render stats count the head as a compile sends it, the changes before a reset head included; the structural repair's check takes the newest message that isn't a change as the newest turn; the coverage check names an unrendered change before a reset head as the head's. - CompileResult.firstChanged: on a committing compile, the index of the first message, injections included, whose participant or content bytes differ from the last committing compile's, or null for the first. It's measured after injections, recorded last, and keeps no copy of stored content. Tests: test/system-changes.test.ts (48 tests) and one in test/kv-unified-render-state.test.ts. Co-Authored-By: Claude Opus 5.5 --- changelog.d/system-changes.added.md | 41 + changelog.d/system-changes.fixed.md | 16 + package-lock.json | 8 +- package.json | 2 +- src/adaptive/picker.ts | 19 +- src/context-log.ts | 3 + src/context-manager.ts | 64 ++ src/message-store.ts | 4 + src/strategies/autobiographical.ts | 256 ++++-- src/strategies/knowledge.ts | 79 +- src/strategies/passthrough.ts | 82 +- src/strategies/windowed-passthrough.ts | 110 ++- src/system-changes.ts | 55 ++ src/types/context.ts | 26 + src/types/strategy.ts | 5 +- src/well-formed-content.ts | 164 +++- test/kv-unified-render-state.test.ts | 30 + test/l1-recall-raw-pins-frontier.test.ts | 2 +- test/system-changes.test.ts | 1014 ++++++++++++++++++++++ 19 files changed, 1832 insertions(+), 148 deletions(-) create mode 100644 changelog.d/system-changes.added.md create mode 100644 changelog.d/system-changes.fixed.md create mode 100644 src/system-changes.ts create mode 100644 test/system-changes.test.ts diff --git a/changelog.d/system-changes.added.md b/changelog.d/system-changes.added.md new file mode 100644 index 00000000..d6ad6840 --- /dev/null +++ b/changelog.d/system-changes.added.md @@ -0,0 +1,41 @@ +- System changes: a message holding only membrane's `system_change` blocks + (`SystemChangeContent`, membrane's release that adds them) changes the tools + and system text the model is offered from its place on, and every strategy + compiles it unaltered, in its place among the raw messages. (Under the + autobiographical strategy's `positionedRecallPairs: false`, the middle's + summaries go ahead of its raw messages, as they always have.) The + autobiographical strategy, and with it `KnowledgeStrategy`, keeps it raw as + a pin keeps a message, so no fold takes it and no chunk holds or spans it; + after a head-window reset that leaves messages in the head window, the + changes stored before the new head are part of the head, ahead of the head + window, in order. `PassthroughStrategy` and + `WindowedPassthroughStrategy` keep one that falls before their window, + ahead of the window and priced into it, and the windowed strategy sizes a + re-anchored window as its fraction of the room the changes leave. None is + dropped: when the changes alone exceed the budget (beyond the + autobiographical strategy's grace, and on its hierarchical path only where + `enforceBudget` is on), every strategy refuses with `OverBudgetError`, + `clearedBy` empty, until the caller consolidates them into `tools` and + `system`; a dry compile reports where the strategy's other refusals do, + which for the passthrough strategies is nowhere. They also refuse when the + newest conversation message doesn't fit beside the changes, naming + `newer-message` while they leave room for one. + Compression never reads one. It's priced from its JSON. A write refuses one + beside other blocks or inside a tool result, and one whose `removeTools` or + `addSystem` isn't an array of strings, or whose `defineTools` isn't an array + of tool definitions, each with a non-empty string `name`, a string + `description` and an object `inputSchema` whose `type` is `'object'`; a read + leaves such a change out. Ingress sharding never splits a message of them, + since it holds no text block. + Only membrane's `NativeFormatter`, on an adapter that sends system changes, + sends one; elsewhere membrane refuses the request. Rolling back to a + release without them isn't supported once a store holds one. +- `CompileResult.firstChanged`, optional: on a committing compile, the index + of the first compiled message, injections included, whose participant or + content bytes differ from the last committing compile's message at that + index, or that has none there, so after appends alone it is the first + appended index; an injection placed beside the newest user message moves + with each new user message, so after one it's that injection's old index. + It compares bytes, so it holds across a branch switch or a new strategy, + and it's `null` only for the manager's first committing compile. A dry + compile leaves it out and doesn't move it. diff --git a/changelog.d/system-changes.fixed.md b/changelog.d/system-changes.fixed.md new file mode 100644 index 00000000..ee816531 --- /dev/null +++ b/changelog.d/system-changes.fixed.md @@ -0,0 +1,16 @@ +- A summarizer request no longer carries an empty turn, or a `---` separator + with nothing after it, where a message left empty for it sat, as one of + thinking alone was: merging consecutive messages passes over an empty one, + and its neighbours join as they would without it. +- With the head window disabled (`headWindowTokens: 0`), a compile after + `resetHeadWindow` under `adaptiveResolution` no longer sends history from + before the reset twice, and under kv-unified no longer throws + `CanonicalForestError` (`duplicate-chunk-id`) for it: the middle's ranges + overlapped each other, and the recent window when it starts before the + reset. +- A `WindowedPassthroughStrategy` compile that refuses no longer moves the + window's anchor; only a compile that sends does. So a retry with a larger + budget still sends the older turns that fit. +- `getRenderStats()` before the first compile no longer counts a message in + both the head and the tail when the recent window starts inside the head + window: the tail starts where the head ends, as a compile's does. diff --git a/package-lock.json b/package-lock.json index 77596b38..33e9383b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -10,7 +10,7 @@ "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", - "@animalabs/membrane": "^0.6.0" + "@animalabs/membrane": "^0.7.0" }, "devDependencies": { "@types/node": "^22.0.0", @@ -27,9 +27,9 @@ "license": "MIT" }, "node_modules/@animalabs/membrane": { - "version": "0.6.0", - "resolved": "https://registry.npmjs.org/@animalabs/membrane/-/membrane-0.6.0.tgz", - "integrity": "sha512-Dt3VMTqmUekLqMdsIzS70PbVbyS+akAiBn9sqKAy28+TNDiUSQiJxKO1o2Pw39koa0J7RcCEBqH/efzouIDx5A==", + "version": "0.7.0", + "resolved": "https://registry.npmjs.org/@animalabs/membrane/-/membrane-0.7.0.tgz", + "integrity": "sha512-e+1ER8NovdCNR2NG9ZO70l7igVSgg798UkP8OtAZu0EzGYVTe1VfPOOkNSo7tEPh8bcav68N2jVB2IkmeC5IEQ==", "license": "MIT", "dependencies": { "@anthropic-ai/sdk": "^0.52.0" diff --git a/package.json b/package.json index f3b4beb9..f4973342 100644 --- a/package.json +++ b/package.json @@ -39,7 +39,7 @@ "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", - "@animalabs/membrane": "^0.6.0" + "@animalabs/membrane": "^0.7.0" }, "devDependencies": { "@types/node": "^22.0.0", diff --git a/src/adaptive/picker.ts b/src/adaptive/picker.ts index 5a26635a..0774af03 100644 --- a/src/adaptive/picker.ts +++ b/src/adaptive/picker.ts @@ -55,10 +55,12 @@ export interface OverBudgetDiagnostics { /** * What clears an over-budget refusal at run time besides a larger budget, - * which clears any of them: waiting, or a hot setting. Cold configuration, + * which clears any of them but the structural repair's (a newest turn the + * repair dropped, which no budget brings back): waiting, or a hot setting. Cold configuration, * such as the head window's size, isn't named, and neither is a head-window * reset (`resetHeadWindow`), which clears a head that's over but starts a new - * topic, nor newer messages that would shrink a head holding images, since + * topic (the system changes stored before the new head stay in it, so a + * reset clears none of what they cost), nor newer messages that would shrink a head holding images, since * the image strip takes the oldest images first but may not take enough. * The strategy that refuses sets it, since it knows which part of the * context is over and what can shrink that part: @@ -115,14 +117,17 @@ export interface OverBudgetDiagnostics { * plan must also still have summaries to produce. Whether that work is * enough to fit isn't checked, so a host can tell an agent that * compression may make room, not that it will. - * - `'newer-message'`: the newest message alone is over the budget, and a - * newer message that fits moves the window past it. + * - `'newer-message'`: the newest message, with the system changes sent + * beside it, is over the budget, and a newer conversation message that + * fits beside them moves the window past it (a newer change doesn't). * - `'smaller-recent-window'`: the raw recent window, which compression never * touches, is what's over the budget, and the head fits beside the newest * turn as the compile must keep it raw, or beside everything from a - * compression hold if one starts earlier. That's the newest message (for - * one split into shards, its newest shard, since the earlier shards can - * fold), unless the newest message's body holds a tool_result, on its first + * compression hold if one starts earlier, and beside the system changes + * outside the head, which stay raw wherever the window starts. That's the + * newest conversation message, a system change being none, with the + * changes after it (for one split into shards, its newest shard, since the + * earlier shards can fold), unless the newest message's body holds a tool_result, on its first * shard: then it's that whole body and the whole call before it, which the * result must follow raw. A smaller * `recentWindowTokens` (the hot `tailTokens`) moves the rest out of the raw diff --git a/src/context-log.ts b/src/context-log.ts index f2856e90..3089cac1 100644 --- a/src/context-log.ts +++ b/src/context-log.ts @@ -341,6 +341,9 @@ export class ContextLog { case 'audio': case 'video': return 1000; + case 'system_change': + // Mirrors MessageStore.computeBlockTokensRaw: priced from its JSON. + return this.tokenEstimator(JSON.stringify(block)); default: // Mirrors MessageStore.computeBlockTokensRaw: membrane's XML-history // carriers (tool_attempt, tool_notice) are priced as the markup they diff --git a/src/context-manager.ts b/src/context-manager.ts index baa0e19a..71204fde 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -71,6 +71,21 @@ import type { type InitializationStamp = StoreBranchGeneration & { id: string; attempt: number }; +/** + * A compiled message as `firstChanged` compares it. An injection's content + * is the caller's, which may change in place, so its bytes are taken when + * it's compiled. Other content is the store's or the strategy's, which + * nothing changes in place (as the store's own per-message caches assume), + * so the same array is the same content, and its bytes are taken only when a + * comparison with another array needs them. + */ +interface CompiledMessageKey { + readonly participant: string; + readonly content: readonly ContentBlock[]; + readonly injected: boolean; + bytes?: string; +} + /** * Base configuration for ContextManager. */ @@ -254,6 +269,12 @@ export class ContextManager { private compressionHolds = new Set(); /** Summaries the strategy rendered but couldn't describe, already reported. */ private undescribedSummariesReported = new Set(); + /** + * The last committing compile's messages, for the next one's + * `firstChanged`, whatever branch or strategy it read; null before this + * manager's first committing compile. + */ + private lastCompiled: CompiledMessageKey[] | null = null; /** Sharded adds held on ingress: first-shard id -> every shard id. */ private compressionHoldGroups = new Map(); /** heldAt / expiresAt per held id (expiresAt absent = no timeout). */ @@ -1245,6 +1266,9 @@ export class ContextManager { ...(layout ? { journalMark: this.foldJournal.mark() } : {}), }; if (this.debugLogContext) this.logCompiledContext(result); + // Last, so a compile that throws before returning, which no caller + // sends, is never the one the next is compared with. + if (!opts?.dryRun) result.firstChanged = this.measureFirstChanged(result.messages, injected); return result; }; @@ -1317,6 +1341,46 @@ export class ContextManager { return finish({ messages, systemInjections }, injectedContent); } + /** + * The index of the first compiled message whose participant or content + * bytes differ from the last committing compile's message at that index, or + * that has none there (so the first appended index after appends alone, and + * the message count when these messages are that compile's or a prefix of + * them); null before this manager's first committing compile. The + * comparison is by bytes, so it holds across a branch or strategy switch, + * which the manager can't always see (a direct switch of the store's + * branch away and back). Cache markers are message flags, not content, so + * they don't count. Records these messages for the next + * (CompiledMessageKey). + */ + private measureFirstChanged( + messages: readonly NormalizedMessage[], + injected: readonly (readonly ContentBlock[])[], + ): number | null { + const fromInjection = new Set(injected); + const keys: CompiledMessageKey[] = messages.map((message) => fromInjection.has(message.content) + ? { participant: message.participant, content: message.content, injected: true, bytes: JSON.stringify(message.content) } + : { participant: message.participant, content: message.content, injected: false }); + const before = this.lastCompiled; + if (!before) { + this.lastCompiled = keys; + return null; + } + const bytesOf = (key: CompiledMessageKey): string => (key.bytes ??= JSON.stringify(key.content)); + let i = 0; + for (; i < keys.length && i < before.length; i++) { + const now = keys[i]!; + const then = before[i]!; + if (now.participant !== then.participant) break; + if (now.content === then.content && !now.injected && !then.injected) continue; + if (bytesOf(now) !== bytesOf(then)) break; + } + // Recorded once the comparison has returned: one that throws leaves the + // last compile that returned as the next one's comparison. + this.lastCompiled = keys; + return i; + } + /** * The rendered layout of one compile, or null when the strategy does not * report its layout, or names a summary it cannot describe (a layout that diff --git a/src/message-store.ts b/src/message-store.ts index f0572851..bb2d479a 100644 --- a/src/message-store.ts +++ b/src/message-store.ts @@ -1157,6 +1157,10 @@ export class MessageStore { case 'audio': case 'video': return 1000; // Default estimate for media + case 'system_change': + // Its tool definitions and system text go out whole, so it's priced + // from its JSON, as a tool call's input is. + return jsonTokenEstimator(JSON.stringify(block)); default: return MessageStore.xmlHistoryCarrierTokens(block); } diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 4bad3d90..883c9463 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -81,6 +81,7 @@ import { CAP_OMISSION_TEXT, capContent, cappedPriceExtra, cappedWhole, isCapMark import type { MessageId } from '../types/message.js'; import type { IngressChunkResult } from '../types/strategy.js'; import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; +import { holdsSystemChange, isSystemChange, systemChangePositions } from '../system-changes.js'; /** A persisted reset record is trusted only if it has the expected shape. */ function isCalibrationReset(v: unknown): v is CalibrationReset { @@ -1655,6 +1656,19 @@ function stripThinkingBlocks(content: ContentBlock[]): ContentBlock[] { ); } +/** + * A raw message's content as it enters compression input: its thinking + * stripped (stripThinkingBlocks), and its system changes left out. A system + * change alters the request's tools and system text, not the conversation, and + * a summarizer's request would apply it, or be refused for it + * (system-changes.ts). A message of system changes alone enters empty, and + * every request path drops an empty message before sending. + */ +function compressionInputContent(content: ContentBlock[]): ContentBlock[] { + const stripped = stripThinkingBlocks(content); + return holdsSystemChange(stripped) ? stripped.filter((b) => !isSystemChange(b)) : stripped; +} + /** * Capture the summarizer's verbatim response blocks for storage on the * SummaryEntry — but only when the response carried reasoning blocks. @@ -3787,14 +3801,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected resolvedPinBounds(messages: StoredMessage[]): Map { const out = new Map(); const pinned = this.pinnedPositions(messages); - if (pinned.size === 0) return out; - const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; - for (const position of pinned) { - const bound = bounds?.get(position); - if (bound?.level !== undefined) out.set(position, { level: bound.level }); - else if (bound?.maxLevel !== undefined) out.set(position, { maxLevel: bound.maxLevel }); - else out.set(position, 'raw'); + if (pinned.size > 0) { + const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; + for (const position of pinned) { + const bound = bounds?.get(position); + if (bound?.level !== undefined) out.set(position, { level: bound.level }); + else if (bound?.maxLevel !== undefined) out.set(position, { maxLevel: bound.maxLevel }); + else out.set(position, 'raw'); + } } + // A message of system changes is kept raw, as a pin keeps a message: a + // fold would take its change out of what follows it (holdsSystemChange). + for (const position of systemChangePositions(messages)) out.set(position, 'raw'); return out; } @@ -8353,9 +8371,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { const pos = messages.findIndex(x => x.id === id); if (pos < 0) return 'not-in-window'; // Before the LIVE head start (head-window reset moved the anchor): - // this is compressible middle history, not head — a distinct defect - // from a verbatim-head message going unrendered. - if (pos < regions.headStart) return 'before-head-start-not-rendered'; + // this is middle history, not head — a distinct defect from a + // verbatim-head message going unrendered — compressible but for a + // system change, which stays raw there, except while the head window + // holds messages, which is where a reset leaves it (headPositions). + if (pos < regions.headStart) { + return holdsSystemChange(messages[pos].content) && Math.min(regions.headEnd, messages.length) > regions.headStart + ? 'in-head-not-rendered' + : 'before-head-start-not-rendered'; + } if (pos < regions.headEnd) return 'in-head-not-rendered'; if (pos >= regions.recentStart) return 'in-tail-not-rendered'; return 'in-middle-not-rendered'; @@ -8629,8 +8653,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { const sumTokens = (slice: StoredMessage[]): number => slice.reduce((acc, m) => acc + store.estimateTokens(this.prunedForSend(store, m)), 0); - const headMsgs = messages.slice(headStart, headEnd); - const tailMsgs = messages.slice(recentStart); + // The head as a compile sends it: after a reset, the system changes + // stored before the window, then the window (headPositions). + const headMsgs = this.headPositions(messages, headStart, headEnd).map((i) => messages[i]!); + // The tail starts no earlier than the head ends, as a compile's does, so + // no message counts in both. + const tailMsgs = messages.slice(Math.max(recentStart, headEnd)); const live = (level: SummaryLevel) => this.summaries.filter(s => s.level === level && !s.mergedInto); @@ -9225,7 +9253,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { headCoveredSkipped++; continue; } - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } // ---- 2. Prior recall pairs (budget-capped, above) and the raw middle ---- @@ -9244,7 +9272,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { ].sort((a, b) => a.position - b.position); for (const { summary: s, message: m } of middle) { if (m) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); continue; } llmMessages.push({ @@ -9277,7 +9305,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ---- 5. Chunk messages raw ---- for (const m of chunk.messages) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } // ---- 6. Instruction (reading-mode aware; worked out above) ---- @@ -10643,7 +10671,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const buildSub = (subset: typeof chunk.messages, target: number): NormalizedRequest => { const msgs = [ { participant: 'Context Manager', content: [{ type: 'text', text: COMPRESSION_MARKER }] as ContentBlock[] }, - ...subset.map((m) => ({ participant: m.participant, content: stripThinkingBlocks(m.content) })), + ...subset.map((m) => ({ participant: m.participant, content: compressionInputContent(m.content) })), { participant: 'Context Manager', content: [{ type: 'text', text: this.applyIdentityReminder(this.getCompressionInstruction(chunk, target), this.chunkIsWitnessed(chunk)) }] as ContentBlock[] }, ]; const cleaned = stripUnpairedToolBlocks(this.collapseConsecutiveMessages(splitMixedToolMessages(msgs))); @@ -11795,7 +11823,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { headCoveredSkipped++; continue; } - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } if (headCoveredSkipped > 0) { this.diag.warn( @@ -11879,7 +11907,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const m = allMessages[i]; if (priorSummaryMessageIds.has(m.id)) continue; if (sourceLeafIds.has(m.id)) continue; - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } } @@ -11935,7 +11963,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const messageId of src.sourceIds) { const m = messageById.get(messageId); if (m) { - llmMessages.push({ participant: m.participant, content: stripThinkingBlocks(m.content) }); + llmMessages.push({ participant: m.participant, content: compressionInputContent(m.content) }); } } } else { @@ -12840,13 +12868,24 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Compute the fixed raw windows first. They are hard reservations, not // best-effort phases: foldable history may use only the space left after // every head and tail message has been accounted for. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + // The head's positions: after a reset, the system changes stored before + // the head window, then the window (headPositions). + const head = this.headPositions(messages, headStart, headEnd); + for (const i of head) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; headMessageIds.add(msg.id); headTokens += tokens; } const effectiveRecentStart = Math.max(recentStart, headEnd); + // System changes stay raw wherever they are (resolvedPinBounds), and + // nothing a refusal can name shrinks them: the ones outside the head and + // the tail are sent raw in the middle, beside what compression folds. + let middleChangeTokens = 0; + for (const i of systemChangePositions(messages)) { + if (i >= effectiveRecentStart || headMessageIds.has(messages[i].id)) continue; + middleChangeTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; + } for (let i = effectiveRecentStart; i < messages.length; i++) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; @@ -12877,14 +12916,26 @@ export class AutobiographicalStrategy implements ResettableStrategy { // does a newest turn that starts inside the head. let floorTokens = 0; if (messages.length > 0) { - const newestBody = bodyStart((k) => messages[k], messages.length - 1); + // A system change is no conversation message, so the newest one is + // the newest that isn't one, and the changes after it stay raw with + // it (counted from floorStart below). + let newest = messages.length - 1; + while (newest > 0 && holdsSystemChange(messages[newest].content)) newest--; + const newestBody = bodyStart((k) => messages[k], newest); const newestTurn = newestBody > 0 && this.hasToolResult(messages[newestBody]) ? bodyStart((k) => messages[k], newestBody - 1) - : messages.length - 1; + : newest; const floorStart = Math.min(newestTurn, this.holdBoundary(store)); for (let i = floorStart; i < messages.length; i++) { floorTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; } + // A system change outside the head stays raw wherever the recent + // window starts, so the floor holds every one before it too, and + // when the changes alone exceed the budget nothing is named. + for (const i of systemChangePositions(messages)) { + if (i >= floorStart) break; + if (!headMessageIds.has(messages[i].id)) floorTokens += pse[i] + this.cappedForSend(store, messages[i], msgCap).extra; + } } throw this.overBudgetError(budget, { stage: 'Head/tail reservation exceeded budget', @@ -12909,7 +12960,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Emit the already-reserved head entries raw, each cut at the cap as any // raw message is. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + for (const i of head) { const msg = messages[i]; const { content, extra } = this.cappedForSend(store, msg, msgCap); const tokens = pse[i] + extra; @@ -12978,15 +13029,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Excluding that zone made pre-transition history invisible to the picker // (never planned, never rendered): the adaptive variant of the frontier // gap the coverage invariant exists to refuse. - const middleSegments: ReadonlyArray = headStart > 0 - ? [[0, headStart], [headEnd, effectiveRecentStart]] - : [[headEnd, effectiveRecentStart]]; + // A system change before a reset head is in the head (headPositions), so + // the middle walks pass over it. + const middleSegments = AutobiographicalStrategy.middleRanges(headStart, headEnd, effectiveRecentStart); const pickerChunks: PickerChunk[] = []; const salience = this.salienceByPosition(messages); for (const [segStart, segEnd] of middleSegments) for (let i = segStart; i < segEnd && i < messages.length; i++) { const msg = messages[i]; + if (headMessageIds.has(msg.id)) continue; const ch = chunkAtPosition ? chunkAtPosition[i] : chunksByMessageId.get(msg.id); // This is the chunk's raw price. The shard path in step 5 sends a run // of a body's raw shards as one entry, their text joined and uncut, so @@ -13015,7 +13067,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Also include head and tail in PickerChunks (so token accounting matches) // — but mark them as in-head/in-tail so the picker won't fold them. - for (let i = headStart; i < headEnd && i < messages.length; i++) { + for (const i of head) { const msg = messages[i]; const tokens = pse[i] + this.cappedForSend(store, msg, msgCap).extra; pickerChunks.push({ @@ -13329,8 +13381,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Taken at a refusal of this select (the plan check's, or a later // stage's), or for a dry run's preview of a plan over the budget, since a // plan that fits and renders needs no list. + // Compression shrinks neither the head and the tail nor a system change + // between them: when those alone exceed the budget, nothing the list + // names clears it. const planClearedBy = (): OverBudgetClearedBy[] => - result.produced.length > 0 && this.compressionWorkQueued(store, 'adaptive') ? ['compression'] : []; + headTokens + tailTokens + middleChangeTokens <= rejectionBudget && result.produced.length > 0 && this.compressionWorkQueued(store, 'adaptive') ? ['compression'] : []; const planFits = result.finalTokens <= rejectionBudget; // Record what a dry run needs (select() puts back the transition @@ -13422,6 +13477,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { let i = segStart; while (i < segEnd && i < messages.length) { const msg = messages[i]; + if (headMessageIds.has(msg.id)) { + i++; // sent in the head (headPositions) + continue; + } if (msg.bodyGroupId) { // Collect this body's consecutive shards. Another ingestion of the @@ -13655,9 +13714,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { // newest shard alone outgrows the recent window. Only that body's shards // count: an earlier ingestion of the same text shares the group id, and its // survival proves nothing about the newest turn. - const newest = messages[messages.length - 1]; + // A system change is no turn and always stays raw, so the newest turn is + // the newest message that isn't one, as for the floor above. + let newestAt = messages.length - 1; + while (newestAt > 0 && holdsSystemChange(messages[newestAt].content)) newestAt--; + const newest = messages[newestAt]; if (newest) { - const newestBody = bodyBoundsIn(messages, messages.length - 1); + const newestBody = bodyBoundsIn(messages, newestAt); const newestBodyIds = new Set(messages.slice(newestBody.from, newestBody.to + 1).map(m => m.id)); const newestRetained = merged.some( entry => (entry.sourceMessageId !== undefined && newestBodyIds.has(entry.sourceMessageId)) || @@ -13668,10 +13731,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { stage: 'Structural repair did not retain the newest turn', budget: rejectionBudget, actual: totalTokens + tailTokens, - // The newest turn is what's missing, and only room for it brings it - // back. A newer message makes a different turn the newest, and this - // one then sits in the middle raw until its chunk closes and - // compression summarizes it, so the next compile is no smaller. + // The newest turn is what's missing: the repair trimmed it as an + // unanswered call, which no budget brings back. A newer message + // makes a different turn the newest, and this one then sits in the + // middle raw until its chunk closes and compression summarizes it, + // so the next compile is no smaller. clearedBy: [], diagnostics: { middleCountUnit: 'picker-chunks', @@ -14955,14 +15019,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { } pending = []; }; - for (let i = window.headStart; i < window.headEnd && i < messages.length; i++) pushRaw(messages[i]!); - const segments: ReadonlyArray = window.headStart > 0 - ? [[0, window.headStart], [window.headEnd, window.recentStart]] - : [[window.headEnd, window.recentStart]]; - for (const [segStart, segEnd] of segments) { + const head = this.headPositions(messages, window.headStart, window.headEnd); + for (const i of head) pushRaw(messages[i]!); + const inHead = new Set(head); + for (const [segStart, segEnd] of AutobiographicalStrategy.middleRanges(window.headStart, window.headEnd, window.recentStart)) { let i = segStart; while (i < segEnd && i < messages.length) { const msg = messages[i]!; + if (inHead.has(i)) { + i++; + continue; + } if (msg.bodyGroupId) { const groupStart = i; i = bodyEnd(Math.min(segEnd, messages.length), (k) => messages[k]!, groupStart) + 1; @@ -15652,7 +15719,38 @@ export class AutobiographicalStrategy implements ResettableStrategy { // any raw message is (from headStart, not necessarily 0) const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); - for (let i = headStart; i < headEnd && i < messages.length; i++) { + // System changes stay raw wherever they are (resolvedPinBounds), and no + // newer message or compression shrinks them, so when they alone exceed + // the grace limit only the caller, consolidating them, clears it. That + // refusal comes first, as every strategy gives it; an eviction from the + // recent window while the changes fit stays a loss the coverage + // invariant refuses, as any there is. + let changeTokens = 0; + for (const i of systemChangePositions(messages)) { + const msg = messages[i]; + changeTokens += store.estimateTokens(this.prunedForSend(store, msg)) + this.cappedForSend(store, msg, msgCap).extra; + } + if (changeTokens > graceLimit) { + throw this.overBudgetError(budget, { + stage: 'Hierarchical: its system changes alone exceed the budget', + budget: graceLimit, + actual: changeTokens, + clearedBy: [], + diagnostics: { + middleCountUnit: 'selection-items', + deepestLevelKind: 'attempted', + headTokens: changeTokens, + tailTokens: 0, + middleTokens: 0, + middleChunkCount: 0, + deepestLevel: 0, + }, + }); + } + // After a reset, the system changes stored before the head window go + // first, in the head (headPositions). + const head = this.headPositions(messages, headStart, headEnd); + for (const i of head) { const msg = messages[i]; const { content, extra } = this.cappedForSend(store, msg, msgCap); // The pruned form's price with its images whole: this selection's @@ -15667,8 +15765,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { actual: totalTokens + tokens, // The head is sent whole by definition, and its size is cold // configuration (headWindowTokens isn't a hot setting). Besides a - // larger budget, a head-window reset clears it, which starts a new - // topic and isn't a lever the list names (OverBudgetClearedBy). + // larger budget, a head-window reset clears the window's share of + // it, though not the system changes stored before the new head, + // which go in it; a reset starts a new topic and isn't a lever the + // list names (OverBudgetClearedBy). // Newer messages can shrink a head that holds images, since the // image strip takes the oldest images first (maxLiveImages, // maxLiveImageBytes, imageStripDepthTokens), but whether they'd @@ -15844,7 +15944,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The middle is everything before the recent window that is not the LIVE // head window — including [0, headStart) after a head-window reset, which // is compressible-but-often-unsummarized history, not head. - const inLiveHead = (i: number): boolean => i >= headStart && i < headEnd; + // The head, as Phase 0 sent it: the live window, and after a reset the + // system changes before it (headPositions). + const inHead = new Set(head); + const inLiveHead = (i: number): boolean => inHead.has(i); for (let i = 0; i < effectiveRecentStart && i < messages.length; i++) { if (inLiveHead(i)) continue; if (pinnedPositionsSet.has(i)) { @@ -15936,7 +16039,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const covered = new Set(); for (const s of selectedSummaries) for (const leaf of leavesOf(s)) covered.add(leaf); const rawPlanned = new Set(); - for (let i = headStart; i < headEnd && i < messages.length; i++) rawPlanned.add(messages[i].id); + for (const i of head) rawPlanned.add(messages[i].id); for (let i = effectiveRecentStart; i < messages.length; i++) rawPlanned.add(messages[i].id); for (const p of middleRaw) rawPlanned.add(p.msg.id); const selectedIdSet = new Set(selectedSummaries.map(s => s.id)); @@ -16265,13 +16368,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * A stored message's content priced as a compression request sends it: its - * thinking stripped, as a request's messages are built from raw messages, - * then priced as prepared (preparedTokensOf). Images count at the store's + * thinking stripped and its system changes left out, as a request's messages + * are built from raw messages (compressionInputContent), then priced as + * prepared (preparedTokensOf). Images count at the store's * estimate, though a request's inline image cap can replace some with * placeholders. */ protected sentTokensOf(content: ContentBlock[], ctx: StrategyContext): number { - return this.preparedTokensOf(stripThinkingBlocks(content), ctx); + return this.preparedTokensOf(compressionInputContent(content), ctx); } /** @@ -16497,6 +16601,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { let headContent = ''; let headReplacements = 0; for (const m of headMessages) { + // A message of system changes has no text: it adds no entry. + if (holdsSystemChange(m.content)) continue; // Counted as the request sends it: only the message's text blocks // reach it, with their serialized media replaced on a copy, so a // payload doesn't spend the cap. The request carries the markers, so @@ -17964,6 +18070,46 @@ export class AutobiographicalStrategy implements ResettableStrategy { return message.content.some(block => block.type === 'tool_result'); } + /** + * The head's positions, in the order they're sent: after a head-window + * reset, the system changes stored before the head window, then the window + * [headStart, headEnd). Every selector sends the head first and the + * history before it later, with the middle, so a change left in that + * history would follow the head's own changes and apply after them: a + * removal could run before the definition it removes. In the head, every + * change keeps its order, and the solved layout, the emission, the cache + * check's rebuild and the markers read one head. The history before the + * head, which the reset already sends after the head, then reads them too, + * which the provider takes: it doesn't check a past tool call against the + * tools offered where the call sits. With nothing in the window (the head + * window disabled, or a window that ends where it starts), nothing is sent + * ahead of that history, and the changes stay where they are. + */ + protected headPositions(messages: readonly StoredMessage[], headStart: number, headEnd: number): number[] { + const end = Math.min(headEnd, messages.length); + const out: number[] = []; + if (headStart > 0 && end > headStart) { + for (const i of systemChangePositions(messages)) { + if (i >= headStart) break; + out.push(i); + } + } + for (let i = headStart; i < end; i++) out.push(i); + return out; + } + + /** + * The middle's position ranges, in order: what comes before the tail + * outside the live head window, [0, headStart) after a head-window reset + * included. With the head window disabled, headEnd is 0 under a positive + * headStart, so the live window ends at the later of the two, and no + * position is in two ranges or in a range and the tail. A walk over them + * passes over the head's other positions (headPositions). + */ + protected static middleRanges(headStart: number, headEnd: number, tailStart: number): ReadonlyArray { + return [[0, Math.min(headStart, tailStart)], [Math.max(headStart, headEnd), tailStart]]; + } + /** * Shared structural pass for both selectors: repair tool pairing over the * entries as selected. What sending does to a raw message's tool blocks @@ -18428,19 +18574,21 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Collapse consecutive messages from the same participant into single messages. * Required because Claude API rejects consecutive same-role messages. + * A message left empty, as compression input leaves one of thinking or + * system changes alone (compressionInputContent), is passed over, so its + * neighbours join as they would without it and no separator marks it. */ protected collapseConsecutiveMessages( messages: Array<{ participant: string; content: ContentBlock[] }> ): Array<{ participant: string; content: ContentBlock[] }> { - if (messages.length === 0) return []; + const result: Array<{ participant: string; content: ContentBlock[] }> = []; - const result: Array<{ participant: string; content: ContentBlock[] }> = [ - { participant: messages[0].participant, content: [...messages[0].content] }, - ]; - - for (let i = 1; i < messages.length; i++) { + for (let i = 0; i < messages.length; i++) { + if (messages[i].content.length === 0) continue; const last = result[result.length - 1]; - if (messages[i].participant === last.participant) { + if (!last) { + result.push({ participant: messages[i].participant, content: [...messages[i].content] }); + } else if (messages[i].participant === last.participant) { // Merge: add separator then content last.content.push({ type: 'text', text: '\n\n---\n\n' } as ContentBlock); last.content.push(...messages[i].content); diff --git a/src/strategies/knowledge.ts b/src/strategies/knowledge.ts index 92bc2afe..ce983e98 100644 --- a/src/strategies/knowledge.ts +++ b/src/strategies/knowledge.ts @@ -9,6 +9,7 @@ import type { } from '../types/index.js'; import { AutobiographicalStrategy, type Chunk } from './autobiographical.js'; import type { ConfigLayer } from '../config-provenance.js'; +import { systemChangePositions } from '../system-changes.js'; const DEFAULT_RESEARCH_PREFIXES = ['mcpl:', 'zulip:']; const DEFAULT_SUBAGENT_PREFIXES = ['subagent:']; @@ -65,6 +66,12 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { protected rebuildChunks(store: MessageStoreView): void { const messagesToChunk = this.getCompressibleMessages(store); + const all = store.getAll(); + const position = this.positionIndex(all); + const changes = systemChangePositions(all); + // The walk below is in store order, so one cursor finds the changes + // stored between each message and the one before it. + let nextChange = 0; // Preserve existing compressed chunks const existingCompressed = new Map(); @@ -73,6 +80,20 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { existingCompressed.set(this.chunkKey(chunk), chunk); } } + // A chunk holds the messages of one summary, or of none, but for a call's + // result: it closes where the summary covering the walk's messages + // changes, except between a call and its result (below). The walk otherwise + // decides boundaries afresh, so once a message that closed a chunk is + // removed (a consolidation removes its system changes), a chunk could run + // into or across a summary's span; the overlap guard refuses to compress + // such a chunk for good, and its messages go out raw beside the summary + // that already covers some of them. Where the chunks already match the + // summaries, the walk closes there anyway. An L2's sources are summaries, + // not messages. + const coveredBy = new Map(); + for (const s of this.summaries) { + if (s.level === 1) for (const id of s.sourceIds) coveredBy.set(id, s.id); + } this.chunks = []; this.compressionQueue = []; @@ -82,9 +103,49 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { let currentPhase: PhaseType | null = null; let lastToolPhase: PhaseType = 'synthesis'; let chunkFilteredStart = 0; + const closeAt = (end: number): void => { + const chunk = this.createChunk( + this.chunks.length, + chunkFilteredStart, + end, + currentChunk, + currentTokens, + existingCompressed + ); + chunk.phaseType = currentPhase!; + this.chunks.push(chunk); + if (!chunk.compressed) this.compressionQueue.push(chunk.index); + + currentChunk = []; + currentTokens = 0; + chunkFilteredStart = end; + }; for (let i = 0; i < messagesToChunk.length; i++) { const msg = messagesToChunk[i]; + const previous = currentChunk[currentChunk.length - 1]; + let changeBefore = false; + const at = position.get(msg.id)!; + while (nextChange < changes.length && changes[nextChange]! < at) { + changeBefore = true; + nextChange++; + } + // Not between a call and its result: closing there would leave the + // result with its call folded into the summary, and pairing repair + // drops a result whose call isn't before it. So when a summary ends at + // a call answered only later, the result joins the summary's chunk, + // which the overlap guard then refuses, as on main. + const summaryChanges = previous !== undefined + && coveredBy.get(previous.id) !== coveredBy.get(msg.id) + && !(this.hasToolUse(previous) && this.hasToolResult(msg)); + if (previous && (changeBefore || summaryChanges)) { + // The compressible stream skips a system change, which stays raw. A + // chunk across it would summarize both sides as one, rendered at its + // first message, ahead of the change, so what came after the change + // would be read before it. Close even a thin run there. A store + // without changes whose chunks match its summaries chunks as before. + closeAt(i); + } const phase = this.classifyMessage(msg, lastToolPhase); // Track tool phase for tool_result inheritance @@ -118,23 +179,7 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { (phaseChanged || sizeExceeded) && !this.lastMessageContainsToolUse(currentChunk); - if (shouldClose) { - const chunk = this.createChunk( - this.chunks.length, - chunkFilteredStart, - i, - currentChunk, - currentTokens, - existingCompressed - ); - chunk.phaseType = currentPhase!; - this.chunks.push(chunk); - if (!chunk.compressed) this.compressionQueue.push(chunk.index); - - currentChunk = []; - currentTokens = 0; - chunkFilteredStart = i; - } + if (shouldClose) closeAt(i); currentChunk.push(msg); currentTokens += msgTokens; diff --git a/src/strategies/passthrough.ts b/src/strategies/passthrough.ts index ba8cfaf1..7517dccf 100644 --- a/src/strategies/passthrough.ts +++ b/src/strategies/passthrough.ts @@ -7,7 +7,10 @@ import type { TokenBudget, ContextEntry, StoredMessage, + SelectOptions, } from '../types/index.js'; +import { OverBudgetError } from '../adaptive/picker.js'; +import { holdsSystemChange, systemChangePositions } from '../system-changes.js'; /** * Simple passthrough strategy that copies all messages to context. @@ -16,7 +19,12 @@ import type { */ export class PassthroughStrategy implements ContextStrategy { readonly name = 'passthrough'; - /** Renders history raw; when it cannot all fit, the oldest is not rendered. */ + /** + * Renders history raw; when it cannot all fit, the oldest is not rendered, + * but for its system changes, and when those alone, or those and the + * newest conversation message, can't fit, a committing select refuses with + * OverBudgetError. + */ readonly renderedForms = ['raw', 'omitted'] as const; checkReadiness(): ReadinessState { @@ -30,7 +38,8 @@ export class PassthroughStrategy implements ContextStrategy { select( store: MessageStoreView, _log: ContextLogView, - budget: TokenBudget + budget: TokenBudget, + opts?: SelectOptions, ): ContextEntry[] { const messages = store.getAll(); const entries: ContextEntry[] = []; @@ -61,40 +70,87 @@ export class PassthroughStrategy implements ContextStrategy { // If we hit the limit before adding all messages, // keep the most recent ones that fit if (entries.length < messages.length) { - return this.selectFromEnd(store, maxTokens); + return this.selectFromEnd(store, maxTokens, opts); } return entries; } /** - * Select messages from the end when we can't fit everything. + * Select messages from the end when we can't fit everything. A system + * change before the cut is kept too, in its order ahead of the rest: it + * changes the tools and system text everything after it was read with + * (system-changes.ts). Its tokens are set aside first, and when they alone + * exceed the budget a committing select refuses: only the caller, by + * consolidating its changes, can clear that. A change is no conversation + * message, so when the newest conversation message doesn't fit beside + * them a committing select refuses too, as the windowed strategy does: a + * budget refusal, which a newer conversation message that fits clears + * while they leave room. Without changes, a select that fits no message + * sends nothing, as on main. Where a change can go in a request is + * membrane's to say (after a user turn): this strategy can't tell a user + * participant from another, and membrane refuses a request holding a + * change it can't place. */ - private selectFromEnd(store: MessageStoreView, maxTokens: number): ContextEntry[] { + private selectFromEnd(store: MessageStoreView, maxTokens: number, opts?: SelectOptions): ContextEntry[] { const messages = store.getAll(); - const entries: ContextEntry[] = []; - let totalTokens = 0; + const changes = systemChangePositions(messages); + let changeTokens = 0; + for (const i of changes) changeTokens += store.estimateTokens(messages[i]!); + let totalTokens = changeTokens; + if (totalTokens > maxTokens && changes.length > 0 && !opts?.dryRun) { + throw new OverBudgetError({ + budget: maxTokens, + actual: totalTokens, + stage: 'Passthrough kept only its system changes', + clearedBy: [], + diagnostics: { headTokens: totalTokens, tailTokens: 0, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, + }); + } // Work backwards from the most recent + let start = messages.length; for (let i = messages.length - 1; i >= 0; i--) { - const msg = messages[i]; + const msg = messages[i]!; + if (holdsSystemChange(msg.content)) continue; // counted above const tokens = store.estimateTokens(msg); if (totalTokens + tokens > maxTokens) { break; } - entries.unshift({ + totalTokens += tokens; + start = i; + } + // A store of changes alone gets here only when they're over the budget: + // select() sends every message when all fit. A committing select refused + // them above, and a dry one skips this refusal too. + if (start === messages.length && changes.length > 0 && !opts?.dryRun) { + let newest = messages.length - 1; + while (holdsSystemChange(messages[newest]!.content)) newest--; + const newestTokens = store.estimateTokens(messages[newest]!); + throw new OverBudgetError({ + budget: maxTokens, + actual: changeTokens + newestTokens, + stage: 'Passthrough: its system changes and the newest conversation message', + clearedBy: changeTokens < maxTokens ? ['newer-message'] : [], + diagnostics: { headTokens: changeTokens, tailTokens: newestTokens, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, + }); + } + + const entries: ContextEntry[] = []; + const add = (i: number): void => { + const msg = messages[i]!; + entries.push({ index: i, sourceMessageId: msg.id, sourceRelation: 'copy', participant: msg.participant, content: msg.content, }); - - totalTokens += tokens; - } - + }; + for (const i of changes) if (i < start) add(i); + for (let i = start; i < messages.length; i++) add(i); return entries; } } diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 8ec09277..75588040 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -20,6 +20,7 @@ import { OverBudgetError } from '../adaptive/picker.js'; import { observeStoreBranch, type StoreBranchGeneration } from '../branch-generation.js'; import { hasImage, imagesIn, inlineImageBytes, replaceImages, strippedImageSaving } from '../inline-images.js'; import { capContent, cappedPriceExtra } from './message-cap.js'; +import { holdsSystemChange, systemChangePositions } from '../system-changes.js'; /** * Options for {@link WindowedPassthroughStrategy}. @@ -28,12 +29,14 @@ export interface WindowedPassthroughOptions { /** * When the window no longer fits the budget, the anchor jumps forward so * the new window is roughly this fraction of the usable budget (default - * 0.5). Between jumps the anchor never moves, so compiled output between - * re-anchors is a byte-stable prefix plus pure appends — and the strategy + * 0.5), or, when the store holds system changes, which go out wherever the + * window starts, this fraction of the room they leave. Between jumps the + * anchor never moves, so compiled output between re-anchors is a + * byte-stable prefix plus pure appends — and the strategy * marks that prefix for the provider's prompt cache (see * {@link WindowedPassthroughStrategy} on cache markers), so the accepted - * cost is one cache miss per jump, amortized over ~(1 - fraction) × budget - * of appended tokens, instead of the per-turn prefix churn a naively + * cost is one cache miss per jump, amortized over ~(1 - fraction) × that + * room of appended tokens, instead of the per-turn prefix churn a naively * sliding front produces. */ reAnchorFraction?: number; @@ -120,14 +123,16 @@ const DEFAULT_MAX_LIVE_IMAGE_BYTES = 20 * 1024 * 1024; * **Hard budget.** After per-message shaping (truncation, image policy) the * window is priced against `maxTokens - reserveForResponse`. Overflow * re-anchors; if the single newest message alone still exceeds the usable - * budget the strategy refuses with {@link OverBudgetError} (the framework's - * over-budget breaker handles it) rather than emitting a request the - * provider will reject. Previews report the oversized window instead. + * budget, or with system changes, the changes and the newest conversation + * message, or the changes alone, the strategy refuses with + * {@link OverBudgetError} (the framework's over-budget breaker handles it) + * rather than emitting a request the provider will reject. Previews report + * the oversized window instead. */ export class WindowedPassthroughStrategy implements ContextStrategy { readonly name = 'windowed-passthrough'; readonly maxMessageTokens?: number; - /** Renders the window raw; history before the anchor is not rendered. */ + /** Renders the window raw; history before the anchor is not rendered, but for its system changes. */ readonly renderedForms = ['raw', 'omitted'] as const; private readonly reAnchorFraction: number; @@ -299,17 +304,33 @@ export class WindowedPassthroughStrategy implements ContextStrategy { return estimate + cappedPriceExtra(store, msg, content); }; + // A system change before the window is sent too, in its order ahead of + // it: it changes the tools and system text everything after it was read + // with (system-changes.ts). So every change is priced wherever the + // window starts. + const changes = systemChangePositions(messages); + const keptBefore = (from: number): number[] => changes.filter((i) => i < from); + let changeTokens = 0; + for (const i of changes) changeTokens += priced(i); let total = 0; + for (const i of keptBefore(start)) total += priced(i); for (let i = start; i < messages.length; i++) total += priced(i); + let movedAnchor: number | undefined; if (total > maxTokens) { - // Coarse re-anchor: jump forward so the window is ~fraction × budget, - // then emit from the new anchor. One prefix invalidation, then - // byte-stable prefix + appends again until the next overflow. - const target = maxTokens * this.reAnchorFraction; - let acc = 0; + // Coarse re-anchor: jump forward so the window holds the changes and + // about that fraction of the room they leave, then emit from the new + // anchor. One prefix invalidation, then + // byte-stable prefix + appends again until the next overflow. The + // system changes go out wherever the window starts, so its messages + // take that fraction of the room the changes leave, and appends get + // the rest of that room, (1 - fraction) × (budget - changes), before + // the next. + const target = changeTokens + (maxTokens - changeTokens) * this.reAnchorFraction; + let acc = changeTokens; let newStart = messages.length; for (let i = messages.length - 1; i >= start; i--) { + if (holdsSystemChange(messages[i]!.content)) continue; // priced above const tokens = priced(i); if (acc + tokens > target) break; acc += tokens; @@ -317,20 +338,18 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } // Always make progress: if even the newest message exceeds the // target, emit just that message rather than nothing — subject to the - // hard-budget check below. + // hard-budget check below. A system change is no conversation message, + // so that's the newest message that isn't one, if the window holds one. if (newStart === messages.length && messages.length > 0) { newStart = messages.length - 1; + while (newStart > start && holdsSystemChange(messages[newStart]!.content)) newStart--; } start = newStart; total = 0; + for (const i of keptBefore(start)) total += priced(i); for (let i = start; i < messages.length; i++) total += priced(i); - if (!opts?.dryRun && start < messages.length) { - const newAnchor = messages[start].sequence; - if (newAnchor !== this.anchor) { - this.anchor = newAnchor; - this.persistAnchor(); - this.pendingCause = 'window'; - } + if (start < messages.length && messages[start].sequence !== this.anchor) { + movedAnchor = messages[start].sequence; } } @@ -339,27 +358,41 @@ export class WindowedPassthroughStrategy implements ContextStrategy { // would reject the request; refuse loudly instead (the framework // recognizes OverBudgetError and runs its over-budget breaker), as the // autobiographical strategy does. Previews report rather than throw. + // The anchor moves only for a compile that sends, below: a refused one + // leaves it where the last sent window started, so the turns that fit + // once the caller consolidates its changes are still in the window. if (total > maxTokens && !opts?.dryRun) { throw new OverBudgetError({ budget: maxTokens, actual: total, - stage: 'Windowed passthrough: the newest message alone', + stage: changeTokens === 0 + ? 'Windowed passthrough: the newest message alone' + : total > changeTokens + ? 'Windowed passthrough: its system changes and the newest conversation message' + : 'Windowed passthrough: its system changes alone', // No compression here: the window moves past this message once a - // newer one that fits arrives (the progress clause above). - clearedBy: ['newer-message'], + // newer one that fits arrives (the progress clause above). System + // changes go out wherever the window starts, so once they leave no + // room for a message only the caller, consolidating them, clears it. + clearedBy: changeTokens < maxTokens ? ['newer-message'] : [], diagnostics: { - headTokens: 0, - tailTokens: total, + headTokens: changeTokens, + tailTokens: total - changeTokens, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0, }, }); } + if (movedAnchor !== undefined && !opts?.dryRun) { + this.anchor = movedAnchor; + this.persistAnchor(); + this.pendingCause = 'window'; + } const entries: ContextEntry[] = []; - for (let i = start; i < messages.length; i++) { - const msg = messages[i]; + const add = (i: number): void => { + const msg = messages[i]!; entries.push({ index: i, sourceMessageId: msg.id, @@ -367,7 +400,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { participant: msg.participant, content: shaped.get(i) ?? msg.content, }); - } + }; + for (const i of keptBefore(start)) add(i); + for (let i = start; i < messages.length; i++) add(i); this.stripImages(entries, policy); if (!opts?.dryRun) this.placeCacheMarkers(entries); @@ -387,13 +422,16 @@ export class WindowedPassthroughStrategy implements ContextStrategy { * message to its newest: each cut as select cuts it (cappedContent), then * the live-image policy's verdicts applied (liveImagePolicy, stripImages), * each decided from its message and newer ones, with depth from the view's - * newest message, so they're select's. A message before the anchor, which select - * leaves out, comes back as null. Like select, it first reloads the - * branch the store has moved to. A re-anchor the next compile's budget - * forces isn't foreseen: it moves the anchor only past a window bigger - * than the budget allows. A body sharded under the autobiographical - * strategy comes back shard by shard, since select sends each as its own - * message. + * newest message, so they're select's. A message before the anchor comes + * back as null: select leaves it out, but for a system change, which it + * sends ahead of the window. A change there isn't a just-stored message, + * which is what this is for, since the anchor only moves to a message a + * compile sent, and a message stored later comes after it. Like select, it + * first reloads the branch the store has moved to. A re-anchor the next + * compile's budget forces isn't foreseen: it moves the anchor only past a + * window bigger than the budget allows. A body sharded under the + * autobiographical strategy comes back shard by shard, since select sends + * each as its own message. */ renderAsNewest(store: MessageStoreView, ids: readonly MessageId[]): Array { this.syncToBranch(); diff --git a/src/system-changes.ts b/src/system-changes.ts new file mode 100644 index 00000000..e2790103 --- /dev/null +++ b/src/system-changes.ts @@ -0,0 +1,55 @@ +/** + * System changes: membrane's `system_change` block (SystemChangeContent), a + * change to the tools and system text the model is offered from its place in + * the conversation on. One stands alone in its message + * (well-formed-content.ts), and membrane sends such a message as a system + * message of its own, at the first point at or after its place that follows + * a user turn, so always before the reply the request asks for. + * + * Every strategy compiles such a message unaltered, in its place among the + * raw messages. A fold or a window that left one out would take its change + * out of everything after it, so the autobiographical strategy keeps it raw + * as a pin keeps a message, and the passthrough strategies keep it when + * their window starts after it. Under the autobiographical strategy's + * positionedRecallPairs: false, the middle's summaries go ahead of its raw + * messages, so there a change keeps its place among raw messages only. The + * one move is after a head-window reset, when the changes stored before the + * head become part of the head, ahead of the head window, so that they keep + * their order (AutobiographicalStrategy.headPositions). + * + * None is dropped, so they're carried until the caller consolidates them. + * When they alone exceed the budget (beyond the autobiographical strategy's + * grace, and on its hierarchical path only where enforceBudget is on), every + * strategy refuses with OverBudgetError, its clearedBy empty, since + * no newer message or compression shrinks them; a dry compile reports where + * the strategy's other refusals do (the passthrough strategies refuse none + * on a dry compile). While they fit, the passthrough strategies refuse when + * the newest conversation message doesn't fit beside them (OverBudgetError, + * naming a newer message while they leave room for one) + * and otherwise drop the oldest messages as they do without changes, and + * the autobiographical strategy refuses a loss as it refuses any: the + * hierarchical path's recent window refuses an eviction with + * UncoveredDropError. What compression reads + * leaves them out: a change alters the request, not the conversation, and a + * summarizer's request would apply it, or be refused for it. + */ + +import type { ContentBlock } from '@animalabs/membrane'; + +/** Whether a block is a system change. */ +export function isSystemChange(block: ContentBlock | { type?: unknown }): boolean { + return (block as { type?: unknown }).type === 'system_change'; +} + +/** Whether a message's content holds a system change, and so, as written, nothing else. */ +export function holdsSystemChange(content: readonly ContentBlock[]): boolean { + for (let i = 0; i < content.length; i++) if (isSystemChange(content[i]!)) return true; + return false; +} + +/** The positions in a listing of the messages that hold a system change, in order. */ +export function systemChangePositions(messages: readonly { content: readonly ContentBlock[] }[]): readonly number[] { + const positions: number[] = []; + for (let i = 0; i < messages.length; i++) if (holdsSystemChange(messages[i]!.content)) positions.push(i); + return positions; +} diff --git a/src/types/context.ts b/src/types/context.ts index 7290edc7..cf8992da 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -171,6 +171,32 @@ export interface CompileResult { * read-only. It's transient and isn't part of the persisted provenance. */ rawBodies?: ReadonlyMap; + + /** + * Where this compile's messages first differ from the last committing + * compile's, for every strategy: the index of the first message, injections + * included, whose participant or content bytes differ from that compile's + * message at the same index, or that has none there. So when this compile + * only appended to that one's messages it is the first appended index, and + * it equals `messages.length` when these messages are that compile's or a + * prefix of them. An injection placed beside the newest user message moves + * with each new user message, so after one it's that injection's old index + * rather than the first appended one. Null only for the manager's first + * committing compile, since the comparison is with what this manager last + * compiled, held in memory: a manager opened again, as after a restart, + * starts with none. The comparison is by bytes, so it holds across a + * branch switch or a new strategy, whichever way the switch was made. The + * store's content is taken as the same bytes where it's the same array, so + * a block a caller changed in place after storing it, which the store's + * contract rules out, isn't seen. + * Cache markers don't count, and neither do `systemInjections`, which the + * caller adds to the system prompt itself. A dry compile leaves it out and + * doesn't move the comparison; a committing compile that returns moves it + * whether or not its request is sent, and one that throws doesn't. A + * caller can tell from it whether a compile rewrote what the last + * committing compile gave, such as a system change it wrote. + */ + firstChanged?: number | null; } /** A branch as a compile or receipt saw it: native id, name, creation time. */ diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 70e51c72..9e40dc77 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -351,7 +351,10 @@ export interface SelectOptions { * throw as for a real compile, leaving nothing behind; `previewContext` * reports an over-budget throw, as the plan's own preview when the plan was * already over the budget (the live compile refuses it at the plan check), - * and passes any other throw through. + * and passes any other throw through. The passthrough strategies throw + * none of their budget refusals on a dry select: it returns what they would + * send, over the budget or without the newest message, with no verdict, + * and they have no `previewContext` report. * * A compression hold ends at its deadline, whoever looks: the manager * releases an expired hold at any compile or hold query, a dry compile diff --git a/src/well-formed-content.ts b/src/well-formed-content.ts index a27848ac..25cd7e67 100644 --- a/src/well-formed-content.ts +++ b/src/well-formed-content.ts @@ -14,11 +14,21 @@ * base64, or no media type, neither a declared one nor, for an image, one * its data's signature names. The store keeps a block of the content as a * blob, not media inside a `tool_result`, which it keeps as given. - * Nothing else is checked: not ids, names or signatures, which a provider may + * Nothing else is checked: not ids, a tool call's name or signatures, which a provider may * still refuse a block without, and nothing else of a media source, such as - * a URL image's URL, which the formatters and the providers read. A block of - * a type none of these names passes as it is, since membrane adds types: - * whether a provider can take one is the provider's to say. One type is the + * a URL image's URL, which the formatters and the providers read. A + * `system_change` (membrane's SystemChangeContent) carries its content in + * `removeTools` and `addSystem`, arrays of strings, and `defineTools`, an + * array of tool definitions, each where present. A definition has the + * fields membrane's ToolDefinition types and sends to the provider with every + * later request: a non-empty string `name`, a string `description` and an + * object `inputSchema` whose `type` is `'object'`, which the API requires. + * One stands alone in its message, at + * its top level: membrane sends such a message as a system message of its + * own, and refuses one that mixes it with other blocks. A write refuses one + * that doesn't, and a read leaves it out. A + * block of a type none of these names passes as it is, since membrane adds + * types: whether a provider can take one is the provider's to say. One type is the * store's own: a `blob_ref`, which the store writes in place of inline media. * A write can't give one, and a read takes one whose `ref` is an object with * a string `hash` and an `originalType` of image, document, audio or video, @@ -92,6 +102,105 @@ function kindOf(value: unknown): string { /** The block types whose `source` carries their media. */ const MEDIA_TYPES: ReadonlySet = new Set(['image', 'document', 'audio', 'video']); +/** Whether a value is a system change (membrane's `system_change` block, SystemChangeContent). */ +function isSystemChange(value: unknown): boolean { + return isRecord(value) && value.type === 'system_change'; +} + +function isString(value: unknown): value is string { + return typeof value === 'string'; +} + +/** + * Whether a value is a tool definition as membrane reads one (ToolDefinition): + * a non-empty string `name`, a string `description` and an object + * `inputSchema` whose `type` is `'object'`. Membrane sends each to the + * provider with every request from the change on, so one without them would + * be refused on every request: the API refuses a definition whose schema has + * no `type`, or another, with a 400. + */ +function isToolDefinition(value: unknown): boolean { + return isRecord(value) && typeof value.name === 'string' && value.name !== '' && + typeof value.description === 'string' && isRecord(value.inputSchema) && value.inputSchema.type === 'object'; +} + +function stringPlaces(value: unknown, at: string, places: string[], what: string): void { + if (typeof value !== 'string') places.push(`${at} is ${kindOf(value)}, not ${what}`); +} + +function toolDefinitionPlaces(value: unknown, at: string, places: string[]): void { + if (!isRecord(value)) { + places.push(`${at} is ${kindOf(value)}, not a tool definition (an object)`); + return; + } + if (value.name === '') places.push(`${at}.name is an empty string, not a tool name`); + else stringPlaces(value.name, `${at}.name`, places, 'a tool name (a string)'); + stringPlaces(value.description, `${at}.description`, places, 'a string'); + if (!isRecord(value.inputSchema)) places.push(`${at}.inputSchema is ${kindOf(value.inputSchema)}, not a JSON Schema (an object)`); + else if (value.inputSchema.type !== 'object') places.push(`${at}.inputSchema.type is ${kindOf(value.inputSchema.type)}, not 'object'`); +} + +/** + * Whether every element of the content is a system change. An index loop, so + * a hole counts as an element that isn't one, as every walk here names a hole + * and refuses it like any other. + */ +function standsAlone(content: readonly unknown[]): boolean { + for (let i = 0; i < content.length; i++) if (!isSystemChange(content[i])) return false; + return true; +} + +/** Whether any element of the content is a system change (a hole isn't one). */ +function holdsChange(content: readonly unknown[]): boolean { + for (let i = 0; i < content.length; i++) if (isSystemChange(content[i])) return true; + return false; +} + +/** Whether a value is absent, or an array whose every element passes `each`. */ +function absentOrArrayOf(value: unknown, each: (element: unknown) => boolean): boolean { + if (value === undefined) return true; + if (!Array.isArray(value)) return false; + for (let i = 0; i < value.length; i++) if (!each(value[i])) return false; + return true; +} + +/** + * The fields of a system change that carry its content, as membrane reads + * them when it sends the change: names to withdraw and system texts as arrays + * of strings, and definitions as an array of tool definitions + * (isToolDefinition), each where present. Each with the places an element + * that fails names. + */ +const SYSTEM_CHANGE_FIELDS: ReadonlyArray boolean, + what: string, + elementPlaces: (element: unknown, at: string, places: string[]) => void, +]> = [ + ['removeTools', isString, 'an array of tool names', (v, at, places) => stringPlaces(v, at, places, 'a tool name (a string)')], + ['defineTools', isToolDefinition, 'an array of tool definitions', toolDefinitionPlaces], + ['addSystem', isString, 'an array of strings', (v, at, places) => stringPlaces(v, at, places, 'a string')], +]; + +function systemChangeWellFormed(block: Record): boolean { + for (const [field, each] of SYSTEM_CHANGE_FIELDS) if (!absentOrArrayOf(block[field], each)) return false; + return true; +} + +function systemChangePlaces(block: Record, at: string, places: string[]): void { + for (const [field, each, what, elementPlaces] of SYSTEM_CHANGE_FIELDS) { + const value = block[field]; + if (value === undefined) continue; + if (!Array.isArray(value)) { + places.push(`${at}.${field} is ${kindOf(value)}, not ${what}`); + continue; + } + for (let i = 0; i < value.length; i++) { + if (!each(value[i])) elementPlaces(value[i], `${at}.${field}[${i}]`, places); + } + } +} + /** Whether a block matches where its readers read it, without allocating: each side's common case. */ function wellFormed(block: unknown, side: Side): boolean { if (!isRecord(block)) return false; @@ -108,9 +217,13 @@ function wellFormed(block: unknown, side: Side): boolean { const inner = block.content; if (typeof inner === 'string') return true; if (!Array.isArray(inner)) return false; - for (let i = 0; i < inner.length; i++) if (!wellFormed(inner[i], side)) return false; + for (let i = 0; i < inner.length; i++) { + if (!wellFormed(inner[i], side) || isSystemChange(inner[i])) return false; + } return true; } + case 'system_change': + return systemChangeWellFormed(block); case 'generated_image': return typeof block.data === 'string' && typeof block.mimeType === 'string'; case 'blob_ref': @@ -152,10 +265,13 @@ function storeRefuses(block: unknown): boolean { } function allWellFormed(content: readonly unknown[], side: Side): boolean { + let changes = 0; for (let i = 0; i < content.length; i++) { if (!wellFormed(content[i], side) || (side === 'write' && storeRefuses(content[i]))) return false; + if (isSystemChange(content[i])) changes++; } - return true; + // A message holding a system change holds nothing else. + return changes === 0 || changes === content.length; } /** @@ -231,9 +347,18 @@ function placesIn(block: unknown, at: string, places: string[], side: Side): voi places.push(`${at}.content is ${kindOf(inner)}, not a string or an array of content blocks`); return; } - for (let i = 0; i < inner.length; i++) placesIn(inner[i], `${at}.content[${i}]`, places, side); + for (let i = 0; i < inner.length; i++) { + if (isSystemChange(inner[i])) { + places.push(`${at}.content[${i}] is a system_change inside a tool_result; a system change goes in a message of its own`); + } else { + placesIn(inner[i], `${at}.content[${i}]`, places, side); + } + } return; } + case 'system_change': + systemChangePlaces(block, at, places); + return; case 'generated_image': if (typeof block.data !== 'string') places.push(`${at}.data is ${kindOf(block.data)}, not a string`); if (typeof block.mimeType !== 'string') places.push(`${at}.mimeType is ${kindOf(block.mimeType)}, not a string`); @@ -274,9 +399,13 @@ function storePlacesIn(block: unknown, at: string, places: string[]): void { function malformedPlaces(content: readonly unknown[], side: Side, at = 'content'): string[] { const places: string[] = []; + const mixed = !standsAlone(content) && holdsChange(content); for (let i = 0; i < content.length; i++) { placesIn(content[i], `${at}[${i}]`, places, side); if (side === 'write') storePlacesIn(content[i], `${at}[${i}]`, places); + if (mixed && isSystemChange(content[i])) { + places.push(`${at}[${i}] is a system_change beside other blocks; a system change goes in a message of its own`); + } } return places; } @@ -349,7 +478,10 @@ export function blockForWrite(block: ContentBlock, what: string): ContentBlock { * so each level is reconciled on its own, and a block mended inside a * `tool_result` gives up its form like one at the top. */ -function transformed(content: readonly unknown[], side: Side): ContentBlock[] { +function transformed(content: readonly unknown[], side: Side, nested = false): ContentBlock[] { + // A system change that doesn't stand alone in its message, at its top + // level, is left out by a read, and a write has refused it already. + const changesStand = !nested && standsAlone(content); const before: ContentBlock[] = []; const after: ContentBlock[] = []; for (let i = 0; i < content.length; i++) { @@ -358,7 +490,7 @@ function transformed(content: readonly unknown[], side: Side): ContentBlock[] { // read leaves it out, and a write has refused it already. if (!isRecord(block)) continue; before.push(block as unknown as ContentBlock); - const out = side === 'write' ? writeBlock(block) : readBlock(block); + const out = isSystemChange(block) && !changesStand ? undefined : side === 'write' ? writeBlock(block) : readBlock(block); if (out) after.push(out); } return releaseEditedRawForms(before, after); @@ -369,7 +501,7 @@ function writeBlock(block: Record): ContentBlock { if (wellFormed(block, 'write')) return block as unknown as ContentBlock; if (block.type === 'tool_use') return mendedToolUse(block)!; if (block.type === 'tool_result' && Array.isArray(block.content)) { - return { ...block, content: transformed(block.content, 'write') } as unknown as ContentBlock; + return { ...block, content: transformed(block.content, 'write', true) } as unknown as ContentBlock; } return block as unknown as ContentBlock; } @@ -386,10 +518,14 @@ function writeBlock(block: Record): ContentBlock { * out (undefined): a value that isn't an object, an array, a block without a * string `type`, a text block without its string, unsigned thinking without * its string, media without a source or the data and media type to make - * one, a generated image without its data or media type, and a `blob_ref` + * one, a generated image without its data or media type, a `blob_ref` * that doesn't reference a blob, whose `originalType` isn't image, document, * audio or video, or, for a document, audio or video, without a string - * `mediaType`. + * `mediaType`, and a `system_change` whose `removeTools`, `defineTools` or + * `addSystem` isn't the array its content needs (one element of + * `defineTools` that isn't a tool definition leaves the whole change out), + * or that doesn't stand alone in its message, at its top level + * (transformed). */ function readBlock(block: Record): ContentBlock | undefined { if (wellFormed(block, 'read')) return block as unknown as ContentBlock; @@ -406,9 +542,9 @@ function readBlock(block: Record): ContentBlock | undefined { case 'tool_result': { const inner = block.content; const content = Array.isArray(inner) - ? transformed(inner, 'read') + ? transformed(inner, 'read', true) : isRecord(inner) - ? transformed([inner], 'read') + ? transformed([inner], 'read', true) : typeof inner === 'number' ? String(inner) : []; diff --git a/test/kv-unified-render-state.test.ts b/test/kv-unified-render-state.test.ts index 71aeae96..05cc1182 100644 --- a/test/kv-unified-render-state.test.ts +++ b/test/kv-unified-render-state.test.ts @@ -1218,3 +1218,33 @@ describe('a reference loaded from the store', () => { }); } }); + +describe('system changes before a reset head', () => { + it("they lead the head, and the check's rebuild puts them there, as the emission sends them", async () => { + const { dir, strategy, manager } = await open({ headWindowTokens: 4_000, recentWindowTokens: 4_000 }); + try { + const change = { type: 'system_change', addSystem: ['Answer in metric units.'] } as unknown as ContentBlock; + add(manager, [['u', [text('a')]], ['Harness', [change]], ['a', [text('b')]], ...MIDDLE, ...CLOSE]); + strategy.resetHeadWindow(ids(manager)[3]!); + const one = await compile(manager); + assert.deepEqual(one[0]!.content, [change], 'the change before the head leads'); + assert.deepEqual(one[1]!.content, manager.getMessage(ids(manager)[3]!)!.content, 'then the head window'); + accept(strategy, one); + const ref = reference(strategy)!; + assert.ok(ref.markers.length > 0, 'premise: the accepted request has markers'); + const hooks = strategy as unknown as { kvUnifiedKeepEntries(...args: unknown[]): unknown }; + const keep = hooks.kvUnifiedKeepEntries.bind(strategy); + let rebuilt = 0; + hooks.kvUnifiedKeepEntries = (...args) => { rebuilt++; return keep(...args); }; + add(manager, SMALL); + const two = await compile(manager); + assert.ok(rebuilt > 0, 'premise: the check rebuilt'); + assert.equal(firstChange(one, two), one.length, 'premise: nothing the accepted request sent changes'); + assert.equal(solveMarkers(strategy), ref.markers.length, 'every accepted marker survives the check'); + // afterEach: the self-check found the rebuild matching what was sent. + } finally { + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/test/l1-recall-raw-pins-frontier.test.ts b/test/l1-recall-raw-pins-frontier.test.ts index d1f7de68..bb17ad47 100644 --- a/test/l1-recall-raw-pins-frontier.test.ts +++ b/test/l1-recall-raw-pins-frontier.test.ts @@ -835,7 +835,7 @@ describe('the raw pins add is spent from the recall budget', () => { }); describe('a pin resolves once, for compression and every selector', () => { - const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}` })); + const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}`, content: [] })); const pins: Array & { at: number }> = [ { at: 1 }, // classic raw pin { at: 2, level: 0 }, diff --git a/test/system-changes.test.ts b/test/system-changes.test.ts new file mode 100644 index 00000000..2157b2ab --- /dev/null +++ b/test/system-changes.test.ts @@ -0,0 +1,1014 @@ +/** + * System changes (membrane's `system_change` block, SystemChangeContent): a + * message of them changes the tools and system text the model is offered from + * its place on (src/system-changes.ts). context-manager prices one from its + * JSON, keeps one alone in its message (a write refuses one that isn't, and + * a read leaves it out), compiles one in place and unaltered under every + * strategy (the autobiographical strategy keeps it raw as a pin keeps a + * message, and puts the ones before a reset head in the head, ahead of the + * head window; the passthrough strategies keep it when their window starts + * after it), refuses over budget when the changes alone don't fit (beyond + * grace, where enforcement is on), or, under the passthrough strategies, + * when the newest conversation message doesn't fit beside them, + * and leaves it out of what compression reads. A committing compile reports + * `firstChanged`, where its messages first differ from the last committing + * compile's. + */ +import { describe, it, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock } from '@animalabs/membrane'; +import { + AutobiographicalStrategy, + ContextManager, + KnowledgeStrategy, + MalformedContentError, + OverBudgetError, + PassthroughStrategy, + UncoveredDropError, + WindowedPassthroughStrategy, + malformedContentPlaces, +} from '../src/index.js'; +import { jsonTokenEstimator } from '../src/message-store.js'; +import { wellFormedContent } from '../src/well-formed-content.js'; +import type { CompileResult, ContextStrategy, StoredMessage, TokenBudget } from '../src/types/index.js'; + +const dir = mkdtempSync(join(tmpdir(), 'cm-system-changes-')); +after(() => rmSync(dir, { recursive: true, force: true })); +let opened = 0; + +const tool = { name: 'get_weather', description: 'Weather.', inputSchema: { type: 'object', properties: { city: { type: 'string' } } } }; +const change = (extra: Record = {}): ContentBlock => + ({ type: 'system_change', defineTools: [tool], addSystem: ['Use metric units.'], ...extra }) as unknown as ContentBlock; +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); +const BIG = { maxTokens: 1_000_000, reserveForResponse: 0 }; + +async function open(strategy: ContextStrategy, membrane?: unknown): Promise { + return ContextManager.open({ path: join(dir, `store-${opened++}`), strategy, ...(membrane ? { membrane: membrane as never } : {}) }); +} + +const holdsChange = (content: unknown[]): boolean => content.some((b) => (b as { type?: string }).type === 'system_change'); +/** The content of each compiled message that holds a change, in order. */ +const changesSent = (compiled: CompileResult): unknown[] => compiled.messages.filter((m) => holdsChange(m.content)).map((m) => m.content); +/** The tools a compile's changes leave offered, each applied in compiled order as membrane applies one: its removals, then its definitions. */ +const toolsAfter = (compiled: CompileResult): string[] => { + const offered = new Set(); + for (const m of compiled.messages) { + for (const b of m.content as Array<{ type?: string; removeTools?: string[]; defineTools?: Array<{ name: string }> }>) { + if (b.type !== 'system_change') continue; + for (const name of b.removeTools ?? []) offered.delete(name); + for (const t of b.defineTools ?? []) offered.add(t.name); + } + } + return [...offered]; +}; +const estimateOf = (cm: ContextManager, id: string): number => + (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore.estimateTokens(cm.getMessage(id)!); +/** The stored message ids a compile sends raw, in its order (a summary's recall pair adds none). */ +const rawIds = (compiled: CompileResult): string[] => + compiled.provenance!.messages.flatMap((s) => (s.kind === 'raw' ? s.bodies.map((b) => b.messageId) : [])); +/** What a compile sends of the store, priced by the store: every raw message's estimate. */ +const sentTokens = (cm: ContextManager, compiled: CompileResult): number => rawIds(compiled).reduce((n, id) => n + estimateOf(cm, id), 0); +const KV_UNIFIED = { + policy: { + alpha: 0.7, budgetLowRatio: 0.5, budgetHighRatio: 0.9, budgetUnderLambda: 10, budgetOverLambda: 10, + cacheLambda: 1, cacheScale: 1000, cacheReadPrice: 0.1, cacheWritePrice: 1.25, + continuityLambda: 1, continuityScale: 1000, continuityRecencyHalfLifeTokens: 1000, continuityRecencyFloor: 0.2, + continuityStableHalfLife: 10, continuityStableFloor: 0.25, + }, + tokenBucketSize: 100, continuityBucketSize: 100, fidelityBucketSize: 100, labelCeiling: 10_000, adoptEpsilon: 0, + treeifyNonContiguousSummaries: false, preserveGapBearingSummaries: false, +}; +/** A summarizer stand-in that records every request it gets. */ +function recordingMembrane(): { requests: unknown[]; membrane: unknown } { + const requests: unknown[] = []; + return { requests, membrane: { complete: async (req: unknown) => { + requests.push(JSON.parse(JSON.stringify(req))); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `Summary #${requests.length}.` }], usage: { inputTokens: 500, outputTokens: 100 } }; + } } }; +} +async function settle(cm: ContextManager, requests: unknown[]): Promise { + let before = -1; + for (let r = 0; r < 200 && requests.length !== before; r++) { before = requests.length; await cm.compile(); await cm.tick(); await cm.tick(); } +} + +describe('a system change, written and priced', () => { + it('is priced from its JSON, where it used to cost nothing', async () => { + const cm = await open(new PassthroughStrategy()); + const block = change(); + const id = cm.addMessage('Harness', [block]); + const store = (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore; + const estimate = store.estimateTokens(cm.getMessage(id)!); + assert.equal(estimate, jsonTokenEstimator(JSON.stringify(block))); + assert.ok(estimate > 0); + // The context log prices it by the same rule, with its own estimator. + const log = (cm as unknown as { contextLog: { estimateTokens(entry: { content: ContentBlock[] }): number; tokenEstimator(t: string): number } }).contextLog; + assert.equal(log.estimateTokens({ content: [block] }), log.tokenEstimator(JSON.stringify(block))); + }); + + it('is refused beside other blocks, or inside a tool result, by name', async () => { + assert.deepEqual(malformedContentPlaces([text('hi'), change()]), [ + 'content[1] is a system_change beside other blocks; a system change goes in a message of its own', + ]); + assert.deepEqual(malformedContentPlaces([{ type: 'tool_result', toolUseId: 't1', content: [change()] }]), [ + 'content[0].content[0] is a system_change inside a tool_result; a system change goes in a message of its own', + ]); + const cm = await open(new PassthroughStrategy()); + assert.throws(() => cm.addMessage('Harness', [text('hi'), change()]), MalformedContentError); + // Beside a hole, which counts as another entry. + const sparse: unknown[] = [change()]; + sparse.length = 2; + assert.ok(malformedContentPlaces(sparse).includes('content[0] is a system_change beside other blocks; a system change goes in a message of its own'), JSON.stringify(malformedContentPlaces(sparse))); + assert.throws(() => cm.addMessage('Harness', sparse as ContentBlock[]), MalformedContentError); + assert.equal(cm.getAllMessages().length, 0); + }); + + it('is refused when a field that carries its content has the wrong type, each field and element by name, and taken with several changes alone', () => { + for (const [fields, place] of [ + [{ removeTools: 'get_time' }, 'content[0].removeTools is a string, not an array of tool names'], + [{ removeTools: ['get_time', 1] }, 'content[0].removeTools[1] is a number, not a tool name (a string)'], + [{ defineTools: tool }, 'content[0].defineTools is an object, not an array of tool definitions'], + [{ defineTools: [tool, [tool]] }, 'content[0].defineTools[1] is an array, not a tool definition (an object)'], + [{ defineTools: [{ ...tool, name: '' }] }, 'content[0].defineTools[0].name is an empty string, not a tool name'], + [{ defineTools: [{ ...tool, name: 7 }] }, 'content[0].defineTools[0].name is a number, not a tool name (a string)'], + [{ defineTools: [{ name: 'x', inputSchema: tool.inputSchema }] }, 'content[0].defineTools[0].description is undefined, not a string'], + [{ defineTools: [{ ...tool, inputSchema: 'object' }] }, 'content[0].defineTools[0].inputSchema is a string, not a JSON Schema (an object)'], + [{ defineTools: [{ ...tool, inputSchema: {} }] }, "content[0].defineTools[0].inputSchema.type is undefined, not 'object'"], + [{ defineTools: [{ ...tool, inputSchema: { type: 'string' } }] }, "content[0].defineTools[0].inputSchema.type is a string, not 'object'"], + [{ addSystem: 'Use metric units.' }, 'content[0].addSystem is a string, not an array of strings'], + [{ addSystem: [null] }, 'content[0].addSystem[0] is null, not a string'], + ] as const) { + assert.deepEqual(malformedContentPlaces([change(fields)]), [place], JSON.stringify(fields)); + } + assert.deepEqual(malformedContentPlaces([change(), change({ removeTools: ['get_time'], defineTools: undefined, addSystem: undefined })]), []); + }); + + it('is left out by a read where a write would refuse it, the rest kept, with one line naming where', () => { + const lines: string[] = []; + const log = { error: (line: string) => { lines.push(line); }, warn: () => {}, info: () => {}, debug: () => {} } as never; + const read = (content: unknown[]): unknown[] => wellFormedContent(content as never, 'message m-1', new Set(), log); + // Alone and well formed: as stored, and nothing logged. + const alone = [change(), change({ removeTools: ['get_time'] })]; + assert.equal(read(alone), alone); + assert.equal(lines.length, 0); + // Beside other blocks: the change goes, the conversation stays. + assert.deepEqual(read([text('hi'), change()]), [text('hi')]); + assert.match(lines.pop()!, /message m-1: .*content\[1\] is a system_change beside other blocks/); + // Inside a tool result: the result keeps what else it holds. + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't1', content: [text('42'), change()] }]), [{ type: 'tool_result', toolUseId: 't1', content: [text('42')] }]); + assert.match(lines.pop()!, /content\[0\]\.content\[1\] is a system_change inside a tool_result/); + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't2', content: [change()] }]), [{ type: 'tool_result', toolUseId: 't2', content: [] }]); + assert.match(lines.pop()!, /content\[0\]\.content\[0\] is a system_change inside a tool_result/); + // A field of the wrong type: the change goes. + assert.deepEqual(read([change({ addSystem: [2] })]), []); + assert.match(lines.pop()!, /content\[0\]\.addSystem\[0\] is a number, not a string/); + // A definition without the fields the provider needs: the change goes. + assert.deepEqual(read([change({ defineTools: [{ name: 'x' }] })]), []); + assert.match(lines.pop()!, /content\[0\]\.defineTools\[0\]\.description is undefined, not a string/); + // A schema that isn't of type 'object', which the API refuses: the change goes. + assert.deepEqual(read([change({ defineTools: [{ ...tool, inputSchema: {} }] })]), []); + assert.match(lines.pop()!, /content\[0\]\.defineTools\[0\]\.inputSchema\.type is undefined, not 'object'/); + // Beside a hole, which counts as another entry: the change goes. + const sparse: unknown[] = [change()]; + sparse.length = 2; + assert.deepEqual(read(sparse), []); + assert.match(lines.pop()!, /content\[0\] is a system_change beside other blocks/); + // A damaged tool result whose content is one change object, not an array: left out, and named. + assert.deepEqual(read([{ type: 'tool_result', toolUseId: 't3', content: change() }]), [{ type: 'tool_result', toolUseId: 't3', content: [] }]); + assert.match(lines.pop()!, /message m-1: .*content\[0\]\.content/); + }); +}); + +describe('the autobiographical strategy and a system change', () => { + /** Every stored position a compiled message stands for: its raw bodies, or its summaries' source ranges. */ + const spansOf = (cm: ContextManager, strategy: AutobiographicalStrategy, compiled: CompileResult): Array<[number, number] | null> => { + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + return compiled.provenance!.messages.map((s) => { + const at: number[] = s.kind === 'raw' ? s.bodies.map((b) => position.get(b.messageId)!) + : s.kind === 'summary' ? s.summaries.flatMap((x) => { + const range = strategy.getSummary(x.id)!.sourceRange; + return [position.get(range.first)!, position.get(range.last)!]; + }) : []; + return at.length ? [Math.min(...at), Math.max(...at)] : null; + }); + }; + + for (const [label, overrides, headWindowTokens] of [ + ['hierarchical', {}, 0], + ['adaptive', { adaptiveResolution: true }, 0], + ['hierarchical, the head window holding a change', {}, 300], + ['adaptive, the head window holding a change', { adaptiveResolution: true }, 300], + ] as const) { + it(`${label}: no fold takes it, it compiles exactly in place as stored, and no summarizer request reads it`, async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new AutobiographicalStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', hierarchical: true, mergeThreshold: 6, ...overrides, + }); + const cm = await open(strategy, membrane); + const block = change(); + const headBlock = change({ addSystem: ['Answer briefly.'] }); + let changeId = ''; + for (let i = 0; i < 160; i++) { + if (i === 41) changeId = cm.addMessage('Harness', [block]); + else if (i === 1 && headWindowTokens > 0) cm.addMessage('Harness', [headBlock]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await settle(cm, requests); + assert.ok(requests.length > 5, `setup: the strategy compressed (${requests.length} calls)`); + const compiled = await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const changes = compiled.messages.filter((m) => holdsChange(m.content)); + assert.equal(changes.length, headWindowTokens > 0 ? 2 : 1, 'the changes are in the compile'); + const at = compiled.messages.indexOf(changes.at(-1)!); + assert.deepEqual(compiled.messages[at]!.content, [block]); + assert.ok(compiled.messages.length < 160, 'setup: the history folded'); + // Everything before it stands for history before it, and everything after for history after. + const spans = spansOf(cm, strategy, compiled); + const position = cm.getAllMessages().findIndex((m) => m.id === changeId); + spans.forEach((span, i) => { + if (!span || i === at) return; + if (i < at) assert.ok(span[1] < position, `compiled ${i} (${span}) comes before the change at ${position}`); + else assert.ok(span[0] > position, `compiled ${i} (${span}) comes after the change at ${position}`); + }); + assert.ok(!JSON.stringify(requests).includes('system_change'), 'no summarizer request carries one'); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.length > 0 && chunks.every((c) => c.messages.every((m) => m.id !== changeId)), 'no chunk holds it'); + }); + } + + for (const [label, overrides, opts] of [ + ['hierarchical', {}, undefined], + ['adaptive', { adaptiveResolution: true }, undefined], + ['kv-unified', { adaptiveResolution: true, foldingStrategy: 'kv-unified', kvUnified: KV_UNIFIED }, { kvUnifiedImmutablePrefixHash: 'v1' }], + ] as const) { + it(`${label}: after a head-window reset, the changes before the head are part of it, ahead of the head window, in order, once`, async () => { + const strategy = new AutobiographicalStrategy({ headWindowTokens: 400, recentWindowTokens: 100, autoTickOnNewMessage: false, ...overrides } as never); + const cm = await open(strategy); + const define = change(); + const remove = change({ defineTools: undefined, addSystem: undefined, removeTools: ['get_weather'] }); + const inHead = change({ defineTools: undefined, addSystem: ['Answer briefly.'] }); + const ids: string[] = []; + for (let i = 0; i < 30; i++) { + if (i === 5) ids.push(cm.addMessage('Harness', [define])); + else if (i === 12) ids.push(cm.addMessage('Harness', [remove])); + else if (i === 22) ids.push(cm.addMessage('Harness', [inHead])); + else ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(10)}`)])); + } + strategy.resetHeadWindow(ids[20]!); + // Before any compile, render stats come from the store alone. + const before = cm.getRenderStats()!; + const compiled = await cm.compile(BIG, undefined, opts as never); + const order = rawIds(compiled); + const where = (i: number): number => order.indexOf(ids[i]!); + assert.ok(where(21) === where(20) + 1 && where(0) > where(20), 'setup: the head runs from message 20, and the history before it comes after it'); + assert.deepEqual([where(5), where(12), where(20), where(21), where(22)], [0, 1, 2, 3, 4], 'the changes lead, in their order, then the head window, a change inside it in its place'); + assert.equal(order.filter((id) => id === ids[5] || id === ids[12] || id === ids[22]).length, 3, 'each once'); + assert.deepEqual(toolsAfter(compiled), [], 'the removal applies after the definition it removes'); + // The head costs what it sends: the changes before it and the window, each once. + const head = order.slice(0, where(0)); + const headTokens = head.reduce((n, id) => n + estimateOf(cm, id), 0); + assert.deepEqual(before.head, { messages: head.length, tokens: headTokens }, 'render stats before a compile'); + assert.deepEqual(cm.getRenderStats()!.head, { messages: head.length, tokens: headTokens }, "the compile's render stats"); + // The adaptive paths reserve the head at that price: below it they refuse the head and tail, and say what the head cost. + if (label !== 'hierarchical') { + await assert.rejects(cm.compile({ maxTokens: headTokens - 1, reserveForResponse: 0 }, undefined, opts as never), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.equal(err.diagnostics.headTokens, headTokens); + return true; + }); + } + }); + } + + for (const [label, overrides, opts] of [ + ['hierarchical', {}, undefined], + ['adaptive', { adaptiveResolution: true }, undefined], + ['kv-unified', { adaptiveResolution: true, foldingStrategy: 'kv-unified', kvUnified: KV_UNIFIED }, { kvUnifiedImmutablePrefixHash: 'v1' }], + ] as const) { + it(`${label}: with the head window disabled, a reset sends the history before it once, in order, a change in it in its place`, async () => { + const strategy = new AutobiographicalStrategy({ headWindowTokens: 0, recentWindowTokens: 100, autoTickOnNewMessage: false, ...overrides } as never); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 30; i++) ids.push(i === 15 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(10)}`)])); + // A reset before the recent window starts, and one inside it. + for (const at of [20, 27]) { + strategy.resetHeadWindow(ids[at]!); + assert.deepEqual(rawIds(await cm.compile(BIG, undefined, opts as never)), ids, `reset at ${at}`); + } + }); + } + + it('render stats before a first compile count no message in both the head and the tail', async () => { + const strategy = new AutobiographicalStrategy({ headWindowTokens: 60, recentWindowTokens: 100_000, autoTickOnNewMessage: false }); + const cm = await open(strategy); + for (let i = 0; i < 12; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(10)}`)]); + const before = cm.getRenderStats()!; + assert.ok(before.head.messages > 0 && before.head.messages < 12, `setup: the head holds ${before.head.messages} of 12`); + assert.equal(before.head.messages + before.tail.messages, 12); + const after = (await cm.compile(BIG), cm.getRenderStats()!); + assert.deepEqual([before.head, before.tail], [after.head, after.tail], "the compile's"); + }); + + it('adaptive: a change after the newest turn is no turn, so a structural repair that drops that turn still refuses', async () => { + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, headWindowTokens: 400, recentWindowTokens: 0, autoTickOnNewMessage: false }); + const cm = await open(strategy); + cm.addMessage('User', [text('message ' + 'word '.repeat(10))]); + // An unanswered call is the newest turn. + cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + // A reset at a change after it puts the change in the head, so the call is + // the last compiled message, and the structural repair trims it. + strategy.resetHeadWindow(cm.addMessage('Harness', [change()])); + await assert.rejects(cm.compile(BIG), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Structural repair did not retain the newest turn/); + return true; + }); + }); + + it('a transition summary reads no change: a head message of changes alone adds no entry to its request', async () => { + const { requests, membrane } = recordingMembrane(); + const cm = await open(new AutobiographicalStrategy({ + compressionModel: 'm', headWindowTokens: 100_000, recentWindowTokens: 100_000, autoTickOnNewMessage: false, + }), membrane); + cm.addMessage('User', [text('first')]); + cm.addMessage('Harness', [change()]); + cm.addMessage('Claude', [text('second')]); + await cm.resetHeadWindow(); + assert.equal(requests.length, 1); + const sent = JSON.stringify(requests[0]); + assert.ok(sent.includes('User: first') && sent.includes('Claude: second'), 'setup: the head is in the request'); + assert.ok(!sent.includes('Harness'), 'no entry for the message of changes'); + }); + + it('the coverage check names an unrendered change before the head\'s start as the head\'s only while the head window holds messages', async () => { + const strategy = new AutobiographicalStrategy({ autoTickOnNewMessage: false }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 10; i++) ids.push(i === 3 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i}`)])); + const messages = cm.getAllMessages() as StoredMessage[]; + // Every message but the change, as a compile that lost it would hand the check. + const entries = messages.filter((m) => m.id !== ids[3]).map((m, index) => ({ index, sourceMessageId: m.id, sourceRelation: 'raw' as const, participant: m.participant, content: m.content })); + const check = strategy as unknown as { assertFullCoverage(e: unknown[], m: StoredMessage[], r: { headStart: number; headEnd: number; recentStart: number }): void }; + const label = (headStart: number, headEnd: number): string => { + try { check.assertFullCoverage(entries, messages, { headStart, headEnd, recentStart: 8 }); } catch (err) { + assert.ok(err instanceof UncoveredDropError, String(err)); + return err.site; + } + return assert.fail('no refusal'); + }; + // After a reset at 5 with the head window holding messages, the change at 3 is the head's (headPositions). + assert.match(label(5, 8), /in-head-not-rendered=1/); + // With the head window empty or disabled, no head holds it: it's history before the head's start. + assert.match(label(5, 5), /before-head-start-not-rendered=1/); + assert.match(label(5, 0), /before-head-start-not-rendered=1/); + }); + + it('KnowledgeStrategy: no chunk holds or spans a change, and a chunk closes there', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + let changeId = ''; + for (let i = 0; i < 80; i++) { + if (i === 21) changeId = cm.addMessage('Harness', [change()]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.some((c) => c.messages.some((m) => position.get(m.id)! < 21)), 'setup: chunks before the change'); + assert.ok(chunks.every((c) => c.messages.every((m) => m.id !== changeId)), 'no chunk holds it'); + for (const chunk of chunks) { + const at = chunk.messages.map((m) => position.get(m.id)!); + assert.ok(!(at[0]! < 21 && at.at(-1)! > 21), `chunk ${at[0]}..${at.at(-1)} doesn't span the change`); + } + // It closes only the chunk it ends: the chunks after it hold several messages, as the rest do. + const later = chunks.filter((c) => position.get(c.messages[0]!.id)! > 21); + assert.ok(later.length > 0 && later.every((c) => c.messages.length > 1), `chunks after it: ${later.map((c) => c.messages.length)}`); + }); + + it('KnowledgeStrategy: no chunk holds or spans a change first, side by side, beside a pin or last, none is empty, and a pair of changes closes one chunk', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + const at = [0, 30, 31, 51, 79]; + const ids: string[] = []; + for (let i = 0; i < 80; i++) { + ids.push(at.includes(i) ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + cm.pinRange(ids[50]!, ids[50]!); + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.length > 4, `setup: ${chunks.length} chunks`); + assert.ok(chunks.every((c) => c.messages.length > 0), 'no chunk is empty'); + for (const chunk of chunks) { + const span = chunk.messages.map((m) => position.get(m.id)!); + for (const c of at) { + assert.ok(!span.includes(c), `chunk ${span[0]}..${span.at(-1)} doesn't hold the change at ${c}`); + assert.ok(!(span[0]! < c && span.at(-1)! > c), `chunk ${span[0]}..${span.at(-1)} doesn't span the change at ${c}`); + } + } + // The two changes side by side close one chunk between them, not two. + const after = chunks.find((c) => position.get(c.messages[0]!.id) === 32); + assert.ok(after && after.messages.length > 1, `the chunk after the pair holds ${after?.messages.length} messages`); + }); + + it('KnowledgeStrategy: once a change between two compressed chunks is removed, each keeps its span, so compression goes on past them', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 30; i++) { + ids.push(i === 22 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + const span = (c: { messages: Array<{ id: string }> }): string => c.messages.map((m) => m.id).join(','); + await settle(cm, requests); + const compressed = chunks().filter((c) => c.compressed).map(span); + assert.ok(compressed.some((s) => s.endsWith(ids[21]!)) && compressed.some((s) => s.startsWith(ids[23]!)), 'setup: the chunks either side of the change are compressed'); + cm.removeMessage(ids[22]!); + for (let i = 30; i < 45; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await settle(cm, requests); + for (const s of compressed) assert.ok(chunks().map(span).includes(s), 'each compressed chunk keeps its span'); + assert.ok(chunks().slice(0, -1).every((c) => c.compressed), `every chunk but the newest is compressed: ${chunks().map((c) => (c.compressed ? 'C' : 'u')).join('')}`); + }); + + it('KnowledgeStrategy: a summary ending at an unanswered call doesn\'t part the call from the result that comes after it', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + for (let i = 0; i < 9; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const use = cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + await settle(cm, requests); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'setup: a summary ends at the call'); + const result = cm.addMessage('User', [{ type: 'tool_result', toolUseId: 't1', content: 'read it' } as ContentBlock]); + await cm.compile(BIG); + assert.ok(!chunks().some((c) => c.messages.at(-1)!.id === use), 'no chunk ends at the call now that its result follows it'); + assert.ok(chunks().some((c) => c.messages.some((m) => m.id === use) && c.messages.some((m) => m.id === result)), 'the call and its result share a chunk'); + // As on main, that chunk overlaps the summary, which refuses it, so both go out raw, the result with its call. + await settle(cm, requests); + const sent = rawIds(await cm.compile(BIG)); + assert.ok(sent.includes(use) && sent.includes(result), 'the call and its result are sent raw'); + }); + + it('KnowledgeStrategy: a summary ending at a call that a plain message follows still closes there', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + for (let i = 0; i < 9; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const use = cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); + await settle(cm, requests); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'setup: a summary ends at the call'); + // No result comes; the conversation goes on. + const after = cm.addMessage('User', [text(`never mind ${'word '.repeat(30)}`)]); + await cm.compile(BIG); + assert.ok(chunks().some((c) => c.compressed && c.messages.at(-1)!.id === use), 'the summary keeps its chunk'); + assert.ok(!chunks().some((c) => c.messages.some((m) => m.id === use) && c.messages.some((m) => m.id === after)), 'the message after it starts a chunk of its own'); + }); + + it('KnowledgeStrategy: a chunk closes where a summary covers its messages, even once the summary\'s last message is pinned, and where an unpinned message precedes a summary', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 1000, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 30; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + // A message pinned before anything compresses is in no summary. + const early = cm.pinRange(ids[12]!, ids[12]!); + const chunks = (): Array<{ messages: Array<{ id: string }>; compressed: boolean }> => + (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }>; compressed: boolean }> }).chunks; + const chunkOf = (id: string): Array<{ id: string }> | undefined => chunks().find((c) => c.messages.some((m) => m.id === id))?.messages; + await settle(cm, requests); + // The last message of the summary holding M-20, pinned after it compressed. + const covered = chunks().find((c) => c.compressed && c.messages.some((m) => m.id === ids[20]))!; + const last = covered.messages.at(-1)!.id; + const next = ids[ids.indexOf(last) + 1]!; + assert.ok(chunks().filter((c) => c.compressed).length >= 3 && next !== undefined, 'setup: compressed chunks, and a message after the one pinned'); + cm.pinRange(last, last); + cm.unpin(early); + for (let i = 30; i < 45; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await settle(cm, requests); + const beforeLast = covered.messages.at(-2)!.id; + assert.ok(!chunkOf(beforeLast)?.some((m) => m.id === next), 'no chunk runs on past a summary whose last message is pinned'); + // Summaries made before it was unpinned, which don't cover it. + const others = new Set((strategy as unknown as { summaries: Array<{ level: number; sourceIds: string[] }> }).summaries + .filter((x) => x.level === 1 && !x.sourceIds.includes(ids[12]!)).flatMap((x) => x.sourceIds)); + assert.ok(chunkOf(ids[12]!)!.every((m) => !others.has(m.id)), 'the unpinned message shares no chunk with another summary\'s messages'); + }); + + it('KnowledgeStrategy: once a change is removed, the chunks rebuilt around where it was run across that place', async () => { + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 40; i++) { + ids.push(i === 22 ? cm.addMessage('Harness', [change()]) : cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + const chunks = (): Array<{ messages: Array<{ id: string }> }> => (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + const spans = (): string[][] => chunks().map((c) => c.messages.map((m) => m.id)); + await cm.compile(BIG); + assert.ok(spans().some((s) => s.at(-1) === ids[21]) && spans().some((s) => s[0] === ids[23]), 'setup: a chunk ends at the message before the change, and one starts after it'); + cm.removeMessage(ids[22]!); + await cm.compile(BIG); + assert.ok(spans().some((s) => s.includes(ids[21]!) && s.includes(ids[23]!)), `a chunk now holds both neighbours: ${JSON.stringify(spans().map((s) => s.length))}`); + }); + + it('KnowledgeStrategy: a store without changes chunks by its other rules, so a chunk can span a pinned message', async () => { + const { requests, membrane } = recordingMembrane(); + const strategy = new KnowledgeStrategy({ + compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0, autoTickOnNewMessage: false, + minChunkCharsForLLM: 0, summaryParticipant: 'Claude', mergeThreshold: 6, + }); + const cm = await open(strategy, membrane); + const ids: string[] = []; + for (let i = 0; i < 80; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + cm.pinRange(ids[21]!, ids[21]!); + await settle(cm, requests); + await cm.compile({ maxTokens: 4000, reserveForResponse: 0 }); + const position = new Map(cm.getAllMessages().map((m, i) => [m.id, i] as const)); + const chunks = (strategy as unknown as { chunks: Array<{ messages: Array<{ id: string }> }> }).chunks; + assert.ok(chunks.some((c) => { + const at = c.messages.map((m) => position.get(m.id)!); + return at[0]! < 21 && at.at(-1)! > 21; + }), 'a chunk runs across the pin'); + }); + + it('stays raw under a level pin over it, where pin bounds are read', async () => { + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, foldingStrategy: 'kv-stable', autoTickOnNewMessage: false } as never); + const cm = await open(strategy); + const ids = [cm.addMessage('User', [text('a')]), cm.addMessage('Harness', [change()]), cm.addMessage('User', [text('b')])]; + cm.pinRange(ids[0]!, ids[2]!, { name: 'level-1', level: 1 }); + const bounds = (strategy as unknown as { resolvedPinBounds(m: StoredMessage[]): Map }).resolvedPinBounds(cm.getAllMessages()); + assert.deepEqual([bounds.get(0), bounds.get(1), bounds.get(2)], [{ level: 1 }, 'raw', { level: 1 }]); + }); + + it('compression input passes over a message it left empty: no separator where it sat, and its neighbours join', () => { + const collapse = (AutobiographicalStrategy.prototype as unknown as { + collapseConsecutiveMessages(m: Array<{ participant: string; content: ContentBlock[] }>): Array<{ participant: string; content: ContentBlock[] }>; + }).collapseConsecutiveMessages; + const sep = text('\n\n---\n\n'); + assert.deepEqual(collapse([{ participant: 'User', content: [text('a')] }, { participant: 'User', content: [] }, { participant: 'User', content: [text('b')] }]), + [{ participant: 'User', content: [text('a'), sep, text('b')] }]); + assert.deepEqual(collapse([{ participant: 'User', content: [text('a')] }, { participant: 'Harness', content: [] }, { participant: 'User', content: [text('b')] }]), + [{ participant: 'User', content: [text('a'), sep, text('b')] }]); + // An empty first message, and nothing but empty ones. + assert.deepEqual(collapse([{ participant: 'Harness', content: [] }, { participant: 'User', content: [text('a')] }]), + [{ participant: 'User', content: [text('a')] }]); + assert.deepEqual(collapse([{ participant: 'User', content: [] }, { participant: 'Claude', content: [] }]), []); + }); + + it('a large change is never sharded at ingress', async () => { + const cm = await open(new AutobiographicalStrategy({ compressionModel: 'm', targetChunkTokens: 200, adaptiveResolution: true, autoTickOnNewMessage: false })); + cm.addMessage('Harness', [change({ addSystem: ['x '.repeat(5000)] })]); + assert.equal(cm.getAllMessages().length, 1); + }); +}); + +describe('the passthrough strategies and a system change before their window', () => { + /** + * `first`, a change A, M-00..M-15, a change B, M-16..M-19: B sits among the + * newest five messages, and the budget fits both changes and those five. + */ + async function seeded(strategy: ContextStrategy): Promise<{ cm: ContextManager; ids: Map; T: number; A: number; B: number }> { + const cm = await open(strategy); + const ids = new Map(); + ids.set('first', cm.addMessage('User', [text('first ' + 'word '.repeat(50))])); + ids.set('A', cm.addMessage('Harness', [change()])); + for (let i = 0; i < 20; i++) { + if (i === 16) ids.set('B', cm.addMessage('Harness', [change({ addSystem: ['Use metric units, always.'] })])); + const label = `M-${String(i).padStart(2, '0')}`; + ids.set(label, cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`${label} ${'word '.repeat(30)}`)])); + } + const T = estimateOf(cm, ids.get('M-00')!); + const A = estimateOf(cm, ids.get('A')!); + const B = estimateOf(cm, ids.get('B')!); + assert.ok(B > T / 2, `setup: a change counted twice would cost a message (B ${B}, T ${T})`); + return { cm, ids, T, A, B }; + } + const expected = ['A', 'M-15', 'B', 'M-16', 'M-17', 'M-18', 'M-19']; + const labelsOf = (ids: Map, compiled: CompileResult): string[] => { + const label = new Map([...ids].map(([k, v]) => [v, k] as const)); + return rawIds(compiled).map((id) => label.get(id)!); + }; + + it('passthrough: when the oldest no longer fit, the changes are sent ahead of the rest, counted once, within the budget', async () => { + const { cm, ids, T, A, B } = await seeded(new PassthroughStrategy()); + const budget = A + B + 5 * T + Math.floor(T / 2); + const compiled = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + assert.deepEqual(labelsOf(ids, compiled), expected); + assert.ok(sentTokens(cm, compiled) <= budget); + assert.deepEqual(changesSent(compiled), [cm.getMessage(ids.get('A')!)!.content, cm.getMessage(ids.get('B')!)!.content], 'each as stored'); + }); + + it('windowed-passthrough: a re-anchor past a change keeps it ahead of the window, sizes the window as that fraction of the room the changes leave, and no compile exceeds the budget', async () => { + const strategy = new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 }); + const { cm, ids, T, A, B } = await seeded(strategy); + const budget = 2 * (A + B + 5 * T) + T; + const compiled = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + assert.ok(strategy.getAnchor() > cm.getMessage(ids.get('A')!)!.sequence, 'setup: the anchor moved past the change'); + const labels = labelsOf(ids, compiled); + assert.equal(labels[0], 'A', 'the change before the window leads'); + assert.deepEqual(changesSent(compiled), [cm.getMessage(ids.get('A')!)!.content, cm.getMessage(ids.get('B')!)!.content], 'each once, as stored'); + // The window's turns are the newest that fit in half the room the changes leave, and no more. + const room = (budget - A - B) * 0.5; + const turns = labels.filter((l) => l.startsWith('M-')); + const W = turns.reduce((n, l) => n + estimateOf(cm, ids.get(l)!), 0); + const older = `M-${String(Number(turns[0]!.slice(2)) - 1).padStart(2, '0')}`; + assert.ok(W <= room && W + estimateOf(cm, ids.get(older)!) > room, `window ${W}, room ${room}, next older ${older}`); + // Appends grow the window until the next re-anchor, the change before it priced in all along. + let anchors = 0; + for (let i = 20; i < 60; i++) { + const anchor = strategy.getAnchor(); + cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const next = await cm.compile({ maxTokens: budget, reserveForResponse: 0 }); + if (strategy.getAnchor() !== anchor) anchors++; + assert.ok(sentTokens(cm, next) <= budget, `compile after M-${i}: ${sentTokens(cm, next)} within ${budget}`); + assert.ok(holdsChange(next.messages[0]!.content), 'a change still leads'); + } + assert.ok(anchors >= 2, `setup: the window moved again (${anchors})`); + }); + + it('windowed-passthrough: when the changes fill the target, the window still starts at the newest turn, not at a trailing change', async () => { + const cm = await open(new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 })); + cm.addMessage('User', [text('old ' + 'word '.repeat(40))]); + const big = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + const user = cm.addMessage('User', [text('the newest turn ' + 'word '.repeat(10))]); + const last = cm.addMessage('Harness', [change({ addSystem: ['Use metric units.'] })]); + const C = estimateOf(cm, big) + estimateOf(cm, last); + const U = estimateOf(cm, user); + assert.ok(U > (U + 5) * 0.5, 'setup: the turn alone passes the window the changes leave room for, so the progress clause picks it'); + const compiled = await cm.compile({ maxTokens: C + U + 5, reserveForResponse: 0 }); + assert.deepEqual(rawIds(compiled), [big, user, last]); + }); + + it('every strategy refuses with OverBudgetError, rather than drop one, when the changes alone exceed the budget, and names nothing that would clear it', async () => { + const huge = change({ addSystem: ['Rule. '.repeat(400)] }); + const budget: TokenBudget = { maxTokens: 300, reserveForResponse: 0 }; + for (const [label, strategy] of [ + ['passthrough', new PassthroughStrategy()], + ['windowed-passthrough', new WindowedPassthroughStrategy()], + ['hierarchical', new AutobiographicalStrategy({ autoTickOnNewMessage: false })], + ['adaptive', new AutobiographicalStrategy({ adaptiveResolution: true, autoTickOnNewMessage: false })], + ] as const) { + const cm = await open(strategy); + cm.addMessage('User', [text('hello')]); + cm.addMessage('Harness', [huge]); + cm.addMessage('User', [text('again')]); + await assert.rejects(cm.compile(budget), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); + assert.deepEqual(err.clearedBy, [], `${label}: only consolidating clears it`); + if (label === 'passthrough') assert.match(err.message, /^Passthrough kept only its system changes/, label); + // Its window still holds the newest conversation message, which its numbers count. + if (label === 'windowed-passthrough') assert.match(err.message, /^Windowed passthrough: its system changes and the newest conversation message/, label); + if (label === 'hierarchical') assert.match(err.message, /system changes/, label); + return true; + }); + } + // A store of changes alone: over the budget, each passthrough strategy names them alone; + // filling it exactly, each sends them. + for (const [label, make, stage] of [ + ['passthrough', () => new PassthroughStrategy(), /^Passthrough kept only its system changes/], + ['windowed-passthrough', () => new WindowedPassthroughStrategy(), /^Windowed passthrough: its system changes alone/], + ] as const) { + const alone = await open(make()); + const id = alone.addMessage('Harness', [huge]); + await assert.rejects(alone.compile(budget), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); + assert.match(err.message, stage, label); + return true; + }); + assert.deepEqual(rawIds(await alone.compile({ maxTokens: estimateOf(alone, id), reserveForResponse: 0 })), [id], label); + } + // A preview reports rather than refuses: passthrough sends the change alone. + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('hello')]); + const id = cm.addMessage('Harness', [huge]); + assert.deepEqual(rawIds(await cm.compile(budget, undefined, { dryRun: true })), [id]); + }); + + it('the hierarchical path refuses an eviction from its recent window while the changes fit, as it refuses any', async () => { + const cm = await open(new AutobiographicalStrategy({ autoTickOnNewMessage: false })); + cm.addMessage('Harness', [change()]); + for (let i = 0; i < 12; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + await assert.rejects(cm.compile({ maxTokens: 300, reserveForResponse: 0 }), UncoveredDropError); + }); + + it('adaptive: a plan that can\'t fit beside a change in the middle names nothing, though compression is queued', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, recentWindowTokens: 50, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + for (let i = 0; i < 30; i++) { + if (i === 10) cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(400)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + await assert.rejects(cm.compile({ maxTokens: 300, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, []); + return true; + }); + }); + + it('adaptive: a plan refusal names compression only while the head, the tail and the changes between them fit', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 200, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + let c = ''; + for (let i = 0; i < 30; i++) { + if (i === 10) c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + const C = estimateOf(cm, c); + const refusal = async (maxTokens: number): Promise => { + try { await cm.compile({ maxTokens, reserveForResponse: 0 }); } catch (err) { assert.ok(err instanceof OverBudgetError, String(err)); return err; } + return assert.fail(`no refusal at ${maxTokens}`); + }; + // The change alone fits, beside the tail it doesn't: compression can't clear that. + const tight = await refusal(C + 10); + assert.ok(tight.diagnostics.tailTokens > 10, `setup: tail ${tight.diagnostics.tailTokens}, change ${C}`); + assert.deepEqual(tight.clearedBy, []); + // Room for the tail and the change, but not the unfolded middle: compression can. + const roomy = await refusal(tight.diagnostics.tailTokens + C + 20); + assert.deepEqual(roomy.clearedBy, ['compression']); + }); + + it('adaptive: a change in the recent window counts once toward what compression can\'t shrink', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + })); + let c = ''; + for (let i = 0; i < 30; i++) { + if (i === 28) c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })]); + else cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + } + const C = estimateOf(cm, c); + let first: OverBudgetError | undefined; + try { await cm.compile({ maxTokens: C + 1, reserveForResponse: 0 }); } catch (err) { first = err as OverBudgetError; } + // The tail holds the change; a budget between the tail and the tail with the change again fits it once, not the middle. + const tail = first!.diagnostics.tailTokens; + assert.ok(tail > C, `setup: the tail (${tail}) holds the change (${C})`); + await assert.rejects(cm.compile({ maxTokens: tail + Math.floor(C / 2), reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, ['compression']); + return true; + }); + }); + + it('adaptive: a change a reset left in the head counts once toward what compression can\'t shrink', async () => { + const strategy = new AutobiographicalStrategy({ + adaptiveResolution: true, compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 200, recentWindowTokens: 200, + overBudgetGraceRatio: 0, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, + }); + const cm = await open(strategy); + let c = ''; + const ids: string[] = []; + for (let i = 0; i < 40; i++) { + if (i === 10) ids.push(c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(60)] })])); + else ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + } + strategy.resetHeadWindow(ids[20]!); + const C = estimateOf(cm, c); + let first: OverBudgetError | undefined; + try { await cm.compile({ maxTokens: 1, reserveForResponse: 0 }); } catch (err) { first = err as OverBudgetError; } + assert.ok(first instanceof OverBudgetError, String(first)); + const { headTokens, tailTokens } = first.diagnostics; + assert.ok(headTokens > C, `setup: the head (${headTokens}) holds the change (${C})`); + // Room for the head and the tail, the change among them once, but not the unfolded middle: compression can. + await assert.rejects(cm.compile({ maxTokens: headTokens + tailTokens + Math.floor(C / 2), reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.doesNotMatch(err.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(err.clearedBy, ['compression']); + return true; + }); + }); + + it('adaptive: a refusal of the head and tail finds the newest conversation message past the changes after it, so it names a smaller recent window only when that message fits beside them', async () => { + const cm = await open(new AutobiographicalStrategy({ + adaptiveResolution: true, headWindowTokens: 0, recentWindowTokens: 100_000, overBudgetGraceRatio: 0, autoTickOnNewMessage: false, + })); + for (let i = 0; i < 6; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); + const turn = cm.addMessage('User', [text('the newest turn ' + 'word '.repeat(60))]); + const c = cm.addMessage('Harness', [change()]); + const U = estimateOf(cm, turn); + const C = estimateOf(cm, c); + const refusal = async (maxTokens: number): Promise => { + try { await cm.compile({ maxTokens, reserveForResponse: 0 }); } catch (err) { assert.ok(err instanceof OverBudgetError, String(err)); return err; } + return assert.fail(`no refusal at ${maxTokens}`); + }; + const tight = await refusal(U + C - 1); + assert.match(tight.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(tight.clearedBy, [], 'the newest message and the change after it don\'t fit together'); + const roomy = await refusal(U + C + 5); + assert.match(roomy.message, /Head\/tail reservation exceeded budget/); + assert.deepEqual(roomy.clearedBy, ['smaller-recent-window']); + }); + + for (const [label, make] of [ + ['passthrough', () => new PassthroughStrategy()], + ['windowed-passthrough', () => new WindowedPassthroughStrategy()], + ] as const) { + it(`${label}: refuses when the newest conversation message doesn't fit beside its changes, naming a newer one, which clears it, while they leave room for one`, async () => { + const cm = await open(make()); + const c = cm.addMessage('Harness', [change()]); + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(400))]); + const C = estimateOf(cm, c); + const U = estimateOf(cm, big); + await assert.rejects(cm.compile({ maxTokens: C + 10 + 50, reserveForResponse: 50 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, ['newer-message']); + assert.match(err.message, /its system changes and the newest conversation message/); + assert.deepEqual([err.budget, err.actual], [C + 10, C + U], 'the budget less the reserve, and the changes with the newest message'); + assert.deepEqual([err.diagnostics.headTokens, err.diagnostics.tailTokens], [C, U], 'the changes, then the newest message'); + return true; + }); + // Changes that fill the budget leave no room for any message. + await assert.rejects(cm.compile({ maxTokens: C, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual(err.clearedBy, []); + return true; + }); + // A change after the newest message: that message is still the one priced as the newest. + const trailing = cm.addMessage('Harness', [change({ addSystem: ['Answer briefly.'] })]); + const C2 = estimateOf(cm, trailing); + await assert.rejects(cm.compile({ maxTokens: C + C2 + 10, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.deepEqual([err.diagnostics.headTokens, err.diagnostics.tailTokens], [C + C2, U], 'the changes, then the newest conversation message'); + return true; + }); + // A newer conversation message that fits beside the changes clears it. + const small = cm.addMessage('User', [text('ok')]); + const compiled = await cm.compile({ maxTokens: C + C2 + estimateOf(cm, small), reserveForResponse: 0 }); + assert.deepEqual(rawIds(compiled), [c, trailing, small]); + }); + } + + it('passthrough: without changes, a newest message over the budget still sends nothing, as on main', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('hello')]); + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(400))]); + const compiled = await cm.compile({ maxTokens: estimateOf(cm, big) - 1, reserveForResponse: 0 }); + assert.equal(compiled.messages.length, 0); + // Nor does a response reserve larger than the budget make it refuse. + assert.equal((await cm.compile({ maxTokens: 10, reserveForResponse: 20 })).messages.length, 0); + // The windowed strategy refuses it, naming the newest message alone. + const windowed = await open(new WindowedPassthroughStrategy()); + windowed.addMessage('User', [text('hello')]); + const big2 = windowed.addMessage('User', [text('big ' + 'word '.repeat(400))]); + await assert.rejects(windowed.compile({ maxTokens: estimateOf(windowed, big2) - 1, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Windowed passthrough: the newest message alone/); + return true; + }); + }); + + it('windowed-passthrough: a refused compile leaves the anchor where the last sent window started', async () => { + const strategy = new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 }); + const cm = await open(strategy); + const ids: string[] = []; + for (let i = 0; i < 6; i++) ids.push(cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)])); + const budget: TokenBudget = { maxTokens: 20 * estimateOf(cm, ids[0]!), reserveForResponse: 0 }; + await cm.compile(budget); + const anchor = strategy.getAnchor(); + const c = cm.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(2000)] })]); + cm.addMessage('User', [text('again')]); + assert.ok(estimateOf(cm, c) > budget.maxTokens, 'setup: the change alone is over the budget'); + await assert.rejects(cm.compile(budget), OverBudgetError); + assert.equal(strategy.getAnchor(), anchor, 'the refusal moved nothing'); + // Consolidated: the turns that fit are still in the window. + cm.removeMessage(c); + assert.equal(rawIds(await cm.compile(budget))[0], ids[0]); + // Without any change too: a newest message alone over the budget is refused, and a retry with a larger budget sends the older turns. + const big = cm.addMessage('User', [text('big ' + 'word '.repeat(3000))]); + assert.ok(estimateOf(cm, big) > budget.maxTokens, 'setup: the newest alone is over the budget'); + await assert.rejects(cm.compile(budget), OverBudgetError); + assert.equal(strategy.getAnchor(), anchor, 'that refusal moved nothing either'); + assert.equal(rawIds(await cm.compile(BIG))[0], ids[0]); + }); +}); + +describe('firstChanged', () => { + it('is null at first, the count when nothing changed, the first new index after an append, and a dry compile neither reads nor moves it', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + assert.equal((await cm.compile(BIG)).firstChanged, null); + assert.equal((await cm.compile(BIG)).firstChanged, 2); + cm.addMessage('User', [text('c')]); + const dry = await cm.compile(BIG, undefined, { dryRun: true }); + assert.equal(dry.firstChanged, undefined); + assert.equal((await cm.compile(BIG)).firstChanged, 2); + // An injection placed before the last user message changes from there. + const injected = await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + assert.equal(injected.firstChanged, 2); + // Taking it out again changes from there too. + assert.equal((await cm.compile(BIG)).firstChanged, 2); + // An injection rebuilt as a new array of the same bytes is the same message. + await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + const same = await cm.compile(BIG, [{ namespace: 'n', position: 'beforeUser', content: [text('note')] }]); + assert.equal(same.firstChanged, same.messages.length); + }); + + it('tells a participant apart from content, and sees an edit, a removal, and an injection changed in place', async () => { + const cm = await open(new PassthroughStrategy()); + const ids = [cm.addMessage('User', [text('a')]), cm.addMessage('Claude', [text('b')]), cm.addMessage('User', [text('c')])]; + await cm.compile(BIG); + // The same content under another participant: the injection's namespace names it. + await cm.compile(BIG, [{ namespace: 'one', position: 'afterUser', content: [text('note')] }]); + assert.equal((await cm.compile(BIG, [{ namespace: 'two', position: 'afterUser', content: [text('note')] }])).firstChanged, 3); + // An injection's own array, changed in place between compiles. + const note = [text('note v1')]; + await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }]); + note[0] = text('note v2'); + assert.equal((await cm.compile(BIG, [{ namespace: 'n', position: 'afterUser', content: note }])).firstChanged, 3); + await cm.compile(BIG); + cm.editMessage(ids[1]!, [text('b, edited')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + cm.removeMessage(ids[0]!); + assert.equal((await cm.compile(BIG)).firstChanged, 0); + }); + + it('is not compared with a compile that throws before it returns', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + await cm.compile(BIG); + cm.addMessage('User', [text('c')]); + const internals = cm as unknown as { renderedLayoutFor: (...args: unknown[]) => unknown }; + const layout = internals.renderedLayoutFor; + internals.renderedLayoutFor = () => { internals.renderedLayoutFor = layout; throw new Error('layout failed'); }; + await assert.rejects(cm.compile(BIG), /layout failed/); + // The same three messages again: the last compile that returned sent two. + assert.equal((await cm.compile(BIG)).firstChanged, 2); + }); + + it('does not count cache markers', async () => { + const cm = await open(new WindowedPassthroughStrategy()); + cm.addMessage('User', [text('a')]); + let previous = await cm.compile(BIG); + let moved = false; + for (let i = 0; i < 6 && !moved; i++) { + cm.addMessage(i % 2 ? 'User' : 'Claude', [text(`m${i}`)]); + const next = await cm.compile(BIG); + moved = previous.messages.some((m, k) => !!m.cacheBreakpoint !== !!next.messages[k]!.cacheBreakpoint); + if (moved) assert.equal(next.firstChanged, previous.messages.length, 'a marker moved, and nothing else changed'); + previous = next; + } + assert.ok(moved, 'setup: a marker moved on a message both compiles hold'); + }); + + it('compares bytes across a fork, a branch switch, a switch made on the store itself, and a new strategy, and is null only at the first compile', async () => { + const cm = await open(new PassthroughStrategy()); + const a = cm.addMessage('User', [text('a')]); + assert.equal((await cm.compile(BIG)).firstChanged, null); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + const main = cm.currentBranch().name; + await cm.fork(); + const forked = cm.currentBranch().name; + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'a fork holds the same messages'); + cm.addMessage('Claude', [text('on the fork')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + await cm.switchBranch(main); + assert.equal((await cm.compile(BIG)).firstChanged, 1, "back on main, a prefix of the fork's"); + const alt = cm.branchAt(a, 'alt'); + cm.addMessage('User', [text('b')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + cm.getStore().switchBranch(alt); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'after the host switched the store'); + await cm.setStrategy(new PassthroughStrategy()); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'a new strategy compiling the same messages'); + // Two branches of the same length and participants, whose second messages differ. + cm.addMessage('Claude', [text('on alt')]); + assert.equal((await cm.compile(BIG)).firstChanged, 1); + await cm.switchBranch(forked); + assert.equal((await cm.compile(BIG)).firstChanged, 1, 'the fork differs from the second message on'); + }); +}); From a15cdf7c88854a2cab7c984b922a73569da1dc61 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Thu, 22 Oct 2026 08:29:27 -0400 Subject: [PATCH 18/67] fix: refuse a system change between a tool call and its result; compare firstChanged when read; find changes once per listing Answers the automated review of bbca9e33. - A write that would put a system change between a tool call and a result that answers it is refused (MessageStore.refuseToolPairSplit): a message of changes while one of the nearest calls before it has no result between it and the change, a tool result answering a call that a change already follows, and an edit giving a message a tool_use that a change follows before its results. It reads back from the write over changes and over messages holding results but no call, to the nearest message holding a call, reading each message whole, every shard of its body, since ingress sharding puts a message's calls and results on its first shard. It checks that message's calls by id: a result in a later message answers one, as parallel calls' results in messages of their own do, and so does the message's own later result, as in a bundled cycle (splitMixedToolMessages). It stops at a message of any other kind, so a call no result answered further back refuses nothing; the refusal says to store a result for a call that will get none, or remove the call. MessageStore.append and edit check it, and ContextManager.addMessage checks first, so a refused add releases no expired hold. The API takes a system message with content only after a user turn, or an assistant turn ending in a server tool result, so membrane sends a change stored there after the result anyway. Stored between them, a change parted them: the base chunker and KnowledgeStrategy close a chunk at a kept-raw change, so the call and its result were folded into different summaries, and once the call's chunk was summarized pairing repair dropped the result. What a write can't see is left as on main, and its doc says so: a message of another kind between a call and its result, a call in an earlier message than the nearest one holding a call, a shape that exists only in a view, filtered or merged from several stores, and a removal or edit that takes away a result or a message between. The adaptive test of a structural repair after a reset at a trailing change reaches its store through a removal. - CompileResult.firstChanged is compared when read, through an accessor. A committing compile still records its keys (references, and an injection's bytes, taken then since a caller may change its own array), but a caller that never reads the field serializes nothing else. The first read, or an assignment, settles the value and lets go of both compiles' keys; it stays an accessor, so a frozen or sealed result reads it too. Read after a later compile, it still compares its compile with the one before it. - systemChangePositions remembers its answer per listing (a WeakMap on the store's array, which callers never change and a write replaces), since a select reads it several times. - WindowedPassthroughStrategy.renderAsNewest gives a system change before the anchor as select sends it, ahead of the window, where it gave null: setAnchor can move the anchor past a message just stored. - Fragments: the membrane floor's raise to ^0.7.0 gets a breaking entry, as 0.16.0's raise to ^0.6.0 had; the KnowledgeStrategy summary-span fix joins the fixed fragment; the added fragment names the refusal and that firstChanged is compared when read. Tests in system-changes.test.ts: the refusal through the manager before an expired hold is released and through the store, and a change taken after the result; parallel calls answered in messages of their own; a call on a sharded body's first shard; a sharded results message read whole walking back from a change and forward from an edited call; a change after a bundled cycle and edits giving one, a bundled message's call that none of its later blocks answers left open, a plain message and the store's end ending the forward walk; a call further back than a message of another kind refusing nothing; a tool result answering a call a change follows, as a removal leaves them, and another change there; edits from either side, through the manager and the store; renderAsNewest for a change before windowed's anchor; a listing's changes found once; firstChanged compared when read, serializing nothing on a read of the same arrays, read first after a later compile, carried by a copy and JSON, read on a frozen result, taking an assignment, and leaving out system injections. Two existing tests now check what they name: the Knowledge call/result test asserts the compiled blocks, not provenance ids, and the pinned-summary test checks its subject chunk exists. Co-Authored-By: Claude Opus 5.5 --- changelog.d/system-changes.added.md | 14 +- changelog.d/system-changes.breaking.md | 3 + changelog.d/system-changes.fixed.md | 5 + src/context-manager.ts | 86 +++++--- src/message-store.ts | 142 +++++++++++++ src/strategies/autobiographical.ts | 4 + src/strategies/knowledge.ts | 3 + src/strategies/windowed-passthrough.ts | 14 +- src/system-changes.ts | 18 +- src/types/context.ts | 9 +- test/system-changes.test.ts | 268 ++++++++++++++++++++++++- 11 files changed, 517 insertions(+), 49 deletions(-) create mode 100644 changelog.d/system-changes.breaking.md diff --git a/changelog.d/system-changes.added.md b/changelog.d/system-changes.added.md index d6ad6840..73c56596 100644 --- a/changelog.d/system-changes.added.md +++ b/changelog.d/system-changes.added.md @@ -25,8 +25,15 @@ `addSystem` isn't an array of strings, or whose `defineTools` isn't an array of tool definitions, each with a non-empty string `name`, a string `description` and an object `inputSchema` whose `type` is `'object'`; a read - leaves such a change out. Ingress sharding never splits a message of them, - since it holds no text block. + leaves such a change out. A write also refuses a change while one of the + nearest calls before it has no result between it and the change (counting + back over other changes and over results, by id, to the nearest message + holding a call, every shard of it), + a tool result answering a call that a change already follows, and an edit + giving a message a `tool_use` that a change follows before its results: + membrane sends a change there after the result anyway, so write it after + the result. Ingress sharding never splits a message of them, since it holds + no text block. Only membrane's `NativeFormatter`, on an adapter that sends system changes, sends one; elsewhere membrane refuses the request. Rolling back to a release without them isn't supported once a store holds one. @@ -38,4 +45,5 @@ with each new user message, so after one it's that injection's old index. It compares bytes, so it holds across a branch switch or a new strategy, and it's `null` only for the manager's first committing compile. A dry - compile leaves it out and doesn't move it. + compile leaves it out and doesn't move it. It's compared when read, + so a compile whose caller never reads it serializes only its injections. diff --git a/changelog.d/system-changes.breaking.md b/changelog.d/system-changes.breaking.md new file mode 100644 index 00000000..9758a413 --- /dev/null +++ b/changelog.d/system-changes.breaking.md @@ -0,0 +1,3 @@ +- **Host operators:** take `@animalabs/membrane` 0.7.0 or later in your own range with this release. context-manager now requires `^0.7.0`, where it required `^0.6.0`, since what it stores and compiles is typed with 0.7.0's `ContentBlock`, which adds `system_change`, and npm installs context-manager's own 0.7.0 nested beside a host's older membrane without complaint. A host whose own membrane is older doesn't send a stored system change: 0.6.0's `NativeFormatter` leaves out a block type it doesn't convert. + - **Migration:** raise the host's own `@animalabs/membrane` range to `^0.7.0`. + - **Unchanged:** what context-manager stores, and how it calls membrane, for a host that writes no system change. diff --git a/changelog.d/system-changes.fixed.md b/changelog.d/system-changes.fixed.md index ee816531..6683104b 100644 --- a/changelog.d/system-changes.fixed.md +++ b/changelog.d/system-changes.fixed.md @@ -14,3 +14,8 @@ - `getRenderStats()` before the first compile no longer counts a message in both the head and the tail when the recent window starts inside the head window: the tail starts where the head ends, as a compile's does. +- A `KnowledgeStrategy` chunk no longer runs into or across an L1 summary's + span once a removal moves where its walk closes, which left the chunk after + it uncompressed for good and its messages sent raw beside the summary: a + chunk now closes where the L1 summary covering its messages changes, except + between a call and the result right after it. diff --git a/src/context-manager.ts b/src/context-manager.ts index 71204fde..1a83cd76 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -86,6 +86,20 @@ interface CompiledMessageKey { bytes?: string; } +/** The index of the first of `keys` that differs from `before`'s at its index, or has none there (ContextManager.recordFirstChanged). */ +function firstDifference(keys: readonly CompiledMessageKey[], before: readonly CompiledMessageKey[]): number { + const bytesOf = (key: CompiledMessageKey): string => (key.bytes ??= JSON.stringify(key.content)); + let i = 0; + for (; i < keys.length && i < before.length; i++) { + const now = keys[i]!; + const then = before[i]!; + if (now.participant !== then.participant) break; + if (now.content === then.content && !now.injected && !then.injected) continue; + if (bytesOf(now) !== bytesOf(then)) break; + } + return i; +} + /** * Base configuration for ContextManager. */ @@ -551,6 +565,9 @@ export class ContextManager { // later (appendMessage), once expired holds have been released, as at // any add. content = contentForWrite(content, 'Message refused'); + // So is a system change between a tool call and its result + // (MessageStore.refuseToolPairSplit). + this.messageStore.refuseToolPairSplit(content, 'Message refused'); return this.write(() => { this.expireCompressionHolds(); if (!options?.holdCompression) { @@ -1268,7 +1285,7 @@ export class ContextManager { if (this.debugLogContext) this.logCompiledContext(result); // Last, so a compile that throws before returning, which no caller // sends, is never the one the next is compared with. - if (!opts?.dryRun) result.firstChanged = this.measureFirstChanged(result.messages, injected); + if (!opts?.dryRun) this.recordFirstChanged(result, injected); return result; }; @@ -1342,43 +1359,48 @@ export class ContextManager { } /** - * The index of the first compiled message whose participant or content - * bytes differ from the last committing compile's message at that index, or - * that has none there (so the first appended index after appends alone, and - * the message count when these messages are that compile's or a prefix of - * them); null before this manager's first committing compile. The - * comparison is by bytes, so it holds across a branch or strategy switch, - * which the manager can't always see (a direct switch of the store's - * branch away and back). Cache markers are message flags, not content, so - * they don't count. Records these messages for the next - * (CompiledMessageKey). + * Records a committing compile's messages for the next one's comparison + * (CompiledMessageKey), and gives `result` its `firstChanged`, compared + * when it's read: a caller that never reads it pays for no comparison. + * That is the index of the first compiled message whose + * participant or content bytes differ from the last committing compile's + * message at that index, or that has none there (so the first appended + * index after appends alone, and the message count when these messages are + * that compile's or a prefix of them); null before this manager's first + * committing compile. The comparison is by bytes, so it holds across a + * branch or strategy switch, which the manager can't always see (a direct + * switch of the store's branch away and back). Cache markers are message + * flags, not content, so they don't count. Read after later compiles, it + * still compares this compile with the one before it. Until it's read, a + * result keeps this compile's keys and the last one's. */ - private measureFirstChanged( - messages: readonly NormalizedMessage[], - injected: readonly (readonly ContentBlock[])[], - ): number | null { + private recordFirstChanged(result: CompileResult, injected: readonly (readonly ContentBlock[])[]): void { const fromInjection = new Set(injected); - const keys: CompiledMessageKey[] = messages.map((message) => fromInjection.has(message.content) + const keys: CompiledMessageKey[] = result.messages.map((message) => fromInjection.has(message.content) ? { participant: message.participant, content: message.content, injected: true, bytes: JSON.stringify(message.content) } : { participant: message.participant, content: message.content, injected: false }); const before = this.lastCompiled; - if (!before) { - this.lastCompiled = keys; - return null; - } - const bytesOf = (key: CompiledMessageKey): string => (key.bytes ??= JSON.stringify(key.content)); - let i = 0; - for (; i < keys.length && i < before.length; i++) { - const now = keys[i]!; - const then = before[i]!; - if (now.participant !== then.participant) break; - if (now.content === then.content && !now.injected && !then.injected) continue; - if (bytesOf(now) !== bytesOf(then)) break; - } - // Recorded once the comparison has returned: one that throws leaves the - // last compile that returned as the next one's comparison. this.lastCompiled = keys; - return i; + // The first read, or a write, settles the value and lets go of both + // compiles' keys, so the result then holds a number. It stays an + // accessor, so a frozen or sealed result reads it too. + let pending: { keys: CompiledMessageKey[]; before: typeof before } | null = { keys, before }; + let value: number | null | undefined; + Object.defineProperty(result, 'firstChanged', { + enumerable: true, + configurable: true, + get: () => { + if (pending) { + value = pending.before ? firstDifference(pending.keys, pending.before) : null; + pending = null; + } + return value; + }, + set: (next: number | null | undefined) => { + value = next; + pending = null; + }, + }); } /** diff --git a/src/message-store.ts b/src/message-store.ts index bb2d479a..c0edd846 100644 --- a/src/message-store.ts +++ b/src/message-store.ts @@ -22,6 +22,37 @@ import { BlobManager } from './blob-manager.js'; import { bodyBoundsIn, bodyEnd, bodyStart, continuesBody } from './body-runs.js'; import { contentForWrite, wellFormedContent } from './well-formed-content.js'; +/** Whether content, written or stored, holds a top-level block of `type`; content that isn't an array holds none. */ +function holdsBlock(content: unknown, type: string): boolean { + return Array.isArray(content) && content.some((block) => (block as { type?: unknown } | null)?.type === type); +} + +/** + * Each call content holds, by id, and whether it's still open after the + * content's own later blocks: a result after its call in the same content, + * as in a bundled cycle (splitMixedToolMessages), answers it. + */ +function callsInOrder(content: unknown): Map { + const calls = new Map(); + if (!Array.isArray(content)) return calls; + for (const block of content as Array<{ type?: unknown; id?: unknown; toolUseId?: unknown } | null>) { + if (block?.type === 'tool_use' && typeof block.id === 'string') calls.set(block.id, true); + else if (block?.type === 'tool_result' && typeof block.toolUseId === 'string' && calls.has(block.toolUseId)) calls.set(block.toolUseId, false); + } + return calls; +} + +/** The ids of content's top-level calls (`tool_use`), or of the calls its results (`tool_result`) answer, in block order. */ +function toolIds(content: unknown, type: 'tool_use' | 'tool_result'): string[] { + if (!Array.isArray(content)) return []; + const ids: string[] = []; + for (const block of content as Array<{ type?: unknown; id?: unknown; toolUseId?: unknown } | null>) { + const id = type === 'tool_use' ? block?.id : block?.toolUseId; + if (block?.type === type && typeof id === 'string') ids.push(id); + } + return ids; +} + const DEFAULT_MESSAGE_STATE_ID = 'messages'; /** @@ -459,6 +490,7 @@ export class MessageStore { } ): StoredMessage { content = contentForWrite(content, 'Message refused'); + this.refuseToolPairSplit(content, 'Message refused'); // Extract blobs from content const storedContent = this.blobManager.extractBlobs(content); @@ -567,6 +599,115 @@ export class MessageStore { return message; } + /** + * Refuses a write that would put a system change between a tool call and + * a result that answers it: a message of changes while one of the nearest + * calls before it has no result between it and the change, a tool result + * answering a call that a change already follows, or an edit giving a + * message a `tool_use` that a change follows before the call's results. + * The API takes a system message with content only after a user turn, or an + * assistant turn ending in a server tool result, so membrane sends a change + * stored after a call after the call's result anyway, and one written after + * the result goes out the same. Stored between them, a change parts them: + * the autobiographical and Knowledge strategies keep a change raw and close + * a chunk at it, so the call and its result would be folded into different + * summaries, and a compile's pairing repair (enforceToolPairing) would drop + * the result once the call was folded. + * + * It reads back from `index` over changes and messages holding results but + * no call, to the nearest message that holds a call, reading each message + * whole, every shard of its body, since ingress sharding puts a message's + * calls and results on its body's first shard. It checks that message's + * calls, by id: those a result in a later message answers, as parallel + * calls' results in messages of their own do, and those its own later + * blocks answer, as in a bundled cycle (splitMixedToolMessages), are + * answered. The walk stops at a message of any other kind, so a call no + * result answered, further back, refuses nothing. `index` is where the + * content goes: the end for an append, the message's place for an edit. + * + * It leaves alone what a write can't see, as main does: a message of + * another kind between a call and its result, which is already off the + * API's shape; a call in an earlier message than the nearest one holding + * a call; a shape that exists only in a view, filtered or merged from + * several stores; and a removal, or an edit, that takes away a result or a + * message between. Each parts the pair once the call is folded. + */ + refuseToolPairSplit(content: ContentBlock[], what: string, index: number = this.length()): void { + const length = this.length(); + const at = (i: number): StoredMessageInternal | null => (i >= 0 && i < length ? this.getInternal(i) : null); + // A body is one step of either walk, read whole: ingress sharding puts a + // message's non-text blocks, its calls and results, on its first shard. + const bodyAt = (i: number): { from: number; to: number; shards: StoredMessageInternal[] } => { + const from = bodyStart(at, i); + const to = bodyEnd(length, at, i); + const shards: StoredMessageInternal[] = []; + for (let k = from; k <= to; k++) shards.push(at(k)!); + return { from, to, shards }; + }; + const holds = (shards: StoredMessageInternal[], type: string): boolean => shards.some((shard) => holdsBlock(shard.content, type)); + const idsIn = (shards: StoredMessageInternal[], type: 'tool_use' | 'tool_result'): string[] => + shards.flatMap((shard) => toolIds(shard.content, type)); + const change = holdsBlock(content, 'system_change'); + const answers = change ? [] : toolIds(content, 'tool_result'); + if (change || answers.length > 0) { + // Back over changes and results, to the nearest body holding a call. + const answered = new Set(); + let changeBetween: StoredMessageInternal | undefined; + const calls = new Map(); + for (let i = index - 1; i >= 0; ) { + const { from, shards } = bodyAt(i); + if (holds(shards, 'tool_use')) { + // The body's calls left open by its own blocks, read in order + // across its shards, each by the shard that holds it. + const holder = new Map(); + for (const shard of shards) for (const id of toolIds(shard.content, 'tool_use')) holder.set(id, shard.id); + for (const [id, open] of callsInOrder(shards.flatMap((shard) => shard.content))) if (open) calls.set(id, holder.get(id)!); + break; + } + if (holds(shards, 'system_change')) changeBetween ??= shards[0]; + else if (holds(shards, 'tool_result')) for (const id of idsIn(shards, 'tool_result')) answered.add(id); + else break; + i = from - 1; + } + if (calls.size > 0) { + if (change) { + const open = [...calls].filter(([id]) => !answered.has(id)); + if (open.length > 0) { + throw new Error( + `${what}: a system change can't go between a tool call and its result, and message ${open[0]![1]} holds a tool_use ` + + `(${open.map(([id]) => id).join(', ')}) with no result between it and this change. Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.`, + ); + } + } else if (changeBetween) { + const id = answers.find((answer) => calls.has(answer)); + if (id !== undefined) { + throw new Error( + `${what}: a tool_result can't follow a system change (message ${changeBetween.id}) that follows its tool call ` + + `(${id}, message ${calls.get(id)}). Remove the change and write it after the result.`, + ); + } + } + } + } + if (index < length && holdsBlock(content, 'tool_use')) { + // Forward over changes and results, while a call here is unanswered. + const open = new Set([...callsInOrder(content)].filter(([, stillOpen]) => stillOpen).map(([id]) => id)); + for (let i = index + 1; i < length && open.size > 0; ) { + const { to, shards } = bodyAt(i); + if (holds(shards, 'tool_use')) break; + if (holds(shards, 'system_change')) { + throw new Error( + `${what}: a tool_use (${[...open].join(', ')}) can't go before a system change (message ${shards[0]!.id}) that comes before its result; ` + + "a change goes after the call's result.", + ); + } + if (!holds(shards, 'tool_result')) break; + for (const id of idsIn(shards, 'tool_result')) open.delete(id); + i = to + 1; + } + } + } + /** * Edit a message's content. * @@ -593,6 +734,7 @@ export class MessageStore { `Sharded messages are immutable — remove the whole bodyGroup and re-append instead.`, ); } + this.refuseToolPairSplit(newContent, `Edit of ${messageId} refused`, index); // The old content as a read gives it, so an edit can replace a message // whose stored content doesn't match ContentBlock. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 883c9463..d9e5164d 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -16907,6 +16907,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // distant chronological regions and poisoned every ancestor merge. // Close even a thin run at the ownership seam. A short summary (or // raw fallback while it is pending) is cheaper than a non-tree span. + // A system change, which stays raw, is such a seam. The store refuses + // a change between a call and its result where a write can see it + // (MessageStore.refuseToolPairSplit), so this close parts a pair only + // in a shape that refusal leaves alone. closeCurrent(i); } let msgTokens = store.estimateTokens(msg); diff --git a/src/strategies/knowledge.ts b/src/strategies/knowledge.ts index ce983e98..fa4abf2c 100644 --- a/src/strategies/knowledge.ts +++ b/src/strategies/knowledge.ts @@ -144,6 +144,9 @@ export class KnowledgeStrategy extends AutobiographicalStrategy { // first message, ahead of the change, so what came after the change // would be read before it. Close even a thin run there. A store // without changes whose chunks match its summaries chunks as before. + // The store refuses a change between a call and its result where a + // write can see it (MessageStore.refuseToolPairSplit), so this close + // parts a pair only in a shape that refusal leaves alone. closeAt(i); } const phase = this.classifyMessage(msg, lastToolPhase); diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 75588040..af9e7fb6 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -423,10 +423,10 @@ export class WindowedPassthroughStrategy implements ContextStrategy { * the live-image policy's verdicts applied (liveImagePolicy, stripImages), * each decided from its message and newer ones, with depth from the view's * newest message, so they're select's. A message before the anchor comes - * back as null: select leaves it out, but for a system change, which it - * sends ahead of the window. A change there isn't a just-stored message, - * which is what this is for, since the anchor only moves to a message a - * compile sent, and a message stored later comes after it. Like select, it + * back as null, since select leaves it out, but for a system change, which + * comes back as select sends it ahead of the window, cut as any message + * is: a compile moves the anchor only to a message it sent, but a + * caller's setAnchor can move it past one just stored. Like select, it * first reloads the branch the store has moved to. A re-anchor the next * compile's budget forces isn't foreseen: it moves the anchor only past a * window bigger than the budget allows. A body sharded under the @@ -459,7 +459,11 @@ export class WindowedPassthroughStrategy implements ContextStrategy { this.stripImages(entries, this.liveImagePolicy(messages, store, from)); const rendered = new Map(); for (const entry of entries) rendered.set(entry.sourceMessageId!, entry.content); - return ids.map((id) => rendered.get(id) ?? null); + return ids.map((id) => { + const i = at.get(id); + if (i !== undefined && !shown(i) && holdsSystemChange(messages[i]!.content)) return this.cappedContent(messages[i]!); + return rendered.get(id) ?? null; + }); } /** 'window' when the last committed select moved the window's start. */ diff --git a/src/system-changes.ts b/src/system-changes.ts index e2790103..ef36e71a 100644 --- a/src/system-changes.ts +++ b/src/system-changes.ts @@ -4,7 +4,11 @@ * the conversation on. One stands alone in its message * (well-formed-content.ts), and membrane sends such a message as a system * message of its own, at the first point at or after its place that follows - * a user turn, so always before the reply the request asks for. + * a user turn, so always before the reply the request asks for. So one + * stored between a tool call and its result would go out after the result, + * and the store refuses that write where a write can see it + * (MessageStore.refuseToolPairSplit): kept raw there, it would part the call + * from its result. * * Every strategy compiles such a message unaltered, in its place among the * raw messages. A fold or a window that left one out would take its change @@ -47,9 +51,19 @@ export function holdsSystemChange(content: readonly ContentBlock[]): boolean { return false; } -/** The positions in a listing of the messages that hold a system change, in order. */ +const positionsOf = new WeakMap(); + +/** + * The positions in a listing of the messages that hold a system change, in + * order. A select reads them several times, so they're found once per + * listing: a listing is the store's, which callers never change + * (MessageStore.getAll), and a write makes a new one. + */ export function systemChangePositions(messages: readonly { content: readonly ContentBlock[] }[]): readonly number[] { + const known = positionsOf.get(messages); + if (known) return known; const positions: number[] = []; for (let i = 0; i < messages.length; i++) if (holdsSystemChange(messages[i]!.content)) positions.push(i); + positionsOf.set(messages, positions); return positions; } diff --git a/src/types/context.ts b/src/types/context.ts index cf8992da..77c5cde3 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -194,7 +194,14 @@ export interface CompileResult { * doesn't move the comparison; a committing compile that returns moves it * whether or not its request is sent, and one that throws doesn't. A * caller can tell from it whether a compile rewrote what the last - * committing compile gave, such as a system change it wrote. + * committing compile gave, such as a system change it wrote. It's compared + * when read, so a compile whose caller never reads it serializes only its + * injections, whose bytes a committing compile takes since a caller may + * change its own array: read it only where it's used. + * Read after later compiles, it still compares this compile with the one + * before it. Until it's read, the result keeps this compile's compiled + * messages and the last committing compile's, by reference; once read, it + * holds the number. */ firstChanged?: number | null; } diff --git a/test/system-changes.test.ts b/test/system-changes.test.ts index 2157b2ab..456eab90 100644 --- a/test/system-changes.test.ts +++ b/test/system-changes.test.ts @@ -33,6 +33,7 @@ import { } from '../src/index.js'; import { jsonTokenEstimator } from '../src/message-store.js'; import { wellFormedContent } from '../src/well-formed-content.js'; +import { systemChangePositions } from '../src/system-changes.js'; import type { CompileResult, ContextStrategy, StoredMessage, TokenBudget } from '../src/types/index.js'; const dir = mkdtempSync(join(tmpdir(), 'cm-system-changes-')); @@ -43,6 +44,8 @@ const tool = { name: 'get_weather', description: 'Weather.', inputSchema: { type const change = (extra: Record = {}): ContentBlock => ({ type: 'system_change', defineTools: [tool], addSystem: ['Use metric units.'], ...extra }) as unknown as ContentBlock; const text = (t: string): ContentBlock => ({ type: 'text', text: t }); +const toolUse = (id: string): ContentBlock => ({ type: 'tool_use', id, name: 'get_weather', input: { city: 'Oslo' } }) as ContentBlock; +const toolResult = (id: string, t = 'Sunny.'): ContentBlock => ({ type: 'tool_result', toolUseId: id, content: t }) as ContentBlock; const BIG = { maxTokens: 1_000_000, reserveForResponse: 0 }; async function open(strategy: ContextStrategy, membrane?: unknown): Promise { @@ -125,6 +128,161 @@ describe('a system change, written and priced', () => { assert.equal(cm.getAllMessages().length, 0); }); + it('is refused while the nearest call before it has no result between them, by the manager before an expired hold is released and by the store, and taken after the result', async () => { + const clock = { t: 0 }; + let released = 0; + const strategy = new (class extends PassthroughStrategy { + onCompressionHoldsReleased(): void { released++; } + })(); + const cm = await ContextManager.open({ path: join(dir, `store-${opened++}`), strategy, now: () => clock.t }); + cm.addMessage('User', [text('Weather in Oslo?')], undefined, undefined, { holdCompression: { timeoutMs: 100 } }); + const call = cm.addMessage('Claude', [toolUse('t1')]); + clock.t = 150; + assert.throws(() => cm.addMessage('Harness', [change()]), (e: Error) => e.message === + `Message refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (t1) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + assert.equal(released, 0, 'refused before the expired hold was released'); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[]): unknown } }).messageStore; + assert.throws(() => store.append('Harness', [change()]), /a system change can't go between a tool call and its result/); + assert.equal(cm.getAllMessages().length, 2); + cm.addMessage('User', [toolResult('t1')]); + assert.equal(released, 1); + // After the result, and after another change there. + cm.addMessage('Harness', [change()]); + cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]); + assert.equal(cm.getAllMessages().length, 5); + }); + + it('is refused while any of parallel calls is unanswered, their results in messages of their own, and taken once the last lands', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather in Oslo and Bergen?')]); + const call = cm.addMessage('Claude', [toolUse('a'), toolUse('b')]); + cm.addMessage('User', [toolResult('a')]); + assert.throws(() => cm.addMessage('Harness', [change()]), (e: Error) => e.message === + `Message refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (b) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + cm.addMessage('User', [toolResult('b')]); + cm.addMessage('Harness', [change()]); + assert.equal(cm.getAllMessages().length, 5); + }); + + it("is refused after a call on a sharded body's first shard, whichever shard is last, and the refusal names the shard holding the call", async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[], metadata?: unknown, causedBy?: unknown, extra?: unknown): StoredMessage } }).messageStore; + cm.addMessage('User', [text('Summarize this and look it up.')]); + // Ingress sharding puts a message's non-text blocks on its body's first shard. + const first = store.append('Claude', [text('part one '), toolUse('t1')], undefined, undefined, { bodyGroupId: 'g', shardIndex: 0, shardCount: 2 }); + store.append('Claude', [text('part two')], undefined, undefined, { bodyGroupId: 'g', shardIndex: 1, shardCount: 2 }); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${first.id} holds a tool_use \\(t1\\) with no result between it and this change`)); + cm.addMessage('User', [toolResult('t1')]); + cm.addMessage('Harness', [change()]); + }); + + it("reads a sharded results message whole, from its first shard, walking back from a change and forward from an edited call", async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { append(participant: string, content: ContentBlock[], metadata?: unknown, causedBy?: unknown, extra?: unknown): StoredMessage } }).messageStore; + cm.addMessage('User', [text('Weather in Oslo and Bergen?')]); + const call = cm.addMessage('Claude', [toolUse('a'), toolUse('b')]); + // Ingress sharding puts a message's results on its first shard, its long text after. + store.append('User', [toolResult('a')], undefined, undefined, { bodyGroupId: 'r', shardIndex: 0, shardCount: 2 }); + store.append('User', [text('a long note beside the result')], undefined, undefined, { bodyGroupId: 'r', shardIndex: 1, shardCount: 2 }); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${call} holds a tool_use \\(b\\) with no result between it and this change`)); + cm.addMessage('User', [toolResult('b')]); + const later = cm.addMessage('Harness', [change()]); + // Forward: an edit giving a message calls that the sharded results answer only in part, before the change. + cm.editMessage(call, [text('Checking.')]); + assert.throws(() => cm.editMessage(call, [toolUse('a'), toolUse('c')]), (e: Error) => e.message === + `Edit of ${call} refused: a tool_use (c) can't go before a system change (message ${later}) that comes before its result; a change goes after the call's result.`); + cm.editMessage(call, [toolUse('a'), toolUse('b')]); + assert.equal(cm.getAllMessages().length, 6); + }); + + it('takes a change after a bundled cycle, whose own later blocks answer its calls, an edit giving one to a message right before a change, and an edit giving a call to a message a plain one follows', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + // A bundled cycle, as claude.ai exports store one: the compile splits it (splitMixedToolMessages). + cm.addMessage('Claude', [text('Checking.'), toolUse('t1'), toolResult('t1'), text('Sunny.')]); + cm.addMessage('Harness', [change()]); + const reply = cm.addMessage('Claude', [text('Anything else?')]); + cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]); + cm.editMessage(reply, [toolUse('t2'), toolResult('t2')]); + // A plain message between a call and the next change ends the forward walk, as it ends the back walk. + const asked = cm.addMessage('Claude', [text('Let me see.')]); + cm.addMessage('User', [text('ok')]); + cm.addMessage('Harness', [change({ addSystem: ['Be kind.'] })]); + cm.editMessage(asked, [toolUse('t3')]); + // A call given to the last message: the forward walk ends with the store. + const last = cm.addMessage('Claude', [text('Bye.')]); + cm.editMessage(last, [toolUse('t4')]); + assert.deepEqual(cm.getAllMessages().map((m) => m.content.map((b) => b.type).join('+')), + ['text', 'text+tool_use+tool_result+text', 'system_change', 'tool_use+tool_result', 'system_change', 'tool_use', 'text', 'system_change', 'tool_use']); + }); + + it("leaves open a bundled message's call that none of its later blocks answers: a parallel call beside an answered one, and a call after its result", async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + const parallel = cm.addMessage('Claude', [toolUse('a'), toolUse('b'), toolResult('a')]); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${parallel} holds a tool_use \\(b\\) with no result`)); + cm.addMessage('User', [toolResult('b')]); + cm.addMessage('Harness', [change()]); + // A result before its call in the same message answers nothing: the compile splits it ahead of the call. + const reversed = cm.addMessage('Claude', [toolResult('c'), toolUse('c')]); + assert.throws(() => cm.addMessage('Harness', [change()]), new RegExp(`message ${reversed} holds a tool_use \\(c\\) with no result`)); + // And an edit giving a message right before a change such a call is refused. + assert.throws(() => cm.editMessage(cm.getAllMessages()[2]!.id, [toolUse('d'), toolUse('e'), toolResult('d')]), /a tool_use \(e\) can't go before a system change/); + }); + + it('refuses nothing for a call further back than a message of another kind, whose result never came', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + cm.addMessage('Claude', [toolUse('t1')]); + // An interrupted round: the call's result never came, and the conversation went on. + cm.addMessage('User', [text('Never mind.')]); + cm.addMessage('Harness', [change()]); + assert.equal(cm.getAllMessages().length, 4); + }); + + it('refuses a tool result answering a call that a change follows, as a removal can leave them, and another change there', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('Weather?')]); + const call = cm.addMessage('Claude', [toolUse('t1'), toolUse('t2')]); + const result = cm.addMessage('User', [toolResult('t1')]); + cm.addMessage('User', [toolResult('t2')]); + const first = cm.addMessage('Harness', [change()]); + // Removing a result leaves its call unanswered before the change. + cm.removeMessage(result); + assert.throws(() => cm.addMessage('User', [toolResult('t1')]), (e: Error) => e.message === + `Message refused: a tool_result can't follow a system change (message ${first}) that follows its tool call (t1, message ${call}). ` + + 'Remove the change and write it after the result.'); + assert.throws(() => cm.addMessage('Harness', [change({ addSystem: ['Be brief.'] })]), new RegExp(`message ${call} holds a tool_use \\(t1\\)`)); + // A result answering no call there isn't refused. + cm.addMessage('User', [toolResult('other')]); + cm.addMessage('User', [text('Never mind.')]); + assert.equal(cm.getAllMessages().length, 6); + }); + + it('refuses an edit that puts a change between a call and its result, from either side, through the manager and the store, and takes one that keeps them together', async () => { + const cm = await open(new PassthroughStrategy()); + const store = (cm as unknown as { messageStore: { edit(id: string, content: ContentBlock[]): void } }).messageStore; + cm.addMessage('User', [text('Weather?')]); + const call = cm.addMessage('Claude', [toolUse('t1')]); + const result = cm.addMessage('User', [toolResult('t1')]); + const reply = cm.addMessage('Claude', [text('Sunny.')]); + const partial = cm.addMessage('User', [toolResult('t2')]); + const later = cm.addMessage('Harness', [change()]); + assert.throws(() => cm.editMessage(result, [change()]), (e: Error) => e.message === + `Edit of ${result} refused: a system change can't go between a tool call and its result, and message ${call} holds a tool_use (t1) with no result between it and this change. ` + + 'Write it after the result, where membrane sends it anyway; for a call that will get none, store a result for it first or remove the call.'); + assert.throws(() => store.edit(result, [change()]), /a system change can't go between a tool call and its result/); + // A call whose results, in the messages after it, run into a change first. + assert.throws(() => cm.editMessage(reply, [toolUse('t2'), toolUse('t3')]), (e: Error) => e.message === + `Edit of ${reply} refused: a tool_use (t3) can't go before a system change (message ${later}) that comes before its result; a change goes after the call's result.`); + cm.editMessage(reply, [toolUse('t2')]); + cm.editMessage(result, [toolResult('t1', 'Rainy.')]); + assert.deepEqual(cm.getAllMessages().map((m) => m.content[0]!.type), ['text', 'tool_use', 'tool_result', 'tool_use', 'tool_result', 'system_change']); + assert.equal(cm.getAllMessages()[4]!.id, partial); + }); + it('is refused when a field that carries its content has the wrong type, each field and element by name, and taken with several changes alone', () => { for (const [fields, place] of [ [{ removeTools: 'get_time' }, 'content[0].removeTools is a string, not an array of tool names'], @@ -315,11 +473,15 @@ describe('the autobiographical strategy and a system change', () => { const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, headWindowTokens: 400, recentWindowTokens: 0, autoTickOnNewMessage: false }); const cm = await open(strategy); cm.addMessage('User', [text('message ' + 'word '.repeat(10))]); - // An unanswered call is the newest turn. cm.addMessage('Claude', [text('calling'), { type: 'tool_use', id: 't1', name: 'read', input: {} } as ContentBlock]); - // A reset at a change after it puts the change in the head, so the call is - // the last compiled message, and the structural repair trims it. - strategy.resetHeadWindow(cm.addMessage('Harness', [change()])); + const result = cm.addMessage('User', [{ type: 'tool_result', toolUseId: 't1', content: 'read.' } as ContentBlock]); + const trailing = cm.addMessage('Harness', [change()]); + // Removing the result leaves an unanswered call as the newest turn, right + // before the change (a write can't put one there). A reset at the change + // puts it in the head, so the call is the last compiled message, and the + // structural repair trims it. + cm.removeMessage(result); + strategy.resetHeadWindow(trailing); await assert.rejects(cm.compile(BIG), (err: unknown) => { assert.ok(err instanceof OverBudgetError, String(err)); assert.match(err.message, /^Structural repair did not retain the newest turn/); @@ -466,8 +628,12 @@ describe('the autobiographical strategy and a system change', () => { assert.ok(chunks().some((c) => c.messages.some((m) => m.id === use) && c.messages.some((m) => m.id === result)), 'the call and its result share a chunk'); // As on main, that chunk overlaps the summary, which refuses it, so both go out raw, the result with its call. await settle(cm, requests); - const sent = rawIds(await cm.compile(BIG)); + const compiled = await cm.compile(BIG); + const sent = rawIds(compiled); assert.ok(sent.includes(use) && sent.includes(result), 'the call and its result are sent raw'); + const blocks = compiled.messages.flatMap((m) => m.content as Array<{ type: string; id?: string; toolUseId?: string; content?: unknown }>); + assert.ok(blocks.some((b) => b.type === 'tool_use' && b.id === 't1'), "the call's tool_use is sent"); + assert.ok(blocks.some((b) => b.type === 'tool_result' && b.toolUseId === 't1' && b.content === 'read it'), 'the result is sent with its content'); }); it('KnowledgeStrategy: a summary ending at a call that a plain message follows still closes there', async () => { @@ -515,7 +681,8 @@ describe('the autobiographical strategy and a system change', () => { for (let i = 30; i < 45; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [text(`M-${i} ${'word '.repeat(30)}`)]); await settle(cm, requests); const beforeLast = covered.messages.at(-2)!.id; - assert.ok(!chunkOf(beforeLast)?.some((m) => m.id === next), 'no chunk runs on past a summary whose last message is pinned'); + assert.ok(chunkOf(beforeLast), 'setup: the message before the pinned one is in a chunk'); + assert.ok(!chunkOf(beforeLast)!.some((m) => m.id === next), 'no chunk runs on past a summary whose last message is pinned'); // Summaries made before it was unpinned, which don't cover it. const others = new Set((strategy as unknown as { summaries: Array<{ level: number; sourceIds: string[] }> }).summaries .filter((x) => x.level === 1 && !x.sourceIds.includes(ids[12]!)).flatMap((x) => x.sourceIds)); @@ -585,6 +752,14 @@ describe('the autobiographical strategy and a system change', () => { assert.deepEqual(collapse([{ participant: 'User', content: [] }, { participant: 'Claude', content: [] }]), []); }); + it("finds a listing's changes once, and a new listing's afresh", () => { + const m = (...content: ContentBlock[]): { content: ContentBlock[] } => ({ content }); + const listing = [m(text('a')), m(change()), m(text('b'))]; + assert.deepEqual(systemChangePositions(listing), [1]); + assert.equal(systemChangePositions(listing), systemChangePositions(listing), 'the same listing: the same answer'); + assert.notEqual(systemChangePositions([...listing]), systemChangePositions(listing), 'a new listing: found again'); + }); + it('a large change is never sharded at ingress', async () => { const cm = await open(new AutobiographicalStrategy({ compressionModel: 'm', targetChunkTokens: 200, adaptiveResolution: true, autoTickOnNewMessage: false })); cm.addMessage('Harness', [change({ addSystem: ['x '.repeat(5000)] })]); @@ -890,6 +1065,18 @@ describe('the passthrough strategies and a system change before their window', ( }); }); + it('windowed-passthrough: renderAsNewest gives a change before the anchor as select sends it, ahead of the window', async () => { + const strategy = new WindowedPassthroughStrategy(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + const id = cm.addMessage('Harness', [change()]); + const after = cm.addMessage('User', [text('b')]); + strategy.setAnchor(cm.getMessage(after)!.sequence); + const compiled = await cm.compile(BIG); + assert.deepEqual(changesSent(compiled), [[change()]], 'setup: select sends the change ahead of the window'); + assert.deepEqual(cm.renderAsNewest([id]), changesSent(compiled)); + }); + it('windowed-passthrough: a refused compile leaves the anchor where the last sent window started', async () => { const strategy = new WindowedPassthroughStrategy({ reAnchorFraction: 0.5 }); const cm = await open(strategy); @@ -970,6 +1157,75 @@ describe('firstChanged', () => { assert.equal((await cm.compile(BIG)).firstChanged, 2); }); + it('is compared when read, serializing nothing again on a second read, and read after a later compile still compares with the compile before it', async () => { + // A strategy sending new arrays each compile, as caps and recall pairs + // do, whose blocks count each serialization. + let serialized = 0; + const counted = (block: ContentBlock): ContentBlock => ({ ...block, toJSON: () => { serialized++; return block; } }) as unknown as ContentBlock; + const strategy = new (class extends PassthroughStrategy { + select(...args: Parameters): ReturnType { + return super.select(...args).map((entry) => ({ ...entry, content: entry.content.map(counted) })); + } + })(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + const first = await cm.compile(BIG); + const second = await cm.compile(BIG); + const before = serialized; + assert.equal(second.firstChanged, 2, 'the same bytes in new arrays'); + assert.ok(serialized > before, 'the comparison serialized them when read'); + const once = serialized; + assert.equal(second.firstChanged, 2); + assert.equal(serialized, once, 'and not again'); + cm.editMessage(cm.getAllMessages()[1]!.id, [text('b, edited')]); + const third = await cm.compile(BIG); + cm.addMessage('User', [text('c')]); + await cm.compile(BIG); + assert.equal(first.firstChanged, null); + assert.equal(third.firstChanged, 1, 'read first after a later compile, the third against the second'); + assert.equal(second.firstChanged, 2, 'still the second compile against the first'); + }); + + it('serializes nothing on a read where the compiled arrays are the last compile\'s', async () => { + let serialized = 0; + const counted = (block: ContentBlock): ContentBlock => ({ ...block, toJSON: () => { serialized++; return block; } }) as unknown as ContentBlock; + const sent = new Map(); + const strategy = new (class extends PassthroughStrategy { + select(...args: Parameters): ReturnType { + return super.select(...args).map((entry) => { + const id = entry.sourceMessageId!; + if (!sent.has(id)) sent.set(id, entry.content.map(counted)); + return { ...entry, content: sent.get(id)! }; + }); + } + })(); + const cm = await open(strategy); + cm.addMessage('User', [text('a')]); + cm.addMessage('Claude', [text('b')]); + await cm.compile(BIG); + const second = await cm.compile(BIG); + const before = serialized; + assert.equal(second.firstChanged, 2); + assert.equal(serialized, before, 'the same arrays are compared by identity'); + }); + + it('is carried by a copy and JSON, read on a frozen result, takes an assignment, and leaves out system injections', async () => { + const cm = await open(new PassthroughStrategy()); + cm.addMessage('User', [text('a')]); + await cm.compile(BIG); + const copied = await cm.compile(BIG); + assert.equal({ ...copied }.firstChanged, 1); + assert.equal(JSON.parse(JSON.stringify(copied)).firstChanged, 1); + const frozen = Object.freeze(await cm.compile(BIG)); + assert.equal(frozen.firstChanged, 1, 'a frozen result reads it'); + const assigned = await cm.compile(BIG); + assigned.firstChanged = 7; + assert.equal(assigned.firstChanged, 7); + const withSystem = await cm.compile(BIG, [{ namespace: 'n', position: 'system', content: [text('system note')] }]); + assert.equal(withSystem.firstChanged, withSystem.messages.length, 'a system injection goes in the system prompt, not the messages'); + }); + it('does not count cache markers', async () => { const cm = await open(new WindowedPassthroughStrategy()); cm.addMessage('User', [text('a')]); From c2ecdac7082a404584601bd6c303ceb24b5dc3e6 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Thu, 22 Oct 2026 20:17:13 -0400 Subject: [PATCH 19/67] chore(deps): membrane floor ^0.7.1, not ^0.7.0 context-manager 0.19.0 raised its floor to ^0.6.1 for antra-tess/membrane#168's backport: on 0.6.1 a round a provider ends with `stop_reason: end_turn` while holding complete tool calls runs as a `tool_use` round. membrane 0.7.0 doesn't have that fix (v0.6.1 isn't an ancestor of v0.7.0; antra-tess/membrane#168 shipped in 0.7.1), so `^0.7.0` admitted a membrane that drops it. The summarizer path declares the agent's live tools, so on 0.7.0 a draft returned in a tool call under end_turn would be lost. - package.json: @animalabs/membrane ^0.7.0 -> ^0.7.1; the lockfile's root line follows (regenerated with --package-lock-only --ignore-scripts); it still resolves 0.7.1. - changelog.d/system-changes.breaking.md: the floor and migration name 0.7.1, with the reason. Co-Authored-By: Claude Opus 5.5 --- changelog.d/system-changes.breaking.md | 4 ++-- package-lock.json | 2 +- package.json | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/changelog.d/system-changes.breaking.md b/changelog.d/system-changes.breaking.md index 9758a413..e3095b41 100644 --- a/changelog.d/system-changes.breaking.md +++ b/changelog.d/system-changes.breaking.md @@ -1,3 +1,3 @@ -- **Host operators:** take `@animalabs/membrane` 0.7.0 or later in your own range with this release. context-manager now requires `^0.7.0`, where it required `^0.6.0`, since what it stores and compiles is typed with 0.7.0's `ContentBlock`, which adds `system_change`, and npm installs context-manager's own 0.7.0 nested beside a host's older membrane without complaint. A host whose own membrane is older doesn't send a stored system change: 0.6.0's `NativeFormatter` leaves out a block type it doesn't convert. - - **Migration:** raise the host's own `@animalabs/membrane` range to `^0.7.0`. +- **Host operators:** take `@animalabs/membrane` 0.7.1 or later in your own range with this release. context-manager now requires `^0.7.1`, where it required `^0.6.1`, since what it stores and compiles is typed with 0.7's `ContentBlock`, which adds `system_change`, and npm installs context-manager's own 0.7 nested beside a host's older membrane without complaint. A host whose own membrane is older doesn't send a stored system change: 0.6's `NativeFormatter` leaves out a block type it doesn't convert. The floor is 0.7.1, not 0.7.0, because 0.7.0 lacks the fix the 0.6.1 floor exists for: a round a provider ends with `stop_reason: end_turn` while holding complete tool calls runs as a `tool_use` round on 0.6.1 and 0.7.1, and ends the turn on 0.7.0. + - **Migration:** raise the host's own `@animalabs/membrane` range to `^0.7.1`. - **Unchanged:** what context-manager stores, and how it calls membrane, for a host that writes no system change. diff --git a/package-lock.json b/package-lock.json index 0eec2354..a6f991c2 100644 --- a/package-lock.json +++ b/package-lock.json @@ -10,7 +10,7 @@ "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", - "@animalabs/membrane": "^0.7.0" + "@animalabs/membrane": "^0.7.1" }, "devDependencies": { "@types/node": "^22.0.0", diff --git a/package.json b/package.json index bfe67680..d9b8597b 100644 --- a/package.json +++ b/package.json @@ -39,7 +39,7 @@ "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", - "@animalabs/membrane": "^0.7.0" + "@animalabs/membrane": "^0.7.1" }, "devDependencies": { "@types/node": "^22.0.0", From 4e8ee43b098cdf9c6c79ff72ca4f10024cfcd483 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Thu, 22 Oct 2026 20:33:04 -0400 Subject: [PATCH 20/67] changelog: drop the membrane-floor entry; main requires ^0.7.1 since 0.20.0 This branch no longer changes context-manager's membrane range: 0.20.0 already requires ^0.7.1, with its own Breaking entry for hosts. The fragment held only that floor, so it goes; the added and fixed fragments stay. Co-Authored-By: Claude Opus 5.5 --- changelog.d/system-changes.breaking.md | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 changelog.d/system-changes.breaking.md diff --git a/changelog.d/system-changes.breaking.md b/changelog.d/system-changes.breaking.md deleted file mode 100644 index e3095b41..00000000 --- a/changelog.d/system-changes.breaking.md +++ /dev/null @@ -1,3 +0,0 @@ -- **Host operators:** take `@animalabs/membrane` 0.7.1 or later in your own range with this release. context-manager now requires `^0.7.1`, where it required `^0.6.1`, since what it stores and compiles is typed with 0.7's `ContentBlock`, which adds `system_change`, and npm installs context-manager's own 0.7 nested beside a host's older membrane without complaint. A host whose own membrane is older doesn't send a stored system change: 0.6's `NativeFormatter` leaves out a block type it doesn't convert. The floor is 0.7.1, not 0.7.0, because 0.7.0 lacks the fix the 0.6.1 floor exists for: a round a provider ends with `stop_reason: end_turn` while holding complete tool calls runs as a `tool_use` round on 0.6.1 and 0.7.1, and ends the turn on 0.7.0. - - **Migration:** raise the host's own `@animalabs/membrane` range to `^0.7.1`. - - **Unchanged:** what context-manager stores, and how it calls membrane, for a host that writes no system change. From acc1feb611a4b40007c79a0df63b8a7d72cd915b Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 22:38:01 -0400 Subject: [PATCH 21/67] docs(images): the ceilings' docs and fragments follow the rule's credit The line a ceiling says is now credited by the strip under the limits, or by replaying the hysteresis rule's set without the ceilings, and reads " holds to it". The texts that still described the watermark's record follow: - imageCeiling's doc says when the line comes: an image past the ceiling that the host's own limits, the depth limit included, don't strip anyway, or, with imageStripHysteresisRatio, an image the set strips that the set without the ceilings keeps and the set with this ceiling alone strips. imageByteCeiling's says it does so as for imageCeiling. - The breaking fragment says the line comes where images are shown as placeholders because of a ceiling, and no longer names a loaded watermark. Its siblings bullet takes today's ranges: agent-framework 0.26.0 and connectome-host 0.13.0 depend on ^0.20.0 (npm, 2026-10-23). - The changed fragment names the rule's set among the places that read the clamped limits, and says when the line comes under the limits and under the rule. --- changelog.d/request-image-ceilings.breaking.md | 4 ++-- changelog.d/request-image-ceilings.changed.md | 4 ++-- src/types/strategy.ts | 15 +++++++-------- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index e1cd6638..e6738206 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling. It doesn't say so for an image the host's own limits would strip anyway, or for what a watermark loaded from the store strips, which carries no record of what moved it. +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling. It doesn't say so for an image the host's own limits would strip anyway. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** agent-framework 0.24.0 and connectome-host 0.12.0 depend on `^0.18.0` (npm, 2026-10-22), which admits a 0.18.x release and not a 0.19.0 one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. + - **Siblings:** agent-framework 0.26.0 and connectome-host 0.13.0 depend on `^0.20.0` (npm, 2026-10-23), which admits a 0.20.x release and not a 0.21.0 one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index a84b0e1a..00f3d067 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ -- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it held, at the first render a select returns where images are shown as placeholders because an image passed that ceiling, rather than because of a limit the host set. That's the image past the ceiling, unless the depth limit, the hysteresis watermark or a limit the host has in force under its ceiling strips it anyway, or the images the hysteresis watermark strips when an image past the ceiling moves it, unless the host's own limits, the depth limit included, would strip them too. A watermark the strategy loads from the store carries no record of what moved it, so the instance that loads it says nothing for what it strips. +- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips that the set without the ceilings keeps and the set with that ceiling alone strips. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 313b0aea..5a5b9dc6 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1433,11 +1433,12 @@ export interface AutobiographicalConfig { * initialize names it. A fraction is floored, to at least 1. Under a * window limit of 0 or above this ceiling, the strategy says so once, at * the first render a select returns where images are shown as - * placeholders because an image passed this ceiling, rather than because - * of a limit the host set: the image past it, or those the hysteresis - * watermark strips with it that the host's own limits wouldn't strip. A - * watermark the strategy loads from the store carries no record of what - * moved it, so the instance that loads it says nothing for what it strips. + * placeholders because of this ceiling, rather than because of a limit + * the host set: an image past it that the host's own limits, the depth + * limit included, don't strip anyway, or, with + * `imageStripHysteresisRatio`, an image the rule's set strips that the + * set without the ceilings keeps and the set with this ceiling alone + * strips. */ imageCeiling?: number; @@ -1467,9 +1468,7 @@ export interface AutobiographicalConfig { * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A * value that isn't 0 or a finite number above it takes the default, and a * fraction is floored, to at least 1. Under a window limit of 0 or above - * this ceiling, the strategy says so once, as for `imageCeiling`, with the - * same exceptions: not for an image the host's own limits would strip, and - * not for what a watermark loaded from the store strips. + * this ceiling, the strategy says so once, as for `imageCeiling`. */ imageByteCeiling?: number; From 585df8d7eec00738c1b0bfb48f3b95377f06823c Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 22:46:25 -0400 Subject: [PATCH 22/67] test(images): pin four pieces of the ceiling line no test caught A removal run at the merge's tree left four of its 85 pieces with no failing test. Each now has one: - The returned guard on the strip's notes, in the verdict path: the compare-layout test also strips with asEstimated, as the adaptive selection and the kv-unified cache check do. - The depth limit's verdict: the adaptive selection, whose strip applies the post-strip estimates' verdicts, says nothing when the depth limit strips the image a byte ceiling would, and says it without the depth limit. - The credit's "that ceiling alone strips it": both limits set by their ceilings, the byte limit taking part in the count ceiling's trim but stripping nothing alone, so only the count ceiling is named. - The credit coming from the set alone under the rule: four live images after a trim put the stripped ones past the count ceiling by the limits' own account, while the set without the ceiling strips them too, so nothing is said. --- test/request-image-ceilings.test.ts | 68 +++++++++++++++++++++++++---- 1 file changed, 60 insertions(+), 8 deletions(-) diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 0257dec4..d33b2015 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -612,10 +612,15 @@ describe('the line when an image ceiling first strips', () => { }); it("the memory strategy's strip leaves the line only from a render select returns, not a layout rendered to compare against", async () => { - for (const options of [ - { maxLiveImages: 0, imageCeiling: 1 }, - { maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, - ]) { + // Both ways the strip decides: counting what the entries send, and the + // verdicts of the post-strip estimates (asEstimated), as the adaptive + // selection and the kv-unified cache check ask for them. + for (const [options, asEstimated] of [ + [{ maxLiveImages: 0, imageCeiling: 1 }, false], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, false], + [{ maxLiveImages: 0, imageCeiling: 1 }, true], + [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 4 }, true], + ] as const) { const strategy = memory(options); const said: string[] = []; const original = console.warn; @@ -632,15 +637,15 @@ describe('the line when an image ceiling first strips', () => { index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content, })); const strip = strategy as unknown as { - applyImageStripping(entries: unknown[], store: unknown, opts?: { returned?: boolean }): void; + applyImageStripping(entries: unknown[], store: unknown, opts?: { returned?: boolean; asEstimated?: boolean }): void; reportCeilingHits(store: unknown): void; }; const kept = entries(); - strip.applyImageStripping(kept, view); + strip.applyImageStripping(kept, view, { asEstimated }); assert.equal(kept.filter((e) => e.content.some((b) => b.type === 'image')).length, 1, 'the strip ran: the ceiling replaced one'); strip.reportCeilingHits(view); - assert.deepEqual(ours(), [], `a layout rendered to compare against leaves nothing for ${JSON.stringify(options)}`); - strip.applyImageStripping(entries(), view, { returned: true }); + assert.deepEqual(ours(), [], `a layout rendered to compare against leaves nothing for ${JSON.stringify(options)}, asEstimated ${asEstimated}`); + strip.applyImageStripping(entries(), view, { returned: true, asEstimated }); assert.deepEqual(ours(), [], 'the strip itself says nothing: select reports once it returns the render'); strip.reportCeilingHits(view); assert.equal(ours().length, 1, JSON.stringify(said)); @@ -681,6 +686,22 @@ describe('the line when an image ceiling first strips', () => { } }); + it("the memory strategy's adaptive selection says nothing when the depth limit strips the image the ceiling would, and says it when only the ceiling does", async () => { + // The adaptive selection's strip applies the post-strip estimates' + // verdicts, which carry the depth limit's own: two 8-byte images under a + // 12-byte ceiling, the older one also past a 2,500-token depth limit. + const adaptive = (options: Record) => memory({ + hierarchical: false, adaptiveResolution: true, compressionModel: 'mock', targetChunkTokens: 100, + autoTickOnNewMessage: false, maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 12, ...options, + }); + const depth = await ceilingLines(adaptive({ imageStripDepthTokens: 2500 }), [{ add: 2, bytes: 8 }]); + assert.equal(depth.last, '.I', 'the depth limit strips the older image'); + assert.deepEqual(depth.lines, [], JSON.stringify(depth.lines)); + const ceilingOnly = await ceilingLines(adaptive({}), [{ add: 2, bytes: 8 }]); + assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); + assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); + }); + it("the memory strategy says nothing for the rule's set under a preview's own ceiling", async () => { const said: string[] = []; const original = console.warn; @@ -1097,6 +1118,37 @@ describe('the line when an image ceiling first strips', () => { assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); }); + it("the memory strategy's rule names only the ceiling whose limit alone strips, though the other took part", async () => { + // Both window limits are set by their ceilings: 2 images and 16 bytes. + // The third 4-byte image passes the count ceiling, and the trim, to 1 + // image and 8 bytes, strips the two older ones. The byte limit takes part + // in that trim (12 bytes are over 8), but the set under the byte ceiling + // alone, with no count, strips nothing, so only the count ceiling decided. + const { lines, last } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), + [3], + ); + assert.equal(last, '..I'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); + }); + + it("the memory strategy's rule credits only its set, not the limits' account of the images it strips", async () => { + // A host byte limit of 40, in force, and a count ceiling of 4. Four 8-byte + // images, then a 12-byte one, which passes both (5 images, 44 bytes): the + // trim, to 2 images and 20 bytes, strips the three oldest, and the host's + // byte limit alone strips the same three. Two 4-byte images then make four + // live, 28 bytes. By the limits' own account, newest first, the third + // oldest is now past the count ceiling and within the host's bytes, but + // the set decides it, and the set without the ceiling strips it too. + const { lines, patterns } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), + [{ add: 4, bytes: 8 }, { add: 1, bytes: 12 }, { add: 2, bytes: 4 }], + ); + assert.deepEqual(patterns, ['IIII', '...II', '...IIII']); + assert.deepEqual(lines, [], JSON.stringify(lines)); + }); + it("the memory strategy replays the rule for the credit only for a ceiling whose limit took part and is still to be said", async () => { // The credit replays the rule with the ceilings' limits lifted. Once each // line is said, a later compile replays nothing for it; nor does a host From 8c40a652d943f4766082f48bc5293d949b042263 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 23:23:50 -0400 Subject: [PATCH 23/67] docs(images): review round: the rule's credit names its third condition and its replays Our review of this round found the texts stating the hysteresis rule's credit with two conditions where the code has three: a ceiling is named only where its limit took part in the set the render shows, as ceilingsDeciding names only a limit the walk passes. And they didn't say what working it out costs. - imageCeiling's doc and the changed fragment state all three conditions, "that ceiling alone over the host's own limits" included, and the replays: once or twice more per compile whose render the set stripped, while a line is unsaid and its ceiling's limit takes part, three times when both take part and neither line is said. - ceilingsDeciding's doc says the same, and names the memory strategy's ruleCeilingHits. - stripReplayHeld's doc says lifting only uninvolved limits gives the same set, since lifting one beside an involved limit can make it bind. - reportCeilingHits' opening says "because of that ceiling", as the line now does, where it said an image passed the ceiling. - The counting path's comment on the returned guard names renderAsNewest's strip, the one there that select doesn't return; kv-unified's compare layout takes the verdict path. - The byte-cap note names membrane 0.7.1, the version the lockfile installs, beside 0.5.87 and 0.6.0. Its 28 MiB default, the check in transformRequest unless shedOversizeImages is set, the Anthropic adapter's own check, and the continuation builders that skip transformRequest are all the same there. --- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 9 ++++++--- src/strategies/autobiographical.ts | 12 ++++++------ src/types/strategy.ts | 10 +++++++--- 4 files changed, 20 insertions(+), 13 deletions(-) diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 00f3d067..96f67b8b 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips that the set without the ceilings keeps and the set with that ceiling alone strips. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips where that ceiling's limit took part in the set, and that the set without the ceilings keeps and the set with that ceiling alone over the host's own limits strips. Working that out replays the set: while a line is unsaid and its ceiling's limit takes part, a compile whose render the set stripped replays it once or twice more, and three times when both ceilings' limits take part and neither line is said. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 0dc9ae98..2684e744 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -26,7 +26,8 @@ export const DEFAULT_IMAGE_CEILING = 500; /** * membrane's byte cap is 28 MiB of serialized messages unless - * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise (membrane 0.5.87 and 0.6.0). + * `MEMBRANE_MAX_REQUEST_BYTES` says otherwise (membrane 0.5.87, 0.6.0 and 0.7.1, + * the version this package's lockfile installs). * It checks a request when it builds it through `transformRequest`, which a * call's first request and each native tool-loop round go through; an * XML-mode continuation round doesn't, and of the provider adapters only the @@ -165,8 +166,10 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b * the window keeps, so only that one says its line. With the memory * strategy's `imageStripHysteresisRatio`, whose set decides in place of the * limits, the strategy credits the set's images itself, not here: a ceiling - * decides an image the set strips when the set without the ceilings keeps - * it and the set with that ceiling alone strips it (ruleCeilingHits). + * decides an image the set strips when its limit took part in that set, the + * set without the ceilings keeps the image, and the set with that ceiling + * alone over the host's own limits strips it (the memory strategy's + * ruleCeilingHits). */ export function ceilingsDeciding( over: { images: boolean; bytes: boolean }, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index fabd5a88..9bbb8662 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -17440,8 +17440,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { } // A render select returns (`returned`) leaves what its ceilings decided // for select to report once it has returned it; the last such strip is - // the render it returns. A layout rendered only to compare against, such - // as kv-unified's kept one, leaves nothing. + // the render it returns. Any other strip, such as renderAsNewest's, + // leaves nothing. if (opts.returned) this._ceilingNotes = notes; } @@ -17512,8 +17512,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the tool pruning in use (withToolPruning). 0 is no limit; with none, it * strips nothing. `involved` says whether the count limit, and the byte * limit, was ever over its full or trimmed value where the replay asked: - * one that never was decided nothing, so the set with it lifted is this - * same set (ruleCeilingHits). */ + * one that never was decided nothing, so lifting only such limits gives + * this same set (ruleCeilingHits). */ private stripReplayHeld( store: MessageStoreView, messages: readonly StoredMessage[], @@ -17744,8 +17744,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { private _ceilingNotes: CeilingNotes = { images: false, bytes: false, ruleStripped: [] }; /** Say once for each ceiling, when a select returns a render in which - * images are shown as placeholders because an image passed that ceiling, - * rather than because of a limit the host set. Under the limits, that's + * images are shown as placeholders because of that ceiling, rather than + * because of a limit the host set. Under the limits, that's * an image past the ceiling that nothing else strips (see * ceilingsDeciding). With imageStripHysteresisRatio, the rule's set * decides in place of the limits, so an image it replaces is credited diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 5a5b9dc6..3b1fa4c2 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1436,9 +1436,13 @@ export interface AutobiographicalConfig { * placeholders because of this ceiling, rather than because of a limit * the host set: an image past it that the host's own limits, the depth * limit included, don't strip anyway, or, with - * `imageStripHysteresisRatio`, an image the rule's set strips that the - * set without the ceilings keeps and the set with this ceiling alone - * strips. + * `imageStripHysteresisRatio`, an image the rule's set strips where this + * ceiling's limit took part in that set, and that the set without the + * ceilings keeps and the set with this ceiling alone over the host's own + * limits strips. Working that out replays the set: while the line is + * unsaid and this ceiling's limit takes part, a compile whose render the + * set stripped replays it once or twice more, and three times when both + * ceilings' limits take part and neither line is said. */ imageCeiling?: number; From b7e73124b4db41b04dd4355906aa29c2b2620e45 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Thu, 22 Oct 2026 23:23:50 -0400 Subject: [PATCH 24/67] test(images): review round: four cases of the ceiling line the round's review named - A message whose first image passes the byte ceiling and whose second fits says the line, for both strategies: windowed passthrough ORs each image's deciding ceilings into its message's, and a plain assignment would let the kept image clear it. - Windowed passthrough says nothing for a ceiling's verdict on a message its re-anchor drops: the verdicts are decided from the old anchor, and only the entries the render holds are reported. - Under the hysteresis rule, an image in a tool result that pairing repair moves into another message's entry is credited to its own message, under the hierarchical and the adaptive select. - The credit replays nothing when asked about a render that replaced none of the set's images, though a ceiling's limit took part. --- test/request-image-ceilings.test.ts | 128 ++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index d33b2015..541e2695 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -491,6 +491,35 @@ describe('the line when an image ceiling first strips', () => { assert.ok(byStep[1]![0]!.includes('imageByteCeiling 16 bytes holds'), byStep[1]![0]); }); + it(`${name}: says it when a message's first image passes the byte ceiling and its second fits`, async () => { + // One message: a 12-byte image, then a 4-byte one, under a byte + // ceiling of 8. The first is past it on its own and the second fits, + // so the message holds one image the ceiling decided and one it kept. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }) }); + try { + manager.addMessage('user', [ + { type: 'image', source: { type: 'base64', data: 'A'.repeat(12), mediaType: 'image/png' } }, + { type: 'image', source: { type: 'base64', data: 'B'.repeat(4), mediaType: 'image/png' } }, + { type: 'text', text: 'pair' }, + ] as ContentBlock[]); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const pair = compiled.messages.find((m) => m.content.some((b) => b.type === 'text' && b.text === 'pair'))!; + assert.deepEqual(pair.content.filter((b) => b.type === 'image').map((b) => (b as { source: { data: string } }).source.data), ['B'.repeat(4)], + 'the first image is replaced, the second kept'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageByteCeiling 8 bytes holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + it(`${name}: says it for a single image bigger than the byte ceiling`, async () => { const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }), [{ add: 1, bytes: 12 }]); assert.deepEqual(patterns, ['.'], 'the one image is past the ceiling on its own'); @@ -1196,6 +1225,105 @@ describe('the line when an image ceiling first strips', () => { } }); + it("windowed passthrough says nothing for a ceiling's verdict on a message its window drops", async () => { + // Two image messages under a count ceiling of 1, about 2,000 tokens of + // text between them, past a 3,000-token budget: the window re-anchors to + // the newest message, whose image the ceiling keeps. The verdicts are + // decided from the old anchor, where the ceiling strips the older image, + // which the render no longer holds. With room for both, the same images + // say the line. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: windowed({ maxLiveImages: 0, imageCeiling: 1 }) }); + try { + manager.addMessage('user', imageMessage('img-1')); + manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(1200) }]); + manager.addMessage('user', imageMessage('img-2')); + const compiled = await manager.compile({ maxTokens: 3000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 2), '-I', 'the window holds the newest message alone'); + assert.equal(compiled.messages.length, 1, 'premise: the text went with the older image'); + assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), [], JSON.stringify(said)); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + const both = await ceilingLines(windowed({ maxLiveImages: 0, imageCeiling: 1 }), [2]); + assert.equal(both.last, '.I'); + assert.equal(both.lines.length, 1, JSON.stringify(both.lines)); + }); + + for (const hierarchical of [true, false]) { + it(`the memory strategy's rule credits a moved tool result's image to its own message (${hierarchical ? 'hierarchical' : 'adaptive'})`, async () => { + // Result a holds image A, ahead of image B in its message. Under a count + // ceiling of 1 the rule strips that message's first image, A, and keeps + // B. Pairing repair moves result a up beside result b, into another + // message's entry; A is still its own message's first image, which the + // set without the ceiling keeps. + const image = (data: string): ContentBlock => ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ hierarchical, adaptiveResolution: !hierarchical, maxLiveImages: 0, imageCeiling: 1, imageStripHysteresisRatio: 0.5 }), + }); + try { + manager.addMessage('Claude', [{ type: 'tool_use', id: 'a', name: 'shot', input: {} }, { type: 'tool_use', id: 'b', name: 'shot', input: {} }] as ContentBlock[]); + manager.addMessage('User', [{ type: 'tool_result', toolUseId: 'b', content: [{ type: 'text', text: 'result b' }] }] as unknown as ContentBlock[]); + manager.addMessage('User', [{ type: 'tool_result', toolUseId: 'a', content: [image('AAAA'), { type: 'text', text: 'result a' }] }, image('BBBB')] as unknown as ContentBlock[]); + manager.addMessage('Claude', [{ type: 'text', text: 'done' }]); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const results = (content: ContentBlock[]) => content.filter((b) => b.type === 'tool_result').map((b) => (b as unknown as { toolUseId: string }).toolUseId).sort().join(','); + assert.ok(compiled.messages.some((x) => results(x.content) === 'a,b'), 'premise: result a moved beside result b'); + const data = (content: ContentBlock[]): string[] => content.flatMap((b) => + b.type === 'image' ? [(b as { source: { data: string } }).source.data] + : b.type === 'tool_result' && Array.isArray((b as { content?: unknown }).content) ? data((b as unknown as { content: ContentBlock[] }).content) : []); + assert.deepEqual(compiled.messages.flatMap((x) => data(x.content)), ['BBBB'], 'A stripped, B live'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageCeiling 1 holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + } + + it("the memory strategy's credit replays nothing for a render that replaced none of the set's images", async () => { + // A count ceiling of 4 takes part in the set of five images, so a render + // that replaced some would be credited by replay; asked for a render that + // replaced none, the credit replays nothing and names nothing. + const original = console.warn; + console.warn = () => {}; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const s = strategy as unknown as { + stripReplayHeld: (...args: unknown[]) => unknown; + ruleCeilingHits: (store: unknown, stripped: unknown[], wanted: { images: boolean; bytes: boolean }) => { images: boolean; bytes: boolean }; + stripReplayAtLimits: (store: unknown) => { involved: { images: boolean; bytes: boolean } } | undefined; + }; + try { + for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const view = (manager as unknown as { messageStore: { createView(): unknown } }).messageStore.createView(); + assert.equal(s.stripReplayAtLimits(view)?.involved.images, true, 'premise: the count limit took part'); + let replays = 0; + const held = s.stripReplayHeld.bind(strategy); + s.stripReplayHeld = (...args: unknown[]) => { replays++; return held(...args); }; + assert.deepEqual(s.ruleCeilingHits(view, [], { images: true, bytes: true }), { images: false, bytes: false }); + assert.equal(replays, 0); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { const auto = await ceilingLines(memory({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); From f5c42161a6855277fad80df0ca7e7b3076a51e2c Mon Sep 17 00:00:00 2001 From: Cedric-2315 Date: Fri, 23 Oct 2026 03:14:51 -0400 Subject: [PATCH 25/67] fix(compression): a request for history gives it only what came before it, after a head-window reset too (#317) compressChunkHierarchical and executeMerge put the head window first, raw, as the summarizer's context ("the raw chronicle opening ... exactly where the original instance saw it"), and keep later events out. After resetHeadWindow the head window starts at the transition, so every request for history from before the reset opened with the new topic, which came after the history being summarized (#317). Recall pairs were a second route for later history. A compression request recalled every summary on the frontier, so a chunk compressed after later history had been summarized got those later summaries ahead of it. That happens to the old head's chunk, which a reset makes compressible late, and to a pinned or quarantined chunk released late. headAsOf(store, pos) gives the head in force when the message at pos was the newest, cut at pos and at the compression holds. From the live head's start on it is the live head, cut at pos, so a chunk inside a widened live head no longer gets the head's later messages. Before the live head's start it is the head starting at the latest topic transition message at or before pos (isTopicTransitionMessage, the test the restart uses), or at the opening, and it ends by the live head's walk (getHeadWindowEnd's body, now headWindowEndFrom). A chunk that starts at a transition, and the opening's own chunk, get no head. Compression requests and merges use it. So does requestCarriesToolBlocks, which the count of queued work uses to predict tick's tool deferral. It reads each chunk's own head, memoized by pins and head start. A compression request now recalls only summaries that start before its chunk, the rule a merge's prior recall set already used. A summary that starts before the chunk and reaches past it is still recalled, because the messages it covers before the chunk have no other form in the request. That is the one way later content still reaches a request. Coverage is unchanged. Hosts can remove stored messages while a summary over them, or a queued chunk holding them, lives on (agent-framework's Discord module removes them on a delete). startInStore gives the first of a set of messages still in the store. summaryStartInStore gives a summary's first leaf still in the store, found through its children, which is the set its coverage hides. A chunk starts at its first message still in the store, and a merge at the first of its sources' leaves. Both recall sets take summaries by that start, and the compression request's middle places each recall pair there. History with no message left gets no head and recalls nothing. Every compile selects and sends what it did before, and summaries already written stay as they are. An over-budget refusal's clearedBy follows tick's deferral, which reads each request's head. Prompt cache, measured with a stub summarizer at connectome-host's defaults scaled by 1/10: against main, the requests after a reset rewrite fewer tokens in every case measured. Some cases send more, mainly the two with a transition bigger than the head window, where main's head held none or part of it and this sends the opening. The pull request description has the table. test/compression-head-as-of.test.ts has 31 tests, and 27 of them fail at main fa6e0d6a. The four that pass there are a queued chunk whose first message is removed during the compression before it, and three cases of a summary whose first leaf was removed or isn't its sourceRange.first. main passes those because it recalls the whole frontier. Five existing tests change. Three in overbudget-diagnostics and one in l1-recall-raw-pins-frontier now pin the new behaviour. One in compression-transient-ladder reached a tool cycle only through the head, and now uses a chunk that holds the cycle, so it checks what it checked before. A comment in tool-prose-hoist changes too. Removing any one of 23 parts of the change fails at least one test. Co-authored-by: Claude Opus 5.5 --- .../317-compression-head-as-of.fixed.md | 9 + src/strategies/autobiographical.ts | 283 ++++++-- test/compression-head-as-of.test.ts | 645 ++++++++++++++++++ test/compression-transient-ladder.test.ts | 6 +- test/l1-recall-raw-pins-frontier.test.ts | 6 +- test/overbudget-diagnostics.test.ts | 53 +- test/tool-prose-hoist.test.ts | 5 +- 7 files changed, 904 insertions(+), 103 deletions(-) create mode 100644 changelog.d/317-compression-head-as-of.fixed.md create mode 100644 test/compression-head-as-of.test.ts diff --git a/changelog.d/317-compression-head-as-of.fixed.md b/changelog.d/317-compression-head-as-of.fixed.md new file mode 100644 index 00000000..aecd10e1 --- /dev/null +++ b/changelog.d/317-compression-head-as-of.fixed.md @@ -0,0 +1,9 @@ +- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request no longer gives the history it summarizes what came after it, ahead of it (#317). It came by two routes: + - **The head:** a request for history from before the reset gave the new topic's opening as its head. It now gives the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. + - **Recall pairs:** a compression request recalled every summary on the frontier, so a chunk compressed after later history had been summarized was given those later summaries too. That's the old head's chunk, which a reset makes compressible late, or a pinned or quarantined chunk let go late. It now recalls only summaries that start before it, the rule merges already used; the later summaries' messages stay out of it as before. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, the one way later content does, and only where a summary straddles the chunk. + - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. + - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. + - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. + - **Unchanged:** what every compile selects and sends, and every summary already written. What an over-budget refusal names as clearing it (`clearedBy`) follows tick's deferral, which reads the request's head. + - **Prompt cache:** a request for older history after a reset carries on the prefix it had before the reset, where it used to restart with the new topic, and restart again each time the live head grew. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index d272bfb2..c20a7631 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -9110,9 +9110,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ("Antra came to me to explore the transformation story // again…"), compounding across merges into runaway false // memories (the "68 initiations" incident). Chronological - // order is also the KV-stable order — the head never changes. - // (executeMerge always had head-first; this site was the odd - // one out.) + // order is also the KV-stable order: chunks of one topic after its + // head share that head, so their requests share its prefix. The + // head is the head as of the chunk (headAsOf): after a head-window + // reset, the head that history followed, the opening or its own + // topic's, never the later topic's. (executeMerge always had + // head-first; this site was the odd one out.) // 2. Prior summaries — narrativized as CM-asks / agent-recalls // pairs, in source order. The unmerged frontier of the // summary forest: any summary that has not yet been merged @@ -9130,10 +9133,25 @@ export class AutobiographicalStrategy implements ResettableStrategy { // future information into the model's KV state and corrupt the // as-of framing of memory formation. const allMessages = ctx.messageStore.getAll(); - const headStartIdx = this.getHeadWindowStartIndex(ctx.messageStore); - // Compression holds: provisional content never enters a prompt, not even - // as head context (a reset head can sit past the hold boundary). - const headEndIdx = Math.min(this.getHeadWindowEnd(ctx.messageStore), this.holdBoundary(ctx.messageStore)); + // Message IDs are opaque strings, not sortable sequence numbers. Chronicle's + // default IDs happen to be decimal text, where lexical order corrupts the + // timeline at the first width transition (`"103" < "15"`). Use the store's + // canonical order instead so recall pairs remain chronological for every ID + // representation and newly-created summaries extend rather than splice the + // provider-cache prefix. + const messageOrder = new Map( + allMessages.map((message, index) => [message.id, index]), + ); + const chunkFirstId = chunk.messages[0]?.id; + // The chunk's start is its first message still in the store: a host can + // remove a stored message (agent-framework's Discord module does, on a + // delete) while the chunk holding it waits in the queue. -1 only when + // none of its messages is left (startInStore). + const chunkStartIdx = this.startInStore(chunk.messages.map((m) => m.id), messageOrder); + // The head as of the chunk (headAsOf): after a head-window reset, history + // from before it is given the head it followed, not the new topic's, which + // came after it. Clipped at a hold there, as before. + const { start: headStartIdx, end: headEndIdx } = this.headAsOf(ctx.messageStore, chunkStartIdx); // ---- Prior recall set (the unmerged frontier) ---- // Computed BEFORE the head emission because the head loop needs the @@ -9146,15 +9164,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { // optimization that scales catastrophically: a 4000-message import // converged to ~500 L1s that never aged out, blowing the 200k // window around chunk 118. - // Message IDs are opaque strings, not sortable sequence numbers. Chronicle's - // default IDs happen to be decimal text, where lexical order corrupts the - // timeline at the first width transition (`"103" < "15"`). Use the store's - // canonical order instead so recall pairs remain chronological for every ID - // representation and newly-created summaries extend rather than splice the - // provider-cache prefix. - const messageOrder = new Map( - allMessages.map((message, index) => [message.id, index]), - ); // Pins in the canonical L1 prompt. The live view shows a message that a // pin keeps raw (forceRawPinPositions) raw, so this prompt does too. A // frontier summary that can't be shown as a recall pair (it covers such a @@ -9192,20 +9201,39 @@ export class AutobiographicalStrategy implements ResettableStrategy { // stay declared (reasoning_extraction guard). L1 only; merge/transition // paths do not consult this flag. const sourceOnly = this.config.compressionSourceOnly === true; - const chunkFirstId = chunk.messages[0]?.id; - const chunkStartIdx = chunkFirstId ? allMessages.findIndex((m) => m.id === chunkFirstId) : -1; // History before the chunk, laid out for the messages that pins keep raw. const layoutHistory = (forceRawMessageIds: ReadonlySet) => { // The recall frontier, the chunks pins open and the leaf coverage, as // compressionWorkQueued's count reads them too (recallCoverage). const { frontier, openedMessageIds, coveredMessageIds: priorSummaryMessageIds } = this.recallCoverage(forceRawMessageIds); + // Recall pairs only for history before the chunk: a summary whose + // source range starts before the chunk's first message, as a merge's + // prior recall set takes only what starts before its range (1a). A + // chunk compressed after later history already has summaries (the old + // head's chunk, which a head-window reset makes compressible after the + // chunks behind it; a pinned or quarantined chunk let go late) was + // given their recall pairs too, ahead of it: what came after it + // (#317). It now sees fewer pairs: only those that start before it. + // Coverage is the whole frontier's still, so a later summary's + // messages stay out of the raw middle (one-to-one). A summary that + // starts before the chunk and reaches past it (a sparse sweep + // artifact, issue #42) is still recalled, since the messages it covers + // before the chunk have no other form here; so its later part, what it + // says of history after the chunk, still reaches this request, the one + // way later content does, and only where a summary straddles the + // chunk. Where a summary starts is summaryStartInStore's rule, the one + // a merge's prior recall set uses: a summary whose first message a host + // removed starts at its first message still there, and one with none + // left isn't recalled. A chunk with no message left (chunkStartIdx -1) + // recalls none. + const summariesById = new Map(); + for (const s of this.summaries) summariesById.set(s.id, s); + const startOf = new Map(); + for (const s of frontier) startOf.set(s.id, this.summaryStartInStore(s, messageOrder, summariesById)); const priorSummaries = frontier - .sort((a, b) => { - const aOrder = messageOrder.get(a.sourceRange.first) ?? Number.MAX_SAFE_INTEGER; - const bOrder = messageOrder.get(b.sourceRange.first) ?? Number.MAX_SAFE_INTEGER; - return aOrder - bOrder || a.sourceRange.first.localeCompare(b.sourceRange.first); - }); + .filter((s) => { const at = startOf.get(s.id)!; return at >= 0 && at < chunkStartIdx; }) + .sort((a, b) => startOf.get(a.id)! - startOf.get(b.id)! || a.sourceRange.first.localeCompare(b.sourceRange.first)); // Raw messages before the chunk outside the head window, by position. // From the head window's end on, any message no summary represents yet @@ -9249,7 +9277,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { rawMiddle.push({ position: i, message: m }); } } - return { priorSummaries, priorSummaryMessageIds, rawMiddle, pinnedRaw, pinnedRawTokens }; + return { priorSummaries, startOf, priorSummaryMessageIds, rawMiddle, pinnedRaw, pinnedRawTokens }; }; // When the chunk is a portion of a substantially larger sharded message @@ -9275,7 +9303,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Pins count only before the chunk being compressed: this prompt is the // history the chunk follows. A summary after it (a chunk compressed out of - // order, after a head-window reset or an unpin) is recalled as on main. + // order, after a head-window reset or an unpin) isn't recalled at all + // (layoutHistory), and its messages stay covered. const forceRawMessageIds = new Set(); for (const i of this.forceRawPinPositions(allMessages)) { if (i < chunkStartIdx) forceRawMessageIds.add(allMessages[i].id); @@ -9285,7 +9314,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The canonical request's messages for one layout of history, sections 1 // to 6, shaped as they are sent. Admitting the pins prices this list. const assemble = ( - { priorSummaryMessageIds, rawMiddle }: ReturnType, + { startOf, priorSummaryMessageIds, rawMiddle }: ReturnType, keptSummaries: readonly SummaryEntry[], ) => { const llmMessages: Array<{ participant: string; content: ContentBlock[] }> = []; @@ -9304,7 +9333,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the chunk." For doc-heavy chronicles, "everything before" would // be hundreds of thousands of tokens; the recall pairs below // represent that intermediate content. The head is just the - // permanent prefix that the original instance always saw. + // prefix the original instance saw ahead of this history: the head + // window as of the chunk (headAsOf), which after a head-window reset + // is the opening, or the transition and opening of the chunk's topic. // // ONE-TO-ONE INVARIANT: chunk records (and the summaries over them) // are the persistent authority over how a message is represented. @@ -9329,13 +9360,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ---- 2. Prior recall pairs (budget-capped, above) and the raw middle ---- // One sequence by source position after the head window, which always - // comes first, so the summarizer never reads an older raw turn as coming - // after a later memory. Where the raw middle follows every recall pair, - // as it does when chunking runs contiguously, this is the order the - // prompt always had: recall pairs, then raw. + // comes first, so the summarizer never reads an older raw turn, or an + // older memory, as coming after a later memory. A recall pair sits at + // its summary's start as the recall filter read it (layoutHistory's + // startOf, summaryStartInStore): a summary whose first message a host + // removed sits at its first message still in the store, not last. + // Where the raw middle follows every recall pair, as it does when + // chunking runs contiguously, this is the order the prompt always had: + // recall pairs, then raw. const middle: Array<{ position: number; summary?: SummaryEntry; message?: StoredMessage }> = [ ...keptSummaries.map((summary) => ({ - position: messageOrder.get(summary.sourceRange.first) ?? Number.MAX_SAFE_INTEGER, + position: startOf.get(summary.id) ?? Number.MAX_SAFE_INTEGER, summary, })), ...rawMiddle, @@ -11820,9 +11855,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; for (const src of sources) collectLeaves(src); - // Find the start of the merge range in the message store. - const mergeFirstMsgId = sources[0].sourceRange.first; - const mergeStartIdx = allMessages.findIndex((m) => m.id === mergeFirstMsgId); + // The merge range's start in the store: its first leaf still there + // (startInStore), as a summary's start is read (summaryStartInStore). -1 + // only when none is: nothing in the store says when that history + // happened, so the merge gets no head (headAsOf) and recalls no summary + // (1a), as a compression chunk in that state does. + const positionOf = this.positionIndex(allMessages); + const mergeStartIdx = this.startInStore(sourceLeafIds, positionOf); // ---- 1a. PRIOR RECALL SET (chronologically before merge range) ---- // Computed BEFORE the head emission because the head loop needs the @@ -11836,20 +11875,22 @@ export class AutobiographicalStrategy implements ResettableStrategy { // (`!mergedInto`) lets a merged L1 drop out in favour of its L2/L3 // parent — the same rule used everywhere else and now in // `compressChunkHierarchical`. The cap below is the defense-in-depth. + const priorStart = new Map(); const priorSummariesAll = this.summaries // Skip empty-content summaries (see compressChunkHierarchical): an empty // text block in the merge recall header 400s and stalls compression. .filter((s) => !s.mergedInto && !!s.content && s.content.trim().length > 0) .filter((s) => { for (const lid of s.sourceIds) if (sourceLeafIds.has(lid)) return false; - const firstIdx = allMessages.findIndex((m) => m.id === s.sourceRange.first); - return firstIdx >= 0 && (mergeStartIdx < 0 || firstIdx < mergeStartIdx); + // Where it starts (summaryStartInStore): a summary whose first message + // a host removed starts at its first message still in the store, so + // it's recalled and its other messages, which its coverage holds out + // of the raw middle, keep a form here. One with none left isn't. + const firstIdx = this.summaryStartInStore(s, positionOf, summariesById); + priorStart.set(s.id, firstIdx); + return firstIdx >= 0 && firstIdx < mergeStartIdx; }) - .sort((a, b) => { - const ai = allMessages.findIndex((m) => m.id === a.sourceRange.first); - const bi = allMessages.findIndex((m) => m.id === b.sourceRange.first); - return ai - bi; - }); + .sort((a, b) => priorStart.get(a.id)! - priorStart.get(b.id)!); // Leaf coverage of every live summary — the rendering authority // (one-to-one invariant; see compressChunkHierarchical). Uses the @@ -11872,9 +11913,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // ---- 1b. HEAD WINDOW (raw, FIRST, ownership-capped) ---- // - // The head window is the foundational identity anchor — the actual - // opening of the chronicle. It establishes who is speaking to whom. - // Without it, when the merge target's content is heavily first-person + // The head window is the foundational identity anchor — the opening + // the merged history followed (headAsOf: the chronicle's opening, or + // after a head-window reset the opening of its topic). It establishes + // who is speaking to whom. Without it, when the merge target's content + // is heavily first-person // from someone other than the agent, the agent loses its first-person // grounding and drifts into the content author's voice. // @@ -11882,10 +11925,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { // wins over the token-derived head boundary. A head-range message // covered by a live summary renders via its recall pair; one inside // the merge tree renders in the TARGET expansion. Never raw here too. - const headStartIdx = this.getHeadWindowStartIndex(ctx.messageStore); - // Compression holds: provisional content never enters a prompt, not even - // as head context (a reset head can sit past the hold boundary). - const headEndIdx = Math.min(this.getHeadWindowEnd(ctx.messageStore), this.holdBoundary(ctx.messageStore)); + // The head as of the merge's first message (headAsOf), as for an L1: a + // merge over history from before a head-window reset is given the head + // that history followed. Clipped at a hold there, as before. + const { start: headStartIdx, end: headEndIdx } = this.headAsOf(ctx.messageStore, mergeStartIdx); let headCoveredSkipped = 0; for (let i = headStartIdx; !mergeSourceOnly && i < headEndIdx && i < allMessages.length; i++) { const m = allMessages[i]; @@ -12804,36 +12847,42 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the chunk's own messages, and, unless compressionSourceOnly, each message * the request carries raw ahead of the chunk, by the coverage tick's request * reads with the pins before the chunk (recallCoverage). From the lower of - * the head window's start and headEnd (headEnd clipped at a hold as tick - * clips it, and 0 with the head window off, headWindowTokens <= 0), that's - * every message the coverage leaves uncovered, up to the chunk or to - * headEnd, whichever is later. Before it, only what pins keep raw: the + * the start and end of the head that request gives (headAsOf, the head as + * of the chunk, its end clipped at a hold as tick clips it, and 0 with the + * head window off, headWindowTokens <= 0), that's every message the coverage + * leaves uncovered, up to the chunk or to that head's end, whichever is + * later. Before it, only what pins keep raw: the * messages of an L1 a pin opens, and a pinned message. Tick lays its * history out without the pins when the raw they add doesn't fit, so the * pins can only add raw here that tick leaves out; and tick strips unpaired * tool blocks first. So this can say yes where tick's request has none, * never the reverse. The test it returns takes one chunk. Chunks with the - * same pins before them read the same coverage, so each coverage is read - * once, for the first message its request carries raw with a tool block, - * and a chunk's answer is whether that message comes before the later of - * the chunk's start and headEnd. + * same pins before them, and the same head start, read the same coverage + * from the same place, so each is read once, for the first message its + * request carries raw with a tool block, and a chunk's answer is whether + * that message comes before the later of the chunk's start and its head's + * end. */ private requestCarriesToolBlocks(store: MessageStoreView): (chunk: Chunk) => boolean { const all = store.getAll(); const sourceOnly = this.config.compressionSourceOnly === true; - const headEnd = Math.min(this.getHeadWindowEnd(store), this.holdBoundary(store)); - const from = Math.min(this.getHeadWindowStartIndex(store), headEnd); const position = new Map(all.map((message, i) => [message.id, i])); const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all)].sort((a, b) => a - b); - // For each count of pins before a chunk, the index of the first message - // that request carries raw with a tool block, or all.length for none. - const firstToolBlock = new Map(); + // For each count of pins before a chunk and the start its head's raw + // reading takes, the index of the first message that request carries raw + // with a tool block, or all.length for none. + const firstToolBlock = new Map(); return (chunk) => { if (chunk.messages.some((m) => hasToolBlocks(m.content))) return true; if (sourceOnly) return false; - const start = position.get(chunk.messages[0]?.id ?? '') ?? -1; + // The chunk's start as tick's request takes it (startInStore). + const start = this.startInStore(chunk.messages.map((m) => m.id), position); + const head = this.headAsOf(store, start); + const headEnd = head.end; + const from = Math.min(head.start, headEnd); const pinned = pins.filter((i) => i < start); - let first = firstToolBlock.get(pinned.length); + const key = `${pinned.length}:${from}`; + let first = firstToolBlock.get(key); if (first === undefined) { const pinnedIds = new Set(pinned.map((i) => all[i].id)); const seen = this.recallCoverage(pinnedIds); @@ -12845,7 +12894,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { && (i >= from || seen.openedMessageIds.has(message.id) || pinnedIds.has(message.id)); const found = all.findIndex((message, i) => carried(message, i) && hasToolBlocks(message.content)); first = found === -1 ? all.length : found; - firstToolBlock.set(pinned.length, first); + firstToolBlock.set(key, first); } return first < Math.max(headEnd, start); }; @@ -18083,6 +18132,98 @@ export class AutobiographicalStrategy implements ResettableStrategy { return result; } + /** + * Where history starts in the store: the position of the first of `ids` + * still there (store order), or -1 when none is. A host can remove stored + * messages (agent-framework's Discord module does, on a delete or a history + * sync) while a summary over them, or a queued chunk holding them, lives + * on; -1 means nothing in the store says when that history happened. + */ + protected startInStore(ids: Iterable, position: ReadonlyMap): number { + let start = -1; + for (const id of ids) { + const at = position.get(id); + if (at !== undefined && (start < 0 || at < start)) start = at; + } + return start; + } + + /** + * Where a summary starts in the store: its first leaf message still there + * (expandSummaryToLeafMessageIds, startInStore), whether or not that is + * sourceRange.first; -1 when none is. The one rule for both recall sets, a + * compression request's and a merge's (1a), and for where a recall pair + * sits among them: a summary starting before the history being summarized + * is recalled, at its start. A summary with no message left isn't: its + * place is unknown, and its coverage holds no message out of the request. + */ + protected summaryStartInStore( + summary: SummaryEntry, + position: ReadonlyMap, + summariesById: ReadonlyMap, + ): number { + const leaves = new Set(); + this.expandSummaryToLeafMessageIds(summary, summariesById, leaves); + return this.startInStore(leaves, position); + } + + /** + * The head window a compression or merge request gives as context for the + * history it summarizes from `pos` on: the head in force when the message at + * `pos` was the newest, cut at `pos` and at the compression holds + * (holdBoundary). From the live head's start on, that is the live head, + * cut at `pos`, so a chunk inside the live head window (one a widened + * window grew over) no longer gets the head's messages after its start. + * A `pos` that's unknown (-1: none of the history's messages is left in + * the store, startInStore) gets no head: nothing in the store says when + * that history happened, and the live head may have come after it. Before + * the live head's start, it is the head starting at the latest topic + * transition message at or before `pos` (isTopicTransitionMessage, the test + * that restores the live head's start at a restart), or at the opening when + * there is none, ending by the walk the live head takes from its start + * (headWindowEndFrom). So history from before a head-window reset gets the + * head it followed: the opening, or the transition and opening of the topic + * it was in. There, a chunk that starts at a transition, or at the + * store's first message (the opening's own chunk, which a reset releases), + * gets none: nothing came before it. + * + * After a reset, these requests gave the new topic's opening ahead of the + * older history they summarize (#317), where section 1 of + * compressChunkHierarchical places the chronicle's opening "exactly where + * the original instance saw it" and keeps later events out. + * + * Three things it is, by choice: + * - Before the live head's start it reads the store alone, as a restart + * does. A strategy-level resetHeadWindow to a message that isn't a + * transition isn't in the store: history before that message gets the + * latest transition before it, or the opening, as after a restart. History + * from that message on gets the live head, which a restart moves to the + * last transition in the store. So a restart gives the same heads while + * the live head starts at the last transition, as it does when every + * transition is written by ContextManager.resetHeadWindow, which resets + * the head to it. + * - A past head is priced with today's headWindowTokens and calibration, as + * the live head is, not with those in force when it was the head. + * - Before the live head's start, any Context Manager message whose text + * starts with "[Topic Transition]" starts a head, as the restart reads + * one, whether or not resetHeadWindow wrote it. + */ + protected headAsOf(store: MessageStoreView, pos: number): { start: number; end: number } { + const liveStart = this.getHeadWindowStartIndex(store); + const hold = this.holdBoundary(store); + if (pos < 0) return { start: liveStart, end: liveStart }; + if (pos >= liveStart) return { start: liveStart, end: Math.min(this.getHeadWindowEnd(store), hold, pos) }; + const messages = store.getAll(); + let start = 0; + for (let i = pos; i > 0; i--) { + if (this.isTopicTransitionMessage(messages[i])) { + start = i; + break; + } + } + return { start, end: Math.min(this.headWindowEndFrom(store, messages, start), hold, pos) }; + } + /** * Index of the first message AFTER the head window. * Messages [headStart, headEnd) are sent raw, never summarized. @@ -18099,10 +18240,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { * material (issue #122). See anchorHeadToCoverage. */ protected getHeadWindowEnd(store: MessageStoreView): number { - if (this.config.headWindowTokens <= 0) return 0; + return this.headWindowEndFrom(store, store.getAll(), this.getHeadWindowStartIndex(store)); + } - const messages = store.getAll(); - const startIdx = this.getHeadWindowStartIndex(store); + /** + * Where a head window starting at `startIdx` ends, by the walk the live head + * window takes (getHeadWindowEnd): the live head's start, or a past head's + * (headAsOf). + */ + protected headWindowEndFrom(store: MessageStoreView, messages: StoredMessage[], startIdx: number): number { + if (this.config.headWindowTokens <= 0) return 0; let tokens = 0; for (let i = startIdx; i < messages.length; i++) { diff --git a/test/compression-head-as-of.test.ts b/test/compression-head-as-of.test.ts new file mode 100644 index 00000000..d5a56d86 --- /dev/null +++ b/test/compression-head-as-of.test.ts @@ -0,0 +1,645 @@ +/** + * The head a compression or merge request gives as context is the head in force when the history it summarizes + * happened: before the live head's start, the head starting at the latest topic transition at or before the + * request's first message, or the opening; from the live head's start on, the live head; cut at that message (#317). After resetHeadWindow, a request for history from before the reset gave the + * new topic's opening ahead of it, where the code places the chronicle's opening and keeps later events out. + */ + +import { describe, it, after } from 'node:test'; +import assert from 'node:assert'; +import { rmSync, existsSync } from 'node:fs'; +import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; +import type { Chunk } from '../src/strategies/autobiographical.js'; +import type { SummaryEntry } from '../src/types/index.js'; +import type { StrategyContext } from '../src/types/strategy.js'; +import type { ContentBlock } from '@animalabs/membrane'; + +const ROOT = './test-compression-head-as-of'; +let counter = 0; +const freshPath = (): string => `${ROOT}-${counter++}`; + +class ProbeStrategy extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { + this.pushSummary(entry); + } + runCompress(chunk: Chunk, ctx: StrategyContext): Promise { + return this.compressChunkHierarchical(chunk, ctx); + } + runMerge(targetLevel: number, sourceIds: string[], ctx: StrategyContext): Promise { + return this.executeMerge(targetLevel as never, sourceIds, ctx); + } + /** The head a request gives for history from `pos` on. */ + asOf(ctx: StrategyContext, pos: number): { start: number; end: number } { + return (this as unknown as { headAsOf(store: unknown, pos: number): { start: number; end: number } }) + .headAsOf(ctx.messageStore, pos); + } + /** Whether the count of queued work reads tick's request for `chunk` as carrying a tool block. */ + carriesToolBlocks(ctx: StrategyContext, chunk: Chunk): boolean { + return this.toolBlockReader(ctx)(chunk); + } + /** One reader of the count's, as one count holds it across the queue (its answers memoized). */ + toolBlockReader(ctx: StrategyContext): (chunk: Chunk) => boolean { + return (this as unknown as { requestCarriesToolBlocks(store: unknown): (c: Chunk) => boolean }) + .requestCarriesToolBlocks(ctx.messageStore); + } + /** Sets the head window to `count` of this store's equal-sized messages, with half a message to spare. */ + headOfMessages(ctx: StrategyContext, count: number): void { + const messages = ctx.messageStore.getAll(); + let sum = 0; + for (let i = 0; i < count; i++) sum += ctx.messageStore.estimateTokens(messages[i]!); + this.config.headWindowTokens = sum + Math.floor(ctx.messageStore.estimateTokens(messages[count]!) / 2); + } + headWindowTokens(): number { + return this.config.headWindowTokens; + } +} + +/** Lets every merge through the topology check, so a merge whose source messages a host removed reaches its + * request and executeMerge's own start rule is what these tests pin. The real check may refuse such a merge (a + * merge whose first source message was removed returned false in Effie-2289's P5): these tests don't show that + * production reaches it. */ +class PlacingTopologyStrategy extends ProbeStrategy { + protected override mergeTopologyProblem(): string | null { + return null; + } +} + +const context = (manager: ContextManager): StrategyContext => + (manager as unknown as { createStrategyContext(): StrategyContext }).createStrategyContext(); + +const text = (t: string): ContentBlock[] => [{ type: 'text', text: t }]; +/** Equal-sized messages, so a head of `count` messages is the same count whichever message it starts at. */ +const body = (marker: string): string => `${marker} ${'word '.repeat(18)}`.slice(0, 110); + +function countIn(req: unknown, needle: string): number { + const blob = JSON.stringify(req); + let n = 0; + for (let at = blob.indexOf(needle); at >= 0; at = blob.indexOf(needle, at + needle.length)) n++; + return n; +} + +function summary(id: string, sourceIds: string[]): SummaryEntry { + return { + id, level: 1, content: `authored ${id}`, tokens: 20, sourceLevel: 0, sourceIds, + sourceRange: { first: sourceIds[0]!, last: sourceIds[sourceIds.length - 1]! }, + created: Number(id.replace(/\D/g, '')) || 1, + }; +} + +const chunkOf = (ctx: StrategyContext, from: number, to: number): Chunk => ({ + index: 900 + from, startIndex: from, endIndex: to, messages: ctx.messageStore.getAll().slice(from, to), tokens: 100, compressed: false, +}); + +/** + * A store with the opening OPEN0..OPEN9, a topic transition T1 and ONE0..ONE9, and a second transition T2 and + * TWO0..TWO5, every message the same size, the head window three messages. `resets` says how many of the two + * transitions the strategy has been reset to. + */ +async function build(resets: 0 | 1 | 2, opts: { path?: string; openingTool?: boolean; placing?: boolean } = {}) { + const captured: unknown[] = []; + const membrane = { + complete: async (req: unknown) => { + captured.push(structuredClone(req)); + return { stopReason: 'end_turn', content: [{ type: 'text', text: 'a summary' }] }; + }, + }; + const strategy = new (opts.placing ? PlacingTopologyStrategy : ProbeStrategy)({ + compressionModel: 'test-compression-model', headWindowTokens: 1, recentWindowTokens: 0, + targetChunkTokens: 1_000_000, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + }); + const path = opts.path ?? freshPath(); + const manager = await ContextManager.open({ path, strategy, membrane: membrane as never }); + const at: Record = {}; + let n = 0; + const add = (name: string, participant: string, content: ContentBlock[]): string => { + at[name] = n++; + return manager.addMessage(participant, content); + }; + for (let i = 0; i < 10; i++) { + if (opts.openingTool && i === 1) { + add('OPEN1', 'assistant', [{ type: 'text', text: body('OPEN1').slice(0, 60) }, { type: 'tool_use', id: 'call_1', name: 'fs:read', input: { p: 'a' } } as ContentBlock]); + } else if (opts.openingTool && i === 2) { + add('OPEN2', 'user', [{ type: 'tool_result', toolUseId: 'call_1', content: body('OPEN2').slice(0, 60) } as ContentBlock]); + } else { + add(`OPEN${i}`, i % 2 ? 'assistant' : 'user', text(body(`OPEN${i}`))); + } + } + const t1 = add('T1', 'Context Manager', text(`[Topic Transition]\n\n${body('TOPIC_ONE')}`.slice(0, 110))); + for (let i = 0; i < 10; i++) add(`ONE${i}`, i % 2 ? 'assistant' : 'user', text(body(`ONE${i}`))); + const t2 = add('T2', 'Context Manager', text(`[Topic Transition]\n\n${body('TOPIC_TWO')}`.slice(0, 110))); + for (let i = 0; i < 6; i++) add(`TWO${i}`, i % 2 ? 'assistant' : 'user', text(body(`TWO${i}`))); + if (resets >= 1) strategy.resetHeadWindow(t1); + if (resets >= 2) strategy.resetHeadWindow(t2); + const ctx = context(manager); + strategy.headOfMessages(ctx, 3); + return { manager, strategy, ctx, captured, at, path }; +} + +describe('the head a compression or merge request gives is the head as of the history it summarizes (#317)', () => { + after(() => { + for (let i = 0; i < counter; i++) if (existsSync(`${ROOT}-${i}`)) rmSync(`${ROOT}-${i}`, { recursive: true, force: true }); + }); + + it("a chunk from before a reset is compressed with the opening, not the new topic's head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + await strategy.runCompress(chunkOf(ctx, at.OPEN6!, at.OPEN8!), ctx); + assert.equal(captured.length, 1); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m} is the head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head's transition isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0, "nor the live head's messages"); + assert.equal(countIn(req, 'TOPIC_ONE'), 0, 'nor a later topic'); + assert.ok(JSON.stringify(req).indexOf('OPEN0 ') < JSON.stringify(req).indexOf('OPEN6 '), 'the head comes before the chunk'); + await manager.close(); + }); + + it('a chunk between two resets is compressed with the head of the topic it was in, cut at the chunk', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.ONE5!), { start: at.T1!, end: at.T1! + 3 }); + await strategy.runCompress(chunkOf(ctx, at.ONE5!, at.ONE7!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_ONE'), 1, "its topic's transition is the head"); + for (const m of ['ONE0', 'ONE1']) assert.equal(countIn(req, `${m} `), 1, `${m}, in that head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, 'not the later topic'); + assert.equal(countIn(req, 'TWO0'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor the opening, which comes before that head'); + await manager.close(); + }); + + it('a chunk that starts at a transition is compressed with no head: nothing came before it in that topic', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + await strategy.runCompress(chunkOf(ctx, at.T1!, at.ONE1!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_ONE'), 1, 'the transition is sent once, in the chunk'); + assert.equal(countIn(req, 'TOPIC_TWO'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0); + assert.deepEqual(strategy.asOf(ctx, at.T1!), { start: at.T1!, end: at.T1! }); + await manager.close(); + }); + + it('a merge across a reset gives the head as of its first message', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const a = summary('L1-501', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + const b = summary('L1-502', [ids[at.T1!]!, ids[at.ONE0!]!]); + strategy.seed(a); + strategy.seed(b); + await strategy.runMerge(2, [a.id, b.id], ctx); + assert.equal(captured.length, 1); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m} is the head, raw once`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head's transition isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + await manager.close(); + }); + + it("a chunk after the live head's start keeps the live head, as before", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.TWO4!), { start: at.T2!, end: at.T2! + 3 }); + await strategy.runCompress(chunkOf(ctx, at.TWO4!, at.TWO6!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 1, "the live head's transition is the head"); + for (const m of ['TWO0', 'TWO1']) assert.equal(countIn(req, `${m} `), 1); + assert.equal(countIn(req, 'TOPIC_ONE'), 0); + await manager.close(); + }); + + it("a strategy reset to a message that isn't a transition isn't in the store: history before it gets the latest transition or the opening, as a restart reads it", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + strategy.resetHeadWindow(ids[at.OPEN6!]!); + assert.deepEqual(strategy.asOf(ctx, at.OPEN3!), { start: 0, end: 3 }, 'the opening, before the reset message'); + assert.deepEqual(strategy.asOf(ctx, at.OPEN7!), { start: at.OPEN6!, end: at.OPEN7! }, 'from the live start on, the live head, cut at the chunk'); + await strategy.runCompress(chunkOf(ctx, at.OPEN3!, at.OPEN5!), ctx); + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(captured[0], `${m} `), 1, `${m} is the head, raw once`); + await manager.close(); + }); + + it("a past head is priced with today's head window, as the live head is", async () => { + const { manager, strategy, ctx, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: 3 }); + strategy.headOfMessages(ctx, 1); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: 1 }, "today's window decides, not the one in force then"); + strategy.headOfMessages(ctx, 5); + assert.deepEqual(strategy.asOf(ctx, at.ONE7!), { start: at.T1!, end: at.T1! + 5 }); + await manager.close(); + }); + + it('before the live head\'s start, a Context Manager message whose text starts with "[Topic Transition]" starts a past head, as the restart reads one; a user\'s doesn\'t', async () => { + const { manager, strategy, ctx, at } = await build(0); + const quoted = ctx.messageStore.getAll().length; + manager.addMessage('Context Manager', text('[Topic Transition] noted by the host')); + manager.addMessage('user', text(body('LATE0'))); + const userQuoted = quoted + 2; + manager.addMessage('user', text('[Topic Transition] quoted by someone')); + for (let i = 1; i < 4; i++) manager.addMessage('user', text(body(`LATE${i}`))); + const t3 = manager.addMessage('Context Manager', text('[Topic Transition]\n\nA third topic.')); + strategy.resetHeadWindow(t3); + assert.ok(at.T2! < quoted); + assert.equal(strategy.asOf(ctx, userQuoted + 2).start, quoted, "the host's message starts a head; the user's quoting doesn't"); + await manager.close(); + }); + + it('a restart reads the same heads while the live head starts at the last transition: past heads come from the store alone', async () => { + const path = freshPath(); + const first = await build(2, { path }); + const positions = [first.at.OPEN6!, first.at.ONE5!, first.at.T1!, first.at.TWO4!]; + const before = positions.map((p) => first.strategy.asOf(first.ctx, p)); + await first.manager.close(); + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: first.strategy.headWindowTokens(), recentWindowTokens: 0, + targetChunkTokens: 1_000_000, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + }); + const manager = await ContextManager.open({ path, strategy, membrane: { complete: async () => ({ stopReason: 'end_turn', content: [] }) } as never }); + const ctx = context(manager); + assert.deepEqual(positions.map((p) => strategy.asOf(ctx, p)), before); + await manager.close(); + }); + + it("a chunk whose first message has left the store starts at its first message still there, and gets that message's head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + await strategy.runCompress({ index: 950, startIndex: at.OPEN6!, endIndex: at.OPEN8!, messages: [gone, kept], tokens: 100, compressed: false }, ctx); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + await manager.close(); + }); + + it('a chunk with no message left in the store gets no head and recalls nothing: nothing in the store says when its history happened', async () => { + const { manager, strategy, ctx, captured, at } = await build(2); + assert.deepEqual(strategy.asOf(ctx, -1), { start: at.T2!, end: at.T2! }); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + const ids = ctx.messageStore.getAll().map((m) => m.id); + const earlier = summary('L1-841', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + strategy.seed(earlier); + await strategy.runCompress({ index: 950, startIndex: at.OPEN6!, endIndex: at.OPEN7!, messages: [gone], tokens: 100, compressed: false }, ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'TWO0'), 0); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor any other head'); + assert.equal(countIn(req, `authored ${earlier.id}`), 0, 'and nothing is recalled: its place is unknown'); + await manager.close(); + }); + + it('a chunk whose first message a host removes while it waits in the queue still recalls the summaries before it', async () => { + // Twelve equal messages in chunks of four; the first message of the second chunk is removed during the first + // chunk's compression, as a delete arriving then would (agent-framework's Discord module removes stored rows). + let manager!: ContextManager; + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: 1, recentWindowTokens: 0, l1HoldbackChunks: 0, + targetChunkTokens: 60, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + } as never); + const captured: unknown[] = []; + const ids: string[] = []; + const membrane = { + complete: async (req: unknown) => { + captured.push(structuredClone(req)); + if (captured.length === 1) manager.removeMessage(ids[4]! as never); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `MEMORY_${captured.length}` }] }; + }, + }; + manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never }); + for (let i = 0; i < 12; i++) ids.push(manager.addMessage(i % 2 ? 'assistant' : 'user', text(body(`M${i}`)))); + for (let i = 0; i < 20 && captured.length < 2; i++) await manager.tick(); + assert.ok(captured.length >= 2, 'premise: the second chunk was compressed'); + const req = captured[1]!; + assert.equal(countIn(req, 'M4 '), 1, 'premise: the queued chunk still holds the removed message'); + assert.equal(countIn(req, 'MEMORY_1'), 1, "the first chunk's summary, before it, is recalled"); + await manager.close(); + }); + + it("a summary whose first message a host removed starts at its first message still there: it's recalled, so its other messages keep a form", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const s = summary('L1-801', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + strategy.seed(s); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunk, ctx); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${s.id}`), 1, 'the summary is recalled'); + assert.equal(countIn(req, 'OPEN4 '), 0, 'its messages are represented by it, not raw'); + await manager.close(); + }); + + it('a summary whose first message a host removed is recalled in its place, before a later one, so older history never follows a later memory', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN7!, at.OPEN9!); + const older = summary('L1-851', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const newer = summary('L1-852', [ids[at.OPEN5!]!, ids[at.OPEN6!]!]); + strategy.seed(older); + strategy.seed(newer); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunk, ctx); + const blob = JSON.stringify(captured[0]!); + const o = blob.indexOf(`authored ${older.id}`); + const n = blob.indexOf(`authored ${newer.id}`); + assert.ok(o >= 0 && n >= 0, 'premise: both are recalled'); + assert.ok(o < n, 'the older summary comes first'); + await manager.close(); + }); + + it("a summary with no message left in the store isn't recalled: its place is unknown", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const s = summary('L1-802', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + strategy.seed(s); + manager.removeMessage(ids[at.OPEN3!]! as never); + manager.removeMessage(ids[at.OPEN4!]! as never); + await strategy.runCompress(chunk, ctx); + assert.equal(countIn(captured[0]!, `authored ${s.id}`), 0); + await manager.close(); + }); + + it("a merge recalls a summary before it whose first message a host removed, so that summary's other messages keep a form", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const a = summary('L1-811', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + const b = summary('L1-812', [ids[at.OPEN6!]!, ids[at.OPEN7!]!]); + const c = summary('L1-813', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const x of [a, b, c]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN3!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${a.id}`), 1, 'the summary before the merge is recalled'); + assert.equal(countIn(req, 'OPEN4 '), 0, 'its messages are represented by it, not raw'); + await manager.close(); + }); + + it("a merge whose first source message a host removed starts at its first message still there: that message's head, and recall only of what starts before it", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-821', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-822', [ids[at.OPEN6!]!, ids[at.OPEN7!]!]); + const c = summary('L1-823', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + const after = summary('L1-824', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, `authored ${before.id}`), 1, 'the summary before it is recalled'); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'the summary after it is not'); + await manager.close(); + }); + + it('a merge whose sources have no message left in the store gets no head and recalls nothing, as a chunk in that state', async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-831', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-832', [ids[at.OPEN6!]!]); + const c = summary('L1-833', [ids[at.OPEN7!]!]); + const after = summary('L1-834', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + manager.removeMessage(ids[at.OPEN7!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, 'OPEN0 '), 0, 'nor any other head'); + assert.equal(countIn(req, `authored ${before.id}`), 0); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'nothing is recalled: its place is unknown'); + await manager.close(); + }); + + it("a merge whose first source has no message left starts at its next source's first message still there", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-861', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const b = summary('L1-862', [ids[at.OPEN6!]!]); + const c = summary('L1-863', [ids[at.OPEN7!]!, ids[at.OPEN8!]!]); + const after = summary('L1-864', [ids[at.ONE3!]!, ids[at.ONE4!]!]); + for (const x of [before, b, c, after]) strategy.seed(x); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const req = captured[0]!; + for (const m of ['OPEN0', 'OPEN1', 'OPEN2']) assert.equal(countIn(req, `${m} `), 1, `${m}: the opening is the head as of OPEN7`); + assert.equal(countIn(req, 'TOPIC_TWO'), 0, "the live head isn't sent"); + assert.equal(countIn(req, `authored ${before.id}`), 1, 'the summary before it is recalled'); + assert.equal(countIn(req, `authored ${after.id}`), 0, 'the summary after it is not'); + await manager.close(); + }); + + it('a merge recalls the summaries before it in the order they start in the store, and none with no message left', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const older = summary('L1-871', [ids[at.OPEN3!]!]); + const newer = summary('L1-872', [ids[at.OPEN4!]!, ids[at.OPEN5!]!]); + const gone = summary('L1-873', [ids[at.OPEN6!]!]); + const b = summary('L1-874', [ids[at.OPEN7!]!]); + const c = summary('L1-875', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + // Seeded newest first, so the order the request gives comes from the sort by start, not from the seeding. + for (const x of [c, b, gone, newer, older]) strategy.seed(x); + // The newer summary's first message is removed: it starts at OPEN5, its first leaf still in the store, after + // the older one's OPEN3. (main's 1a, by sourceRange.first, left it out; ordering by sourceRange.first would put + // it first, at -1.) + manager.removeMessage(ids[at.OPEN4!]! as never); + manager.removeMessage(ids[at.OPEN6!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + const blob = JSON.stringify(captured[0]!); + const o = blob.indexOf(`authored ${older.id}`); + const n = blob.indexOf(`authored ${newer.id}`); + assert.ok(o >= 0 && n >= 0, 'premise: both are recalled'); + assert.ok(o < n, 'the older comes first'); + assert.equal(blob.indexOf(`authored ${gone.id}`), -1, 'a summary with no message left is not recalled'); + await manager.close(); + }); + + it('a merge recalls a higher-level summary before it whose first leaf a host removed, its start found through its L1s', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const x = { ...summary('L1-901', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-903' }; + const y = { ...summary('L1-902', [ids[at.OPEN5!]!]), mergedInto: 'L2-903' }; + const z: SummaryEntry = { + id: 'L2-903', level: 2, content: 'authored L2-903', tokens: 20, sourceLevel: 1, sourceIds: [x.id, y.id], + sourceRange: { first: ids[at.OPEN3!]!, last: ids[at.OPEN5!]! }, created: 903, + }; + const b = summary('L1-904', [ids[at.OPEN7!]!]); + const c = summary('L1-905', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const e of [x, y, z, b, c]) strategy.seed(e); + manager.removeMessage(ids[at.OPEN3!]! as never); + assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); + assert.equal(countIn(captured[0]!, 'authored L2-903'), 1, 'the L2 is recalled'); + await manager.close(); + }); + + it("a merge starts at the first of all its sources' leaves still in the store, whatever order its sources are listed in", async () => { + const { manager, strategy, ctx, captured, at } = await build(0, { placing: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const between = summary('L1-911', [ids[at.OPEN7!]!]); + const b = summary('L1-912', [ids[at.OPEN6!]!]); + const c = summary('L1-913', [ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + for (const e of [between, b, c]) strategy.seed(e); + // Listed later source first: by its first source alone the merge would start at OPEN8, after the summary between. + assert.equal(await strategy.runMerge(2, [c.id, b.id], ctx), true, 'premise: the merge runs'); + assert.equal(countIn(captured[0]!, `authored ${between.id}`), 0, 'the summary after its first leaf, OPEN6, is not recalled'); + await manager.close(); + }); + + it('a higher-level summary whose first leaf a host removed starts at its first leaf still there', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const x = { ...summary('L1-881', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-883' }; + const y = { ...summary('L1-882', [ids[at.OPEN5!]!]), mergedInto: 'L2-883' }; + const z: SummaryEntry = { + id: 'L2-883', level: 2, content: 'authored L2-883', tokens: 20, sourceLevel: 1, sourceIds: [x.id, y.id], + sourceRange: { first: ids[at.OPEN3!]!, last: ids[at.OPEN5!]! }, created: 883, + }; + for (const e of [x, y, z]) strategy.seed(e); + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunkOf(ctx, at.OPEN7!, at.OPEN9!), ctx); + assert.equal(countIn(captured[0]!, 'authored L2-883'), 1, 'the L2 is recalled, its start found through its L1s'); + await manager.close(); + }); + + it("a summary starts at its first leaf in the store, whatever its sourceRange.first says", async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // An L2 whose sources came in the order [later, earlier], so its sourceRange.first names a leaf after the + // chunk while two of its leaves come before it. (An L1's range is its sourceIds' ends: pushSummary sets it.) + const earlier = { ...summary('L1-891', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]), mergedInto: 'L2-893' }; + const later = { ...summary('L1-892', [ids[at.OPEN9!]!]), mergedInto: 'L2-893' }; + const z: SummaryEntry = { + id: 'L2-893', level: 2, content: 'authored L2-893', tokens: 20, sourceLevel: 1, sourceIds: [later.id, earlier.id], + sourceRange: { first: ids[at.OPEN9!]!, last: ids[at.OPEN4!]! }, created: 893, + }; + for (const e of [earlier, later, z]) strategy.seed(e); + await strategy.runCompress(chunkOf(ctx, at.OPEN6!, at.OPEN8!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, 'authored L2-893'), 1, 'it is recalled, so OPEN3 and OPEN4, which it covers, keep a form'); + assert.equal(countIn(req, 'OPEN3 '), 0); + await manager.close(); + }); + + it('a compression hold inside a past head cuts it at the hold', async () => { + const { manager, strategy, ctx, at } = await build(2); + const ids = ctx.messageStore.getAll().map((m) => m.id); + manager.holdCompression([ids[at.OPEN1!]!]); + assert.deepEqual(strategy.asOf(ctx, at.OPEN6!), { start: 0, end: at.OPEN1! }); + await manager.close(); + }); + + it('a chunk compressed after the history behind it was summarized recalls only the summaries before it: the old head after a reset', async () => { + const { manager, strategy, ctx, captured, at } = await build(1); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // The history after the opening's head was summarized while that head was the head; the reset released the + // head, so its chunk is compressed last. + const later = [summary('L1-601', [ids[at.OPEN3!]!, ids[at.OPEN4!]!, ids[at.OPEN5!]!]), summary('L1-602', [ids[at.OPEN6!]!, ids[at.OPEN7!]!, ids[at.OPEN8!]!, ids[at.OPEN9!]!])]; + for (const s of later) strategy.seed(s); + await strategy.runCompress(chunkOf(ctx, at.OPEN0!, at.OPEN3!), ctx); + const req = captured[0]!; + for (const s of later) { + assert.equal(countIn(req, `Recall memory ${s.id}.`), 0, `${s.id}, after the chunk, isn't recalled`); + assert.equal(countIn(req, `authored ${s.id}`), 0); + } + for (const m of ['OPEN3', 'OPEN6', 'OPEN9']) assert.equal(countIn(req, `${m} `), 0, `${m} stays covered: not raw either`); + assert.equal(countIn(req, 'TOPIC_ONE'), 0, "and the live head isn't the head"); + await manager.close(); + }); + + it('a chunk compressed after the history behind it was summarized recalls only the summaries before it: a chunk let go late', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const before = summary('L1-701', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const after = summary('L1-702', [ids[at.OPEN7!]!, ids[at.OPEN8!]!, ids[at.OPEN9!]!]); + strategy.seed(before); + strategy.seed(after); + await strategy.runCompress(chunkOf(ctx, at.OPEN5!, at.OPEN7!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, `Recall memory ${before.id}.`), 1, 'the summary before the chunk is recalled'); + assert.equal(countIn(req, `Recall memory ${after.id}.`), 0, 'the one after it is not'); + assert.equal(countIn(req, `authored ${after.id}`), 0); + assert.equal(countIn(req, 'OPEN8 '), 0, 'and its messages stay covered'); + await manager.close(); + }); + + it("one count's reader gives each chunk its own head's answer, asked in either order, and leaves out a tool cycle before that head", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { openingTool: true }); + const opening = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + const topicOne = chunkOf(ctx, at.ONE5!, at.ONE7!); + // The opening's head carries the cycle; topic one's head starts after it, so the cycle is left out. + const forward = strategy.toolBlockReader(ctx); + assert.deepEqual([forward(opening), forward(topicOne)], [true, false]); + const backward = strategy.toolBlockReader(ctx); + assert.deepEqual([backward(topicOne), backward(opening)], [false, true]); + // Tick agrees for the chunk whose head leaves the cycle out: it asks the model. + await strategy.runCompress(topicOne, ctx); + assert.equal(captured.length, 1, 'tick asked the model, the cycle being before its head'); + await manager.close(); + }); + + it("the count reads a chunk whose first message has left the store from its first message still there, as tick does", async () => { + const { manager, strategy, ctx, at } = await build(2, { openingTool: true }); + const kept = ctx.messageStore.getAll()[at.OPEN7!]!; + const gone = { ...kept, id: 'not-in-the-store', content: text(body('GONE')) }; + const chunk: Chunk = { index: 960, startIndex: at.OPEN6!, endIndex: at.OPEN8!, messages: [gone, kept], tokens: 100, compressed: false }; + // As of OPEN7 the head is the opening, which carries the cycle raw, so tick defers this chunk before the tools. + assert.equal(strategy.carriesToolBlocks(ctx, chunk), true); + await manager.close(); + }); + + it('with merges on, through two resets, no compression request carries anything from at or after its chunk', async () => { + // From a review: Effie-2289's P8, the two-reset flow with merges on, run through ContextManager as a host does. + const MARKER = 'You will soon form a new memory'; + const NAME = /\b(OPEN\d|ONE\d|TWO\d|TOPIC_ONE|TOPIC_TWO)\b/g; + const namesIn = (t: string): string[] => [...new Set([...t.replace(/\\n/g, ' ').matchAll(NAME)].map((m) => m[1]!))]; + const captured: string[] = []; + const membrane = { + complete: async (req: unknown) => { + const blob = JSON.stringify(req); + captured.push(blob); + const at = blob.indexOf(MARKER); + return { stopReason: 'end_turn', content: [{ type: 'text', text: `${at >= 0 ? 'MEM' : 'MERGE'}[${namesIn(at >= 0 ? blob.slice(at) : blob).join(',')}]` }] }; + }, + }; + const probe = await ContextManager.open({ path: freshPath(), strategy: new ProbeStrategy({ compressionModel: 'm' } as never), membrane: membrane as never }); + probe.addMessage('user', text(body('OPEN0'))); + const per = context(probe).messageStore.estimateTokens(context(probe).messageStore.getAll()[0]!); + await probe.close(); + const strategy = new ProbeStrategy({ + compressionModel: 'test-compression-model', headWindowTokens: 3 * per + Math.floor(per / 2), recentWindowTokens: 0, l1HoldbackChunks: 0, + targetChunkTokens: 60, autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 2, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never }); + const order: string[] = []; + const add = (name: string, participant: string): void => { manager.addMessage(participant, text(body(name))); order.push(name); }; + const drain = async (): Promise => { for (let i = 0; i < 60; i++) await manager.tick(); }; + for (let i = 0; i < 10; i++) add(`OPEN${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + await manager.resetHeadWindow(body('TOPIC_ONE')); order.push('TOPIC_ONE'); + for (let i = 0; i < 10; i++) add(`ONE${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + await manager.resetHeadWindow(body('TOPIC_TWO')); order.push('TOPIC_TWO'); + for (let i = 0; i < 6; i++) add(`TWO${i}`, i % 2 ? 'assistant' : 'user'); + await drain(); + const pos = new Map(order.map((n, i) => [n, i])); + let compressions = 0; + let merges = 0; + for (const blob of captured) { + const at = blob.indexOf(MARKER); + if (at < 0) { if (blob.includes('You have just reviewed')) merges++; continue; } + compressions++; + const first = Math.min(...namesIn(blob.slice(at)).map((n) => pos.get(n)!)); + const later = namesIn(blob.slice(0, at)).filter((n) => pos.get(n)! >= first); + assert.deepEqual(later, [], `a request for history from ${order[first]} carries, before its chunk: ${later.join(',')}`); + } + assert.ok(compressions >= 5 && merges >= 1, `premise: the flow compresses and merges (${compressions} compressions, ${merges} merges)`); + await manager.close(); + }); + + it("tick's tool deferral and the count of queued work agree for a chunk before a reset whose head holds a tool block", async () => { + const { manager, strategy, ctx, captured, at } = await build(2, { openingTool: true }); + const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + await strategy.runCompress(chunk, ctx); + assert.equal(captured.length, 0, 'tick defers it: its request would carry the opening, whose tool blocks need the host\'s tools'); + assert.equal(strategy.carriesToolBlocks(ctx, chunk), true, 'and the count reads the same request'); + await manager.close(); + }); +}); diff --git a/test/compression-transient-ladder.test.ts b/test/compression-transient-ladder.test.ts index fff506ad..18907b85 100644 --- a/test/compression-transient-ladder.test.ts +++ b/test/compression-transient-ladder.test.ts @@ -712,8 +712,10 @@ describe('progress is a shared record, not an instance snapshot', () => { await assert.rejects(a.strategy.run(a.target(), managerContext(a.manager)), /zz network failure/); assert.equal(progressSlot(a.manager).length, 1); - const want = new Set(a.ids.slice(6, 9)); - const other: Chunk = { index: 998, startIndex: 6, endIndex: 9, messages: managerContext(b.manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; + // B's family: a chunk of its own that carries the skip_reply cycle, so its hoist rung applies. (A chunk before + // the cycle no longer does: its request's head ends at the chunk, #317.) + const want = new Set(a.ids.slice(11, 13)); + const other: Chunk = { index: 998, startIndex: 11, endIndex: 13, messages: managerContext(b.manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; await assert.rejects(b.strategy.run(other, managerContext(b.manager)), /zz network failure/); assert.equal(progressSlot(a.manager).length, 2, "B's write kept A's family"); diff --git a/test/l1-recall-raw-pins-frontier.test.ts b/test/l1-recall-raw-pins-frontier.test.ts index d1f7de68..d5924be5 100644 --- a/test/l1-recall-raw-pins-frontier.test.ts +++ b/test/l1-recall-raw-pins-frontier.test.ts @@ -225,7 +225,7 @@ describe('raw pins open only their own branch of the L1 recall frontier', () => cm.close(); }); - it('a pin after the chunk being compressed opens nothing: its summary is recalled, as on main', async () => { + it('a pin after the chunk being compressed opens nothing, and its summary, which comes after the chunk, is not recalled (#317)', async () => { // A message the chunker skipped while it was pinned raw, then unpinned, is // compressed out of order, after the history that follows it. let skippedPin = ''; @@ -247,7 +247,9 @@ describe('raw pins open only their own branch of the L1 recall frontier', () => const request = requests.find(compressing); assert.ok(request, 'setup: the skipped message was compressed'); const texts = textsOf(request); - assert.ok(texts.includes(`[CM] Recall memory ${root.id}.`), 'the summary holding the later pin is recalled'); + const rootFirst = ids.indexOf((root as unknown as { sourceRange: { first: string } }).sourceRange.first); + assert.ok(rootFirst > 30, `setup: the summary holding the later pin starts after the chunk (at ${rootFirst})`); + assert.ok(!texts.includes(`[CM] Recall memory ${root.id}.`), 'the summary holding the later pin, after the chunk, is not recalled: the request holds only what came before'); assert.ok(!texts.includes(textOf.get(ids[100])!), 'the later pin adds no raw'); cm.close(); }); diff --git a/test/overbudget-diagnostics.test.ts b/test/overbudget-diagnostics.test.ts index 81b36e04..c32186ba 100644 --- a/test/overbudget-diagnostics.test.ts +++ b/test/overbudget-diagnostics.test.ts @@ -1684,7 +1684,7 @@ describe('OverBudgetError.clearedBy', () => { }); } - it('before the host declares its tools, counts no queued chunk that starts inside the head window when the head carries a tool cycle after the chunk\'s start, which tick\'s head carries raw', async () => { + it('before the host declares its tools, counts no queued chunk that lies inside the head window, and tick asks the model for the first: its request carries the head only up to the chunk, not the tool cycle after it (#317)', async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1712,19 +1712,22 @@ describe('OverBudgetError.clearedBy', () => { const queued = await settledQueue(manager, strategy); assert.ok(!queued[0].some((id) => cycle.includes(id)), 'the first queued chunk is text ahead of the cycle'); strategy.setHeadWindowTokens(100_000); - assert.equal(strategy.countsCompressionWork(), false, 'tick defers every queued chunk until the tools come'); - // The premise: tick's head carries the cycle raw, so it defers each queued chunk, with no model call. - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); - assert.equal(calls.n, 0, 'tick asked no model'); - assert.deepEqual(strategy.summaryIds(), [], 'and compressed none'); + assert.equal(strategy.countsCompressionWork(), false, 'chunks inside the head fold nothing, so none counts'); + // The head a request gives is cut at its chunk's start (headAsOf), so the cycle after the first chunk isn't in + // its request: tick asks the model rather than deferring, and the counting membrane throws. Before #317 the + // head went whole, messages after the chunk included. + await assert.rejects(async () => { + for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); + }, /no model in this test/); + assert.equal(calls.n, 1, 'tick asked the model'); }, { membrane: countingMembrane(calls) }); }); // The chunks the test above widens the head over lie wholly inside it, so they fold nothing and don't count - // whatever the tools say. A chunk that lies before a reset head start folds, and the head after it, which - // tick's request carries raw, is what holds it back: a tool message in the head lies past the chunk's start. - it('before the host declares its tools, counts no queued chunk that lies before a reset head start when the head carries a tool cycle, which tick\'s request carries raw though it lies after the chunk', async () => { + // whatever the tools say. A chunk that lies before a reset head start folds. Tick's request for it gives the + // head that history followed, the opening here, and not the reset head after it (#317), so a tool cycle in the + // reset head doesn't hold it back. + it('before the host declares its tools, names compression for a queued chunk that lies before a reset head start when that head carries a tool cycle, which lies after the chunk and tick\'s request leaves out (#317)', async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1756,14 +1759,13 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(chunkAt.every((i) => i >= 0 && i < headStart), 'the premise: the first queued chunk lies before the head window\'s start, so it folds'); assert.ok(!queued[0]!.some((id) => cycle.includes(id)), 'the premise: and holds no tool block of its own'); assert.ok(cycleAt.every((i) => i >= headStart && i < headEnd), 'the premise: the cycle sits in the head window, past the chunk\'s start'); - assert.deepEqual(error.clearedBy, [], 'tick defers every queued chunk until the tools come'); - // The premise: tick's request carries the head raw, cycle included, so it defers each queued chunk, with no model call. - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); - assert.equal(calls.n, 0, 'tick asked no model'); - assert.deepEqual(strategy.summaryIds(), [], 'and compressed none'); - manager.setToolDefinitions([{ name: 'zz_tool', description: 'zz', inputSchema: { type: 'object' } }]); - assert.deepEqual((await refusal(manager)).clearedBy, ['compression'], 'with the tools declared, they count'); + assert.deepEqual(error.clearedBy, ['compression'], 'the chunk folds and tick will take it without the tools'); + // The premise: tick's request gives the opening as its head, without the cycle, so tick asks the model, and the + // counting membrane throws. + await assert.rejects(async () => { + for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); + }, /no model in this test/); + assert.equal(calls.n, 1, 'tick asked the model'); }, { membrane: countingMembrane(calls) }); }); @@ -1814,7 +1816,7 @@ describe('OverBudgetError.clearedBy', () => { } for (const pinned of [false, true]) { - it(`names ${pinned ? 'nothing' : 'compression'} before the host declares its tools for text chunks after a tool cycle no L1 covers, before the head window's start, ${pinned ? 'pinned, which tick\'s request then carries raw' : 'which tick\'s request leaves out'}`, async () => { + it(`names nothing before the host declares its tools for text chunks after a tool cycle no L1 covers, before the head window's start, ${pinned ? 'pinned' : 'unpinned'}: tick's request carries the cycle raw in the opening, the head of that history (#317)`, async () => { const strategy = new DiagnosticStrategy({ adaptiveResolution: false, targetChunkTokens: 40, @@ -1841,17 +1843,10 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(indices.every((i) => i >= 0 && i < headStart), 'the premise: the cycle sits before the head window\'s start'); const queued = strategy.queuedChunkIds(); assert.ok(queued.some((ids) => !ids.some((id) => cycle.includes(id))), 'text chunks are queued'); - if (!pinned) { - assert.deepEqual(error.clearedBy, ['compression']); - // The premise: tick's request leaves the cycle out, so tick asks the model, and the counting membrane throws. - await assert.rejects(async () => { - for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); - }, /no model in this test/); - assert.equal(calls.n, 1, 'tick asked the model'); - return; - } assert.deepEqual(error.clearedBy, [], 'nothing tick will do clears it'); - // The premise: tick's request carries the pinned cycle raw, so tick defers every queued chunk, with no model call. + // The premise: tick's request for a chunk before the reset gives the opening as its head, cycle included, raw + // (pinned or not), so tick defers every queued chunk, with no model call. Before #317 an unpinned cycle there + // was left out, since the head was the reset one. for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); assert.equal(calls.n, 0, 'tick asked no model'); diff --git a/test/tool-prose-hoist.test.ts b/test/tool-prose-hoist.test.ts index 558c431f..a0d8d6e4 100644 --- a/test/tool-prose-hoist.test.ts +++ b/test/tool-prose-hoist.test.ts @@ -125,8 +125,9 @@ function scripted(responses: Array>) { return { calls, membrane: { complete: async (request: NormalizedRequest) => { calls.push(structuredClone(request)); return responses[Math.min(n++, responses.length - 1)]; } } as never }; } -// The fixture's large head window replays the chunk's messages in the head as -// well as in the target slot, so canonical requests carry each call twice. +// The fixture's large head window holds every message, but a request's head is +// cut at its chunk's start (headAsOf, #317), so canonical requests carry each of +// the chunk's calls once, in the target slot. const names = (req: NormalizedRequest): string => [...new Set(toolUses(req).map((u) => u.name))].join('+'); describe('tool-prose hoist rung', () => { From 895c368f0d9175e1ac850c1fee1b911767796b4a Mon Sep 17 00:00:00 2001 From: Cedric-2315 Date: Fri, 23 Oct 2026 04:48:44 -0400 Subject: [PATCH 26/67] fix(compression): read heads once per count of queued work; narrow claims (#317) From the automated review of f5c4216. The count of queued work (requestCarriesToolBlocks) asked headAsOf once per queued chunk it read. Each call read the live head's start and the holds (a scan of the store when a message is held), scanned back from the chunk to the latest transition, and walked a head. With a long backlog from before a reset, each chunk the count read cost scans of the store, where main read the head once per count. headsAsOf(store) now returns a reader that reads the live head's start and the holds once, the live head's end once when a position in it is asked for, the transitions before the live head's start once when a position before it is asked for, and walks each past head once per start. headAsOf(store, pos) is that reader asked once, so tick's requests get the heads they got before. The count builds its reader at its first head read, so a count that reads no head scans no holds. Tests (compression-head-as-of, now 35): - one reader of heads gives every position, from -1 to the store's end, the head the rule gives it alone, worked out in the test by scanning back from the position as headAsOf did per call. It is asked in three orders, with three transitions before the live head's start, then with the live head's end overridden (getHeadWindowEnd), then with a hold in the live head; - one count reads the holds once and walks a head once for the chunks that share it, and a count that reads no head scans no holds; - a pin that opens a summary over a tool cycle changes the count's answer at the same head start; - under a recall budget for one pair, a compression request keeps the newer summary when the older one has its first message removed. Text: - headAsOf's doc: where a past head starts comes from the store alone, and its end is priced with today's window and calibration and extended by today's chunk ownership (the ratchet guard). The restart sentence keeps only its condition: while the live head starts at the last transition in the store. - headsAsOf's doc: a subclass that changes heads overrides it, since tick's requests and the count both read it. - The L1 recall comment, the merge's as-of comment and the changelog: a summary written before this fix may already carry what came after its history, and it is recalled as written. So the first commit's "the one way later content still reaches a request" no longer holds. - The changelog: a source-only request carries no history beside its chunk, so neither route reached it. - The cache-marker comments: consecutive requests share the head window only while they summarize one topic's history. - The count's comment on pinned messages, scoped to those before `from`. Co-authored-by: Claude Opus 5.5 --- .../317-compression-head-as-of.fixed.md | 4 +- src/strategies/autobiographical.ts | 112 +++++++++++---- test/compression-head-as-of.test.ts | 131 +++++++++++++++++- 3 files changed, 210 insertions(+), 37 deletions(-) diff --git a/changelog.d/317-compression-head-as-of.fixed.md b/changelog.d/317-compression-head-as-of.fixed.md index aecd10e1..dec8a6a6 100644 --- a/changelog.d/317-compression-head-as-of.fixed.md +++ b/changelog.d/317-compression-head-as-of.fixed.md @@ -1,7 +1,7 @@ -- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request no longer gives the history it summarizes what came after it, ahead of it (#317). It came by two routes: +- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request no longer gives the history it summarizes what came after it, ahead of it (#317). It came by two routes, in requests that carry history beside their chunk (a source-only request, `compressionSourceOnly` or `compressionMergeSourceOnly`, carries none): - **The head:** a request for history from before the reset gave the new topic's opening as its head. It now gives the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. - **Recall pairs:** a compression request recalled every summary on the frontier, so a chunk compressed after later history had been summarized was given those later summaries too. That's the old head's chunk, which a reset makes compressible late, or a pinned or quarantined chunk let go late. It now recalls only summaries that start before it, the rule merges already used; the later summaries' messages stay out of it as before. - - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, the one way later content does, and only where a summary straddles the chunk. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. Summaries written before this fix are recalled as written, and one written after a reset may already carry the new topic. - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index c20a7631..b76c6b28 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -4016,12 +4016,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { * Consecutive compression/merge requests share a long, append-mostly * prefix — head window + prior recall pairs (~60% of input on synthetic * workloads, ~93% on mature stores) — that previously went to the API - * with zero `cache_control` and was re-read cold on every call. Markers + * with zero `cache_control` and was re-read cold on every call. They share + * the head window only while they summarize one topic's history: after a + * head-window reset, requests for different topics' history each carry + * that topic's head (headAsOf). Markers * land at the three boundaries whose survival probability ranks highest * (measured optimum; `analysis/2026-08-20-cm37-cache-partition-experiments.md` * in the workspace root): * - * 1. end of head window — survives everything short of identity edits + * 1. end of head window — survives everything short of identity edits, + * among requests for one topic's history * 2. last level>=2 recall pair — survives appends AND L1→L2 merges (a * merge replaces the run at this boundary and extends it) * 3. last recall pair — the append frontier @@ -4085,7 +4089,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (frontierEnd === -1) return false; // no ladder yet — nothing stable to cache // A capped ladder front-evicts on every mint, so the pair seams shift and // writing them is waste. The head seam does not: tools, system prompt and - // head window precede the ladder and are the same request after request, + // head window precede the ladder and are the same request after request + // for one topic's history (headAsOf gives each topic its own head), // unless image shedding (one byte budget, spent newest-first) replaces a // head image for one payload and not for the next. let placed = false; @@ -9220,9 +9225,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // starts before the chunk and reaches past it (a sparse sweep // artifact, issue #42) is still recalled, since the messages it covers // before the chunk have no other form here; so its later part, what it - // says of history after the chunk, still reaches this request, the one - // way later content does, and only where a summary straddles the - // chunk. Where a summary starts is summaryStartInStore's rule, the one + // says of history after the chunk, still reaches this request, and only + // where a summary straddles the chunk. A summary written before this + // rule may also carry what came after its own history, from the head or + // recall pairs it was written with; that stays in its text. Where a + // summary starts is summaryStartInStore's rule, the one // a merge's prior recall set uses: a summary whose first message a host // removed starts at its first message still there, and one with none // left isn't recalled. A chunk with no message left (chunkStartIdx -1) @@ -11829,7 +11836,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // // No tail-after-merge: same as-of principle as L1 compression. The // consolidation is being formed at the moment the last source was - // ready, so nothing after that is visible. + // ready, so nothing after that is visible, except what a recalled + // summary's own text carries: one written before this as-of rule may + // say what came after its history. const llmMessages: Array<{ participant: string; content: ContentBlock[] }> = []; // Build lookup maps @@ -12868,6 +12877,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const sourceOnly = this.config.compressionSourceOnly === true; const position = new Map(all.map((message, i) => [message.id, i])); const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all)].sort((a, b) => a - b); + // Built at the first head read, so a count that reads none (source-only, + // or each chunk holding a tool block itself) scans no holds. + let heads: ((pos: number) => { start: number; end: number }) | undefined; // For each count of pins before a chunk and the start its head's raw // reading takes, the index of the first message that request carries raw // with a tool block, or all.length for none. @@ -12877,7 +12889,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (sourceOnly) return false; // The chunk's start as tick's request takes it (startInStore). const start = this.startInStore(chunk.messages.map((m) => m.id), position); - const head = this.headAsOf(store, start); + heads ??= this.headsAsOf(store); + const head = heads(start); const headEnd = head.end; const from = Math.min(head.start, headEnd); const pinned = pins.filter((i) => i < start); @@ -12887,8 +12900,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const pinnedIds = new Set(pinned.map((i) => all[i].id)); const seen = this.recallCoverage(pinnedIds); // Before `from`, the request carries only what pins keep raw: an L1 a - // pin opens, and a pinned message (which lies before headEnd, since - // from is at most headEnd). + // pin opens, and a pinned message. (Either lies before headEnd there, + // since from is at most headEnd.) const carried = (message: StoredMessage, i: number): boolean => !seen.coveredMessageIds.has(message.id) && (i >= from || seen.openedMessageIds.has(message.id) || pinnedIds.has(message.id)); @@ -18193,35 +18206,74 @@ export class AutobiographicalStrategy implements ResettableStrategy { * the original instance saw it" and keeps later events out. * * Three things it is, by choice: - * - Before the live head's start it reads the store alone, as a restart - * does. A strategy-level resetHeadWindow to a message that isn't a - * transition isn't in the store: history before that message gets the - * latest transition before it, or the opening, as after a restart. History - * from that message on gets the live head, which a restart moves to the - * last transition in the store. So a restart gives the same heads while - * the live head starts at the last transition, as it does when every - * transition is written by ContextManager.resetHeadWindow, which resets - * the head to it. + * - Before the live head's start, where a head starts comes from the store + * alone, as a restart reads it. A strategy-level resetHeadWindow to a + * message that isn't a transition isn't in the store: history before that + * message gets the latest transition before it, or the opening, as after a + * restart. History from that message on gets the live head, which a + * restart moves to the last transition in the store. So a restart gives + * the same heads while the live head starts at the last transition in the + * store. * - A past head is priced with today's headWindowTokens and calibration, as - * the live head is, not with those in force when it was the head. + * the live head is, and the ratchet guard (anchorHeadToCoverage) extends + * it by today's chunk ownership, not by those in force when it was the + * head. * - Before the live head's start, any Context Manager message whose text * starts with "[Topic Transition]" starts a head, as the restart reads * one, whether or not resetHeadWindow wrote it. */ protected headAsOf(store: MessageStoreView, pos: number): { start: number; end: number } { + return this.headsAsOf(store)(pos); + } + + /** + * headAsOf for many positions in one store state, such as each queued chunk + * a count of queued work reads (requestCarriesToolBlocks): the live head's + * start, the holds and the live head's end are read once, the transitions + * before the live head's start once when a position before it is asked + * for, and each past head's end once per start. Tick's requests (through + * headAsOf) and the count both read heads here, so a subclass that changes + * heads overrides this. + */ + protected headsAsOf(store: MessageStoreView): (pos: number) => { start: number; end: number } { const liveStart = this.getHeadWindowStartIndex(store); const hold = this.holdBoundary(store); - if (pos < 0) return { start: liveStart, end: liveStart }; - if (pos >= liveStart) return { start: liveStart, end: Math.min(this.getHeadWindowEnd(store), hold, pos) }; - const messages = store.getAll(); - let start = 0; - for (let i = pos; i > 0; i--) { - if (this.isTopicTransitionMessage(messages[i])) { - start = i; - break; + let liveEnd: number | undefined; + let messages: StoredMessage[] | undefined; + let transitions: number[] | undefined; + const pastEnds = new Map(); + return (pos) => { + if (pos < 0) return { start: liveStart, end: liveStart }; + if (pos >= liveStart) { + liveEnd ??= this.getHeadWindowEnd(store); + return { start: liveStart, end: Math.min(liveEnd, hold, pos) }; } - } - return { start, end: Math.min(this.headWindowEndFrom(store, messages, start), hold, pos) }; + messages ??= store.getAll(); + if (!transitions) { + const found: number[] = []; + for (let i = 1; i < liveStart && i < messages.length; i++) { + if (this.isTopicTransitionMessage(messages[i])) found.push(i); + } + transitions = found; + } + // The latest transition at or before pos, or the opening. + let start = 0; + for (let lo = 0, hi = transitions.length - 1; lo <= hi;) { + const mid = (lo + hi) >> 1; + if (transitions[mid]! <= pos) { + start = transitions[mid]!; + lo = mid + 1; + } else { + hi = mid - 1; + } + } + let end = pastEnds.get(start); + if (end === undefined) { + end = this.headWindowEndFrom(store, messages, start); + pastEnds.set(start, end); + } + return { start, end: Math.min(end, hold, pos) }; + }; } /** diff --git a/test/compression-head-as-of.test.ts b/test/compression-head-as-of.test.ts index d5a56d86..b17039e7 100644 --- a/test/compression-head-as-of.test.ts +++ b/test/compression-head-as-of.test.ts @@ -1,8 +1,9 @@ /** * The head a compression or merge request gives as context is the head in force when the history it summarizes * happened: before the live head's start, the head starting at the latest topic transition at or before the - * request's first message, or the opening; from the live head's start on, the live head; cut at that message (#317). After resetHeadWindow, a request for history from before the reset gave the - * new topic's opening ahead of it, where the code places the chronicle's opening and keeps later events out. + * request's first message, or the opening; from the live head's start on, the live head; cut at that message + * (#317). After resetHeadWindow, a request for history from before the reset gave the new topic's opening ahead of + * it, where the code places the chronicle's opening and keeps later events out. */ import { describe, it, after } from 'node:test'; @@ -28,6 +29,33 @@ class ProbeStrategy extends AutobiographicalStrategy { runMerge(targetLevel: number, sourceIds: string[], ctx: StrategyContext): Promise { return this.executeMerge(targetLevel as never, sourceIds, ctx); } + /** One reader of heads for many positions, as the count of queued work holds one. */ + heads(ctx: StrategyContext): (pos: number) => { start: number; end: number } { + return (this as unknown as { headsAsOf(store: unknown): (pos: number) => { start: number; end: number } }) + .headsAsOf(ctx.messageStore); + } + /** The head rule worked out for one position alone, scanning back from it, as headAsOf computed it per call + * before heads were read once per count: the oracle the shared reader is checked against. */ + headAlone(ctx: StrategyContext, pos: number): { start: number; end: number } { + const store = ctx.messageStore; + const liveStart = this.getHeadWindowStartIndex(store); + const hold = this.holdBoundary(store); + if (pos < 0) return { start: liveStart, end: liveStart }; + if (pos >= liveStart) return { start: liveStart, end: Math.min(this.getHeadWindowEnd(store), hold, pos) }; + const messages = store.getAll(); + let start = 0; + for (let i = pos; i > 0; i--) { + if (this.isTopicTransitionMessage(messages[i]!)) { + start = i; + break; + } + } + return { start, end: Math.min(this.headWindowEndFrom(store, messages, start), hold, pos) }; + } + /** Timeline scans of holdBoundary so far. */ + holdScans(): number { + return this.holdBoundaryScans; + } /** The head a request gives for history from `pos` on. */ asOf(ctx: StrategyContext, pos: number): { start: number; end: number } { return (this as unknown as { headAsOf(store: unknown, pos: number): { start: number; end: number } }) @@ -240,7 +268,7 @@ describe('the head a compression or merge request gives is the head as of the hi await manager.close(); }); - it('a restart reads the same heads while the live head starts at the last transition: past heads come from the store alone', async () => { + it("a restart reads the same heads while the live head starts at the last transition: where a past head starts comes from the store alone", async () => { const path = freshPath(); const first = await build(2, { path }); const positions = [first.at.OPEN6!, first.at.ONE5!, first.at.T1!, first.at.TWO4!]; @@ -438,8 +466,9 @@ describe('the head a compression or merge request gives is the head as of the hi // Seeded newest first, so the order the request gives comes from the sort by start, not from the seeding. for (const x of [c, b, gone, newer, older]) strategy.seed(x); // The newer summary's first message is removed: it starts at OPEN5, its first leaf still in the store, after - // the older one's OPEN3. (main's 1a, by sourceRange.first, left it out; ordering by sourceRange.first would put - // it first, at -1.) + // the older one's OPEN3. (Before this change the merge's prior recall set, section 1a of executeMerge, placed a + // summary by sourceRange.first, which has no place here: it left this one out. Ordering by sourceRange.first + // would put it first, at -1.) manager.removeMessage(ids[at.OPEN4!]! as never); manager.removeMessage(ids[at.OPEN6!]! as never); assert.equal(await strategy.runMerge(2, [b.id, c.id], ctx), true, 'premise: the merge runs'); @@ -634,6 +663,98 @@ describe('the head a compression or merge request gives is the head as of the hi await manager.close(); }); + it("one reader of heads gives every position the head the rule gives it alone, asked in any order, with three topics before the live head, a hold in the live head, and the live head's end overridden", async () => { + const { manager, strategy, ctx, at } = await build(2); + // Two resets more, so three transitions lie before the live head's start. + for (const topic of ['THREE', 'FOUR']) { + await manager.resetHeadWindow(`${topic} begins.`); + for (let i = 0; i < 4; i++) manager.addMessage(i % 2 ? 'assistant' : 'user', text(body(`${topic}${i}`))); + } + const messages = ctx.messageStore.getAll(); + const liveStart = strategy.asOf(ctx, messages.length - 1).start; + const transitions = messages.flatMap((m, i) => + i > 0 && i < liveStart && m.content.some((c) => c.type === 'text' && c.text.startsWith('[Topic Transition]')) ? [i] : []); + assert.equal(transitions.length, 3, 'premise: three transitions before the live head'); + const positions = Array.from({ length: messages.length + 1 }, (_, i) => i - 1); + const orders = { + ascending: positions, + descending: [...positions].reverse(), + 'evens, then odds': [...positions.filter((p) => p % 2 === 0), ...positions.filter((p) => p % 2 !== 0)], + }; + const check = (state: string): void => { + for (const [name, order] of Object.entries(orders)) { + const reader = strategy.heads(ctx); + for (const p of order) assert.deepEqual(reader(p), strategy.headAlone(ctx, p), `${state}, ${name}: position ${p}`); + } + }; + check('as built'); + assert.deepEqual(strategy.heads(ctx)(at.ONE5!), { start: at.T1!, end: at.T1! + 3 }, "topic one's head"); + const live = strategy as unknown as { getHeadWindowEnd(store: unknown): number }; + live.getHeadWindowEnd = () => liveStart + 1; + assert.deepEqual(strategy.heads(ctx)(liveStart + 2), { start: liveStart, end: liveStart + 1 }, "premise: the override shortens the live head"); + check("with the live head's end overridden"); + delete (live as { getHeadWindowEnd?: unknown }).getHeadWindowEnd; + manager.holdCompression([messages[liveStart + 2]!.id]); + assert.deepEqual(strategy.heads(ctx)(liveStart + 3), { start: liveStart, end: liveStart + 2 }, 'premise: the hold cuts the live head'); + check('with a hold in the live head'); + await manager.close(); + }); + + it('one count reads the holds once and walks a head once for the chunks that share it, however many it reads', async () => { + const { manager, strategy, ctx, at } = await build(2, { openingTool: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + manager.holdCompression([ids[at.TWO5!]!]); + const walker = strategy as unknown as { headWindowEndFrom(...args: unknown[]): number }; + const walk = walker.headWindowEndFrom.bind(strategy); + let walks = 0; + walker.headWindowEndFrom = (...args: unknown[]): number => { + walks++; + return walk(...args); + }; + const scans = strategy.holdScans(); + const reader = strategy.toolBlockReader(ctx); + const chunks = [chunkOf(ctx, at.OPEN4!, at.OPEN5!), chunkOf(ctx, at.OPEN6!, at.OPEN7!), chunkOf(ctx, at.OPEN8!, at.OPEN9!)]; + assert.deepEqual(chunks.map((c) => reader(c)), [true, true, true], "premise: the opening's head carries its tool cycle into each chunk's request"); + assert.equal(strategy.holdScans() - scans, 1, 'one scan of the holds'); + assert.equal(walks, 1, "one walk of the opening's head"); + const before = strategy.holdScans(); + const own = strategy.toolBlockReader(ctx); + assert.equal(own(chunkOf(ctx, at.OPEN1!, at.OPEN3!)), true, 'premise: a chunk holding a tool block answers from its own messages'); + assert.equal(strategy.holdScans(), before, 'so its count reads no head and scans no holds'); + await manager.close(); + }); + + it("one count's reader keys its memo by the pins before a chunk too: a pin that opens a summary over a tool cycle changes the answer at the same head start", async () => { + const { manager, strategy, ctx, at } = await build(0, { openingTool: true }); + const ids = ctx.messageStore.getAll().map((m) => m.id); + // The opening's tool cycle (OPEN1, OPEN2) is covered by an L1, and a pin on OPEN3 opens that L1's messages. + strategy.seed(summary('L1-921', [ids[at.OPEN1!]!, ids[at.OPEN2!]!, ids[at.OPEN3!]!])); + manager.pinRange(ids[at.OPEN3!]! as never, ids[at.OPEN3!]! as never); + const before = chunkOf(ctx, at.OPEN0!, at.OPEN1!); + const after = chunkOf(ctx, at.OPEN6!, at.OPEN8!); + assert.equal(strategy.asOf(ctx, at.OPEN0!).start, strategy.asOf(ctx, at.OPEN6!).start, 'premise: both chunks have the same head start'); + const reader = strategy.toolBlockReader(ctx); + assert.equal(reader(before), false, 'before the pin, the L1 covers the cycle'); + assert.equal(reader(after), true, 'after it, the pin opens the L1 and its cycle is carried raw'); + await manager.close(); + }); + + it('under a recall budget for one pair, a compression request keeps the newer summary when the older one has its first message removed', async () => { + const { manager, strategy, ctx, captured, at } = await build(0); + const ids = ctx.messageStore.getAll().map((m) => m.id); + const older = summary('L1-931', [ids[at.OPEN3!]!, ids[at.OPEN4!]!]); + const newer = summary('L1-932', [ids[at.OPEN5!]!, ids[at.OPEN6!]!]); + strategy.seed(older); + strategy.seed(newer); + (strategy as unknown as { config: { compressionRecallBudgetTokens: number } }).config.compressionRecallBudgetTokens = 100; + manager.removeMessage(ids[at.OPEN3!]! as never); + await strategy.runCompress(chunkOf(ctx, at.OPEN7!, at.OPEN9!), ctx); + const req = captured[0]!; + assert.equal(countIn(req, `authored ${older.id}`) + countIn(req, `authored ${newer.id}`), 1, 'premise: the budget holds one pair'); + assert.equal(countIn(req, `authored ${newer.id}`), 1, 'the newer summary, which starts later, is kept'); + await manager.close(); + }); + it("tick's tool deferral and the count of queued work agree for a chunk before a reset whose head holds a tool block", async () => { const { manager, strategy, ctx, captured, at } = await build(2, { openingTool: true }); const chunk = chunkOf(ctx, at.OPEN6!, at.OPEN8!); From bcf8bcd072c4650e12bf7eda86d0e5d78bc4d5f6 Mon Sep 17 00:00:00 2001 From: Cedric-2315 Date: Fri, 23 Oct 2026 05:30:42 -0400 Subject: [PATCH 27/67] test(compression): drop a reviewer's name from two comments (#317) Two comments in compression-head-as-of named the reviewer whose probes they came from. They now say "a review's probe" and "From a review", and nothing else in the file changes. Co-authored-by: Claude Opus 5.5 --- test/compression-head-as-of.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/compression-head-as-of.test.ts b/test/compression-head-as-of.test.ts index b17039e7..90dedf96 100644 --- a/test/compression-head-as-of.test.ts +++ b/test/compression-head-as-of.test.ts @@ -84,7 +84,7 @@ class ProbeStrategy extends AutobiographicalStrategy { /** Lets every merge through the topology check, so a merge whose source messages a host removed reaches its * request and executeMerge's own start rule is what these tests pin. The real check may refuse such a merge (a - * merge whose first source message was removed returned false in Effie-2289's P5): these tests don't show that + * merge whose first source message was removed returned false in a review's probe): these tests don't show that * production reaches it. */ class PlacingTopologyStrategy extends ProbeStrategy { protected override mergeTopologyProblem(): string | null { @@ -615,7 +615,7 @@ describe('the head a compression or merge request gives is the head as of the hi }); it('with merges on, through two resets, no compression request carries anything from at or after its chunk', async () => { - // From a review: Effie-2289's P8, the two-reset flow with merges on, run through ContextManager as a host does. + // From a review: the two-reset flow with merges on, run through ContextManager as a host does. const MARKER = 'You will soon form a new memory'; const NAME = /\b(OPEN\d|ONE\d|TWO\d|TOPIC_ONE|TOPIC_TWO)\b/g; const namesIn = (t: string): string[] => [...new Set([...t.replace(/\\n/g, ' ').matchAll(NAME)].map((m) => m[1]!))]; From 86984f6f6bde8e80f4feab24894b9673c7d18234 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Fri, 23 Oct 2026 06:28:55 -0400 Subject: [PATCH 28/67] fix(calibration): price message-position injections into the sample; two agreeing declared samples move a store that learned the overhead (#323) From our internal review of #324. - Message-position injections. ContextManager.compile inserts beforeUser and afterUser injections after the select, so the request carries them and the window estimate doesn't. With system and tools declared, an injection large beside its window was learned as a per-token rate, or, as a consistent declared out-of-band run, at the clamp. compile now passes their content to the select as SelectOptions.messageInjections; the adaptive select prices it where it arms the sample, at the multiplier it priced the window at, and the sample adds it to the estimate, declared or not. A dry select puts the count back. system injections stay in the caller's declared system prompt. - A store that learned the overhead as a rate. Declared samples stepped such a multiplier back over about ten applied changes, each a reprice of every message and summary. The record now carries declaredOverhead once declared samples agree with its multiplier. On a record without it, a declared sample inside the band is held in memory, and the next declared sample decides: within OUT_OF_BAND_RUN_AGREEMENT their mean replaces the multiplier, and otherwise the EMA steps; either way the record is marked. Two, not one, since this reaches every store at upgrade and one wild sample shouldn't set a correct store's ruler. A restart or branch switch, a provisional replacement or a run ends a held sample; an undeclared sample doesn't, since an implied multiplier doesn't depend on the multiplier in force. The flag is read on load, kept through a re-stamp, and dropped by an undeclared write that moves the multiplier. A provisional multiplier, or a run, still moves on one sample. - A consistent out-of-band run agrees with its first sample, not the one before, as the changelog says, so a drifting run isn't learned. - Declared means passed, {} included and null not, throughout. - RequestOverhead is exported from the package root, and its doc says passing it claims completeness. Docs, comments and the declared rejection's words follow the code; the changelog keeps the provisional seed for a store whose multiplier puts every compile over budget. - 29 tests in test/calibration-overhead.test.ts; #324's first test sends two samples, since one is held. Co-Authored-By: Claude Opus 5.5 --- .../323-calibration-request-overhead.fixed.md | 40 +- src/context-manager.ts | 8 +- src/index.ts | 1 + src/strategies/autobiographical.ts | 144 ++++-- src/types/strategy.ts | 19 +- test/calibration-overhead.test.ts | 423 +++++++++++++++++- 6 files changed, 591 insertions(+), 44 deletions(-) diff --git a/changelog.d/323-calibration-request-overhead.fixed.md b/changelog.d/323-calibration-request-overhead.fixed.md index 7c06e8fb..3b1a141b 100644 --- a/changelog.d/323-calibration-request-overhead.fixed.md +++ b/changelog.d/323-calibration-request-overhead.fixed.md @@ -5,16 +5,34 @@ estimate doesn't, so a tool-heavy agent's multiplier learned its schemas as a per-token rate and folded history the budget had room for, or sat at 1.00 rejecting every sample as out of band. A caller that passes nothing - gets the old comparison. + gets the old comparison, plus the compile's message-position injections + (below). - With the overhead declared, three out-of-band calibration samples in a row - whose implied multipliers agree within 10% are learned, outright, at the - clamp edge, instead of being rejected for good. At 1.00 the ratio and the - implied multiplier are the same number, so a store whose true ratio was - past 1.8, where every pricing-epoch discard restarts, could never learn. - Undeclared samples are rejected as before, and the rejection line says the - caller may be leaving out the system prompt and tools it sends. + whose implied multipliers agree within 10% of the first are learned, + outright, at the clamp edge, instead of being rejected for good. At 1.00 + the ratio and the implied multiplier are the same number, so a store whose + true ratio was past 1.8, where every pricing-epoch discard restarts, could + never learn. Undeclared samples are rejected as before, and the rejection + line says the caller may be leaving out the system prompt and tools it + sends. +- The sample also counts the compile's message-position injections + (`beforeUser` and `afterUser`), which the request carries and the window + estimate doesn't: `ContextManager.compile` passes their content to the + select as `SelectOptions.messageInjections`, and the select prices it with + the window. Left out, an agent whose injections are large beside its window + learned them as a per-token rate, or, with the overhead declared, at the + clamp. - **Host operators:** a multiplier learned before the caller declared its - overhead holds that overhead, and steps back down over about ten compiles - once samples declare it. To relearn in one sample, seed it as provisional - on the stopped store: `setStateJson('/autobio:calibration', - { multiplier, at, pricing, provisional: true })`. + overhead holds that overhead. Inside the calibration band, the first two + samples that declare it replace the multiplier with their mean when they + agree within 10%, and the EMA steps as before when they don't; either way + the record then carries `declaredOverhead: true`, so later declared + samples step by the EMA. (Out of the band, the three-sample run above + applies.) The first of the two is held in memory, so a restart between + them costs one more sample. An undeclared sample that moves the multiplier + writes the record without the flag. A store whose multiplier puts every + compile over budget gets no sample to move it, as before: to unwedge it, + seed the record as provisional on the stopped store, and the refused + compile drops it to 1: `setStateJson('/autobio:calibration', + { multiplier, at, pricing: , provisional: true })`. + `RequestOverhead` is exported from the package. diff --git a/src/context-manager.ts b/src/context-manager.ts index ea298d8c..04fca26e 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1178,12 +1178,16 @@ export class ContextManager { const branch = branchRefOf(this.store); const view = this.strategyMessageView(); - // Get selected entries from strategy + // Get selected entries from strategy. The injections it will sit among + // go with it, for its calibration's count of the request (SelectOptions). + const messageInjections = (injections ?? []) + .filter((inj) => inj.position === 'beforeUser' || inj.position === 'afterUser') + .map((inj) => inj.content); const entries = this.strategy.select( view, this.contextLog.createView(), effectiveBudget, - opts + messageInjections.length > 0 ? { ...opts, messageInjections } : opts ); const selectionCause = this.strategy.takeSelectionCause?.(); // Read now, before anything else selects: these are this select's own. diff --git a/src/index.ts b/src/index.ts index af88b229..82687b0b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -81,6 +81,7 @@ export type { RenderedLayout, LayoutUnit, // Strategy types + RequestOverhead, MessageStoreView, ContextLogView, StrategyContext, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 615138a7..9472770a 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -1856,8 +1856,10 @@ function imageGroup(block: ContentBlock): unknown { } /** Out-of-band calibration samples in a row it takes to learn from them, and - * how close each one's implied multiplier must be to the one before it to - * continue the run (see reportRealInputTokens). */ + * how close each one's implied multiplier must be to the run's first to + * continue the run (see reportRealInputTokens). The agreement also decides + * whether two declared samples move a store whose multiplier wasn't + * learned from declared samples. */ const OUT_OF_BAND_RUN_LENGTH = 3; const OUT_OF_BAND_RUN_AGREEMENT = 0.1; @@ -2852,6 +2854,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._lastRenderStats = null; this._lastSelectStats = null; this._lastCompileEstimate = 0; + this._lastCompileInjected = 0; this._storeView = null; // Prices memoized over the view just cleared: the next one may estimate otherwise. this._answerCostCache.clear(); @@ -2861,7 +2864,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibrationLoaded = false; this._calibrationReset = null; this._calibrationProvisional = null; + this._calibrationDeclared = false; this._outOfBandRun = null; + this._declaredCandidate = null; this._imageStripReplayMemo = null; this._lastKvStable = null; this._storeOrder = new Map(); @@ -13862,23 +13867,36 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._lastCompileEstimate = rs.head.tokens + rs.tail.tokens + rs.middleRaw.tokens + rs.summaries.l1.tokens + rs.summaries.l2.tokens + rs.summaries.l3.tokens; + // What the compile inserts among these entries after the select (its + // message-position injections) is in the request and not in the + // window: priced here, at the multiplier the window was priced at, so + // the sample counts it as the request does (#323). + let injected = 0; + for (const content of opts?.messageInjections ?? []) { + injected += store.estimateTokens({ content } as unknown as StoredMessage); + } + this._lastCompileInjected = injected; this._calibrationArmed = true; // exactly one sample per compile } return merged; } private _lastCompileEstimate = 0; + /** The armed compile's message-position injections, priced with its window. */ + private _lastCompileInjected = 0; private _storeView: MessageStoreView | null = null; /** * Closed-loop estimator calibration (2026-07-12). Feed the REAL input total * for a request built from the latest compile (fresh + cache_read + * cache_creation, as the provider reports it), and what the request sent - * besides the window: its system prompt and tool definitions (#323). Those - * are priced as the store prices the window and added to the estimate, so - * the sample compares the whole request with the whole request; left out, - * the multiplier learns them as a per-token rate, which no value of it can - * be (see RequestOverhead). Maintains an EMA of real/estimated and applies + * besides the window and the compile's message-position injections: its + * system prompt and tool definitions (#323). The injections are priced at + * the compile (SelectOptions.messageInjections) and the overhead here, as + * the store prices the window, and both are added to the estimate, so the + * sample compares the whole request with the whole request; left out, the + * multiplier learns them as a per-token rate, which no value of it can be + * (see RequestOverhead). Maintains an EMA of real/estimated and applies * it as the store's global multiplier, persisted across restarts. The * per-class rates carry the shape; this carries the residual level. */ @@ -13898,21 +13916,30 @@ export class AutobiographicalStrategy implements ResettableStrategy { // exhausting the picker on every wake. One sample per compile, always. if (!this._calibrationArmed) return; this._calibrationArmed = false; + // Declared: the caller passed what the request sent besides the window + // and the compile's injections ({} included). A JS caller's null isn't. + const declared = overhead != null; // The overhead in the window's units (current calibrated), so the implied // multiplier below is real / raw(window + overhead) and doesn't depend on // the multiplier it's measured at: one sample states it. const overheadTokens = overhead ? this.priceRequestOverhead(overhead) : 0; - const est = window + overheadTokens; + const injected = this._lastCompileInjected; + const est = window + injected + overheadTokens; + const parts = [ + ...(overheadTokens > 0 ? [`${Math.round(overheadTokens / 1000)}k system+tools`] : []), + ...(injected > 0 ? [`${Math.round(injected / 1000)}k injected`] : []), + ]; const counts = `(est=${Math.round(est / 1000)}k` + - `${overheadTokens > 0 ? ` incl. ${Math.round(overheadTokens / 1000)}k system+tools` : ''}` + + `${parts.length > 0 ? ` incl. ${parts.join(', ')}` : ''}` + ` real=${Math.round(realTotal / 1000)}k)`; const ratio = realTotal / est; const current = this._calibration; const observed = ratio * current; // back out the multiplier already applied // SANITY BAND: a representative sample sits near 1. Anything wilder is a // structural mismatch (a request we didn't compile, a partial compile, a - // provider quirk) — never evidence about chars-per-token. Log, don't learn. + // provider quirk) — never evidence about chars-per-token. Log, don't learn, + // except a declared consistent run (below). // The band is also checked on the IMPLIED raw multiplier: when the // multiplier sits at a clamp edge because the per-class rates were wrong // (signed thinking at a flat 600) and the rates are then fixed, every @@ -13926,8 +13953,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the same number, so a store whose true ratio is past the band (every // epoch discard restarts there) rejects every sample and never learns. // A single wild sample is still noise, but a run of them that agree with - // each other is a ruler that's off, not a request we didn't compile: the - // third in a row is learned, outright, which lands it on the clamp edge. + // the run's first, within OUT_OF_BAND_RUN_AGREEMENT, is a ruler that's + // off, not a request we didn't compile: the third in a row is learned, + // outright, which lands it on the clamp edge. Agreement is with the first, + // not the one before, so a drifting run (a growing unpriced part) isn't + // learned as a constant. // Only for samples whose caller declared what it sent besides the window: // undeclared, a consistent ratio past the band is what tool schemas and a // system prompt look like, and learning it would fold the window to the @@ -13936,18 +13966,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (!ratioInBand && !observedInBand) { const run = this._outOfBandRun; this._outOfBandRun = !overhead ? null - : run && Math.abs(observed / run.observed - 1) <= OUT_OF_BAND_RUN_AGREEMENT - ? { observed, count: run.count + 1 } - : { observed, count: 1 }; + : run && Math.abs(observed / run.first - 1) <= OUT_OF_BAND_RUN_AGREEMENT + ? { first: run.first, count: run.count + 1 } + : { first: observed, count: 1 }; const count = this._outOfBandRun?.count ?? 0; if (count < OUT_OF_BAND_RUN_LENGTH) { this.diag.error( `[estimator-calibration] REJECTED out-of-band sample real/est=${ratio.toFixed(2)} ${counts} — ` + - `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}` + (overhead - ? ` (${count} of ${OUT_OF_BAND_RUN_LENGTH} consistent samples to learn from)` - : `; if every sample reads about the same, the caller may be leaving out the system prompt ` + - `and tools it sends (reportRealInputTokens' second argument)`), + ? `multiplier stays ${this._calibration.toFixed(2)} until ${OUT_OF_BAND_RUN_LENGTH} samples in a row ` + + `agree within ${OUT_OF_BAND_RUN_AGREEMENT * 100}% (${count} of ${OUT_OF_BAND_RUN_LENGTH})` + : `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}; if every sample ` + + `reads about the same, the caller may be leaving out the system prompt and tools it sends ` + + `(reportRealInputTokens' second argument)`), ); return; } @@ -13957,6 +13988,44 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._outOfBandRun = null; } + // A multiplier learned before samples declared their overhead learned it + // as a rate (#323). Inside the band, the first declared sample on such a + // record is held, and the next declared sample decides: if the two agree + // within OUT_OF_BAND_RUN_AGREEMENT their mean replaces the multiplier, + // instead of about ten EMA steps, each a reprice of every message and + // summary; if not, the EMA steps as for any sample. Two, not one: this + // reaches every store that samples after the upgrade, those whose + // multiplier is already right included, and one wild sample shouldn't + // set their ruler alone. The held sample is in memory only, so a restart + // between them costs one more sample; an undeclared sample between them + // leaves it, since an implied multiplier doesn't depend on the multiplier + // in force. Either way the record then says declared samples agree with + // its multiplier (declaredOverhead), and an undeclared write takes that + // back. A provisional multiplier, or a consistent run (out of the band, + // three samples), is replaced by one sample as before. + const markDeclared = declared && !this._calibrationDeclared; + let replacement: number | null = null; + if (markDeclared && !this._calibrationProvisional && !fromRun) { + const candidate = this._declaredCandidate; + if (candidate === null) { + this._declaredCandidate = observed; + this.diag.error( + `[estimator-calibration] real/est=${ratio.toFixed(2)} implies ${observed.toFixed(2)}: the first sample ` + + `with its overhead declared, held for the next; if they agree within ` + + `${OUT_OF_BAND_RUN_AGREEMENT * 100}% their mean replaces the multiplier, and if not it steps; ` + + `multiplier stays ${current.toFixed(2)} ${counts}`, + ); + return; + } + this._declaredCandidate = null; + if (Math.abs(observed / candidate - 1) <= OUT_OF_BAND_RUN_AGREEMENT) replacement = (candidate + observed) / 2; + } else if (declared) { + // A declared sample that moves the ruler another way (a provisional + // replacement, a run) or meets a marked record ends any held sample: + // it was measured against a ruler this one replaces. + this._declaredCandidate = null; + } + const alpha = 0.2; // slow EMA: one wild request shouldn't yank the ruler const ema = this._calibrationEma ?? current; // A provisional multiplier is a guess (see CALIBRATION_PRICING_EPOCH). This @@ -13966,7 +14035,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // would leave the next sample just as far out of band. const provisional = this._calibrationProvisional; this._calibrationProvisional = null; - const next = provisional || fromRun ? observed : ema + alpha * (observed - ema); + const next = replacement ?? (provisional || fromRun ? observed : ema + alpha * (observed - ema)); const clamped = Math.min(1.8, Math.max(0.6, next)); this._calibrationEma = clamped; // DEAD BAND. The multiplier is applied only once the EMA has moved more @@ -13985,6 +14054,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { `[estimator-calibration] real/est=${ratio.toFixed(2)} confirms the provisional multiplier ` + `${current.toFixed(2)} ${counts}; stamped`, ); + if (declared) this._calibrationDeclared = true; + this.persistCalibration(); + } else if (markDeclared) { + this._calibrationDeclared = true; this.persistCalibration(); } return; @@ -13992,10 +14065,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.diag.error( `[estimator-calibration] real/est=${ratio.toFixed(2)} ` + `multiplier ${current.toFixed(2)} -> ${clamped.toFixed(2)}` + - `${fromRun ? ` (${OUT_OF_BAND_RUN_LENGTH} consistent out-of-band samples: learned at the clamp)` : provisional ? ' (provisional: replaced, not stepped)' : ''} ` + + `${fromRun ? ` (${OUT_OF_BAND_RUN_LENGTH} consistent out-of-band samples: learned at the clamp)` : provisional ? ' (provisional: replaced, not stepped)' : replacement !== null ? ' (two samples with their overhead declared agree: replaced by their mean, not stepped)' : ''} ` + counts, ); this._calibration = clamped; + this._calibrationDeclared = declared; this.applyCalibration(); this.persistCalibration(); } @@ -14024,6 +14098,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { at: Date.now(), pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, ...(this._calibrationReset ? { reset: this._calibrationReset } : {}), + ...(this._calibrationDeclared ? { declaredOverhead: true } : {}), }); } catch { /* persistence is best-effort */ } } @@ -14167,10 +14242,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { private _calibration = 1; /** The EMA itself; `_calibration` is its last applied value (see the dead band). */ private _calibrationEma: number | null = null; - /** The out-of-band samples rejected in a row that agree with each other, - * and the last one's implied multiplier (see reportRealInputTokens). In - * memory only: a restart counts afresh. */ - private _outOfBandRun: { observed: number; count: number } | null = null; + /** The out-of-band samples rejected in a row that agree with the first, + * and the first one's implied multiplier (see reportRealInputTokens). In + * memory only: a restart, or a branch's re-initialisation + * (clearBranchMirrors), counts afresh. */ + private _outOfBandRun: { first: number; count: number } | null = null; + /** The implied multiplier of the first declared sample on a record whose + * multiplier wasn't learned from declared samples, held for the next + * declared sample (see reportRealInputTokens). In memory only. */ + private _declaredCandidate: number | null = null; + /** Whether declared samples agree with the multiplier in force: set by + * one, or confirmed by two inside the dead band. Persisted as the record's + * declaredOverhead (see reportRealInputTokens). */ + private _calibrationDeclared = false; private _calibrationLoaded = false; /** Set when an older-epoch multiplier was discarded; persisted with the record. */ private _calibrationReset: CalibrationReset | null = null; @@ -14372,9 +14456,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibration = 1; this._calibrationReset = null; this._calibrationProvisional = null; + this._calibrationDeclared = false; try { const saved = this.store?.getStateJson(this.calibrationStateId) as - { multiplier?: number; pricing?: number; reset?: CalibrationReset; provisional?: boolean } | null; + { multiplier?: number; pricing?: number; reset?: CalibrationReset; provisional?: boolean; declaredOverhead?: boolean } | null; if (saved && Number.isFinite(saved.multiplier)) { const current = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; const stamped = typeof saved.pricing === 'number' && Number.isFinite(saved.pricing); @@ -14389,6 +14474,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (epoch === current) { this._calibration = Math.min(1.8, Math.max(0.6, saved.multiplier!)); this._calibrationReset = isCalibrationReset(saved.reset) ? saved.reset : null; + this._calibrationDeclared = saved.declaredOverhead === true; // A discard's 1 that no sample has confirmed yet was persisted as // provisional, and stays a guess across the restart: the first // sample still sets it, instead of stepping toward it. @@ -14409,6 +14495,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // still a guess. this._calibration = Math.min(1.8, Math.max(0.6, saved.multiplier!)); this._calibrationReset = isCalibrationReset(saved.reset) ? saved.reset : null; + this._calibrationDeclared = saved.declaredOverhead === true; if (saved.provisional === true) { this._calibrationProvisional = { fromEpoch: this._calibrationReset?.fromEpoch ?? null }; } @@ -14429,6 +14516,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { pricing: current, ...(this._calibrationReset ? { reset: this._calibrationReset } : {}), ...(this._calibrationProvisional ? { provisional: true } : {}), + ...(this._calibrationDeclared ? { declaredOverhead: true } : {}), }); } catch { /* persistence is best-effort */ } } @@ -19286,11 +19374,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { transitionBlocked: this.transitionBlocked, lastRenderStats: this._lastRenderStats, lastCompileEstimate: this._lastCompileEstimate, + lastCompileInjected: this._lastCompileInjected, calibrationArmed: this._calibrationArmed, calibrationLoaded: this._calibrationLoaded, calibration: this._calibration, calibrationReset: this._calibrationReset, calibrationProvisional: this._calibrationProvisional, + calibrationDeclared: this._calibrationDeclared, storeCalibration: store.getTokenCalibration?.(), storeView: this._storeView, storeOrder: this._storeOrder, @@ -19313,11 +19403,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.transitionBlocked = saved.transitionBlocked; this._lastRenderStats = saved.lastRenderStats; this._lastCompileEstimate = saved.lastCompileEstimate; + this._lastCompileInjected = saved.lastCompileInjected; this._calibrationArmed = saved.calibrationArmed; this._calibrationLoaded = saved.calibrationLoaded; this._calibration = saved.calibration; this._calibrationReset = saved.calibrationReset; this._calibrationProvisional = saved.calibrationProvisional; + this._calibrationDeclared = saved.calibrationDeclared; if (saved.storeCalibration !== undefined) store.setTokenCalibration?.(saved.storeCalibration); this._storeView = saved.storeView; this._storeOrder = saved.storeOrder; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 45b643a8..813f42c3 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -370,6 +370,14 @@ export interface SelectOptions { multiplier: number; expiresAt: number; }; + /** The content of each injection the compile inserts among the selected + * entries after the select (`beforeUser` and `afterUser`), which the + * request carries and the select's window doesn't. A strategy whose + * calibration compares the provider's input total with the compile prices + * it there (AutobiographicalStrategy.reportRealInputTokens). + * ContextManager.compile fills it from its injections; `system` ones go + * into the caller's system prompt instead. */ + messageInjections?: ReadonlyArray; } /** @@ -667,10 +675,13 @@ export function isSummaryOverviewStrategy(s: ContextStrategy): s is SummaryOverv } /** - * What a request carried besides the compiled window, as the caller sent it - * (see `AutobiographicalStrategy.reportRealInputTokens`). The provider counts - * it in the input total and the compile estimate doesn't, so a calibration - * sample that leaves it out learns it as a per-token rate. + * What a request carried besides the compiled window and the compile's + * message-position injections (which the compile prices itself), as the + * caller sent it (see `AutobiographicalStrategy.reportRealInputTokens`). The + * provider counts it in the input total and the compile estimate doesn't, so + * a calibration sample that leaves it out learns it as a per-token rate. + * Passing it says this is everything the request sent besides those; a + * caller that can't see the request's head passes nothing. */ export interface RequestOverhead { /** The system prompt the request sent. */ diff --git a/test/calibration-overhead.test.ts b/test/calibration-overhead.test.ts index 9bd493ca..8f1d1bfd 100644 --- a/test/calibration-overhead.test.ts +++ b/test/calibration-overhead.test.ts @@ -16,6 +16,8 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { Membrane, MockAdapter, NativeFormatter, type ContentBlock, type ToolDefinition } from '@animalabs/membrane'; +import type { ContextInjection } from '../src/types/index.js'; +import type { RequestOverhead } from '../src/index.js'; import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; import type { StoredMessage } from '../src/types/index.js'; @@ -82,8 +84,11 @@ describe('estimator calibration and request overhead (#323)', () => { const overhead = overheadPrice(cm, SYSTEM, tools); assert.ok(overhead > window * 0.4 && overhead < window * 0.7, `overhead ${overhead} vs window ${window}`); // Undeclared, this sample reads real/est ≈ 1.5: in band, so it would be - // learned as a per-token rate 50% above the truth. + // learned as a per-token rate 50% above the truth. Two samples, since a + // store's first declared sample is held until a second agrees. strategy.reportRealInputTokens(window + overhead, { system: SYSTEM, tools }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overhead, { system: SYSTEM, tools }); assert.equal(internals._calibration, 1); cm.close(); }); @@ -194,3 +199,419 @@ describe('estimator calibration and request overhead (#323)', () => { cm.close(); }); }); + +/** The store's price of some content, at its current multiplier. */ +function priceOf(cm: ContextManager, content: ContentBlock[]): number { + return (cm as unknown as { messageStore: { estimateTokens(m: StoredMessage): number } }).messageStore.estimateTokens({ content } as unknown as StoredMessage); +} + +/** An injection at a position, of about `tokens` estimated tokens. */ +function injectionOf(position: 'beforeUser' | 'afterUser' | 'system', tokens: number): ContextInjection { + return { namespace: 'probe', position, content: [{ type: 'text', text: 'Retrieved knowledge. '.repeat(Math.ceil(tokens / 4)) }] }; +} + +type DeclaredInternals = Internals & { _calibrationDeclared: boolean; _calibrationArmed: boolean }; +const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + +async function seeded(name: string, record: Record) { + const first = await openManager(name); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, record); + first.cm.close(); + const opened = await openManager(name); + await opened.cm.compile(BUDGET); + return { ...opened, internals: opened.internals as DeclaredInternals }; +} + +/** Compile, then one sample at implied multiplier k, with system and tools declared or not. */ +async function sampleAt(cm: ContextManager, strategy: AutobiographicalStrategy, internals: Internals, k: number, declared: boolean): Promise { + await cm.compile(BUDGET); + const m = internals._calibration; + const tools = toolsOf(internals._lastCompileEstimate / m / 2); + const rawWindow = internals._lastCompileEstimate / m; + const rawOverhead = overheadPrice(cm, SYSTEM, tools) / m; + strategy.reportRealInputTokens(k * (rawWindow + (declared ? rawOverhead : 0)), declared ? { system: SYSTEM, tools } : undefined); +} + +const declaredOf = (internals: Internals) => + (internals.store!.getStateJson(internals.calibrationStateId) as { declaredOverhead?: boolean } | null)?.declaredOverhead; + +describe('estimator calibration: what the compile adds among the messages', () => { + /** Two compiles with the injection, each sampled with system, tools and the injection in the provider's count. */ + async function twoSamples(name: string, injection: (window: number) => ContextInjection) { + const { cm, strategy, internals } = await openManager(name); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const inj = injection(window); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, [inj]); + assert.equal(internals._lastCompileEstimate, window, 'the window estimate leaves the injection out'); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, inj.content), { system: SYSTEM, tools }); + } + return { cm, internals, window, inj }; + } + + it('a message-position injection is priced into the sample: in band, it is not learned as a per-token rate', async () => { + const { cm, internals, window, inj } = await twoSamples('injected-in-band', (w) => injectionOf('afterUser', w / 2)); + // Premise: priced as the window and the declared head alone, these samples read about 1.33. + assert.ok(priceOf(cm, inj.content) > window * 0.4, `injection ${priceOf(cm, inj.content)} vs window ${window}`); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('a large injection on a small window is not learned at the clamp', async () => { + const { cm, strategy, internals } = await openManager('injected-out-of-band'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('beforeUser', window * 3); + const tools = toolsOf(window / 2); + for (let i = 0; i < 4; i++) { + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'the samples were taken in band'); + cm.close(); + }); + + it('a dry compile between the compile and its sample leaves the compile\'s injection priced', async () => { + const { cm, strategy, internals } = await openManager('injected-dry'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, [injection]); + await cm.compile(BUDGET, undefined, { dryRun: true }); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('a compile without injections prices none, after one with', async () => { + const { cm, strategy, internals } = await openManager('injected-then-none'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + const tools = toolsOf(window / 2); + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('a system-position injection is priced once, as part of the declared system prompt', async () => { + const { cm, strategy, internals } = await openManager('injected-system'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('system', window / 2); + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + const result = await cm.compile(BUDGET, [injection]); + // The caller appends systemInjections to its system prompt, and declares that. + const system = `${SYSTEM}\n${result.systemInjections.map((b) => (b as { text: string }).text).join('\n')}`; + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, tools), { system, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('a dry compile with injections leaves the committing compile\'s count', async () => { + const { cm, strategy, internals } = await openManager('dry-with-injections'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET); + await cm.compile(BUDGET, [injectionOf('afterUser', window / 2)], { dryRun: true }); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('several message-position injections are each priced', async () => { + const { cm, strategy, internals } = await openManager('two-injections'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injections = [injectionOf('beforeUser', window / 4), injectionOf('afterUser', window / 4)]; + const tools = toolsOf(window / 2); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET, injections); + const injected = injections.reduce((sum, inj) => sum + priceOf(cm, inj.content), 0); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + injected, { system: SYSTEM, tools }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); + + it('an injection is priced at the multiplier in force, as the window is', async () => { + const { cm, strategy, internals } = await seeded('injected-calibrated', { multiplier: 1.5, at: 1, pricing: epoch, declaredOverhead: true }); + assert.equal(internals._calibration, 1.5); + const tools = toolsOf(internals._lastCompileEstimate / 2); + const injection = injectionOf('afterUser', internals._lastCompileEstimate / 2); + await cm.compile(BUDGET, [injection]); + // A provider at k = 1.5: the window, the head and the injection, each at the store's price. + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); + assert.equal(internals._calibration, 1.5); + assert.equal((internals as unknown as { _calibrationEma: number | null })._calibrationEma, 1.5, 'the sample was taken'); + cm.close(); + }); + + it('an undeclared sample counts the injections too', async () => { + const { cm, strategy, internals } = await openManager('injected-undeclared'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const injection = injectionOf('afterUser', window / 2); + await cm.compile(BUDGET, [injection]); + strategy.reportRealInputTokens(window + priceOf(cm, injection.content)); + assert.equal(internals._calibration, 1); + assert.equal((internals as unknown as { _calibrationEma: number | null })._calibrationEma, 1, 'the sample was taken'); + cm.close(); + }); + + it('a system prompt alone is priced into the sample', async () => { + const { cm, strategy, internals } = await openManager('system-only'); + await cm.compile(BUDGET); + const window = internals._lastCompileEstimate; + const system = `You are a resident. ${'Keep to your purpose. '.repeat(Math.ceil(window / 14))}`; + // Premise: left out, the samples read 1.25 to 1.7, inside the band, where they'd be learned. + const share = overheadPrice(cm, system, []) / window; + assert.ok(share > 0.25 && share < 0.7, `the system prompt is ${share} of the window`); + for (let i = 0; i < 2; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, []), { system }); + } + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + cm.close(); + }); +}); + +describe('estimator calibration: consistent out-of-band runs', () => { + /** Compile, then one declared sample at `ratio` of the compile's estimate. */ + async function oob(cm: ContextManager, strategy: AutobiographicalStrategy, internals: Internals, ratio: number, declared = true) { + await cm.compile(BUDGET); + const nothingBesides: RequestOverhead = {}; // the package's type, from its root + strategy.reportRealInputTokens(internals._lastCompileEstimate * ratio, declared ? nothingBesides : undefined); + } + + it('an undeclared out-of-band sample ends a declared run', async () => { + const { cm, strategy, internals } = await openManager('undeclared-ends-run'); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1, false); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8, 'a fresh run of three is learned'); + cm.close(); + }); + + it('a run agrees with its first sample: within 10% it is learned, and a drift past 10% of the first is not', async () => { + const drift = await openManager('run-drift'); + for (const ratio of [2.0, 2.18, 2.38]) await oob(drift.cm, drift.strategy, drift.internals, ratio); + assert.equal(drift.internals._calibration, 1, 'each within 10% of the one before, the third 19% past the first'); + drift.cm.close(); + const near = await openManager('run-near'); + for (const ratio of [2.0, 2.18, 2.08]) await oob(near.cm, near.strategy, near.internals, ratio); + assert.equal(near.internals._calibration, 1.8, 'each within 10% of the first'); + near.cm.close(); + }); + + it('a consistent run below the band is learned at the floor', async () => { + const { cm, strategy, internals } = await openManager('run-low'); + for (let i = 0; i < 3; i++) await oob(cm, strategy, internals, 0.3); + assert.equal(internals._calibration, 0.6); + cm.close(); + }); + + it('one compile counts once toward a run, however many reports follow it', async () => { + const { cm, strategy, internals } = await openManager('run-one-arm'); + await cm.compile(BUDGET); + for (let i = 0; i < 3; i++) strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, {}); + assert.equal(internals._calibration, 1, 'three reports of one compile'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1, 'the second compile is the run\'s second sample'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8, 'the third compile is its third'); + cm.close(); + }); + + it('a branch switch starts a run afresh', async () => { + const { cm, strategy, internals } = await openManager('run-branch'); + await oob(cm, strategy, internals, 2.1); + await oob(cm, strategy, internals, 2.1); + await cm.fork('other'); + await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1); + assert.equal((internals as unknown as { _outOfBandRun: { count: number } | null })._outOfBandRun?.count, 1, 'the sample on the new branch starts a run'); + cm.close(); + }); +}); + +describe('estimator calibration: a store that learned the overhead as a rate', () => { + it('two declared samples that agree replace a multiplier learned without them, by their mean', async () => { + const { cm, strategy, internals } = await seeded('learned-undeclared', { multiplier: 1.8, at: 1, pricing: epoch }); + assert.equal(internals._calibration, 1.8); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'the first is held'); + assert.equal(declaredOf(internals), undefined); + await sampleAt(cm, strategy, internals, 1.04, true); + assert.ok(Math.abs(internals._calibration - 1.02) <= 0.01, `replaced by the mean, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true, 'and the record says its samples declare their overhead'); + cm.close(); + }); + + it('two declared samples that disagree step by the EMA, and mark the record declared', async () => { + const { cm, strategy, internals } = await seeded('learned-disagree', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1.5, true); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `stepped toward the second, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.592) <= 0.01, `then steps by the EMA, got ${internals._calibration}`); + cm.close(); + }); + + it('a branch switch between the two loses the held sample', async () => { + const { cm, strategy, internals } = await seeded('held-branch', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await cm.fork('other'); + await cm.compile(BUDGET); + assert.equal(internals._calibration, 1.8, 'the fork has the record'); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'held, not paired with the other branch\'s sample'); + assert.notEqual((internals as unknown as { _declaredCandidate: number | null })._declaredCandidate, null, 'and held afresh'); + cm.close(); + }); + + it('a consistent run between the two ends the held sample', async () => { + const { cm, strategy, internals } = await seeded('held-run', { multiplier: 1, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + for (let i = 0; i < 3; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, {}); + } + assert.equal(internals._calibration, 1.8, 'the run is learned'); + await sampleAt(cm, strategy, internals, 1.5, false); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `an undeclared sample steps and unmarks, got ${internals._calibration}`); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.74) <= 0.01, `held, not paired with the sample from before the run, got ${internals._calibration}`); + cm.close(); + }); + + it('a restart between the two loses the held sample: it starts over, and the record is not marked', async () => { + const first = await seeded('held-restart', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + first.cm.close(); + const { cm, strategy, internals } = await openManager('held-restart'); + await cm.compile(BUDGET); + const typed = internals as DeclaredInternals; + assert.deepEqual([internals._calibration, typed._calibrationDeclared, declaredOf(internals)], [1.8, false, undefined]); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'held again'); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.02, `then replaced, got ${internals._calibration}`); + cm.close(); + }); + + it('a multiplier learned from declared samples steps by the EMA', async () => { + const { cm, strategy, internals } = await seeded('learned-declared', { multiplier: 1.8, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('after the replacement, a restart steps by the EMA', async () => { + const first = await seeded('declared-restart', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + await sampleAt(first.cm, first.strategy, first.internals, 1, true); + first.cm.close(); + const { cm, strategy, internals } = await openManager('declared-restart'); + await cm.compile(BUDGET); + await sampleAt(cm, strategy, internals, 1.3, true); + assert.ok(Math.abs(internals._calibration - 1.06) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('an undeclared sample that moves the multiplier unmarks the record, and the next two declared replace it', async () => { + const { cm, strategy, internals } = await seeded('undeclared-after-declared', { multiplier: 1, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(cm, strategy, internals, 1.5, false); + assert.ok(internals._calibration > 1.05, `the undeclared sample stepped, got ${internals._calibration}`); + assert.equal(declaredOf(internals), undefined); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.02, `replaced, got ${internals._calibration}`); + cm.close(); + }); + + it('two declared samples that confirm the multiplier mark the record declared', async () => { + const { cm, strategy, internals } = await seeded('confirmed-declared', { multiplier: 1, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1); + assert.equal(declaredOf(internals), true); + await sampleAt(cm, strategy, internals, 1.5, true); + assert.ok(Math.abs(internals._calibration - 1.1) <= 0.01, `stepped, got ${internals._calibration}`); + cm.close(); + }); + + it('a provisional multiplier is replaced by one declared sample, and marked declared', async () => { + const { cm, strategy, internals } = await seeded('provisional-declared', { multiplier: 1, at: 1, pricing: epoch, provisional: true }); + await sampleAt(cm, strategy, internals, 1, true); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { provisional?: boolean; declaredOverhead?: boolean }; + assert.deepEqual([saved.provisional, saved.declaredOverhead], [undefined, true]); + cm.close(); + }); + + it('a dry compile after the load keeps a record\'s declared overhead', async () => { + const { cm, strategy, internals } = await seeded('declared-dry', { multiplier: 1.8, at: 1, pricing: epoch, declaredOverhead: true }); + await cm.compile(BUDGET, undefined, { dryRun: true }); + assert.equal(internals._calibrationDeclared, true); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `stepped, not held, got ${internals._calibration}`); + cm.close(); + }); + + it('an undeclared sample inside the dead band leaves a record marked, and a rejected declared one leaves it unmarked', async () => { + const marked = await seeded('deadband-undeclared', { multiplier: 1, at: 1, pricing: epoch, declaredOverhead: true }); + await sampleAt(marked.cm, marked.strategy, marked.internals, 1.01, false); + assert.deepEqual([marked.internals._calibration, marked.internals._calibrationDeclared, declaredOf(marked.internals)], [1, true, true]); + marked.cm.close(); + const unmarked = await seeded('rejected-declared', { multiplier: 1, at: 1, pricing: epoch }); + await unmarked.cm.compile(BUDGET); + unmarked.strategy.reportRealInputTokens(unmarked.internals._lastCompileEstimate * 2.1, {}); + assert.deepEqual([unmarked.internals._calibration, unmarked.internals._calibrationDeclared, declaredOf(unmarked.internals)], [1, false, undefined]); + unmarked.cm.close(); + }); + + it('a null overhead from a JS caller counts as undeclared', async () => { + const { cm, strategy, internals } = await seeded('null-overhead', { multiplier: 1, at: 1, pricing: epoch }); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.5, null as unknown as RequestOverhead); + assert.ok(Math.abs(internals._calibration - 1.1) <= 0.01, `stepped as undeclared, not held, got ${internals._calibration}`); + assert.deepEqual([internals._calibrationDeclared, declaredOf(internals)], [false, undefined]); + cm.close(); + }); + + it('a re-stamp keeps a record\'s declared overhead', async () => { + const { cm, internals } = await seeded('restamp-declared', { multiplier: 1.2, at: 1, pricing: epoch - 1, declaredOverhead: true }); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { pricing?: number; declaredOverhead?: boolean }; + assert.equal(saved.pricing, epoch, 're-stamped'); + assert.equal(saved.declaredOverhead, true); + assert.equal(internals._calibrationDeclared, true); + cm.close(); + }); +}); From b8a617a72dafb1b131bbb81d198d66689953c959 Mon Sep 17 00:00:00 2001 From: Cedric-2315 Date: Fri, 23 Oct 2026 08:18:41 -0400 Subject: [PATCH 29/67] docs(changelog): #322's recall change under Changed (#317) The fragment for #322 listed its two routes together under Fixed. The recall route changes what 0.15.0's entry told callers ("Only pins before the chunk being compressed count, so a summary after it is recalled as before"), so it moves to its own Changed fragment, old beside new, with the bullet that qualifies it. The head, removed messages, tick's deferral and the prompt cache stay under Fixed. No code changes. Co-authored-by: Claude Opus 5.5 --- changelog.d/317-compression-head-as-of.fixed.md | 6 ++---- changelog.d/317-compression-recall-before.changed.md | 4 ++++ 2 files changed, 6 insertions(+), 4 deletions(-) create mode 100644 changelog.d/317-compression-recall-before.changed.md diff --git a/changelog.d/317-compression-head-as-of.fixed.md b/changelog.d/317-compression-head-as-of.fixed.md index dec8a6a6..581fc94c 100644 --- a/changelog.d/317-compression-head-as-of.fixed.md +++ b/changelog.d/317-compression-head-as-of.fixed.md @@ -1,7 +1,5 @@ -- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request no longer gives the history it summarizes what came after it, ahead of it (#317). It came by two routes, in requests that carry history beside their chunk (a source-only request, `compressionSourceOnly` or `compressionMergeSourceOnly`, carries none): - - **The head:** a request for history from before the reset gave the new topic's opening as its head. It now gives the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. - - **Recall pairs:** a compression request recalled every summary on the frontier, so a chunk compressed after later history had been summarized was given those later summaries too. That's the old head's chunk, which a reset makes compressible late, or a pinned or quarantined chunk let go late. It now recalls only summaries that start before it, the rule merges already used; the later summaries' messages stay out of it as before. - - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. Summaries written before this fix are recalled as written, and one written after a reset may already carry the new topic. +- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request for history from before the reset no longer gets the new topic's opening as its head, ahead of the history it summarizes (#317). A source-only request (`compressionSourceOnly` or `compressionMergeSourceOnly`) carries no head, so this didn't reach it. The other way later history reached a compression request, its recall pairs, is under Changed. + - **The head it gets now:** the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. diff --git a/changelog.d/317-compression-recall-before.changed.md b/changelog.d/317-compression-recall-before.changed.md new file mode 100644 index 00000000..a0a375ca --- /dev/null +++ b/changelog.d/317-compression-recall-before.changed.md @@ -0,0 +1,4 @@ +- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). + - **Before:** it recalled every summary on the frontier, as 0.15.0's entry said: "Only pins before the chunk being compressed count, so a summary after it is recalled as before." So a chunk compressed after later history had been summarized was given those later summaries too, ahead of it. That's the old head's chunk, which a head-window reset makes compressible late, or a pinned or quarantined chunk let go late. + - **Now:** a summary that starts after the history being summarized isn't recalled, and its messages stay out of the request, as before. Pins still count only before the chunk. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. From ca9b9d0f4697e302e6e69f3df9fe74c142929e8e Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 08:12:38 -0400 Subject: [PATCH 30/67] fix(pins): a pin renders whole messages in the selects; a compile that can't hold them narrows it in memory A document mark on a long message stored in parts, made with its first part's id as connectome-host's panel makes it, kept only that part raw and let the rest fold. - pinSpan reads a pin in one of two ways. 'whole' runs from the first part of the message its first id is in to the last part of the message its last id is in. 'ids' covers its ids alone, as before. - The selects read pins whole under adaptive resolution (selectsPinReading), the only ingress that stores messages in parts, and by their ids otherwise. - The summarizer's prompt, its pinned-raw warning filter and requestCarriesToolBlocks read pins by their ids, as before: the prompt already has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. - Compression leaves out a position only when it is raw both by the pins' ids and as the selects read them (compressionPinnedPositions). A marked message's other parts are chunked and summarized as before. A mark's own part that another pin's whole reading holds at a level is summarized too, so kv-unified isn't left infeasible. - A select refused over budget with a non-empty middle narrows pins to their ids one at a time, the one that frees the most first, selecting again after each (selectNarrowingPins). - What a pin frees counts the shards its whole reading adds beyond its ids that compression can summarize and that have an L1 the select can fold them to, priced as the selects price them, and only where the selects resolve them raw and they aren't locked. - A narrowing that lets a select through is held for the strategy instance's life (narrowedPins, keyed by a pin's id, ends and creation time, so a reload of the store's state keeps it), with a warning per pin. Nothing records it, so a reopened store reads the pin whole. - When none fits, the pins go back and the first refusal is thrown with its preview. An error mid-loop puts them back too. A dry compile narrows for itself only. The cache-marker baseline is put back before each attempt. - Docs: ProtectedRange, pinRange and markDocument (strategy and manager), getCompressibleMessages, the README, and changelog.d/whole-body-pins. test/whole-body-pins.test.ts adds 30 tests. test/l1-recall-raw-pins-frontier.test.ts passes the 'whole' reading to the two internals it calls. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 3 + src/context-manager.ts | 11 +- src/strategies/autobiographical.ts | 324 ++++++++-- src/types/strategy.ts | 19 +- test/l1-recall-raw-pins-frontier.test.ts | 11 +- test/whole-body-pins.test.ts | 758 +++++++++++++++++++++++ 7 files changed, 1069 insertions(+), 59 deletions(-) create mode 100644 changelog.d/whole-body-pins.fixed.md create mode 100644 test/whole-body-pins.test.ts diff --git a/README.md b/README.md index 1f125ebc..4e5affe4 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts sends the whole message raw where the budget holds it, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows it, says so in a warning, and keeps it that way until the manager is opened again. `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md new file mode 100644 index 00000000..56a39e1a --- /dev/null +++ b/changelog.d/whole-body-pins.fixed.md @@ -0,0 +1,3 @@ +- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. + - **When the whole messages don't fit.** A compile refused over budget narrows pins back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed while the strategy instance runs, and a warning names it and says how to have it whole again. Nothing is stored: a store opened again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. + - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on; under kv-unified, whose solver weighs the whole request, the rewrite can start earlier. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/src/context-manager.ts b/src/context-manager.ts index ea298d8c..774d9c02 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1639,8 +1639,9 @@ export class ContextManager { /** * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel`. A raw pin, with no bound or a bound of 0, keeps the - * messages out of compression and renders them raw at their original - * chronological position. A bound above 0 is read only under + * messages it names out of compression and renders them raw at their + * original chronological position, a message stored in parts whole, as + * `ProtectedRange` describes. A bound above 0 is read only under * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution`: there the range is still compressed, since only a * summary can supply its level, and is held at `level`, or no deeper than @@ -1667,7 +1668,7 @@ export class ContextManager { * it renders where one does, or raw (`ProtectedRange.level`). Honored * by `foldingStrategy: 'kv-stable'` and `'kv-unified'` with adaptive * resolution; under any other selector the range is raw, in the live view - * and in the canonical L1 compression prompt where its budgets allow (see + * and, read by its ids, in the canonical L1 compression prompt where its budgets allow (see * `compressionRecallBudgetTokens`). Returns the new pin id. */ pinAtLevel(firstMessageId: MessageId, lastMessageId: MessageId, level: number, opts?: { name?: string }): string { @@ -1681,7 +1682,9 @@ export class ContextManager { * `kind: 'document'`, levels included (see `pinRange`): without a `level` * or `maxLevel` the message is kept raw, while * `markDocument(id, { maxLevel })` lets it fold no deeper than that level - * where the selector reads level bounds. + * where the selector reads level bounds. Under adaptive resolution, a mark + * on any part of a message stored in parts renders the whole message, as + * `pinRange` does. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b76c6b28..4da013d0 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2154,6 +2154,20 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected pins: ProtectedRange[] = []; /** Monotonically increasing counter for pin ids. Persisted as part of the pins snapshot. */ protected pinIdCounter = 0; + /** + * Pins a compile narrowed (`selectNarrowingPins`), by `narrowKey`: their + * whole reading (`pinSpan`) covers only their own ids, for as long as this + * instance runs, a reload of the store's state included. Nothing records + * the narrowing itself, so a store opened again reads every pin whole; what + * a narrowed compile sent is recorded as any compile's is. A pin unpinned + * and marked again is a new pin, and whole. + */ + protected readonly narrowedPins = new Set(); + + /** A pin's key in `narrowedPins`: its id, ends and creation time, which no API changes. */ + protected narrowKey(pin: ProtectedRange): string { + return JSON.stringify([pin.id, pin.firstMessageId, pin.lastMessageId, pin.created]); + } /** * Per-message resolution state for the adaptive-resolution picker. @@ -3512,8 +3526,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel` (`normalizePinLevels`). A raw pin, with no bound or a - * bound of 0, keeps the messages out of compression and renders them raw - * at their original position. A bound above 0 is read only under + * bound of 0, keeps the messages it names out of compression and renders + * them raw at their original position, a message stored in parts whole + * (`pinSpan`): where the whole messages don't fit the budget but the rest + * does with the pin narrowed to its ids, a compile narrows it while this + * instance runs (`selectNarrowingPins`). A bound above 0 is read only under * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution` (`readsPinBounds`): there the range is still * compressed, since only a summary can supply its level, and is held at @@ -3545,6 +3562,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * with `kind: 'document'`, and `opts` reads as `pinRange`'s: without a * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. + * Under adaptive resolution, a mark on any part of a message stored in + * parts renders the whole message, as `pinRange` does. */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -3569,8 +3588,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * summary at `level` doesn't list it renders where one does, or raw * (`ProtectedRange.level`). Honored by `foldingStrategy: 'kv-stable'` * and `'kv-unified'` with adaptive resolution; under any other selector the - * range is raw (`resolvedPinBounds`), in the live view and in the canonical - * L1 compression prompt where its budgets allow (see + * range is raw (`resolvedPinBounds`), in the live view and, read by its + * ids, in the canonical L1 compression prompt where its budgets allow (see * `compressionRecallBudgetTokens`). * Equivalent to `pinRange(first, last, { level })`. */ @@ -3762,21 +3781,62 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * Build a set of message-store positions covered by any pin. O(N pins · K range). - * Returns positions for which the message exists; orphan pins (deleted - * messages) are silently skipped. + * The store positions a pin covers, first and last inclusive, in one of its + * two readings (`ProtectedRange`): + * - `'ids'`: the messages from its first id to its last, a part of a + * message stored in parts covering only itself. The summarizer's prompt + * reads this, as do its pinned-raw warning and the count of queued work + * that predicts it (`requestCarriesToolBlocks`): the prompt already has a + * marked message's other parts as summaries, and carrying them raw would + * multiply every later compression request's input. Compression keeps + * out only parts raw this way (`compressionPinnedPositions`). + * - `'whole'`: from the first part of the message its first id is in to the + * last part of the message its last id is in. The selects read this under + * adaptive resolution (`selectsPinReading`), so the whole message goes out + * raw where the budget holds it. A pin narrowed in this instance + * (`narrowedPins`) reads by its ids here too. + * Undefined when either id is no longer in the store. + */ + protected pinSpan( + pin: ProtectedRange, + positionOf: ReadonlyMap, + messages: readonly StoredMessage[], + reading: 'ids' | 'whole', + ): { lo: number; hi: number } | undefined { + const first = positionOf.get(pin.firstMessageId); + const last = positionOf.get(pin.lastMessageId); + if (first === undefined || last === undefined) return undefined; + const lo = Math.min(first, last); + const hi = Math.max(first, last); + if (reading === 'ids' || this.narrowedPins.has(this.narrowKey(pin))) return { lo, hi }; + return { lo: bodyBoundsIn(messages, lo).from, hi: bodyBoundsIn(messages, hi).to }; + } + + /** + * The reading the selects take (`pinSpan`), the agent's view, with what + * must agree with them (the narrowing's candidates, and what compression + * keeps out): `'whole'` under adaptive resolution, the only ingress that + * stores a message in parts, and `'ids'` otherwise, so a store written with + * it and opened without it reads its pins as before. The summarizer's prompt + * reads pins by their ids either way. */ - protected pinnedPositions(messages: StoredMessage[]): Set { + protected selectsPinReading(): 'ids' | 'whole' { + return this.config.adaptiveResolution ? 'whole' : 'ids'; + } + + /** + * Build a set of message-store positions covered by any pin, in `reading` + * (`pinSpan`). O(N pins · K range). Returns positions for which the message + * exists; orphan pins (deleted messages) are silently skipped. + */ + protected pinnedPositions(messages: StoredMessage[], reading: 'ids' | 'whole'): Set { if (this.pins.length === 0) return new Set(); const positionOf = this.positionIndex(messages); const out = new Set(); for (const pin of this.pins) { - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) continue; - const lo = Math.min(first, last); - const hi = Math.max(first, last); - for (let i = lo; i <= hi; i++) out.add(i); + const span = this.pinSpan(pin, positionOf, messages, reading); + if (!span) continue; + for (let i = span.lo; i <= span.hi; i++) out.add(i); } return out; } @@ -3791,13 +3851,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { * kv-stable's reading applies: a bound there takes the place of a classic * pin, a fixed level takes the place of a cap, and within each kind the * lowest wins (`pinLevelBounds`). A level or cap of 0 is raw - * (`isRawPinBound`). + * (`isRawPinBound`). Pins are read in `reading` (`pinSpan`): the summarizer's + * prompt by their ids, the selects as whole messages, and compression keeps + * out what both keep raw. */ - protected resolvedPinBounds(messages: StoredMessage[]): Map { + protected resolvedPinBounds(messages: StoredMessage[], reading: 'ids' | 'whole'): Map { const out = new Map(); - const pinned = this.pinnedPositions(messages); + const pinned = this.pinnedPositions(messages, reading); if (pinned.size === 0) return out; - const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; + const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages, reading) : undefined; for (const position of pinned) { const bound = bounds?.get(position); if (bound?.level !== undefined) out.set(position, { level: bound.level }); @@ -3821,10 +3883,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { return 'level' in bound ? bound.level === 0 : bound.maxLevel === 0; } - /** Positions of the messages that pins keep raw (`resolvedPinBounds`). */ - protected forceRawPinPositions(messages: StoredMessage[]): Set { + /** Positions of the messages that pins keep raw, in `reading` (`resolvedPinBounds`). */ + protected forceRawPinPositions(messages: StoredMessage[], reading: 'ids' | 'whole'): Set { const out = new Set(); - for (const [position, bound] of this.resolvedPinBounds(messages)) { + for (const [position, bound] of this.resolvedPinBounds(messages, reading)) { if (AutobiographicalStrategy.isRawPinBound(bound)) out.add(position); } return out; @@ -3835,21 +3897,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `maxLevel`) by message position. Only pins that carry a bound appear; a * classic raw pin (no bound) is absent here and handled by `pinnedPositions`. * Where bounded pins overlap, the lowest fixed level and the lowest cap are - * kept; `resolvedPinBounds` decides between them and a classic pin. + * kept; `resolvedPinBounds` decides between them and a classic pin. Pins + * are read in `reading` (`pinSpan`). */ - protected pinLevelBounds(messages: StoredMessage[]): Map { + protected pinLevelBounds(messages: StoredMessage[], reading: 'ids' | 'whole'): Map { const out = new Map(); if (this.pins.length === 0) return out; const positionOf = this.positionIndex(messages); for (const pin of this.pins) { if (pin.level === undefined && pin.maxLevel === undefined) continue; - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) continue; - const lo = Math.min(first, last); - const hi = Math.max(first, last); - for (let i = lo; i <= hi; i++) { + const span = this.pinSpan(pin, positionOf, messages, reading); + if (!span) continue; + for (let i = span.lo; i <= span.hi; i++) { const prev = out.get(i) ?? {}; // A fixed level is the strongest constraint; when two pins fix the same // position, the shallower (lower) level wins (finest requirement). @@ -7941,6 +8001,151 @@ export class AutobiographicalStrategy implements ResettableStrategy { } private selectPath(store: MessageStoreView, budget: TokenBudget, opts?: SelectOptions): ContextEntry[] { + const cacheKeys = this._prevCacheKeys; + try { + return this.selectOnce(store, budget, opts); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + return this.selectNarrowingPins(store, budget, opts, err, cacheKeys); + } + } + + /** + * Answers a select refused over budget (`refusal`) where the whole messages + * pins cover may be what doesn't fit: a mark on a long message's first part + * keeps the whole message raw (`pinSpan`), which can be more than the + * budget holds beside the rest. Pins are narrowed to their own ids one at a + * time, the one that frees the most first (`freedByNarrowing`), and the + * select runs again after each. Compression reads pins by their ids, so the + * parts a narrowing frees are chunked and summarized like any history, and + * the narrowed select can send their summaries. + * - A refusal whose middle holds nothing, as when the head and recent + * windows alone are over the budget, stands as it was: nothing is + * narrowed and no select runs again. + * - When a select goes ahead, the pins narrowed so far stay narrowed while + * this instance runs (`narrowedPins`), and a warning names each. + * Nothing records them, so a store opened again reads every pin whole: with + * a larger budget it sends the whole message, and one that still doesn't + * fit narrows again at its first refused compile. + * - When no pin is left to narrow, the pins go back to whole messages and + * the first refusal is thrown, with its preview. + * - A dry compile narrows for its own select only: it puts the pins back + * and warns nothing. + */ + private selectNarrowingPins( + store: MessageStoreView, + budget: TokenBudget, + opts: SelectOptions | undefined, + refusal: OverBudgetError, + cacheKeys: string[] | undefined, + ): ContextEntry[] { + if (!(refusal.diagnostics.middleTokens > 0)) throw refusal; + const preview = this._lastPreview; + const narrowed: ProtectedRange[] = []; + const putBack = (): void => { + for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); + }; + let entries: ContextEntry[] | undefined; + try { + // Narrowing a raw pin changes neither another pin's reading nor what + // compression leaves out (a part it stops covering stays raw as the + // selects read it only where another raw pin's ids keep it so), so the + // order taken here holds throughout. + for (const { pin } of this.freedByNarrowing(store)) { + this.narrowedPins.add(this.narrowKey(pin)); + narrowed.push(pin); + // A refused select can have placed its cache markers; each attempt is + // measured against the request before the compile, as select puts back. + this._prevCacheKeys = cacheKeys; + try { + entries = this.selectOnce(store, budget, opts); + break; + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + } + } + } catch (err) { + putBack(); + throw err; + } + if (!entries) { + putBack(); + this._lastPreview = preview; + throw refusal; + } + if (opts?.dryRun === true) putBack(); + else this.warnNarrowedPins(narrowed, refusal); + return entries; + } + + /** + * What narrowing each raw pin would free, the most first: the raw tokens of + * the messages its reading as the selects take it (`selectsPinReading`) + * covers beyond its own ids that compression can summarize + * (`getCompressibleMessages`: outside the head and recent windows) and that + * have an L1 the select can fold them to (their chunk's summary, else + * `l1ByMessageIndex`, as selectAdaptive finds it), priced as the selects + * price them (`postStripEstimates`), less those the selects hold at a + * level or that are locked. Only those can go out as summaries once it's + * narrowed. Where the selects read bounds, a pin with a level or cap above + * 0 frees nothing, since its parts are held at that level: its parts + * already fold, and where it overlaps a raw pin, narrowing it could turn + * parts raw. Elsewhere every pin renders raw and can be narrowed. A pin + * that frees nothing, a narrowed one included, isn't listed. + */ + private freedByNarrowing(store: MessageStoreView): Array<{ pin: ProtectedRange; tokens: number }> { + if (this.pins.length === 0) return []; + const messages = store.getAll(); + const positionOf = this.positionIndex(messages); + const compressible = new Set(this.getCompressibleMessages(store).map((m) => m.id)); + const summarized = new Set(this.l1ByMessageIndex().keys()); + for (const chunk of this.chunks) if (chunk.summaryId !== undefined) for (const m of chunk.messages) summarized.add(m.id); + const reading = this.selectsPinReading(); + // A pin keeps raw only where the selects resolve it raw: where they read + // bounds, a position a pin holds at a level (this one's own, if it has a + // level or cap) stays held, and a locked one stays as it is, whether or + // not this pin covers it. So a pin with a level or cap frees nothing there. + const bounds = this.resolvedPinBounds(messages, reading); + const prices = this.postStripEstimates(store); + const out: Array<{ pin: ProtectedRange; tokens: number }> = []; + for (const pin of this.pins) { + const whole = this.pinSpan(pin, positionOf, messages, reading); + const ids = this.pinSpan(pin, positionOf, messages, 'ids'); + if (!whole || !ids) continue; + let tokens = 0; + for (let i = whole.lo; i <= whole.hi; i++) { + const id = messages[i].id; + const bound = bounds.get(i); + const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound) && !this.locked.has(id); + if ((i < ids.lo || i > ids.hi) && raw && compressible.has(id) && summarized.has(id)) tokens += prices[i] ?? 0; + } + if (tokens > 0) out.push({ pin, tokens }); + } + return out.sort((a, b) => b.tokens - a.tokens); + } + + /** Warn, naming each pin `selectNarrowingPins` narrowed, of what it now covers and how to have it whole again. */ + private warnNarrowedPins(narrowed: readonly ProtectedRange[], refusal: OverBudgetError): void { + for (const pin of narrowed) { + const what = pin.kind === 'document' ? 'document mark' : 'pin'; + const label = pin.name !== undefined ? `"${pin.name}" (${pin.id})` : pin.id; + const one = pin.firstMessageId === pin.lastMessageId; + const covers = one + ? `only message ${pin.firstMessageId}, one part of a message stored in parts, not the whole message` + : `only messages ${pin.firstMessageId} to ${pin.lastMessageId}, not the whole of the messages stored in parts at its ends`; + this.diag.warn( + `autobio: ${what} ${label} now covers ${covers}. With it covering ${one ? 'the whole message' : 'them whole'}, ` + + `the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget}` + + `${refusal.diagnostics.configuredBudget === undefined ? '' : ` (maxTokens ${refusal.diagnostics.configuredBudget})`}. Until this ` + + `context is opened again, the parts it no longer covers fold like any history. Opened again, as at a restart, the ${what} ` + + `covers ${one ? 'the whole message' : 'them'} again where the budget holds ${one ? 'it' : 'them'}: to have ` + + `${one ? 'it' : 'them'} whole, raise the budget, then restart or unpin ${pin.id} and ` + + `${pin.kind === 'document' ? 'mark the message' : 'pin the range'} again.`, + ); + } + } + + private selectOnce(store: MessageStoreView, budget: TokenBudget, opts?: SelectOptions): ContextEntry[] { // Image stripping runs inside each select path (before stats commit / cache // markers), so the returned entries are already bounded — see // applyImageStripping. @@ -7951,7 +8156,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { // A provisional multiplier above 1 that puts the compile over budget // is dropped to 1 and the select runs again: with the refusal // standing, no inference would run, so no sample could ever correct - // it (#179, the startup wedge). Any other refusal stands. + // it (#179, the startup wedge). Any other refusal stands here, for + // selectNarrowingPins to answer. if (!(err instanceof OverBudgetError) || !this.dropProvisionalCalibration(err, opts?.dryRun === true)) throw err; // The retry answers for the select: a preview the discarded attempt // recorded mustn't stand in for a retry that refuses before recording @@ -9169,8 +9375,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // optimization that scales catastrophically: a 4000-message import // converged to ~500 L1s that never aged out, blowing the 200k // window around chunk 118. - // Pins in the canonical L1 prompt. The live view shows a message that a - // pin keeps raw (forceRawPinPositions) raw, so this prompt does too. A + // Pins in the canonical L1 prompt, read by their ids as compression reads + // them (forceRawPinPositions): a marked message's other parts reach the + // prompt as their summaries, though the selects send them raw. A // frontier summary that can't be shown as a recall pair (it covers such a // message, or its content is empty) is DESCENDED, not dropped: its // children take its place, so only the branch holding the pin opens, @@ -9313,7 +9520,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // order, after a head-window reset or an unpin) isn't recalled at all // (layoutHistory), and its messages stay covered. const forceRawMessageIds = new Set(); - for (const i of this.forceRawPinPositions(allMessages)) { + for (const i of this.forceRawPinPositions(allMessages, 'ids')) { if (i < chunkStartIdx) forceRawMessageIds.add(allMessages[i].id); } const recallBudget = sourceOnly ? 0 : (this.config.compressionRecallBudgetTokens ?? 100_000); @@ -9828,17 +10035,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The pins that keep one of those messages raw: a pin whose own bound // reads as raw here, over a message the pins keep raw. const pinnedRawSet = new Set(pinned.pinnedRaw); - const forceRawPositions = this.forceRawPinPositions(allMessages); + const forceRawPositions = this.forceRawPinPositions(allMessages, 'ids'); const readsBounds = this.readsPinBounds(); const positionOf = this.positionIndex(allMessages); const pins = this.pins.filter((pin) => { const own: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; if (readsBounds && !AutobiographicalStrategy.isRawPinBound(own)) return false; - const first = positionOf.get(pin.firstMessageId); - const last = positionOf.get(pin.lastMessageId); - if (first === undefined || last === undefined) return false; - for (let i = Math.min(first, last); i <= Math.max(first, last); i++) { + const span = this.pinSpan(pin, positionOf, allMessages, 'ids'); + if (!span) return false; + for (let i = span.lo; i <= span.hi; i++) { if (pinnedRawSet.has(i) && forceRawPositions.has(i)) return true; } return false; @@ -12726,9 +12932,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { within?: ReadonlySet, ): boolean { // Work frees room only where its summary folds a message the selection - // can fold now: both selectors keep the head and recent windows raw, and - // a message a pin keeps raw. That's the chunker's own reading of what's - // compressible (getCompressibleMessages). + // can fold: both selectors keep the head and recent windows raw, and a + // message a pin keeps raw. That's the chunker's own reading of what's + // compressible (getCompressibleMessages), which counts the parts a pin + // covers only as the selects read it: their summaries are what a + // narrowing sends (selectNarrowingPins). const foldable = new Set( this.getCompressibleMessages(store).map((m) => m.id).filter((id) => within === undefined || within.has(id)), ); @@ -12876,7 +13084,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const all = store.getAll(); const sourceOnly = this.config.compressionSourceOnly === true; const position = new Map(all.map((message, i) => [message.id, i])); - const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all)].sort((a, b) => a - b); + const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all, 'ids')].sort((a, b) => a - b); // Built at the first head read, so a count that reads none (source-only, // or each chunk holding a tool block itself) scans no holds. let heads: ((pos: number) => { start: number; end: number }) | undefined; @@ -13119,15 +13327,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { } // Each pinned position's fold bound, resolved once (resolvedPinBounds), - // so the picker doesn't fold what the user marked keep-raw and every - // selector reads a pin as compression does. A chunk carries at most one + // so the picker doesn't fold what the user marked keep-raw, read as the + // selects read pins (selectsPinReading). A chunk carries at most one // of `pinned` (raw: a classic pin, or a level or cap of 0), `pinLevel` or // `pinMaxLevel`, so every reader of `pinned` agrees on what stays raw. // The last two appear only under the strategies that honor them // (kv-stable, kv-unified), which must be able to move the chunk to or // within its bound; elsewhere a bounded pin resolves to `pinned`. Built // once and reused. - const pinBounds = this.resolvedPinBounds(messages); + const pinBounds = this.resolvedPinBounds(messages, this.selectsPinReading()); // O(1) summary lookup for findAncestorAt — avoids O(summaries) find() // calls during emission. @@ -16077,7 +16285,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const positionOf = this.positionIndex(messages); // Without adaptive resolution no selector reads a pin's level bound, so // every pinned position resolves to raw (resolvedPinBounds). - const pinnedPositionsSet = this.forceRawPinPositions(messages); + const pinnedPositionsSet = this.forceRawPinPositions(messages, this.selectsPinReading()); // Pinned messages between head and recent (head/recent pinned ones // already emit raw via Phase 0 / Phase 4). const pinnedInMiddle: { msg: StoredMessage; position: number }[] = []; @@ -16858,11 +17066,31 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Shared utilities // ============================================================================ + /** + * The positions compression leaves out for pins: raw both by the pins' ids + * and as the selects read them (`forceRawPinPositions`, `'ids'` and + * `selectsPinReading`). A part a pin's ids name that another pin's whole + * reading holds at a level isn't one, since the selects need its summary, + * and a part a pin covers only as the selects read it isn't either: its + * summary is what a narrowing sends. No chunk made after the pin takes a + * position in this set, so what relies on that reads it. + */ + protected compressionPinnedPositions(messages: StoredMessage[]): Set { + const rawAsSelected = this.forceRawPinPositions(messages, this.selectsPinReading()); + const out = new Set(); + for (const i of this.forceRawPinPositions(messages, 'ids')) if (rawAsSelected.has(i)) out.add(i); + return out; + } + /** * Get messages in the compressible zone: outside both head window and * recent window AND not kept raw by a pin. Returns messages from * [0, headStart) ∪ [headEnd, recentStart) minus any position a pin or - * document mark keeps raw (forceRawPinPositions). A pin with a positive + * document mark keeps raw both by its ids and as the selects read it + * (compressionPinnedPositions): of a message stored in parts, only the + * parts a pin's ids name stay out, though the selects send the whole + * message raw, and a part another pin's whole reading holds at a level + * comes in for its summary. A pin with a positive * level or cap is compressible where bounds are read (kv-stable and * kv-unified, with adaptive resolution: readsPinBounds): it renders at a * level, which only a summary can supply. @@ -16872,7 +17100,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); const recentStart = this.getRecentWindowStart(store); - const pinned = this.forceRawPinPositions(messages); + const pinned = this.compressionPinnedPositions(messages); const out: StoredMessage[] = []; for (let i = 0; i < recentStart; i++) { if (i >= headStart && i < headEnd) continue; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 8a3be3bf..f0e724db 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -2084,6 +2084,23 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * how far the span folds instead, and where pins overlap at a message, one * bound holds it (see `level`), so a raw pin there might not keep it raw. * + * A message stored in parts (a long message that ingress splits under + * `adaptiveResolution`) counts as one where a pin is rendered: under adaptive + * resolution, an end inside one covers it from its first part to its last in + * the selects, the agent's view. The summarizer's prompt reads a pin by its + * ids, as compression does, so a raw pin keeps out of compression only the + * parts its ids name (a part chunked before the pin was made stays in its + * chunk, as before), and the message's other parts are chunked and + * summarized like any history. A summary whose chunk runs past the message's + * edge goes out beside the raw message, so some of the message is sent twice. + * A pin with a fixed level reads whole too: under kv-unified, as for any + * pinned level, compiles fail until each part of the message has the summary + * that level needs. + * When a compile can't hold the whole messages within its budget but can + * hold the rest with the pin narrowed to its ids, it narrows it for as long + * as the strategy instance runs, and warns. Nothing records the narrowing, so + * a store opened again reads the pin whole. + * * - `kind: 'pin'` — generic protected range (any first/last span). * - `kind: 'document'` — typically a single message containing a body of * information the agent wants to keep; semantically identical to a @@ -2120,7 +2137,7 @@ export interface ProtectedRange { * conflict, and every compile fails until one does. Under * any other selector, and without adaptive resolution, the range is raw (a * safe superset). The canonical L1 compression prompt shows raw what the - * configured selector keeps raw, where its budgets allow (see + * configured selector keeps raw, reading pins by their ids, where its budgets allow (see * `compressionRecallBudgetTokens`); it doesn't hold a span at its pinned * level. Where pins overlap at a message, a bound takes a classic pin's * place there, `level` takes `maxLevel`'s, and the lowest of each kind diff --git a/test/l1-recall-raw-pins-frontier.test.ts b/test/l1-recall-raw-pins-frontier.test.ts index d5924be5..8f6b130f 100644 --- a/test/l1-recall-raw-pins-frontier.test.ts +++ b/test/l1-recall-raw-pins-frontier.test.ts @@ -8,7 +8,8 @@ * window too, in source order with the recall pairs; nothing else loses * coverage; the raw pins add is spent from the recall budget, and a * compression whose pinned raw can't fit it recalls what the pins override, - * and says so. A pin resolves once, for compression and every selector. + * and says so. A pin's bounds resolve by one rule, for compression and every + * selector, in the reading each takes (resolvedPinBounds). */ import { describe, it, after } from 'node:test'; import assert from 'node:assert/strict'; @@ -836,7 +837,7 @@ describe('the raw pins add is spent from the recall budget', () => { }); }); -describe('a pin resolves once, for compression and every selector', () => { +describe('a pin\'s bounds resolve by one rule, for compression and every selector', () => { const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}` })); const pins: Array & { at: number }> = [ { at: 1 }, // classic raw pin @@ -856,10 +857,10 @@ describe('a pin resolves once, for compression and every selector', () => { id: `pin-${n}`, firstMessageId: `m${at}`, lastMessageId: `m${at}`, kind: 'pin', created: 0, ...bound, })); const internals = strategy as unknown as { - resolvedPinBounds(m: typeof messages): Map; - forceRawPinPositions(m: typeof messages): Set; + resolvedPinBounds(m: typeof messages, reading: 'ids' | 'whole'): Map; + forceRawPinPositions(m: typeof messages, reading: 'ids' | 'whole'): Set; }; - return { bounds: internals.resolvedPinBounds(messages), raw: internals.forceRawPinPositions(messages) }; + return { bounds: internals.resolvedPinBounds(messages, 'whole'), raw: internals.forceRawPinPositions(messages, 'whole') }; }; for (const foldingStrategy of ['kv-stable', 'kv-unified'] as const) { diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts new file mode 100644 index 00000000..79e59cc4 --- /dev/null +++ b/test/whole-body-pins.test.ts @@ -0,0 +1,758 @@ +/** + * A pin renders whole messages: a pin or document mark whose first or last + * id names a part of a message stored in parts covers that message from its + * first part to its last in the selects, while the summarizer's prompt and + * compression read it by its ids, so the message's other parts are chunked + * and summarized like any history (`pinSpan`). A compile refused + * over budget narrows pins back to their own ids, the one that frees the + * most first, one at a time, until a select goes ahead, and keeps them so + * while the strategy instance holds them; otherwise it puts them back + * (`selectNarrowingPins`). + * + * The store: six history messages, then long messages that ingress stores + * in parts (adaptive resolution with 1,500-token chunks), then 360 messages + * of traffic. A head window of 500 tokens and a recent window of 8,000 leave + * the long messages in the middle. With one 6,000-word message marked, its + * first part alone fits at 22,000 tokens and the whole message needs about + * 38,000; the tests use 30,000 for "can't be whole" and 42,000 for "fits". + */ + +import { describe, it } from 'node:test'; +import assert from 'node:assert'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, OverBudgetError } from '../src/index.js'; +import type { AutobiographicalConfig } from '../src/index.js'; + +const words = (label: string, n: number): ContentBlock[] => + [{ type: 'text', text: `${label} ${Array.from({ length: n }, (_, i) => `word${i}`).join(' ')}` }]; + +const KV_UNIFIED = { + policy: { + alpha: 0.7, budgetLowRatio: 0.5, budgetHighRatio: 0.9, budgetUnderLambda: 10, budgetOverLambda: 10, + cacheLambda: 1, cacheScale: 1000, cacheReadPrice: 0.1, cacheWritePrice: 1.25, continuityLambda: 1, + continuityScale: 1000, continuityRecencyHalfLifeTokens: 1000, continuityRecencyFloor: 0.2, + continuityStableHalfLife: 10, continuityStableFloor: 0.25, + }, + tokenBucketSize: 100, continuityBucketSize: 100, fidelityBucketSize: 100, labelCeiling: 10_000, adoptEpsilon: 0, + treeifyNonContiguousSummaries: false, preserveGapBearingSummaries: false, +}; + +/** Reaches the strategy's pins and windows. */ +class PinStore extends AutobiographicalStrategy { + private pinById(pinId: string) { + return this.pins.find((p) => p.id === pinId)!; + } + /** Narrow a pin to its ids, or put it back, as a compile does: the request an earlier version sent. */ + setNarrowed(pinId: string, narrowed: boolean): void { + if (narrowed) this.narrowedPins.add(this.narrowKey(this.pinById(pinId))); + else this.narrowedPins.delete(this.narrowKey(this.pinById(pinId))); + } + /** Whether a compile narrowed this pin. */ + isNarrowed(pinId: string): boolean { + return this.narrowedPins.has(this.narrowKey(this.pinById(pinId))); + } + /** Read the store's state again, as this instance does when another instance on the store has written it. */ + reload(): void { + this.loadPersistedState({ keepKvUnifiedPresentation: true }); + } + /** The fold bound each pinned position resolves to, as every selector reads it. */ + boundsAt(messages: unknown[]): Map { + return this.resolvedPinBounds(messages as never, 'whole'); + } + /** The ids of the messages in compressed chunks. */ + summarizedIds(): Set { + const out = new Set(); + for (const chunk of this.chunks) if (chunk.compressed) for (const m of chunk.messages) out.add(m.id); + return out; + } + /** The ids of the messages in any chunk. */ + chunkedIds(): Set { + const out = new Set(); + for (const chunk of this.chunks) for (const m of chunk.messages) out.add(m.id); + return out; + } + /** Where the head window starts, and where the recent window starts. */ + windows(store: never): { headStart: number; recentStart: number } { + return { headStart: this.getHeadWindowStartIndex(store), recentStart: this.getRecentWindowStart(store) }; + } + /** Where the head window ends. */ + headEnd(store: never): number { + return this.getHeadWindowEnd(store); + } + /** How many adaptive selects ran. */ + selects = 0; + protected selectAdaptive(store: never, budget: never, opts?: never): never { + this.selects++; + return super.selectAdaptive(store, budget, opts) as never; + } + /** Fail a select with an error that isn't a refusal while a pin is narrowed. */ + failWhileNarrowed = false; + protected resolvedPinBounds(messages: never, reading: 'ids' | 'whole'): never { + if (this.failWhileNarrowed && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) throw new Error('select failed'); + return super.resolvedPinBounds(messages, reading) as never; + } +} + +type Kind = 'adaptive' | 'kv-unified'; +const config = (kind: Kind, recentWindowTokens: number): AutobiographicalConfig => ({ + headWindowTokens: 500, recentWindowTokens, imageStripDepthTokens: 0, compressionModel: 'test-model', + hierarchical: false, adaptiveResolution: true, targetChunkTokens: 1500, + ...(kind === 'kv-unified' ? { foldingStrategy: 'kv-unified', kvUnified: { ...KV_UNIFIED, policy: { ...KV_UNIFIED.policy } } } : {}), +} as AutobiographicalConfig); + +function summarizer(words = 30): { membrane: never; calls: () => number; requests: string[] } { + let n = 0; + const requests: string[] = []; + const membrane = { + complete: async (request: unknown) => (requests.push(JSON.stringify(request)), { + stopReason: 'end_turn', content: [{ type: 'text', text: `[S${++n}] ` + 'recalled '.repeat(words) }], + usage: { input_tokens: 100, output_tokens: 40 }, + }), + } as never; + return { membrane, calls: () => n, requests }; +} + +async function open(dir: string, kind: Kind = 'adaptive', recentWindowTokens = 8000, extra: Partial = {}) { + const strategy = new PinStore({ ...config(kind, recentWindowTokens), ...extra } as AutobiographicalConfig); + const sum = summarizer(); + const cm = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: sum.membrane }); + return { cm, strategy, calls: sum.calls, requests: sum.requests }; +} + +const budget = (maxTokens: number) => ({ maxTokens, reserveForResponse: 0 }); +type Sent = { participant: string; content: ContentBlock[] }; +const sent = (r: { messages: unknown[] }): Sent[] => r.messages as Sent[]; +const textOf = (m: Sent): string => m.content.map((b) => (b.type === 'text' ? b.text : '')).join(''); +/** The raw words of a marked message the request carries, by their numbers. */ +function rawWords(messages: Sent[], label: string): number[] { + return messages.map(textOf) + .filter((t) => t.startsWith(label) || /^\s*word\d+/.test(t)) + .flatMap((t) => [...t.matchAll(/\bword(\d+)\b/g)].map((m) => Number(m[1]))) + .sort((a, b) => a - b); +} +const allOf = (n: number): number[] => Array.from({ length: n }, (_, i) => i); +const ready = async (cm: ContextManager) => { for (let i = 0; i < 800 && !cm.isReady(); i++) await cm.tick(); }; + +async function history(cm: ContextManager): Promise { + for (let i = 0; i < 6; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`hist${i}`, 200)); + await cm.compile(budget(42_000)); +} +async function traffic(cm: ContextManager, n: number, label = 'event'): Promise { + for (let i = 0; i < n; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`${label}${i}`, 200)); +} + +/** + * A store as an earlier version left it: the marks on long messages covered + * their first part only, so the rest of each was chunked and summarized, as + * it still is. Returns the request it sent then, and the marks read whole + * again, as the upgraded store reads them. + */ +async function upgradedStore(dir: string, opts: { kind?: Kind; bodies: Array<[string, number]>; budget: number }) { + const s = await open(dir, opts.kind); + await history(s.cm); + const marks: string[] = []; + for (const [label, n] of opts.bodies) { + const mark = s.cm.markDocument(s.cm.addMessage('user', words(label, n))); + s.strategy.setNarrowed(mark, true); + marks.push(mark); + } + await traffic(s.cm, 360); + await ready(s.cm); + const before = sent(await s.cm.compile(budget(opts.budget))); + for (const mark of marks) s.strategy.setNarrowed(mark, false); + return { ...s, marks, before }; +} + +function warnings(t: { mock: { method: (o: object, m: string, f: (...a: unknown[]) => void) => unknown } }): string[] { + const out: string[] = []; + t.mock.method(console, 'warn', (...args: unknown[]) => { out.push(String(args[0])); }); + return out; +} +const narrowingWarnings = (out: string[]) => out.filter((w) => w.includes(' now covers only ')); + +type View = { getAll(): Array<{ id: string; bodyGroupId?: string }> }; +const viewOf = (cm: ContextManager): View => (cm as unknown as { strategyMessageView(): View }).strategyMessageView(); +/** The ids of the parts of the message `id` is in, in store order. */ +const partsOf = (cm: ContextManager, id: string): string[] => { + const all = viewOf(cm).getAll(); + const group = all.find((m) => m.id === id)!.bodyGroupId; + return all.filter((m) => m.bodyGroupId !== undefined && m.bodyGroupId === group).map((m) => m.id); +}; +async function withDir(body: (dir: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), 'cm-whole-pins-')); + try { + await body(dir); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +} + +describe('pins render whole messages', () => { + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: a document mark on a long message's first part sends the whole message raw, while compression summarizes its other parts`, async () => { + await withDir(async (dir) => { + // A recent window of 20,000 holds the whole message as it arrives, so the mark is there before + // compression chunks any part of it. + const { cm, strategy } = await open(dir, kind, 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + assert.deepStrictEqual(partsOf(cm, b).filter((id) => strategy.chunkedIds().has(id)), [], 'no part is chunked yet'); + cm.markDocument(b); + await traffic(cm, 360); + await ready(cm); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); + const parts = partsOf(cm, b); + assert.ok(parts.length > 2, 'the message is stored in parts'); + const summarized = strategy.summarizedIds(); + assert.deepStrictEqual(parts.map((id) => summarized.has(id)), parts.map((_, i) => i > 0), 'every part but the marked one is summarized'); + await cm.close(); + }); + }); + } + + it('a range pin whose ends fall inside long messages covers both of them whole', async () => { + await withDir(async (dir) => { + const { cm } = await open(dir); + await history(cm); + const bParts = partsOf(cm, cm.addMessage('user', words('Bdoc', 2500))); + const cParts = partsOf(cm, cm.addMessage('user', words('Cdoc', 2500))); + assert.ok(bParts.length > 2 && cParts.length > 2, 'both messages are stored in parts'); + // From B's second part to C's second part. + cm.pinRange(bParts[1]!, cParts[1]!); + await traffic(cm, 360); + await ready(cm); + const request = sent(await cm.compile(budget(42_000))); + assert.deepStrictEqual(rawWords(request, 'Bdoc'), allOf(2500)); + assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(2500)); + await cm.close(); + }); + }); + + it('a mark\'s level bound holds every part of its message', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + cm.markDocument(b, { maxLevel: 1 }); + const all = viewOf(cm).getAll(); + const parts = new Set(partsOf(cm, b)); + const at = all.flatMap((m, i) => (parts.has(m.id) ? [i] : [])); + assert.ok(at.length > 2, 'the message is stored in parts'); + const bounds = strategy.boundsAt(all); + assert.deepStrictEqual(at.map((i) => bounds.get(i)), at.map(() => ({ maxLevel: 1 }))); + await cm.close(); + }); + }); + + it('kv-unified: a mark\'s own part that another pin\'s whole reading holds at a level is summarized, so the compile is feasible', async () => { + await withDir(async (dir) => { + // A recent window of 20,000 holds the message as it arrives, so both pins are there before any part is chunked. + const { cm, strategy } = await open(dir, 'kv-unified', 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + // A raw mark on the first part; a range at level 1 from the second part to the third, which read whole covers + // the first part too, where a bound takes a raw pin's place. + cm.markDocument(b); + cm.pinRange(parts[1]!, parts[2]!, { level: 1 }); + await traffic(cm, 360); + await ready(cm); + assert.ok(strategy.summarizedIds().has(parts[0]!), 'the first part is summarized'); + await cm.compile(budget(60_000)); + await cm.close(); + }); + }); + + it('a compression request after the mark carries the marked part raw and the other parts as summaries, as main gives it', async () => { + await withDir(async (dir) => { + const { cm, requests } = await open(dir); + await history(cm); + cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + await traffic(cm, 360); + await ready(cm); + // A request whose chunk holds traffic message 300, long after the marked message. + const later = requests.filter((r) => r.includes('"event300 word0')); + assert.ok(later.length > 0, 'the chunk was summarized'); + for (const r of later) { + assert.ok(r.includes('Bdoc word0 word1'), 'the marked part is in the prompt raw'); + assert.ok(!r.includes('word5999'), 'the message\'s last part is not'); + } + await cm.close(); + }); + }); + + it('the summarizer\'s over-budget warning doesn\'t count a mark\'s later parts, which its prompt sends as summaries', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // A head window of 3,000 holds the message's first part (about 2,070 tokens), where the mark's own + // id is. Read whole, its later parts would be raw history ahead of the traffic's chunks, more than + // the summarizer's recall budget of 2,000. + const { cm, strategy, calls } = await open(dir, 'adaptive', 8000, { headWindowTokens: 3000, compressionRecallBudgetTokens: 2000 }); + cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + await traffic(cm, 120); + await ready(cm); + assert.strictEqual(strategy.headEnd(viewOf(cm) as never), 1, 'the head holds the first part alone'); + assert.ok(calls() > 0, 'the summarizer was called'); + assert.deepStrictEqual(out.filter((w) => w.includes('recalls the summaries its pins override')), []); + await cm.close(); + }); + }); + + it('a store opened without adaptive resolution reads a mark by its ids, as before', async () => { + await withDir(async (dir) => { + const first = await open(dir); + await history(first.cm); + const b = first.cm.addMessage('user', words('Bdoc', 6000)); + first.cm.markDocument(b); + await traffic(first.cm, 360); + await ready(first.cm); + const summarized = first.strategy.summarizedIds(); + assert.ok(partsOf(first.cm, b).slice(1).every((id) => summarized.has(id)), 'the later parts are summarized'); + await first.cm.close(); + const { cm } = await open(dir, 'adaptive', 8000, { adaptiveResolution: false }); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(763)); + await cm.close(); + }); + }); +}); + +describe('a compile that can\'t hold the whole messages narrows pins', () => { + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: an upgraded store whose mark can't be whole narrows it in memory and sends the request it sent before`, async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { kind, bodies: [['Bdoc', 6000]], budget: 30_000 }); + assert.deepStrictEqual(rawWords(before, 'Bdoc'), allOf(763), 'before, the mark kept its first part raw'); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.ok(strategy.isNarrowed(marks[0]!)); + const said = narrowingWarnings(out); + assert.strictEqual(said.length, 1); + assert.match(said[0]!, new RegExp(`^autobio: document mark ${marks[0]} now covers only message `)); + assert.match(said[0]!, /With it covering the whole message, the compile came to ~\d+ tokens against a hard budget of 30600 \(maxTokens 30000\)\./); + assert.match(said[0]!, /Until this context is opened again, the parts it no longer covers fold like any history\./); + assert.match(said[0]!, new RegExp(`raise the budget, then restart or unpin ${marks[0]} and mark the message again\\.`)); + // Narrowed, it fits: the next compile sends the same and warns no more. + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.strictEqual(narrowingWarnings(out).length, 1); + await cm.close(); + }); + }); + } + + it('an upgraded store whose mark fits sends the whole message and narrows nothing', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 42_000 }); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await cm.close(); + }); + }); + + it('a store opened again reads the mark whole: with room it sends the whole message, without it narrows again', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const first = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + await first.cm.compile(budget(30_000)); + assert.ok(first.strategy.isNarrowed(first.marks[0]!)); + await first.cm.close(); + const roomy = await open(dir); + assert.deepStrictEqual(rawWords(sent(await roomy.cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + assert.ok(!roomy.strategy.isNarrowed(first.marks[0]!)); + await roomy.cm.close(); + const tight = await open(dir); + assert.notDeepStrictEqual(rawWords(sent(await tight.cm.compile(budget(30_000))), 'Bdoc'), allOf(6000)); + assert.ok(tight.strategy.isNarrowed(first.marks[0]!)); + assert.strictEqual(narrowingWarnings(out).length, 2, 'each instance that narrows warns once'); + await tight.cm.close(); + }); + }); + + it('a narrowing holds when this instance reads the store\'s state again', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + await cm.compile(budget(30_000)); + strategy.reload(); + assert.ok(strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), rawWords(before, 'Bdoc')); + assert.strictEqual(narrowingWarnings(out).length, 1); + await cm.close(); + }); + }); + + it('a mark whose other parts aren\'t summarized yet frees nothing, so another mark\'s narrowing doesn\'t take it along', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + // C's parts are summarized; B's, which arrive after, aren't, with no tick since. + const c = cm.markDocument(cm.addMessage('user', words('Cdoc', 3000))); + await traffic(cm, 120); + await ready(cm); + const b = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + await traffic(cm, 60, 'later'); + for (let max = 80_000; !strategy.isNarrowed(c) && max > 10_000; max -= 1_000) { + try { + await cm.compile(budget(max)); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + } + } + assert.ok(strategy.isNarrowed(c), 'C was narrowed'); + assert.ok(!strategy.isNarrowed(b), 'B was not'); + await cm.close(); + }); + }); + + it('unpinning a narrowed mark and marking the message again covers it whole', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + await cm.compile(budget(30_000)); + const b = cm.listPins()[0]!.firstMessageId; + assert.ok(cm.unpin(marks[0]!)); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a new mark on a long message narrows as it stops fitting, with no compile refused', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + const turn = async (content: ContentBlock[], role: 'user' | 'assistant') => { + cm.addMessage(role, content); + const request = sent(await cm.compile(budget(30_000))); + await ready(cm); + return request; + }; + for (let i = 0; i < 6; i++) await turn(words(`hist${i}`, 200), i % 2 ? 'assistant' : 'user'); + const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + let last: Sent[] = []; + for (let i = 0; i < 120; i++) last = await turn(words(`event${i}`, 200), i % 2 ? 'assistant' : 'user'); + assert.ok(strategy.isNarrowed(mark)); + assert.strictEqual(narrowingWarnings(out).length, 1); + assert.match(narrowingWarnings(out)[0]!, new RegExp(`document mark ${mark} now covers only message `)); + assert.deepStrictEqual(rawWords(last, 'Bdoc'), allOf(763)); + await cm.close(); + }); + }); + + it('narrows the mark whose whole message adds the most first, and no more than the compile needs', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // Both whole need about 40,800; B narrowed alone, about 25,200; C narrowed alone, about 38,700. + // C is marked first, so taking the first mark that frees anything would narrow C. + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Cdoc', 1500], ['Bdoc', 6000]], budget: 30_000 }); + const request = sent(await cm.compile(budget(30_000))); + assert.deepStrictEqual(marks.map((m) => strategy.isNarrowed(m)), [false, true]); + assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(1500)); + assert.strictEqual(narrowingWarnings(out).length, 1); + assert.match(narrowingWarnings(out)[0]!, new RegExp(`document mark ${marks[1]} `)); + await cm.close(); + }); + }); + + it('narrows a second mark, after the first, when the first alone doesn\'t let the compile through', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // Both narrowed need about 23,000; B narrowed alone, about 25,200. + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000], ['Cdoc', 1500]], budget: 24_000 }); + assert.deepStrictEqual(sent(await cm.compile(budget(24_000))), before); + assert.deepStrictEqual(marks.map((m) => strategy.isNarrowed(m)), [true, true]); + const said = narrowingWarnings(out); + assert.strictEqual(said.length, 2); + assert.match(said[0]!, new RegExp(`document mark ${marks[0]} `)); + assert.match(said[1]!, new RegExp(`document mark ${marks[1]} `)); + await cm.close(); + }); + }); + + it('two marks on one message: narrowing one frees nothing while the other covers the message, so both are narrowed', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + const parts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); + const first = cm.markDocument(parts[0]!); + const second = cm.markDocument(parts[2]!); + strategy.setNarrowed(first, true); + strategy.setNarrowed(second, true); + await traffic(cm, 360); + await ready(cm); + const before = sent(await cm.compile(budget(30_000))); + strategy.setNarrowed(first, false); + strategy.setNarrowed(second, false); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.deepStrictEqual([strategy.isNarrowed(first), strategy.isNarrowed(second)], [true, true]); + assert.strictEqual(narrowingWarnings(out).length, 2); + await cm.close(); + }); + }); + + it('doesn\'t narrow a mark whose message lies in the recent window, which is sent raw either way', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // A recent window of 20,000. B, marked before the upgrade, is in the middle; C, 6,000 words marked + // now, is in the recent window. Both whole need about 39,100; B narrowed, about 33,900, with C whole + // or not. C's whole message adds more than B's, so without the window check C would go first. + const { cm, strategy } = await open(dir, 'adaptive', 20_000); + await history(cm); + const b = cm.markDocument(cm.addMessage('user', words('Bdoc', 2500))); + strategy.setNarrowed(b, true); + await traffic(cm, 360); + const c = cm.markDocument(cm.addMessage('user', words('Cdoc', 6000))); + await ready(cm); + strategy.setNarrowed(b, false); + const request = sent(await cm.compile(budget(36_000))); + assert.deepStrictEqual([strategy.isNarrowed(b), strategy.isNarrowed(c)], [true, false]); + assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(6000)); + assert.strictEqual(narrowingWarnings(out).length, 1); + await cm.close(); + }); + }); + + it('a mark whose message is in the head window frees nothing, so a refusal runs no select again', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // A head window of 25,000 holds the history and the whole 6,000-word message. + const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 25_000 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const mark = cm.markDocument(b); + await traffic(cm, 360); + await ready(cm); + const all = viewOf(cm).getAll(); + const last = all.findIndex((m) => m.id === partsOf(cm, b).at(-1)); + assert.ok(strategy.headEnd(viewOf(cm) as never) > last, 'the head holds the whole message'); + // Lower the budget until a compile is refused for its middle, where narrowing would free nothing. + let refused: OverBudgetError | undefined; + let ran = 0; + for (let max = 44_000; !refused && max > 20_000; max -= 1_000) { + const selects = strategy.selects; + try { + await cm.compile(budget(max)); + } catch (err) { + assert.ok(err instanceof OverBudgetError); + refused = err; + ran = strategy.selects - selects; + } + } + assert.ok(refused && refused.diagnostics.middleTokens > 0, 'refused for its middle'); + assert.strictEqual(ran, 1, 'nothing to narrow, so no select again'); + assert.ok(!strategy.isNarrowed(mark)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await cm.close(); + }); + }); + + it('after a head-window reset, a mark in the history before the head frees its message once it leaves the recent window', async (t) => { + await withDir(async (dir) => { + warnings(t); + // A recent window of 20,000. The marked message is history before the head, which the selects fold. + const { cm, strategy } = await open(dir, 'adaptive', 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 2500)); + const mark = cm.markDocument(b); + await cm.resetHeadWindow('A new topic.'); + const narrows = async (): Promise => { + for (let max = 60_000; !strategy.isNarrowed(mark) && max > 8_000; max -= 1_000) { + try { + await cm.compile(budget(max)); + } catch (err) { + if (!(err instanceof OverBudgetError)) throw err; + } + } + return strategy.isNarrowed(mark); + }; + const positions = () => { + const at = viewOf(cm).getAll().findIndex((m) => m.id === b); + return { at, ...strategy.windows(viewOf(cm) as never) }; + }; + // Within the recent window its parts can't be summarized, so narrowing it would free nothing. + cm.addMessage('user', words('after', 20)); + await ready(cm); + const inRecent = positions(); + assert.ok(inRecent.at > inRecent.recentStart && inRecent.at < inRecent.headStart, JSON.stringify(inRecent)); + assert.strictEqual(await narrows(), false); + // Out of it, they're summarized, and narrowing it frees them. + await traffic(cm, 120); + await ready(cm); + const outOfRecent = positions(); + assert.ok(outOfRecent.at < outOfRecent.recentStart && outOfRecent.at < outOfRecent.headStart, JSON.stringify(outOfRecent)); + assert.strictEqual(await narrows(), true); + await cm.close(); + }); + }); + + it('narrows a range pin to its own ids, warning in a range\'s words, and sends the request it sent before', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + const bParts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); + const cParts = partsOf(cm, cm.addMessage('user', words('Cdoc', 1500))); + // From B's last part to C's first: whole, both messages; by its ids, B's last part and C's first. + const range = cm.pinRange(bParts.at(-1)!, cParts[0]!); + strategy.setNarrowed(range, true); + await traffic(cm, 360); + await ready(cm); + const before = sent(await cm.compile(budget(30_000))); + strategy.setNarrowed(range, false); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.ok(strategy.isNarrowed(range)); + const said = narrowingWarnings(out); + assert.strictEqual(said.length, 1); + assert.match(said[0]!, new RegExp(`^autobio: pin ${range} now covers only messages ${bParts.at(-1)} to ${cParts[0]}, `)); + assert.match(said[0]!, new RegExp(`unpin ${range} and pin the range again\\.`)); + await cm.close(); + }); + }); + + it('a refusal narrowing doesn\'t answer puts the mark back and stands as it was, preview included', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + // At 15,000 the narrowed request is over too. + const preview = cm.previewContext(budget(15_000))!; + assert.ok(!strategy.isNarrowed(marks[0]!), 'the dry compile kept nothing'); + const selects = strategy.selects; + let thrown: OverBudgetError | undefined; + await assert.rejects(cm.compile(budget(15_000)), (err: unknown) => { + assert.ok(err instanceof OverBudgetError); + thrown = err; + return true; + }); + assert.strictEqual(strategy.selects - selects, 2, 'the refused select and one narrowed'); + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + // A preview reports the plan the refusal describes, not a narrowed retry's. + assert.strictEqual(preview.fits, false); + assert.strictEqual(preview.finalTokens, thrown!.actual); + // The refusal thrown is the whole reading's: the narrowed reading's is smaller. + strategy.setNarrowed(marks[0]!, true); + await assert.rejects(cm.compile(budget(15_000)), (err: unknown) => { + assert.ok(err instanceof OverBudgetError); + assert.ok(thrown!.actual > err.actual, `whole ${thrown!.actual}, narrowed ${err.actual}`); + return true; + }); + strategy.setNarrowed(marks[0]!, false); + // Put back whole: the next compile with room sends it whole. + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a refusal while compression is behind puts the mark back, and the whole message goes out once it catches up', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + // No tick while 360 messages arrive: nothing after the message is summarized. + await traffic(cm, 360); + await assert.rejects(cm.compile(budget(42_000)), OverBudgetError); + assert.ok(!strategy.isNarrowed(mark)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await ready(cm); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('under a solver that reads no bounds, a mark with a level cap renders raw and is narrowed like any', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { maxLevel: 1 }); + strategy.setNarrowed(mark, true); + await traffic(cm, 360); + await ready(cm); + const before = sent(await cm.compile(budget(30_000))); + strategy.setNarrowed(mark, false); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.ok(strategy.isNarrowed(mark)); + assert.strictEqual(narrowingWarnings(out).length, 1); + await cm.close(); + }); + }); + + it('kv-unified: a mark with a level cap isn\'t narrowed, so a refusal runs no select again', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { maxLevel: 1 }); + await traffic(cm, 360); + await ready(cm); + const selects = strategy.selects; + await assert.rejects(cm.compile(budget(12_000)), (err: unknown) => { + assert.ok(err instanceof OverBudgetError); + assert.ok(err.diagnostics.middleTokens > 0, 'refused for its middle'); + return true; + }); + assert.strictEqual(strategy.selects - selects, 1); + assert.ok(!strategy.isNarrowed(mark)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await cm.close(); + }); + }); + + it('a refusal the head and recent windows alone cause narrows nothing and runs no select again', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy } = await open(dir); + await history(cm); + const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + await traffic(cm, 360); + await ready(cm); + // A recent window of 8,000 doesn't fit 6,000. + const selects = strategy.selects; + await assert.rejects(cm.compile(budget(6_000)), (err: unknown) => { + assert.ok(err instanceof OverBudgetError); + assert.strictEqual(err.diagnostics.middleTokens, 0); + return true; + }); + assert.strictEqual(strategy.selects - selects, 1); + assert.ok(!strategy.isNarrowed(mark)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await cm.close(); + }); + }); + + it('a dry compile narrows for itself only: it sends what the live compile will, and keeps and warns nothing', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + const dry = sent(await cm.compile(budget(30_000), undefined, { dryRun: true })); + assert.deepStrictEqual(dry, before); + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), dry); + assert.ok(strategy.isNarrowed(marks[0]!)); + assert.strictEqual(narrowingWarnings(out).length, 1); + await cm.close(); + }); + }); + + it('a select that fails while narrowing puts the marks back and fails the compile with its error', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + strategy.failWhileNarrowed = true; + await assert.rejects(cm.compile(budget(30_000)), /select failed/); + strategy.failWhileNarrowed = false; + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + await cm.close(); + }); + }); +}); From 7049ff3f3670f13be4123162bebf69e263f38668 Mon Sep 17 00:00:00 2001 From: Cedric-2315 Date: Fri, 23 Oct 2026 08:36:15 -0400 Subject: [PATCH 31/67] docs: the recall change's exceptions, and a comment #322 left stale (#317) From the automated review of b8a617a. - The Changed entry says a source-only request recalls nothing, before or after: its recall budget is zero. - Its straddle bullet adds the recall-curve fallback: a refused request retried with a recalled summary above L1 replaced by its children recalls a straddling summary's later child too, the same later part as a summary of its own (buildRecallCurveVariants checks coverage and order, not the chunk's start). - chunkRunKey's comment said a pending chunk's request carries the whole unmerged frontier, so every other L1 that lands changes it. Since #322 it carries the summaries that start before the chunk. No code changes. Co-authored-by: Claude Opus 5.5 --- changelog.d/317-compression-recall-before.changed.md | 4 ++-- src/strategies/autobiographical.ts | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/changelog.d/317-compression-recall-before.changed.md b/changelog.d/317-compression-recall-before.changed.md index a0a375ca..d2644fd4 100644 --- a/changelog.d/317-compression-recall-before.changed.md +++ b/changelog.d/317-compression-recall-before.changed.md @@ -1,4 +1,4 @@ -- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). +- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). A source-only request (`compressionSourceOnly`) recalls none, before or after. - **Before:** it recalled every summary on the frontier, as 0.15.0's entry said: "Only pins before the chunk being compressed count, so a summary after it is recalled as before." So a chunk compressed after later history had been summarized was given those later summaries too, ahead of it. That's the old head's chunk, which a head-window reset makes compressible late, or a pinned or quarantined chunk let go late. - **Now:** a summary that starts after the history being summarized isn't recalled, and its messages stay out of the request, as before. Pins still count only before the chunk. - - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. When a refused request is retried with a recalled summary above L1 replaced by its children (the recall-curve fallback), a straddling summary's later child is recalled too: the same later part, as a summary of its own. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b76c6b28..53c00abd 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -354,9 +354,9 @@ function nextFailureStreak( * An L1 run's subject: the chunk's rung, by the chunk's sources (see * chunkRunSubject) and the rung's curve label, the stripped retry after a * carrier rejection being its own. Not the request's bytes: a pending - * chunk's request carries the whole unmerged frontier as recall pairs, so - * every other L1 that lands changes it, and those landings are the evidence - * a run needs. + * chunk's request carries the unmerged frontier summaries that start before + * it as recall pairs, so an L1 that lands for history before it changes it, + * and those landings are the evidence a run needs. */ function chunkRunKey(runSubject: string, curveLabel: string, stripped: boolean): string { return sha256Json({ runSubject, curveLabel, stripped }); From 7df5bb6e50af1de63a36bc3e30653e45d59a818c Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Fri, 23 Oct 2026 09:10:01 -0400 Subject: [PATCH 32/67] fix(calibration): a store with no record starts from a provisional 1 (#328) A store with no calibration record loaded 1 as learned, so the store with the least evidence took the slowest path: nine applied EMA steps to reach a true 1.4, each a reprice of every message and summary. A discard already restarts from a provisional 1, which the first sample the band accepts sets. A store with no record, or a record with no finite multiplier, now loads the same, with fromEpoch null, and writes nothing: with no record the next load decides the same, and the first sample's write gives the stamp. So its first in-band sample sets it, declared or not; a declared one isn't held, as for any provisional multiplier. A record from a newer epoch and a read of the record that throws still load a learned 1: each may hold a multiplier worth keeping, which a provisional 1's first sample would write over even when it only confirms the 1. Render stats report calibration.provisional: true for such a store until its first sample. Tests: a store with no record loads a provisional 1, a rejected sample leaves it, the first in-band sample sets and stamps it, and the EMA steps after; a first sample inside the dead band confirms and stamps it; a restart before any sample leaves it provisional; a record with no usable multiplier loads as none; a read that throws loads a learned 1; a first declared sample sets it with nothing held; a consistent run on a learned record without the flag is learned at the clamp, not held. Two tests that meant a learned multiplier now seed one, and the no-history render-stats test expects provisional. The new assertions that compare with null are strict, since the legacy deepEqual these files use passes an object against null. Co-Authored-By: Claude Opus 5.5 --- ...328-fresh-calibration-provisional.fixed.md | 7 ++ src/strategies/autobiographical.ts | 19 +++- src/types/strategy.ts | 3 +- test/calibration-band.test.ts | 105 +++++++++++++++++- test/calibration-overhead.test.ts | 40 +++++-- 5 files changed, 157 insertions(+), 17 deletions(-) create mode 100644 changelog.d/328-fresh-calibration-provisional.fixed.md diff --git a/changelog.d/328-fresh-calibration-provisional.fixed.md b/changelog.d/328-fresh-calibration-provisional.fixed.md new file mode 100644 index 00000000..a1be6553 --- /dev/null +++ b/changelog.d/328-fresh-calibration-provisional.fixed.md @@ -0,0 +1,7 @@ +- A store with no calibration record starts from a provisional 1, as a + discard's restart does (#328): its first sample inside the calibration + band sets the multiplier, declared or not, instead of stepping a fifth of + the way per sample (nine applied changes, each a reprice of every message + and summary, to reach a true 1.4). Until that sample, render stats report + `calibration.provisional: true`. A record from a newer pricing epoch, or a + read of the record that throws, still loads a learned 1. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 9472770a..b677f878 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -14261,7 +14261,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Set while the multiplier is provisional (see CALIBRATION_PRICING_EPOCH): * the first sample the band accepts replaces it outright, and one above 1 * that puts a compile over budget is dropped to 1. `fromEpoch` is the - * record's it came from, null for one without a stamp. */ + * record's it came from, null for one without a stamp or for no record. */ private _calibrationProvisional: { fromEpoch: number | null } | null = null; protected applyCalibration(): void { @@ -14320,7 +14320,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * is persisted as such (`provisional: true`), so a restart before that * sample doesn't turn the guess into a learned value, and a later epoch's * load re-stamps it, still provisional, rather than discarding it again - * over the reset it carries. A dry run under a + * over the reset it carries. A store with no record starts from a + * provisional 1 too, and writes nothing until its first sample, since no + * record already loads that way (#328). A dry run under a * provisional multiplier above 1 takes the same fallback at the picker's * refusal, which it otherwise only reports. * @@ -14574,6 +14576,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { // re-stamping it as the discard warning advises would destroy it. // This binary's learner still writes its own record over it on its // first applied sample (see CALIBRATION_PRICING_EPOCH). + } else { + // No record, or none with a multiplier to use: no sample has ever + // set this store's 1, so it's a guess, as a discard's restart is, + // and the first sample the band accepts sets it (#328). Learned, it + // would take the slowest path for the store with the least + // evidence, nine applied EMA steps to reach a true 1.4, each a + // reprice of every message and summary. Nothing is written: with no + // record, the next load decides the same, and the first sample's + // write gives the stamp. A record from a newer epoch (above) and a + // read that throws (below) aren't this case: each may hold a + // multiplier worth keeping, which a provisional 1's first sample + // would write over even when it only confirms the 1. + this._calibrationProvisional = { fromEpoch: null }; } } catch { /* absent slot is fine */ } } diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 813f42c3..fbc2dfd1 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -746,7 +746,8 @@ export interface RenderStats { * provider sample the calibration's band accepts replaces outright, rather * than stepping toward it (see * `AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH`): one kept from a - * record without a stamp, or the 1 a discard restarts from. That 1 stays + * record without a stamp, the 1 a discard restarts from, or the 1 a store + * with no calibration record starts from. A discard's 1 stays * provisional through a later epoch's load, which re-stamps it whatever * that epoch repriced. * `reset` is present once a multiplier from an older or unknown epoch has diff --git a/test/calibration-band.test.ts b/test/calibration-band.test.ts index 0c108c8c..fcc391e0 100644 --- a/test/calibration-band.test.ts +++ b/test/calibration-band.test.ts @@ -95,11 +95,16 @@ async function openManager(name: string, opts: { signedThinking?: boolean; stamp describe('estimator calibration band', () => { it('a multiplier pinned at the ceiling comes back down once the raw estimate is honest', async () => { + // A learned 1.8: a store with no record would be provisional, and set + // outright by the first sample (#328). + const first = await openManager('pinned'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1.8, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('pinned'); - await cm.compile(BUDGET); - internals._calibration = 1.8; - internals.applyCalibration(); await cm.compile(BUDGET); // arms one sample; estimate is in calibrated units (raw × 1.8) + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1.8, null], 'learned'); const est = internals._lastCompileEstimate; assert.ok(est > 0); // The provider bills exactly the raw estimate: real/est = 1/1.8 ≈ 0.56. @@ -120,8 +125,15 @@ describe('estimator calibration band', () => { }); it('an in-band ratio is learned from as before', async () => { + // A learned 1: a store with no record would be set outright (#328). + const first = await openManager('inband'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('inband'); await cm.compile(BUDGET); + assert.equal(internals._calibrationProvisional, null, 'learned'); const est = internals._lastCompileEstimate; strategy.reportRealInputTokens(est * 1.5); assert.ok(internals._calibration > 1.0 && internals._calibration < 1.5, `got ${internals._calibration}`); @@ -401,10 +413,93 @@ describe('estimator calibration band', () => { reopened.cm.close(); }); - it('a store with no calibration history reports its multiplier and no reset', async () => { + it('a store with no calibration history reports a provisional 1 and no reset', async () => { const { cm } = await openManager('no-history'); await cm.compile(BUDGET); - assert.deepEqual(cm.getRenderStats()!.calibration, { multiplier: 1, pricingEpoch: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + assert.deepStrictEqual(cm.getRenderStats()!.calibration, + { multiplier: 1, pricingEpoch: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, provisional: true }); + cm.close(); + }); + + it('a store with no calibration record loads a provisional 1: the first sample the band accepts sets it, and stamps the record', async () => { + // The store with the least evidence: learned, its 1 would step a fifth of + // the way per sample, nine applied changes to reach a true 1.4 (#328). + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + const { cm, strategy, internals } = await openManager('fresh-provisional'); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }]); + assert.equal(internals.store!.getStateJson(internals.calibrationStateId), null, 'the load writes nothing'); + // A sample the band rejects is no evidence: the 1 stays provisional. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 3); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }]); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.4); + assert.ok(Math.abs(internals._calibration - 1.4) < 1e-9, `set by the sample, got ${internals._calibration}`); + assert.equal(internals._calibrationProvisional, null); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as + { multiplier?: number; pricing?: number; provisional?: boolean; reset?: unknown }; + assert.deepStrictEqual([saved.multiplier, saved.pricing, saved.provisional, saved.reset], [1.4, epoch, undefined, undefined]); + // From here the EMA steps as it always did. + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.2); // implies 1.68 + assert.ok(internals._calibration > 1.4 && internals._calibration < 1.6, `stepped, not replaced: ${internals._calibration}`); + cm.close(); + }); + + it("a store with no calibration record whose first sample falls inside the dead band confirms its 1, and stamps the record", async () => { + const { cm, strategy, internals } = await openManager('fresh-confirm'); + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; pricing?: number; provisional?: boolean }; + assert.deepStrictEqual([saved.multiplier, saved.pricing, saved.provisional], [1, AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH, undefined]); + cm.close(); + }); + + it('a store with no calibration record is still provisional after a restart before its first sample', async () => { + const first = await openManager('fresh-restart'); + await first.cm.compile(BUDGET); + first.cm.close(); + const { cm, strategy, internals } = await openManager('fresh-restart'); + await cm.compile(BUDGET); + assert.deepStrictEqual(internals._calibrationProvisional, { fromEpoch: null }); + assert.equal(internals.store!.getStateJson(internals.calibrationStateId), null); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 0.8); + assert.ok(Math.abs(internals._calibration - 0.8) < 1e-9, `set by the sample, got ${internals._calibration}`); + cm.close(); + }); + + it('a record with no multiplier to use loads as no record: a provisional 1', async () => { + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + for (const [i, record] of ([{ at: 1, pricing: epoch }, { multiplier: '1.4', at: 1, pricing: epoch }, { multiplier: null, at: 1, pricing: epoch }, 7] as unknown[]).entries()) { + const name = `fresh-unusable-${i}`; + const first = await openManager(name); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, record); + first.cm.close(); + const { cm, internals } = await openManager(name); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, { fromEpoch: null }], JSON.stringify(record)); + cm.close(); + } + }); + + it('a calibration read that throws loads a learned 1, as before: the record it could not read may hold one worth keeping', async () => { + const { cm, internals } = await openManager('fresh-unreadable'); + const store = internals.store as unknown as { getStateJson(id: string): unknown }; + const read = store.getStateJson; + let refused = 0; + store.getStateJson = (id: string) => { + if (id === internals.calibrationStateId) { refused++; throw new Error('unreadable'); } + return read.call(store, id); + }; + try { + await cm.compile(BUDGET); + } finally { + store.getStateJson = read; + } + assert.ok(refused > 0, 'premise: the load read the record, and the read threw'); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); cm.close(); }); diff --git a/test/calibration-overhead.test.ts b/test/calibration-overhead.test.ts index 8f1d1bfd..c80bdb45 100644 --- a/test/calibration-overhead.test.ts +++ b/test/calibration-overhead.test.ts @@ -84,8 +84,8 @@ describe('estimator calibration and request overhead (#323)', () => { const overhead = overheadPrice(cm, SYSTEM, tools); assert.ok(overhead > window * 0.4 && overhead < window * 0.7, `overhead ${overhead} vs window ${window}`); // Undeclared, this sample reads real/est ≈ 1.5: in band, so it would be - // learned as a per-token rate 50% above the truth. Two samples, since a - // store's first declared sample is held until a second agrees. + // learned as a per-token rate 50% above the truth. The first sample sets + // a store with no record; the second steps it. strategy.reportRealInputTokens(window + overhead, { system: SYSTEM, tools }); await cm.compile(BUDGET); strategy.reportRealInputTokens(internals._lastCompileEstimate + overhead, { system: SYSTEM, tools }); @@ -257,7 +257,7 @@ describe('estimator calibration: what the compile adds among the messages', () = // Premise: priced as the window and the declared head alone, these samples read about 1.33. assert.ok(priceOf(cm, inj.content) > window * 0.4, `injection ${priceOf(cm, inj.content)} vs window ${window}`); assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -288,7 +288,7 @@ describe('estimator calibration: what the compile adds among the messages', () = strategy.reportRealInputTokens(window + overheadPrice(cm, SYSTEM, tools) + priceOf(cm, injection.content), { system: SYSTEM, tools }); } assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -303,7 +303,7 @@ describe('estimator calibration: what the compile adds among the messages', () = await cm.compile(BUDGET); strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -320,7 +320,7 @@ describe('estimator calibration: what the compile adds among the messages', () = strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, tools), { system, tools }); } assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -335,7 +335,7 @@ describe('estimator calibration: what the compile adds among the messages', () = strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools), { system: SYSTEM, tools }); } assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -351,7 +351,7 @@ describe('estimator calibration: what the compile adds among the messages', () = strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, SYSTEM, tools) + injected, { system: SYSTEM, tools }); } assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); @@ -393,7 +393,7 @@ describe('estimator calibration: what the compile adds among the messages', () = strategy.reportRealInputTokens(internals._lastCompileEstimate + overheadPrice(cm, system, []), { system }); } assert.equal(internals._calibration, 1); - assert.equal(declaredOf(internals), true, 'both samples were taken, and agreed'); + assert.equal(declaredOf(internals), true, 'the samples were taken'); cm.close(); }); }); @@ -406,6 +406,17 @@ describe('estimator calibration: consistent out-of-band runs', () => { strategy.reportRealInputTokens(internals._lastCompileEstimate * ratio, declared ? nothingBesides : undefined); } + it('a consistent run on a learned record without the flag is learned at the clamp, not held', async () => { + // The upgrade case: a learned multiplier no declared sample has confirmed. + // A store with no record is provisional, which skips the hold anyway. + const { cm, strategy, internals } = await seeded('run-on-learned', { multiplier: 1, at: 1, pricing: epoch }); + assert.equal(internals._calibrationProvisional, null, 'learned'); + for (let i = 0; i < 3; i++) await oob(cm, strategy, internals, 2.1); + assert.equal(internals._calibration, 1.8); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + it('an undeclared out-of-band sample ends a declared run', async () => { const { cm, strategy, internals } = await openManager('undeclared-ends-run'); await oob(cm, strategy, internals, 2.1); @@ -568,6 +579,17 @@ describe('estimator calibration: a store that learned the overhead as a rate', ( cm.close(); }); + it('a store with no record is set by its first declared sample, with nothing held, and marked declared', async () => { + const { cm, strategy, internals } = await openManager('fresh-declared'); + await cm.compile(BUDGET); + assert.deepEqual(internals._calibrationProvisional, { fromEpoch: null }); + await sampleAt(cm, strategy, internals as DeclaredInternals, 1.4, true); + assert.ok(Math.abs(internals._calibration - 1.4) <= 0.01, `set by one sample, got ${internals._calibration}`); + assert.equal((internals as unknown as { _declaredCandidate: number | null })._declaredCandidate, null, 'nothing held'); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + it('a provisional multiplier is replaced by one declared sample, and marked declared', async () => { const { cm, strategy, internals } = await seeded('provisional-declared', { multiplier: 1, at: 1, pricing: epoch, provisional: true }); await sampleAt(cm, strategy, internals, 1, true); From e93803d74b65ac421839b968117e8542180d794d Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Fri, 23 Oct 2026 09:10:24 -0400 Subject: [PATCH 33/67] docs(calibration): give a consistent run's actual reason for replacing outright The comment said an EMA step from 1.00 toward a ratio past the band would leave the next sample just as far out of band. It wouldn't: with the overhead declared, one step toward an implied 2.1 gives 1.22, and the next sample's ratio, 1.72, is inside the band (#324's review, R1-5). The run replaces outright because its samples already agree on the multiplier, and stepping toward it would take several more applied changes, each a reprice of every message and summary. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b677f878..f5947840 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -14031,8 +14031,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // A provisional multiplier is a guess (see CALIBRATION_PRICING_EPOCH). This // sample is the first evidence under this release's pricing, so it replaces // the guess outright; the EMA steps from here on. A consistent run replaces - // it outright too: an EMA step from 1.00 toward a ratio past the band - // would leave the next sample just as far out of band. + // it outright too: its samples agree on the multiplier, and stepping toward + // it would take several more applied changes, each a reprice of every + // message and summary. const provisional = this._calibrationProvisional; this._calibrationProvisional = null; const next = replacement ?? (provisional || fromRun ? observed : ema + alpha * (observed - ema)); From dd98a9838bdd40a1f90cc9c24ae4abe214c2994e Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 09:16:46 -0400 Subject: [PATCH 34/67] fix(pins): narrowings end when the strategy loads the store again; one reading for the summarizer's side; the replay exporter reads as the selects do - narrowedPins is cleared with the branch's other mirrors (clearBranchMirrors), so an open, a branch switch, a fork or setStrategy reads every pin whole again. Before, a store reopened with the same strategy object kept its narrowings, against what the warning and docs say, and a sibling branch inherited them. A reload of the store's state mid-life (takeUpStoreState) still keeps them. - PROMPT_PIN_READING names the reading the summarizer's prompt takes for pins, by their ids, and the four sites that describe that prompt read it: the prompt's forced-raw set, both reads in its pinned-raw warning filter, and requestCarriesToolBlocks. - scripts/export-kv-replay-metadata.mjs passed no reading to pinnedPositions and pinLevelBounds, so every pin read whole whatever the configuration. It now passes selectsPinReading(), the live selector's reading. - When no narrowing lets the compile through, the refused select's frontier (lastFrontierTokens, transitionBlocked, which getHotContextSettings reports) goes back with its preview: a narrowed attempt had overwritten it. - Where the selects read pins by their ids (no adaptive resolution), selectNarrowingPins lets the refusal stand without computing candidates, and compressionPinnedPositions resolves the pins once: no pin covers more than its ids there. - Docs: the warning says "loaded again"; README (a mark with a level is held at it, not raw); the level-pin note names the middle; markDocument names the narrowing; adaptive-resolution-design.md section 3.10 names the narrowing that answers some refusals; selectNarrowingPins says a provisional calibration multiplier is dropped before any narrowing. - Tests (37 in whole-body-pins): - a fork narrows again, and the same strategy reopened with room sends the whole message; - a later narrowing that fails, or that a dry compile makes, leaves an earlier one as it was; - the warning names a named mark and its part, and gives the refusal's exact total; - a provisional multiplier the whole reading is over is dropped before any narrowing; - a mark whose parts' L1s only the summaries record is narrowed as the select folds them; - the summarizer's over-budget warning names the pins that keep its raw history raw, adaptive and kv-unified; - the compression-behind test asserts that no narrowed select ran, and is named for what it shows; - a refusal narrowing doesn't answer leaves the frontier its plan set. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- docs/adaptive-resolution-design.md | 8 ++ scripts/export-kv-replay-metadata.mjs | 6 +- src/context-manager.ts | 3 +- src/strategies/autobiographical.ts | 78 ++++++++----- src/types/strategy.ts | 11 +- test/whole-body-pins.test.ts | 157 ++++++++++++++++++++++++-- 8 files changed, 223 insertions(+), 44 deletions(-) diff --git a/README.md b/README.md index 4e5affe4..f352e91d 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts sends the whole message raw where the budget holds it, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows it, says so in a warning, and keeps it that way until the manager is opened again. `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows it, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch or a fork). `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 56a39e1a..254beec3 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ - **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. - - **When the whole messages don't fit.** A compile refused over budget narrows pins back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed while the strategy instance runs, and a warning names it and says how to have it whole again. Nothing is stored: a store opened again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. + - **When the whole messages don't fit.** A compile refused over budget narrows pins back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch or a fork), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on; under kv-unified, whose solver weighs the whole request, the rewrite can start earlier. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/docs/adaptive-resolution-design.md b/docs/adaptive-resolution-design.md index 00d5a930..1614a1bd 100644 --- a/docs/adaptive-resolution-design.md +++ b/docs/adaptive-resolution-design.md @@ -373,6 +373,14 @@ This case should be rare in practice with unbounded L_n: the picker can almost a > pressure, but any such eviction that would silently lose content instead > **refuses the turn** (`UncoveredDropError`) — pressure never ships silent > loss. The philosophy stands: surface the overage, let the host decide. +> +> *(Pins on messages stored in parts)* A pin or document mark covers the +> whole of a message stored in parts in the selects. Where that is what +> doesn't fit, a refusal is answered first by narrowing the pin to the parts +> its ids name (`selectNarrowingPins`), and the select runs again: the +> narrowing lasts until the store is loaded again, and a host-log warning +> names the pin. When no narrowing lets the compile through, the refusal is +> thrown as above. ### 3.11 Recent-window slide diff --git a/scripts/export-kv-replay-metadata.mjs b/scripts/export-kv-replay-metadata.mjs index 6823dc5d..4f70ea78 100644 --- a/scripts/export-kv-replay-metadata.mjs +++ b/scripts/export-kv-replay-metadata.mjs @@ -19,8 +19,10 @@ try { strategy.loadCalibration(view); const messages = view.getAll(); const estimates = strategy.postStripEstimates(view); - const pins = strategy.pinnedPositions(messages); - const bounds = strategy.pinLevelBounds(messages); + // The live selector's reading: under adaptive resolution a pin covers the + // whole of a message stored in parts. + const pins = strategy.pinnedPositions(messages, strategy.selectsPinReading()); + const bounds = strategy.pinLevelBounds(messages, strategy.selectsPinReading()); const metadata = messages.map((message, index) => ({ id: message.id, index, timestamp: message.timestamp.getTime(), postStripTokens: estimates[index], toolResult: strategy.hasToolResult(message), diff --git a/src/context-manager.ts b/src/context-manager.ts index 774d9c02..fb069917 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1684,7 +1684,8 @@ export class ContextManager { * `markDocument(id, { maxLevel })` lets it fold no deeper than that level * where the selector reads level bounds. Under adaptive resolution, a mark * on any part of a message stored in parts renders the whole message, as - * `pinRange` does. + * `pinRange` does, narrowed to the part it names where the whole message + * doesn't fit. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 4da013d0..9ba10066 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2156,11 +2156,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected pinIdCounter = 0; /** * Pins a compile narrowed (`selectNarrowingPins`), by `narrowKey`: their - * whole reading (`pinSpan`) covers only their own ids, for as long as this - * instance runs, a reload of the store's state included. Nothing records - * the narrowing itself, so a store opened again reads every pin whole; what - * a narrowed compile sent is recorded as any compile's is. A pin unpinned - * and marked again is a new pin, and whole. + * whole reading (`pinSpan`) covers only their own ids until the strategy + * loads the store again (`initialize`: an open, a branch switch, a fork or + * `setStrategy`, which clear it with the branch's other mirrors), a reload + * of the store's state mid-life (`takeUpStoreState`) excluded. Nothing + * records the narrowing itself, so a store loaded again reads every pin + * whole; what a narrowed compile sent is recorded as any compile's is. A pin + * unpinned and marked again is a new pin, and whole. */ protected readonly narrowedPins = new Set(); @@ -2169,6 +2171,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { return JSON.stringify([pin.id, pin.firstMessageId, pin.lastMessageId, pin.created]); } + /** + * The reading the summarizer's prompt takes for pins (`pinSpan`), and so + * everything that describes that prompt: its pinned-raw warning and the + * count of queued work that predicts it (`requestCarriesToolBlocks`). By + * their ids: the prompt already has a marked message's other parts as + * summaries, and carrying them raw multiplied later compression requests' + * input. + */ + protected static readonly PROMPT_PIN_READING = 'ids' as const; + /** * Per-message resolution state for the adaptive-resolution picker. * - Key: MessageId @@ -2850,6 +2862,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.warnedQuarantineSkips.clear(); this.pins = []; this.pinIdCounter = 0; + this.narrowedPins.clear(); this.resolutions.clear(); this.locked.clear(); this._adaptivePicker = null; @@ -3529,9 +3542,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * bound of 0, keeps the messages it names out of compression and renders * them raw at their original position, a message stored in parts whole * (`pinSpan`): where the whole messages don't fit the budget but the rest - * does with the pin narrowed to its ids, a compile narrows it while this - * instance runs (`selectNarrowingPins`). A bound above 0 is read only under - * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with + * does with the pin narrowed to its ids, a compile narrows it until the + * strategy loads the store again (`selectNarrowingPins`). A bound above 0 + * is read only under `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution` (`readsPinBounds`): there the range is still * compressed, since only a summary can supply its level, and is held at * `level`, or no deeper than `maxLevel`, as `ProtectedRange` describes. @@ -3563,7 +3576,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. * Under adaptive resolution, a mark on any part of a message stored in - * parts renders the whole message, as `pinRange` does. + * parts renders the whole message, as `pinRange` does, narrowed to the part + * it names where the whole message doesn't fit (`selectNarrowingPins`). */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -3785,11 +3799,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * two readings (`ProtectedRange`): * - `'ids'`: the messages from its first id to its last, a part of a * message stored in parts covering only itself. The summarizer's prompt - * reads this, as do its pinned-raw warning and the count of queued work - * that predicts it (`requestCarriesToolBlocks`): the prompt already has a - * marked message's other parts as summaries, and carrying them raw would - * multiply every later compression request's input. Compression keeps - * out only parts raw this way (`compressionPinnedPositions`). + * reads this (`PROMPT_PIN_READING`). Compression keeps out only parts raw + * this way (`compressionPinnedPositions`). * - `'whole'`: from the first part of the message its first id is in to the * last part of the message its last id is in. The selects read this under * adaptive resolution (`selectsPinReading`), so the whole message goes out @@ -8022,13 +8033,20 @@ export class AutobiographicalStrategy implements ResettableStrategy { * - A refusal whose middle holds nothing, as when the head and recent * windows alone are over the budget, stands as it was: nothing is * narrowed and no select runs again. - * - When a select goes ahead, the pins narrowed so far stay narrowed while - * this instance runs (`narrowedPins`), and a warning names each. - * Nothing records them, so a store opened again reads every pin whole: with - * a larger budget it sends the whole message, and one that still doesn't - * fit narrows again at its first refused compile. + * - Where the selects read pins by their ids, as without adaptive + * resolution, no pin covers more than its ids, so the refusal stands + * without that look. + * - When a select goes ahead, the pins narrowed so far stay narrowed until + * the strategy loads the store again (`narrowedPins`), and a warning + * names each. Nothing records them, so a store loaded again reads every + * pin whole: with a larger budget it sends the whole message, and one + * that still doesn't fit narrows again at its first refused compile. + * - A provisional calibration multiplier above 1 that puts the whole + * reading over is dropped first (`selectOnce`), as for any refusal, so a + * guess the first sample replaces doesn't narrow a pin. * - When no pin is left to narrow, the pins go back to whole messages and - * the first refusal is thrown, with its preview. + * the first refusal is thrown, with its preview and the frontier it + * planned (`getHotContextSettings`). * - A dry compile narrows for its own select only: it puts the pins back * and warns nothing. */ @@ -8039,8 +8057,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { refusal: OverBudgetError, cacheKeys: string[] | undefined, ): ContextEntry[] { - if (!(refusal.diagnostics.middleTokens > 0)) throw refusal; + if (this.selectsPinReading() === 'ids' || !(refusal.diagnostics.middleTokens > 0)) throw refusal; + // What the refused select recorded for observers, which a narrowed + // attempt overwrites and a refusal that stands puts back. const preview = this._lastPreview; + const frontier = this.lastFrontierTokens; + const blocked = this.transitionBlocked; const narrowed: ProtectedRange[] = []; const putBack = (): void => { for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); @@ -8071,6 +8093,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (!entries) { putBack(); this._lastPreview = preview; + this.lastFrontierTokens = frontier; + this.transitionBlocked = blocked; throw refusal; } if (opts?.dryRun === true) putBack(); @@ -8137,7 +8161,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { `autobio: ${what} ${label} now covers ${covers}. With it covering ${one ? 'the whole message' : 'them whole'}, ` + `the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget}` + `${refusal.diagnostics.configuredBudget === undefined ? '' : ` (maxTokens ${refusal.diagnostics.configuredBudget})`}. Until this ` + - `context is opened again, the parts it no longer covers fold like any history. Opened again, as at a restart, the ${what} ` + + `context is loaded again, the parts it no longer covers fold like any history. Loaded again, as at a restart, the ${what} ` + `covers ${one ? 'the whole message' : 'them'} again where the budget holds ${one ? 'it' : 'them'}: to have ` + `${one ? 'it' : 'them'} whole, raise the budget, then restart or unpin ${pin.id} and ` + `${pin.kind === 'document' ? 'mark the message' : 'pin the range'} again.`, @@ -9520,7 +9544,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // order, after a head-window reset or an unpin) isn't recalled at all // (layoutHistory), and its messages stay covered. const forceRawMessageIds = new Set(); - for (const i of this.forceRawPinPositions(allMessages, 'ids')) { + for (const i of this.forceRawPinPositions(allMessages, AutobiographicalStrategy.PROMPT_PIN_READING)) { if (i < chunkStartIdx) forceRawMessageIds.add(allMessages[i].id); } const recallBudget = sourceOnly ? 0 : (this.config.compressionRecallBudgetTokens ?? 100_000); @@ -10035,14 +10059,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The pins that keep one of those messages raw: a pin whose own bound // reads as raw here, over a message the pins keep raw. const pinnedRawSet = new Set(pinned.pinnedRaw); - const forceRawPositions = this.forceRawPinPositions(allMessages, 'ids'); + const forceRawPositions = this.forceRawPinPositions(allMessages, AutobiographicalStrategy.PROMPT_PIN_READING); const readsBounds = this.readsPinBounds(); const positionOf = this.positionIndex(allMessages); const pins = this.pins.filter((pin) => { const own: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; if (readsBounds && !AutobiographicalStrategy.isRawPinBound(own)) return false; - const span = this.pinSpan(pin, positionOf, allMessages, 'ids'); + const span = this.pinSpan(pin, positionOf, allMessages, AutobiographicalStrategy.PROMPT_PIN_READING); if (!span) return false; for (let i = span.lo; i <= span.hi; i++) { if (pinnedRawSet.has(i) && forceRawPositions.has(i)) return true; @@ -13084,7 +13108,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const all = store.getAll(); const sourceOnly = this.config.compressionSourceOnly === true; const position = new Map(all.map((message, i) => [message.id, i])); - const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all, 'ids')].sort((a, b) => a - b); + const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all, AutobiographicalStrategy.PROMPT_PIN_READING)].sort((a, b) => a - b); // Built at the first head read, so a count that reads none (source-only, // or each chunk holding a tool block itself) scans no holds. let heads: ((pos: number) => { start: number; end: number }) | undefined; @@ -17076,6 +17100,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * position in this set, so what relies on that reads it. */ protected compressionPinnedPositions(messages: StoredMessage[]): Set { + // Where the selects read pins by their ids too, the two sets are one. + if (this.selectsPinReading() === 'ids') return this.forceRawPinPositions(messages, 'ids'); const rawAsSelected = this.forceRawPinPositions(messages, this.selectsPinReading()); const out = new Set(); for (const i of this.forceRawPinPositions(messages, 'ids')) if (rawAsSelected.has(i)) out.add(i); diff --git a/src/types/strategy.ts b/src/types/strategy.ts index f0e724db..6c52f1fe 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -2094,12 +2094,13 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * summarized like any history. A summary whose chunk runs past the message's * edge goes out beside the raw message, so some of the message is sent twice. * A pin with a fixed level reads whole too: under kv-unified, as for any - * pinned level, compiles fail until each part of the message has the summary - * that level needs. + * pinned level, compiles fail until each part of the message in the middle + * (outside the head and recent windows) has the summary that level needs. * When a compile can't hold the whole messages within its budget but can - * hold the rest with the pin narrowed to its ids, it narrows it for as long - * as the strategy instance runs, and warns. Nothing records the narrowing, so - * a store opened again reads the pin whole. + * hold the rest with the pin narrowed to its ids, it narrows it until the + * strategy loads the store again (an open, a branch switch or a fork), and + * warns. Nothing records the narrowing, so a store loaded again reads the + * pin whole. * * - `kind: 'pin'` — generic protected range (any first/last span). * - `kind: 'document'` — typically a single message containing a body of diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 79e59cc4..074ca370 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -82,11 +82,39 @@ class PinStore extends AutobiographicalStrategy { headEnd(store: never): number { return this.getHeadWindowEnd(store); } - /** How many adaptive selects ran. */ + /** How many adaptive selects ran, and the refusals they threw. */ selects = 0; + refusals: OverBudgetError[] = []; protected selectAdaptive(store: never, budget: never, opts?: never): never { this.selects++; - return super.selectAdaptive(store, budget, opts) as never; + try { + return super.selectAdaptive(store, budget, opts) as never; + } catch (err) { + if (err instanceof OverBudgetError) this.refusals.push(err); + throw err; + } + } + /** + * Leave the L1s covering these messages as the only record of their + * coverage, as an adopted summary can: their chunks lose the pointer to it + * and read as not compressed. + */ + dropChunkPointers(ids: readonly string[]): void { + const of = new Set(ids); + for (const record of this.chunkRecords) { + if (!record.sourceIds.some((id) => of.has(id))) continue; + record.summaryId = undefined; + record.compressed = false; + } + } + /** The id a pin's range starts at. */ + firstIdOf(pinId: string): string { + return this.pinById(pinId).firstMessageId; + } + /** Persist a provisional calibration multiplier, as a restart no sample has confirmed leaves one. */ + saveProvisionalCalibration(multiplier: number): void { + const epoch = (AutobiographicalStrategy as unknown as { CALIBRATION_PRICING_EPOCH: number }).CALIBRATION_PRICING_EPOCH; + this.store!.setStateJson(this.calibrationStateId, { multiplier, at: 1, pricing: epoch, provisional: true }); } /** Fail a select with an error that isn't a refusal while a pin is narrowed. */ failWhileNarrowed = false; @@ -150,12 +178,12 @@ async function traffic(cm: ContextManager, n: number, label = 'event'): Promise< * it still is. Returns the request it sent then, and the marks read whole * again, as the upgraded store reads them. */ -async function upgradedStore(dir: string, opts: { kind?: Kind; bodies: Array<[string, number]>; budget: number }) { +async function upgradedStore(dir: string, opts: { kind?: Kind; bodies: Array<[string, number, string?]>; budget: number }) { const s = await open(dir, opts.kind); await history(s.cm); const marks: string[] = []; - for (const [label, n] of opts.bodies) { - const mark = s.cm.markDocument(s.cm.addMessage('user', words(label, n))); + for (const [label, n, name] of opts.bodies) { + const mark = s.cm.markDocument(s.cm.addMessage('user', words(label, n)), name === undefined ? undefined : { name }); s.strategy.setNarrowed(mark, true); marks.push(mark); } @@ -301,6 +329,34 @@ describe('pins render whole messages', () => { }); }); + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: the summarizer's over-budget warning names the pins that keep its raw history raw, read by their ids as its prompt reads them`, async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + // The summarizer shows a pinned message raw where a summary would cover it. Under kv-unified, Q's whole + // reading holds P's part at a level, so it's summarized; a recent window of 20,000 holds the message as it + // arrives, so both pins are there before any part is chunked. Elsewhere, a recent window of 8,000 lets the + // message's early parts be chunked as they arrive, before the pins. + const { cm } = await open(dir, kind, kind === 'kv-unified' ? 20_000 : 8000, { compressionRecallBudgetTokens: 1000 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + // P keeps the first part raw, about 2,070 tokens, over the recall budget of 1,000. Q names the sixth part: + // raw where no bounds are read, and at level 1 under kv-unified, where read whole it would hold the first + // part at that level in P's place. + cm.markDocument(b, { name: 'P' }); + cm.pinRange(parts[5]!, parts[5]!, kind === 'kv-unified' ? { name: 'Q', level: 1 } : { name: 'Q' }); + await traffic(cm, 360); + await ready(cm); + const over = out.filter((w) => w.includes('recalls the summaries its pins override')); + assert.ok(over.length > 0, 'the warning was given'); + // Only P's part is the prompt's pinned raw history: Q's whole reading would reach it, its ids don't. + for (const w of over) assert.ok(w.includes('pins: P)'), w); + await cm.close(); + }); + }); + } + it('a store opened without adaptive resolution reads a mark by its ids, as before', async () => { await withDir(async (dir) => { const first = await open(dir); @@ -332,7 +388,7 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { assert.strictEqual(said.length, 1); assert.match(said[0]!, new RegExp(`^autobio: document mark ${marks[0]} now covers only message `)); assert.match(said[0]!, /With it covering the whole message, the compile came to ~\d+ tokens against a hard budget of 30600 \(maxTokens 30000\)\./); - assert.match(said[0]!, /Until this context is opened again, the parts it no longer covers fold like any history\./); + assert.match(said[0]!, /Until this context is loaded again, the parts it no longer covers fold like any history\./); assert.match(said[0]!, new RegExp(`raise the budget, then restart or unpin ${marks[0]} and mark the message again\\.`)); // Narrowed, it fits: the next compile sends the same and warns no more. assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); @@ -385,6 +441,88 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); }); + it('a narrowing ends when the strategy loads the store again: a fork narrows again, and the same strategy reopened with room sends the whole message', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + await cm.compile(budget(30_000)); + assert.ok(strategy.isNarrowed(marks[0]!)); + await cm.fork(); + assert.ok(!strategy.isNarrowed(marks[0]!), 'the fork loaded the store again'); + await cm.compile(budget(30_000)); + assert.ok(strategy.isNarrowed(marks[0]!), 'and narrowed again'); + assert.strictEqual(narrowingWarnings(out).length, 2); + await cm.close(); + const again = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: summarizer().membrane }); + assert.deepStrictEqual(rawWords(sent(await again.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + assert.ok(!strategy.isNarrowed(marks[0]!)); + await again.close(); + }); + }); + + it('a later narrowing that fails, or that a dry compile makes, leaves an earlier one as it was', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000], ['Cdoc', 1500]], budget: 30_000 }); + strategy.setNarrowed(marks[0]!, true); + // With the first narrowed, the second fits at 25,000 only narrowed, and at 15,000 not even so. + let selects = strategy.selects; + await assert.rejects(cm.compile(budget(15_000)), OverBudgetError); + assert.strictEqual(strategy.selects - selects, 2, 'the refused select and one narrowing the second mark'); + assert.ok(strategy.isNarrowed(marks[0]!), 'the first stays narrowed'); + assert.ok(!strategy.isNarrowed(marks[1]!)); + selects = strategy.selects; + assert.strictEqual(cm.previewContext(budget(25_000))!.fits, true); + assert.strictEqual(strategy.selects - selects, 2, 'the dry select narrowed the second mark'); + assert.ok(strategy.isNarrowed(marks[0]!), 'the first stays narrowed'); + assert.ok(!strategy.isNarrowed(marks[1]!)); + await cm.close(); + }); + }); + + it('a mark whose parts\' L1s only the summaries record, with no chunk pointing to them, is narrowed as the select folds them', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + strategy.dropChunkPointers(partsOf(cm, strategy.firstIdOf(marks[0]!))); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc'), rawWords(before, 'Bdoc')); + assert.ok(strategy.isNarrowed(marks[0]!)); + await cm.close(); + }); + }); + + it('the warning names a named mark and its part, and gives the whole reading\'s refusal', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000, 'spec']], budget: 30_000 }); + strategy.refusals = []; + await cm.compile(budget(30_000)); + const refusal = strategy.refusals[0]!; + const said = narrowingWarnings(out); + assert.strictEqual(said.length, 1); + assert.ok(said[0]!.startsWith(`autobio: document mark "spec" (${marks[0]}) now covers only message ${strategy.firstIdOf(marks[0]!)}, one part of a message stored in parts, not the whole message. `), said[0]); + assert.ok(said[0]!.includes(`the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget} (maxTokens 30000).`), said[0]); + await cm.close(); + }); + }); + + it('a provisional calibration multiplier the whole reading is over is dropped before any narrowing, as for any refusal', async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const first = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 42_000 }); + first.strategy.saveProvisionalCalibration(1.3); + await first.cm.close(); + // At 1.3 the whole message is over 42,000 and the narrowed one isn't; at 1 the whole one fits. + const { cm, strategy } = await open(dir); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + assert.ok(!strategy.isNarrowed(first.marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + const calibration = cm.getRenderStats()!.calibration!; + assert.deepStrictEqual([calibration.multiplier, calibration.reset?.discardedMultiplier], [1, 1.3]); + await cm.close(); + }); + }); + it('a mark whose other parts aren\'t summarized yet frees nothing, so another mark\'s narrowing doesn\'t take it along', async (t) => { await withDir(async (dir) => { warnings(t); @@ -632,9 +770,10 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { assert.strictEqual(strategy.selects - selects, 2, 'the refused select and one narrowed'); assert.ok(!strategy.isNarrowed(marks[0]!)); assert.deepStrictEqual(narrowingWarnings(out), []); - // A preview reports the plan the refusal describes, not a narrowed retry's. + // A preview reports the plan the refusal describes, not a narrowed retry's, and so does the frontier. assert.strictEqual(preview.fits, false); assert.strictEqual(preview.finalTokens, thrown!.actual); + assert.strictEqual(strategy.getHotContextSettings().currentFrontierTokens, thrown!.actual); // The refusal thrown is the whole reading's: the narrowed reading's is smaller. strategy.setNarrowed(marks[0]!, true); await assert.rejects(cm.compile(budget(15_000)), (err: unknown) => { @@ -649,7 +788,7 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); }); - it('a refusal while compression is behind puts the mark back, and the whole message goes out once it catches up', async (t) => { + it('a refusal while nothing the mark covers is summarized narrows nothing, and the whole message goes out once compression catches up', async (t) => { await withDir(async (dir) => { const out = warnings(t); const { cm, strategy } = await open(dir); @@ -657,7 +796,9 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); // No tick while 360 messages arrive: nothing after the message is summarized. await traffic(cm, 360); + const selects = strategy.selects; await assert.rejects(cm.compile(budget(42_000)), OverBudgetError); + assert.strictEqual(strategy.selects - selects, 1, 'no narrowed select ran'); assert.ok(!strategy.isNarrowed(mark)); assert.deepStrictEqual(narrowingWarnings(out), []); await ready(cm); From 0215b1bec118ee4e054c3a66bbbdca95caa3513b Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Fri, 23 Oct 2026 09:33:47 -0400 Subject: [PATCH 35/67] fix(calibration): answer our internal review of the round - A record whose pricing stamp is newer than this binary's is left alone whatever its multiplier field holds, so a later binary's record in a shape this one can't read isn't treated as no record and written over by a provisional 1's first sample. - Dropping a provisional multiplier to 1 clears the declared mark in memory as the record it writes does, so an operator's seed over a marked record (the unwedge advice) can't bring the mark back on the 1. - Declared-ness is one test, overhead != null, at the run gate, the rejection line and the pricing too, as at the mark. - Text: the declared mark means the two-sample move has run for the multiplier (a disagreeing pair steps and marks too); a held sample ends when what it was held for is settled another way; a store with no record writes nothing because each load decides the same; the unwedge advice is for a multiplier above 1; the #328 fragment states the trade, one sample without the two-sample hold. - Tests: an undeclared sample between the two leaves the held sample; a falsy non-null overhead is declared in the run; a newer-epoch record with no readable multiplier is left alone; the drop clears the mark; the dead-band and rejected-declared tests show their samples were taken; the provisional-replacement test samples at 1.4; the dead-band drift test seeds a learned 1. Co-Authored-By: Claude Opus 5.5 --- .../323-calibration-request-overhead.fixed.md | 8 +-- ...328-fresh-calibration-provisional.fixed.md | 9 +-- src/strategies/autobiographical.ts | 63 +++++++++++-------- test/calibration-band.test.ts | 45 ++++++++++++- test/calibration-overhead.test.ts | 37 +++++++++-- 5 files changed, 121 insertions(+), 41 deletions(-) diff --git a/changelog.d/323-calibration-request-overhead.fixed.md b/changelog.d/323-calibration-request-overhead.fixed.md index 3b1a141b..68050e01 100644 --- a/changelog.d/323-calibration-request-overhead.fixed.md +++ b/changelog.d/323-calibration-request-overhead.fixed.md @@ -30,9 +30,9 @@ samples step by the EMA. (Out of the band, the three-sample run above applies.) The first of the two is held in memory, so a restart between them costs one more sample. An undeclared sample that moves the multiplier - writes the record without the flag. A store whose multiplier puts every - compile over budget gets no sample to move it, as before: to unwedge it, - seed the record as provisional on the stopped store, and the refused - compile drops it to 1: `setStateJson('/autobio:calibration', + writes the record without the flag. A store whose multiplier, above 1, + puts every compile over budget gets no sample to move it, as before: to + unwedge it, seed the record as provisional on the stopped store, and the + refused compile drops it to 1: `setStateJson('/autobio:calibration', { multiplier, at, pricing: , provisional: true })`. `RequestOverhead` is exported from the package. diff --git a/changelog.d/328-fresh-calibration-provisional.fixed.md b/changelog.d/328-fresh-calibration-provisional.fixed.md index a1be6553..354929cf 100644 --- a/changelog.d/328-fresh-calibration-provisional.fixed.md +++ b/changelog.d/328-fresh-calibration-provisional.fixed.md @@ -1,7 +1,8 @@ - A store with no calibration record starts from a provisional 1, as a - discard's restart does (#328): its first sample inside the calibration - band sets the multiplier, declared or not, instead of stepping a fifth of - the way per sample (nine applied changes, each a reprice of every message - and summary, to reach a true 1.4). Until that sample, render stats report + discard's restart does (#328). Its first sample inside the calibration + band sets the multiplier, declared or not: one sample, without the + two-sample hold, where it stepped a fifth of the way per sample (nine + applied changes, each a reprice of every message and summary, to reach a + true 1.4). Until that sample, render stats report `calibration.provisional: true`. A record from a newer pricing epoch, or a read of the record that throws, still loads a learned 1. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index f5947840..9f4d86a3 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -13923,7 +13923,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The overhead in the window's units (current calibrated), so the implied // multiplier below is real / raw(window + overhead) and doesn't depend on // the multiplier it's measured at: one sample states it. - const overheadTokens = overhead ? this.priceRequestOverhead(overhead) : 0; + const overheadTokens = declared ? this.priceRequestOverhead(overhead!) : 0; const injected = this._lastCompileInjected; const est = window + injected + overheadTokens; const parts = [ @@ -13965,7 +13965,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { let fromRun = false; if (!ratioInBand && !observedInBand) { const run = this._outOfBandRun; - this._outOfBandRun = !overhead ? null + this._outOfBandRun = !declared ? null : run && Math.abs(observed / run.first - 1) <= OUT_OF_BAND_RUN_AGREEMENT ? { first: run.first, count: run.count + 1 } : { first: observed, count: 1 }; @@ -13973,7 +13973,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (count < OUT_OF_BAND_RUN_LENGTH) { this.diag.error( `[estimator-calibration] REJECTED out-of-band sample real/est=${ratio.toFixed(2)} ${counts} — ` + - (overhead + (declared ? `multiplier stays ${this._calibration.toFixed(2)} until ${OUT_OF_BAND_RUN_LENGTH} samples in a row ` + `agree within ${OUT_OF_BAND_RUN_AGREEMENT * 100}% (${count} of ${OUT_OF_BAND_RUN_LENGTH})` : `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}; if every sample ` + @@ -13999,10 +13999,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // set their ruler alone. The held sample is in memory only, so a restart // between them costs one more sample; an undeclared sample between them // leaves it, since an implied multiplier doesn't depend on the multiplier - // in force. Either way the record then says declared samples agree with - // its multiplier (declaredOverhead), and an undeclared write takes that - // back. A provisional multiplier, or a consistent run (out of the band, - // three samples), is replaced by one sample as before. + // in force. Either way the record is then marked (declaredOverhead): the + // move has run for it, and later declared samples step by the EMA; an + // undeclared sample that moves the multiplier takes the mark back. A + // provisional multiplier, or a consistent run (out of the band, three + // samples), is replaced by one sample as before. const markDeclared = declared && !this._calibrationDeclared; let replacement: number | null = null; if (markDeclared && !this._calibrationProvisional && !fromRun) { @@ -14020,9 +14021,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._declaredCandidate = null; if (Math.abs(observed / candidate - 1) <= OUT_OF_BAND_RUN_AGREEMENT) replacement = (candidate + observed) / 2; } else if (declared) { - // A declared sample that moves the ruler another way (a provisional - // replacement, a run) or meets a marked record ends any held sample: - // it was measured against a ruler this one replaces. + // A declared sample that decides the multiplier another way (a + // provisional replacement, a run) or meets a marked record ends any + // held sample: what it was held for, whether two declared samples + // should replace an unmarked multiplier, has been settled. this._declaredCandidate = null; } @@ -14252,9 +14254,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { * multiplier wasn't learned from declared samples, held for the next * declared sample (see reportRealInputTokens). In memory only. */ private _declaredCandidate: number | null = null; - /** Whether declared samples agree with the multiplier in force: set by - * one, or confirmed by two inside the dead band. Persisted as the record's - * declaredOverhead (see reportRealInputTokens). */ + /** Whether the two-sample move has run for the multiplier in force: set + * once declared samples have set, stepped or confirmed it (one sample for a + * provisional multiplier or a run, two for a learned one). Persisted as the + * record's declaredOverhead; an undeclared sample that moves the multiplier + * clears it, and so does dropping a provisional multiplier to 1 (see + * reportRealInputTokens). */ private _calibrationDeclared = false; private _calibrationLoaded = false; /** Set when an older-epoch multiplier was discarded; persisted with the record. */ @@ -14322,8 +14327,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * sample doesn't turn the guess into a learned value, and a later epoch's * load re-stamps it, still provisional, rather than discarding it again * over the reset it carries. A store with no record starts from a - * provisional 1 too, and writes nothing until its first sample, since no - * record already loads that way (#328). A dry run under a + * provisional 1 too (#328). It writes nothing until its first sample: with + * no record, each load decides the same. A dry run under a * provisional multiplier above 1 takes the same fallback at the picker's * refusal, which it otherwise only reports. * @@ -14577,18 +14582,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { // re-stamping it as the discard warning advises would destroy it. // This binary's learner still writes its own record over it on its // first applied sample (see CALIBRATION_PRICING_EPOCH). - } else { - // No record, or none with a multiplier to use: no sample has ever - // set this store's 1, so it's a guess, as a discard's restart is, - // and the first sample the band accepts sets it (#328). Learned, it - // would take the slowest path for the store with the least - // evidence, nine applied EMA steps to reach a true 1.4, each a - // reprice of every message and summary. Nothing is written: with no - // record, the next load decides the same, and the first sample's - // write gives the stamp. A record from a newer epoch (above) and a - // read that throws (below) aren't this case: each may hold a - // multiplier worth keeping, which a provisional 1's first sample - // would write over even when it only confirms the 1. + } else if (!(typeof saved?.pricing === 'number' && saved.pricing > AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH)) { + // No record, or none with a multiplier to use: nothing this store + // kept says what its multiplier is, so its 1 is a guess, as a + // discard's restart is, and the first sample the band accepts sets + // it (#328). Learned, it would take the slowest path for the store + // with the least evidence, nine applied EMA steps to reach a true + // 1.4, each a reprice of every message and summary. Nothing is + // written: with no record, the next load decides the same, and the + // first sample's write gives the stamp. A record from a newer epoch, + // with a multiplier this binary can read or not, and a read that + // throws (below) aren't this case: each may hold what a later binary + // or the store keeps, which a provisional 1's first sample would + // write over even when it only confirms the 1. this._calibrationProvisional = { fromEpoch: null }; } } catch { /* absent slot is fine */ } @@ -14619,6 +14625,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { const reset: CalibrationReset = { discardedMultiplier: discarded, fromEpoch: provisional.fromEpoch, at: Date.now() }; this._calibration = 1; this._calibrationReset = reset; + // The mark was about the multiplier dropped; the record below is written + // without it, and memory agrees. + this._calibrationDeclared = false; this.applyCalibration(); if (dryRun) return true; this.diag.warn( diff --git a/test/calibration-band.test.ts b/test/calibration-band.test.ts index fcc391e0..f40bc215 100644 --- a/test/calibration-band.test.ts +++ b/test/calibration-band.test.ts @@ -216,6 +216,21 @@ describe('estimator calibration band', () => { assert.deepEqual(warned.filter((l) => l.includes('[estimator-calibration]')), []); }); + it('a record from a newer pricing epoch with no multiplier this binary can read is left alone too', async () => { + const first = await openManager('newer-epoch-unreadable'); + await first.cm.compile(BUDGET); + const future = { multipliers: { default: 1.4 }, at: 123, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH + 1 }; + first.internals.store!.setStateJson(first.internals.calibrationStateId, future); + first.cm.close(); + const { cm, strategy, internals } = await openManager('newer-epoch-unreadable'); + await cm.compile(BUDGET); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); + // A first sample that only confirms the 1 writes nothing over it. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + assert.deepStrictEqual(internals.store!.getStateJson(internals.calibrationStateId), future); + cm.close(); + }); + it('a multiplier learned under the current pricing epoch survives reopen', async () => { const first = await openManager('current-epoch'); await first.cm.compile(BUDGET); @@ -731,6 +746,28 @@ describe('estimator calibration band', () => { belowReopened.cm.close(); }); + it('dropping a provisional multiplier to 1 drops its declared mark in memory as on disk', async () => { + // An operator following the unwedge advice may seed provisional over a + // record a declared caller had marked. + const epoch = AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH; + const tight = await tightBudget('drop-declared-probe', 1.2); // fits at 1.0, not at 1.45 + const first = await openManager('drop-declared'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1.45, at: 1, pricing: epoch, provisional: true, declaredOverhead: true }); + first.cm.close(); + const { cm, strategy, internals } = await openManager('drop-declared'); + const declared = () => (internals as unknown as { _calibrationDeclared: boolean })._calibrationDeclared; + await calibrationWarnings(() => cm.compile(tight)); + assert.equal(internals._calibration, 1, 'premise: the refused compile dropped it'); + assert.equal(declared(), false); + // An undeclared sample confirming the 1 writes no mark either. + strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + const saved = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; declaredOverhead?: boolean }; + assert.deepStrictEqual([saved.multiplier, saved.declaredOverhead], [1, undefined]); + cm.close(); + }); + it('a discard\'s provisional 1 survives a restart until a sample confirms it', async () => { // Both discard writers (the load's and the drop's) persist the 1 as // provisional; a reopen before the first sample must not read it as @@ -1260,9 +1297,15 @@ describe('estimator calibration band', () => { }); it('a drift inside 2% is not applied; once the EMA passes 2% it is', async () => { + // A learned 1: a store with no record would be set by its first sample (#328). + const first = await openManager('deadband'); + await first.cm.compile(BUDGET); + first.internals.store!.setStateJson(first.internals.calibrationStateId, + { multiplier: 1, at: 1, pricing: AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH }); + first.cm.close(); const { cm, strategy, internals } = await openManager('deadband'); await cm.compile(BUDGET); - assert.equal(internals._calibration, 1); + assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null], 'learned'); // real/est = 1.01 every compile: the EMA creeps toward 1.01 and never // leaves the band, so nothing is repriced between compiles. for (let i = 0; i < 12; i++) { diff --git a/test/calibration-overhead.test.ts b/test/calibration-overhead.test.ts index c80bdb45..9841942f 100644 --- a/test/calibration-overhead.test.ts +++ b/test/calibration-overhead.test.ts @@ -508,6 +508,18 @@ describe('estimator calibration: a store that learned the overhead as a rate', ( cm.close(); }); + it('an undeclared sample between the two leaves the held sample, and the pair still replaces', async () => { + const { cm, strategy, internals } = await seeded('held-undeclared', { multiplier: 1.8, at: 1, pricing: epoch }); + await sampleAt(cm, strategy, internals, 1, true); + assert.equal(internals._calibration, 1.8, 'the first is held'); + await sampleAt(cm, strategy, internals, 1, false); + assert.ok(Math.abs(internals._calibration - 1.64) <= 0.01, `the undeclared sample steps, got ${internals._calibration}`); + await sampleAt(cm, strategy, internals, 1, true); + assert.ok(Math.abs(internals._calibration - 1) <= 0.01, `paired with the held sample and replaced, got ${internals._calibration}`); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + it('a consistent run between the two ends the held sample', async () => { const { cm, strategy, internals } = await seeded('held-run', { multiplier: 1, at: 1, pricing: epoch }); await sampleAt(cm, strategy, internals, 1, true); @@ -592,9 +604,10 @@ describe('estimator calibration: a store that learned the overhead as a rate', ( it('a provisional multiplier is replaced by one declared sample, and marked declared', async () => { const { cm, strategy, internals } = await seeded('provisional-declared', { multiplier: 1, at: 1, pricing: epoch, provisional: true }); - await sampleAt(cm, strategy, internals, 1, true); - const saved = internals.store!.getStateJson(internals.calibrationStateId) as { provisional?: boolean; declaredOverhead?: boolean }; - assert.deepEqual([saved.provisional, saved.declaredOverhead], [undefined, true]); + await sampleAt(cm, strategy, internals, 1.4, true); + assert.ok(Math.abs(internals._calibration - 1.4) <= 0.01, `replaced, not stepped, got ${internals._calibration}`); + assert.equal(internals._calibrationProvisional, null); + assert.equal(declaredOf(internals), true); cm.close(); }); @@ -610,15 +623,29 @@ describe('estimator calibration: a store that learned the overhead as a rate', ( it('an undeclared sample inside the dead band leaves a record marked, and a rejected declared one leaves it unmarked', async () => { const marked = await seeded('deadband-undeclared', { multiplier: 1, at: 1, pricing: epoch, declaredOverhead: true }); await sampleAt(marked.cm, marked.strategy, marked.internals, 1.01, false); - assert.deepEqual([marked.internals._calibration, marked.internals._calibrationDeclared, declaredOf(marked.internals)], [1, true, true]); + const ema = (marked.internals as unknown as { _calibrationEma: number | null })._calibrationEma; + assert.ok(ema !== null && Math.abs(ema - 1.002) < 0.001, `premise: the sample reached the EMA, got ${ema}`); + assert.deepStrictEqual([marked.internals._calibration, marked.internals._calibrationDeclared, declaredOf(marked.internals)], [1, true, true]); marked.cm.close(); const unmarked = await seeded('rejected-declared', { multiplier: 1, at: 1, pricing: epoch }); await unmarked.cm.compile(BUDGET); unmarked.strategy.reportRealInputTokens(unmarked.internals._lastCompileEstimate * 2.1, {}); - assert.deepEqual([unmarked.internals._calibration, unmarked.internals._calibrationDeclared, declaredOf(unmarked.internals)], [1, false, undefined]); + assert.equal((unmarked.internals as unknown as { _outOfBandRun: { count: number } | null })._outOfBandRun?.count, 1, 'premise: the sample was rejected as the start of a run'); + assert.deepStrictEqual([unmarked.internals._calibration, unmarked.internals._calibrationDeclared, declaredOf(unmarked.internals)], [1, false, undefined]); unmarked.cm.close(); }); + it('any overhead a JS caller passes but null or undefined counts as declared, in the run as in the mark', async () => { + const { cm, strategy, internals } = await seeded('falsy-overhead', { multiplier: 1, at: 1, pricing: epoch }); + for (let i = 0; i < 3; i++) { + await cm.compile(BUDGET); + strategy.reportRealInputTokens(internals._lastCompileEstimate * 2.1, 0 as unknown as RequestOverhead); + } + assert.equal(internals._calibration, 1.8, 'learned as a declared run'); + assert.equal(declaredOf(internals), true); + cm.close(); + }); + it('a null overhead from a JS caller counts as undeclared', async () => { const { cm, strategy, internals } = await seeded('null-overhead', { multiplier: 1, at: 1, pricing: epoch }); await cm.compile(BUDGET); From 7099a7b186c7621a2465e394dbf57e9c7d6967b4 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Fri, 23 Oct 2026 09:51:00 -0400 Subject: [PATCH 36/67] docs(calibration): say what the no-record, newer-epoch and drop paths do; pin the drop's dry restore From our internal review of the round. No behaviour changes. - The load comment separates the callers: an undeclared one took nine applied EMA steps to reach a true 1.4, a declared one two samples with the first held. - The newer-epoch comment and the epoch doc say this binary also writes its own record over a newer one when two declared samples confirm and mark the 1, beside its first applied sample. - The catch says a record the store can't read loads 1, not provisional. - The drop's comment says the mark goes with the multiplier dropped. - The #328 fragment and RenderStats.calibration's doc include a record with no multiplier to use, and say a newer-epoch or unreadable record loads 1 as before, not provisional. - Tests: the drop test loads the marked seed with a committing compile, shows a dry compile's drop is put back, then reads the record the live drop writes, with the seeded 1.45 in the reset; the newer-epoch test shows its confirming sample reached the EMA. Co-Authored-By: Claude Opus 5.5 --- .../328-fresh-calibration-provisional.fixed.md | 17 +++++++++-------- src/strategies/autobiographical.ts | 18 +++++++++++------- src/types/strategy.ts | 3 ++- test/calibration-band.test.ts | 13 ++++++++++++- 4 files changed, 34 insertions(+), 17 deletions(-) diff --git a/changelog.d/328-fresh-calibration-provisional.fixed.md b/changelog.d/328-fresh-calibration-provisional.fixed.md index 354929cf..1c7c4836 100644 --- a/changelog.d/328-fresh-calibration-provisional.fixed.md +++ b/changelog.d/328-fresh-calibration-provisional.fixed.md @@ -1,8 +1,9 @@ -- A store with no calibration record starts from a provisional 1, as a - discard's restart does (#328). Its first sample inside the calibration - band sets the multiplier, declared or not: one sample, without the - two-sample hold, where it stepped a fifth of the way per sample (nine - applied changes, each a reprice of every message and summary, to reach a - true 1.4). Until that sample, render stats report - `calibration.provisional: true`. A record from a newer pricing epoch, or a - read of the record that throws, still loads a learned 1. +- A store with no calibration record, or a record with no finite + multiplier, starts from a provisional 1, as a discard's restart does + (#328). Its first sample inside the calibration band sets the multiplier, + declared or not: one sample, without the two-sample hold, where it + stepped a fifth of the way per sample (nine applied changes, each a + reprice of every message and summary, to reach a true 1.4). Until that + sample, render stats report `calibration.provisional: true`. A record + from a newer pricing epoch, or a read of the record that throws, still + loads 1 as before, not provisional. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 9f4d86a3..a11798ca 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -14333,7 +14333,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * refusal, which it otherwise only reports. * * A binary from before a bump still learns after a rollback, and its first - * applied sample overwrites the newer epoch's record with its own stamp + * applied sample, or two declared samples that confirm and mark its 1, + * overwrite the newer epoch's record with its own stamp * (reportRealInputTokens); rolling forward again treats that as an older * record. */ @@ -14581,14 +14582,17 @@ export class AutobiographicalStrategy implements ResettableStrategy { // leaves it on disk and doesn't use it, and logs nothing, since // re-stamping it as the discard warning advises would destroy it. // This binary's learner still writes its own record over it on its - // first applied sample (see CALIBRATION_PRICING_EPOCH). + // first applied sample, or when two declared samples confirm and + // mark the 1 (see CALIBRATION_PRICING_EPOCH). } else if (!(typeof saved?.pricing === 'number' && saved.pricing > AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH)) { // No record, or none with a multiplier to use: nothing this store // kept says what its multiplier is, so its 1 is a guess, as a // discard's restart is, and the first sample the band accepts sets // it (#328). Learned, it would take the slowest path for the store - // with the least evidence, nine applied EMA steps to reach a true - // 1.4, each a reprice of every message and summary. Nothing is + // with the least evidence: for an undeclared caller nine applied + // EMA steps to reach a true 1.4, each a reprice of every message and + // summary, and for a declared one two samples, the first held. + // Nothing is // written: with no record, the next load decides the same, and the // first sample's write gives the stamp. A record from a newer epoch, // with a multiplier this binary can read or not, and a read that @@ -14597,7 +14601,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // write over even when it only confirms the 1. this._calibrationProvisional = { fromEpoch: null }; } - } catch { /* absent slot is fine */ } + } catch { /* a record the store can't read loads 1, not provisional: see the no-record branch above */ } } // A dry select changes the store's multiplier only where select() read it // first and can put it back (see MessageStoreView). @@ -14625,8 +14629,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const reset: CalibrationReset = { discardedMultiplier: discarded, fromEpoch: provisional.fromEpoch, at: Date.now() }; this._calibration = 1; this._calibrationReset = reset; - // The mark was about the multiplier dropped; the record below is written - // without it, and memory agrees. + // The mark was about the multiplier dropped, and goes with it, as the + // record below is written without it. this._calibrationDeclared = false; this.applyCalibration(); if (dryRun) return true; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index fbc2dfd1..096b557d 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -747,7 +747,8 @@ export interface RenderStats { * than stepping toward it (see * `AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH`): one kept from a * record without a stamp, the 1 a discard restarts from, or the 1 a store - * with no calibration record starts from. A discard's 1 stays + * with no calibration record, or none with a multiplier to use, starts + * from. A discard's 1 stays * provisional through a later epoch's load, which re-stamps it whatever * that epoch repriced. * `reset` is present once a multiplier from an older or unknown epoch has diff --git a/test/calibration-band.test.ts b/test/calibration-band.test.ts index f40bc215..0325ef91 100644 --- a/test/calibration-band.test.ts +++ b/test/calibration-band.test.ts @@ -227,6 +227,8 @@ describe('estimator calibration band', () => { assert.deepStrictEqual([internals._calibration, internals._calibrationProvisional], [1, null]); // A first sample that only confirms the 1 writes nothing over it. strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); + const ema = (internals as unknown as { _calibrationEma: number | null })._calibrationEma; + assert.ok(ema !== null && Math.abs(ema - 1.002) < 0.001, `premise: the sample reached the EMA, got ${ema}`); assert.deepStrictEqual(internals.store!.getStateJson(internals.calibrationStateId), future); cm.close(); }); @@ -758,9 +760,18 @@ describe('estimator calibration band', () => { first.cm.close(); const { cm, strategy, internals } = await openManager('drop-declared'); const declared = () => (internals as unknown as { _calibrationDeclared: boolean })._calibrationDeclared; + await cm.compile(BUDGET); // loads the seed, which fits this budget + assert.deepStrictEqual([internals._calibration, declared()], [1.45, true], 'premise: loaded, marked'); + // A dry compile drops it too, in the dry compile only: select() puts the mark back. + const dry = await calibrationWarnings(() => cm.compile(tight, undefined, { dryRun: true })); + assert.equal(dry.result.renderStats!.calibration!.multiplier, 1, 'premise: the dry compile dropped it'); + assert.deepStrictEqual([internals._calibration, declared()], [1.45, true], 'put back after the dry compile that dropped it'); await calibrationWarnings(() => cm.compile(tight)); - assert.equal(internals._calibration, 1, 'premise: the refused compile dropped it'); + assert.equal(internals._calibration, 1); + assert.equal(cm.getRenderStats()!.calibration!.reset?.discardedMultiplier, 1.45, 'premise: the seeded 1.45 was loaded and dropped'); assert.equal(declared(), false); + const dropped = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; provisional?: boolean; declaredOverhead?: boolean }; + assert.deepStrictEqual([dropped.multiplier, dropped.provisional, dropped.declaredOverhead], [1, true, undefined], 'the record the drop writes'); // An undeclared sample confirming the 1 writes no mark either. strategy.reportRealInputTokens(internals._lastCompileEstimate * 1.01); const saved = internals.store!.getStateJson(internals.calibrationStateId) as { multiplier?: number; declaredOverhead?: boolean }; From eb80967c66afce5d0518c3a18f32991ee395359c Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 10:10:18 -0400 Subject: [PATCH 37/67] fix(pins): a shard locked at a fold counts as freed; every put-back restores the refused select's frontier; several narrowed pins' warnings say so - freedByNarrowing counted no locked shard as freed. A pin renders a locked message raw over its lock (lockChunk), and narrowed, the message returns to its locked level, so only a message locked raw stays raw either way. A compile that narrowing the pin would answer was refused where the message's other parts were locked at a fold. - The error path puts back the refused select's preview, frontier and transition status with the pins, as the no-fit path does. - With several pins narrowed by one compile, each warning's total is the compile with all of them whole, and the warning now says so. - pinSpan refuses a reading other than 'ids' or 'whole', so a JavaScript caller that leaves it out fails rather than reading every pin whole. - Text: narrowing applies to a pin that renders the message raw (README, the changelog fragment, ProtectedRange, markDocument, design doc 3.10); the upgrade's rewrite can start earlier wherever the select folds older history further to make room; setStrategy is among the loads that clear narrowings; comments on the narrowing loop's order, the prompt's reading, compressionPinnedPositions and the replay exporter. - Tests (40 in whole-body-pins): parts locked at a fold; an error from a narrowed select after it planned; a mark made after its message's early parts were chunked as it arrived; the two-pin warning's wording. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 4 +-- docs/adaptive-resolution-design.md | 5 +-- scripts/export-kv-replay-metadata.mjs | 5 +-- src/context-manager.ts | 4 +-- src/strategies/autobiographical.ts | 48 ++++++++++++++----------- src/types/strategy.ts | 5 +-- test/whole-body-pins.test.ts | 52 ++++++++++++++++++++++++++- 8 files changed, 92 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index f352e91d..eda52912 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows it, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch or a fork). `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 254beec3..069e0c7e 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ - **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. - - **When the whole messages don't fit.** A compile refused over budget narrows pins back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch or a fork), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on; under kv-unified, whose solver weighs the whole request, the rewrite can start earlier. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. + - **When the whole messages don't fit.** A compile refused over budget narrows pins that render the message raw back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch, a fork or `setStrategy`), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. + - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/docs/adaptive-resolution-design.md b/docs/adaptive-resolution-design.md index 1614a1bd..7a999cee 100644 --- a/docs/adaptive-resolution-design.md +++ b/docs/adaptive-resolution-design.md @@ -376,8 +376,9 @@ This case should be rare in practice with unbounded L_n: the picker can almost a > > *(Pins on messages stored in parts)* A pin or document mark covers the > whole of a message stored in parts in the selects. Where that is what -> doesn't fit, a refusal is answered first by narrowing the pin to the parts -> its ids name (`selectNarrowingPins`), and the select runs again: the +> doesn't fit, a refusal is answered first by narrowing a pin that renders +> the message raw to the parts its ids name (`selectNarrowingPins`), and the +> select runs again: the > narrowing lasts until the store is loaded again, and a host-log warning > names the pin. When no narrowing lets the compile through, the refusal is > thrown as above. diff --git a/scripts/export-kv-replay-metadata.mjs b/scripts/export-kv-replay-metadata.mjs index 4f70ea78..6b082a1a 100644 --- a/scripts/export-kv-replay-metadata.mjs +++ b/scripts/export-kv-replay-metadata.mjs @@ -19,8 +19,9 @@ try { strategy.loadCalibration(view); const messages = view.getAll(); const estimates = strategy.postStripEstimates(view); - // The live selector's reading: under adaptive resolution a pin covers the - // whole of a message stored in parts. + // The selects' reading, as at a restart: under adaptive resolution a pin + // covers the whole of a message stored in parts. A narrowing a live compile + // made isn't stored, so this reads that pin whole too. const pins = strategy.pinnedPositions(messages, strategy.selectsPinReading()); const bounds = strategy.pinLevelBounds(messages, strategy.selectsPinReading()); const metadata = messages.map((message, index) => ({ diff --git a/src/context-manager.ts b/src/context-manager.ts index fb069917..e8efb3c9 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1684,8 +1684,8 @@ export class ContextManager { * `markDocument(id, { maxLevel })` lets it fold no deeper than that level * where the selector reads level bounds. Under adaptive resolution, a mark * on any part of a message stored in parts renders the whole message, as - * `pinRange` does, narrowed to the part it names where the whole message - * doesn't fit. + * `pinRange` does: where the whole message doesn't fit, a mark that renders + * it raw is narrowed to the part it names. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 9ba10066..7cf87a1f 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -3576,8 +3576,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. * Under adaptive resolution, a mark on any part of a message stored in - * parts renders the whole message, as `pinRange` does, narrowed to the part - * it names where the whole message doesn't fit (`selectNarrowingPins`). + * parts renders the whole message, as `pinRange` does: where the whole + * message doesn't fit, a mark that renders it raw is narrowed to the part + * it names (`selectNarrowingPins`). */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -3814,6 +3815,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { messages: readonly StoredMessage[], reading: 'ids' | 'whole', ): { lo: number; hi: number } | undefined { + if (reading !== 'ids' && reading !== 'whole') throw new TypeError(`pinSpan: a reading is 'ids' or 'whole', not ${String(reading)}`); const first = positionOf.get(pin.firstMessageId); const last = positionOf.get(pin.lastMessageId); if (first === undefined || last === undefined) return undefined; @@ -8066,13 +8068,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { const narrowed: ProtectedRange[] = []; const putBack = (): void => { for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); + this._lastPreview = preview; + this.lastFrontierTokens = frontier; + this.transitionBlocked = blocked; }; let entries: ContextEntry[] | undefined; try { - // Narrowing a raw pin changes neither another pin's reading nor what - // compression leaves out (a part it stops covering stays raw as the - // selects read it only where another raw pin's ids keep it so), so the - // order taken here holds throughout. + // Narrowing a raw pin changes no other pin's reading, and nothing it + // stops covering leaves what compression leaves out (a position there + // is raw by some pin's ids, and so by that pin's whole reading too), so + // the order taken here holds throughout. for (const { pin } of this.freedByNarrowing(store)) { this.narrowedPins.add(this.narrowKey(pin)); narrowed.push(pin); @@ -8092,12 +8097,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { } if (!entries) { putBack(); - this._lastPreview = preview; - this.lastFrontierTokens = frontier; - this.transitionBlocked = blocked; throw refusal; } - if (opts?.dryRun === true) putBack(); + if (opts?.dryRun === true) for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); else this.warnNarrowedPins(narrowed, refusal); return entries; } @@ -8110,8 +8112,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { * have an L1 the select can fold them to (their chunk's summary, else * `l1ByMessageIndex`, as selectAdaptive finds it), priced as the selects * price them (`postStripEstimates`), less those the selects hold at a - * level or that are locked. Only those can go out as summaries once it's - * narrowed. Where the selects read bounds, a pin with a level or cap above + * level, and those locked raw: a pin renders a locked message raw over its + * lock (`lockChunk`), and narrowed, the message returns to its locked + * level, which frees something only where that level is a fold. Only those + * can go out as summaries once it's narrowed. Where the selects read bounds, a pin with a level or cap above * 0 frees nothing, since its parts are held at that level: its parts * already fold, and where it overlaps a raw pin, narrowing it could turn * parts raw. Elsewhere every pin renders raw and can be narrowed. A pin @@ -8127,8 +8131,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const reading = this.selectsPinReading(); // A pin keeps raw only where the selects resolve it raw: where they read // bounds, a position a pin holds at a level (this one's own, if it has a - // level or cap) stays held, and a locked one stays as it is, whether or - // not this pin covers it. So a pin with a level or cap frees nothing there. + // level or cap) stays held, so a pin with a level or cap frees nothing + // there. A message locked raw stays raw narrowed too. const bounds = this.resolvedPinBounds(messages, reading); const prices = this.postStripEstimates(store); const out: Array<{ pin: ProtectedRange; tokens: number }> = []; @@ -8140,7 +8144,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (let i = whole.lo; i <= whole.hi; i++) { const id = messages[i].id; const bound = bounds.get(i); - const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound) && !this.locked.has(id); + const lockedRaw = this.locked.has(id) && (this.resolutions.get(id) ?? 0) === 0; + const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound) && !lockedRaw; if ((i < ids.lo || i > ids.hi) && raw && compressible.has(id) && summarized.has(id)) tokens += prices[i] ?? 0; } if (tokens > 0) out.push({ pin, tokens }); @@ -8158,7 +8163,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { ? `only message ${pin.firstMessageId}, one part of a message stored in parts, not the whole message` : `only messages ${pin.firstMessageId} to ${pin.lastMessageId}, not the whole of the messages stored in parts at its ends`; this.diag.warn( - `autobio: ${what} ${label} now covers ${covers}. With it covering ${one ? 'the whole message' : 'them whole'}, ` + + `autobio: ${what} ${label} now covers ${covers}. ${narrowed.length > 1 ? `With the ${narrowed.length} pins narrowed here covering their whole messages` : `With it covering ${one ? 'the whole message' : 'them whole'}`}, ` + `the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget}` + `${refusal.diagnostics.configuredBudget === undefined ? '' : ` (maxTokens ${refusal.diagnostics.configuredBudget})`}. Until this ` + `context is loaded again, the parts it no longer covers fold like any history. Loaded again, as at a restart, the ${what} ` + @@ -9399,9 +9404,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { // optimization that scales catastrophically: a 4000-message import // converged to ~500 L1s that never aged out, blowing the 200k // window around chunk 118. - // Pins in the canonical L1 prompt, read by their ids as compression reads - // them (forceRawPinPositions): a marked message's other parts reach the - // prompt as their summaries, though the selects send them raw. A + // Pins in the canonical L1 prompt, read by their ids (PROMPT_PIN_READING, + // forceRawPinPositions): a marked message's other parts reach the prompt + // as their summaries, though the selects send them raw. A // frontier summary that can't be shown as a recall pair (it covers such a // message, or its content is empty) is DESCENDED, not dropped: its // children take its place, so only the branch holding the pin opens, @@ -17096,8 +17101,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `selectsPinReading`). A part a pin's ids name that another pin's whole * reading holds at a level isn't one, since the selects need its summary, * and a part a pin covers only as the selects read it isn't either: its - * summary is what a narrowing sends. No chunk made after the pin takes a - * position in this set, so what relies on that reads it. + * summary is what a narrowing sends. No chunk made while a position is in + * this set takes it (a chunk recorded before keeps its messages), so what + * relies on that reads it. */ protected compressionPinnedPositions(messages: StoredMessage[]): Set { // Where the selects read pins by their ids too, the two sets are one. diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 6c52f1fe..97a11b22 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -2097,8 +2097,9 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * pinned level, compiles fail until each part of the message in the middle * (outside the head and recent windows) has the summary that level needs. * When a compile can't hold the whole messages within its budget but can - * hold the rest with the pin narrowed to its ids, it narrows it until the - * strategy loads the store again (an open, a branch switch or a fork), and + * hold the rest with the pin narrowed to its ids, it narrows a pin it renders + * raw (one held at a level frees nothing narrowed) until the strategy loads + * the store again (an open, a branch switch, a fork or `setStrategy`), and * warns. Nothing records the narrowing, so a store loaded again reads the * pin whole. * diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 074ca370..bc005325 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -85,14 +85,19 @@ class PinStore extends AutobiographicalStrategy { /** How many adaptive selects ran, and the refusals they threw. */ selects = 0; refusals: OverBudgetError[] = []; + /** Fail a select that a narrowed pin let through, after it planned. */ + failNarrowedAfterPlanning = false; protected selectAdaptive(store: never, budget: never, opts?: never): never { this.selects++; + let entries: never; try { - return super.selectAdaptive(store, budget, opts) as never; + entries = super.selectAdaptive(store, budget, opts) as never; } catch (err) { if (err instanceof OverBudgetError) this.refusals.push(err); throw err; } + if (this.failNarrowedAfterPlanning && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) throw new Error('select failed'); + return entries; } /** * Leave the L1s covering these messages as the only record of their @@ -241,6 +246,21 @@ describe('pins render whole messages', () => { }); } + it('a mark made after its message\'s early parts were chunked as it arrived sends the whole message raw', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + assert.ok(strategy.chunkedIds().has(parts[1]!), 'the second part was chunked before the mark'); + cm.markDocument(b); + await traffic(cm, 360); + await ready(cm); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + it('a range pin whose ends fall inside long messages covers both of them whole', async () => { await withDir(async (dir) => { const { cm } = await open(dir); @@ -491,6 +511,34 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); }); + it('a mark over parts locked at a fold is narrowed: narrowed, they return to their locked level', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + // The earlier version's compile folded the message's later parts; lock them there. + const parts = partsOf(cm, strategy.firstIdOf(marks[0]!)); + for (const id of parts.slice(1)) strategy.lockChunk(id); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc'), rawWords(before, 'Bdoc')); + assert.ok(strategy.isNarrowed(marks[0]!)); + await cm.close(); + }); + }); + + it('an error from a narrowed select leaves the refused select\'s frontier and preview', async (t) => { + await withDir(async (dir) => { + warnings(t); + const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); + strategy.refusals = []; + // The narrowed select plans, setting its own frontier, and then fails. + strategy.failNarrowedAfterPlanning = true; + await assert.rejects(cm.compile(budget(30_000)), /select failed/); + strategy.failNarrowedAfterPlanning = false; + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.strictEqual(strategy.getHotContextSettings().currentFrontierTokens, strategy.refusals[0]!.actual); + await cm.close(); + }); + }); + it('the warning names a named mark and its part, and gives the whole reading\'s refusal', async (t) => { await withDir(async (dir) => { const out = warnings(t); @@ -608,6 +656,8 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { assert.strictEqual(said.length, 2); assert.match(said[0]!, new RegExp(`document mark ${marks[0]} `)); assert.match(said[1]!, new RegExp(`document mark ${marks[1]} `)); + // The refusal's total had both whole, and each warning says so. + for (const w of said) assert.ok(w.includes('With the 2 pins narrowed here covering their whole messages, the compile came to ~'), w); await cm.close(); }); }); From 666de273a2b69839f525e726cba8e61237d83e4d Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 10:21:25 -0400 Subject: [PATCH 38/67] fix(pins): a part locked with lockChunk keeps its lock under a mark on another part of its message Under kv-unified, a raw pin and a lock are two constraints, so a mark's whole reading over parts locked at a fold made every compile fail ("kv-unified constraints are infeasible"), at any budget, where the earlier version compiled. Narrowing couldn't answer it, since the conflict isn't a refusal. Under the greedy solvers the whole reading rendered those parts raw over their locks instead. pinPositions gives the positions a pin covers in a reading: its span, less the locked messages that only the whole reading adds. A lock is said of that part, while the whole reading is what a mark on another part implies, so the lock holds there. A pin over its own locked messages keeps the rule lockChunk describes. pinnedPositions, pinLevelBounds and freedByNarrowing read it, so the lock rule the previous commit added to freedByNarrowing is gone. Docs: lockChunk, pinSpan, ProtectedRange, the README and the changelog fragment. Tests (42 in whole-body-pins): parts locked at a fold keep their lock, and the store sends the request it sent before (adaptive and kv-unified); a mark over its own part locked at a fold renders it raw, as before. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- src/strategies/autobiographical.ts | 58 +++++++++++++++++++--------- src/types/strategy.ts | 3 +- test/whole-body-pins.test.ts | 41 ++++++++++++++++---- 5 files changed, 77 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index eda52912..2cc1f260 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk`, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 069e0c7e..32603990 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ -- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. +- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` keeps its lock: the whole reading leaves it out. - **When the whole messages don't fit.** A compile refused over budget narrows pins that render the message raw back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch, a fork or `setStrategy`), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 7cf87a1f..c4e93ff6 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2567,7 +2567,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * lock while it stands: under the greedy solvers and kv-stable, the pinned * message renders raw, and its locked level stays stored and returns when * the pin is removed. kv-unified treats a pin and a lock as constraints and - * reports a conflict when they are incompatible. + * reports a conflict when they are incompatible. A pin whose ids name + * another part of a message stored in parts doesn't cover a locked part + * (`pinPositions`): there the lock holds, under every solver. * No-op when adaptiveResolution is false. Set-only programmatic API per * the design (no agent-facing tool in V1). Persisted to chronicle. */ @@ -3806,7 +3808,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * last part of the message its last id is in. The selects read this under * adaptive resolution (`selectsPinReading`), so the whole message goes out * raw where the budget holds it. A pin narrowed in this instance - * (`narrowedPins`) reads by its ids here too. + * (`narrowedPins`) reads by its ids here too. A part locked with + * `lockChunk` outside the pin's ids isn't covered (`pinPositions`). * Undefined when either id is no longer in the store. */ protected pinSpan( @@ -3825,6 +3828,31 @@ export class AutobiographicalStrategy implements ResettableStrategy { return { lo: bodyBoundsIn(messages, lo).from, hi: bodyBoundsIn(messages, hi).to }; } + /** + * The store positions a pin covers in `reading`: its span (`pinSpan`), + * less the messages locked with `lockChunk` that only the whole reading + * adds. A lock is said of that part, while the whole reading is what a mark + * on another part implies, so the lock holds there. A pin over its own + * locked messages keeps the rule `lockChunk` describes. Undefined when + * either id is no longer in the store. + */ + protected pinPositions( + pin: ProtectedRange, + positionOf: ReadonlyMap, + messages: readonly StoredMessage[], + reading: 'ids' | 'whole', + ): number[] | undefined { + const span = this.pinSpan(pin, positionOf, messages, reading); + if (!span) return undefined; + const ids = reading === 'ids' ? span : this.pinSpan(pin, positionOf, messages, 'ids')!; + const out: number[] = []; + for (let i = span.lo; i <= span.hi; i++) { + if ((i < ids.lo || i > ids.hi) && this.locked.has(messages[i].id)) continue; + out.push(i); + } + return out; + } + /** * The reading the selects take (`pinSpan`), the agent's view, with what * must agree with them (the narrowing's candidates, and what compression @@ -3847,9 +3875,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const positionOf = this.positionIndex(messages); const out = new Set(); for (const pin of this.pins) { - const span = this.pinSpan(pin, positionOf, messages, reading); - if (!span) continue; - for (let i = span.lo; i <= span.hi; i++) out.add(i); + for (const i of this.pinPositions(pin, positionOf, messages, reading) ?? []) out.add(i); } return out; } @@ -3920,9 +3946,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const pin of this.pins) { if (pin.level === undefined && pin.maxLevel === undefined) continue; - const span = this.pinSpan(pin, positionOf, messages, reading); - if (!span) continue; - for (let i = span.lo; i <= span.hi; i++) { + for (const i of this.pinPositions(pin, positionOf, messages, reading) ?? []) { const prev = out.get(i) ?? {}; // A fixed level is the strongest constraint; when two pins fix the same // position, the shallower (lower) level wins (finest requirement). @@ -8112,11 +8136,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { * have an L1 the select can fold them to (their chunk's summary, else * `l1ByMessageIndex`, as selectAdaptive finds it), priced as the selects * price them (`postStripEstimates`), less those the selects hold at a - * level, and those locked raw: a pin renders a locked message raw over its - * lock (`lockChunk`), and narrowed, the message returns to its locked - * level, which frees something only where that level is a fold. Only those - * can go out as summaries once it's narrowed. Where the selects read bounds, a pin with a level or cap above - * 0 frees nothing, since its parts are held at that level: its parts + * level. A part locked with `lockChunk` isn't among them: the whole reading + * leaves it out (`pinPositions`). Only those can go out as summaries once + * it's narrowed. Where the selects read bounds, a pin with a level or cap + * above 0 frees nothing, since its parts are held at that level: its parts * already fold, and where it overlaps a raw pin, narrowing it could turn * parts raw. Elsewhere every pin renders raw and can be narrowed. A pin * that frees nothing, a narrowed one included, isn't listed. @@ -8132,20 +8155,19 @@ export class AutobiographicalStrategy implements ResettableStrategy { // A pin keeps raw only where the selects resolve it raw: where they read // bounds, a position a pin holds at a level (this one's own, if it has a // level or cap) stays held, so a pin with a level or cap frees nothing - // there. A message locked raw stays raw narrowed too. + // there. const bounds = this.resolvedPinBounds(messages, reading); const prices = this.postStripEstimates(store); const out: Array<{ pin: ProtectedRange; tokens: number }> = []; for (const pin of this.pins) { - const whole = this.pinSpan(pin, positionOf, messages, reading); + const whole = this.pinPositions(pin, positionOf, messages, reading); const ids = this.pinSpan(pin, positionOf, messages, 'ids'); if (!whole || !ids) continue; let tokens = 0; - for (let i = whole.lo; i <= whole.hi; i++) { + for (const i of whole) { const id = messages[i].id; const bound = bounds.get(i); - const lockedRaw = this.locked.has(id) && (this.resolutions.get(id) ?? 0) === 0; - const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound) && !lockedRaw; + const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound); if ((i < ids.lo || i > ids.hi) && raw && compressible.has(id) && summarized.has(id)) tokens += prices[i] ?? 0; } if (tokens > 0) out.push({ pin, tokens }); diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 97a11b22..a3d6b1f2 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -2087,7 +2087,8 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * A message stored in parts (a long message that ingress splits under * `adaptiveResolution`) counts as one where a pin is rendered: under adaptive * resolution, an end inside one covers it from its first part to its last in - * the selects, the agent's view. The summarizer's prompt reads a pin by its + * the selects, the agent's view, less a part locked with `lockChunk` that the + * pin's ids don't name, where the lock holds. The summarizer's prompt reads a pin by its * ids, as compression does, so a raw pin keeps out of compression only the * parts its ids name (a part chunked before the pin was made stays in its * chunk, as before), and the message's other parts are chunked and diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index bc005325..650954c5 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -112,6 +112,10 @@ class PinStore extends AutobiographicalStrategy { record.compressed = false; } } + /** The stored resolution of a message (0 is raw). */ + resolutionOf(id: string): number { + return this.resolutions.get(id) ?? 0; + } /** The id a pin's range starts at. */ firstIdOf(pinId: string): string { return this.pinById(pinId).firstMessageId; @@ -511,15 +515,36 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); }); - it('a mark over parts locked at a fold is narrowed: narrowed, they return to their locked level', async (t) => { + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: a part locked at a fold keeps its lock, so a mark on another part sends the request it sent before`, async (t) => { + await withDir(async (dir) => { + const out = warnings(t); + const { cm, strategy, marks, before } = await upgradedStore(dir, { kind, bodies: [['Bdoc', 6000]], budget: 30_000 }); + // The earlier version's compile folded the message's later parts; lock them there. + const parts = partsOf(cm, strategy.firstIdOf(marks[0]!)); + for (const id of parts.slice(1)) strategy.lockChunk(id); + assert.ok(parts.slice(1).every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); + assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), rawWords(before, 'Bdoc')); + assert.ok(!strategy.isNarrowed(marks[0]!)); + assert.deepStrictEqual(narrowingWarnings(out), []); + await cm.close(); + }); + }); + } + + it('a mark over its own part locked at a fold renders it raw over the lock, as before', async () => { await withDir(async (dir) => { - warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - // The earlier version's compile folded the message's later parts; lock them there. - const parts = partsOf(cm, strategy.firstIdOf(marks[0]!)); - for (const id of parts.slice(1)) strategy.lockChunk(id); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc'), rawWords(before, 'Bdoc')); - assert.ok(strategy.isNarrowed(marks[0]!)); + const { cm, strategy } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + await cm.compile(budget(30_000)); + assert.ok(strategy.resolutionOf(b) > 0, 'the first part folded'); + strategy.lockChunk(b); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); await cm.close(); }); }); From 15ccf84c14094f0a829e4847d1185eb048690f99 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Fri, 23 Oct 2026 21:16:38 +0200 Subject: [PATCH 39/67] chore(release): 0.20.1 Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 25 +++++++++++++++++++ changelog.d/175-load-repair-report.added.md | 1 - .../317-compression-head-as-of.fixed.md | 7 ------ .../317-compression-recall-before.changed.md | 4 --- changelog.d/compile-render-stats.added.md | 1 - package-lock.json | 4 +-- package.json | 2 +- 7 files changed, 28 insertions(+), 16 deletions(-) delete mode 100644 changelog.d/175-load-repair-report.added.md delete mode 100644 changelog.d/317-compression-head-as-of.fixed.md delete mode 100644 changelog.d/317-compression-recall-before.changed.md delete mode 100644 changelog.d/compile-render-stats.added.md diff --git a/CHANGELOG.md b/CHANGELOG.md index a2b40b8b..1c6db09b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,31 @@ Releases up to and including 0.6.2 predate this file; for their contents see ## Unreleased +## 0.20.1 — 2026-10-23 + +### Added + +- `AutobiographicalStrategy.setLoadRepairHandler(fn)` tells a host when a load's repair of the summary state drops summaries holding text, so whoever keeps the agent can learn that its memory changed from more than a log line (#175). A load runs at each initialization (an open, a branch switch, a fork, `setStrategy`) and at each take-up of what another instance on the store wrote. The handler gets one `LoadRepairReport` per such load. The report holds no summary text. For each dropped summary that held text, it gives the id, the level and the stored `tokens` (each null where it isn't a number), why it was dropped (`unreadable`, `names-missing`, `names-not-below` or `names-non-child`), and, where its range can be read, the messages it covered, with each end's stored time where the strategy's message view gives it. It also gives the sum of those tokens, how many empty summaries the same repair dropped, the namespace, and where the old summaries read back from: `getStateJsonAt(stateId, sequenceBefore)`, with `branch`, the Chronicle branch the repair wrote on, current. A repair that drops only empty summaries, or none, makes no report, and the repair's diagnostic line is written as before. A report made while a handler is set goes to it once its load's synchronous run has ended, never during it. Reports made while no handler is set, such as an open's where the host sets the handler after opening, are held in memory and handed to the handler, oldest first, when it's set. Each strategy instance holds its own. A handler that throws, or whose returned promise or thenable rejects, is logged, and stops neither the load nor the next report. `getLatestLoadRepairReport()` gives the newest report the instance has made, whether held or handed on, for a health surface to show. A load that drops text makes no report only where the store's branch or sequence can't be read, which is logged. + +- `CompileResult.renderStats`: each compile's result carries the render stats its own select recorded. A dry compile's (`dryRun`) are its own, which `getRenderStats()` never returns, so a host can show the composition of the request a dry compile built, beside the last live compile's, without sending anything. A live compile's are the object `getRenderStats()` then returns. Strategies supply them through an optional `RenderStatsCapableStrategy.getLastSelectStats()`; a strategy without it gives compiles none. + +### Changed + +- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). A source-only request (`compressionSourceOnly`) recalls none, before or after. + - **Before:** it recalled every summary on the frontier, as 0.15.0's entry said: "Only pins before the chunk being compressed count, so a summary after it is recalled as before." So a chunk compressed after later history had been summarized was given those later summaries too, ahead of it. That's the old head's chunk, which a head-window reset makes compressible late, or a pinned or quarantined chunk let go late. + - **Now:** a summary that starts after the history being summarized isn't recalled, and its messages stay out of the request, as before. Pins still count only before the chunk. + - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. When a refused request is retried with a recalled summary above L1 replaced by its children (the recall-curve fallback), a straddling summary's later child is recalled too: the same later part, as a summary of its own. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. + +### Fixed + +- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request for history from before the reset no longer gets the new topic's opening as its head, ahead of the history it summarizes (#317). A source-only request (`compressionSourceOnly` or `compressionMergeSourceOnly`) carries no head, so this didn't reach it. The other way later history reached a compression request, its recall pairs, is under Changed. + - **The head it gets now:** the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. + - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. + - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. + - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. + - **Unchanged:** what every compile selects and sends, and every summary already written. What an over-budget refusal names as clearing it (`clearedBy`) follows tick's deferral, which reads the request's head. + - **Prompt cache:** a request for older history after a reset carries on the prefix it had before the reset, where it used to restart with the new topic, and restart again each time the live head grew. + ## 0.20.0 — 2026-10-23 ### Breaking diff --git a/changelog.d/175-load-repair-report.added.md b/changelog.d/175-load-repair-report.added.md deleted file mode 100644 index a7de72eb..00000000 --- a/changelog.d/175-load-repair-report.added.md +++ /dev/null @@ -1 +0,0 @@ -- `AutobiographicalStrategy.setLoadRepairHandler(fn)` tells a host when a load's repair of the summary state drops summaries holding text, so whoever keeps the agent can learn that its memory changed from more than a log line (#175). A load runs at each initialization (an open, a branch switch, a fork, `setStrategy`) and at each take-up of what another instance on the store wrote. The handler gets one `LoadRepairReport` per such load. The report holds no summary text. For each dropped summary that held text, it gives the id, the level and the stored `tokens` (each null where it isn't a number), why it was dropped (`unreadable`, `names-missing`, `names-not-below` or `names-non-child`), and, where its range can be read, the messages it covered, with each end's stored time where the strategy's message view gives it. It also gives the sum of those tokens, how many empty summaries the same repair dropped, the namespace, and where the old summaries read back from: `getStateJsonAt(stateId, sequenceBefore)`, with `branch`, the Chronicle branch the repair wrote on, current. A repair that drops only empty summaries, or none, makes no report, and the repair's diagnostic line is written as before. A report made while a handler is set goes to it once its load's synchronous run has ended, never during it. Reports made while no handler is set, such as an open's where the host sets the handler after opening, are held in memory and handed to the handler, oldest first, when it's set. Each strategy instance holds its own. A handler that throws, or whose returned promise or thenable rejects, is logged, and stops neither the load nor the next report. `getLatestLoadRepairReport()` gives the newest report the instance has made, whether held or handed on, for a health surface to show. A load that drops text makes no report only where the store's branch or sequence can't be read, which is logged. diff --git a/changelog.d/317-compression-head-as-of.fixed.md b/changelog.d/317-compression-head-as-of.fixed.md deleted file mode 100644 index 581fc94c..00000000 --- a/changelog.d/317-compression-head-as-of.fixed.md +++ /dev/null @@ -1,7 +0,0 @@ -- After a head-window reset (`ContextManager.resetHeadWindow()`), a compression or merge request for history from before the reset no longer gets the new topic's opening as its head, ahead of the history it summarizes (#317). A source-only request (`compressionSourceOnly` or `compressionMergeSourceOnly`) carries no head, so this didn't reach it. The other way later history reached a compression request, its recall pairs, is under Changed. - - **The head it gets now:** the head that history followed, the opening or its own topic's transition and opening, cut at the request's first message. A merge gets the head as of its first message. - - **Which head:** before the live head's start, the one starting at the latest Context Manager message whose text starts with `[Topic Transition]` at or before the request's first message, or the opening when there is none. That's the store as a restart reads it, so for that history a strategy-level `resetHeadWindow` to a message that isn't a transition doesn't count. It ends by the walk the live head takes, priced with today's `headWindowTokens` and calibration. There, a chunk that starts at a transition gets no head, and so does the opening's own chunk: nothing came before either. - - **From the live head's start on,** a request still gets the live head, but now cut at its first message. So a queued chunk inside the live head window (one a widened window grew over) no longer gets the head's messages after it, and before the host declares its tools, tick no longer defers it for a tool cycle among them. - - **Removed messages:** where history starts is its first message still in the store, for a queued chunk, a summary (its first leaf still there) and a merge alike: a host can remove stored messages (agent-framework's Discord module does) while a summary over them or a queued chunk holding them lives on. A summary whose first message was removed is still recalled, in its place, so its other messages keep a form. History with no message left gets no head and recalls nothing: nothing in the store says when it happened. Before, a summary whose first message was removed was left out of a merge's recall while its other messages stayed out too, and a compression request placed it after newer ones. - - **Unchanged:** what every compile selects and sends, and every summary already written. What an over-budget refusal names as clearing it (`clearedBy`) follows tick's deferral, which reads the request's head. - - **Prompt cache:** a request for older history after a reset carries on the prefix it had before the reset, where it used to restart with the new topic, and restart again each time the live head grew. diff --git a/changelog.d/317-compression-recall-before.changed.md b/changelog.d/317-compression-recall-before.changed.md deleted file mode 100644 index d2644fd4..00000000 --- a/changelog.d/317-compression-recall-before.changed.md +++ /dev/null @@ -1,4 +0,0 @@ -- A compression request recalls only the summaries that start before the history it summarizes, the rule a merge's recall already used (#317). A source-only request (`compressionSourceOnly`) recalls none, before or after. - - **Before:** it recalled every summary on the frontier, as 0.15.0's entry said: "Only pins before the chunk being compressed count, so a summary after it is recalled as before." So a chunk compressed after later history had been summarized was given those later summaries too, ahead of it. That's the old head's chunk, which a head-window reset makes compressible late, or a pinned or quarantined chunk let go late. - - **Now:** a summary that starts after the history being summarized isn't recalled, and its messages stay out of the request, as before. Pins still count only before the chunk. - - **What still reaches a request from after it:** a summary that starts before the history being summarized and reaches past it is still recalled, since what it covers before that history has no other form there. So its later part reaches the request, and only where a summary straddles the chunk. When a refused request is retried with a recalled summary above L1 replaced by its children (the recall-curve fallback), a straddling summary's later child is recalled too: the same later part, as a summary of its own. Summaries written before this change are recalled as written, and one written after a reset may already carry the new topic. diff --git a/changelog.d/compile-render-stats.added.md b/changelog.d/compile-render-stats.added.md deleted file mode 100644 index 11342502..00000000 --- a/changelog.d/compile-render-stats.added.md +++ /dev/null @@ -1 +0,0 @@ -- `CompileResult.renderStats`: each compile's result carries the render stats its own select recorded. A dry compile's (`dryRun`) are its own, which `getRenderStats()` never returns, so a host can show the composition of the request a dry compile built, beside the last live compile's, without sending anything. A live compile's are the object `getRenderStats()` then returns. Strategies supply them through an optional `RenderStatsCapableStrategy.getLastSelectStats()`; a strategy without it gives compiles none. diff --git a/package-lock.json b/package-lock.json index bfe407bd..3d8f444b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.20.1", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.20.1", "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", diff --git a/package.json b/package.json index 011bbc36..fe90bb8d 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@animalabs/context-manager", - "version": "0.20.0", + "version": "0.20.1", "description": "Context management layer for LLM-based agents using Chronicle and Membrane", "type": "module", "main": "dist/src/index.js", From d6bb4c5b0e0683e06d23a2150ca2a742d5df75b8 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 16:05:01 -0400 Subject: [PATCH 40/67] fix(pins): name kv-stable's summary hold over a lock; tests for both sides of the lock rule, the bounds the exporter reads, a refused reading and the error path's transition status Docs (the README, markDocument in the strategy and in the manager, and the changelog): a mark's whole reading leaves out a part locked with lockChunk that the mark's ids don't name, and under kv-stable a pin with a level holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers takes the pin's level, as one sharing a summary with the part its ids name did before. Changing kv-stable's hold would change it for pins read by their ids too. The narrowing loop's comment: the order is taken once, before the first narrowing; where a pin with a level overlaps a raw pin's ids, the level can take the raw pin's place once a pin is narrowed (resolvedPinBounds), so later candidates' counts can be off there. Tests: - parts locked at a fold before the part a mark names keep their lock (adaptive and kv-unified), beside the existing case of parts after it; - a level pin's bounds leave out a locked part that its ids don't name, read as the replay exporter reads them, since the selects read bounds only for positions pinnedPositions keeps; - a reading other than 'ids' or 'whole' is refused, as from a JavaScript caller that leaves it out; - an error from a narrowed select leaves the refused select's transition status: the test's hook now writes one before it fails, as a narrowed select's plan can. The test is renamed from "frontier and preview", since nothing reads the preview after such an error: previewContext rethrows it and clears its preview. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- src/context-manager.ts | 9 ++-- src/strategies/autobiographical.ts | 21 +++++--- test/whole-body-pins.test.ts | 72 +++++++++++++++++++++++++++- 5 files changed, 92 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 2cc1f260..f97f4494 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk`, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk` that the mark's ids don't name, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. Under `kv-stable`, a mark with a level holds whole each summary it reaches, so a locked part that shares a summary with a part the mark covers takes the mark's level, as one sharing a summary with the part its ids name did before. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 32603990..b40565f3 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ -- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` keeps its lock: the whole reading leaves it out. +- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` that the pin's ids don't name keeps its lock: the whole reading leaves it out. Under `kv-stable`, a pin with a level holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers takes the pin's level, as one sharing a summary with the part its ids name did before. - **When the whole messages don't fit.** A compile refused over budget narrows pins that render the message raw back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch, a fork or `setStrategy`), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/src/context-manager.ts b/src/context-manager.ts index e8efb3c9..433ecc38 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1683,9 +1683,12 @@ export class ContextManager { * or `maxLevel` the message is kept raw, while * `markDocument(id, { maxLevel })` lets it fold no deeper than that level * where the selector reads level bounds. Under adaptive resolution, a mark - * on any part of a message stored in parts renders the whole message, as - * `pinRange` does: where the whole message doesn't fit, a mark that renders - * it raw is narrowed to the part it names. + * on any part of a message stored in parts renders the whole message, less + * a part locked with `lockChunk` that it doesn't name, as `pinRange` does: + * where the whole message doesn't fit, a mark that renders it raw is + * narrowed to the part it names. Under `kv-stable`, a mark with a level + * holds whole each summary it reaches, so a locked part that shares a + * summary with a part the mark covers takes the mark's level. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index c4e93ff6..2ccf735b 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -3578,9 +3578,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. * Under adaptive resolution, a mark on any part of a message stored in - * parts renders the whole message, as `pinRange` does: where the whole - * message doesn't fit, a mark that renders it raw is narrowed to the part - * it names (`selectNarrowingPins`). + * parts renders the whole message, less a part locked with `lockChunk` + * that it doesn't name (`pinPositions`), as `pinRange` does: where the + * whole message doesn't fit, a mark that renders it raw is narrowed to the + * part it names (`selectNarrowingPins`). Under `kv-stable`, a mark with a + * level holds whole each summary it reaches, so a locked part that shares a + * summary with a part the mark covers takes the mark's level. */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -8098,10 +8101,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; let entries: ContextEntry[] | undefined; try { - // Narrowing a raw pin changes no other pin's reading, and nothing it - // stops covering leaves what compression leaves out (a position there - // is raw by some pin's ids, and so by that pin's whole reading too), so - // the order taken here holds throughout. + // The order is taken once, before the first narrowing. Narrowing a raw + // pin changes no other pin's reading, and what it stops covering + // mostly stays out of compression as it was (a position there is raw + // by some pin's ids, and so by that pin's whole reading too). Where a + // pin with a level overlaps that position, the level can take the + // other pin's place once this one is narrowed (`resolvedPinBounds`), so + // later candidates' counts can be off there; the loop doesn't take + // them again. for (const { pin } of this.freedByNarrowing(store)) { this.narrowedPins.add(this.narrowKey(pin)); narrowed.push(pin); diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 650954c5..e1e3c9bf 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -62,6 +62,14 @@ class PinStore extends AutobiographicalStrategy { boundsAt(messages: unknown[]): Map { return this.resolvedPinBounds(messages as never, 'whole'); } + /** The positions pins cover in a reading, as a caller passes it, from JavaScript too. */ + pinnedWith(messages: unknown[], reading: unknown): Set { + return this.pinnedPositions(messages as never, reading as never); + } + /** The level bounds pins give in a reading, as the replay exporter reads them. */ + levelBoundsWith(messages: unknown[], reading: 'ids' | 'whole'): Map { + return this.pinLevelBounds(messages as never, reading); + } /** The ids of the messages in compressed chunks. */ summarizedIds(): Set { const out = new Set(); @@ -96,7 +104,11 @@ class PinStore extends AutobiographicalStrategy { if (err instanceof OverBudgetError) this.refusals.push(err); throw err; } - if (this.failNarrowedAfterPlanning && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) throw new Error('select failed'); + if (this.failNarrowedAfterPlanning && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) { + // As a narrowed select's plan can leave a transition status before a later failure. + (this as unknown as { transitionBlocked: string }).transitionBlocked = 'transition-pace-floor'; + throw new Error('select failed'); + } return entries; } /** @@ -218,6 +230,11 @@ const partsOf = (cm: ContextManager, id: string): string[] => { const group = all.find((m) => m.id === id)!.bodyGroupId; return all.filter((m) => m.bodyGroupId !== undefined && m.bodyGroupId === group).map((m) => m.id); }; +/** The word numbers of one stored part's own text. */ +const wordsOfPart = (cm: ContextManager, id: string): number[] => { + const part = (viewOf(cm).getAll() as unknown as Array<{ id: string; content: ContentBlock[] }>).find((m) => m.id === id)!; + return part.content.flatMap((b) => (b.type === 'text' ? [...b.text.matchAll(/\bword(\d+)\b/g)].map((m) => Number(m[1])) : [])); +}; async function withDir(body: (dir: string) => Promise): Promise { const dir = mkdtempSync(join(tmpdir(), 'cm-whole-pins-')); try { @@ -533,6 +550,56 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); } + for (const kind of ['adaptive', 'kv-unified'] as const) { + it(`${kind}: parts locked at a fold before the part a mark names keep their lock`, async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, kind); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + await cm.compile(budget(30_000)); + const parts = partsOf(cm, b); + const earlier = parts.slice(0, -1); + for (const id of earlier) strategy.lockChunk(id); + assert.ok(earlier.every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); + cm.markDocument(parts.at(-1)!); + const last = wordsOfPart(cm, parts.at(-1)!); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), last, 'only the part the mark names'); + await cm.close(); + }); + }); + } + + it('a level pin\'s bounds leave out a part locked with lockChunk that its ids don\'t name, as the replay exporter reads them', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + strategy.lockChunk(parts[3]!); + cm.pinAtLevel(b, b, 1); + const messages = viewOf(cm).getAll(); + const at = (id: string) => messages.findIndex((m) => m.id === id); + const bounds = strategy.levelBoundsWith(messages, 'whole'); + assert.deepStrictEqual(parts.filter((id) => bounds.has(at(id))), parts.filter((_, i) => i !== 3)); + assert.deepStrictEqual(bounds.get(at(parts[0]!)), { level: 1 }); + await cm.close(); + }); + }); + + it('a reading other than ids or whole is refused, as from a JavaScript caller that leaves it out', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir); + cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + const messages = viewOf(cm).getAll(); + assert.throws(() => strategy.pinnedWith(messages, undefined), /a reading is 'ids' or 'whole', not undefined/); + assert.throws(() => strategy.pinnedWith(messages, 'all'), TypeError); + assert.ok(strategy.pinnedWith(messages, 'whole').size > 1); + await cm.close(); + }); + }); + it('a mark over its own part locked at a fold renders it raw over the lock, as before', async () => { await withDir(async (dir) => { const { cm, strategy } = await open(dir); @@ -549,7 +616,7 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { }); }); - it('an error from a narrowed select leaves the refused select\'s frontier and preview', async (t) => { + it('an error from a narrowed select leaves the refused select\'s frontier and transition status', async (t) => { await withDir(async (dir) => { warnings(t); const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); @@ -560,6 +627,7 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { strategy.failNarrowedAfterPlanning = false; assert.ok(!strategy.isNarrowed(marks[0]!)); assert.strictEqual(strategy.getHotContextSettings().currentFrontierTokens, strategy.refusals[0]!.actual); + assert.strictEqual(strategy.getHotContextSettings().blocked, undefined, 'the refused select\'s transition status'); await cm.close(); }); }); From f1d69e7d85c012f55da911cec92b469b56045b68 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 16:07:05 -0400 Subject: [PATCH 41/67] docs(pins): kv-stable's summary hold is a pin with level's, not maxLevel's, at the level it holds that summary at kv-stable holds a whole summary only for a pin set with `level` (fixedPins); a pin with `maxLevel` caps its own chunks and holds no summary group, so a locked part it doesn't name keeps its lock. And the hold is at the level the pin holds that summary at, which is the pin's level or the nearest below where the chunk has a summary (renderableLevel), not always the pin's level. The README, the changelog fragment and both markDocument docs say so. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- src/context-manager.ts | 7 ++++--- src/strategies/autobiographical.ts | 7 ++++--- 4 files changed, 10 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index f97f4494..2b141e92 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk` that the mark's ids don't name, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. Under `kv-stable`, a mark with a level holds whole each summary it reaches, so a locked part that shares a summary with a part the mark covers takes the mark's level, as one sharing a summary with the part its ids name did before. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk` that the mark's ids don't name, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the mark covers renders at the level the mark holds that summary at, as one sharing a summary with the part its ids name did before. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index b40565f3..7f4e8558 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ -- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` that the pin's ids don't name keeps its lock: the whole reading leaves it out. Under `kv-stable`, a pin with a level holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers takes the pin's level, as one sharing a summary with the part its ids name did before. +- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` that the pin's ids don't name keeps its lock: the whole reading leaves it out. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - **When the whole messages don't fit.** A compile refused over budget narrows pins that render the message raw back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch, a fork or `setStrategy`), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. diff --git a/src/context-manager.ts b/src/context-manager.ts index 433ecc38..2b2931d8 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1686,9 +1686,10 @@ export class ContextManager { * on any part of a message stored in parts renders the whole message, less * a part locked with `lockChunk` that it doesn't name, as `pinRange` does: * where the whole message doesn't fit, a mark that renders it raw is - * narrowed to the part it names. Under `kv-stable`, a mark with a level - * holds whole each summary it reaches, so a locked part that shares a - * summary with a part the mark covers takes the mark's level. + * narrowed to the part it names. Under `kv-stable`, a mark with `level` + * (not `maxLevel`) holds whole each summary it reaches, so a locked part + * that shares a summary with a part the mark covers renders at the level + * the mark holds that summary at. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 2ccf735b..8c72108f 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -3581,9 +3581,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { * parts renders the whole message, less a part locked with `lockChunk` * that it doesn't name (`pinPositions`), as `pinRange` does: where the * whole message doesn't fit, a mark that renders it raw is narrowed to the - * part it names (`selectNarrowingPins`). Under `kv-stable`, a mark with a - * level holds whole each summary it reaches, so a locked part that shares a - * summary with a part the mark covers takes the mark's level. + * part it names (`selectNarrowingPins`). Under `kv-stable`, a mark with + * `level` (not `maxLevel`) holds whole each summary it reaches, so a locked + * part that shares a summary with a part the mark covers renders at the + * level the mark holds that summary at. */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); From e06edf00a750a50fad9c5314786a0b5e07a86f34 Mon Sep 17 00:00:00 2001 From: Ezra-2423 Date: Fri, 23 Oct 2026 19:59:11 -0400 Subject: [PATCH 42/67] feat(thinking-binding): leave out thinking a binding model can no longer take (#155) For a model that binds each signed thinking block to the request it was minted in, compile() takes SelectOptions.thinkingBinding and identity, and leaves out every thinking block whose request has changed since it was minted. On an account that enforces the binding, a request carrying such a block is refused with a 400; with the provider's drop control the block is set aside. - src/thinking-binding.ts: the per-branch record (a Chronicle snapshot state, /thinking-binding) of what the branch last sent: a digest of each message the last committing compile sent, the rows a host noted after them, the stored bodies the compile sent without their thinking, the compile's head digests and compile id, and the sequence where the record began. bindThinking compares a compile with it in order: a message keeps its thinking while nothing up to it differs and it lies within the record; past the record's end nothing does; a body sent without thinking stays without; only the participant's own stored replies keep thinking. With no record, every reply's thinking is left out once, unless dropBlock. - ContextManager.noteRequestSent(rows, identity, compileId): a host notes what its stream's requests carried after the compiled messages, every round so far, and at the stream's end its last reply: stored ids, digested as renderAsNewest renders them, or { unstored: true }, with the provenance.compileId of the compile the stream was built from. A note for another compile's record writes nothing. An id the store doesn't hold vouches for nothing. A note under another head leaves the record vouching for nothing until the next committing compile. - context-manager: registers the state at open; refuses a binding compile without participant or identity, or with a non-string digest, before selecting; binds the compile's messages after injections are placed and reports thinkingStrip and thinkingRecordedAfter; on a committing compile, writes the record last and then hands the strategy the stored thinking left out; a committing compile without binds clears the record. A record it can't read is logged and read as none. Provenance reads a body sent without its thinking as incomplete, and the layout prices what was sent. - AutobiographicalStrategy: noteThinkingLeftOut takes the left-out thinking's price off the calibration sample's estimate; on a binding select, carriers render and price as under carrierPolicy 'live-strip'; previewContext takes thinkingBinding for its dry select. - canonicalJson moves to src/canonical-json.ts, and stripThinkingBlocks is the shared withoutThinking. - Types and exports: ThinkingBinding, RequestIdentity, ThinkingStrip, CompileResult.thinkingStrip and thinkingRecordedAfter, ContextStrategy.noteThinkingLeftOut. README paragraph and changelog fragment. - test/thinking-binding.test.ts; the adaptive recall fixture takes select options, a committing compile before the rendered one and an inspect hook, and returns the strategy's estimate. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 + changelog.d/155-thinking-binding.added.md | 13 + src/canonical-json.ts | 14 + src/context-manager.ts | 206 +++- src/index.ts | 3 + src/strategies/autobiographical.ts | 80 +- src/thinking-binding.ts | 325 ++++++ src/types/context.ts | 51 +- src/types/index.ts | 3 + src/types/strategy.ts | 123 +- test/fixtures/recall-envelope-fixture.ts | 17 +- test/thinking-binding.test.ts | 1284 +++++++++++++++++++++ 12 files changed, 2074 insertions(+), 47 deletions(-) create mode 100644 changelog.d/155-thinking-binding.added.md create mode 100644 src/canonical-json.ts create mode 100644 src/thinking-binding.ts create mode 100644 test/thinking-binding.test.ts diff --git a/README.md b/README.md index 2b141e92..1f4d2e1e 100644 --- a/README.md +++ b/README.md @@ -90,6 +90,8 @@ With a strategy that supports them, as this one does, the manager also offers: - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. - `getRenderStats` reports how much of the last compile was raw and how much was summaries, with counts for levels 1 to 3. It's `null` until a live compile has run since the strategy loaded its branch, which opening the store, `setStrategy()` and a branch switch each do; a dry compile or preview leaves it as it was. Each compile's result carries its own as `renderStats`, so a dry compile (`dryRun`) shows the composition of the request it built without sending it. +For a model that binds each signed thinking block to the request it was minted in, pass `{ thinkingBinding: { binds: true, dropBlock, participant }, identity }` as compile's third argument, with `participant` the name the model's own replies are stored under and `identity` digests of the request's system text, tools, model and the rest of its head. The compile then leaves out the thinking such a model can no longer take, against a per-branch record of the last request the branch sent, and says what it newly left out in `thinkingStrip`. Once the branch has a record, a reply keeps its thinking at the compile after it's stored only if the host has noted it with `noteRequestSent(rows, identity, compileId)`, naming the `provenance.compileId` of the compile its stream was built from: as it sends each later round of a stream, the ids of the stored messages the request carried after the compiled ones, every round's so far, in order, and once the stream has ended, those and its last reply's, with `{ unstored: true }` for a row it can't vouch was carried or returned as stored. A host that doesn't call it loses each reply's thinking at the compile after it's stored, and is never refused for it. While one of a branch's requests is still streaming, compile for that branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing (see `SelectOptions.thinkingBinding` in `src/types/strategy.ts`). + Every option is documented on `AutobiographicalConfig` in `src/types/strategy.ts`. ## Branches diff --git a/changelog.d/155-thinking-binding.added.md b/changelog.d/155-thinking-binding.added.md new file mode 100644 index 00000000..e4cd5d59 --- /dev/null +++ b/changelog.d/155-thinking-binding.added.md @@ -0,0 +1,13 @@ +- `compile()` takes `thinkingBinding` and `identity` (`SelectOptions`) for a model that binds each signed thinking block to the request it was minted in (#155). On an account that enforces the binding, such a model refuses a request carrying a block whose request has changed since, with a 400 ("Invalid `signature` in `thinking` block. The block is bound to a different conversation…"), and with the provider's drop control on it sets the block aside instead. With `thinkingBinding.binds`, a compile leaves such blocks out. + - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. + - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message the last committing compile sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. + - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. + - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. A record it can't read is logged and read as none. + - **A dry compile** reports what it would leave out and writes no record. + - **Without `binds`,** a compile is as before, except that a committing one clears the branch's record, since what it sends no record vouches for. + - **While a request is streaming,** compile for its branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing, so its replies go without their thinking. + - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read. +- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the branch's last committing compile, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. +- `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. +- On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. diff --git a/src/canonical-json.ts b/src/canonical-json.ts new file mode 100644 index 00000000..2428d37b --- /dev/null +++ b/src/canonical-json.ts @@ -0,0 +1,14 @@ +/** + * JSON with each object's keys in one order whatever order they were set in + * (sorted, though integer-like keys come first, ascending, as in any object), + * and each array's items in their own order, so two values that differ only + * in the order their keys were set give the same text. An object the store + * keeps comes back with its keys sorted, while one built in code keeps the + * order it was built in. + */ +export function canonicalJson(value: unknown): string { + return JSON.stringify(value, (_key, inner: unknown) => { + if (inner === null || typeof inner !== 'object' || Array.isArray(inner)) return inner; + return Object.fromEntries(Object.keys(inner).sort().map((key) => [key, (inner as Record)[key]])); + }); +} diff --git a/src/context-manager.ts b/src/context-manager.ts index 2b2931d8..919f518b 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -32,6 +32,8 @@ import type { SummaryEntry, TimeRangeSummaryEntry, HotContextSettingsUpdate, + RequestIdentity, + ThinkingBinding, SelectOptions, PreviewResult, HotContextSettingsStatus, @@ -58,6 +60,11 @@ import { randomUUID } from 'node:crypto'; import { attributeEntries, buildRenderedLayout, rawBodiesOf } from './compile-provenance.js'; import { FoldJournal, branchRefOf, cachedStoreIdentity, existingStoreIdentity } from './fold-journal.js'; import { describeThrown, logThrown } from './describe-thrown.js'; +import { + UNVOUCHED, bindThinking, clearBindingRecord, continueRecord, isThinkingBlock, markThinkingLeftOut, messageDigest, readBindingRecord, + refuseUnboundBinding, refuseUnreadableIdentity, registerBindingState, withoutThinking, writeBindingRecord, + type BindOutcome, type BindingRecord, +} from './thinking-binding.js'; import { type Diagnostics, namespaceDiagnostics } from './diagnostics.js'; import type { FoldQuery, FoldQueryResult, FoldReceipt, Presentation, ReceiptSource, RoundUsage } from './fold-journal.js'; import type { @@ -430,6 +437,7 @@ export class ContextManager { // so strategies always have something to scope state IDs by, even when // the caller didn't supply a namespace. const strategyNamespace = config.namespace ?? 'default'; + registerBindingState(store, strategyNamespace); const messageStore = new MessageStore(store, { estimator: config.tokenEstimator, namespace: messageNamespace, @@ -1146,6 +1154,10 @@ export class ContextManager { * - "afterUser": inserted after the last user message, or at the end when * the messages hold none * + * With `opts.thinkingBinding.binds`, it then leaves out the thinking a + * model that binds thinking can no longer take, injections placed (see + * `SelectOptions.thinkingBinding`). + * * May block if strategy has pending work. */ async compile( @@ -1154,6 +1166,9 @@ export class ContextManager { opts?: SelectOptions ): Promise { this.requireInitialized('compile'); + // Before anything is selected or committed: a compile refused here leaves + // nothing behind. + if (opts?.thinkingBinding?.binds === true) refuseUnboundBinding(opts); this.expireCompressionHolds(); // Don't block the agent's turn on speculative compression — let it // run in the background. The strategy renders whatever's available @@ -1234,11 +1249,39 @@ export class ContextManager { } const finish = (result: CompileResult, injected: ContentBlock[][]): CompileResult => { - result.rawBodies = rawBodiesOf(attribution); + const rawBodies = rawBodiesOf(attribution); + const compileId = randomUUID(); + const binding = opts?.thinkingBinding?.binds === true + ? this.bindThinkingIn(result, messageSources, opts, compileId) + : null; + result.rawBodies = rawBodies; if (renderStats) result.renderStats = renderStats; - const layout = this.renderedLayoutFor(viewMessages, entries, attribution.rawComplete, injected, selectionCause); + // A body sent without its thinking isn't carried whole, and the layout + // prices what was sent: thinking only where a stored body kept it. + let layoutEntries = entries; + let layoutInjected = injected; + let rawComplete = attribution.rawComplete; + if (binding) { + const without = new Set(binding.record.without); + if (without.size > 0) { + markThinkingLeftOut(messageSources, without); + // Keyed by each stored member of a body, as attribution keys it. + const complete = new Map(rawComplete); + for (const id of without) { + for (const member of rawBodies.get(id) ?? []) if (complete.has(member.id)) complete.set(member.id, false); + } + rawComplete = complete; + } + layoutEntries = entries.map((entry, e) => { + const source = attribution.sources[e]; + const kept = source?.kind === 'raw' && !source.bodies.some((body) => without.has(body.messageId)); + return kept || !entry.content.some(isThinkingBlock) ? entry : { ...entry, content: withoutThinking(entry.content) }; + }); + layoutInjected = injected.map((content) => (content.some(isThinkingBlock) ? withoutThinking(content) : content)); + } + const layout = this.renderedLayoutFor(viewMessages, layoutEntries, rawComplete, layoutInjected, selectionCause); result.provenance = { - compileId: randomUUID(), + compileId, namespace: this.strategyNamespace, branch, messages: messageSources, @@ -1248,6 +1291,27 @@ export class ContextManager { ...(layout ? { journalMark: this.foldJournal.mark() } : {}), }; if (this.debugLogContext) this.logCompiledContext(result); + // Last, once nothing of the compile can still fail: what a committing + // compile leaves behind for the next one. + if (opts?.dryRun !== true) { + if (binding) { + // The store write is the compile's last step that can fail. The + // strategy's note only adjusts its calibration, so a strategy that + // throws there is logged and the compile stands. + writeBindingRecord(this.store, this.strategyNamespace, binding.record); + if (binding.leftOut.length > 0) { + try { + this.strategy.noteThinkingLeftOut?.(binding.leftOut); + } catch (err) { + this.diag.error('[context-manager] the strategy failed to take note of thinking left out:', describeThrown(err)); + } + } + } else { + // What it sends, no record vouches for: the next binding compile + // starts a record, as a branch's first does. + clearBindingRecord(this.store, this.strategyNamespace); + } + } return result; }; @@ -1320,6 +1384,36 @@ export class ContextManager { return finish({ messages, systemInjections }, injectedContent); } + /** + * Leave out of `result.messages` the thinking the branch's record says a + * binding model can no longer take, and report it (see + * `SelectOptions.thinkingBinding`). Returns what a committing compile then + * leaves behind: the record, and the stored thinking it left out. + */ + private bindThinkingIn( + result: CompileResult, + sources: readonly CompiledMessageSources[], + opts: SelectOptions, + compileId: string, + ): BindOutcome { + const record = this.readBindingRecordFor('this compile goes on as if the branch had no record'); + const outcome = bindThinking({ + messages: result.messages, + sources, + record, + dropBlock: opts.thinkingBinding?.dropBlock === true, + // Checked at the compile's start (refuseUnboundBinding). + identity: opts.identity!, + participant: opts.thinkingBinding!.participant!, + sequence: this.store.currentSequence(), + compileId, + }); + result.messages = outcome.messages; + if (outcome.strip) result.thinkingStrip = outcome.strip; + result.thinkingRecordedAfter = outcome.record.recordedAfter; + return outcome; + } + /** * The rendered layout of one compile, or null when the strategy does not * report its layout, or names a summary it cannot describe (a layout that @@ -1556,7 +1650,8 @@ export class ContextManager { /** * What the next compile sends for each of `ids`, messages this manager has * just stored, rendered as the newest of the window (see - * `ContextStrategy.renderAsNewest`). Give each message by the id + * `ContextStrategy.renderAsNewest`), before any thinking the thinking + * binding leaves out (`SelectOptions.thinkingBinding`). Give each message by the id * `addMessage` returned. For a body stored as shards that's its first * shard's, which stands for the whole body where the compile joins its * shards; where the compile sends each shard on its own, an id gives back @@ -1586,6 +1681,102 @@ export class ContextManager { return this.strategy.renderAsNewest(this.strategyMessageView(), ids); } + /** + * Note what a stream built from this branch's last committing compile sent + * after the compiled messages (see `SelectOptions.thinkingBinding`): as it + * sends each later round, every row its request carried so far, and once + * the stream has ended, those rows and its last reply. Call it on the + * branch that compile was on, with `compileId`, that compile's + * `provenance.compileId`. Each row, in order, is the id `addMessage` + * returned for a stored message, which stands for the whole body, or + * `{ unstored: true }`. `identity` is the head the stream was sent with. + * The branch's record then holds these rows, so at the next compile they + * keep their thinking while nothing up to them differs. Past the record's + * end nothing keeps its thinking: a reply keeps its thinking at the next + * compile only once a note holds it. + * + * Note only what the host can vouch it sent, or for the last reply, got + * back: a stored row whose copy, as `renderAsNewest` renders it and the + * host's request then carries it, differs from what was sent or returned + * is `{ unstored: true }`, and so is a row the store doesn't hold that the + * next request won't carry where this one did. A request-only row the + * host puts back in the same place at every later request needs no row, + * since no compile holds it. A note sent with another head than the + * record's leaves the record vouching for nothing until the next + * committing compile: the stream's earlier thinking was minted under the + * old head. A host that notes nothing keeps no reply's thinking past the + * compile after it was stored, and is never refused for it. + * + * Each stored row is digested as `renderAsNewest` renders it, so one it + * renders as null, or that the next compile renders otherwise, matches + * nothing there, and what follows goes without its thinking; a note that + * renders rows this way releases compression holds that have timed out, as + * `renderAsNewest` does. + * Without `renderAsNewest`, a row is its stored content, and a body stored + * as shards matches nothing, and so does an id the store doesn't hold, as + * after a removal. + * + * Returns false and writes nothing when `compileId` isn't the compile that + * wrote the branch's record, as for a stream whose note arrives after + * another committing compile: its replies were minted under a prefix the + * record no longer holds, and go without their thinking. So it does when + * the branch has no record, as after a committing compile without + * `thinkingBinding.binds`, or one this version can't read, which it logs. + * A dry compile writes no record, so it leaves a stream's notes as they + * were. Throws, writing nothing, for a row that isn't an id or the marker, + * a `compileId` that isn't a string, or a digest that isn't a string. + */ + noteRequestSent(rows: ReadonlyArray, identity: RequestIdentity, compileId: string): boolean { + this.requireInitialized('noteRequestSent'); + if (!Array.isArray(rows)) throw new TypeError(`noteRequestSent: \`rows\` must be an array, got ${typeof rows}`); + rows.forEach((row: unknown, i) => { + if (typeof row === 'string') return; + if (typeof row === 'object' && row !== null && (row as { unstored?: unknown }).unstored === true) return; + throw new TypeError(`noteRequestSent: row ${i} must be a message id or { unstored: true }, got ${JSON.stringify(row) ?? typeof row}`); + }); + refuseUnreadableIdentity(identity); + if (typeof compileId !== 'string') { + throw new TypeError( + `noteRequestSent: \`compileId\` must be a string, the provenance.compileId of the compile the stream was built from; got ${typeof compileId}`, + ); + } + const record = this.readBindingRecordFor('this note writes nothing, and the branch\'s next committing compile replaces it'); + // A note is for the stream of the compile that wrote the record; after + // another committing compile its replies were minted under a prefix the + // record no longer holds. + if (record === null || record.compileId !== compileId) return false; + const view = this.mergedMessageView(); + // An id the store doesn't hold, as after a removal, is nothing the next + // compile sends: it vouches for nothing, as the marker does. + const ids = rows.filter((row): row is MessageId => typeof row === 'string' && view.get(row) !== null); + const rendered = ids.length > 0 ? this.renderAsNewest(ids) : []; + let next = 0; + const continued = rows.map((row) => { + if (typeof row !== 'string') return UNVOUCHED; + const stored = view.get(row); + if (stored === null) return UNVOUCHED; + // With no renderAsNewest, a row is its stored content, which for a body + // stored as shards is only its first shard's. + const content = rendered === undefined ? (stored.bodyGroupId === undefined ? stored.content : null) : rendered[next]; + next++; + return content === null || content === undefined ? UNVOUCHED : messageDigest({ participant: stored.participant, content }); + }); + writeBindingRecord(this.store, this.strategyNamespace, continueRecord(record, continued, identity)); + return true; + } + + /** The branch's binding record; one this version can't read is logged, with what follows from it, and read as none. */ + private readBindingRecordFor(consequence: string): BindingRecord | null { + const { record, malformed } = readBindingRecord(this.store, this.strategyNamespace); + if (malformed) { + this.diag.warn( + `[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} isn't ` + + `one this version reads; ${consequence}`, + ); + } + return record; + } + /** Read the active strategy's allowlisted live settings, if supported. */ getHotContextSettings(): HotContextSettingsStatus | null { this.requireInitialized('getHotContextSettings'); @@ -1606,11 +1797,14 @@ export class ContextManager { * Non-committing preview of the layout at a hypothetical budget and/or * settings. Commits nothing — see `AutobiographicalStrategy.previewContext`. * Returns null when the active strategy has no fold plan to preview. + * `opts.thinkingBinding` previews a compile with that option: a binding + * one prices a summary's carriers as such a compile renders them (see + * `SelectOptions.thinkingBinding`); without it, as configured. */ previewContext( budget: TokenBudget, overrides?: Record, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ): PreviewResult | null { this.requireInitialized('previewContext'); const s = this.strategy as unknown as { @@ -1619,7 +1813,7 @@ export class ContextManager { log: ReturnType, budget: TokenBudget, overrides?: Record, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ) => PreviewResult; }; if (typeof s.previewContext !== 'function') return null; diff --git a/src/index.ts b/src/index.ts index af88b229..064837d5 100644 --- a/src/index.ts +++ b/src/index.ts @@ -80,6 +80,7 @@ export type { JournalMark, RenderedLayout, LayoutUnit, + ThinkingStrip, // Strategy types MessageStoreView, ContextLogView, @@ -109,6 +110,8 @@ export type { CompressionHoldOptions, CompressionHoldInfo, RenderedSummaryInfo, + ThinkingBinding, + RequestIdentity, } from './types/index.js'; // Accepted rounds and fold receipts diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 125dec76..ab7e2559 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -34,6 +34,7 @@ import type { SelectOptions, PreviewResult, RenderedSummaryInfo, + ThinkingBinding, } from '../types/index.js'; import { DEFAULT_AUTOBIOGRAPHICAL_CONFIG } from '../types/index.js'; import { getSummaryParentId } from '../types/strategy.js'; @@ -77,12 +78,14 @@ import type { import { chunkMessage, DEFAULT_CHUNKER_OPTIONS } from '../adaptive/chunker.js'; import { observeStoreBranch } from '../branch-generation.js'; import { bodyBoundsIn, bodyEnd, bodyStart } from '../body-runs.js'; +import { canonicalJson } from '../canonical-json.js'; +import { withoutThinking } from '../thinking-binding.js'; import { releaseEditedRawForms, releaseSharedRawFormsOnMove, releaseTrailingReasoning, releaseVanishedRawForms, withoutRawForms } from '../raw-forms.js'; import { isCompressionWithdrawn } from '../compression-withdrawn.js'; import { hasImage, imagePayload, imagesIn, inlineImageBytes, isImageBlock, replaceImages, strippedImageSaving } from '../inline-images.js'; import { CAP_OMISSION_TEXT, capContent, cappedPriceExtra, cappedWhole, isCapMarker, isEmptyTextBlock } from './message-cap.js'; import type { MessageId } from '../types/message.js'; -import type { IngressChunkResult } from '../types/strategy.js'; +import type { CarrierPolicy, IngressChunkResult } from '../types/strategy.js'; import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; import { storedJson } from '../stored-json.js'; @@ -1433,21 +1436,6 @@ function sha256Json(value: unknown): string { return createHash('sha256').update(JSON.stringify(value), 'utf8').digest('hex'); } -/** - * JSON with each object's keys in one order whatever order they were set in - * (sorted, though integer-like keys come first, ascending, as in any object), - * and each array's items in their own order, so two values that differ only - * in the order their keys were set give the same text. An object the store - * keeps comes back with its keys sorted, while one built here keeps the order - * it was built in. - */ -function canonicalJson(value: unknown): string { - return JSON.stringify(value, (_key, inner: unknown) => { - if (inner === null || typeof inner !== 'object' || Array.isArray(inner)) return inner; - return Object.fromEntries(Object.keys(inner).sort().map((key) => [key, (inner as Record)[key]])); - }); -} - /** * Sources as readers iterate them: a non-empty list of ids. Every index is * read, since `every` skips an empty slot of a sparse array, which the store @@ -1648,16 +1636,10 @@ export function stripEmptyTextBlocks(content: ContentBlock[]): ContentBlock[] { * `full` replay arm). Passing raw thinking through requires keeping those * turns byte-identical — a follow-up experiment, not a blanket strip removal. * - * A stripped Responses reasoning item releases the raw item it was paired - * with, so the follower isn't replayed with an id whose reasoning is gone - * (see raw-forms.ts). + * The blocks are left out as `withoutThinking` leaves them, raw forms + * included (thinking-binding.ts). */ -function stripThinkingBlocks(content: ContentBlock[]): ContentBlock[] { - return releaseEditedRawForms( - content, - content.filter((b) => b.type !== 'thinking' && b.type !== 'redacted_thinking'), - ); -} +const stripThinkingBlocks = withoutThinking; /** * Capture the summarizer's verbatim response blocks for storage on the @@ -2404,7 +2386,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { log: ContextLogView, budget: TokenBudget, overrides?: AutobiographicalOptions, - opts?: { render?: boolean }, + opts?: { render?: boolean; thinkingBinding?: ThinkingBinding }, ): PreviewResult { this.requireLoadedBranch('previewContext'); if (this._previewInFlight) { @@ -2439,7 +2421,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } let rendered: ContextEntry[] | undefined; try { - const out = this.select(store, log, budget, { dryRun: true }); + const out = this.select(store, log, budget, { dryRun: true, thinkingBinding: opts?.thinkingBinding }); // Only retained on request: these are full rendered entries and can be // megabytes on a large store. if (opts?.render) rendered = out; @@ -7993,12 +7975,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { // the next compile's stable prefix and fold diff are measured against // the request before it. const prevCacheKeys = this._prevCacheKeys; + this.thinkingBound = opts?.thinkingBinding?.binds === true; try { return this.withToolPruning(store, () => this.selectPath(store, budget, opts)); } catch (err) { this._prevCacheKeys = prevCacheKeys; throw err; } finally { + this.thinkingBound = false; if (this._lastRenderStats !== lastRenderStats) this._lastSelectStats = this._lastRenderStats; } } @@ -8030,6 +8014,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // one that re-queued its chunk (requeuePendingChunk). const restore = this.saveDryRunState(store); this._dryRunSelect = true; + this.thinkingBound = opts?.thinkingBinding?.binds === true; try { this.rebuildChunks(store); if (_diag) this.diag.error(`[cm-cache] select: rebuildChunks ${Date.now() - _t0}ms`); @@ -8038,6 +8023,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._dryRunSelect = false; if (this._lastRenderStats !== lastRenderStats) this._lastSelectStats = this._lastRenderStats; restore(); + this.thinkingBound = false; } } @@ -14191,6 +14177,32 @@ export class AutobiographicalStrategy implements ResettableStrategy { private _lastCompileEstimate = 0; private _storeView: MessageStoreView | null = null; + /** Thinking the context manager left out after this strategy's committing + * select priced it isn't sent, so the calibration sample's estimate leaves + * out what this strategy priced it at: each block at the store's estimate. + * The select priced each raw message's thinking blocks at just that: the + * image strip and the cap that also enter a message's price never change + * a thinking block's. */ + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { + if (blocks.length === 0) return; + const tokens = this.estimateTokens([...blocks]); + if (Number.isFinite(tokens) && tokens > 0) this._lastCompileEstimate = Math.max(0, this._lastCompileEstimate - tokens); + } + + /** Whether the select in progress is for a model that binds thinking + * (`SelectOptions.thinkingBinding`): set from the select's own option for + * the run of its select path, and false outside it. A summary's carriers + * were minted in its summarizer's request, so such a model never takes + * them in the live window, and the live window renders and prices them as + * under `carrierPolicy: 'live-strip'`. */ + private thinkingBound = false; + + /** The carrier policy the live window renders and prices under: the + * configured one, or `'live-strip'` while thinking is bound. */ + private liveCarrierPolicy(): CarrierPolicy { + return this.thinkingBound || this.config.carrierPolicy === 'live-strip' ? 'live-strip' : 'full'; + } + /** * Closed-loop estimator calibration (2026-07-12). Feed the REAL input total * for a request built from the latest compile (fresh + cache_read + @@ -15845,7 +15857,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Memoized price of a summary's recall answer as the LIVE WINDOW renders * it (carriers included under `carrierPolicy: 'full'`, omitted under - * `'live-strip'`, because this price feeds the fold planner for that same + * `'live-strip'` or while thinking is bound, as liveCarrierPolicy says, + * because this price feeds the fold planner for that same * emission): `estimate` is the estimate of that content, and `cost` is * what it's priced at, the estimate floored at the provider's measurement * where that measurement counts what's sent. Keyed by the summary identity @@ -15857,7 +15870,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected recallAnswerCost(s: SummaryEntry): { estimate: number; cost: number } { const recallEnvelope = this.config.recallEnvelope === 'xml' ? 'xml' : 'none'; - const carrierPolicy = this.config.carrierPolicy === 'live-strip' ? 'live-strip' : 'full'; + const carrierPolicy = this.liveCarrierPolicy(); const calibration = this._storeView?.getTokenCalibration?.() ?? 1; const key = JSON.stringify([s.id, s.tokens, recallEnvelope, carrierPolicy, calibration]); const cached = this._answerCostCache.get(key); @@ -19282,7 +19295,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * The answer body for the LIVE WINDOW — the surface the agent whose memory - * this is reads back — under `carrierPolicy`. + * this is reads back — under `carrierPolicy`, or under `'live-strip'` + * while thinking is bound (liveCarrierPolicy). * * Under `'full'` (default) this IS `summaryAnswerProse`, block for block. * Under `'live-strip'` the captured reasoning carriers are omitted from the @@ -19291,8 +19305,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * here would break the signature that the mint surface depends on — hence * omission, never rewriting. Every surviving block is passed through by * reference, except a Responses item a stripped reasoning item led to: every - * block that shares it is copied without its raw item, and so without its id - * (see raw-forms.ts). + * block that shares it, other than an opaque carrier, is copied without its + * raw item, and so without its id (see raw-forms.ts). * * A carrier-only entry (reasoning captured, no prose block — degenerate, and * guarded against at mint time) would strip to an empty assistant turn, @@ -19302,7 +19316,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { */ protected liveWindowAnswerProse(summary: SummaryEntry): ContentBlock[] { const prose = this.summaryAnswerProse(summary); - if (this.config.carrierPolicy !== 'live-strip') return prose; + if (this.liveCarrierPolicy() !== 'live-strip') return prose; const stripped = stripThinkingBlocks(prose); const carriesProse = stripped.some( (block) => block.type === 'text' && block.text.trim().length > 0, diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts new file mode 100644 index 00000000..d9b09b17 --- /dev/null +++ b/src/thinking-binding.ts @@ -0,0 +1,325 @@ +/** + * Thinking binding (#155): which signed thinking blocks a compile can send to + * a model that binds each block to the request it was minted in. + * + * Such a provider refuses a request that carries a thinking block, or with + * its drop control sets the block aside, once anything sent before the block + * differs from the request that minted it: a fold, an edit, a moved + * injection, a changed tool or system text. A compile can't know what minted + * a block, but it can know what its branch last sent. So the manager keeps, + * per branch, a record of what the branch last sent: the last committing + * compile's messages as sent, and the rows a host noted after them + * (`ContextManager.noteRequestSent`): what its stream's requests went on to + * carry and, once the stream ended, its last reply. A compile keeps a + * message's thinking only while nothing up to it differs from the record, + * and past the record's end nothing keeps its thinking: only the host knows + * which stored reply answered which request. On a branch with no record, a + * reply keeps its thinking only with the drop control, for the provider to + * judge. + * + * The record is a Chronicle snapshot state in the manager's namespace, so it + * is per branch, survives a restart, and a branch made from another starts + * from the parent's record as of the branch point. + */ + +import { createHash } from 'node:crypto'; +import type { JsStore } from '@animalabs/chronicle'; +import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; +import { canonicalJson } from './canonical-json.js'; +import { releaseEditedRawForms } from './raw-forms.js'; +import { registerStateIfAbsent } from './state-registration.js'; +import type { CompiledMessageSources, RawBodySource, ThinkingStrip } from './types/context.js'; +import type { MessageId, Sequence } from './types/message.js'; +import type { RequestIdentity, SelectOptions } from './types/strategy.js'; + +export function isThinkingBlock(block: ContentBlock): boolean { + return block.type === 'thinking' || (block.type as string) === 'redacted_thinking'; +} + +/** + * The content without its thinking blocks, each left out whole, with every + * raw form a left-out block carried released from the blocks that kept it, + * as `releaseEditedRawForms` releases them (see raw-forms.ts). + */ +export function withoutThinking(content: ContentBlock[]): ContentBlock[] { + return releaseEditedRawForms(content, content.filter((block) => !isThinkingBlock(block))); +} + +/** What a branch last sent, as its record keeps it. */ +export interface BindingRecord { + /** Each message the last committing compile sent, in order: a digest of + * its participant and content (`messageDigest`). `[UNVOUCHED]` once a + * note came with another head than `identity`: then the record vouches + * for nothing until the next committing compile. */ + sent: string[]; + /** The rows a host noted after those messages, in order: each a stored + * body's digest as the strategy renders it among the newest, or + * `UNVOUCHED` for a row the next compile can't reproduce. Empty until a + * note. */ + continued: string[]; + /** The stored bodies the last committing compile sent without their + * thinking. */ + without: MessageId[]; + /** The head digests the last committing compile sent. */ + identity: RequestIdentity; + /** The branch's sequence (`JsStore.currentSequence`) when the record + * began: a message at or below it was stored before. */ + recordedAfter: Sequence; + /** The `provenance.compileId` of the committing compile that wrote it: a + * note is for that compile's stream only. */ + compileId: string; +} + +/** A row of a noted request that matches nothing a compile sends. */ +export const UNVOUCHED = ''; + +export const IDENTITY_KEYS = ['system', 'tools', 'model', 'request'] as const; + +/** + * Refuse a binding compile that can't vouch for the thinking it would keep: + * one without `thinkingBinding.participant`, which says whose replies are the + * model's own, or without `identity`, or with a digest that isn't a string. + */ +export function refuseUnboundBinding(opts: SelectOptions): void { + const participant: unknown = opts.thinkingBinding?.participant; + if (typeof participant !== 'string' || participant.length === 0) { + throw new TypeError( + '`thinkingBinding.binds` needs `thinkingBinding.participant`: the participant whose messages are ' + + "the model's own replies, such as the agent's name", + ); + } + if (typeof opts.identity !== 'object' || opts.identity === null) { + throw new TypeError("`thinkingBinding.binds` needs `identity`: the digests of what the request carries before its messages"); + } + refuseUnreadableIdentity(opts.identity); +} + +/** Refuse head digests that aren't all strings: written to a record, they + * would leave it unreadable. */ +export function refuseUnreadableIdentity(identity: unknown): void { + if (typeof identity !== 'object' || identity === null) { + throw new TypeError(`\`identity\` must be an object of four digests, got ${identity === null ? 'null' : typeof identity}`); + } + for (const key of IDENTITY_KEYS) { + const value: unknown = (identity as Record)[key]; + if (typeof value !== 'string') { + throw new TypeError(`\`identity.${key}\` must be a string, got ${value === null ? 'null' : typeof value}`); + } + } +} + +export function bindingStateId(namespace: string): string { + return `${namespace}/thinking-binding`; +} + +export function registerBindingState(store: JsStore, namespace: string): void { + registerStateIfAbsent(store, { id: bindingStateId(namespace), strategy: 'snapshot' }); +} + +/** + * The branch's record, or null when it has none. A value that isn't a + * record's shape is reported as malformed and read as no record. + */ +export function readBindingRecord(store: JsStore, namespace: string): { record: BindingRecord | null; malformed: boolean } { + const value = store.getStateJson(bindingStateId(namespace)) as unknown; + if (value === null || value === undefined) return { record: null, malformed: false }; + return isBindingRecord(value) ? { record: value, malformed: false } : { record: null, malformed: true }; +} + +export function writeBindingRecord(store: JsStore, namespace: string, record: BindingRecord | null): void { + store.setStateJson(bindingStateId(namespace), record); +} + +/** Clear the branch's record, whatever it holds, malformed included. A + * branch with none is left unwritten. */ +export function clearBindingRecord(store: JsStore, namespace: string): void { + const value = store.getStateJson(bindingStateId(namespace)) as unknown; + if (value !== null && value !== undefined) store.setStateJson(bindingStateId(namespace), null); +} + +function isStringArray(value: unknown): value is string[] { + return Array.isArray(value) && value.every((item) => typeof item === 'string'); +} + +function isSequence(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value); +} + +function isBindingRecord(value: unknown): value is BindingRecord { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const v = value as Record; + if (!isStringArray(v.sent) || !isStringArray(v.continued) || !isStringArray(v.without)) return false; + if (!isSequence(v.recordedAfter) || typeof v.compileId !== 'string') return false; + if (typeof v.identity !== 'object' || v.identity === null || Array.isArray(v.identity)) return false; + const identity = v.identity as Record; + return IDENTITY_KEYS.every((key) => typeof identity[key] === 'string'); +} + +/** + * A compiled message's digest: its participant and content, keys in canonical + * order, so a block read back from the store (keys sorted) and the same block + * as built compare equal, as the provider reads them. A cache breakpoint isn't + * part of it, nor a block's own cache marker (`cache_control`, + * `cacheControl`): the binding doesn't count them. Anthropic's docs don't + * say so; in a run on Anthropic's API on 2026-10-23 (Haiku 5.5 with the + * drop control, on an account that enforces the binding and on one that + * doesn't), moving only a cache marker, onto or off a block before two + * replies, dropped neither reply's thinking, where changing one word + * dropped both. Other binding models weren't run for markers. 66 bits of + * SHA-256 are kept: a digest is only ever compared with the one recorded at + * the same place. + */ +export function messageDigest(message: Pick): string { + return createHash('sha256') + .update(canonicalJson({ p: message.participant, c: message.content.map(withoutCacheMarker) })) + .digest('base64url') + .slice(0, 11); +} + +function withoutCacheMarker(block: ContentBlock): unknown { + const marked = block as unknown as Record; + if (!('cache_control' in marked) && !('cacheControl' in marked)) return block; + const { cache_control: _snake, cacheControl: _camel, ...rest } = marked; + return rest; +} + +export interface BindInput { + /** The compiled messages, injections placed. */ + messages: readonly NormalizedMessage[]; + /** Parallel to `messages`. */ + sources: readonly CompiledMessageSources[]; + record: BindingRecord | null; + dropBlock: boolean; + identity: RequestIdentity; + /** The participant whose messages are the model's own replies. */ + participant: string; + /** The branch's sequence now (`JsStore.currentSequence`). */ + sequence: Sequence; + /** This compile's `provenance.compileId`. */ + compileId: string; +} + +export interface BindOutcome { + /** The messages to send. */ + messages: NormalizedMessage[]; + /** Set when a reply's thinking is newly left out. */ + strip?: ThinkingStrip; + /** What a committing compile records. Its `without` is every stored body + * whose thinking this compile leaves out. */ + record: BindingRecord; + /** The thinking blocks left out of messages with a stored body, which a + * strategy's select priced; an injection's never were. */ + leftOut: ContentBlock[]; +} + +/** + * Mark each raw body in `without` as not carried whole: a compile left its + * thinking out, a block removed (`RawBodySource.missing` gains 'content'). + */ +export function markThinkingLeftOut(sources: readonly CompiledMessageSources[], without: ReadonlySet): void { + for (const source of sources) { + if (source.kind !== 'raw') continue; + for (const body of source.bodies) { + if (!without.has(body.messageId)) continue; + body.complete = false; + body.missing = [...new Set<'shards' | 'content'>([...(body.missing ?? []), 'content'])]; + } + } +} + +function rawBodies(source: CompiledMessageSources | undefined): RawBodySource[] { + return source?.kind === 'raw' ? source.bodies : []; +} + +/** + * Decide which compiled messages keep their thinking (see + * `SelectOptions.thinkingBinding`), and the record that compile leaves. + * + * A message keeps its thinking only when it is the participant's own, from a + * stored body; none of its bodies was sent without thinking by the record's + * compile; it was stored after the record began, unless the request carries + * the drop control; and, where there is a record, nothing up to it differs + * from the record, which it is within. With no record, only the drop control + * keeps a reply's thinking. Any other message's thinking (another + * participant's, a summary's recall, an injection) was never minted in this + * agent's requests, and is always left out. + */ +export function bindThinking(input: BindInput): BindOutcome { + const { record, identity, participant } = input; + const recordedAfter = record?.recordedAfter ?? input.sequence; + const recorded = record ? [...record.sent, ...record.continued] : []; + const changed = record ? IDENTITY_KEYS.filter((key) => identity[key] !== record.identity[key]) : []; + let diverged = changed.length > 0; + const wasWithout = new Set(record?.without ?? []); + const messages: NormalizedMessage[] = []; + const sent: string[] = []; + const without = new Set(); + const newly = new Set(); + let newlyBeforeRecord = false; + const leftOut: ContentBlock[] = []; + const stripped = (message: NormalizedMessage): NormalizedMessage => ({ ...message, content: withoutThinking(message.content) }); + + for (let i = 0; i < input.messages.length; i++) { + const message = input.messages[i]!; + const bodies = rawBodies(input.sources[i]); + const thinking = message.content.some(isThinkingBlock); + // The participant's own reply, from stored bodies, not left out before. + const owned = thinking && bodies.length > 0 && message.participant === participant && + !bodies.some((body) => wasWithout.has(body.messageId)); + // Stored before the record began, and the provider won't set it aside. + const beforeRecord = !input.dropBlock && bodies.some((body) => body.sequence <= recordedAfter); + let keep = owned && !beforeRecord; + if (owned && beforeRecord) { + for (const body of bodies) newly.add(body.messageId); + newlyBeforeRecord = true; + } + let out = thinking && !keep ? stripped(message) : message; + let digest = messageDigest(out); + // Past the record's end, nothing was vouched for. + if (!diverged && record !== null) diverged = i >= recorded.length || digest !== recorded[i]; + if (diverged && keep) { + keep = false; + out = stripped(message); + digest = messageDigest(out); + for (const body of bodies) newly.add(body.messageId); + } + if (thinking && !keep && bodies.length > 0) { + for (const block of message.content) if (isThinkingBlock(block)) leftOut.push(block); + for (const body of bodies) without.add(body.messageId); + } + messages.push(out); + sent.push(digest); + } + + const cause: ThinkingStrip['cause'] = changed.length > 0 ? 'head' : newlyBeforeRecord ? 'first' : 'messages'; + return { + messages, + ...(newly.size > 0 ? { strip: { newly: [...newly], cause, ...(cause === 'head' ? { changed } : {}) } } : {}), + record: { + sent, + continued: [], + without: [...without], + identity: copyIdentity(identity), + recordedAfter, + compileId: input.compileId, + }, + leftOut, + }; +} + +function copyIdentity(identity: RequestIdentity): RequestIdentity { + return { system: identity.system, tools: identity.tools, model: identity.model, request: identity.request }; +} + +/** + * The record once a host has noted the rows it sent after the last + * committing compile's messages (`continued`, each a digest or `UNVOUCHED`), + * with the head `identity`. A note with another head than the record's + * carried thinking minted under the old one: the record keeps its head and + * vouches for nothing until the next committing compile, so the next + * compile under the new head reports the head change. + */ +export function continueRecord(record: BindingRecord, continued: string[], identity: RequestIdentity): BindingRecord { + const sameHead = IDENTITY_KEYS.every((key) => identity[key] === record.identity[key]); + return sameHead ? { ...record, continued } : { ...record, sent: [UNVOUCHED], continued: [] }; +} diff --git a/src/types/context.ts b/src/types/context.ts index e18a5792..38718cc3 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -1,6 +1,6 @@ import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; -import type { MessageId, StoredContentBlock, StoredMessage } from './message.js'; -import type { RenderStats } from './strategy.js'; +import type { MessageId, Sequence, StoredContentBlock, StoredMessage } from './message.js'; +import type { RenderStats, RequestIdentity } from './strategy.js'; /** * Describes how a context entry relates to its source message. @@ -188,6 +188,53 @@ export interface CompileResult { * sending it. Absent when the strategy records none. */ renderStats?: RenderStats; + + /** + * Set on a compile with `thinkingBinding.binds` that newly leaves out a + * stored reply's thinking: what, and why (see + * `SelectOptions.thinkingBinding`). + */ + thinkingStrip?: ThinkingStrip; + + /** + * On every compile with `thinkingBinding.binds`: the branch's sequence + * (`JsStore.currentSequence`) when its binding record began, at or above + * every message stored before then. A branch that inherits its parent's + * record keeps its value, which is at most the branch point; a dry compile + * with no record reports what the first record would hold. A reply at or + * below it was stored before the context manager recorded what the branch + * sent, so the provider may refuse or set aside its thinking without any + * strip having missed. Sequences are branch-global across slots, so a + * stored message's `sequence` compares with it directly. + */ + thinkingRecordedAfter?: Sequence; +} + +/** What a compile with `thinkingBinding.binds` newly leaves out. */ +export interface ThinkingStrip { + /** + * The stored bodies (`RawBodySource.messageId`) of the participant's + * replies whose thinking this compile leaves out and the branch's last + * committing compile didn't, whether or not its request went out: that + * compile sent them with their thinking, they were stored since, or it + * didn't send them as stored replies at all. + * So a reply can be reported again: one whose thinking went, that a later + * compile didn't send, and that renders again. A host that tells each + * loss once keeps its own set of what it has told. + */ + newly: MessageId[]; + /** + * `'head'`: an `identity` digest changed, and every message goes without + * its thinking. `'first'`: a reply stored before the branch's record + * began, which only the drop control lets keep its thinking, goes without + * it, as every reply does at a branch's first compile without the control. + * `'messages'`: a compiled message differs from the record, and every + * message from it on goes without. When a compile has more than one, + * `'head'` is given before `'first'`, and `'first'` before `'messages'`. + */ + cause: 'messages' | 'head' | 'first'; + /** With `'head'`: which digests changed. */ + changed?: Array; } /** A branch as a compile or receipt saw it: native id, name, creation time. */ diff --git a/src/types/index.ts b/src/types/index.ts index 42b8a36f..05aa374a 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -40,6 +40,7 @@ export type { JournalMark, RenderedLayout, LayoutUnit, + ThinkingStrip, } from './context.js'; // Strategy types @@ -82,6 +83,8 @@ export type { CalibrationReset, RenderStatsCapableStrategy, RenderedSummaryInfo, + ThinkingBinding, + RequestIdentity, } from './strategy.js'; export { diff --git a/src/types/strategy.ts b/src/types/strategy.ts index a3d6b1f2..9dd103bb 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -160,15 +160,16 @@ export interface ContextStrategy { readonly maxMessageTokens?: number; /** - * What the next compile sends for each of `ids`, messages just stored and - * so among the newest of the window, in the order given. Each comes back + * What the next compile renders for each of `ids`, messages just stored + * and so among the newest of the window, in the order given, before the + * manager leaves out any thinking under `SelectOptions.thinkingBinding`. Each comes back * cut by the per-message cap, and as one message where the compile joins a * body's shards into one; the strategy makes every image decision it makes * there: its count, depth and byte limits, counted newest-first over the * view from the oldest of these messages to its newest, as a compile * counts them. It's for a caller that appends messages it has just stored * to a compiled request without recompiling, such as a turn's mid-turn - * injections, so that what it appends is what the next compile sends. A + * injections, so that what it appends is what the next compile renders. A * later compile can still strip an image of theirs as newer ones arrive. * * `store` is the view a compile reads. An id comes back as null when the @@ -311,6 +312,49 @@ export interface ContextStrategy { * (for example 'budget-fit' or 'window'). Reading clears it. */ takeSelectionCause?(): string | undefined; + + /** + * The context manager left these thinking blocks of stored messages out of + * the request the last committing select priced, after the select (see + * `SelectOptions.thinkingBinding`). A strategy that calibrates its + * estimates against the provider's count of that request takes their + * price off what it expects the count to be. Not called for a dry compile. + */ + noteThinkingLeftOut?(blocks: readonly ContentBlock[]): void; +} + +/** + * Whether the model a compile is for binds each signed thinking block to the + * request it was minted in, so that a block sent after anything before it + * changed is refused, or, with the provider's drop control on, set aside. + * See `SelectOptions.thinkingBinding`. + */ +export interface ThinkingBinding { + /** The model binds thinking. Absent or false: the compile goes on without + * the option, and clears the branch's record if it commits. */ + binds?: boolean; + /** The request carries the provider's drop control, so a stale block is + * set aside rather than refused. Absent means false. */ + dropBlock?: boolean; + /** The participant whose messages are the model's own replies, such as + * the agent's name. Required with `binds`: another participant's reply + * never keeps its thinking. */ + participant?: string; +} + +/** + * Digests of what a request carries before its messages, as the host sends + * it, each opaque to the context manager and compared only for equality: + * the system text (system-position injections included), the tools in the + * order sent, the model, and the rest the provider sees ahead of the + * messages (such as the tool choice, the thinking config and the output + * limit). See `SelectOptions.identity`. + */ +export interface RequestIdentity { + system: string; + tools: string; + model: string; + request: string; } /** What a rendered summary covers, for layout reporting. */ @@ -360,6 +404,74 @@ export interface SelectOptions { * found after that rebuild. */ dryRun?: boolean; + /** + * The model this compile is for binds signed thinking (`binds`), so the + * compile leaves out every thinking block whose request has changed since + * it was minted, and reports it in `CompileResult.thinkingStrip`. With + * `binds`, `participant` and `identity` are required, and a compile + * without them is refused before anything is selected. Without `binds`, + * the compile is as it would be without the option, except that a + * committing one clears the branch's record: what it sends, no record + * vouches for. + * + * The manager keeps a record per branch of what the branch last sent: the + * last committing compile's messages as sent (cache breakpoints aside, + * injections included), then the rows a host noted after them + * (`ContextManager.noteRequestSent`): what its stream's requests went on + * to carry and, once the stream ended, its last reply. With them it keeps + * the stored bodies the compile sent without their thinking, the + * compile's `identity`, and the branch's sequence where the record began. + * A compile compares what it renders with the record in order. From the + * first message that differs, or from the first message when an + * `identity` digest changed, every message goes without its thinking, and + * so does every message past the record's end: only the host knows which + * stored reply answered which request. A reply whose stored body went + * without its thinking goes without it again, since what came after it + * was minted without it. A message whose participant isn't + * `participant`, or that has no stored body (a summary's recall answer, + * an injection), never keeps thinking: nothing it carries was minted in + * this agent's requests. A manager's record serves one participant, so + * compile each namespace for one. + * + * A reply stored at or before the sequence where the branch's record began + * (`CompileResult.thinkingRecordedAfter`), which no record vouches for, + * keeps its thinking only with `dropBlock`, for the provider to judge. So + * a branch with no record, its own or (on a branch made since) the one it + * inherits as of its branch point, leaves out every reply's thinking + * without `dropBlock`; with `dropBlock` it leaves the stored replies' + * thinking as it is. Either way it starts the record. Each committing compile writes + * the record; a dry compile reports what it would leave out and writes no + * record. The record survives a restart. + * + * So once a branch has a record, a reply keeps its thinking at the compile + * after it's stored only if the host has noted it: each later round of a + * stream as the host sends it, and the stream's last reply once it's + * stored. A host that passes `thinkingBinding` and notes nothing loses + * each reply's thinking at the compile after it's stored, and is never + * refused for it. A note names the compile its stream was built from, so + * a committing compile while a stream is open makes that stream's later + * notes vouch for nothing, and its replies go without their thinking: + * while one of a branch's requests is streaming, compile for its branch + * only with `dryRun`, which writes no record. A preview prices a + * summary's carriers by the option it's given (`previewContext`'s + * `thinkingBinding`), as a compile does. + * + * The comparison is of compile output. A host that turns compile output + * into a request (sanitizing it, adding rows) must do so the same way at + * every request, and must note a row only where the next request will + * carry it as the stream sent or got it: context-manager can't see what + * happens to a message after it leaves the compile. + */ + thinkingBinding?: ThinkingBinding; + /** + * Digests of what this compile's request carries before its messages, for + * a compile with `thinkingBinding.binds`, which requires them: when one + * differs from the branch's record, the compile leaves out every thinking + * block. `system` is the system text as sent, with the content of this + * compile's `'system'`-position injections, which the compile returns as + * `CompileResult.systemInjections`. + */ + identity?: RequestIdentity; /** Exact normalized tools/system/context-prefix identity supplied by the * host for kv-unified cache relevance. */ kvUnifiedImmutablePrefixHash?: string; @@ -1221,6 +1333,11 @@ export interface AutobiographicalConfig { * - `'live-strip'` — carriers are OMITTED from the live window only. Mint * and merge requests still recall them verbatim. * + * A select for a model that binds thinking (`SelectOptions.thinkingBinding`) + * renders and prices the live window as under `'live-strip'` whatever this + * says: a carrier was minted in its summarizer's request, so such a model + * never takes it there. Mint and merge requests are unchanged. + * * TWO SURFACES, ONE STORE. A captured carrier is the ARCHIVIST'S cognition: * the thinking of writing the memory, not the thinking of living the span it * describes. It is replayed on two surfaces that want different things from diff --git a/test/fixtures/recall-envelope-fixture.ts b/test/fixtures/recall-envelope-fixture.ts index cdc75d95..2eb632f5 100644 --- a/test/fixtures/recall-envelope-fixture.ts +++ b/test/fixtures/recall-envelope-fixture.ts @@ -25,7 +25,7 @@ import { join } from 'node:path'; import type { ContentBlock } from '@animalabs/membrane'; import { ContextManager, AutobiographicalStrategy } from '../../src/index.js'; -import type { AutobiographicalOptions, SummaryEntry } from '../../src/types/index.js'; +import type { AutobiographicalOptions, SelectOptions, SummaryEntry } from '../../src/types/index.js'; export interface RenderedMessage { participant: string; @@ -51,6 +51,9 @@ export interface FixtureRender { * Empty for fixtures that seed summaries instead of minting them. */ compressionRequests: RenderedMessage[][]; + /** The strategy's calibration estimate of the compiled request, where the + * fixture reads it (renderAdaptiveFixture). */ + estimate?: number; } /** Strategy subclass that can seed summaries without an LLM round trip. */ @@ -238,6 +241,11 @@ export async function renderAdaptiveFixture( options: AutobiographicalOptions = {}, responseContentForCall: (callCount: number) => ContentBlock[] = adaptiveTextResponseContent, maxTokens = 900, + selectOptions?: SelectOptions, + /** Options for a committing compile before the one rendered, if any. */ + firstSelectOptions?: SelectOptions, + /** Called with the manager after the rendered compile. */ + inspect?: (manager: ContextManager) => void | Promise, ): Promise { return withTempStore(async (path) => { const strategy = new AutobiographicalStrategy({ @@ -262,9 +270,12 @@ export async function renderAdaptiveFixture( // Drain compression to a fixed point: eight ticks take this workload // past L1 into merged L2s, so the adaptive render carries both levels. for (let tick = 0; tick < 8; tick++) await manager.tick(); - const compiled = await manager.compile({ maxTokens, reserveForResponse: 0 }); + if (firstSelectOptions) await manager.compile({ maxTokens, reserveForResponse: 0 }, undefined, firstSelectOptions); + const compiled = await manager.compile({ maxTokens, reserveForResponse: 0 }, undefined, selectOptions); const summaries = summariesOf(strategy); + const estimate = (strategy as unknown as { _lastCompileEstimate: number })._lastCompileEstimate; + await inspect?.(manager); manager.close(); - return { messages: project(compiled.messages), summaries, compressionRequests: mock.requests }; + return { messages: project(compiled.messages), summaries, compressionRequests: mock.requests, estimate }; }); } diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts new file mode 100644 index 00000000..7fccc48e --- /dev/null +++ b/test/thinking-binding.test.ts @@ -0,0 +1,1284 @@ +/** + * Thinking binding (#155): on a model that binds each signed thinking block + * to the request it was minted in, a compile leaves out the thinking whose + * request has changed since, against a per-branch record of what the branch + * last sent (src/thinking-binding.ts, SelectOptions.thinkingBinding). + */ + +import { describe, it, before, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { rmSync, existsSync } from 'node:fs'; +import type { ContentBlock, NormalizedMessage } from '@animalabs/membrane'; +import { AutobiographicalStrategy, ContextManager, PassthroughStrategy } from '../src/index.js'; +import type { + CompileResult, + CompiledMessageSources, + ContextEntry, + ContextInjection, + ContextLogView, + ContextStrategy, + MessageId, + MessageStoreView, + RawBodySource, + ReadinessState, + RequestIdentity, + ThinkingBinding, + ThinkingStrip, + TokenBudget, +} from '../src/index.js'; +import type { RequestIdentity as RequestIdentityFromTypes, SelectOptions, ThinkingStrip as ThinkingStripFromTypes } from '../src/types/index.js'; +import { UNVOUCHED, bindThinking, markThinkingLeftOut, messageDigest, type BindingRecord } from '../src/thinking-binding.js'; +import { renderAdaptiveFixture } from './fixtures/recall-envelope-fixture.js'; + +const STORE = './test-thinking-binding'; +const STORE_B = './test-thinking-binding-b'; + +function cleanup() { + for (const path of [STORE, STORE_B]) if (existsSync(path)) rmSync(path, { recursive: true, force: true }); +} + +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); +const thought = (n: number): ContentBlock => + ({ type: 'thinking', thinking: `zz-reasoning-${n} `.repeat(20), signature: `zz-sig-${n}-0123456789abcdef` }) as ContentBlock; +const redacted = (n: number): ContentBlock => ({ type: 'redacted_thinking', data: `zz-enc-${n}-payload==` }) as unknown as ContentBlock; +const isThinking = (b: ContentBlock) => b.type === 'thinking' || (b.type as string) === 'redacted_thinking'; +const toolUse = (id: string): ContentBlock => ({ type: 'tool_use', id, name: 'zz_search', input: { q: id } }) as ContentBlock; +const toolResult = (id: string, content: string): ContentBlock => ({ type: 'tool_result', toolUseId: id, content }) as ContentBlock; + +const identity = (over: Partial = {}): RequestIdentity => + ({ system: 'zz-system-1', tools: 'zz-tools-1', model: 'zz-model-1', request: 'zz-request-1', ...over }); +/** The options a binding compile takes, as agent-framework passes them. */ +const bind = (over: { dropBlock?: boolean; identity?: Partial; dryRun?: boolean } = {}): SelectOptions => { + const binding: ThinkingBinding = { binds: true, dropBlock: over.dropBlock === true, participant: 'Claude' }; + return { thinkingBinding: binding, identity: identity(over.identity), ...(over.dryRun ? { dryRun: true } : {}) }; +}; +const BIND = bind(); +const BIND_DROP = bind({ dropBlock: true }); +const BIG = { maxTokens: 1_000_000, reserveForResponse: 0 }; + +/** Which compiled messages carry thinking, by their text. */ +function thinkingByText(result: CompileResult): Record { + const out: Record = {}; + for (const m of result.messages) { + const label = m.content.filter((b) => b.type === 'text').map((b) => (b as { text: string }).text).join('|'); + out[label || m.content.map((b) => b.type).join('+')] = m.content.some(isThinking); + } + return out; +} + +async function open(path = STORE, strategy: ContextStrategy = new PassthroughStrategy(), namespace?: string) { + return ContextManager.open({ path, strategy, ...(namespace ? { namespace } : {}) }); +} + +/** U1, A1 (thinking), U2, A2 (thinking). */ +function seed(manager: ContextManager): { a1: MessageId; a2: MessageId } { + manager.addMessage('User', [text('U1')]); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + manager.addMessage('User', [text('U2')]); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + return { a1, a2 }; +} + +function recordOf(manager: ContextManager, namespace = 'default'): unknown { + return manager.getStore().getStateJson(`${namespace}/thinking-binding`); +} + +/** The compile id the branch's record holds, in the manager's namespace, if any. */ +function recordCompileId(manager: ContextManager): string { + const namespace = (manager as unknown as { strategyNamespace: string }).strategyNamespace; + const id = (recordOf(manager, namespace) as { compileId?: unknown } | null)?.compileId; + return typeof id === 'string' ? id : 'zz-no-record'; +} + +/** Note rows for the stream of the compile that wrote the branch's record, + * as a host does with that compile's provenance.compileId. */ +function note( + manager: ContextManager, + rows: ReadonlyArray, + identity: RequestIdentity, + compileId = recordCompileId(manager), +): boolean { + return manager.noteRequestSent(rows, identity, compileId); +} + +/** The budget the newest `n` messages take, as the passthrough strategy prices them. */ +function tokensOfNewest(manager: ContextManager, n: number): number { + const store = (manager as unknown as { messageStore: { estimateTokens(m: unknown): number } }).messageStore; + return manager.getAllMessages().slice(-n).reduce((sum, m) => sum + store.estimateTokens(m), 0); +} + +/** Passthrough that counts its selects, and can render a header ahead of + * each tool result message (`header`). */ +class Watching extends PassthroughStrategy { + selects = 0; + header = false; + select(store: MessageStoreView, log: ContextLogView, budget: TokenBudget): ContextEntry[] { + this.selects++; + return super.select(store, log, budget).map((entry) => { + const headed = this.header && entry.content.some((b) => b.type === 'tool_result'); + return headed ? { ...entry, content: [text('zz-header'), ...entry.content] } : entry; + }); + } +} + +/** U1, a committing compile, then a stream's first round: A1 calling a + * tool, and its result. */ +async function streamTurn(manager: ContextManager): Promise<{ a1: MessageId; r1: MessageId }> { + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + const r1 = manager.addMessage('User', [toolResult('t1', 'zz-result')]); + return { a1, r1 }; +} + +describe('thinking binding: compile', () => { + before(() => cleanup()); + after(() => cleanup()); + + it('without binds, a compile is unchanged and records nothing', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const plain = await manager.compile(BIG); + const off = await manager.compile(BIG, undefined, { thinkingBinding: { binds: false } }); + assert.deepEqual(off.messages, plain.messages); + assert.equal(off.thinkingStrip, undefined); + assert.equal(off.thinkingRecordedAfter, undefined); + assert.ok(plain.messages.filter((m) => m.content.some(isThinking)).length === 2, 'setup: both replies carry thinking'); + assert.equal(recordOf(manager), null); + await manager.close(); + }); + + it('with no record and no drop control, leaves out every reply\u2019s thinking once; a reply the host notes keeps its own', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const u3 = manager.addMessage('User', [text('U3')]); + const first = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(first), { U1: false, A1: false, U2: false, A2: false, U3: false }); + assert.deepEqual(first.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + assert.ok(first.thinkingRecordedAfter! >= manager.getMessage(u3)!.sequence, 'at or above every message stored before'); + + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + manager.addMessage('User', [text('U4')]); + const next = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(next), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: true, U4: false }); + assert.equal(next.thinkingStrip, undefined, 'nothing newly left out'); + assert.equal(next.thinkingRecordedAfter, first.thinkingRecordedAfter, 'where the record began stays put'); + await manager.close(); + }); + + it('with no record and the drop control, leaves the replies\u2019 thinking and starts the record', async () => { + cleanup(); + const manager = await open(); + const { a2 } = seed(manager); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(first), { U1: false, A1: true, U2: false, A2: true }); + assert.equal(first.thinkingStrip, undefined); + assert.ok(first.thinkingRecordedAfter! >= manager.getMessage(a2)!.sequence); + assert.notEqual(recordOf(manager), null); + await manager.close(); + }); + + it('a record begun under the drop control hands its earlier replies to no compile without it', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + manager.addMessage('User', [text('U3')]); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + const without = await manager.compile(BIG, undefined, BIND); + // A3 was minted after A1 and A2 with their thinking, so it goes too. + assert.deepEqual(thinkingByText(without), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: false }); + assert.deepEqual(without.thinkingStrip, { newly: [a1, a2, a3], cause: 'first' }); + const next = await manager.compile(BIG, undefined, BIND); + assert.equal(next.thinkingStrip, undefined, 'control: nothing newly left out after'); + await manager.close(); + }); + + it('a moved injection: replies after its old place lose their thinking, earlier ones keep it, and a lost one stays lost', async () => { + cleanup(); + const manager = await open(); + const { a2 } = seed(manager); + const lesson: ContextInjection = { namespace: 'zz-lessons', position: 'afterUser', content: [text('zz-lesson')] }; + const first = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(first.messages.map((m) => m.participant), ['User', 'Claude', 'User', 'system_context:zz-lessons', 'Claude']); + + manager.addMessage('User', [text('U3')]); + const moved = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(thinkingByText(moved), { U1: false, A1: true, U2: false, A2: false, U3: false, 'zz-lesson': false }); + assert.deepEqual(moved.thinkingStrip, { newly: [a2], cause: 'messages' }); + + const again = await manager.compile(BIG, [lesson], BIND_DROP); + assert.deepEqual(thinkingByText(again), thinkingByText(moved), 'A2 goes without again, and A1 keeps'); + assert.equal(again.thinkingStrip, undefined, 'A2 was left out before, so not newly'); + await manager.close(); + }); + + it('oldest messages leaving the window change the first message, so later replies lose their thinking, without the drop control too', async () => { + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + const kept = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(kept).A3, true, 'setup: A3 keeps its thinking'); + // A budget that fits only the newest two messages: the rest fall out. + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, BIND); + assert.deepEqual(thinkingByText(tight), { U3: false, A3: false }); + assert.deepEqual(tight.thinkingStrip, { newly: [a3], cause: 'messages' }); + await manager.close(); + }); + + it('a stream\u2019s replies keep their thinking when the host notes each round and the last reply, and lose it, never refused, when it doesn\u2019t', async () => { + for (const noted of ['every note', 'rounds only', 'none'] as const) { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + if (noted !== 'none') assert.equal(note(manager, [run.a1, run.r1], identity()), true); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + // The stream's end: the last reply is stored, then noted. + if (noted === 'every note') note(manager, [run.a1, run.r1, a2], identity()); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + const expected = { 'every note': [true, true], 'rounds only': [true, false], none: [false, false] }[noted]; + assert.deepEqual(thinkingByText(next), { U1: false, A1: expected[0], tool_result: false, A2: expected[1], U2: false }, noted); + const newly = [...(expected[0] ? [] : [run.a1]), ...(expected[1] ? [] : [a2])]; + assert.deepEqual(next.thinkingStrip, newly.length > 0 ? { newly, cause: 'messages' } : undefined, noted); + await manager.close(); + } + }); + + it('a write between a stream\u2019s rounds that the stream didn\u2019t send leaves out the reply after it', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + manager.addMessage('Other', [text('zz-interleaved')]); + const r1 = manager.addMessage('User', [toolResult('t1', 'zz-result')]); + note(manager, [a1, r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [a1, r1, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, 'zz-interleaved': false, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a write in an auxiliary slot between a stream\u2019s rounds leaves out the reply after it', async () => { + cleanup(); + const main = await open(); + const side = await ContextManager.open({ + store: main.getStore(), namespace: 'zz-side', isolate: true, strategy: new PassthroughStrategy(), auxiliaryMessageViews: [{}], + }); + side.addMessage('User', [text('U1')]); + await side.compile(BIG, undefined, BIND_DROP); + const a1 = side.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + main.addMessage('Other', [text('zz-auxiliary')]); + const r1 = side.addMessage('User', [toolResult('t1', 'zz-result')]); + note(side, [a1, r1], identity()); + const a2 = side.addMessage('Claude', [thought(2), text('A2')]); + note(side, [a1, r1, a2], identity()); + const next = await side.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, 'zz-auxiliary': false, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await side.close(); + await main.close(); + }); + + it('an edit to a noted row, which keeps its sequence, leaves out the reply after it', async () => { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const sequence = manager.getMessage(run.r1)!.sequence; + manager.editMessage(run.r1, [toolResult('t1', 'zz-edited')]); + assert.equal(manager.getMessage(run.r1)!.sequence, sequence, 'setup: the edit keeps the sequence'); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a noted row removed, or one the view filter leaves out, leaves out the reply after it', async () => { + for (const how of ['removed', 'filtered'] as const) { + cleanup(); + const manager = await ContextManager.open({ + path: STORE, strategy: new PassthroughStrategy(), + viewFilter: (message) => !(message.metadata as { hidden?: unknown } | undefined)?.hidden, + }); + const run = await streamTurn(manager); + const injected = manager.addMessage('User', [text('zz-injected')], how === 'filtered' ? { hidden: true } : undefined); + note(manager, [run.a1, run.r1, injected], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, injected, a2], identity()); + if (how === 'removed') manager.removeMessage(injected); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }, how); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }, how); + await manager.close(); + } + }); + + it('a row the compile renders with an addition leaves out the reply after it', async () => { + cleanup(); + const strategy = new Watching(); + const manager = await open(STORE, strategy); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const control = await manager.compile(BIG, undefined, { ...BIND_DROP, dryRun: true }); + assert.equal(thinkingByText(control).A2, true, 'control: rendered as stored, A2 keeps its thinking'); + strategy.header = true; + const decorated = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(thinkingByText(decorated).A2, false); + await manager.close(); + }); + + it('past a row noted as unstored, nothing keeps its thinking', async () => { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, { unstored: true }], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, { unstored: true }, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a2], cause: 'messages' }); + await manager.close(); + }); + + it('a noted row is digested as renderAsNewest renders it, and one it renders as null matches nothing', async () => { + for (const render of ['as-stored', 'null', 'otherwise'] as const) { + cleanup(); + class Rendering extends PassthroughStrategy { + renderAsNewest(store: MessageStoreView, ids: readonly MessageId[]): Array { + return ids.map((id) => { + const content = store.get(id)!.content; + const result = content.some((b) => b.type === 'tool_result'); + if (!result || render === 'as-stored') return content; + return render === 'null' ? null : [text('zz-other'), ...content]; + }); + } + } + const manager = await open(STORE, new Rendering()); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + note(manager, [run.a1, run.r1, a2], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(thinkingByText(next).A2, render === 'as-stored', render); + await manager.close(); + } + }); + + it('with no renderAsNewest, a noted body stored as shards matches nothing', async () => { + cleanup(); + class Sharding extends PassthroughStrategy { + chunkIngressMessage(_participant: string, content: ContentBlock[]) { + const body = content.length === 1 && content[0]!.type === 'text' ? (content[0] as { text: string }).text : ''; + if (!body.startsWith('zz-shard')) return null; + const half = Math.floor(body.length / 2); + return { + bodyGroupId: 'zz-group', + shards: [ + { content: [text(body.slice(0, half))], shardIndex: 0 }, + { content: [text(body.slice(half))], shardIndex: 1 }, + ], + }; + } + } + const manager = await open(STORE, new Sharding()); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const before = manager.getAllMessages().length; + const shard = manager.addMessage('User', [text('zz-shard ' + 'y'.repeat(100))]); + assert.equal(manager.getAllMessages().length, before + 2, 'setup: stored as two shards'); + const plain = manager.addMessage('User', [text('zz-plain')]); + note(manager, [shard, plain], identity()); + const noted = recordOf(manager) as BindingRecord; + assert.equal(noted.continued[0], UNVOUCHED); + assert.notEqual(noted.continued[1], UNVOUCHED, 'control: an unsharded row is digested'); + await manager.close(); + }); + + it('noteRequestSent writes nothing on a branch with no record, and refuses what it can\u2019t record before writing', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + assert.equal(note(manager, [a1], identity()), false, 'no record yet'); + assert.equal(note(manager, ['zz-missing'], identity()), false, 'an unknown id, with no record'); + assert.equal(recordOf(manager), null); + await manager.compile(BIG, undefined, BIND); + const before = recordOf(manager) as BindingRecord; + assert.deepEqual(before.without, [a1], 'setup: the compile left A1 out'); + const recorded = JSON.stringify(recordOf(manager)); + const refused: Array<[string, () => unknown, RegExp]> = [ + ['rows not a list', () => note(manager, 'zz' as unknown as MessageId[], identity()), /`rows` must be an array/], + ['a row of neither kind', () => note(manager, [a1, { unstored: false } as unknown as { unstored: true }], identity()), /row 1 must be a message id or \{ unstored: true \}/], + ['no identity', () => note(manager, [a1], undefined as unknown as RequestIdentity), /`identity` must be an object/], + ['a digest not a string', () => note(manager, [a1], { ...identity(), model: 3 as unknown as string }), /`identity.model` must be a string, got number/], + ['a compileId not a string', () => manager.noteRequestSent([a1], identity(), undefined as unknown as string), /`compileId` must be a string, the provenance.compileId of the compile the stream was built from; got undefined/], + ]; + for (const [label, call, message] of refused) { + assert.throws(call, (error: unknown) => error instanceof Error && message.test(error.message), label); + } + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as it was'); + // An id the store doesn't hold, as after a removal, vouches for nothing, as the marker does. + assert.equal(note(manager, [a1, 'zz-missing', { unstored: true }], identity()), true); + const noted = recordOf(manager) as BindingRecord; + assert.equal(noted.continued.length, 3); + assert.notEqual(noted.continued[0], UNVOUCHED, 'control: a stored row is digested'); + assert.equal(noted.continued[1], UNVOUCHED); + assert.equal(noted.continued[2], UNVOUCHED); + assert.deepEqual(noted.sent, before.sent, 'the compiled part stays'); + assert.deepEqual(noted.without, before.without, 'and what it left out'); + assert.equal(noted.recordedAfter, before.recordedAfter, 'and where the record began'); + // A compile starts the request over. + await manager.compile(BIG, undefined, BIND); + assert.deepEqual((recordOf(manager) as BindingRecord).continued, []); + await manager.close(); + }); + + it('a note is for the stream of the compile that wrote the record: after another committing compile it writes nothing, and a dry compile changes nothing', async () => { + // previewContext calls the strategy alone and never reaches the record. + for (const between of ['a committing compile', 'a dry compile', 'nothing'] as const) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND); + // Stored while C1's stream is open, so C1's reply was minted without it. + manager.addMessage('User', [text('zz-X')]); + if (between === 'a committing compile') await manager.compile(BIG, undefined, BIND); + if (between === 'a dry compile') await manager.compile(BIG, undefined, bind({ dryRun: true })); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + const accepted = manager.noteRequestSent([a1], identity(), c1.provenance!.compileId); + assert.equal(accepted, between !== 'a committing compile', between); + const next = await manager.compile(BIG, undefined, BIND); + // Where C1's record still stands, X sits where its note put A1, so A1 + // goes without its thinking as well; only the committing compile's + // record would have lined A1 up with the note, and it refused the note. + assert.equal(thinkingByText(next).A1, false, between); + await manager.close(); + } + // The control: C1's own stream, noted under C1's id, keeps its reply's thinking. + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND); + await manager.compile(BIG, undefined, bind({ dryRun: true })); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + assert.equal(manager.noteRequestSent([a1], identity(), c1.provenance!.compileId), true); + assert.equal(manager.noteRequestSent([a1], identity(), 'zz-another-compile'), false, 'another compile\u2019s id writes nothing'); + assert.equal(manager.noteRequestSent([a1], identity(), c1.provenance!.compileId), true, 'a later note of the same stream, under the same id'); + const next = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(next).A1, true, 'control'); + await manager.close(); + }); + + it('each note carries every round so far; a note of the latest round alone leaves out the rounds before', async () => { + for (const cumulative of [true, false]) { + cleanup(); + const manager = await open(); + const run = await streamTurn(manager); + note(manager, [run.a1, run.r1], identity()); + const a2 = manager.addMessage('Claude', [thought(2), text('A2'), toolUse('t2')]); + const r2 = manager.addMessage('User', [toolResult('t2', 'zz-result-2')]); + note(manager, cumulative ? [run.a1, run.r1, a2, r2] : [a2, r2], identity()); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, cumulative ? [run.a1, run.r1, a2, r2, a3] : [a3], identity()); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: cumulative, tool_result: false, A2: cumulative, A3: cumulative }, `cumulative: ${cumulative}`); + if (!cumulative) assert.deepEqual(next.thinkingStrip, { newly: [run.a1, a2, a3], cause: 'messages' }); + await manager.close(); + } + }); + + it('a note sent with another head leaves a record that vouches for no thinking, until the next committing compile', async () => { + for (const later of [false, true]) { + for (const next of ['new head', 'old head'] as const) { + const label = `${next}${later ? ', after a later note under the record\u2019s head' : ''}`; + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity({ tools: 'zz-tools-2' })); + // A later note under the record's own head doesn't take the mark away. + if (later) note(manager, [a3], identity()); + const head = next === 'new head' ? { tools: 'zz-tools-2' } : {}; + const after = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: head })); + assert.deepEqual(thinkingByText(after), { U1: false, A1: false, U2: false, A2: false, U3: false, A3: false }, label); + // The record kept its own head, so only a compile under the new one reports a head change. + assert.equal(after.thinkingStrip?.cause, next === 'new head' ? 'head' : 'messages', label); + if (next === 'new head') assert.deepEqual(after.thinkingStrip?.changed, ['tools'], label); + // The compile starts a record again, which vouches as usual. + const again = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: head })); + assert.equal(again.thinkingStrip, undefined, `${label}: nothing newly left out once the compile records again`); + await manager.close(); + } + } + // A later note's rows can't line up with a compile's first messages either: + // here the window keeps only the noted reply and what follows it. + cleanup(); + const manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity({ tools: 'zz-tools-2' })); + note(manager, [a3], identity()); + manager.addMessage('User', [text('U4')]); + const windowed = await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, { ...BIND_DROP, dryRun: true }); + assert.deepEqual(thinkingByText(windowed), { A3: false, U4: false }, 'the window starts at the noted reply, which goes without its thinking'); + await manager.close(); + }); + + it('a noted row removed, by this manager or another on the shared slot, mid-stream or after, leaves out the replies after it', async () => { + for (const how of ['this manager', 'another manager', 'mid-stream'] as const) { + cleanup(); + const main = await open(); + const side = await ContextManager.open({ store: main.getStore(), namespace: 'zz-side', strategy: new PassthroughStrategy() }); + side.addMessage('User', [text('U1')]); + await side.compile(BIG, undefined, BIND_DROP); + const a1 = side.addMessage('Claude', [thought(1), text('A1'), toolUse('t1')]); + const r1 = side.addMessage('User', [toolResult('t1', 'zz-result')]); + const extra = side.addMessage('User', [text('zz-extra')]); + note(side, [a1, r1, extra], identity()); + if (how === 'mid-stream') side.removeMessage(extra); + const a2 = side.addMessage('Claude', [thought(2), text('A2')]); + // The end note names the removed row by its id, which now vouches for nothing. + assert.equal(note(side, [a1, r1, extra, a2], identity()), true, how); + if (how === 'this manager') side.removeMessage(extra); + if (how === 'another manager') main.removeMessage(extra); + const next = await side.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, tool_result: false, A2: false }, how); + await side.close(); + await main.close(); + } + }); + + it('an edit to the noted last reply, ahead of its thinking or after it, leaves out its thinking', async () => { + for (const where of ['ahead', 'after'] as const) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + note(manager, [a1], identity()); + const control = await manager.compile(BIG, undefined, { ...BIND_DROP, dryRun: true }); + assert.equal(thinkingByText(control).A1, true, `control, ${where}`); + manager.editMessage(a1, where === 'ahead' ? [text('zz-edit'), thought(1), text('A1')] : [thought(1), text('A1 edited')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(next.thinkingStrip, { newly: [a1], cause: 'messages' }, where); + await manager.close(); + } + }); + + it('a single-round turn keeps its reply\u2019s thinking when the host notes it, and loses only that reply\u2019s when it doesn\u2019t, as when the host stops before its note', async () => { + for (const noted of [true, false]) { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); + note(manager, [a1], identity()); + manager.addMessage('User', [text('U2')]); + await manager.compile(BIG, undefined, BIND_DROP); + const a2 = manager.addMessage('Claude', [thought(2), text('A2')]); + if (noted) note(manager, [a2], identity()); + manager.addMessage('User', [text('U3')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, U2: false, A2: noted, U3: false }, `noted: ${noted}`); + assert.deepEqual(next.thinkingStrip, noted ? undefined : { newly: [a2], cause: 'messages' }, `noted: ${noted}`); + await manager.close(); + } + }); + + it('a strategy that throws taking note of the thinking left out is logged, and the compile stands', async () => { + cleanup(); + class Throwing extends PassthroughStrategy { + noteThinkingLeftOut(): void { throw new Error('zz-note-failed'); } + } + const manager = await open(STORE, new Throwing()); + seed(manager); + const logged: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + let result: CompileResult; + try { + result = await manager.compile(BIG, undefined, BIND); + } finally { + console.error = error; + } + assert.equal(result.thinkingStrip?.cause, 'first'); + assert.notEqual(recordOf(manager), null, 'the record is written'); + assert.ok(logged.some((line) => /failed to take note of thinking left out/.test(line) && /zz-note-failed/.test(line)), logged.join('\n')); + await manager.close(); + }); + + it('a compile without binds on a branch with no record writes none', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; + const set = store.setStateJson.bind(store); + let writes = 0; + store.setStateJson = (id: string, value: unknown) => { + if (id === 'default/thinking-binding') writes++; + set(id, value); + }; + await manager.compile(BIG); + store.setStateJson = set; + assert.equal(writes, 0); + await manager.close(); + }); + + it('a reply long enough to shard and not noted goes without its thinking, every shard read as partial', async () => { + cleanup(); + const strategy = new AutobiographicalStrategy({ adaptiveResolution: true, targetChunkTokens: 50, autoTickOnNewMessage: false, headWindowTokens: 0 }); + const manager = await ContextManager.open({ path: STORE, strategy }); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG, undefined, BIND_DROP); + const before = manager.getAllMessages().length; + const a1 = manager.addMessage('Claude', [thought(1), text('zz-long ' + 'w '.repeat(1_500))]); + const shards = manager.getAllMessages().length - before; + assert.ok(shards > 1, 'setup: the reply was stored as shards'); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + const reply = next.messages.find((m) => m.participant === 'Claude')!; + assert.ok(!reply.content.some(isThinking), 'the sharded reply goes without its thinking'); + assert.deepEqual(next.thinkingStrip, { newly: [a1], cause: 'messages' }); + const partial = next.provenance!.layout!.units.filter((unit) => (unit as { p?: number }).p === 1).length; + assert.equal(partial, shards, `every shard's unit is partial (${partial} of ${shards})`); + await manager.close(); + }); + + it('a shorter or empty record doesn\u2019t make older messages look stored since', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const empty = await manager.compile({ maxTokens: 1, reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(empty.messages.length, 0, 'setup: nothing fits'); + const back = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(back), { U1: false, A1: false, U2: false, A2: false }); + assert.deepEqual(back.thinkingStrip, { newly: [a1, a2], cause: 'messages' }); + await manager.close(); + }); + + for (const key of ['system', 'tools', 'model', 'request'] as const) { + it(`a changed ${key} digest leaves out every reply\u2019s thinking`, async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const same = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(same.thinkingStrip, undefined, 'control: the same digests change nothing'); + const changed = await manager.compile(BIG, undefined, bind({ dropBlock: true, identity: { [key]: 'zz-changed' } })); + assert.deepEqual(changed.thinkingStrip, { newly: [a1, a2], cause: 'head', changed: [key] }); + assert.deepEqual(thinkingByText(changed), { U1: false, A1: false, U2: false, A2: false }); + await manager.close(); + }); + } + + it('refuses a binding compile without participant or identity, or with a digest that isn\u2019t a string, before anything is selected or recorded', async () => { + cleanup(); + const strategy = new Watching(); + const manager = await open(STORE, strategy); + seed(manager); + await manager.compile(BIG, undefined, BIND); + const recorded = JSON.stringify(recordOf(manager)); + const selects = strategy.selects; + const refused: Array<[string, SelectOptions, RegExp]> = [ + ['no participant', { thinkingBinding: { binds: true }, identity: identity() }, /needs `thinkingBinding.participant`/], + ['an empty participant', { thinkingBinding: { binds: true, participant: '' }, identity: identity() }, /needs `thinkingBinding.participant`/], + ['no identity', { thinkingBinding: { binds: true, participant: 'Claude' } }, /needs `identity`: the digests of what the request carries before its messages/], + ...(['system', 'tools', 'model', 'request'] as const).map((key): [string, SelectOptions, RegExp] => [ + `a ${key} digest that isn't a string`, + { ...BIND, identity: { ...identity(), [key]: 7 as unknown as string } }, + new RegExp(`identity\\.${key}\` must be a string, got number`), + ]), + ]; + for (const [label, opts, message] of refused) { + await assert.rejects(manager.compile(BIG, undefined, opts), (error: unknown) => error instanceof TypeError && message.test(error.message), label); + } + assert.equal(strategy.selects, selects, 'no select ran'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as it was'); + await manager.close(); + }); + + it('a committing compile without binds clears the record, and a dry one leaves it', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + await manager.compile(BIG, undefined, { dryRun: true }); + assert.notEqual(recordOf(manager), null, 'a dry compile without binds leaves the record'); + await manager.compile(BIG); + assert.equal(recordOf(manager), null, 'a committing one clears it'); + // Back on a binding model, the replies stored meanwhile were minted under + // requests no record saw. + const back = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(back.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + // A record it can't read is cleared too. + manager.getStore().setStateJson('default/thinking-binding', { sent: 'zz-not-a-list' }); + await manager.compile(BIG); + assert.equal(recordOf(manager), null, 'a malformed record is cleared'); + await manager.close(); + }); + + it('a reply a compile didn\u2019t send is reported again when it renders again', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const first = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(first.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 1), reserveForResponse: 0 }, undefined, BIND); + assert.deepEqual(thinkingByText(tight), { A2: false }, 'setup: only A2 is sent'); + assert.equal(tight.thinkingStrip, undefined, 'A2 was left out by the last compile'); + const back = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(back.thinkingStrip, { newly: [a1], cause: 'first' }, 'A1 again, since the last compile didn\u2019t send it'); + await manager.close(); + }); + + it('another participant\u2019s reply never keeps its thinking', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + manager.addMessage('Other', [thought(8), text('O1')]); + manager.addMessage('Claude', [thought(1), text('A1')]); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(first), { U1: false, O1: false, A1: true }); + assert.equal(first.thinkingStrip, undefined, 'another participant\u2019s thinking is never newly left out'); + await manager.close(); + }); + + it('the record survives a restart', async () => { + cleanup(); + let manager = await open(); + seed(manager); + manager.addMessage('User', [text('U3')]); + await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + note(manager, [a3], identity()); + const before = await manager.compile(BIG, undefined, BIND); + assert.equal(thinkingByText(before).A3, true, 'setup: A3 keeps its thinking'); + await manager.close(); + + manager = await open(); + const after = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(thinkingByText(after), thinkingByText(before), 'with no record this would leave out A3 too'); + assert.equal(after.thinkingStrip, undefined); + assert.equal(after.thinkingRecordedAfter, before.thinkingRecordedAfter); + await manager.close(); + }); + + it('a branch starts from the record as of its branch point, and its writes stay its own', async () => { + cleanup(); + const manager = await open(); + seed(manager); + const first = await manager.compile(BIG, undefined, BIND_DROP); + const u3 = manager.addMessage('User', [text('U3')]); + manager.addMessage('Claude', [thought(3), text('A3')]); + // On main, U1 falls out of the window: A1, A2 and A3 lose their thinking. + const tight = await manager.compile({ maxTokens: tokensOfNewest(manager, 4), reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(thinkingByText(tight).A2, false, 'setup: main\u2019s latest record leaves A2 out'); + const mainRecord = JSON.stringify(recordOf(manager)); + + // A branch at U3 sees the record the first compile wrote, before U3 was + // stored, and not main's latest. + const atU3 = manager.branchAt(u3, 'zz-at-u3'); + await manager.switchBranch(atU3); + const there = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(there), { U1: false, A1: true, U2: false, A2: true, U3: false }); + assert.equal(there.thinkingStrip, undefined); + assert.equal(there.thinkingRecordedAfter, first.thinkingRecordedAfter); + assert.ok(there.thinkingRecordedAfter! <= manager.currentBranch().branchPoint!, 'at most the branch point'); + await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, BIND_DROP); + + await manager.switchBranch('main'); + assert.equal(JSON.stringify(recordOf(manager)), mainRecord, 'the branch\u2019s writes left main\u2019s record'); + + const forked = await manager.fork('zz-fork'); + assert.equal(forked, 'zz-fork'); + assert.equal(JSON.stringify(recordOf(manager)), mainRecord, 'the fork holds main\u2019s record'); + const onFork = await manager.compile({ maxTokens: tokensOfNewest(manager, 4), reserveForResponse: 0 }, undefined, BIND_DROP); + assert.equal(onFork.thinkingStrip, undefined, 'the fork starts from main\u2019s record'); + await manager.close(); + }); + + it('a dry compile reports what it would leave out and records nothing', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const dry = await manager.compile(BIG, undefined, bind({ dryRun: true })); + assert.deepEqual(dry.thinkingStrip, { newly: [a1, a2], cause: 'first' }); + assert.deepEqual(thinkingByText(dry), { U1: false, A1: false, U2: false, A2: false }); + assert.equal(recordOf(manager), null); + const real = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(real.thinkingStrip, { newly: [a1, a2], cause: 'first' }, 'the dry compile left no record behind'); + const recorded = JSON.stringify(recordOf(manager)); + await manager.compile({ maxTokens: tokensOfNewest(manager, 2), reserveForResponse: 0 }, undefined, bind({ dryRun: true })); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'a dry compile leaves a record as it was'); + await manager.close(); + }); + + it('a record it cannot read is warned of and read as none', async () => { + const good = { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }; + const malformed: Array<[string, unknown]> = [ + ['sent not a list', { ...good, sent: 'zz-not-a-list' }], + ['sent holding a number', { ...good, sent: [7] }], + ['without missing', { sent: [], continued: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }], + ['continued missing', { sent: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }], + ['compileId missing', { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity() }], + ['compileId not a string', { ...good, compileId: 7 }], + ['continued holding a number', { ...good, continued: [7] }], + ['without holding a number', { ...good, without: [7] }], + ['recordedAfter not a number', { ...good, recordedAfter: '1' }], + ['identity missing', { sent: [], continued: [], without: [], recordedAfter: 1, compileId: 'zz-compile' }], + ['identity null', { ...good, identity: null }], + ['an identity digest not a string', { ...good, identity: { ...identity(), model: 5 } }], + ['not an object', ['zz']], + ]; + for (const [label, value] of malformed) { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + manager.getStore().setStateJson('default/thinking-binding', value); + const warned: string[] = []; + const warn = console.warn; + console.warn = (message: string) => { warned.push(message); }; + let result: CompileResult; + try { + assert.equal(note(manager, [a2], identity()), false, `${label}: a note writes nothing`); + assert.deepEqual(recordOf(manager), value, `${label}: the value is as it was`); + assert.equal(warned.length, 1, label); + assert.match(warned[0]!, /thinking-binding record of branch main isn't one this version reads; this note writes nothing/, label); + warned.length = 0; + result = await manager.compile(BIG, undefined, BIND); + } finally { + console.warn = warn; + } + assert.deepEqual(result.thinkingStrip, { newly: [a1, a2], cause: 'first' }, label); + assert.equal(warned.length, 1, label); + assert.match(warned[0]!, /thinking-binding record of branch main isn't one this version reads; this compile goes on as if the branch had no record/, label); + assert.ok(Array.isArray((recordOf(manager) as { sent: unknown }).sent), `${label}: the compile wrote a record it can read`); + await manager.close(); + } + cleanup(); + const control = await open(); + seed(control); + control.getStore().setStateJson('default/thinking-binding', good); + const read = await control.compile(BIG, undefined, BIND_DROP); + assert.equal(read.thinkingRecordedAfter, 1, 'control: a record of the right shape is read'); + await control.close(); + }); + + it('a stripped body reads as incomplete, and the rendered layout prices what was sent', async () => { + cleanup(); + const manager = await open(); + const { a1, a2 } = seed(manager); + const plain = await manager.compile(BIG, undefined, { dryRun: true }); + const bound = await manager.compile(BIG, undefined, BIND); + const bodies = (result: CompileResult) => { + const out = new Map(); + for (const source of result.provenance!.messages) if (source.kind === 'raw') for (const body of source.bodies) out.set(body.messageId, body); + return out; + }; + assert.equal(bodies(plain).get(a1)!.complete, true, 'setup: carried whole without binding'); + assert.equal(bodies(bound).get(a1)!.complete, false); + assert.deepEqual(bodies(bound).get(a2)!.missing, ['content']); + assert.equal([...bodies(bound).values()].filter((body) => body.complete).length, 2, 'the user messages are still whole'); + const thoughts = (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens([thought(1), thought(2)]); + assert.ok(thoughts > 0, 'setup: the thinking has a price'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - thoughts); + const partial = (result: CompileResult) => result.provenance!.layout!.units.filter((unit) => (unit as { p?: number }).p === 1).length; + assert.equal(partial(plain), 0, 'setup: every raw unit whole without binding'); + assert.equal(partial(bound), 2, 'the two replies\u2019 units are partial'); + await manager.close(); + }); + + it('the rendered layout prices a summary\u2019s and an injection\u2019s thinking as never sent', async () => { + cleanup(); + class Summarizing extends PassthroughStrategy { + select(store: MessageStoreView, log: ContextLogView, budget: TokenBudget): ContextEntry[] { + const entries = super.select(store, log, budget); + return [...entries, { index: entries.length, sourceRelation: 'derived', participant: 'Claude', content: [thought(7), text('zz-summary')], summaries: [{ id: 'zz-s1', level: 1 }] }]; + } + describeRenderedSummaries(ids: readonly string[]) { + return new Map(ids.map((id) => [id, { level: 1, leaves: [] as MessageId[], method: 'authored' }])); + } + } + const manager = await open(STORE, new Summarizing()); + seed(manager); + const odd: ContextInjection = { namespace: 'zz-odd', position: 'afterUser', content: [thought(9), text('zz-odd')] }; + const plain = await manager.compile(BIG, [odd], { dryRun: true }); + const bound = await manager.compile(BIG, [odd], BIND_DROP); + assert.deepEqual(thinkingByText(bound), { U1: false, A1: true, U2: false, 'zz-odd': false, A2: true, 'zz-summary': false }, 'setup: the replies keep their thinking'); + const price = (blocks: ContentBlock[]) => (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens(blocks); + assert.ok(plain.provenance!.layout, 'setup: the strategy reports a layout'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([thought(7), thought(9)]), + 'the kept replies are priced as sent, the summary and the injection without theirs'); + await manager.close(); + }); + + it('a record write that fails tells the strategy nothing', async () => { + cleanup(); + const told: ContentBlock[][] = []; + class Listening extends PassthroughStrategy { + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + } + const manager = await open(STORE, new Listening()); + seed(manager); + const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; + const set = store.setStateJson.bind(store); + store.setStateJson = (id: string, value: unknown) => { + if (id === 'default/thinking-binding') throw new Error('zz-write-failed'); + set(id, value); + }; + await assert.rejects(manager.compile(BIG, undefined, BIND), /zz-write-failed/); + assert.deepEqual(told, []); + store.setStateJson = set; + await manager.close(); + }); + + it('a compile that fails after selecting writes no record and tells the strategy nothing', async () => { + cleanup(); + const told: ContentBlock[][] = []; + class Failing implements ContextStrategy { + readonly name = 'zz-failing'; + readonly renderedForms = ['raw', 'summary', 'omitted'] as const; + checkReadiness(): ReadinessState { return { ready: true }; } + select(store: MessageStoreView): ContextEntry[] { + const entries: ContextEntry[] = store.getAll().map((m, index) => ({ index, sourceMessageId: m.id, sourceRelation: 'copy', participant: m.participant, content: m.content })); + return [...entries, { index: entries.length, sourceRelation: 'derived', participant: 'Claude', content: [text('zz-summary')], summaries: [{ id: 'zz-s1', level: 1 }] }]; + } + describeRenderedSummaries(): ReadonlyMap { throw new Error('zz-layout-failed'); } + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + } + const manager = await open(STORE, new Failing()); + seed(manager); + await assert.rejects(manager.compile(BIG, undefined, BIND), /zz-layout-failed/); + assert.equal(recordOf(manager), null); + assert.deepEqual(told, []); + // Nor does one without binds clear a record when it fails. + const planted = { sent: [], continued: [], without: [], recordedAfter: 1, identity: identity(), compileId: 'zz-compile' }; + manager.getStore().setStateJson('default/thinking-binding', planted); + await assert.rejects(manager.compile(BIG), /zz-layout-failed/); + assert.deepEqual(recordOf(manager), planted); + await manager.close(); + }); + + it('the strategy is told the stored thinking left out, at each committing compile and only then', async () => { + cleanup(); + const told: ContentBlock[][] = []; + class Listening extends PassthroughStrategy { + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { + told.push([...blocks]); + } + } + const manager = await open(STORE, new Listening()); + seed(manager); + manager.addMessage('User', [text('U3')]); + const r3 = manager.addMessage('Claude', [redacted(3)]); + const odd: ContextInjection = { namespace: 'zz-odd', position: 'afterUser', content: [thought(9), text('zz-odd')] }; + const dry = await manager.compile(BIG, [odd], bind({ dryRun: true })); + assert.equal(told.length, 0, 'a dry compile tells nothing'); + assert.equal(thinkingByText(dry)['zz-odd'], false, 'an injection never keeps thinking'); + + const first = await manager.compile(BIG, [odd], BIND); + assert.ok(first.thinkingStrip!.newly.includes(r3), 'a reply of redacted thinking alone is left out too'); + assert.deepEqual(told, [[thought(1), thought(2), redacted(3)]], 'the replies\u2019 thinking, not the injection\u2019s'); + await manager.compile(BIG, [odd], BIND); + assert.deepEqual(told[1], [thought(1), thought(2), redacted(3)], 'told again: each compile\u2019s estimate prices them again'); + await manager.close(); + }); + + it('calibration: the strategy\u2019s estimate leaves out the price of the thinking left out', async () => { + const estimateAfter = async (path: string, opts?: SelectOptions) => { + if (existsSync(path)) rmSync(path, { recursive: true, force: true }); + const strategy = new AutobiographicalStrategy({ autoTickOnNewMessage: false, headWindowTokens: 0, adaptiveResolution: true }); + const manager = await ContextManager.open({ path, strategy }); + seed(manager); + await manager.compile(BIG, undefined, opts); + const reach = strategy as unknown as { _lastCompileEstimate: number; estimateTokens(c: ContentBlock[]): number }; + const result = { estimate: reach._lastCompileEstimate, thoughts: reach.estimateTokens([thought(1), thought(2)]) }; + await manager.close(); + return result; + }; + const plain = await estimateAfter(STORE); + const bound = await estimateAfter(STORE_B, BIND); + assert.ok(plain.thoughts > 0, 'setup: the thinking has a price'); + assert.ok(plain.estimate > plain.thoughts, 'setup: the estimate covers the replies'); + assert.equal(bound.estimate, plain.estimate - plain.thoughts); + cleanup(); + }); + + it('thinkingRecordedAfter is the branch\u2019s sequence, at or above every message stored before, one the view filter hides included', async () => { + cleanup(); + const manager = await ContextManager.open({ + path: STORE, + strategy: new PassthroughStrategy(), + viewFilter: (message) => !(message.metadata as { hidden?: unknown } | undefined)?.hidden, + }); + seed(manager); + const hidden = manager.addMessage('User', [text('zz-hidden')], { hidden: true }); + // A state write after the newest message moves the branch's sequence past it. + manager.getStore().setStateJson('default/thinking-binding', null); + const sequence = manager.getStore().currentSequence(); + assert.ok(sequence > manager.getMessage(hidden)!.sequence, 'setup: the branch is past its newest message'); + const first = await manager.compile(BIG, undefined, BIND_DROP); + assert.equal(first.messages.length, 4, 'setup: the hidden message isn\u2019t sent'); + assert.ok(first.thinkingRecordedAfter! >= sequence); + assert.ok(first.thinkingRecordedAfter! <= manager.getStore().currentSequence()); + await manager.close(); + }); + + it('each namespace on a store keeps its own record', async () => { + cleanup(); + const a = await open(STORE, new PassthroughStrategy(), 'zz-a'); + seed(a); + await a.compile(BIG, undefined, BIND_DROP); + const recorded = JSON.stringify(recordOf(a, 'zz-a')); + const b = await ContextManager.open({ store: a.getStore(), strategy: new PassthroughStrategy(), namespace: 'zz-b' }); + assert.equal(recordOf(b, 'zz-b'), null, 'zz-b starts with none'); + await b.compile({ maxTokens: tokensOfNewest(a, 2), reserveForResponse: 0 }, undefined, BIND); + assert.notEqual(recordOf(b, 'zz-b'), null); + assert.equal(JSON.stringify(recordOf(a, 'zz-a')), recorded, 'zz-b\u2019s write left zz-a\u2019s record'); + await b.close(); + await a.close(); + }); +}); + +describe('thinking binding: a summary\u2019s carriers', () => { + it('a binding select renders and prices the live window as under live-strip', async () => { + // Carriers big enough that pricing them changes what fits in 900 tokens. + const carriers = (call: number): ContentBlock[] => [ + { type: 'thinking', thinking: `zz-archivist-${call} `.repeat(50), signature: `zz-asig-${call}-fedcba9876543210` } as ContentBlock, + { type: 'redacted_thinking', data: `zz-aenc-${call}-payload==` } as ContentBlock, + { type: 'text', text: `zz-adaptive-memory-${call} recollection` }, + ]; + await assert.rejects( + renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900), + /Adaptive picker exhausted/, + 'setup: rendered and priced with their carriers, these summaries don\u2019t fit', + ); + const stripped = await renderAdaptiveFixture({ carrierPolicy: 'live-strip' }, carriers, 900); + const bound = await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, BIND_DROP); + assert.ok(stripped.messages.length > 0, 'setup: the live-strip render fits'); + assert.equal(typeof stripped.estimate, 'number', 'setup: the fixture reads the estimate'); + assert.equal(typeof bound.estimate, 'number'); + assert.deepEqual(bound.messages, stripped.messages); + assert.equal(bound.estimate, stripped.estimate, 'priced as rendered'); + // A dry select prices by its own option, whatever compiled before it. + const preview = await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, { ...BIND_DROP, dryRun: true }); + assert.deepEqual(preview.messages, stripped.messages, 'a binding dry compile, with no committing compile before it'); + await assert.rejects( + renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, { dryRun: true }, BIND_DROP), + /Emission overran the plan/, + 'a dry compile without the option, after a binding compile, prices as configured: it reports an infeasible plan rather than refusing it, so it fails where its render overruns', + ); + // previewContext takes the option too. + const previewed: Record = {}; + await renderAdaptiveFixture({ carrierPolicy: 'full' }, carriers, 900, BIND_DROP, undefined, (manager) => { + const budget = { maxTokens: 900, reserveForResponse: 0 }; + previewed.bound = manager.previewContext(budget, undefined, { thinkingBinding: { binds: true } })!.fits; + previewed.unbound = manager.previewContext(budget)!.fits; + }); + assert.deepEqual(previewed, { bound: true, unbound: false }, 'priced in full, they don\u2019t fit'); + }); + + it('a recall pair is priced as its own select\u2019s binding says', async () => { + cleanup(); + const entry = { + id: 'zz-sum-bound', level: 1, content: 'zz-memory prose.', tokens: 4_000, + sourceIds: ['zz-m1'], sourceRange: { first: 'zz-m1', last: 'zz-m2' }, sourceLevel: 0, timestamp: 0, + responseContent: [ + { type: 'thinking', thinking: 'zz-archivist '.repeat(300), signature: 'zz-sig-bound-0123' }, + { type: 'text', text: 'zz-memory prose.' }, + ], + }; + const strategy = new AutobiographicalStrategy({ carrierPolicy: 'full', autoTickOnNewMessage: false }); + const stripStrategy = new AutobiographicalStrategy({ carrierPolicy: 'live-strip', autoTickOnNewMessage: false }); + const price = (s: AutobiographicalStrategy) => (s as unknown as { recallPairCost(e: unknown): number }).recallPairCost(entry); + const manager = await ContextManager.open({ path: STORE, strategy }); + manager.addMessage('User', [text('U1')]); + await manager.compile(BIG); + // The reference: the same pricing, on a store, under live-strip. + const reference = await ContextManager.open({ path: STORE_B, strategy: stripStrategy }); + reference.addMessage('User', [text('U1')]); + await reference.compile(BIG); + const full = price(strategy); + const live = price(stripStrategy); + assert.ok(full > live + 1_000, 'setup: the measured tokens, carriers included, set the full price'); + + // The price as each select sees it, from inside its select path. + // The price as each select sees it, from inside its select path, and as + // it stands once the select has returned. + const seen: number[] = []; + const after: number[] = []; + const hooked = strategy as unknown as { selectPath: (...args: unknown[]) => unknown }; + const selectPath = hooked.selectPath.bind(strategy); + hooked.selectPath = (...args: unknown[]) => { + seen.push(price(strategy)); + return selectPath(...args); + }; + for (const opts of [BIND, { dryRun: true }, { ...BIND, dryRun: true }, { thinkingBinding: { binds: false }, dryRun: true }, undefined]) { + await manager.compile(BIG, undefined, opts); + after.push(price(strategy)); + } + assert.deepEqual(seen, [live, full, live, full, full], 'each select prices by its own option'); + assert.deepEqual(after, [full, full, full, full, full], 'outside a select, as configured, after a binding one too'); + // And after a binding select that throws, committing or dry. + const refusing = { maxTokens: 1, reserveForResponse: 0 }; + await manager.compile(refusing, undefined, BIND).catch(() => undefined); + assert.equal(price(strategy), full, 'after a refused binding compile'); + await manager.compile(refusing, undefined, { ...BIND, dryRun: true }).catch(() => undefined); + assert.equal(price(strategy), full, 'after a refused binding dry compile'); + await manager.close(); + await reference.close(); + cleanup(); + }); +}); + +describe('thinking binding: bindThinking', () => { + const user = (t: string, participant = 'User'): NormalizedMessage => ({ participant, content: [text(t)] }); + const reply = (n: number, t: string): NormalizedMessage => ({ participant: 'Claude', content: [thought(n), text(t)] }); + const body = (id: string, sequence: number): RawBodySource => ({ messageId: id, sequence, complete: true }); + const raw = (...bodies: RawBodySource[]): CompiledMessageSources => ({ kind: 'raw', bodies }); + const recordFor = (messages: NormalizedMessage[], over: Partial = {}): BindingRecord => ({ + sent: messages.map((m) => messageDigest(m)), + continued: [], + without: [], + identity: identity(), + recordedAfter: 0, + compileId: 'zz-compile', + ...over, + }); + const input = (messages: NormalizedMessage[], sources: CompiledMessageSources[], record: BindingRecord | null) => ({ + messages, sources, record, dropBlock: true, identity: identity(), participant: 'Claude', sequence: 9, compileId: 'zz-compile-2', + }); + + it('compares content as the provider reads it: key order and cache breakpoints aside', () => { + const sent = [user('U1'), reply(1, 'A1')]; + const record = recordFor(sent); + const reordered: NormalizedMessage[] = [ + { participant: 'User', content: [{ text: 'U1', type: 'text' } as ContentBlock], cacheBreakpoint: true }, + reply(1, 'A1'), + ]; + const outcome = bindThinking(input(reordered, [raw(body('u1', 1)), raw(body('a1', 2))], record)); + assert.equal(outcome.strip, undefined); + assert.ok(outcome.messages[1]!.content.some(isThinking)); + assert.equal(outcome.record.recordedAfter, 0, 'kept from the record'); + assert.deepEqual(outcome.record.continued, [], 'a compile starts the request over'); + }); + + it('a message whose participant alone changed is a difference', () => { + const sent = [user('U1'), reply(1, 'A1')]; + const now = [user('U1', 'Someone'), reply(1, 'A1')]; + const outcome = bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 2))], recordFor(sent))); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'messages' }); + }); + + it('after the first difference, a reply left out before is not newly left out', () => { + const sent = [user('U1'), reply(1, 'A1'), user('U2'), { participant: 'Claude', content: [text('A2')] }]; + const record = recordFor(sent, { without: ['a2'] }); + const now = [user('U1-edited'), reply(1, 'A1'), user('U2'), reply(2, 'A2')]; + const outcome = bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 2)), raw(body('u2', 3)), raw(body('a2', 4))], record)); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'messages' }); + assert.deepEqual(outcome.record.without, ['a1', 'a2']); + assert.ok(outcome.messages.every((m) => !m.content.some(isThinking))); + }); + + it('a message of several bodies goes without when any of them did, and records them all', () => { + const merged: NormalizedMessage = { participant: 'Claude', content: [thought(1), text('A1'), text('A1b')] }; + const sent = [user('U1'), { participant: 'Claude', content: [text('A1'), text('A1b')] }]; + const record = recordFor(sent, { without: ['a1b'] }); + const outcome = bindThinking(input([user('U1'), merged], [raw(body('u1', 1)), raw(body('a1', 2), body('a1b', 3))], record)); + assert.equal(outcome.strip, undefined, 'left out before, so not newly'); + assert.ok(!outcome.messages[1]!.content.some(isThinking)); + assert.deepEqual(outcome.record.without, ['a1', 'a1b']); + }); + + it('the participant\u2019s message with no stored body never keeps its thinking', () => { + const sources: CompiledMessageSources[] = [ + raw(body('u1', 1)), + { kind: 'summary', summaries: [{ id: 'zz-s1', level: 1 }] }, + { kind: 'other' }, + raw(body('a1', 2)), + ]; + const now = [user('U1'), reply(8, 'S1'), reply(9, 'O1'), reply(1, 'A1')]; + const outcome = bindThinking(input(now, sources, null)); + assert.deepEqual(outcome.messages.map((m) => m.content.some(isThinking)), [false, false, false, true]); + assert.equal(outcome.strip, undefined, 'no stored body, so nothing is newly left out'); + assert.deepEqual(outcome.record.without, [], 'and nothing recorded as left out'); + assert.deepEqual(outcome.leftOut, [], 'nor handed to the strategy: only stored bodies\u2019 thinking is'); + }); + + it('past the record\u2019s end nothing keeps its thinking; a noted row extends the record, and an unvouched one matches nothing', () => { + const sent = [user('U1')]; + const now = [user('U1'), reply(1, 'A1'), reply(2, 'A2')]; + const keeps = (record: Partial) => + bindThinking(input(now, [raw(body('u1', 1)), raw(body('a1', 5)), raw(body('a2', 6))], recordFor(sent, record))).messages.map((m) => m.content.some(isThinking)); + assert.deepEqual(keeps({}), [false, false, false]); + assert.deepEqual(keeps({ continued: [messageDigest(reply(1, 'A1'))] }), [false, true, false]); + assert.deepEqual(keeps({ continued: [messageDigest(reply(1, 'A1')), messageDigest(reply(2, 'A2'))] }), [false, true, true]); + assert.deepEqual(keeps({ continued: [UNVOUCHED, messageDigest(reply(2, 'A2'))] }), [false, false, false]); + assert.deepEqual(keeps({ sent: [UNVOUCHED], continued: [messageDigest(reply(1, 'A1'))] }), [false, false, false], 'a record that vouches for nothing'); + }); + + it('a head change is the cause given, when replies from before the record go too', () => { + const sent = [user('U1')]; + const now = [user('U1'), reply(1, 'A1')]; + const outcome = bindThinking({ + ...input(now, [raw(body('u1', 1)), raw(body('a1', 2))], recordFor(sent, { recordedAfter: 5, identity: identity({ tools: 'zz-old' }) })), + dropBlock: false, + }); + assert.deepEqual(outcome.strip, { newly: ['a1'], cause: 'head', changed: ['tools'] }); + }); + + it('a body marked as left out gains \u2018content\u2019 once, and keeps what it was missing', () => { + const sources: CompiledMessageSources[] = [ + raw({ messageId: 'a1', sequence: 1, complete: false, missing: ['shards'] }), + raw(body('a2', 2)), + raw(body('u1', 3)), + ]; + markThinkingLeftOut(sources, new Set(['a1', 'a2'])); + markThinkingLeftOut(sources, new Set(['a2'])); + const bodiesOf = (i: number) => (sources[i] as { bodies: RawBodySource[] }).bodies[0]!; + assert.deepEqual(bodiesOf(0).missing, ['shards', 'content']); + assert.deepEqual(bodiesOf(1).missing, ['content']); + assert.equal(bodiesOf(1).complete, false); + assert.equal(bodiesOf(2).complete, true, 'a body not named is left as it was'); + }); + + it('a digest is 11 base64url characters, tells an array from an object, and keeps array order', () => { + const digest = messageDigest(user('U1')); + assert.match(digest, /^[A-Za-z0-9_-]{11}$/); + const withInput = (input: unknown): NormalizedMessage => ({ participant: 'Claude', content: [{ type: 'tool_use', id: 't', name: 'n', input } as ContentBlock] }); + assert.notEqual(messageDigest(withInput({ value: ['x'] })), messageDigest(withInput({ value: { 0: 'x' } }))); + assert.notEqual(messageDigest(withInput({ value: ['x', 'y'] })), messageDigest(withInput({ value: ['y', 'x'] }))); + assert.equal(messageDigest(withInput({ a: 1, b: 2 })), messageDigest(withInput({ b: 2, a: 1 })), 'control: key order aside'); + const marked = (key: string): NormalizedMessage => ({ participant: 'User', content: [{ type: 'text', text: 'U1', [key]: { type: 'ephemeral' } } as unknown as ContentBlock] }); + assert.equal(messageDigest(marked('cache_control')), digest, 'a block\u2019s cache marker aside'); + assert.equal(messageDigest(marked('cacheControl')), digest); + }); +}); + +// The public types, through both barrels: a removed export fails the build. +const exported: { binding: ThinkingBinding; identity: RequestIdentity | RequestIdentityFromTypes; strip: ThinkingStrip | ThinkingStripFromTypes } = { + binding: { binds: true, participant: 'Claude' }, + identity: identity(), + strip: { newly: [], cause: 'messages' }, +}; +void exported; From edef4136001bcaa674e9003108e7f4097d07ea21 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 19:58:56 -0400 Subject: [PATCH 43/67] fix(pins): every reader reads a pin over a long message whole, and the plan's over-budget refusal names the pins Under adaptive resolution a pin on any part of a message stored in parts now covers the whole message for every reader of pins (pinPositions): the selects, compression, the summarizer's prompt and its pinned-raw warning, and the replay exporter, as a pinRange over all its parts does where no part is locked. A part locked with lockChunk outside the span a pin's ids name keeps its lock; one between them is covered, as a classic pin over a lock is. Without adaptive resolution a pin covers its ids, as before. The previous change read pins whole only in the selects and by their ids in compression and the summarizer's prompt, so a mark was the one pin the agent and the summarizer read differently, and a summary of a marked message's other parts could go out beside the raw message. That second reading goes, with what rested on it: narrowing a pin back to its ids when a compile couldn't hold it whole, and its warning. A compile whose budget can't hold a whole message now refuses, as for any pin the budget can't hold. The refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but ...") names the pins keeping raw tokens in that middle: diagnostics.rawPins ({ tokens, pins }, each an OverBudgetRawPin, now exported) gives the raw tokens priced as the plan priced them (middlePrice, now shared with the plan's chunks), and its message ends with a sentence naming each pin and saying that whoever manages the pins can unpin them, which frees up to their tokens less their summaries, or whoever sets the hard budget can raise it where the model's window allows. It says who can act, since a host can pass the message on to an agent. A part locked raw isn't counted, since it stays raw without its pin; where bounds are read, a pin whose own bound isn't raw isn't named. The refusals after the plan, and the hierarchical select's, count other things than the planned middle and name no pins. The summarizer's pinned-raw warning advises unpinning, or narrowing a range pin to fewer messages, since a pin on any part of a message stored in parts covers the rest of it. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/raw-pins-in-refusals.added.md | 1 + changelog.d/whole-body-pins.fixed.md | 6 +- docs/adaptive-resolution-design.md | 15 +- scripts/export-kv-replay-metadata.mjs | 7 +- src/adaptive/index.ts | 1 + src/adaptive/picker.ts | 55 +- src/context-manager.ts | 24 +- src/index.ts | 2 +- src/strategies/autobiographical.ts | 478 +++------- src/types/strategy.ts | 28 +- test/l1-recall-raw-pins-frontier.test.ts | 11 +- test/whole-body-pins.test.ts | 1032 +++++++-------------- 13 files changed, 588 insertions(+), 1074 deletions(-) create mode 100644 changelog.d/raw-pins-in-refusals.added.md diff --git a/README.md b/README.md index 2b141e92..ba4f2d7e 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a mark on any part of a long message stored in parts covers the whole message, less a part locked with `lockChunk` that the mark's ids don't name, where the lock holds: a raw mark sends it raw where the budget holds it, and a mark with a level, where the solver reads levels, holds all of it at that level, while compression still summarizes the parts the mark doesn't name. Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the mark covers renders at the level the mark holds that summary at, as one sharing a summary with the part its ids name did before. A compile that can't hold the whole message, but can hold the rest with the mark narrowed to the part it names, narrows a mark that renders the message raw, says so in a warning, and keeps it that way until the manager loads the store again (an open, a branch switch, a fork or `setStrategy`). `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a pin or mark on any part of a long message stored in parts covers the whole message, in what the agent is sent, in compression and in the summarizer's prompt alike, less a part locked with `lockChunk` outside the span its ids name. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at. A compile whose budget can't hold the raw pins beside the rest refuses with `OverBudgetError`; where the adaptive plan's middle is over the budget, its message and `diagnostics.rawPins` name each pin keeping raw tokens in that middle. `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/raw-pins-in-refusals.added.md b/changelog.d/raw-pins-in-refusals.added.md new file mode 100644 index 00000000..6b7aec5f --- /dev/null +++ b/changelog.d/raw-pins-in-refusals.added.md @@ -0,0 +1 @@ +- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them, a part locked raw left out), and the message ends with a sentence naming each pin and its tokens, and saying that whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to their tokens less their summaries, or whoever sets the hard budget can raise it where the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent. Each pin's tokens are what it keeps raw; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 7f4e8558..85ebd761 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ -- **Pins and document marks render whole messages.** A pin whose first or last id names one part of a message stored in parts (a long message that ingress splits under `adaptiveResolution`) now covers that message from its first part to its last in the adaptive-resolution selects, what the agent is sent. The summarizer's prompt still reads a pin by its ids: it has the message's other parts as summaries, and carrying them raw multiplied later compression requests' input. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. Compression still reads a pin by its ids, so the message's other parts are chunked and summarized as before. A part locked with `lockChunk` that the pin's ids don't name keeps its lock: the whole reading leaves it out. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - - **When the whole messages don't fit.** A compile refused over budget narrows pins that render the message raw back to their own ids, one at a time, the one whose whole messages add the most that compression has summarized first, until a select goes ahead. Where compression has kept up, those parts are summarized already, so that select can send their summaries, as compiles did before this change. A narrowed pin stays narrowed until the store is loaded again (an open, a branch switch, a fork or `setStrategy`), and a warning names it and says how to have it whole again. Nothing is stored: a store loaded again, as at a restart, reads every pin whole, so after the budget is raised the whole message comes back, and one that still doesn't fit is narrowed again at its first refused compile. When no narrowing lets the compile through, the pins go back and the refusal stands. A refusal whose middle holds nothing, as when the head and recent windows alone are over the budget, narrows nothing. A dry compile narrows for itself only. - - **Upgrade.** A stored mark on part of a long message reads whole. Where the whole message fits, the first compile after the upgrade rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't, the first compile narrows the mark and warns, and in a store with that one mark it sends the request the earlier version would have sent. +- **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. + - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. + - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold, and that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages the pins can unpin it, which lets compression fold the message, or whoever sets the hard budget can raise it, where the model's window allows. diff --git a/docs/adaptive-resolution-design.md b/docs/adaptive-resolution-design.md index 7a999cee..f92f74ea 100644 --- a/docs/adaptive-resolution-design.md +++ b/docs/adaptive-resolution-design.md @@ -374,14 +374,13 @@ This case should be rare in practice with unbounded L_n: the picker can almost a > **refuses the turn** (`UncoveredDropError`) — pressure never ships silent > loss. The philosophy stands: surface the overage, let the host decide. > -> *(Pins on messages stored in parts)* A pin or document mark covers the -> whole of a message stored in parts in the selects. Where that is what -> doesn't fit, a refusal is answered first by narrowing a pin that renders -> the message raw to the parts its ids name (`selectNarrowingPins`), and the -> select runs again: the -> narrowing lasts until the store is loaded again, and a host-log warning -> names the pin. When no narrowing lets the compile through, the refusal is -> thrown as above. +> *(Raw pins)* The refusal of a plan whose middle is over the budget names +> each pin that keeps raw tokens there (`OverBudgetDiagnostics.rawPins`), with +> the tokens it keeps. Under adaptive resolution a pin on part of a message +> stored in parts covers the whole message, so a mark on a long message can +> be what doesn't fit: whoever manages the pins can unpin it, which lets +> compression fold it, or whoever sets the budget can raise it, where the +> model's window allows. ### 3.11 Recent-window slide diff --git a/scripts/export-kv-replay-metadata.mjs b/scripts/export-kv-replay-metadata.mjs index 6b082a1a..6823dc5d 100644 --- a/scripts/export-kv-replay-metadata.mjs +++ b/scripts/export-kv-replay-metadata.mjs @@ -19,11 +19,8 @@ try { strategy.loadCalibration(view); const messages = view.getAll(); const estimates = strategy.postStripEstimates(view); - // The selects' reading, as at a restart: under adaptive resolution a pin - // covers the whole of a message stored in parts. A narrowing a live compile - // made isn't stored, so this reads that pin whole too. - const pins = strategy.pinnedPositions(messages, strategy.selectsPinReading()); - const bounds = strategy.pinLevelBounds(messages, strategy.selectsPinReading()); + const pins = strategy.pinnedPositions(messages); + const bounds = strategy.pinLevelBounds(messages); const metadata = messages.map((message, index) => ({ id: message.id, index, timestamp: message.timestamp.getTime(), postStripTokens: estimates[index], toolResult: strategy.hasToolResult(message), diff --git a/src/adaptive/index.ts b/src/adaptive/index.ts index ea7311d3..d22dabac 100644 --- a/src/adaptive/index.ts +++ b/src/adaptive/index.ts @@ -151,6 +151,7 @@ export { accountFrontier, type OverBudgetClearedBy, type OverBudgetDiagnostics, + type OverBudgetRawPin, type PickerInputs, type PickerResult, type PickerChunk, diff --git a/src/adaptive/picker.ts b/src/adaptive/picker.ts index 4cb403a7..316fe3fb 100644 --- a/src/adaptive/picker.ts +++ b/src/adaptive/picker.ts @@ -51,6 +51,34 @@ export interface OverBudgetDiagnostics { reserveForResponse?: number; /** Configured window minus the response reserve, before over-budget grace. */ inputBudget?: number; + /** + * The pins that keep messages raw in the middle, which compression would + * otherwise fold: outside the head and recent windows. `tokens` is the raw + * tokens they keep there together, estimated as the select prices them, + * and each pin gives its own; a message two pins cover counts once in + * `tokens` and toward each pin. Left out when no pin keeps any. + */ + rawPins?: { tokens: number; pins: readonly OverBudgetRawPin[] }; +} + +/** A pin that keeps messages raw in an over-budget refusal's middle (`OverBudgetDiagnostics.rawPins`). */ +export interface OverBudgetRawPin { + /** The pin's id, as `unpin` takes it. */ + pinId: string; + kind: 'pin' | 'document'; + /** Its name, when it was given one. */ + name?: string; + firstMessageId: string; + lastMessageId: string; + /** + * Under adaptive resolution, how many stored messages the pin covers when + * it covers more than its ids name: a pin on part of a message stored in + * parts covers that message's parts, less a part locked with `lockChunk` + * outside the span its ids name. Left out otherwise. + */ + parts?: number; + /** Raw tokens the pin keeps in the middle. */ + tokens: number; } /** @@ -140,6 +168,31 @@ export interface OverBudgetDiagnostics { */ export type OverBudgetClearedBy = 'compression' | 'newer-message' | 'smaller-recent-window'; +/** The sentence naming the pins that keep raw tokens in a refusal's middle, or nothing when none do. */ +function rawPinsText(raw: OverBudgetDiagnostics['rawPins']): string { + if (!raw || raw.pins.length === 0) return ''; + const { tokens: total, pins } = raw; + const each = pins.map((pin) => { + const what = pin.kind === 'document' ? 'document mark' : 'pin'; + const label = pin.name !== undefined ? `"${pin.name}" (${pin.pinId})` : pin.pinId; + const span = pin.firstMessageId === pin.lastMessageId + ? `message ${pin.firstMessageId}` : `messages ${pin.firstMessageId} to ${pin.lastMessageId}`; + const parts = pin.parts === undefined ? '' + : pin.firstMessageId === pin.lastMessageId ? `, covering ${pin.parts} stored parts of its message` + : `, covering ${pin.parts} stored messages`; + return `${what} ${label}, ${span}${parts}, ~${pin.tokens} tokens`; + }); + // Several pins can cover one message, so unpinning one of them frees none + // of it: the remedy speaks of them together, whose tokens count it once. + // It says who can act rather than telling its reader to, since a host can + // pass the text on to an agent that can do neither. + const remedy = pins.length === 1 + ? 'Whoever manages the pins can unpin it, which lets compression fold what it covers and frees up to its tokens less its summary' + : 'Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those tokens less their summaries'; + return `. ${pins.length === 1 ? 'A pin keeps' : 'Pins keep'} ~${total} tokens raw in the middle: ${each.join('; ')}. ` + + `${remedy}; or whoever sets the hard budget can raise it, if the model's window allows.`; +} + /** * Error raised when the picker has folded everything it can and the * resulting context still exceeds the hard token budget. @@ -209,7 +262,7 @@ export class OverBudgetError extends Error { (opts.diagnostics.configuredBudget === undefined ? '' : `, configured=${opts.diagnostics.configuredBudget}`) + (opts.diagnostics.reserveForResponse === undefined ? '' : `, response reserve=${opts.diagnostics.reserveForResponse}`) + (opts.diagnostics.inputBudget === undefined ? '' : `, input budget=${opts.diagnostics.inputBudget}`) + - ')' + ')' + rawPinsText(opts.diagnostics.rawPins) ); // Writable per Error convention so instanceof-by-name works across // iframe / vm boundaries; the field stays writable on the prototype. diff --git a/src/context-manager.ts b/src/context-manager.ts index 2b2931d8..00d58b8f 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1639,9 +1639,12 @@ export class ContextManager { /** * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel`. A raw pin, with no bound or a bound of 0, keeps the - * messages it names out of compression and renders them raw at their - * original chronological position, a message stored in parts whole, as - * `ProtectedRange` describes. A bound above 0 is read only under + * messages out of compression and renders them raw at their original + * chronological position, a message stored in parts whole, as + * `ProtectedRange` describes. A compile whose budget can't hold them beside + * the rest refuses with `OverBudgetError`; where the adaptive plan's middle + * is over the budget, its message and `diagnostics.rawPins` name each pin + * keeping raw tokens in that middle. A bound above 0 is read only under * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution`: there the range is still compressed, since only a * summary can supply its level, and is held at `level`, or no deeper than @@ -1668,7 +1671,7 @@ export class ContextManager { * it renders where one does, or raw (`ProtectedRange.level`). Honored * by `foldingStrategy: 'kv-stable'` and `'kv-unified'` with adaptive * resolution; under any other selector the range is raw, in the live view - * and, read by its ids, in the canonical L1 compression prompt where its budgets allow (see + * and in the canonical L1 compression prompt where its budgets allow (see * `compressionRecallBudgetTokens`). Returns the new pin id. */ pinAtLevel(firstMessageId: MessageId, lastMessageId: MessageId, level: number, opts?: { name?: string }): string { @@ -1683,13 +1686,12 @@ export class ContextManager { * or `maxLevel` the message is kept raw, while * `markDocument(id, { maxLevel })` lets it fold no deeper than that level * where the selector reads level bounds. Under adaptive resolution, a mark - * on any part of a message stored in parts renders the whole message, less - * a part locked with `lockChunk` that it doesn't name, as `pinRange` does: - * where the whole message doesn't fit, a mark that renders it raw is - * narrowed to the part it names. Under `kv-stable`, a mark with `level` - * (not `maxLevel`) holds whole each summary it reaches, so a locked part - * that shares a summary with a part the mark covers renders at the level - * the mark holds that summary at. + * on any part of a message stored in parts covers the whole message, less + * a part locked with `lockChunk` outside the span the mark's id names, as + * `pinRange` does. + * Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each + * summary it reaches, so a locked part that shares a summary with a part + * the mark covers renders at the level the mark holds that summary at. * Returns the new pin id. */ markDocument(messageId: MessageId, opts?: PinLevelOptions): string { diff --git a/src/index.ts b/src/index.ts index af88b229..662bbcfc 100644 --- a/src/index.ts +++ b/src/index.ts @@ -43,7 +43,7 @@ export type { OverBudgetClearedBy } from './adaptive/picker.js'; export { CompressionWithdrawn } from './compression-withdrawn.js'; export { StoreTopologyError, type TopologyViolation } from './strategies/autobiographical.js'; export { planTopologyRepair, type RepairPlan, type RepairInputs, type RepairOptions } from './repair/topology.js'; -export type { OverBudgetDiagnostics } from './adaptive/picker.js'; +export type { OverBudgetDiagnostics, OverBudgetRawPin } from './adaptive/picker.js'; // Types export type { diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 125dec76..263e9b72 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -53,7 +53,7 @@ import { persistMintRequestPreimage } from '../mint-preimage.js'; import { appendFileSync, mkdirSync } from 'node:fs'; import { dirname } from 'node:path'; import { createHash } from 'node:crypto'; -import { Picker, OverBudgetError, UncoveredDropError, type OverBudgetClearedBy, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; +import { Picker, OverBudgetError, UncoveredDropError, type OverBudgetClearedBy, type OverBudgetDiagnostics, type OverBudgetRawPin, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; import { FlatProfileStrategy } from '../adaptive/strategies/flat-profile.js'; import { KvStableStrategy } from '../adaptive/strategies/kv-stable.js'; import { reportOverlap } from '../adaptive/kv-control.js'; @@ -2154,32 +2154,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { protected pins: ProtectedRange[] = []; /** Monotonically increasing counter for pin ids. Persisted as part of the pins snapshot. */ protected pinIdCounter = 0; - /** - * Pins a compile narrowed (`selectNarrowingPins`), by `narrowKey`: their - * whole reading (`pinSpan`) covers only their own ids until the strategy - * loads the store again (`initialize`: an open, a branch switch, a fork or - * `setStrategy`, which clear it with the branch's other mirrors), a reload - * of the store's state mid-life (`takeUpStoreState`) excluded. Nothing - * records the narrowing itself, so a store loaded again reads every pin - * whole; what a narrowed compile sent is recorded as any compile's is. A pin - * unpinned and marked again is a new pin, and whole. - */ - protected readonly narrowedPins = new Set(); - - /** A pin's key in `narrowedPins`: its id, ends and creation time, which no API changes. */ - protected narrowKey(pin: ProtectedRange): string { - return JSON.stringify([pin.id, pin.firstMessageId, pin.lastMessageId, pin.created]); - } - - /** - * The reading the summarizer's prompt takes for pins (`pinSpan`), and so - * everything that describes that prompt: its pinned-raw warning and the - * count of queued work that predicts it (`requestCarriesToolBlocks`). By - * their ids: the prompt already has a marked message's other parts as - * summaries, and carrying them raw multiplied later compression requests' - * input. - */ - protected static readonly PROMPT_PIN_READING = 'ids' as const; /** * Per-message resolution state for the adaptive-resolution picker. @@ -2567,9 +2541,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * lock while it stands: under the greedy solvers and kv-stable, the pinned * message renders raw, and its locked level stays stored and returns when * the pin is removed. kv-unified treats a pin and a lock as constraints and - * reports a conflict when they are incompatible. A pin whose ids name - * another part of a message stored in parts doesn't cover a locked part - * (`pinPositions`): there the lock holds, under every solver. + * reports a conflict when they are incompatible. A pin on another part + * of a message stored in parts doesn't cover a locked part outside the span + * its ids name (`pinPositions`): there the lock holds, under every solver. * No-op when adaptiveResolution is false. Set-only programmatic API per * the design (no agent-facing tool in V1). Persisted to chronicle. */ @@ -2864,7 +2838,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { this.warnedQuarantineSkips.clear(); this.pins = []; this.pinIdCounter = 0; - this.narrowedPins.clear(); this.resolutions.clear(); this.locked.clear(); this._adaptivePicker = null; @@ -3541,12 +3514,16 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * Pin a range of messages. A pin's bound is its `level`, or without one * its `maxLevel` (`normalizePinLevels`). A raw pin, with no bound or a - * bound of 0, keeps the messages it names out of compression and renders - * them raw at their original position, a message stored in parts whole - * (`pinSpan`): where the whole messages don't fit the budget but the rest - * does with the pin narrowed to its ids, a compile narrows it until the - * strategy loads the store again (`selectNarrowingPins`). A bound above 0 - * is read only under `foldingStrategy: 'kv-stable'` and `'kv-unified'` with + * bound of 0, keeps the messages out of compression and renders them raw + * at their original position. A compile whose budget can't hold them + * beside the rest refuses with `OverBudgetError`; where the adaptive plan's + * middle is over the budget, it names each pin keeping raw tokens in that + * middle (`rawPinsDiagnostics`). Under adaptive resolution, a message + * stored in parts counts as one: a pin whose first or last id names one of its parts + * covers it from its first part to its last, less a part locked with + * `lockChunk` outside the span its ids name, for every reader of pins + * (`pinPositions`). A bound above 0 is read only under + * `foldingStrategy: 'kv-stable'` and `'kv-unified'` with * `adaptiveResolution` (`readsPinBounds`): there the range is still * compressed, since only a summary can supply its level, and is held at * `level`, or no deeper than `maxLevel`, as `ProtectedRange` describes. @@ -3578,13 +3555,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `level` or `maxLevel` the message is kept raw, while `{ maxLevel }` lets * it fold no deeper than that level where the selector reads level bounds. * Under adaptive resolution, a mark on any part of a message stored in - * parts renders the whole message, less a part locked with `lockChunk` - * that it doesn't name (`pinPositions`), as `pinRange` does: where the - * whole message doesn't fit, a mark that renders it raw is narrowed to the - * part it names (`selectNarrowingPins`). Under `kv-stable`, a mark with - * `level` (not `maxLevel`) holds whole each summary it reaches, so a locked - * part that shares a summary with a part the mark covers renders at the - * level the mark holds that summary at. + * parts covers the whole message, as `pinRange` does (`pinPositions`). + * Under `kv-stable`, a mark with `level` (not `maxLevel`) holds whole each + * summary it reaches, so a locked part that shares a summary with a part + * the mark covers renders at the level the mark holds that summary at. */ markDocument(messageId: string, opts?: PinLevelOptions): string { this.requireLoadedBranch('markDocument'); @@ -3609,8 +3583,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * summary at `level` doesn't list it renders where one does, or raw * (`ProtectedRange.level`). Honored by `foldingStrategy: 'kv-stable'` * and `'kv-unified'` with adaptive resolution; under any other selector the - * range is raw (`resolvedPinBounds`), in the live view and, read by its - * ids, in the canonical L1 compression prompt where its budgets allow (see + * range is raw (`resolvedPinBounds`), in the live view and in the canonical + * L1 compression prompt where its budgets allow (see * `compressionRecallBudgetTokens`). * Equivalent to `pinRange(first, last, { level })`. */ @@ -3802,84 +3776,50 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * The store positions a pin covers, first and last inclusive, in one of its - * two readings (`ProtectedRange`): - * - `'ids'`: the messages from its first id to its last, a part of a - * message stored in parts covering only itself. The summarizer's prompt - * reads this (`PROMPT_PIN_READING`). Compression keeps out only parts raw - * this way (`compressionPinnedPositions`). - * - `'whole'`: from the first part of the message its first id is in to the - * last part of the message its last id is in. The selects read this under - * adaptive resolution (`selectsPinReading`), so the whole message goes out - * raw where the budget holds it. A pin narrowed in this instance - * (`narrowedPins`) reads by its ids here too. A part locked with - * `lockChunk` outside the pin's ids isn't covered (`pinPositions`). - * Undefined when either id is no longer in the store. + * The store positions a pin covers, in order. Without adaptive resolution, + * the messages from its first id to its last. Under adaptive resolution, + * the only ingress that stores a message in parts, such a message counts as + * one: the span runs from the first part of the message its first id is in + * to the last part of the message its last id is in (`bodyBoundsIn`), less + * a part locked with `lockChunk` outside the span its ids name. A lock is + * said of that part, while covering the rest of its message is what a pin + * on another part implies, so the lock holds there; a pin over its own + * locked messages keeps the rule `lockChunk` describes. Every reader of + * pins takes this: the selects, compression, the summarizer's prompt and + * the replay exporter. Undefined when either id is no longer in the store. */ - protected pinSpan( + protected pinPositions( pin: ProtectedRange, positionOf: ReadonlyMap, messages: readonly StoredMessage[], - reading: 'ids' | 'whole', - ): { lo: number; hi: number } | undefined { - if (reading !== 'ids' && reading !== 'whole') throw new TypeError(`pinSpan: a reading is 'ids' or 'whole', not ${String(reading)}`); + ): number[] | undefined { const first = positionOf.get(pin.firstMessageId); const last = positionOf.get(pin.lastMessageId); if (first === undefined || last === undefined) return undefined; const lo = Math.min(first, last); const hi = Math.max(first, last); - if (reading === 'ids' || this.narrowedPins.has(this.narrowKey(pin))) return { lo, hi }; - return { lo: bodyBoundsIn(messages, lo).from, hi: bodyBoundsIn(messages, hi).to }; - } - - /** - * The store positions a pin covers in `reading`: its span (`pinSpan`), - * less the messages locked with `lockChunk` that only the whole reading - * adds. A lock is said of that part, while the whole reading is what a mark - * on another part implies, so the lock holds there. A pin over its own - * locked messages keeps the rule `lockChunk` describes. Undefined when - * either id is no longer in the store. - */ - protected pinPositions( - pin: ProtectedRange, - positionOf: ReadonlyMap, - messages: readonly StoredMessage[], - reading: 'ids' | 'whole', - ): number[] | undefined { - const span = this.pinSpan(pin, positionOf, messages, reading); - if (!span) return undefined; - const ids = reading === 'ids' ? span : this.pinSpan(pin, positionOf, messages, 'ids')!; + const whole = this.config.adaptiveResolution === true; + const from = whole ? bodyBoundsIn(messages, lo).from : lo; + const to = whole ? bodyBoundsIn(messages, hi).to : hi; const out: number[] = []; - for (let i = span.lo; i <= span.hi; i++) { - if ((i < ids.lo || i > ids.hi) && this.locked.has(messages[i].id)) continue; + for (let i = from; i <= to; i++) { + if ((i < lo || i > hi) && this.locked.has(messages[i].id)) continue; out.push(i); } return out; } /** - * The reading the selects take (`pinSpan`), the agent's view, with what - * must agree with them (the narrowing's candidates, and what compression - * keeps out): `'whole'` under adaptive resolution, the only ingress that - * stores a message in parts, and `'ids'` otherwise, so a store written with - * it and opened without it reads its pins as before. The summarizer's prompt - * reads pins by their ids either way. + * Build a set of message-store positions covered by any pin + * (`pinPositions`). O(N pins · K range). Returns positions for which the + * message exists; orphan pins (deleted messages) are silently skipped. */ - protected selectsPinReading(): 'ids' | 'whole' { - return this.config.adaptiveResolution ? 'whole' : 'ids'; - } - - /** - * Build a set of message-store positions covered by any pin, in `reading` - * (`pinSpan`). O(N pins · K range). Returns positions for which the message - * exists; orphan pins (deleted messages) are silently skipped. - */ - protected pinnedPositions(messages: StoredMessage[], reading: 'ids' | 'whole'): Set { + protected pinnedPositions(messages: StoredMessage[]): Set { if (this.pins.length === 0) return new Set(); const positionOf = this.positionIndex(messages); const out = new Set(); for (const pin of this.pins) { - for (const i of this.pinPositions(pin, positionOf, messages, reading) ?? []) out.add(i); + for (const i of this.pinPositions(pin, positionOf, messages) ?? []) out.add(i); } return out; } @@ -3894,15 +3834,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { * kv-stable's reading applies: a bound there takes the place of a classic * pin, a fixed level takes the place of a cap, and within each kind the * lowest wins (`pinLevelBounds`). A level or cap of 0 is raw - * (`isRawPinBound`). Pins are read in `reading` (`pinSpan`): the summarizer's - * prompt by their ids, the selects as whole messages, and compression keeps - * out what both keep raw. + * (`isRawPinBound`). */ - protected resolvedPinBounds(messages: StoredMessage[], reading: 'ids' | 'whole'): Map { + protected resolvedPinBounds(messages: StoredMessage[]): Map { const out = new Map(); - const pinned = this.pinnedPositions(messages, reading); + const pinned = this.pinnedPositions(messages); if (pinned.size === 0) return out; - const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages, reading) : undefined; + const bounds = this.readsPinBounds() ? this.pinLevelBounds(messages) : undefined; for (const position of pinned) { const bound = bounds?.get(position); if (bound?.level !== undefined) out.set(position, { level: bound.level }); @@ -3926,10 +3864,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { return 'level' in bound ? bound.level === 0 : bound.maxLevel === 0; } - /** Positions of the messages that pins keep raw, in `reading` (`resolvedPinBounds`). */ - protected forceRawPinPositions(messages: StoredMessage[], reading: 'ids' | 'whole'): Set { + /** Positions of the messages that pins keep raw (`resolvedPinBounds`). */ + protected forceRawPinPositions(messages: StoredMessage[]): Set { const out = new Set(); - for (const [position, bound] of this.resolvedPinBounds(messages, reading)) { + for (const [position, bound] of this.resolvedPinBounds(messages)) { if (AutobiographicalStrategy.isRawPinBound(bound)) out.add(position); } return out; @@ -3939,18 +3877,18 @@ export class AutobiographicalStrategy implements ResettableStrategy { * The V2 dynamic-pin fold-depth bounds (`ProtectedRange.level` / * `maxLevel`) by message position. Only pins that carry a bound appear; a * classic raw pin (no bound) is absent here and handled by `pinnedPositions`. - * Where bounded pins overlap, the lowest fixed level and the lowest cap are - * kept; `resolvedPinBounds` decides between them and a classic pin. Pins - * are read in `reading` (`pinSpan`). + * A pin covers what `pinPositions` says. Where bounded pins overlap, the + * lowest fixed level and the lowest cap are kept; `resolvedPinBounds` + * decides between them and a classic pin. */ - protected pinLevelBounds(messages: StoredMessage[], reading: 'ids' | 'whole'): Map { + protected pinLevelBounds(messages: StoredMessage[]): Map { const out = new Map(); if (this.pins.length === 0) return out; const positionOf = this.positionIndex(messages); for (const pin of this.pins) { if (pin.level === undefined && pin.maxLevel === undefined) continue; - for (const i of this.pinPositions(pin, positionOf, messages, reading) ?? []) { + for (const i of this.pinPositions(pin, positionOf, messages) ?? []) { const prev = out.get(i) ?? {}; // A fixed level is the strongest constraint; when two pins fix the same // position, the shallower (lower) level wins (finest requirement). @@ -8042,169 +7980,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { } private selectPath(store: MessageStoreView, budget: TokenBudget, opts?: SelectOptions): ContextEntry[] { - const cacheKeys = this._prevCacheKeys; - try { - return this.selectOnce(store, budget, opts); - } catch (err) { - if (!(err instanceof OverBudgetError)) throw err; - return this.selectNarrowingPins(store, budget, opts, err, cacheKeys); - } - } - - /** - * Answers a select refused over budget (`refusal`) where the whole messages - * pins cover may be what doesn't fit: a mark on a long message's first part - * keeps the whole message raw (`pinSpan`), which can be more than the - * budget holds beside the rest. Pins are narrowed to their own ids one at a - * time, the one that frees the most first (`freedByNarrowing`), and the - * select runs again after each. Compression reads pins by their ids, so the - * parts a narrowing frees are chunked and summarized like any history, and - * the narrowed select can send their summaries. - * - A refusal whose middle holds nothing, as when the head and recent - * windows alone are over the budget, stands as it was: nothing is - * narrowed and no select runs again. - * - Where the selects read pins by their ids, as without adaptive - * resolution, no pin covers more than its ids, so the refusal stands - * without that look. - * - When a select goes ahead, the pins narrowed so far stay narrowed until - * the strategy loads the store again (`narrowedPins`), and a warning - * names each. Nothing records them, so a store loaded again reads every - * pin whole: with a larger budget it sends the whole message, and one - * that still doesn't fit narrows again at its first refused compile. - * - A provisional calibration multiplier above 1 that puts the whole - * reading over is dropped first (`selectOnce`), as for any refusal, so a - * guess the first sample replaces doesn't narrow a pin. - * - When no pin is left to narrow, the pins go back to whole messages and - * the first refusal is thrown, with its preview and the frontier it - * planned (`getHotContextSettings`). - * - A dry compile narrows for its own select only: it puts the pins back - * and warns nothing. - */ - private selectNarrowingPins( - store: MessageStoreView, - budget: TokenBudget, - opts: SelectOptions | undefined, - refusal: OverBudgetError, - cacheKeys: string[] | undefined, - ): ContextEntry[] { - if (this.selectsPinReading() === 'ids' || !(refusal.diagnostics.middleTokens > 0)) throw refusal; - // What the refused select recorded for observers, which a narrowed - // attempt overwrites and a refusal that stands puts back. - const preview = this._lastPreview; - const frontier = this.lastFrontierTokens; - const blocked = this.transitionBlocked; - const narrowed: ProtectedRange[] = []; - const putBack = (): void => { - for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); - this._lastPreview = preview; - this.lastFrontierTokens = frontier; - this.transitionBlocked = blocked; - }; - let entries: ContextEntry[] | undefined; - try { - // The order is taken once, before the first narrowing. Narrowing a raw - // pin changes no other pin's reading, and what it stops covering - // mostly stays out of compression as it was (a position there is raw - // by some pin's ids, and so by that pin's whole reading too). Where a - // pin with a level overlaps that position, the level can take the - // other pin's place once this one is narrowed (`resolvedPinBounds`), so - // later candidates' counts can be off there; the loop doesn't take - // them again. - for (const { pin } of this.freedByNarrowing(store)) { - this.narrowedPins.add(this.narrowKey(pin)); - narrowed.push(pin); - // A refused select can have placed its cache markers; each attempt is - // measured against the request before the compile, as select puts back. - this._prevCacheKeys = cacheKeys; - try { - entries = this.selectOnce(store, budget, opts); - break; - } catch (err) { - if (!(err instanceof OverBudgetError)) throw err; - } - } - } catch (err) { - putBack(); - throw err; - } - if (!entries) { - putBack(); - throw refusal; - } - if (opts?.dryRun === true) for (const pin of narrowed) this.narrowedPins.delete(this.narrowKey(pin)); - else this.warnNarrowedPins(narrowed, refusal); - return entries; - } - - /** - * What narrowing each raw pin would free, the most first: the raw tokens of - * the messages its reading as the selects take it (`selectsPinReading`) - * covers beyond its own ids that compression can summarize - * (`getCompressibleMessages`: outside the head and recent windows) and that - * have an L1 the select can fold them to (their chunk's summary, else - * `l1ByMessageIndex`, as selectAdaptive finds it), priced as the selects - * price them (`postStripEstimates`), less those the selects hold at a - * level. A part locked with `lockChunk` isn't among them: the whole reading - * leaves it out (`pinPositions`). Only those can go out as summaries once - * it's narrowed. Where the selects read bounds, a pin with a level or cap - * above 0 frees nothing, since its parts are held at that level: its parts - * already fold, and where it overlaps a raw pin, narrowing it could turn - * parts raw. Elsewhere every pin renders raw and can be narrowed. A pin - * that frees nothing, a narrowed one included, isn't listed. - */ - private freedByNarrowing(store: MessageStoreView): Array<{ pin: ProtectedRange; tokens: number }> { - if (this.pins.length === 0) return []; - const messages = store.getAll(); - const positionOf = this.positionIndex(messages); - const compressible = new Set(this.getCompressibleMessages(store).map((m) => m.id)); - const summarized = new Set(this.l1ByMessageIndex().keys()); - for (const chunk of this.chunks) if (chunk.summaryId !== undefined) for (const m of chunk.messages) summarized.add(m.id); - const reading = this.selectsPinReading(); - // A pin keeps raw only where the selects resolve it raw: where they read - // bounds, a position a pin holds at a level (this one's own, if it has a - // level or cap) stays held, so a pin with a level or cap frees nothing - // there. - const bounds = this.resolvedPinBounds(messages, reading); - const prices = this.postStripEstimates(store); - const out: Array<{ pin: ProtectedRange; tokens: number }> = []; - for (const pin of this.pins) { - const whole = this.pinPositions(pin, positionOf, messages, reading); - const ids = this.pinSpan(pin, positionOf, messages, 'ids'); - if (!whole || !ids) continue; - let tokens = 0; - for (const i of whole) { - const id = messages[i].id; - const bound = bounds.get(i); - const raw = bound !== undefined && AutobiographicalStrategy.isRawPinBound(bound); - if ((i < ids.lo || i > ids.hi) && raw && compressible.has(id) && summarized.has(id)) tokens += prices[i] ?? 0; - } - if (tokens > 0) out.push({ pin, tokens }); - } - return out.sort((a, b) => b.tokens - a.tokens); - } - - /** Warn, naming each pin `selectNarrowingPins` narrowed, of what it now covers and how to have it whole again. */ - private warnNarrowedPins(narrowed: readonly ProtectedRange[], refusal: OverBudgetError): void { - for (const pin of narrowed) { - const what = pin.kind === 'document' ? 'document mark' : 'pin'; - const label = pin.name !== undefined ? `"${pin.name}" (${pin.id})` : pin.id; - const one = pin.firstMessageId === pin.lastMessageId; - const covers = one - ? `only message ${pin.firstMessageId}, one part of a message stored in parts, not the whole message` - : `only messages ${pin.firstMessageId} to ${pin.lastMessageId}, not the whole of the messages stored in parts at its ends`; - this.diag.warn( - `autobio: ${what} ${label} now covers ${covers}. ${narrowed.length > 1 ? `With the ${narrowed.length} pins narrowed here covering their whole messages` : `With it covering ${one ? 'the whole message' : 'them whole'}`}, ` + - `the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget}` + - `${refusal.diagnostics.configuredBudget === undefined ? '' : ` (maxTokens ${refusal.diagnostics.configuredBudget})`}. Until this ` + - `context is loaded again, the parts it no longer covers fold like any history. Loaded again, as at a restart, the ${what} ` + - `covers ${one ? 'the whole message' : 'them'} again where the budget holds ${one ? 'it' : 'them'}: to have ` + - `${one ? 'it' : 'them'} whole, raise the budget, then restart or unpin ${pin.id} and ` + - `${pin.kind === 'document' ? 'mark the message' : 'pin the range'} again.`, - ); - } - } - - private selectOnce(store: MessageStoreView, budget: TokenBudget, opts?: SelectOptions): ContextEntry[] { // Image stripping runs inside each select path (before stats commit / cache // markers), so the returned entries are already bounded — see // applyImageStripping. @@ -8215,8 +7990,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // A provisional multiplier above 1 that puts the compile over budget // is dropped to 1 and the select runs again: with the refusal // standing, no inference would run, so no sample could ever correct - // it (#179, the startup wedge). Any other refusal stands here, for - // selectNarrowingPins to answer. + // it (#179, the startup wedge). Any other refusal stands. if (!(err instanceof OverBudgetError) || !this.dropProvisionalCalibration(err, opts?.dryRun === true)) throw err; // The retry answers for the select: a preview the discarded attempt // recorded mustn't stand in for a retry that refuses before recording @@ -9434,9 +9208,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { // optimization that scales catastrophically: a 4000-message import // converged to ~500 L1s that never aged out, blowing the 200k // window around chunk 118. - // Pins in the canonical L1 prompt, read by their ids (PROMPT_PIN_READING, - // forceRawPinPositions): a marked message's other parts reach the prompt - // as their summaries, though the selects send them raw. A + // Pins in the canonical L1 prompt. The live view shows a message that a + // pin keeps raw (forceRawPinPositions) raw, so this prompt does too. A // frontier summary that can't be shown as a recall pair (it covers such a // message, or its content is empty) is DESCENDED, not dropped: its // children take its place, so only the branch holding the pin opens, @@ -9579,7 +9352,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // order, after a head-window reset or an unpin) isn't recalled at all // (layoutHistory), and its messages stay covered. const forceRawMessageIds = new Set(); - for (const i of this.forceRawPinPositions(allMessages, AutobiographicalStrategy.PROMPT_PIN_READING)) { + for (const i of this.forceRawPinPositions(allMessages)) { if (i < chunkStartIdx) forceRawMessageIds.add(allMessages[i].id); } const recallBudget = sourceOnly ? 0 : (this.config.compressionRecallBudgetTokens ?? 100_000); @@ -10094,16 +9867,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { // The pins that keep one of those messages raw: a pin whose own bound // reads as raw here, over a message the pins keep raw. const pinnedRawSet = new Set(pinned.pinnedRaw); - const forceRawPositions = this.forceRawPinPositions(allMessages, AutobiographicalStrategy.PROMPT_PIN_READING); + const forceRawPositions = this.forceRawPinPositions(allMessages); const readsBounds = this.readsPinBounds(); const positionOf = this.positionIndex(allMessages); const pins = this.pins.filter((pin) => { const own: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; if (readsBounds && !AutobiographicalStrategy.isRawPinBound(own)) return false; - const span = this.pinSpan(pin, positionOf, allMessages, AutobiographicalStrategy.PROMPT_PIN_READING); - if (!span) return false; - for (let i = span.lo; i <= span.hi; i++) { + for (const i of this.pinPositions(pin, positionOf, allMessages) ?? []) { if (pinnedRawSet.has(i) && forceRawPositions.has(i)) return true; } return false; @@ -10115,7 +9886,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { `autobio: L1 compression of ${chunk.messages.length} message(s) from ${chunkFirstId} recalls the summaries ` + `its pins override: the pinned history it would show raw (~${pinned.pinnedRawTokens} tokens in ` + `${pinned.pinnedRaw.length} message(s); pins: ${pins.join(', ') || 'none found'}) ${limit}. ` + - `Narrow the pins or raise that budget to show them raw here.`, + `Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the ` + + `rest of that message too), or raise that budget to show them raw here.`, ); logCompressionCall({ event: 'compression:pinned-raw-over-budget', @@ -12991,11 +12763,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { within?: ReadonlySet, ): boolean { // Work frees room only where its summary folds a message the selection - // can fold: both selectors keep the head and recent windows raw, and a - // message a pin keeps raw. That's the chunker's own reading of what's - // compressible (getCompressibleMessages), which counts the parts a pin - // covers only as the selects read it: their summaries are what a - // narrowing sends (selectNarrowingPins). + // can fold now: both selectors keep the head and recent windows raw, and + // a message a pin keeps raw. That's the chunker's own reading of what's + // compressible (getCompressibleMessages). const foldable = new Set( this.getCompressibleMessages(store).map((m) => m.id).filter((id) => within === undefined || within.has(id)), ); @@ -13143,7 +12913,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const all = store.getAll(); const sourceOnly = this.config.compressionSourceOnly === true; const position = new Map(all.map((message, i) => [message.id, i])); - const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all, AutobiographicalStrategy.PROMPT_PIN_READING)].sort((a, b) => a - b); + const pins = sourceOnly ? [] : [...this.forceRawPinPositions(all)].sort((a, b) => a - b); // Built at the first head read, so a count that reads none (source-only, // or each chunk holding a tool block itself) scans no holds. let heads: ((pos: number) => { start: number; end: number }) | undefined; @@ -13227,6 +12997,54 @@ export class AutobiographicalStrategy implements ResettableStrategy { }); } + /** + * The pins that keep messages raw in the adaptive plan's middle, for its + * refusal to name (`OverBudgetDiagnostics.rawPins`): the positions the plan + * folds history in (before the head, and from the head's end to the tail's + * start, which is the recent window's start or the head's end where the + * recent window reaches back into the head) that pins keep raw + * (`forceRawPinPositions`), each priced by `price`, the plan's own raw + * price. A position locked raw with `lockChunk` stays raw without its pin, + * so it isn't counted. Where bounds are read, a pin whose own bound isn't + * raw keeps nothing raw, so it isn't named even where it overlaps a raw + * pin; where they aren't, every pin renders raw. Undefined when no pin + * keeps any. + */ + private rawPinsDiagnostics(store: MessageStoreView, price: (position: number) => number): OverBudgetDiagnostics['rawPins'] { + if (this.pins.length === 0) return undefined; + const messages = store.getAll(); + const raw = this.forceRawPinPositions(messages); + const headStart = this.getHeadWindowStartIndex(store); + const headEnd = this.getHeadWindowEnd(store); + const tailStart = Math.max(this.getRecentWindowStart(store), headEnd); + const lockedRaw = (id: MessageId): boolean => this.locked.has(id) && (this.resolutions.get(id) ?? 0) === 0; + const kept = (i: number): boolean => raw.has(i) && (i < headStart || (i >= headEnd && i < tailStart)) + && !lockedRaw(messages[i]!.id); + let tokens = 0; + for (const i of raw) if (kept(i)) tokens += price(i); + if (tokens === 0) return undefined; + const positionOf = this.positionIndex(messages); + const readsBounds = this.readsPinBounds(); + const pins: OverBudgetRawPin[] = []; + for (const pin of this.pins) { + const bound: ResolvedPinBound = pin.level !== undefined ? { level: pin.level } + : pin.maxLevel !== undefined ? { maxLevel: pin.maxLevel } : 'raw'; + if (readsBounds && !AutobiographicalStrategy.isRawPinBound(bound)) continue; + const at = this.pinPositions(pin, positionOf, messages); + if (!at) continue; + let own = 0; + for (const i of at) if (kept(i)) own += price(i); + if (own === 0) continue; + const named = Math.abs(positionOf.get(pin.lastMessageId)! - positionOf.get(pin.firstMessageId)!) + 1; + pins.push({ + pinId: pin.id, kind: pin.kind, ...(pin.name === undefined ? {} : { name: pin.name }), + firstMessageId: pin.firstMessageId, lastMessageId: pin.lastMessageId, + ...(at.length > named ? { parts: at.length } : {}), tokens: own, + }); + } + return { tokens, pins }; + } + // ============================================================================ // Adaptive resolution (picker-driven) path // ============================================================================ @@ -13267,6 +13085,14 @@ export class AutobiographicalStrategy implements ResettableStrategy { // apart from the exceptions applyImageStripping's asEstimated names. const pse = this.postStripEstimates(store); if (_diag) { this.diag.error(`[cm-cache] selectAdaptive: pse ${Date.now() - _t}ms`); _t = Date.now(); } + // A middle message's raw price, as the plan prices it and its refusal + // prices the pins that keep it raw (rawPinsDiagnostics). The shard path + // in step 5 sends a run of a body's raw shards as one entry, their text + // joined and uncut, so a shard takes no extra. + const middlePrice = (i: number): number => { + const msg = messages[i]!; + return msg.bodyGroupId ? pse[i]! : pse[i]! + this.cappedForSend(store, msg, msgCap).extra; + }; // ----- 1. Build head/tail sets and reserve the tail before emitting ----- const headStart = this.getHeadWindowStartIndex(store); @@ -13386,15 +13212,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { } // Each pinned position's fold bound, resolved once (resolvedPinBounds), - // so the picker doesn't fold what the user marked keep-raw, read as the - // selects read pins (selectsPinReading). A chunk carries at most one + // so the picker doesn't fold what the user marked keep-raw and every + // selector reads a pin as compression does. A chunk carries at most one // of `pinned` (raw: a classic pin, or a level or cap of 0), `pinLevel` or // `pinMaxLevel`, so every reader of `pinned` agrees on what stays raw. // The last two appear only under the strategies that honor them // (kv-stable, kv-unified), which must be able to move the chunk to or // within its bound; elsewhere a bounded pin resolves to `pinned`. Built // once and reused. - const pinBounds = this.resolvedPinBounds(messages, this.selectsPinReading()); + const pinBounds = this.resolvedPinBounds(messages); // O(1) summary lookup for findAncestorAt — avoids O(summaries) find() // calls during emission. @@ -13429,15 +13255,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (let i = segStart; i < segEnd && i < messages.length; i++) { const msg = messages[i]; const ch = chunkAtPosition ? chunkAtPosition[i] : chunksByMessageId.get(msg.id); - // This is the chunk's raw price. The shard path in step 5 sends a run - // of a body's raw shards as one entry, their text joined and uncut, so - // a shard takes no extra here. It prices that entry as the shards' - // text it joins, each by estimateTokens, the store's price, so a shard - // of text alone is counted there as here; a shard's other blocks are - // priced here, and the shard path leaves them out. A shard the plan - // folds is priced as its summary. The head and recent windows cut a - // shard as they emit it. - const tokens = msg.bodyGroupId ? pse[i] : pse[i] + this.cappedForSend(store, msg, msgCap).extra; + // This is the chunk's raw price (middlePrice). The shard path in step 5 + // prices its joined entry as the shards' text it joins, each by + // estimateTokens, the store's price, so a shard of text alone is + // counted there as here; a shard's other blocks are priced here, and + // the shard path leaves them out. A shard the plan folds is priced as + // its summary. The head and recent windows cut a shard as they emit it. + const tokens = middlePrice(i); const bound = pinBounds.size > 0 ? pinBounds.get(i) : undefined; const l1Id = ch?.summaryId ?? l1ByMessage.get(msg.id); pickerChunks.push({ @@ -13810,6 +13634,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // real compile will do. If the retry at 1 is still over, it reports. const dryFallback = dryRun && this._calibrationProvisional !== null && this._calibration > 1; if (!planFits && (!dryRun || dryFallback)) { + // The plan's middle is what's over, priced as the plan priced it, so + // the pins keeping raw tokens there are named. The refusals after the + // plan count other things (an emitted prefix, a shard's text alone) + // and don't name pins. + const rawPins = this.rawPinsDiagnostics(store, middlePrice); throw this.overBudgetError(budget, { budget: rejectionBudget, actual: result.finalTokens, @@ -13822,6 +13651,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { middleTokens: Math.max(0, result.finalTokens - headTokens - tailTokens), middleChunkCount: pickerChunks.length - headMessageIds.size - tailMessageIds.size, deepestLevel, + ...(rawPins === undefined ? {} : { rawPins }), }, }); } @@ -16103,7 +15933,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { const maxTokens = budget.maxTokens - budget.reserveForResponse; const messages = store.getAll(); const msgCap = this.config.maxMessageTokens; - // Emission grace (coverage invariant, 76e95a0): selection below still // TARGETS maxTokens, but emission never silently drops content to fit — // it may overshoot up to the grace window, and beyond that the select @@ -16344,7 +16173,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const positionOf = this.positionIndex(messages); // Without adaptive resolution no selector reads a pin's level bound, so // every pinned position resolves to raw (resolvedPinBounds). - const pinnedPositionsSet = this.forceRawPinPositions(messages, this.selectsPinReading()); + const pinnedPositionsSet = this.forceRawPinPositions(messages); // Pinned messages between head and recent (head/recent pinned ones // already emit raw via Phase 0 / Phase 4). const pinnedInMiddle: { msg: StoredMessage; position: number }[] = []; @@ -17125,34 +16954,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { // Shared utilities // ============================================================================ - /** - * The positions compression leaves out for pins: raw both by the pins' ids - * and as the selects read them (`forceRawPinPositions`, `'ids'` and - * `selectsPinReading`). A part a pin's ids name that another pin's whole - * reading holds at a level isn't one, since the selects need its summary, - * and a part a pin covers only as the selects read it isn't either: its - * summary is what a narrowing sends. No chunk made while a position is in - * this set takes it (a chunk recorded before keeps its messages), so what - * relies on that reads it. - */ - protected compressionPinnedPositions(messages: StoredMessage[]): Set { - // Where the selects read pins by their ids too, the two sets are one. - if (this.selectsPinReading() === 'ids') return this.forceRawPinPositions(messages, 'ids'); - const rawAsSelected = this.forceRawPinPositions(messages, this.selectsPinReading()); - const out = new Set(); - for (const i of this.forceRawPinPositions(messages, 'ids')) if (rawAsSelected.has(i)) out.add(i); - return out; - } - /** * Get messages in the compressible zone: outside both head window and * recent window AND not kept raw by a pin. Returns messages from * [0, headStart) ∪ [headEnd, recentStart) minus any position a pin or - * document mark keeps raw both by its ids and as the selects read it - * (compressionPinnedPositions): of a message stored in parts, only the - * parts a pin's ids name stay out, though the selects send the whole - * message raw, and a part another pin's whole reading holds at a level - * comes in for its summary. A pin with a positive + * document mark keeps raw (forceRawPinPositions). A pin with a positive * level or cap is compressible where bounds are read (kv-stable and * kv-unified, with adaptive resolution: readsPinBounds): it renders at a * level, which only a summary can supply. @@ -17162,7 +16968,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); const recentStart = this.getRecentWindowStart(store); - const pinned = this.compressionPinnedPositions(messages); + const pinned = this.forceRawPinPositions(messages); const out: StoredMessage[] = []; for (let i = 0; i < recentStart; i++) { if (i >= headStart && i < headEnd) continue; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index a3d6b1f2..e468ae83 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -2085,24 +2085,14 @@ export function getSummaryParentId(s: SummaryEntry): string | undefined { * bound holds it (see `level`), so a raw pin there might not keep it raw. * * A message stored in parts (a long message that ingress splits under - * `adaptiveResolution`) counts as one where a pin is rendered: under adaptive - * resolution, an end inside one covers it from its first part to its last in - * the selects, the agent's view, less a part locked with `lockChunk` that the - * pin's ids don't name, where the lock holds. The summarizer's prompt reads a pin by its - * ids, as compression does, so a raw pin keeps out of compression only the - * parts its ids name (a part chunked before the pin was made stays in its - * chunk, as before), and the message's other parts are chunked and - * summarized like any history. A summary whose chunk runs past the message's - * edge goes out beside the raw message, so some of the message is sent twice. - * A pin with a fixed level reads whole too: under kv-unified, as for any - * pinned level, compiles fail until each part of the message in the middle - * (outside the head and recent windows) has the summary that level needs. - * When a compile can't hold the whole messages within its budget but can - * hold the rest with the pin narrowed to its ids, it narrows a pin it renders - * raw (one held at a level frees nothing narrowed) until the strategy loads - * the store again (an open, a branch switch, a fork or `setStrategy`), and - * warns. Nothing records the narrowing, so a store loaded again reads the - * pin whole. + * `adaptiveResolution`) counts as one: under adaptive resolution, a pin + * whose first or last id names one of its parts covers it from its first + * part to its last, less a part locked with `lockChunk` outside the span + * its ids name, where the lock holds. The selects, compression and the + * summarizer's prompt all read it so. A compile whose budget can't hold the + * raw pins beside the rest refuses; where the adaptive plan's middle is over + * the budget, the refusal names each pin keeping raw tokens in that middle + * (`OverBudgetDiagnostics.rawPins`). * * - `kind: 'pin'` — generic protected range (any first/last span). * - `kind: 'document'` — typically a single message containing a body of @@ -2140,7 +2130,7 @@ export interface ProtectedRange { * conflict, and every compile fails until one does. Under * any other selector, and without adaptive resolution, the range is raw (a * safe superset). The canonical L1 compression prompt shows raw what the - * configured selector keeps raw, reading pins by their ids, where its budgets allow (see + * configured selector keeps raw, where its budgets allow (see * `compressionRecallBudgetTokens`); it doesn't hold a span at its pinned * level. Where pins overlap at a message, a bound takes a classic pin's * place there, `level` takes `maxLevel`'s, and the lowest of each kind diff --git a/test/l1-recall-raw-pins-frontier.test.ts b/test/l1-recall-raw-pins-frontier.test.ts index 8f6b130f..d5924be5 100644 --- a/test/l1-recall-raw-pins-frontier.test.ts +++ b/test/l1-recall-raw-pins-frontier.test.ts @@ -8,8 +8,7 @@ * window too, in source order with the recall pairs; nothing else loses * coverage; the raw pins add is spent from the recall budget, and a * compression whose pinned raw can't fit it recalls what the pins override, - * and says so. A pin's bounds resolve by one rule, for compression and every - * selector, in the reading each takes (resolvedPinBounds). + * and says so. A pin resolves once, for compression and every selector. */ import { describe, it, after } from 'node:test'; import assert from 'node:assert/strict'; @@ -837,7 +836,7 @@ describe('the raw pins add is spent from the recall budget', () => { }); }); -describe('a pin\'s bounds resolve by one rule, for compression and every selector', () => { +describe('a pin resolves once, for compression and every selector', () => { const messages = Array.from({ length: 11 }, (_, i) => ({ id: `m${i}` })); const pins: Array & { at: number }> = [ { at: 1 }, // classic raw pin @@ -857,10 +856,10 @@ describe('a pin\'s bounds resolve by one rule, for compression and every selecto id: `pin-${n}`, firstMessageId: `m${at}`, lastMessageId: `m${at}`, kind: 'pin', created: 0, ...bound, })); const internals = strategy as unknown as { - resolvedPinBounds(m: typeof messages, reading: 'ids' | 'whole'): Map; - forceRawPinPositions(m: typeof messages, reading: 'ids' | 'whole'): Set; + resolvedPinBounds(m: typeof messages): Map; + forceRawPinPositions(m: typeof messages): Set; }; - return { bounds: internals.resolvedPinBounds(messages, 'whole'), raw: internals.forceRawPinPositions(messages, 'whole') }; + return { bounds: internals.resolvedPinBounds(messages), raw: internals.forceRawPinPositions(messages) }; }; for (const foldingStrategy of ['kv-stable', 'kv-unified'] as const) { diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index e1e3c9bf..ccc2a429 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -1,20 +1,20 @@ /** - * A pin renders whole messages: a pin or document mark whose first or last - * id names a part of a message stored in parts covers that message from its - * first part to its last in the selects, while the summarizer's prompt and - * compression read it by its ids, so the message's other parts are chunked - * and summarized like any history (`pinSpan`). A compile refused - * over budget narrows pins back to their own ids, the one that frees the - * most first, one at a time, until a select goes ahead, and keeps them so - * while the strategy instance holds them; otherwise it puts them back - * (`selectNarrowingPins`). + * A pin covers whole messages: under adaptive resolution, a pin or document + * mark whose first or last id names a part of a message stored in parts + * covers that message from its first part to its last, for every reader of + * pins, the selects, compression and the summarizer's prompt alike, as a + * pinRange over all its parts does where no part is locked (`pinPositions`): + * a part locked with `lockChunk` outside the span the pin's ids name keeps + * its lock. A compile that can't hold a whole message refuses, as for any + * pin the budget can't hold, and the adaptive plan's refusal names the pin. * * The store: six history messages, then long messages that ingress stores * in parts (adaptive resolution with 1,500-token chunks), then 360 messages * of traffic. A head window of 500 tokens and a recent window of 8,000 leave * the long messages in the middle. With one 6,000-word message marked, its * first part alone fits at 22,000 tokens and the whole message needs about - * 38,000; the tests use 30,000 for "can't be whole" and 42,000 for "fits". + * 38,000; the tests use 30,000 for "can't be whole" and 42,000 or 60,000 for + * "fits". */ import { describe, it } from 'node:test'; @@ -42,33 +42,13 @@ const KV_UNIFIED = { /** Reaches the strategy's pins and windows. */ class PinStore extends AutobiographicalStrategy { - private pinById(pinId: string) { - return this.pins.find((p) => p.id === pinId)!; - } - /** Narrow a pin to its ids, or put it back, as a compile does: the request an earlier version sent. */ - setNarrowed(pinId: string, narrowed: boolean): void { - if (narrowed) this.narrowedPins.add(this.narrowKey(this.pinById(pinId))); - else this.narrowedPins.delete(this.narrowKey(this.pinById(pinId))); - } - /** Whether a compile narrowed this pin. */ - isNarrowed(pinId: string): boolean { - return this.narrowedPins.has(this.narrowKey(this.pinById(pinId))); - } - /** Read the store's state again, as this instance does when another instance on the store has written it. */ - reload(): void { - this.loadPersistedState({ keepKvUnifiedPresentation: true }); - } - /** The fold bound each pinned position resolves to, as every selector reads it. */ + /** The fold bound each pinned position resolves to, as every reader of pins takes it. */ boundsAt(messages: unknown[]): Map { - return this.resolvedPinBounds(messages as never, 'whole'); - } - /** The positions pins cover in a reading, as a caller passes it, from JavaScript too. */ - pinnedWith(messages: unknown[], reading: unknown): Set { - return this.pinnedPositions(messages as never, reading as never); + return this.resolvedPinBounds(messages as never); } - /** The level bounds pins give in a reading, as the replay exporter reads them. */ - levelBoundsWith(messages: unknown[], reading: 'ids' | 'whole'): Map { - return this.pinLevelBounds(messages as never, reading); + /** The level bounds pins give, as the replay exporter reads them. */ + levelBounds(messages: unknown[]): Map { + return this.pinLevelBounds(messages as never); } /** The ids of the messages in compressed chunks. */ summarizedIds(): Set { @@ -82,73 +62,21 @@ class PinStore extends AutobiographicalStrategy { for (const chunk of this.chunks) for (const m of chunk.messages) out.add(m.id); return out; } - /** Where the head window starts, and where the recent window starts. */ - windows(store: never): { headStart: number; recentStart: number } { - return { headStart: this.getHeadWindowStartIndex(store), recentStart: this.getRecentWindowStart(store) }; - } /** Where the head window ends. */ headEnd(store: never): number { return this.getHeadWindowEnd(store); } - /** How many adaptive selects ran, and the refusals they threw. */ - selects = 0; - refusals: OverBudgetError[] = []; - /** Fail a select that a narrowed pin let through, after it planned. */ - failNarrowedAfterPlanning = false; - protected selectAdaptive(store: never, budget: never, opts?: never): never { - this.selects++; - let entries: never; - try { - entries = super.selectAdaptive(store, budget, opts) as never; - } catch (err) { - if (err instanceof OverBudgetError) this.refusals.push(err); - throw err; - } - if (this.failNarrowedAfterPlanning && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) { - // As a narrowed select's plan can leave a transition status before a later failure. - (this as unknown as { transitionBlocked: string }).transitionBlocked = 'transition-pace-floor'; - throw new Error('select failed'); - } - return entries; - } - /** - * Leave the L1s covering these messages as the only record of their - * coverage, as an adopted summary can: their chunks lose the pointer to it - * and read as not compressed. - */ - dropChunkPointers(ids: readonly string[]): void { - const of = new Set(ids); - for (const record of this.chunkRecords) { - if (!record.sourceIds.some((id) => of.has(id))) continue; - record.summaryId = undefined; - record.compressed = false; - } - } /** The stored resolution of a message (0 is raw). */ resolutionOf(id: string): number { return this.resolutions.get(id) ?? 0; } - /** The id a pin's range starts at. */ - firstIdOf(pinId: string): string { - return this.pinById(pinId).firstMessageId; - } - /** Persist a provisional calibration multiplier, as a restart no sample has confirmed leaves one. */ - saveProvisionalCalibration(multiplier: number): void { - const epoch = (AutobiographicalStrategy as unknown as { CALIBRATION_PRICING_EPOCH: number }).CALIBRATION_PRICING_EPOCH; - this.store!.setStateJson(this.calibrationStateId, { multiplier, at: 1, pricing: epoch, provisional: true }); - } - /** Fail a select with an error that isn't a refusal while a pin is narrowed. */ - failWhileNarrowed = false; - protected resolvedPinBounds(messages: never, reading: 'ids' | 'whole'): never { - if (this.failWhileNarrowed && this.pins.some((p) => this.narrowedPins.has(this.narrowKey(p)))) throw new Error('select failed'); - return super.resolvedPinBounds(messages, reading) as never; - } } -type Kind = 'adaptive' | 'kv-unified'; +type Kind = 'adaptive' | 'kv-stable' | 'kv-unified'; const config = (kind: Kind, recentWindowTokens: number): AutobiographicalConfig => ({ headWindowTokens: 500, recentWindowTokens, imageStripDepthTokens: 0, compressionModel: 'test-model', hierarchical: false, adaptiveResolution: true, targetChunkTokens: 1500, + ...(kind === 'kv-stable' ? { foldingStrategy: 'kv-stable' } : {}), ...(kind === 'kv-unified' ? { foldingStrategy: 'kv-unified', kvUnified: { ...KV_UNIFIED, policy: { ...KV_UNIFIED.policy } } } : {}), } as AutobiographicalConfig); @@ -183,7 +111,10 @@ function rawWords(messages: Sent[], label: string): number[] { .sort((a, b) => a - b); } const allOf = (n: number): number[] => Array.from({ length: n }, (_, i) => i); -const ready = async (cm: ContextManager) => { for (let i = 0; i < 800 && !cm.isReady(); i++) await cm.tick(); }; +const ready = async (cm: ContextManager) => { + for (let i = 0; i < 800 && !cm.isReady(); i++) await cm.tick(); + assert.ok(cm.isReady(), 'compression caught up'); +}; async function history(cm: ContextManager): Promise { for (let i = 0; i < 6; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`hist${i}`, 200)); @@ -193,34 +124,11 @@ async function traffic(cm: ContextManager, n: number, label = 'event'): Promise< for (let i = 0; i < n; i++) cm.addMessage(i % 2 ? 'assistant' : 'user', words(`${label}${i}`, 200)); } -/** - * A store as an earlier version left it: the marks on long messages covered - * their first part only, so the rest of each was chunked and summarized, as - * it still is. Returns the request it sent then, and the marks read whole - * again, as the upgraded store reads them. - */ -async function upgradedStore(dir: string, opts: { kind?: Kind; bodies: Array<[string, number, string?]>; budget: number }) { - const s = await open(dir, opts.kind); - await history(s.cm); - const marks: string[] = []; - for (const [label, n, name] of opts.bodies) { - const mark = s.cm.markDocument(s.cm.addMessage('user', words(label, n)), name === undefined ? undefined : { name }); - s.strategy.setNarrowed(mark, true); - marks.push(mark); - } - await traffic(s.cm, 360); - await ready(s.cm); - const before = sent(await s.cm.compile(budget(opts.budget))); - for (const mark of marks) s.strategy.setNarrowed(mark, false); - return { ...s, marks, before }; -} - function warnings(t: { mock: { method: (o: object, m: string, f: (...a: unknown[]) => void) => unknown } }): string[] { const out: string[] = []; t.mock.method(console, 'warn', (...args: unknown[]) => { out.push(String(args[0])); }); return out; } -const narrowingWarnings = (out: string[]) => out.filter((w) => w.includes(' now covers only ')); type View = { getAll(): Array<{ id: string; bodyGroupId?: string }> }; const viewOf = (cm: ContextManager): View => (cm as unknown as { strategyMessageView(): View }).strategyMessageView(); @@ -244,26 +152,67 @@ async function withDir(body: (dir: string) => Promise): Promise { } } -describe('pins render whole messages', () => { +/** A store where a long message arrives, is chunked and summarized, and then is marked, as a store an earlier version marked reads now. */ +async function markedAfterSummaries(dir: string, kind: Kind = 'adaptive') { + const s = await open(dir, kind); + await history(s.cm); + const b = s.cm.addMessage('user', words('Bdoc', 6000)); + await traffic(s.cm, 360); + await ready(s.cm); + await s.cm.compile(budget(30_000)); + const parts = partsOf(s.cm, b); + assert.ok(parts.slice(1).every((id) => s.strategy.summarizedIds().has(id)), 'the later parts are summarized'); + return { ...s, b, parts }; +} + +/** The compile's refusal, or a failure when it compiles. */ +async function refusalOf(cm: ContextManager, maxTokens: number): Promise { + try { + await cm.compile(budget(maxTokens)); + } catch (err) { + if (err instanceof OverBudgetError) return err; + throw err; + } + assert.fail(`the compile at ${maxTokens} went ahead`); +} + +describe('a pin covers whole messages, for every reader', () => { for (const kind of ['adaptive', 'kv-unified'] as const) { - it(`${kind}: a document mark on a long message's first part sends the whole message raw, while compression summarizes its other parts`, async () => { - await withDir(async (dir) => { - // A recent window of 20,000 holds the whole message as it arrives, so the mark is there before - // compression chunks any part of it. - const { cm, strategy } = await open(dir, kind, 20_000); - await history(cm); - const b = cm.addMessage('user', words('Bdoc', 6000)); - assert.deepStrictEqual(partsOf(cm, b).filter((id) => strategy.chunkedIds().has(id)), [], 'no part is chunked yet'); - cm.markDocument(b); - await traffic(cm, 360); - await ready(cm); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); - const parts = partsOf(cm, b); - assert.ok(parts.length > 2, 'the message is stored in parts'); - const summarized = strategy.summarizedIds(); - assert.deepStrictEqual(parts.map((id) => summarized.has(id)), parts.map((_, i) => i > 0), 'every part but the marked one is summarized'); - await cm.close(); - }); + it(`${kind}: a document mark on a long message's first part is read as a range pin over all its parts: the same request, chunks and summarizer input`, async () => { + const run = async (how: 'mark' | 'range') => { + let out: { request: string[]; raw: number[]; chunked: boolean[]; prompts: Array<[boolean, boolean]> } | undefined; + await withDir(async (dir) => { + // A recent window of 20,000 holds the whole message as it arrives, so the pin is there before + // compression chunks any part of it. + const { cm, strategy, requests } = await open(dir, kind, 20_000); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const parts = partsOf(cm, b); + assert.ok(parts.length > 2, 'the message is stored in parts'); + assert.deepStrictEqual(parts.filter((id) => strategy.chunkedIds().has(id)), [], 'no part is chunked yet'); + if (how === 'mark') cm.markDocument(b); + else cm.pinRange(parts[0]!, parts.at(-1)!); + await traffic(cm, 360); + await ready(cm); + const request = sent(await cm.compile(budget(60_000))); + // The summarizer's requests for chunks long after the message: does each carry its first and last words raw? + const later = requests.filter((r) => r.includes('"event300 word0')); + assert.ok(later.length > 0, 'the chunk was summarized'); + out = { + request: request.map(textOf), raw: rawWords(request, 'Bdoc'), + chunked: parts.map((id) => strategy.chunkedIds().has(id)), + prompts: later.map((r) => [r.includes('Bdoc word0 word1'), r.includes('word5999')]), + }; + await cm.close(); + }); + return out!; + }; + const mark = await run('mark'); + const range = await run('range'); + assert.deepStrictEqual(mark.raw, allOf(6000), 'the whole message goes out raw'); + assert.deepStrictEqual(mark.chunked, mark.chunked.map(() => false), 'compression leaves every part out'); + assert.ok(mark.prompts.every(([first, last]) => first && last), 'the summarizer\'s prompt carries the whole message raw'); + assert.deepStrictEqual(mark, range); }); } @@ -282,6 +231,41 @@ describe('pins render whole messages', () => { }); }); + it('a mark on a long message whose other parts were summarized before it, as an earlier version left one, sends the whole message where it fits', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir); + assert.ok(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc').length < 6000, 'before the mark, the later parts fold'); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + + it('a range pin given its ends in reverse order covers the same whole message', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'kv-unified'); + await history(cm); + const parts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); + cm.pinRange(parts[3]!, parts[1]!); + const all = viewOf(cm).getAll(); + const bounds = strategy.boundsAt(all); + const of = new Set(parts); + assert.deepStrictEqual(all.flatMap((m, i) => (bounds.has(i) ? [m.id] : [])), all.filter((m) => of.has(m.id)).map((m) => m.id)); + await cm.close(); + }); + }); + + it('a range pin over a part locked at a fold between its ids renders it raw over the lock, as before', async () => { + await withDir(async (dir) => { + const { cm, strategy, parts } = await markedAfterSummaries(dir); + strategy.lockChunk(parts[3]!); + assert.ok(strategy.resolutionOf(parts[3]!) > 0, 'locked at a fold'); + cm.pinRange(parts[1]!, parts[5]!); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), allOf(6000)); + await cm.close(); + }); + }); + it('a range pin whose ends fall inside long messages covers both of them whole', async () => { await withDir(async (dir) => { const { cm } = await open(dir); @@ -335,77 +319,35 @@ describe('pins render whole messages', () => { }); }); - it('a compression request after the mark carries the marked part raw and the other parts as summaries, as main gives it', async () => { - await withDir(async (dir) => { - const { cm, requests } = await open(dir); - await history(cm); - cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - await traffic(cm, 360); - await ready(cm); - // A request whose chunk holds traffic message 300, long after the marked message. - const later = requests.filter((r) => r.includes('"event300 word0')); - assert.ok(later.length > 0, 'the chunk was summarized'); - for (const r of later) { - assert.ok(r.includes('Bdoc word0 word1'), 'the marked part is in the prompt raw'); - assert.ok(!r.includes('word5999'), 'the message\'s last part is not'); - } - await cm.close(); - }); - }); - - it('the summarizer\'s over-budget warning doesn\'t count a mark\'s later parts, which its prompt sends as summaries', async (t) => { + it('the summarizer\'s over-budget warning counts a mark\'s whole message and names the mark', async (t) => { await withDir(async (dir) => { const out = warnings(t); - // A head window of 3,000 holds the message's first part (about 2,070 tokens), where the mark's own - // id is. Read whole, its later parts would be raw history ahead of the traffic's chunks, more than - // the summarizer's recall budget of 2,000. + // A head window of 3,000 holds the message's first part (about 2,070 tokens), where the mark's own id is. + // Its later parts are raw history ahead of the traffic's chunks, more than the summarizer's recall budget of 2,000. const { cm, strategy, calls } = await open(dir, 'adaptive', 8000, { headWindowTokens: 3000, compressionRecallBudgetTokens: 2000 }); - cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); + cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { name: 'P' }); await traffic(cm, 120); await ready(cm); assert.strictEqual(strategy.headEnd(viewOf(cm) as never), 1, 'the head holds the first part alone'); assert.ok(calls() > 0, 'the summarizer was called'); - assert.deepStrictEqual(out.filter((w) => w.includes('recalls the summaries its pins override')), []); + const over = out.filter((w) => w.includes('recalls the summaries its pins override')); + assert.ok(over.length > 0, 'the warning was given'); + for (const w of over) { + assert.ok(w.includes('pins: P)'), w); + assert.ok(w.includes('Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the rest of that message too)'), w); + } await cm.close(); }); }); - for (const kind of ['adaptive', 'kv-unified'] as const) { - it(`${kind}: the summarizer's over-budget warning names the pins that keep its raw history raw, read by their ids as its prompt reads them`, async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - // The summarizer shows a pinned message raw where a summary would cover it. Under kv-unified, Q's whole - // reading holds P's part at a level, so it's summarized; a recent window of 20,000 holds the message as it - // arrives, so both pins are there before any part is chunked. Elsewhere, a recent window of 8,000 lets the - // message's early parts be chunked as they arrive, before the pins. - const { cm } = await open(dir, kind, kind === 'kv-unified' ? 20_000 : 8000, { compressionRecallBudgetTokens: 1000 }); - await history(cm); - const b = cm.addMessage('user', words('Bdoc', 6000)); - const parts = partsOf(cm, b); - // P keeps the first part raw, about 2,070 tokens, over the recall budget of 1,000. Q names the sixth part: - // raw where no bounds are read, and at level 1 under kv-unified, where read whole it would hold the first - // part at that level in P's place. - cm.markDocument(b, { name: 'P' }); - cm.pinRange(parts[5]!, parts[5]!, kind === 'kv-unified' ? { name: 'Q', level: 1 } : { name: 'Q' }); - await traffic(cm, 360); - await ready(cm); - const over = out.filter((w) => w.includes('recalls the summaries its pins override')); - assert.ok(over.length > 0, 'the warning was given'); - // Only P's part is the prompt's pinned raw history: Q's whole reading would reach it, its ids don't. - for (const w of over) assert.ok(w.includes('pins: P)'), w); - await cm.close(); - }); - }); - } - it('a store opened without adaptive resolution reads a mark by its ids, as before', async () => { await withDir(async (dir) => { const first = await open(dir); await history(first.cm); const b = first.cm.addMessage('user', words('Bdoc', 6000)); - first.cm.markDocument(b); await traffic(first.cm, 360); await ready(first.cm); + first.cm.markDocument(b); const summarized = first.strategy.summarizedIds(); assert.ok(partsOf(first.cm, b).slice(1).every((id) => summarized.has(id)), 'the later parts are summarized'); await first.cm.close(); @@ -416,156 +358,28 @@ describe('pins render whole messages', () => { }); }); -describe('a compile that can\'t hold the whole messages narrows pins', () => { +describe('a lock holds against a pin\'s whole reading', () => { for (const kind of ['adaptive', 'kv-unified'] as const) { - it(`${kind}: an upgraded store whose mark can't be whole narrows it in memory and sends the request it sent before`, async (t) => { + it(`${kind}: parts locked at a fold after the part a mark names keep their lock`, async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { kind, bodies: [['Bdoc', 6000]], budget: 30_000 }); - assert.deepStrictEqual(rawWords(before, 'Bdoc'), allOf(763), 'before, the mark kept its first part raw'); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.ok(strategy.isNarrowed(marks[0]!)); - const said = narrowingWarnings(out); - assert.strictEqual(said.length, 1); - assert.match(said[0]!, new RegExp(`^autobio: document mark ${marks[0]} now covers only message `)); - assert.match(said[0]!, /With it covering the whole message, the compile came to ~\d+ tokens against a hard budget of 30600 \(maxTokens 30000\)\./); - assert.match(said[0]!, /Until this context is loaded again, the parts it no longer covers fold like any history\./); - assert.match(said[0]!, new RegExp(`raise the budget, then restart or unpin ${marks[0]} and mark the message again\\.`)); - // Narrowed, it fits: the next compile sends the same and warns no more. - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.strictEqual(narrowingWarnings(out).length, 1); - await cm.close(); - }); - }); - } - - it('an upgraded store whose mark fits sends the whole message and narrows nothing', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 42_000 }); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); - await cm.close(); - }); - }); - - it('a store opened again reads the mark whole: with room it sends the whole message, without it narrows again', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const first = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - await first.cm.compile(budget(30_000)); - assert.ok(first.strategy.isNarrowed(first.marks[0]!)); - await first.cm.close(); - const roomy = await open(dir); - assert.deepStrictEqual(rawWords(sent(await roomy.cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); - assert.ok(!roomy.strategy.isNarrowed(first.marks[0]!)); - await roomy.cm.close(); - const tight = await open(dir); - assert.notDeepStrictEqual(rawWords(sent(await tight.cm.compile(budget(30_000))), 'Bdoc'), allOf(6000)); - assert.ok(tight.strategy.isNarrowed(first.marks[0]!)); - assert.strictEqual(narrowingWarnings(out).length, 2, 'each instance that narrows warns once'); - await tight.cm.close(); - }); - }); - - it('a narrowing holds when this instance reads the store\'s state again', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - await cm.compile(budget(30_000)); - strategy.reload(); - assert.ok(strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), rawWords(before, 'Bdoc')); - assert.strictEqual(narrowingWarnings(out).length, 1); - await cm.close(); - }); - }); - - it('a narrowing ends when the strategy loads the store again: a fork narrows again, and the same strategy reopened with room sends the whole message', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - await cm.compile(budget(30_000)); - assert.ok(strategy.isNarrowed(marks[0]!)); - await cm.fork(); - assert.ok(!strategy.isNarrowed(marks[0]!), 'the fork loaded the store again'); - await cm.compile(budget(30_000)); - assert.ok(strategy.isNarrowed(marks[0]!), 'and narrowed again'); - assert.strictEqual(narrowingWarnings(out).length, 2); - await cm.close(); - const again = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: summarizer().membrane }); - assert.deepStrictEqual(rawWords(sent(await again.compile(budget(42_000))), 'Bdoc'), allOf(6000)); - assert.ok(!strategy.isNarrowed(marks[0]!)); - await again.close(); - }); - }); - - it('a later narrowing that fails, or that a dry compile makes, leaves an earlier one as it was', async (t) => { - await withDir(async (dir) => { - warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000], ['Cdoc', 1500]], budget: 30_000 }); - strategy.setNarrowed(marks[0]!, true); - // With the first narrowed, the second fits at 25,000 only narrowed, and at 15,000 not even so. - let selects = strategy.selects; - await assert.rejects(cm.compile(budget(15_000)), OverBudgetError); - assert.strictEqual(strategy.selects - selects, 2, 'the refused select and one narrowing the second mark'); - assert.ok(strategy.isNarrowed(marks[0]!), 'the first stays narrowed'); - assert.ok(!strategy.isNarrowed(marks[1]!)); - selects = strategy.selects; - assert.strictEqual(cm.previewContext(budget(25_000))!.fits, true); - assert.strictEqual(strategy.selects - selects, 2, 'the dry select narrowed the second mark'); - assert.ok(strategy.isNarrowed(marks[0]!), 'the first stays narrowed'); - assert.ok(!strategy.isNarrowed(marks[1]!)); - await cm.close(); - }); - }); - - it('a mark whose parts\' L1s only the summaries record, with no chunk pointing to them, is narrowed as the select folds them', async (t) => { - await withDir(async (dir) => { - warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - strategy.dropChunkPointers(partsOf(cm, strategy.firstIdOf(marks[0]!))); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(30_000))), 'Bdoc'), rawWords(before, 'Bdoc')); - assert.ok(strategy.isNarrowed(marks[0]!)); - await cm.close(); - }); - }); - - for (const kind of ['adaptive', 'kv-unified'] as const) { - it(`${kind}: a part locked at a fold keeps its lock, so a mark on another part sends the request it sent before`, async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { kind, bodies: [['Bdoc', 6000]], budget: 30_000 }); - // The earlier version's compile folded the message's later parts; lock them there. - const parts = partsOf(cm, strategy.firstIdOf(marks[0]!)); - for (const id of parts.slice(1)) strategy.lockChunk(id); - assert.ok(parts.slice(1).every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), rawWords(before, 'Bdoc')); - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); + const { cm, strategy, b, parts } = await markedAfterSummaries(dir, kind); + const later = parts.slice(1); + for (const id of later) strategy.lockChunk(id); + assert.ok(later.every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); + cm.markDocument(b); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), wordsOfPart(cm, b), 'only the part the mark names'); await cm.close(); }); }); - } - for (const kind of ['adaptive', 'kv-unified'] as const) { it(`${kind}: parts locked at a fold before the part a mark names keep their lock`, async () => { await withDir(async (dir) => { - const { cm, strategy } = await open(dir, kind); - await history(cm); - const b = cm.addMessage('user', words('Bdoc', 6000)); - await traffic(cm, 360); - await ready(cm); - await cm.compile(budget(30_000)); - const parts = partsOf(cm, b); + const { cm, strategy, parts } = await markedAfterSummaries(dir, kind); const earlier = parts.slice(0, -1); for (const id of earlier) strategy.lockChunk(id); assert.ok(earlier.every((id) => strategy.resolutionOf(id) > 0), 'locked at a fold'); cm.markDocument(parts.at(-1)!); - const last = wordsOfPart(cm, parts.at(-1)!); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), last, 'only the part the mark names'); + assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(60_000))), 'Bdoc'), wordsOfPart(cm, parts.at(-1)!), 'only the part the mark names'); await cm.close(); }); }); @@ -581,33 +395,16 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { cm.pinAtLevel(b, b, 1); const messages = viewOf(cm).getAll(); const at = (id: string) => messages.findIndex((m) => m.id === id); - const bounds = strategy.levelBoundsWith(messages, 'whole'); + const bounds = strategy.levelBounds(messages); assert.deepStrictEqual(parts.filter((id) => bounds.has(at(id))), parts.filter((_, i) => i !== 3)); assert.deepStrictEqual(bounds.get(at(parts[0]!)), { level: 1 }); await cm.close(); }); }); - it('a reading other than ids or whole is refused, as from a JavaScript caller that leaves it out', async () => { - await withDir(async (dir) => { - const { cm, strategy } = await open(dir); - cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - const messages = viewOf(cm).getAll(); - assert.throws(() => strategy.pinnedWith(messages, undefined), /a reading is 'ids' or 'whole', not undefined/); - assert.throws(() => strategy.pinnedWith(messages, 'all'), TypeError); - assert.ok(strategy.pinnedWith(messages, 'whole').size > 1); - await cm.close(); - }); - }); - it('a mark over its own part locked at a fold renders it raw over the lock, as before', async () => { await withDir(async (dir) => { - const { cm, strategy } = await open(dir); - await history(cm); - const b = cm.addMessage('user', words('Bdoc', 6000)); - await traffic(cm, 360); - await ready(cm); - await cm.compile(budget(30_000)); + const { cm, strategy, b } = await markedAfterSummaries(dir); assert.ok(strategy.resolutionOf(b) > 0, 'the first part folded'); strategy.lockChunk(b); cm.markDocument(b); @@ -615,428 +412,297 @@ describe('a compile that can\'t hold the whole messages narrows pins', () => { await cm.close(); }); }); +}); - it('an error from a narrowed select leaves the refused select\'s frontier and transition status', async (t) => { - await withDir(async (dir) => { - warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - strategy.refusals = []; - // The narrowed select plans, setting its own frontier, and then fails. - strategy.failNarrowedAfterPlanning = true; - await assert.rejects(cm.compile(budget(30_000)), /select failed/); - strategy.failNarrowedAfterPlanning = false; - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.strictEqual(strategy.getHotContextSettings().currentFrontierTokens, strategy.refusals[0]!.actual); - assert.strictEqual(strategy.getHotContextSettings().blocked, undefined, 'the refused select\'s transition status'); - await cm.close(); - }); - }); - - it('the warning names a named mark and its part, and gives the whole reading\'s refusal', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000, 'spec']], budget: 30_000 }); - strategy.refusals = []; - await cm.compile(budget(30_000)); - const refusal = strategy.refusals[0]!; - const said = narrowingWarnings(out); - assert.strictEqual(said.length, 1); - assert.ok(said[0]!.startsWith(`autobio: document mark "spec" (${marks[0]}) now covers only message ${strategy.firstIdOf(marks[0]!)}, one part of a message stored in parts, not the whole message. `), said[0]); - assert.ok(said[0]!.includes(`the compile came to ~${refusal.actual} tokens against a hard budget of ${refusal.budget} (maxTokens 30000).`), said[0]); - await cm.close(); +describe('a compile that can\'t hold a whole message refuses, naming the pins that keep raw tokens in the middle', () => { + for (const kind of ['adaptive', 'kv-stable', 'kv-unified'] as const) { + it(`${kind}: names the mark, its whole message and its tokens; unpinned, the compile goes ahead`, async () => { + await withDir(async (dir) => { + const { cm, b, parts } = await markedAfterSummaries(dir, kind); + const mark = cm.markDocument(b, { name: 'the spec' }); + const refusal = await refusalOf(cm, 30_000); + const raw = refusal.diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map(({ tokens, ...pin }) => pin), [{ + pinId: mark, kind: 'document', name: 'the spec', firstMessageId: b, lastMessageId: b, parts: parts.length, + }]); + assert.ok(raw.tokens > 10_000 && raw.pins[0]!.tokens === raw.tokens, `the whole message's tokens: ${raw.tokens}`); + assert.ok(raw.tokens <= refusal.diagnostics.middleTokens, `no more than the middle holds: ${raw.tokens} of ${refusal.diagnostics.middleTokens}`); + assert.match(refusal.message, /exceed hard budget/, 'the phrase agent-framework matches stays'); + assert.ok(refusal.message.endsWith( + `). A pin keeps ~${raw.tokens} tokens raw in the middle: document mark "the spec" (${mark}), message ${b}, ` + + `covering ${parts.length} stored parts of its message, ~${raw.tokens} tokens. Whoever manages the pins can unpin it, which ` + + `lets compression fold what it covers and frees up to its tokens less its summary; or whoever sets the hard budget can raise it, if the model's window allows.`), refusal.message); + assert.ok(cm.unpin(mark)); + await cm.compile(budget(30_000)); + await cm.close(); + }); }); - }); + } - it('a provisional calibration multiplier the whole reading is over is dropped before any narrowing, as for any refusal', async (t) => { + it('counts the parts a mark covers, less a part locked with lockChunk that it doesn\'t name', async () => { await withDir(async (dir) => { - const out = warnings(t); - const first = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 42_000 }); - first.strategy.saveProvisionalCalibration(1.3); - await first.cm.close(); - // At 1.3 the whole message is over 42,000 and the narrowed one isn't; at 1 the whole one fits. - const { cm, strategy } = await open(dir); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); - assert.ok(!strategy.isNarrowed(first.marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); - const calibration = cm.getRenderStats()!.calibration!; - assert.deepStrictEqual([calibration.multiplier, calibration.reset?.discardedMultiplier], [1, 1.3]); + const { cm, strategy, b, parts } = await markedAfterSummaries(dir); + strategy.lockChunk(parts[3]!); + const mark = cm.markDocument(b); + const refusal = await refusalOf(cm, 30_000); + assert.deepStrictEqual(refusal.diagnostics.rawPins!.pins.map((p) => [p.pinId, p.parts]), [[mark, parts.length - 1]]); + assert.match(refusal.message, new RegExp(`covering ${parts.length - 1} stored parts of its message`)); await cm.close(); }); }); - it('a mark whose other parts aren\'t summarized yet frees nothing, so another mark\'s narrowing doesn\'t take it along', async (t) => { + it('adaptive: prices a pinned message the cap cuts as the select sends it, cut', async () => { await withDir(async (dir) => { - warnings(t); - const { cm, strategy } = await open(dir); + // A message of about 1,000 tokens, under the size at which ingress stores a message in parts, cut at 300. + const { cm } = await open(dir, 'adaptive', 8000, { maxMessageTokens: 300 }); await history(cm); - // C's parts are summarized; B's, which arrive after, aren't, with no tick since. - const c = cm.markDocument(cm.addMessage('user', words('Cdoc', 3000))); - await traffic(cm, 120); + const m = cm.addMessage('user', words('Mlong', 280)); + await traffic(cm, 360); await ready(cm); - const b = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - await traffic(cm, 60, 'later'); - for (let max = 80_000; !strategy.isNarrowed(c) && max > 10_000; max -= 1_000) { + const pin = cm.pinRange(m, m); + let raw: NonNullable | undefined; + for (const maxTokens of [12_000, 9_000, 7_000, 5_000, 4_000, 3_000, 2_000, 1_500, 1_000, 700]) { try { - await cm.compile(budget(max)); + await cm.compile(budget(maxTokens)); } catch (err) { if (!(err instanceof OverBudgetError)) throw err; + raw = err.diagnostics.rawPins; + if (raw) break; } } - assert.ok(strategy.isNarrowed(c), 'C was narrowed'); - assert.ok(!strategy.isNarrowed(b), 'B was not'); - await cm.close(); - }); - }); - - it('unpinning a narrowed mark and marking the message again covers it whole', async (t) => { - await withDir(async (dir) => { - warnings(t); - const { cm, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - await cm.compile(budget(30_000)); - const b = cm.listPins()[0]!.firstMessageId; - assert.ok(cm.unpin(marks[0]!)); - cm.markDocument(b); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); - await cm.close(); - }); - }); - - it('a new mark on a long message narrows as it stops fitting, with no compile refused', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); - const turn = async (content: ContentBlock[], role: 'user' | 'assistant') => { - cm.addMessage(role, content); - const request = sent(await cm.compile(budget(30_000))); - await ready(cm); - return request; - }; - for (let i = 0; i < 6; i++) await turn(words(`hist${i}`, 200), i % 2 ? 'assistant' : 'user'); - const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - let last: Sent[] = []; - for (let i = 0; i < 120; i++) last = await turn(words(`event${i}`, 200), i % 2 ? 'assistant' : 'user'); - assert.ok(strategy.isNarrowed(mark)); - assert.strictEqual(narrowingWarnings(out).length, 1); - assert.match(narrowingWarnings(out)[0]!, new RegExp(`document mark ${mark} now covers only message `)); - assert.deepStrictEqual(rawWords(last, 'Bdoc'), allOf(763)); - await cm.close(); - }); - }); - - it('narrows the mark whose whole message adds the most first, and no more than the compile needs', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - // Both whole need about 40,800; B narrowed alone, about 25,200; C narrowed alone, about 38,700. - // C is marked first, so taking the first mark that frees anything would narrow C. - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Cdoc', 1500], ['Bdoc', 6000]], budget: 30_000 }); - const request = sent(await cm.compile(budget(30_000))); - assert.deepStrictEqual(marks.map((m) => strategy.isNarrowed(m)), [false, true]); - assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(1500)); - assert.strictEqual(narrowingWarnings(out).length, 1); - assert.match(narrowingWarnings(out)[0]!, new RegExp(`document mark ${marks[1]} `)); - await cm.close(); - }); - }); - - it('narrows a second mark, after the first, when the first alone doesn\'t let the compile through', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - // Both narrowed need about 23,000; B narrowed alone, about 25,200. - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000], ['Cdoc', 1500]], budget: 24_000 }); - assert.deepStrictEqual(sent(await cm.compile(budget(24_000))), before); - assert.deepStrictEqual(marks.map((m) => strategy.isNarrowed(m)), [true, true]); - const said = narrowingWarnings(out); - assert.strictEqual(said.length, 2); - assert.match(said[0]!, new RegExp(`document mark ${marks[0]} `)); - assert.match(said[1]!, new RegExp(`document mark ${marks[1]} `)); - // The refusal's total had both whole, and each warning says so. - for (const w of said) assert.ok(w.includes('With the 2 pins narrowed here covering their whole messages, the compile came to ~'), w); - await cm.close(); - }); - }); - - it('two marks on one message: narrowing one frees nothing while the other covers the message, so both are narrowed', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); - await history(cm); - const parts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); - const first = cm.markDocument(parts[0]!); - const second = cm.markDocument(parts[2]!); - strategy.setNarrowed(first, true); - strategy.setNarrowed(second, true); - await traffic(cm, 360); - await ready(cm); - const before = sent(await cm.compile(budget(30_000))); - strategy.setNarrowed(first, false); - strategy.setNarrowed(second, false); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.deepStrictEqual([strategy.isNarrowed(first), strategy.isNarrowed(second)], [true, true]); - assert.strictEqual(narrowingWarnings(out).length, 2); + assert.ok(raw, 'a refusal named the pin'); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [pin]); + assert.ok(raw.pins[0]!.tokens > 0 && raw.pins[0]!.tokens < 450, `priced cut: ${raw.pins[0]!.tokens}`); await cm.close(); }); }); - it('doesn\'t narrow a mark whose message lies in the recent window, which is sent raw either way', async (t) => { + it('prices a marked message\'s parts uncut, as the plan sends a run of them joined', async () => { await withDir(async (dir) => { - const out = warnings(t); - // A recent window of 20,000. B, marked before the upgrade, is in the middle; C, 6,000 words marked - // now, is in the recent window. Both whole need about 39,100; B narrowed, about 33,900, with C whole - // or not. C's whole message adds more than B's, so without the window check C would go first. - const { cm, strategy } = await open(dir, 'adaptive', 20_000); - await history(cm); - const b = cm.markDocument(cm.addMessage('user', words('Bdoc', 2500))); - strategy.setNarrowed(b, true); - await traffic(cm, 360); - const c = cm.markDocument(cm.addMessage('user', words('Cdoc', 6000))); - await ready(cm); - strategy.setNarrowed(b, false); - const request = sent(await cm.compile(budget(36_000))); - assert.deepStrictEqual([strategy.isNarrowed(b), strategy.isNarrowed(c)], [true, false]); - assert.deepStrictEqual(rawWords(request, 'Cdoc'), allOf(6000)); - assert.strictEqual(narrowingWarnings(out).length, 1); - await cm.close(); - }); - }); - - it('a mark whose message is in the head window frees nothing, so a refusal runs no select again', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - // A head window of 25,000 holds the history and the whole 6,000-word message. - const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 25_000 }); + // Parts of about 2,000 tokens, each over a cap of 300 that the plan doesn't apply to them. + const { cm } = await open(dir, 'adaptive', 8000, { maxMessageTokens: 300 }); await history(cm); const b = cm.addMessage('user', words('Bdoc', 6000)); - const mark = cm.markDocument(b); await traffic(cm, 360); await ready(cm); - const all = viewOf(cm).getAll(); - const last = all.findIndex((m) => m.id === partsOf(cm, b).at(-1)); - assert.ok(strategy.headEnd(viewOf(cm) as never) > last, 'the head holds the whole message'); - // Lower the budget until a compile is refused for its middle, where narrowing would free nothing. - let refused: OverBudgetError | undefined; - let ran = 0; - for (let max = 44_000; !refused && max > 20_000; max -= 1_000) { - const selects = strategy.selects; + const mark = cm.markDocument(b); + let raw: NonNullable | undefined; + for (const maxTokens of [30_000, 25_000, 20_000, 15_000]) { try { - await cm.compile(budget(max)); + await cm.compile(budget(maxTokens)); } catch (err) { - assert.ok(err instanceof OverBudgetError); - refused = err; - ran = strategy.selects - selects; + if (!(err instanceof OverBudgetError)) throw err; + raw = err.diagnostics.rawPins; + if (raw) break; } } - assert.ok(refused && refused.diagnostics.middleTokens > 0, 'refused for its middle'); - assert.strictEqual(ran, 1, 'nothing to narrow, so no select again'); - assert.ok(!strategy.isNarrowed(mark)); - assert.deepStrictEqual(narrowingWarnings(out), []); + assert.ok(raw, 'a refusal named the mark'); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + assert.ok(raw.tokens > 10_000, `priced uncut: ${raw.tokens}`); await cm.close(); }); }); - it('after a head-window reset, a mark in the history before the head frees its message once it leaves the recent window', async (t) => { + it('names every pin keeping raw tokens in the middle, counting a message two pins cover once, and none in the recent window', async () => { await withDir(async (dir) => { - warnings(t); - // A recent window of 20,000. The marked message is history before the head, which the selects fold. - const { cm, strategy } = await open(dir, 'adaptive', 20_000); - await history(cm); - const b = cm.addMessage('user', words('Bdoc', 2500)); + const { cm, b, parts } = await markedAfterSummaries(dir); + const all = viewOf(cm).getAll(); + const at = all.findIndex((m) => m.id === parts.at(-1)); const mark = cm.markDocument(b); - await cm.resetHeadWindow('A new topic.'); - const narrows = async (): Promise => { - for (let max = 60_000; !strategy.isNarrowed(mark) && max > 8_000; max -= 1_000) { - try { - await cm.compile(budget(max)); - } catch (err) { - if (!(err instanceof OverBudgetError)) throw err; - } - } - return strategy.isNarrowed(mark); - }; - const positions = () => { - const at = viewOf(cm).getAll().findIndex((m) => m.id === b); - return { at, ...strategy.windows(viewOf(cm) as never) }; - }; - // Within the recent window its parts can't be summarized, so narrowing it would free nothing. - cm.addMessage('user', words('after', 20)); - await ready(cm); - const inRecent = positions(); - assert.ok(inRecent.at > inRecent.recentStart && inRecent.at < inRecent.headStart, JSON.stringify(inRecent)); - assert.strictEqual(await narrows(), false); - // Out of it, they're summarized, and narrowing it frees them. - await traffic(cm, 120); - await ready(cm); - const outOfRecent = positions(); - assert.ok(outOfRecent.at < outOfRecent.recentStart && outOfRecent.at < outOfRecent.headStart, JSON.stringify(outOfRecent)); - assert.strictEqual(await narrows(), true); + // A range over B's third and fourth parts, which read whole covers all of B, as the mark does. + const inside = cm.pinRange(parts[2]!, parts[3]!); + // A range over two traffic messages in the middle, and a pin on the newest message, in the recent window. + const traffic2 = cm.pinRange(all[at + 10]!.id, all[at + 11]!.id, { name: 'two events' }); + cm.pinRange(all.at(-1)!.id, all.at(-1)!.id); + const refusal = await refusalOf(cm, 30_000); + const raw = refusal.diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => [p.pinId, p.parts]), [[mark, parts.length], [inside, parts.length], [traffic2, undefined]]); + const [m, i, t] = raw.pins; + assert.strictEqual(m!.tokens, i!.tokens, 'both cover the whole message'); + assert.strictEqual(raw.tokens, m!.tokens + t!.tokens, 'the message both cover counts once'); + assert.ok(refusal.message.endsWith('Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those ' + + 'tokens less their summaries; or whoever sets the hard budget can raise it, if the model\'s window allows.'), refusal.message); await cm.close(); }); }); - it('narrows a range pin to its own ids, warning in a range\'s words, and sends the request it sent before', async (t) => { + it('a pin in the head window isn\'t named, since the head is sent raw either way', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); + // A head window of 3,000 holds the first history messages. + const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 3000 }); await history(cm); - const bParts = partsOf(cm, cm.addMessage('user', words('Bdoc', 6000))); - const cParts = partsOf(cm, cm.addMessage('user', words('Cdoc', 1500))); - // From B's last part to C's first: whole, both messages; by its ids, B's last part and C's first. - const range = cm.pinRange(bParts.at(-1)!, cParts[0]!); - strategy.setNarrowed(range, true); + const b = cm.addMessage('user', words('Bdoc', 6000)); await traffic(cm, 360); await ready(cm); - const before = sent(await cm.compile(budget(30_000))); - strategy.setNarrowed(range, false); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.ok(strategy.isNarrowed(range)); - const said = narrowingWarnings(out); - assert.strictEqual(said.length, 1); - assert.match(said[0]!, new RegExp(`^autobio: pin ${range} now covers only messages ${bParts.at(-1)} to ${cParts[0]}, `)); - assert.match(said[0]!, new RegExp(`unpin ${range} and pin the range again\\.`)); + const all = viewOf(cm).getAll(); + assert.ok(strategy.headEnd(viewOf(cm) as never) > 0, 'the head window holds the first message'); + cm.pinRange(all[0]!.id, all[0]!.id); + const mark = cm.markDocument(b); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); await cm.close(); }); }); - it('a refusal narrowing doesn\'t answer puts the mark back and stands as it was, preview included', async (t) => { + it('after a head reset, names a pin in the history before the new head, which the select folds as middle', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - // At 15,000 the narrowed request is over too. - const preview = cm.previewContext(budget(15_000))!; - assert.ok(!strategy.isNarrowed(marks[0]!), 'the dry compile kept nothing'); - const selects = strategy.selects; - let thrown: OverBudgetError | undefined; - await assert.rejects(cm.compile(budget(15_000)), (err: unknown) => { - assert.ok(err instanceof OverBudgetError); - thrown = err; - return true; - }); - assert.strictEqual(strategy.selects - selects, 2, 'the refused select and one narrowed'); - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); - // A preview reports the plan the refusal describes, not a narrowed retry's, and so does the frontier. - assert.strictEqual(preview.fits, false); - assert.strictEqual(preview.finalTokens, thrown!.actual); - assert.strictEqual(strategy.getHotContextSettings().currentFrontierTokens, thrown!.actual); - // The refusal thrown is the whole reading's: the narrowed reading's is smaller. - strategy.setNarrowed(marks[0]!, true); - await assert.rejects(cm.compile(budget(15_000)), (err: unknown) => { - assert.ok(err instanceof OverBudgetError); - assert.ok(thrown!.actual > err.actual, `whole ${thrown!.actual}, narrowed ${err.actual}`); - return true; - }); - strategy.setNarrowed(marks[0]!, false); - // Put back whole: the next compile with room sends it whole. - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + const { cm, strategy } = await markedAfterSummaries(dir); + const all = viewOf(cm).getAll(); + // The head now starts at the newest message, so the recent window reaches back before it: the history before + // the head, recent messages included, is the select's middle. + strategy.resetHeadWindow(all.at(-1)!.id); + const pin = cm.pinRange(all.at(-3)!.id, all.at(-3)!.id); + const raw = (await refusalOf(cm, 3_000)).diagnostics.rawPins; + assert.deepStrictEqual(raw?.pins.map((p) => p.pinId), [pin]); await cm.close(); }); }); - it('a refusal while nothing the mark covers is summarized narrows nothing, and the whole message goes out once compression catches up', async (t) => { + it('kv-stable: names a raw pin but not a pin with a level or a cap that overlaps it, which keeps nothing raw', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); - await history(cm); - const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - // No tick while 360 messages arrive: nothing after the message is summarized. - await traffic(cm, 360); - const selects = strategy.selects; - await assert.rejects(cm.compile(budget(42_000)), OverBudgetError); - assert.strictEqual(strategy.selects - selects, 1, 'no narrowed select ran'); - assert.ok(!strategy.isNarrowed(mark)); - assert.deepStrictEqual(narrowingWarnings(out), []); - await ready(cm); - assert.deepStrictEqual(rawWords(sent(await cm.compile(budget(42_000))), 'Bdoc'), allOf(6000)); + const { cm, b } = await markedAfterSummaries(dir, 'kv-stable'); + const mark = cm.markDocument(b, { level: 0 }); + cm.pinAtLevel(b, b, 2); + cm.pinRange(b, b, { maxLevel: 2 }); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); await cm.close(); }); }); - it('under a solver that reads no bounds, a mark with a level cap renders raw and is narrowed like any', async (t) => { + it('kv-unified: doesn\'t count toward a raw range the message a level pin holds inside it', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); + const { cm } = await open(dir, 'kv-unified'); await history(cm); - const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { maxLevel: 1 }); - strategy.setNarrowed(mark, true); + const b = cm.addMessage('user', words('Bdoc', 3000)); + const c = cm.addMessage('user', words('Cdoc', 3000)); await traffic(cm, 360); await ready(cm); - const before = sent(await cm.compile(budget(30_000))); - strategy.setNarrowed(mark, false); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - assert.ok(strategy.isNarrowed(mark)); - assert.strictEqual(narrowingWarnings(out).length, 1); + // The range keeps B and C raw; a level, which takes a classic pin's place, holds C at level 1. + const range = cm.pinRange(b, c); + cm.pinAtLevel(c, c, 1); + let raw: NonNullable | undefined; + const seen: string[] = []; + for (const maxTokens of [30_000, 25_000, 20_000, 15_000, 12_000]) { + try { + await cm.compile(budget(maxTokens)); + seen.push(`${maxTokens}: compiled`); + } catch (err) { + seen.push(`${maxTokens}: ${(err as Error).message.slice(0, 120)}`); + if (!(err instanceof OverBudgetError)) continue; + raw = err.diagnostics.rawPins; + if (raw) break; + } + } + assert.ok(raw, `a refusal named the range: ${seen.join(' | ')}`); + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [range]); + assert.strictEqual(raw.pins[0]!.tokens, raw.tokens, 'the range keeps raw only what the total counts'); await cm.close(); }); }); - it('kv-unified: a mark with a level cap isn\'t narrowed, so a refusal runs no select again', async (t) => { + it('under a solver that reads no bounds, names a mark with a level, which renders raw there', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir, 'kv-unified'); - await history(cm); - const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000)), { maxLevel: 1 }); - await traffic(cm, 360); - await ready(cm); - const selects = strategy.selects; - await assert.rejects(cm.compile(budget(12_000)), (err: unknown) => { - assert.ok(err instanceof OverBudgetError); - assert.ok(err.diagnostics.middleTokens > 0, 'refused for its middle'); - return true; - }); - assert.strictEqual(strategy.selects - selects, 1); - assert.ok(!strategy.isNarrowed(mark)); - assert.deepStrictEqual(narrowingWarnings(out), []); + const { cm, b } = await markedAfterSummaries(dir); + const mark = cm.markDocument(b, { level: 2 }); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); await cm.close(); }); }); - it('a refusal the head and recent windows alone cause narrows nothing and runs no select again', async (t) => { - await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy } = await open(dir); - await history(cm); - const mark = cm.markDocument(cm.addMessage('user', words('Bdoc', 6000))); - await traffic(cm, 360); - await ready(cm); - // A recent window of 8,000 doesn't fit 6,000. - const selects = strategy.selects; - await assert.rejects(cm.compile(budget(6_000)), (err: unknown) => { - assert.ok(err instanceof OverBudgetError); - assert.strictEqual(err.diagnostics.middleTokens, 0); - return true; + it('doesn\'t count a part locked raw inside a pin\'s span, which stays raw without its pin', async () => { + const named = async (lock: boolean) => { + let tokens = 0; + await withDir(async (dir) => { + const { cm, strategy } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const parts = partsOf(cm, b); + // No compile has folded it yet, so the lock holds it raw. + if (lock) strategy.lockChunk(parts[4]!); + assert.strictEqual(strategy.resolutionOf(parts[4]!), 0); + // A range over every part, so the locked one is inside the span its ids name and the pin covers it. + cm.pinRange(parts[0]!, parts.at(-1)!); + tokens = (await refusalOf(cm, 30_000)).diagnostics.rawPins!.tokens; + await cm.close(); }); - assert.strictEqual(strategy.selects - selects, 1); - assert.ok(!strategy.isNarrowed(mark)); - assert.deepStrictEqual(narrowingWarnings(out), []); + return tokens; + }; + const whole = await named(false); + const lessLocked = await named(true); + assert.ok(lessLocked < whole && lessLocked > whole * 0.7, `${lessLocked} of ${whole}`); + }); + + it('a mark held at a level keeps nothing raw, so a refusal in the middle names no pin, as before', async () => { + await withDir(async (dir) => { + const { cm, b } = await markedAfterSummaries(dir, 'kv-stable'); + cm.markDocument(b, { level: 1 }); + const refusal = await refusalOf(cm, 10_000); + assert.match(refusal.message, /^Adaptive picker exhausted/, 'the plan\'s refusal, the one that names pins'); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); + assert.ok(refusal.message.endsWith(')'), refusal.message); await cm.close(); }); }); - it('a dry compile narrows for itself only: it sends what the live compile will, and keeps and warns nothing', async (t) => { + it('a refusal of the head and recent windows alone names no pin, since unpinning wouldn\'t clear it', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - const dry = sent(await cm.compile(budget(30_000), undefined, { dryRun: true })); - assert.deepStrictEqual(dry, before); - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), dry); - assert.ok(strategy.isNarrowed(marks[0]!)); - assert.strictEqual(narrowingWarnings(out).length, 1); + const { cm, b } = await markedAfterSummaries(dir); + cm.markDocument(b); + const refusal = await refusalOf(cm, 5_000); + assert.match(refusal.message, /^Head\/tail reservation exceeded budget/); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); await cm.close(); }); }); - it('a select that fails while narrowing puts the marks back and fails the compile with its error', async (t) => { + it('a hierarchical refusal names no pin, as before: only the adaptive plan\'s refusal does', async () => { await withDir(async (dir) => { - const out = warnings(t); - const { cm, strategy, marks, before } = await upgradedStore(dir, { bodies: [['Bdoc', 6000]], budget: 30_000 }); - strategy.failWhileNarrowed = true; - await assert.rejects(cm.compile(budget(30_000)), /select failed/); - strategy.failWhileNarrowed = false; - assert.ok(!strategy.isNarrowed(marks[0]!)); - assert.deepStrictEqual(narrowingWarnings(out), []); - assert.deepStrictEqual(sent(await cm.compile(budget(30_000))), before); - await cm.close(); + const strategy = new AutobiographicalStrategy({ + compressionModel: 'test-model', adaptiveResolution: false, targetChunkTokens: 40, headWindowTokens: 0, + recentWindowTokens: 0, overBudgetGraceRatio: 0.25, autoTickOnNewMessage: false, speculativeProduction: false, + } as AutobiographicalConfig); + const cm = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane: summarizer().membrane }); + const ids: string[] = []; + for (let i = 0; i < 12; i++) ids.push(cm.addMessage('user', [{ type: 'text', text: `message ${i} ` + 'word '.repeat(30) }])); + cm.pinRange(ids[2]!, ids[3]!); + const refusal = await refusalOf(cm, 100); + assert.match(refusal.message, /^Hierarchical emission overran the budget/); + assert.strictEqual(refusal.diagnostics.rawPins, undefined); + assert.ok(refusal.message.endsWith(')'), refusal.message); + cm.close(); }); }); }); + +describe('the refusal\'s words for the pins it names', () => { + const diagnostics = { headTokens: 0, tailTokens: 10, middleTokens: 90, middleChunkCount: 3, deepestLevel: 1 }; + it('names one pin, a range read whole, and a pin with no name', () => { + const error = new OverBudgetError({ + budget: 50, actual: 100, + diagnostics: { ...diagnostics, rawPins: { tokens: 70, pins: [{ pinId: 'pin-1', kind: 'pin', firstMessageId: 'a', lastMessageId: 'c', parts: 5, tokens: 70 }] } }, + }); + assert.ok(error.message.endsWith( + '). A pin keeps ~70 tokens raw in the middle: pin pin-1, messages a to c, covering 5 stored messages, ~70 tokens. ' + + 'Whoever manages the pins can unpin it, which lets compression fold what it covers and frees up to its tokens less its summary; or whoever sets the hard budget can raise it, if the model\'s window allows.'), error.message); + }); + it('names several pins, and says nothing of pins without any', () => { + const error = new OverBudgetError({ + budget: 50, actual: 100, + diagnostics: { + ...diagnostics, + rawPins: { tokens: 80, pins: [ + { pinId: 'pin-0', kind: 'document', name: 'notes', firstMessageId: 'm', lastMessageId: 'm', tokens: 50 }, + { pinId: 'pin-2', kind: 'pin', firstMessageId: 'x', lastMessageId: 'x', tokens: 30 }, + ] }, + }, + }); + assert.ok(error.message.endsWith( + '). Pins keep ~80 tokens raw in the middle: document mark "notes" (pin-0), message m, ~50 tokens; pin pin-2, message x, ~30 tokens. ' + + 'Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those tokens less their summaries; or whoever sets the hard budget can raise it, if the model\'s window allows.'), error.message); + assert.ok(new OverBudgetError({ budget: 50, actual: 100, diagnostics }).message.endsWith(')')); + }); +}); From 5f5ed6a4eb4d99cbeb7920717cfe9650418d8bfc Mon Sep 17 00:00:00 2001 From: Ezra-2423 Date: Fri, 23 Oct 2026 20:26:34 -0400 Subject: [PATCH 44/67] fix(thinking-binding): digest a noted row as the compile sends it; price a split body's thinking per part - noteRequestSent digests each stored row as the compile turns it into messages (splitMixedToolMessages): a tool result stored under another participant goes as the user's, and a bundled tool cycle as each of its parts, one digest a part. A noted row like either diverged at the next compile before, so every reply after it lost its thinking. - The rendered layout keeps each entry's thinking as the compile sent it. A bundled tool cycle is bound part by part, so one part can keep the thinking another leaves out; the layout had left out the whole body's. - Tests: a result stored under 'Tool' and a bundled cycle, noted and kept; a split body whose second part loses its thinking, priced with the first part's. Co-Authored-By: Claude Opus 5.5 --- changelog.d/155-thinking-binding.added.md | 2 +- src/context-manager.ts | 47 +++++++++++++++-------- src/thinking-binding.ts | 23 +++++++++++ test/thinking-binding.test.ts | 37 ++++++++++++++++++ 4 files changed, 92 insertions(+), 17 deletions(-) diff --git a/changelog.d/155-thinking-binding.added.md b/changelog.d/155-thinking-binding.added.md index e4cd5d59..bb6eb6a4 100644 --- a/changelog.d/155-thinking-binding.added.md +++ b/changelog.d/155-thinking-binding.added.md @@ -7,7 +7,7 @@ - **Without `binds`,** a compile is as before, except that a committing one clears the branch's record, since what it sends no record vouches for. - **While a request is streaming,** compile for its branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing, so its replies go without their thinking. - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. -- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read. - `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the branch's last committing compile, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. - `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. - On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. diff --git a/src/context-manager.ts b/src/context-manager.ts index 919f518b..fb1eefc0 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -62,7 +62,7 @@ import { FoldJournal, branchRefOf, cachedStoreIdentity, existingStoreIdentity } import { describeThrown, logThrown } from './describe-thrown.js'; import { UNVOUCHED, bindThinking, clearBindingRecord, continueRecord, isThinkingBlock, markThinkingLeftOut, messageDigest, readBindingRecord, - refuseUnboundBinding, refuseUnreadableIdentity, registerBindingState, withoutThinking, writeBindingRecord, + refuseUnboundBinding, refuseUnreadableIdentity, registerBindingState, thinkingAsSent, thinkingSent, withoutThinking, writeBindingRecord, type BindOutcome, type BindingRecord, } from './thinking-binding.js'; import { type Diagnostics, namespaceDiagnostics } from './diagnostics.js'; @@ -1272,10 +1272,13 @@ export class ContextManager { } rawComplete = complete; } - layoutEntries = entries.map((entry, e) => { - const source = attribution.sources[e]; - const kept = source?.kind === 'raw' && !source.bodies.some((body) => without.has(body.messageId)); - return kept || !entry.content.some(isThinkingBlock) ? entry : { ...entry, content: withoutThinking(entry.content) }; + // Each entry keeps the thinking the compile sent of it: an entry the + // compile splits (a bundled tool cycle) is bound part by part, so one + // part can keep what another leaves out. + const sent = thinkingSent(result.messages); + layoutEntries = entries.map((entry) => { + const content = thinkingAsSent(entry.content, sent); + return content === entry.content ? entry : { ...entry, content }; }); layoutInjected = injected.map((content) => (content.some(isThinkingBlock) ? withoutThinking(content) : content)); } @@ -1707,11 +1710,13 @@ export class ContextManager { * old head. A host that notes nothing keeps no reply's thinking past the * compile after it was stored, and is never refused for it. * - * Each stored row is digested as `renderAsNewest` renders it, so one it - * renders as null, or that the next compile renders otherwise, matches - * nothing there, and what follows goes without its thinking; a note that - * renders rows this way releases compression holds that have timed out, as - * `renderAsNewest` does. + * Each stored row is digested as `renderAsNewest` renders it and as the + * compile turns that into messages: a tool result under another + * participant goes as the user's, and a bundled tool cycle as each of its + * parts (`splitMixedToolMessages`). So one it renders as null, or that the + * next compile renders otherwise, matches nothing there, and what follows + * goes without its thinking; a note that renders rows this way releases + * compression holds that have timed out, as `renderAsNewest` does. * Without `renderAsNewest`, a row is its stored content, and a body stored * as shards matches nothing, and so does an id the store doesn't hold, as * after a removal. @@ -1751,16 +1756,26 @@ export class ContextManager { const ids = rows.filter((row): row is MessageId => typeof row === 'string' && view.get(row) !== null); const rendered = ids.length > 0 ? this.renderAsNewest(ids) : []; let next = 0; - const continued = rows.map((row) => { - if (typeof row !== 'string') return UNVOUCHED; - const stored = view.get(row); - if (stored === null) return UNVOUCHED; + const continued: string[] = []; + for (const row of rows) { + const stored = typeof row === 'string' ? view.get(row) : null; + if (stored === null) { + continued.push(UNVOUCHED); + continue; + } // With no renderAsNewest, a row is its stored content, which for a body // stored as shards is only its first shard's. const content = rendered === undefined ? (stored.bodyGroupId === undefined ? stored.content : null) : rendered[next]; next++; - return content === null || content === undefined ? UNVOUCHED : messageDigest({ participant: stored.participant, content }); - }); + if (content === null || content === undefined) { + continued.push(UNVOUCHED); + continue; + } + // As the compile turns it into messages: a tool result under another + // participant goes as the user's, and a bundled tool cycle as each of + // its parts. + for (const part of splitMixedToolMessages([{ participant: stored.participant, content }])) continued.push(messageDigest(part)); + } writeBindingRecord(this.store, this.strategyNamespace, continueRecord(record, continued, identity)); return true; } diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts index d9b09b17..da684e79 100644 --- a/src/thinking-binding.ts +++ b/src/thinking-binding.ts @@ -45,6 +45,29 @@ export function withoutThinking(content: ContentBlock[]): ContentBlock[] { return releaseEditedRawForms(content, content.filter((block) => !isThinkingBlock(block))); } +/** + * The content less the thinking blocks `messages` don't carry, raw forms + * released as `withoutThinking` releases them; the content itself when they + * carry all of its thinking. A block is matched by its canonical form, cache + * marker aside: a signature makes each one distinct. + */ +export function thinkingAsSent(content: ContentBlock[], sent: ReadonlySet): ContentBlock[] { + const left = (block: ContentBlock) => isThinkingBlock(block) && !sent.has(thinkingKey(block)); + if (!content.some(left)) return content; + return releaseEditedRawForms(content, content.filter((block) => !left(block))); +} + +/** Each thinking block `messages` carry, by its canonical form, for `thinkingAsSent`. */ +export function thinkingSent(messages: ReadonlyArray<{ content: ContentBlock[] }>): Set { + const sent = new Set(); + for (const message of messages) for (const block of message.content) if (isThinkingBlock(block)) sent.add(thinkingKey(block)); + return sent; +} + +function thinkingKey(block: ContentBlock): string { + return canonicalJson(withoutCacheMarker(block)); +} + /** What a branch last sent, as its record keeps it. */ export interface BindingRecord { /** Each message the last committing compile sent, in order: a digest of diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts index 7fccc48e..a33cb745 100644 --- a/test/thinking-binding.test.ts +++ b/test/thinking-binding.test.ts @@ -604,6 +604,43 @@ describe('thinking binding: compile', () => { } }); + it('a noted row is digested as the compile sends it: a result under another participant as the user\u2019s, a bundled tool cycle as each part', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND_DROP); + const a1 = manager.addMessage('Claude', [thought(1), toolUse('t1')]); + const r1 = manager.addMessage('Tool', [toolResult('t1', 'zz-result')]); + const b1 = manager.addMessage('Claude', [thought(2), toolUse('t2'), toolResult('t2', 'zz-bundled'), thought(3), text('B1')]); + assert.equal(manager.noteRequestSent([a1, r1, b1], identity(), c1.provenance!.compileId), true); + manager.addMessage('User', [text('U2')]); + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.ok(next.messages.some((m) => m.participant === 'user' && m.content.some((b) => b.type === 'tool_result')), 'setup: the compile sends results as the user\u2019s'); + assert.equal(next.messages.length, 7, 'setup: the bundled cycle goes as three messages'); + assert.equal(next.thinkingStrip, undefined); + assert.equal(next.messages.reduce((n, m) => n + m.content.filter(isThinking).length, 0), 3, 'every noted reply keeps its thinking'); + await manager.close(); + }); + + it('the layout prices a split body\u2019s thinking as each part sent it', async () => { + cleanup(); + const manager = await open(); + manager.addMessage('User', [text('U1')]); + const c1 = await manager.compile(BIG, undefined, BIND_DROP); + const b1 = manager.addMessage('Claude', [thought(1), toolUse('t1'), toolResult('t1', 'zz-bundled'), thought(2), text('B1')]); + manager.noteRequestSent([b1], identity(), c1.provenance!.compileId); + // Placed after the cycle's result, the injection changes what precedes + // the second part, so its thinking goes and the first part's stays. + const late: ContextInjection = { namespace: 'zz-late', position: 'afterUser', content: [text('zz-late')] }; + const plain = await manager.compile(BIG, [late], { dryRun: true }); + const bound = await manager.compile(BIG, [late], { ...BIND_DROP, dryRun: true }); + const kept = bound.messages.flatMap((m) => m.content.filter(isThinking)); + assert.deepEqual(kept, [thought(1)], 'setup: the first part keeps its thinking, the second goes without'); + const price = (blocks: ContentBlock[]) => (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens(blocks); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([thought(2)])); + await manager.close(); + }); + it('a strategy that throws taking note of the thinking left out is logged, and the compile stands', async () => { cleanup(); class Throwing extends PassthroughStrategy { From 1c0f911e864785a9ed26de2d322aa34f282f2094 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 20:26:47 -0400 Subject: [PATCH 45/67] fix(pins): a remedy short enough to survive a quoted cut, every raw position counted, and the rawPins doc's scope Co-Authored-By: Claude Opus 5.5 --- changelog.d/raw-pins-in-refusals.added.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- src/adaptive/picker.ts | 28 +++++++++++--------- src/strategies/autobiographical.ts | 19 +++++++------- test/whole-body-pins.test.ts | 31 ++++++++++++++--------- 5 files changed, 47 insertions(+), 35 deletions(-) diff --git a/changelog.d/raw-pins-in-refusals.added.md b/changelog.d/raw-pins-in-refusals.added.md index 6b7aec5f..b39bfa1a 100644 --- a/changelog.d/raw-pins-in-refusals.added.md +++ b/changelog.d/raw-pins-in-refusals.added.md @@ -1 +1 @@ -- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them, a part locked raw left out), and the message ends with a sentence naming each pin and its tokens, and saying that whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to their tokens less their summaries, or whoever sets the hard budget can raise it where the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent. Each pin's tokens are what it keeps raw; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. +- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them), and the message ends with a sentence naming each pin and the tokens, and saying that whoever manages pins can unpin them, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent, and it comes last and under 140 characters, so a host that quotes a long message by its head and its last 140 characters, as agent-framework does, keeps it whole. Each pin's tokens are what it keeps raw, a part locked with `lockChunk` included, so they're what unpinning frees at most; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 85ebd761..a638ea5e 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ - **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. - - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold, and that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages the pins can unpin it, which lets compression fold the message, or whoever sets the hard budget can raise it, where the model's window allows. + - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold, and that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. diff --git a/src/adaptive/picker.ts b/src/adaptive/picker.ts index 316fe3fb..4dd89fa8 100644 --- a/src/adaptive/picker.ts +++ b/src/adaptive/picker.ts @@ -52,11 +52,14 @@ export interface OverBudgetDiagnostics { /** Configured window minus the response reserve, before over-budget grace. */ inputBudget?: number; /** - * The pins that keep messages raw in the middle, which compression would - * otherwise fold: outside the head and recent windows. `tokens` is the raw - * tokens they keep there together, estimated as the select prices them, - * and each pin gives its own; a message two pins cover counts once in - * `tokens` and toward each pin. Left out when no pin keeps any. + * The pins that keep messages raw in the adaptive plan's middle, which + * compression would otherwise fold: outside the head and recent windows. + * `tokens` is the raw tokens they keep there together, estimated as the + * plan prices them, and each pin gives its own; a message two pins cover + * counts once in `tokens` and toward each pin. Set only by the refusal of a + * plan whose middle is over the budget ("Adaptive picker exhausted but …"); + * every other refusal leaves it out, as does that one when no pin keeps + * any. */ rawPins?: { tokens: number; pins: readonly OverBudgetRawPin[] }; } @@ -180,17 +183,18 @@ function rawPinsText(raw: OverBudgetDiagnostics['rawPins']): string { const parts = pin.parts === undefined ? '' : pin.firstMessageId === pin.lastMessageId ? `, covering ${pin.parts} stored parts of its message` : `, covering ${pin.parts} stored messages`; - return `${what} ${label}, ${span}${parts}, ~${pin.tokens} tokens`; + // A lone pin's own tokens are the total, already given. + return `${what} ${label}, ${span}${parts}${pins.length === 1 ? '' : `, ~${pin.tokens} tokens`}`; }); // Several pins can cover one message, so unpinning one of them frees none // of it: the remedy speaks of them together, whose tokens count it once. // It says who can act rather than telling its reader to, since a host can - // pass the text on to an agent that can do neither. - const remedy = pins.length === 1 - ? 'Whoever manages the pins can unpin it, which lets compression fold what it covers and frees up to its tokens less its summary' - : 'Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those tokens less their summaries'; - return `. ${pins.length === 1 ? 'A pin keeps' : 'Pins keep'} ~${total} tokens raw in the middle: ${each.join('; ')}. ` + - `${remedy}; or whoever sets the hard budget can raise it, if the model's window allows.`; + // pass the text on to an agent that can do neither. It comes last and is + // kept under 140 characters, so a host that quotes a long message by its + // head and last quarter, cut at 600 characters, still quotes it whole. + const remedy = `Whoever manages pins can unpin ${pins.length === 1 ? 'it' : 'them'}, freeing up to those tokens; ` + + `whoever sets the hard budget can raise it if the model's window allows.`; + return `. ${pins.length === 1 ? 'A pin keeps' : 'Pins keep'} ~${total} tokens raw in the middle: ${each.join('; ')}. ${remedy}`; } /** diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 263e9b72..adae9989 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -9887,7 +9887,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { `its pins override: the pinned history it would show raw (~${pinned.pinnedRawTokens} tokens in ` + `${pinned.pinnedRaw.length} message(s); pins: ${pins.join(', ') || 'none found'}) ${limit}. ` + `Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the ` + - `rest of that message too), or raise that budget to show them raw here.`, + `rest of that message too, less a part locked with lockChunk outside the pin's ids), or raise that budget to ` + + `show them raw here.`, ); logCompressionCall({ event: 'compression:pinned-raw-over-budget', @@ -13004,11 +13005,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { * start, which is the recent window's start or the head's end where the * recent window reaches back into the head) that pins keep raw * (`forceRawPinPositions`), each priced by `price`, the plan's own raw - * price. A position locked raw with `lockChunk` stays raw without its pin, - * so it isn't counted. Where bounds are read, a pin whose own bound isn't - * raw keeps nothing raw, so it isn't named even where it overlaps a raw - * pin; where they aren't, every pin renders raw. Undefined when no pin - * keeps any. + * price. Every such position counts, a part locked with `lockChunk` among + * them, so the tokens are what unpinning frees at most: a pin that keeps no + * part raw alone is still named. Where bounds are read, a pin whose own + * bound isn't raw keeps nothing raw, so it isn't named even where it + * overlaps a raw pin; where they aren't, every pin renders raw. Undefined + * when no pin keeps any. */ private rawPinsDiagnostics(store: MessageStoreView, price: (position: number) => number): OverBudgetDiagnostics['rawPins'] { if (this.pins.length === 0) return undefined; @@ -13017,9 +13019,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); const tailStart = Math.max(this.getRecentWindowStart(store), headEnd); - const lockedRaw = (id: MessageId): boolean => this.locked.has(id) && (this.resolutions.get(id) ?? 0) === 0; - const kept = (i: number): boolean => raw.has(i) && (i < headStart || (i >= headEnd && i < tailStart)) - && !lockedRaw(messages[i]!.id); + const kept = (i: number): boolean => raw.has(i) && (i < headStart || (i >= headEnd && i < tailStart)); + let tokens = 0; for (const i of raw) if (kept(i)) tokens += price(i); if (tokens === 0) return undefined; diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index ccc2a429..307b4a2e 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -334,7 +334,7 @@ describe('a pin covers whole messages, for every reader', () => { assert.ok(over.length > 0, 'the warning was given'); for (const w of over) { assert.ok(w.includes('pins: P)'), w); - assert.ok(w.includes('Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the rest of that message too)'), w); + assert.ok(w.includes('Unpin them, or narrow a range pin to fewer messages (a pin on any part of a message stored in parts covers the rest of that message too, less a part locked with lockChunk outside the pin\'s ids)'), w); } await cm.close(); }); @@ -430,8 +430,8 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th assert.match(refusal.message, /exceed hard budget/, 'the phrase agent-framework matches stays'); assert.ok(refusal.message.endsWith( `). A pin keeps ~${raw.tokens} tokens raw in the middle: document mark "the spec" (${mark}), message ${b}, ` + - `covering ${parts.length} stored parts of its message, ~${raw.tokens} tokens. Whoever manages the pins can unpin it, which ` + - `lets compression fold what it covers and frees up to its tokens less its summary; or whoever sets the hard budget can raise it, if the model's window allows.`), refusal.message); + `covering ${parts.length} stored parts of its message. Whoever manages pins can unpin it, freeing up to those tokens; ` + + `whoever sets the hard budget can raise it if the model's window allows.`), refusal.message); assert.ok(cm.unpin(mark)); await cm.compile(budget(30_000)); await cm.close(); @@ -520,8 +520,7 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th const [m, i, t] = raw.pins; assert.strictEqual(m!.tokens, i!.tokens, 'both cover the whole message'); assert.strictEqual(raw.tokens, m!.tokens + t!.tokens, 'the message both cover counts once'); - assert.ok(refusal.message.endsWith('Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those ' + - 'tokens less their summaries; or whoever sets the hard budget can raise it, if the model\'s window allows.'), refusal.message); + assert.ok(refusal.message.endsWith('Whoever manages pins can unpin them, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), refusal.message); await cm.close(); }); }); @@ -611,7 +610,7 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th }); }); - it('doesn\'t count a part locked raw inside a pin\'s span, which stays raw without its pin', async () => { + it('counts a part locked raw inside a pin\'s span, as every raw position in the middle', async () => { const named = async (lock: boolean) => { let tokens = 0; await withDir(async (dir) => { @@ -631,9 +630,7 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th }); return tokens; }; - const whole = await named(false); - const lessLocked = await named(true); - assert.ok(lessLocked < whole && lessLocked > whole * 0.7, `${lessLocked} of ${whole}`); + assert.strictEqual(await named(true), await named(false)); }); it('a mark held at a level keeps nothing raw, so a refusal in the middle names no pin, as before', async () => { @@ -686,8 +683,8 @@ describe('the refusal\'s words for the pins it names', () => { diagnostics: { ...diagnostics, rawPins: { tokens: 70, pins: [{ pinId: 'pin-1', kind: 'pin', firstMessageId: 'a', lastMessageId: 'c', parts: 5, tokens: 70 }] } }, }); assert.ok(error.message.endsWith( - '). A pin keeps ~70 tokens raw in the middle: pin pin-1, messages a to c, covering 5 stored messages, ~70 tokens. ' + - 'Whoever manages the pins can unpin it, which lets compression fold what it covers and frees up to its tokens less its summary; or whoever sets the hard budget can raise it, if the model\'s window allows.'), error.message); + '). A pin keeps ~70 tokens raw in the middle: pin pin-1, messages a to c, covering 5 stored messages. ' + + 'Whoever manages pins can unpin it, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), error.message); }); it('names several pins, and says nothing of pins without any', () => { const error = new OverBudgetError({ @@ -702,7 +699,17 @@ describe('the refusal\'s words for the pins it names', () => { }); assert.ok(error.message.endsWith( '). Pins keep ~80 tokens raw in the middle: document mark "notes" (pin-0), message m, ~50 tokens; pin pin-2, message x, ~30 tokens. ' + - 'Whoever manages the pins can unpin them, which lets compression fold what they cover and frees up to those tokens less their summaries; or whoever sets the hard budget can raise it, if the model\'s window allows.'), error.message); + 'Whoever manages pins can unpin them, freeing up to those tokens; whoever sets the hard budget can raise it if the model\'s window allows.'), error.message); assert.ok(new OverBudgetError({ budget: 50, actual: 100, diagnostics }).message.endsWith(')')); }); + it('ends with a remedy short enough to stay whole in the last quarter of a 600-character cut', () => { + for (const n of [1, 2]) { + const pins = Array.from({ length: n }, (_, i) => ({ pinId: `pin-${i}`, kind: 'pin' as const, name: 'x'.repeat(400), firstMessageId: 'm', lastMessageId: 'm', tokens: 10 })); + const message = new OverBudgetError({ budget: 50, actual: 100, diagnostics: { ...diagnostics, rawPins: { tokens: 10 * n, pins } } }).message; + // agent-framework quotes a compile error cut at 600 characters, keeping the head and the last 140. + const remedy = message.lastIndexOf(' Whoever manages pins'); + assert.ok(message.length > 600 && remedy >= 0, message); + assert.ok(message.length - remedy <= 140, message); + } + }); }); From 6f3b851d5c761885dae084fc4dcbcc67379801e0 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 20:33:39 -0400 Subject: [PATCH 46/67] fix(pins): the plan's middle excludes its tail under a disabled head, and texts and tests from the PR's review Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/whole-body-pins.fixed.md | 2 +- src/strategies/autobiographical.ts | 10 ++++++---- test/whole-body-pins.test.ts | 28 +++++++++++++++++++++++++--- 4 files changed, 33 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index ba4f2d7e..2b8bfc27 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ How it works, with each option's default in parentheses: - With `adaptiveResolution: true` (false), each compile chooses how deeply to fold each part of history to fit the budget. `foldingStrategy` names the solver: `flat-profile` (the default), `oldest-first`, `kv-stable` or `kv-unified`. `kv-unified` needs an explicit `kvUnified` policy. With a strategy that supports them, as this one does, the manager also offers: -- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a pin or mark on any part of a long message stored in parts covers the whole message, in what the agent is sent, in compression and in the summarizer's prompt alike, less a part locked with `lockChunk` outside the span its ids name. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at. A compile whose budget can't hold the raw pins beside the rest refuses with `OverBudgetError`; where the adaptive plan's middle is over the budget, its message and `diagnostics.rawPins` name each pin keeping raw tokens in that middle. `listPins` and `unpin` manage the pins. +- `pinRange` and `markDocument` keep messages at their place in the conversation, raw by default. A pin that sets `level` or `maxLevel` holds its messages at that level, or no deeper than it, only with `adaptiveResolution` and the `kv-stable` or `kv-unified` solver. Under the other solvers, and without adaptive resolution, the pin keeps them raw. Under adaptive resolution, a pin or mark on any part of a long message stored in parts covers the whole message, in what the agent is sent, in compression and in the summarizer's prompt alike, less a part locked with `lockChunk` outside the span its ids name. As with any pin, a chunk recorded before it keeps its messages, so compression can still summarize those parts while the agent is sent them raw. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at. A compile whose budget can't hold the raw pins beside the rest refuses with `OverBudgetError`; where the adaptive plan's middle is over the budget, its message and `diagnostics.rawPins` name each pin keeping raw tokens in that middle. `listPins` and `unpin` manage the pins. - `holdCompression` keeps messages out of compression until `releaseCompression`, or until an optional `timeoutMs` passes, for content you'll still edit. Holds live in memory, so a reopened manager starts with none, and a hold that's never released stalls compression of everything after it. - `searchSummaries`, `getSummary` and `getSummariesInRange` read the summaries, for memory tools an agent can call. - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index a638ea5e..512ac6f0 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,3 @@ - **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. - - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold, and that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. + - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index adae9989..5c717759 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -13001,9 +13001,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** * The pins that keep messages raw in the adaptive plan's middle, for its * refusal to name (`OverBudgetDiagnostics.rawPins`): the positions the plan - * folds history in (before the head, and from the head's end to the tail's - * start, which is the recent window's start or the head's end where the - * recent window reaches back into the head) that pins keep raw + * folds history in (before the tail's start, which is the recent window's + * start or the head's end where the recent window reaches back into the + * head, and outside the head window; with the head window disabled after a + * reset, the plan's tail holds the recent messages before the head) that + * pins keep raw * (`forceRawPinPositions`), each priced by `price`, the plan's own raw * price. Every such position counts, a part locked with `lockChunk` among * them, so the tokens are what unpinning frees at most: a pin that keeps no @@ -13019,7 +13021,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); const tailStart = Math.max(this.getRecentWindowStart(store), headEnd); - const kept = (i: number): boolean => raw.has(i) && (i < headStart || (i >= headEnd && i < tailStart)); + const kept = (i: number): boolean => raw.has(i) && i < tailStart && !(i >= headStart && i < headEnd); let tokens = 0; for (const i of raw) if (kept(i)) tokens += price(i); diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 307b4a2e..712cf6d1 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -130,7 +130,7 @@ function warnings(t: { mock: { method: (o: object, m: string, f: (...a: unknown[ return out; } -type View = { getAll(): Array<{ id: string; bodyGroupId?: string }> }; +type View = { getAll(): Array<{ id: string; bodyGroupId?: string }>; estimateTokens(message: unknown): number }; const viewOf = (cm: ContextManager): View => (cm as unknown as { strategyMessageView(): View }).strategyMessageView(); /** The ids of the parts of the message `id` is in, in store order. */ const partsOf = (cm: ContextManager, id: string): string[] => { @@ -557,6 +557,24 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th }); }); + it('with the head window disabled, a head reset leaves the recent messages before the head in the tail, so a pin there isn\'t named', async () => { + await withDir(async (dir) => { + const { cm, strategy } = await open(dir, 'adaptive', 8000, { headWindowTokens: 0 }); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + await traffic(cm, 360); + await ready(cm); + const mark = cm.markDocument(b); + const all = viewOf(cm).getAll(); + strategy.resetHeadWindow(all.at(-1)!.id); + cm.pinRange(all.at(-3)!.id, all.at(-3)!.id); + const refusal = await refusalOf(cm, 30_000); + assert.match(refusal.message, /^Adaptive picker exhausted/); + assert.deepStrictEqual(refusal.diagnostics.rawPins?.pins.map((p) => p.pinId), [mark]); + await cm.close(); + }); + }); + it('kv-stable: names a raw pin but not a pin with a level or a cap that overlaps it, which keeps nothing raw', async () => { await withDir(async (dir) => { const { cm, b } = await markedAfterSummaries(dir, 'kv-stable'); @@ -587,8 +605,8 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th await cm.compile(budget(maxTokens)); seen.push(`${maxTokens}: compiled`); } catch (err) { - seen.push(`${maxTokens}: ${(err as Error).message.slice(0, 120)}`); - if (!(err instanceof OverBudgetError)) continue; + if (!(err instanceof OverBudgetError)) throw err; + seen.push(`${maxTokens}: ${err.message.slice(0, 120)}`); raw = err.diagnostics.rawPins; if (raw) break; } @@ -596,6 +614,10 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th assert.ok(raw, `a refusal named the range: ${seen.join(' | ')}`); assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [range]); assert.strictEqual(raw.pins[0]!.tokens, raw.tokens, 'the range keeps raw only what the total counts'); + // B and C are the same size, so B alone is about half of both. + const ids = new Set([...partsOf(cm, b), ...partsOf(cm, c)]); + const both = viewOf(cm).getAll().filter((m) => ids.has(m.id)).reduce((n, m) => n + viewOf(cm).estimateTokens(m), 0); + assert.ok(raw.tokens < both * 0.6, `${raw.tokens} of ${both}: C, held at a level, isn't counted`); await cm.close(); }); }); From c22f2519817fa59484340ea5375ea128c61cf66f Mon Sep 17 00:00:00 2001 From: Ezra-2423 Date: Fri, 23 Oct 2026 20:40:35 -0400 Subject: [PATCH 47/67] docs, test(thinking-binding): the record's noted digests per part; refusals that refuse - BindingRecord.continued says a stored row gives one digest for each message the compile makes of it, and thinkingAsSent's doc names the set it takes. - The layout comment says a body the compile splits goes without whole from the next compile, since the record's without is per body. - messageDigest's doc states the cache-marker rule as observed in the run, without a claim about the provider's documentation. - Tests: the carrier flag's checks after a refused binding compile and dry compile now assert the refusal (the budget held the store before); the split-layout test gives the parts' thinking different prices and keeps a redacted block in the first part. Co-Authored-By: Claude Opus 5.5 --- src/context-manager.ts | 3 ++- src/thinking-binding.ts | 24 +++++++++++++----------- test/thinking-binding.test.ts | 16 +++++++++++----- 3 files changed, 26 insertions(+), 17 deletions(-) diff --git a/src/context-manager.ts b/src/context-manager.ts index fb1eefc0..1e439955 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1274,7 +1274,8 @@ export class ContextManager { } // Each entry keeps the thinking the compile sent of it: an entry the // compile splits (a bundled tool cycle) is bound part by part, so one - // part can keep what another leaves out. + // part can keep what another leaves out. The record's `without` is + // per body, so from the next compile such a body goes without whole. const sent = thinkingSent(result.messages); layoutEntries = entries.map((entry) => { const content = thinkingAsSent(entry.content, sent); diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts index da684e79..f9354e17 100644 --- a/src/thinking-binding.ts +++ b/src/thinking-binding.ts @@ -46,9 +46,10 @@ export function withoutThinking(content: ContentBlock[]): ContentBlock[] { } /** - * The content less the thinking blocks `messages` don't carry, raw forms - * released as `withoutThinking` releases them; the content itself when they - * carry all of its thinking. A block is matched by its canonical form, cache + * The content less the thinking blocks `sent` doesn't hold (the canonical + * forms `thinkingSent` takes from the compiled messages), raw forms released + * as `withoutThinking` releases them; the content itself when `sent` holds + * all of its thinking. A block is matched by its canonical form, cache * marker aside: a signature makes each one distinct. */ export function thinkingAsSent(content: ContentBlock[], sent: ReadonlySet): ContentBlock[] { @@ -75,8 +76,9 @@ export interface BindingRecord { * note came with another head than `identity`: then the record vouches * for nothing until the next committing compile. */ sent: string[]; - /** The rows a host noted after those messages, in order: each a stored - * body's digest as the strategy renders it among the newest, or + /** What a host noted after those messages, in order: for each stored + * row, a digest of each message the compile makes of it as the strategy + * renders it among the newest (a bundled tool cycle makes several), or * `UNVOUCHED` for a row the next compile can't reproduce. Empty until a * note. */ continued: string[]; @@ -183,12 +185,12 @@ function isBindingRecord(value: unknown): value is BindingRecord { * order, so a block read back from the store (keys sorted) and the same block * as built compare equal, as the provider reads them. A cache breakpoint isn't * part of it, nor a block's own cache marker (`cache_control`, - * `cacheControl`): the binding doesn't count them. Anthropic's docs don't - * say so; in a run on Anthropic's API on 2026-10-23 (Haiku 5.5 with the - * drop control, on an account that enforces the binding and on one that - * doesn't), moving only a cache marker, onto or off a block before two - * replies, dropped neither reply's thinking, where changing one word - * dropped both. Other binding models weren't run for markers. 66 bits of + * `cacheControl`): the binding doesn't count them, as observed in a run on + * Anthropic's API on 2026-10-23 (Haiku 5.5 with the drop control, on an + * account that enforces the binding and on one that doesn't): moving only a + * cache marker, onto or off a block before two replies, dropped neither + * reply's thinking, where changing one word dropped both. Other binding + * models weren't run for markers. 66 bits of * SHA-256 are kept: a digest is only ever compared with the one recorded at * the same place. */ diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts index a33cb745..21925f9f 100644 --- a/test/thinking-binding.test.ts +++ b/test/thinking-binding.test.ts @@ -627,7 +627,11 @@ describe('thinking binding: compile', () => { const manager = await open(); manager.addMessage('User', [text('U1')]); const c1 = await manager.compile(BIG, undefined, BIND_DROP); - const b1 = manager.addMessage('Claude', [thought(1), toolUse('t1'), toolResult('t1', 'zz-bundled'), thought(2), text('B1')]); + // The first part's thinking includes a redacted block; the second part's + // costs more than the first's, so the price says which part kept its own. + const redacted = { type: 'redacted_thinking', data: 'zz-redacted-payload==' } as ContentBlock; + const long = { type: 'thinking', thinking: 'zz-long '.repeat(80), signature: 'zz-sig-long-0123456789' } as ContentBlock; + const b1 = manager.addMessage('Claude', [thought(1), redacted, toolUse('t1'), toolResult('t1', 'zz-bundled'), long, text('B1')]); manager.noteRequestSent([b1], identity(), c1.provenance!.compileId); // Placed after the cycle's result, the injection changes what precedes // the second part, so its thinking goes and the first part's stays. @@ -635,9 +639,10 @@ describe('thinking binding: compile', () => { const plain = await manager.compile(BIG, [late], { dryRun: true }); const bound = await manager.compile(BIG, [late], { ...BIND_DROP, dryRun: true }); const kept = bound.messages.flatMap((m) => m.content.filter(isThinking)); - assert.deepEqual(kept, [thought(1)], 'setup: the first part keeps its thinking, the second goes without'); + assert.deepEqual(kept, [thought(1), redacted], 'setup: the first part keeps its thinking, the second goes without'); const price = (blocks: ContentBlock[]) => (manager as unknown as { messageStore: { estimateBaseTokens(c: ContentBlock[]): number } }).messageStore.estimateBaseTokens(blocks); - assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([thought(2)])); + assert.ok(price([long]) > price([thought(1), redacted]), 'setup: the parts\u2019 thinking prices differ'); + assert.equal(bound.provenance!.layout!.totalTokens, plain.provenance!.layout!.totalTokens - price([long])); await manager.close(); }); @@ -1178,9 +1183,10 @@ describe('thinking binding: a summary\u2019s carriers', () => { assert.deepEqual(after, [full, full, full, full, full], 'outside a select, as configured, after a binding one too'); // And after a binding select that throws, committing or dry. const refusing = { maxTokens: 1, reserveForResponse: 0 }; - await manager.compile(refusing, undefined, BIND).catch(() => undefined); + manager.addMessage('User', [text('zz-over ' + 'w '.repeat(400))]); + await assert.rejects(manager.compile(refusing, undefined, BIND), 'setup: the binding compile is refused'); assert.equal(price(strategy), full, 'after a refused binding compile'); - await manager.compile(refusing, undefined, { ...BIND, dryRun: true }).catch(() => undefined); + await assert.rejects(manager.compile(refusing, undefined, { ...BIND, dryRun: true }), 'setup: the binding dry compile is refused'); assert.equal(price(strategy), full, 'after a refused binding dry compile'); await manager.close(); await reference.close(); From 6dd2a822ed3add2f30cec6fd1abbbed363e74d00 Mon Sep 17 00:00:00 2001 From: Ezra-2423 Date: Fri, 23 Oct 2026 20:59:45 -0400 Subject: [PATCH 48/67] fix(thinking-binding): a compile without binds touches no binding state; a record the store refuses never fails a compile A committing compile without binds cleared the branch's record, so every host's compile read that state, and a store whose read throws failed a compile that passes on main (agent-framework's "a record that cannot be read is logged" test). - A compile without binds neither reads nor writes the record. A record an earlier binding compile left is compared at the next binding compile like any other: a message keeps its thinking only while everything up to it matches what that compile sent, a prefix the provider took with that thinking, and nothing past the record's end keeps it, so clearing it bought no safety. - With binds, a record the store can't read is logged and read as none, under the first-compile rule; one it can't write is logged, and the compile stands with the record before it, which its stream's notes can't extend. noteRequestSent logs either and returns false. - Docs: ThinkingBinding.binds, SelectOptions.thinkingBinding, noteRequestSent and the changelog. - Tests: a compile without binds over a store that refuses the record state, then a binding compile against the earlier record; a store that can't read or write the record, at a compile and at a note. Co-Authored-By: Claude Opus 5.5 --- changelog.d/155-thinking-binding.added.md | 7 +- src/context-manager.ts | 60 ++++++++++---- src/thinking-binding.ts | 7 -- src/types/strategy.ts | 15 ++-- test/thinking-binding.test.ts | 97 ++++++++++++++++------- 5 files changed, 127 insertions(+), 59 deletions(-) diff --git a/changelog.d/155-thinking-binding.added.md b/changelog.d/155-thinking-binding.added.md index bb6eb6a4..c0b89334 100644 --- a/changelog.d/155-thinking-binding.added.md +++ b/changelog.d/155-thinking-binding.added.md @@ -2,12 +2,13 @@ - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message the last committing compile sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. - - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. A record it can't read is logged and read as none. + - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. - **A dry compile** reports what it would leave out and writes no record. - - **Without `binds`,** a compile is as before, except that a committing one clears the branch's record, since what it sends no record vouches for. + - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent keeps its thinking, and nothing past its end does. + - **A record that can't be read or written** is logged and never fails a compile; one that can't be read is read as none. - **While a request is streaming,** compile for its branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing, so its replies go without their thinking. - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. -- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read, or when the store can't write it. - `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the branch's last committing compile, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. - `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. - On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. diff --git a/src/context-manager.ts b/src/context-manager.ts index 1e439955..118e28f6 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -61,7 +61,7 @@ import { attributeEntries, buildRenderedLayout, rawBodiesOf } from './compile-pr import { FoldJournal, branchRefOf, cachedStoreIdentity, existingStoreIdentity } from './fold-journal.js'; import { describeThrown, logThrown } from './describe-thrown.js'; import { - UNVOUCHED, bindThinking, clearBindingRecord, continueRecord, isThinkingBlock, markThinkingLeftOut, messageDigest, readBindingRecord, + UNVOUCHED, bindThinking, continueRecord, isThinkingBlock, markThinkingLeftOut, messageDigest, readBindingRecord, refuseUnboundBinding, refuseUnreadableIdentity, registerBindingState, thinkingAsSent, thinkingSent, withoutThinking, writeBindingRecord, type BindOutcome, type BindingRecord, } from './thinking-binding.js'; @@ -1299,10 +1299,12 @@ export class ContextManager { // compile leaves behind for the next one. if (opts?.dryRun !== true) { if (binding) { - // The store write is the compile's last step that can fail. The + // Nothing here fails the compile: a record it can't write leaves + // the one before, which this compile's stream can't note onto, so + // what it sends past that record goes without its thinking. The // strategy's note only adjusts its calibration, so a strategy that - // throws there is logged and the compile stands. - writeBindingRecord(this.store, this.strategyNamespace, binding.record); + // throws there is logged and the compile stands too. + this.writeBindingRecordFor(binding.record, 'the next compile compares against the record before it'); if (binding.leftOut.length > 0) { try { this.strategy.noteThinkingLeftOut?.(binding.leftOut); @@ -1310,11 +1312,12 @@ export class ContextManager { this.diag.error('[context-manager] the strategy failed to take note of thinking left out:', describeThrown(err)); } } - } else { - // What it sends, no record vouches for: the next binding compile - // starts a record, as a branch's first does. - clearBindingRecord(this.store, this.strategyNamespace); } + // A compile without binds neither reads nor writes the record. One an + // earlier binding compile left is compared at the next binding compile + // like any other: a message keeps its thinking only while everything + // up to it matches what that compile sent, a prefix the provider took + // with that thinking, and nothing past the record's end keeps it. } return result; }; @@ -1726,8 +1729,8 @@ export class ContextManager { * wrote the branch's record, as for a stream whose note arrives after * another committing compile: its replies were minted under a prefix the * record no longer holds, and go without their thinking. So it does when - * the branch has no record, as after a committing compile without - * `thinkingBinding.binds`, or one this version can't read, which it logs. + * the branch has no record, or one this version can't read or the store + * can't give, and when the store can't write it; it logs each of those. * A dry compile writes no record, so it leaves a stream's notes as they * were. Throws, writing nothing, for a row that isn't an id or the marker, * a `compileId` that isn't a string, or a digest that isn't a string. @@ -1777,20 +1780,45 @@ export class ContextManager { // its parts. for (const part of splitMixedToolMessages([{ participant: stored.participant, content }])) continued.push(messageDigest(part)); } - writeBindingRecord(this.store, this.strategyNamespace, continueRecord(record, continued, identity)); - return true; + return this.writeBindingRecordFor(continueRecord(record, continued, identity), 'this note is lost, so what it named goes without its thinking'); } /** The branch's binding record; one this version can't read is logged, with what follows from it, and read as none. */ private readBindingRecordFor(consequence: string): BindingRecord | null { - const { record, malformed } = readBindingRecord(this.store, this.strategyNamespace); - if (malformed) { + let read: ReturnType; + try { + read = readBindingRecord(this.store, this.strategyNamespace); + } catch (err) { + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} couldn't be read (${describeThrown(err)}); ${consequence}`); + return null; + } + if (read.malformed) { this.diag.warn( - `[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} isn't ` + + `[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} isn't ` + `one this version reads; ${consequence}`, ); } - return record; + return read.record; + } + + /** Write the branch's binding record; a write that throws is logged, with what follows from it, and false. */ + private writeBindingRecordFor(record: BindingRecord, consequence: string): boolean { + try { + writeBindingRecord(this.store, this.strategyNamespace, record); + return true; + } catch (err) { + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} couldn't be written (${describeThrown(err)}); ${consequence}`); + return false; + } + } + + /** The current branch's name for a log line, or '?' when the store can't say. */ + private branchNameForLog(): string { + try { + return this.store.currentBranch().name; + } catch { + return '?'; + } } /** Read the active strategy's allowlisted live settings, if supported. */ diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts index f9354e17..03625420 100644 --- a/src/thinking-binding.ts +++ b/src/thinking-binding.ts @@ -155,13 +155,6 @@ export function writeBindingRecord(store: JsStore, namespace: string, record: Bi store.setStateJson(bindingStateId(namespace), record); } -/** Clear the branch's record, whatever it holds, malformed included. A - * branch with none is left unwritten. */ -export function clearBindingRecord(store: JsStore, namespace: string): void { - const value = store.getStateJson(bindingStateId(namespace)) as unknown; - if (value !== null && value !== undefined) store.setStateJson(bindingStateId(namespace), null); -} - function isStringArray(value: unknown): value is string[] { return Array.isArray(value) && value.every((item) => typeof item === 'string'); } diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 9dd103bb..9cae1464 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -330,8 +330,8 @@ export interface ContextStrategy { * See `SelectOptions.thinkingBinding`. */ export interface ThinkingBinding { - /** The model binds thinking. Absent or false: the compile goes on without - * the option, and clears the branch's record if it commits. */ + /** The model binds thinking. Absent or false: the compile is as it would be + * without the option, and neither reads nor writes the branch's record. */ binds?: boolean; /** The request carries the provider's drop control, so a stale block is * set aside rather than refused. Absent means false. */ @@ -410,9 +410,14 @@ export interface SelectOptions { * it was minted, and reports it in `CompileResult.thinkingStrip`. With * `binds`, `participant` and `identity` are required, and a compile * without them is refused before anything is selected. Without `binds`, - * the compile is as it would be without the option, except that a - * committing one clears the branch's record: what it sends, no record - * vouches for. + * the compile is as it would be without the option: it neither reads nor + * writes the branch's record. A record an earlier binding compile left is + * compared at the next binding compile like any other, so a message keeps + * its thinking only while everything up to it matches what that compile + * sent, a prefix the provider took with that thinking, and nothing past + * the record's end keeps it. A record that can't be read or written is + * logged and never fails a compile; one that can't be read is read as + * none. * * The manager keeps a record per branch of what the branch last sent: the * last committing compile's messages as sent (cache breakpoints aside, diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts index 21925f9f..2f1d74fb 100644 --- a/test/thinking-binding.test.ts +++ b/test/thinking-binding.test.ts @@ -759,23 +759,35 @@ describe('thinking binding: compile', () => { await manager.close(); }); - it('a committing compile without binds clears the record, and a dry one leaves it', async () => { + it('a compile without binds neither reads nor writes the record; one an earlier binding compile left is compared like any other', async () => { cleanup(); const manager = await open(); const { a1, a2 } = seed(manager); await manager.compile(BIG, undefined, BIND_DROP); - await manager.compile(BIG, undefined, { dryRun: true }); - assert.notEqual(recordOf(manager), null, 'a dry compile without binds leaves the record'); + const recorded = JSON.stringify(recordOf(manager)); + // A store that can't give or take the record state doesn't touch a + // compile without the option, committing or dry. + const store = manager.getStore() as unknown as { getStateJson(id: string): unknown; setStateJson(id: string, value: unknown): void }; + const get = store.getStateJson.bind(store); + const set = store.setStateJson.bind(store); + let touched = 0; + store.getStateJson = (id: string) => { if (id === 'default/thinking-binding') { touched++; throw new Error('zz-store-unreadable'); } return get(id); }; + store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') { touched++; throw new Error('zz-store-unwritable'); } set(id, value); }; await manager.compile(BIG); - assert.equal(recordOf(manager), null, 'a committing one clears it'); - // Back on a binding model, the replies stored meanwhile were minted under - // requests no record saw. - const back = await manager.compile(BIG, undefined, BIND); - assert.deepEqual(back.thinkingStrip, { newly: [a1, a2], cause: 'first' }); - // A record it can't read is cleared too. - manager.getStore().setStateJson('default/thinking-binding', { sent: 'zz-not-a-list' }); + await manager.compile(BIG, undefined, { dryRun: true }); + manager.addMessage('User', [text('U3')]); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); await manager.compile(BIG); - assert.equal(recordOf(manager), null, 'a malformed record is cleared'); + store.getStateJson = get; + store.setStateJson = set; + assert.equal(touched, 0, 'no read or write of the record'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record is as the binding compile left it'); + // Back on a binding model, the record is compared as it stands: the + // prefix that matches keeps its thinking, and what came after goes without. + const back = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(back), { U1: false, A1: true, U2: false, A2: true, U3: false, A3: false }); + assert.deepEqual(back.thinkingStrip, { newly: [a3], cause: 'messages' }); + void a1; void a2; await manager.close(); }); @@ -974,24 +986,53 @@ describe('thinking binding: compile', () => { await manager.close(); }); - it('a record write that fails tells the strategy nothing', async () => { - cleanup(); - const told: ContentBlock[][] = []; - class Listening extends PassthroughStrategy { - noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + it('a record the store can\u2019t read or write is logged, and never fails a compile or a note', async () => { + for (const fails of ['write', 'read'] as const) { + cleanup(); + const told: ContentBlock[][] = []; + class Listening extends PassthroughStrategy { + noteThinkingLeftOut(blocks: readonly ContentBlock[]): void { told.push([...blocks]); } + } + const manager = await open(STORE, new Listening()); + const { a1, a2 } = seed(manager); + const store = manager.getStore() as unknown as { getStateJson(id: string): unknown; setStateJson(id: string, value: unknown): void }; + const get = store.getStateJson.bind(store); + const set = store.setStateJson.bind(store); + const failing = () => { + if (fails === 'read') store.getStateJson = (id: string) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unreadable'); return get(id); }; + else store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unwritable'); set(id, value); }; + }; + const warned: string[] = []; + const warn = console.warn; + console.warn = (message: string) => { warned.push(message); }; + try { + failing(); + // Read as no record, the compile strips as a branch's first does. + const result = await manager.compile(BIG, undefined, BIND); + assert.deepEqual(result.thinkingStrip, { newly: [a1, a2], cause: 'first' }, fails); + assert.equal(told.length, 1, `${fails}: the strategy is told what was left out of what was sent`); + assert.equal(warned.length, 1, fails); + assert.match(warned[0]!, fails === 'read' ? /thinking-binding record of branch main couldn't be read \(.*zz-store-unreadable/ : /thinking-binding record of branch main couldn't be written \(.*zz-store-unwritable/, fails); + store.getStateJson = get; + store.setStateJson = set; + // A note can't read or write it either: it writes nothing and says so. + const compiled = await manager.compile(BIG, undefined, BIND); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + const before = JSON.stringify(recordOf(manager)); + failing(); + warned.length = 0; + assert.equal(manager.noteRequestSent([a3], identity(), compiled.provenance!.compileId), false, fails); + assert.equal(warned.length, 1, `${fails}: the note is logged`); + store.getStateJson = get; + store.setStateJson = set; + assert.equal(JSON.stringify(recordOf(manager)), before, `${fails}: the record is as it was`); + } finally { + console.warn = warn; + store.getStateJson = get; + store.setStateJson = set; + } + await manager.close(); } - const manager = await open(STORE, new Listening()); - seed(manager); - const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; - const set = store.setStateJson.bind(store); - store.setStateJson = (id: string, value: unknown) => { - if (id === 'default/thinking-binding') throw new Error('zz-write-failed'); - set(id, value); - }; - await assert.rejects(manager.compile(BIG, undefined, BIND), /zz-write-failed/); - assert.deepEqual(told, []); - store.setStateJson = set; - await manager.close(); }); it('a compile that fails after selecting writes no record and tells the strategy nothing', async () => { From 30a79177e0933365e55a9b7624ab231cf7e3776f Mon Sep 17 00:00:00 2001 From: Ezra-2423 Date: Fri, 23 Oct 2026 21:14:16 -0400 Subject: [PATCH 49/67] docs, test(thinking-binding): the record is a binding compile's; a note after a compile without binds still writes Since a compile without binds touches no binding state, the record is the last committing compile with binds whose write the store took. - Docs, README and changelog: the record, thinkingStrip.newly and the streaming advice name a committing compile with binds; a note after a compile without binds still writes; the record's continued rows are part of what keeps thinking; a record the store can't read is replaced by the next committing compile with binds, and one it can't write leaves the record before it, if any. - Log lines say what each store failure costs: a lost note's rows go without their thinking unless a later note writes, and a compile whose record wasn't written can't have its stream noted. - noteRequestSent's doc says which refusals are logged. - writeBindingRecord takes a record only, since nothing clears it; the log lines name the branch directly, as a compile already reads it. - Tests: a note after a committing compile without binds writes, and its reply keeps its thinking; a compile whose record write fails, with a record before it, leaves that record, refuses its own stream's note, and the next compile compares against the earlier record. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/155-thinking-binding.added.md | 12 ++-- src/context-manager.ts | 70 ++++++++++++----------- src/thinking-binding.ts | 23 ++++---- src/types/context.ts | 10 ++-- src/types/strategy.ts | 29 ++++++---- test/thinking-binding.test.ts | 40 ++++++++++++- 7 files changed, 116 insertions(+), 70 deletions(-) diff --git a/README.md b/README.md index 1f4d2e1e..06a725ae 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ With a strategy that supports them, as this one does, the manager also offers: - `setToolDefinitions` and `setSystemPrompt` give the summarizer's calls the agent's own tools and system prompt. A chunk whose messages hold tool calls or results waits to be compressed until tool definitions are set. - `getRenderStats` reports how much of the last compile was raw and how much was summaries, with counts for levels 1 to 3. It's `null` until a live compile has run since the strategy loaded its branch, which opening the store, `setStrategy()` and a branch switch each do; a dry compile or preview leaves it as it was. Each compile's result carries its own as `renderStats`, so a dry compile (`dryRun`) shows the composition of the request it built without sending it. -For a model that binds each signed thinking block to the request it was minted in, pass `{ thinkingBinding: { binds: true, dropBlock, participant }, identity }` as compile's third argument, with `participant` the name the model's own replies are stored under and `identity` digests of the request's system text, tools, model and the rest of its head. The compile then leaves out the thinking such a model can no longer take, against a per-branch record of the last request the branch sent, and says what it newly left out in `thinkingStrip`. Once the branch has a record, a reply keeps its thinking at the compile after it's stored only if the host has noted it with `noteRequestSent(rows, identity, compileId)`, naming the `provenance.compileId` of the compile its stream was built from: as it sends each later round of a stream, the ids of the stored messages the request carried after the compiled ones, every round's so far, in order, and once the stream has ended, those and its last reply's, with `{ unstored: true }` for a row it can't vouch was carried or returned as stored. A host that doesn't call it loses each reply's thinking at the compile after it's stored, and is never refused for it. While one of a branch's requests is still streaming, compile for that branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing (see `SelectOptions.thinkingBinding` in `src/types/strategy.ts`). +For a model that binds each signed thinking block to the request it was minted in, pass `{ thinkingBinding: { binds: true, dropBlock, participant }, identity }` as compile's third argument, with `participant` the name the model's own replies are stored under and `identity` digests of the request's system text, tools, model and the rest of its head. The compile then leaves out the thinking such a model can no longer take, against a per-branch record of the last request the branch sent, and says what it newly left out in `thinkingStrip`. Once the branch has a record, a reply keeps its thinking at the compile after it's stored only if the host has noted it with `noteRequestSent(rows, identity, compileId)`, naming the `provenance.compileId` of the compile its stream was built from: as it sends each later round of a stream, the ids of the stored messages the request carried after the compiled ones, every round's so far, in order, and once the stream has ended, those and its last reply's, with `{ unstored: true }` for a row it can't vouch was carried or returned as stored. A host that doesn't call it loses each reply's thinking at the compile after it's stored, and is never refused for it. While one of a branch's requests is still streaming, compile for that branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing (see `SelectOptions.thinkingBinding` in `src/types/strategy.ts`). Every option is documented on `AutobiographicalConfig` in `src/types/strategy.ts`. diff --git a/changelog.d/155-thinking-binding.added.md b/changelog.d/155-thinking-binding.added.md index c0b89334..e0803f1d 100644 --- a/changelog.d/155-thinking-binding.added.md +++ b/changelog.d/155-thinking-binding.added.md @@ -1,14 +1,14 @@ - `compile()` takes `thinkingBinding` and `identity` (`SelectOptions`) for a model that binds each signed thinking block to the request it was minted in (#155). On an account that enforces the binding, such a model refuses a request carrying a block whose request has changed since, with a 400 ("Invalid `signature` in `thinking` block. The block is bound to a different conversation…"), and with the provider's drop control on it sets the block aside instead. With `thinkingBinding.binds`, a compile leaves such blocks out. - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. - - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message the last committing compile sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. + - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message its last committing compile with `binds` sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. - **A dry compile** reports what it would leave out and writes no record. - - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent keeps its thinking, and nothing past its end does. - - **A record that can't be read or written** is logged and never fails a compile; one that can't be read is read as none. - - **While a request is streaming,** compile for its branch only with `dryRun`: a committing compile makes that stream's later notes vouch for nothing, so its replies go without their thinking. + - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent and its stream noted keeps its thinking, and nothing past the record's end does. + - **A record the store can't read or write** is logged and never fails a compile or a note. One it can't read is read as none, so a committing compile with `binds` writes its own in its place; when it can't write a compile's record, the record before it, if any, stands. + - **While a request is streaming,** compile for its branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing, so its replies go without their thinking. - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. -- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until the next committing compile; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile, and on a branch with no record or one it can't read, or when the store can't write it. -- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the branch's last committing compile, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until a compile writes it again; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile with `binds` (a compile without it leaves the record, so a later note still writes), and on a branch with no record or one it can't read, or when the store can't write it. +- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the compile that wrote the branch's record, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. - `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. - On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. diff --git a/src/context-manager.ts b/src/context-manager.ts index 118e28f6..a4084327 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -1300,11 +1300,14 @@ export class ContextManager { if (opts?.dryRun !== true) { if (binding) { // Nothing here fails the compile: a record it can't write leaves - // the one before, which this compile's stream can't note onto, so - // what it sends past that record goes without its thinking. The - // strategy's note only adjusts its calibration, so a strategy that - // throws there is logged and the compile stands too. - this.writeBindingRecordFor(binding.record, 'the next compile compares against the record before it'); + // the one before, if any, which this compile's stream can't note + // onto, so what it sends past that record goes without its + // thinking. The strategy's note only adjusts its calibration, so a + // strategy that throws there is logged and the compile stands too. + this.writeBindingRecordFor( + binding.record, + 'this compile\'s stream can\'t note onto the record before it, if any, so its replies go without their thinking until a compile with binds writes the record', + ); if (binding.leftOut.length > 0) { try { this.strategy.noteThinkingLeftOut?.(binding.leftOut); @@ -1316,8 +1319,9 @@ export class ContextManager { // A compile without binds neither reads nor writes the record. One an // earlier binding compile left is compared at the next binding compile // like any other: a message keeps its thinking only while everything - // up to it matches what that compile sent, a prefix the provider took - // with that thinking, and nothing past the record's end keeps it. + // up to it matches what that compile sent and its stream noted, a + // prefix the provider took with that thinking, and nothing past the + // record's end keeps it. } return result; }; @@ -1403,7 +1407,7 @@ export class ContextManager { opts: SelectOptions, compileId: string, ): BindOutcome { - const record = this.readBindingRecordFor('this compile goes on as if the branch had no record'); + const record = this.readBindingRecordFor('this compile goes on as if the branch had no record, and if it commits, its record takes that one\'s place'); const outcome = bindThinking({ messages: result.messages, sources, @@ -1689,7 +1693,8 @@ export class ContextManager { } /** - * Note what a stream built from this branch's last committing compile sent + * Note what a stream built from this branch's last committing compile with + * `binds` sent * after the compiled messages (see `SelectOptions.thinkingBinding`): as it * sends each later round, every row its request carried so far, and once * the stream has ended, those rows and its last reply. Call it on the @@ -1709,8 +1714,8 @@ export class ContextManager { * next request won't carry where this one did. A request-only row the * host puts back in the same place at every later request needs no row, * since no compile holds it. A note sent with another head than the - * record's leaves the record vouching for nothing until the next - * committing compile: the stream's earlier thinking was minted under the + * record's leaves the record vouching for nothing until a compile writes + * it again: the stream's earlier thinking was minted under the * old head. A host that notes nothing keeps no reply's thinking past the * compile after it was stored, and is never refused for it. * @@ -1727,10 +1732,13 @@ export class ContextManager { * * Returns false and writes nothing when `compileId` isn't the compile that * wrote the branch's record, as for a stream whose note arrives after - * another committing compile: its replies were minted under a prefix the - * record no longer holds, and go without their thinking. So it does when - * the branch has no record, or one this version can't read or the store - * can't give, and when the store can't write it; it logs each of those. + * another committing compile with `binds`: its replies were minted under a + * prefix the record no longer holds, and go without their thinking. A + * compile without `binds` leaves the record, so a note after one still + * writes. It returns false too when the branch has no record, and, logging + * it, when the record is one this version can't read, or the store can't + * read or write it; a note it can't write is lost, and the record stands + * as the last one written. * A dry compile writes no record, so it leaves a stream's notes as they * were. Throws, writing nothing, for a row that isn't an id or the marker, * a `compileId` that isn't a string, or a digest that isn't a string. @@ -1749,10 +1757,10 @@ export class ContextManager { `noteRequestSent: \`compileId\` must be a string, the provenance.compileId of the compile the stream was built from; got ${typeof compileId}`, ); } - const record = this.readBindingRecordFor('this note writes nothing, and the branch\'s next committing compile replaces it'); + const record = this.readBindingRecordFor('this note writes nothing, and the branch\'s next committing compile with binds replaces the record'); // A note is for the stream of the compile that wrote the record; after - // another committing compile its replies were minted under a prefix the - // record no longer holds. + // another committing compile with binds its replies were minted under a + // prefix the record no longer holds. if (record === null || record.compileId !== compileId) return false; const view = this.mergedMessageView(); // An id the store doesn't hold, as after a removal, is nothing the next @@ -1780,21 +1788,28 @@ export class ContextManager { // its parts. for (const part of splitMixedToolMessages([{ participant: stored.participant, content }])) continued.push(messageDigest(part)); } - return this.writeBindingRecordFor(continueRecord(record, continued, identity), 'this note is lost, so what it named goes without its thinking'); + return this.writeBindingRecordFor( + continueRecord(record, continued, identity), + 'this note is lost: the record stands as the last one written, so the rows this note adds go without their thinking unless a later note of the stream writes', + ); } - /** The branch's binding record; one this version can't read is logged, with what follows from it, and read as none. */ + /** + * The branch's binding record. One this version doesn't read, or that the + * store can't give, is logged, with what follows from it, and read as + * none; a committing compile with `binds` then writes its own in its place. + */ private readBindingRecordFor(consequence: string): BindingRecord | null { let read: ReturnType; try { read = readBindingRecord(this.store, this.strategyNamespace); } catch (err) { - this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} couldn't be read (${describeThrown(err)}); ${consequence}`); + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} couldn't be read (${describeThrown(err)}); ${consequence}`); return null; } if (read.malformed) { this.diag.warn( - `[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} isn't ` + + `[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} isn't ` + `one this version reads; ${consequence}`, ); } @@ -1807,20 +1822,11 @@ export class ContextManager { writeBindingRecord(this.store, this.strategyNamespace, record); return true; } catch (err) { - this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.branchNameForLog()} couldn't be written (${describeThrown(err)}); ${consequence}`); + this.diag.warn(`[context-manager] the thinking-binding record of branch ${this.store.currentBranch().name} couldn't be written (${describeThrown(err)}); ${consequence}`); return false; } } - /** The current branch's name for a log line, or '?' when the store can't say. */ - private branchNameForLog(): string { - try { - return this.store.currentBranch().name; - } catch { - return '?'; - } - } - /** Read the active strategy's allowlisted live settings, if supported. */ getHotContextSettings(): HotContextSettingsStatus | null { this.requireInitialized('getHotContextSettings'); diff --git a/src/thinking-binding.ts b/src/thinking-binding.ts index 03625420..64e800c2 100644 --- a/src/thinking-binding.ts +++ b/src/thinking-binding.ts @@ -71,10 +71,10 @@ function thinkingKey(block: ContentBlock): string { /** What a branch last sent, as its record keeps it. */ export interface BindingRecord { - /** Each message the last committing compile sent, in order: a digest of - * its participant and content (`messageDigest`). `[UNVOUCHED]` once a - * note came with another head than `identity`: then the record vouches - * for nothing until the next committing compile. */ + /** Each message the compile that wrote the record sent, in order: a + * digest of its participant and content (`messageDigest`). `[UNVOUCHED]` + * once a note came with another head than `identity`: then the record + * vouches for nothing until a compile writes it again. */ sent: string[]; /** What a host noted after those messages, in order: for each stored * row, a digest of each message the compile makes of it as the strategy @@ -82,10 +82,9 @@ export interface BindingRecord { * `UNVOUCHED` for a row the next compile can't reproduce. Empty until a * note. */ continued: string[]; - /** The stored bodies the last committing compile sent without their - * thinking. */ + /** The stored bodies that compile sent without their thinking. */ without: MessageId[]; - /** The head digests the last committing compile sent. */ + /** The head digests that compile sent. */ identity: RequestIdentity; /** The branch's sequence (`JsStore.currentSequence`) when the record * began: a message at or below it was stored before. */ @@ -151,7 +150,7 @@ export function readBindingRecord(store: JsStore, namespace: string): { record: return isBindingRecord(value) ? { record: value, malformed: false } : { record: null, malformed: true }; } -export function writeBindingRecord(store: JsStore, namespace: string, record: BindingRecord | null): void { +export function writeBindingRecord(store: JsStore, namespace: string, record: BindingRecord): void { store.setStateJson(bindingStateId(namespace), record); } @@ -330,12 +329,12 @@ function copyIdentity(identity: RequestIdentity): RequestIdentity { } /** - * The record once a host has noted the rows it sent after the last - * committing compile's messages (`continued`, each a digest or `UNVOUCHED`), + * The record once a host has noted the rows it sent after the messages of + * the compile that wrote it (`continued`, each a digest or `UNVOUCHED`), * with the head `identity`. A note with another head than the record's * carried thinking minted under the old one: the record keeps its head and - * vouches for nothing until the next committing compile, so the next - * compile under the new head reports the head change. + * vouches for nothing until a compile writes it again, so the next compile + * under the new head reports the head change. */ export function continueRecord(record: BindingRecord, continued: string[], identity: RequestIdentity): BindingRecord { const sameHead = IDENTITY_KEYS.every((key) => identity[key] === record.identity[key]); diff --git a/src/types/context.ts b/src/types/context.ts index 38718cc3..3de7d7d8 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -214,10 +214,12 @@ export interface CompileResult { export interface ThinkingStrip { /** * The stored bodies (`RawBodySource.messageId`) of the participant's - * replies whose thinking this compile leaves out and the branch's last - * committing compile didn't, whether or not its request went out: that - * compile sent them with their thinking, they were stored since, or it - * didn't send them as stored replies at all. + * replies whose thinking this compile leaves out and the compile that + * wrote the branch's record didn't, whether or not its request went out: + * that compile sent them with their thinking, they were stored since, or + * it didn't send them as stored replies at all. A record is written by a + * committing compile with `binds`, so a compile without it is passed + * over. * So a reply can be reported again: one whose thinking went, that a later * compile didn't send, and that renders again. A host that tells each * loss once keeps its own set of what it has told. diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 9cae1464..0f622cb5 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -415,13 +415,17 @@ export interface SelectOptions { * compared at the next binding compile like any other, so a message keeps * its thinking only while everything up to it matches what that compile * sent, a prefix the provider took with that thinking, and nothing past - * the record's end keeps it. A record that can't be read or written is - * logged and never fails a compile; one that can't be read is read as - * none. + * the record's end keeps it, unless a host noted it after that compile. + * A record the store can't read or write is logged, and never fails a + * compile or a note. One it can't read is read as none, so a committing + * compile starts a record in its place, as for a record this version + * doesn't read. When it can't write a compile's record, the record before + * it, if any, stands, and that compile's stream can't note onto it. * * The manager keeps a record per branch of what the branch last sent: the - * last committing compile's messages as sent (cache breakpoints aside, - * injections included), then the rows a host noted after them + * messages its last committing compile with `binds` sent (cache + * breakpoints aside, injections included), then the rows a host noted + * after them * (`ContextManager.noteRequestSent`): what its stream's requests went on * to carry and, once the stream ended, its last reply. With them it keeps * the stored bodies the compile sent without their thinking, the @@ -444,9 +448,9 @@ export interface SelectOptions { * a branch with no record, its own or (on a branch made since) the one it * inherits as of its branch point, leaves out every reply's thinking * without `dropBlock`; with `dropBlock` it leaves the stored replies' - * thinking as it is. Either way it starts the record. Each committing compile writes - * the record; a dry compile reports what it would leave out and writes no - * record. The record survives a restart. + * thinking as it is. Either way it starts the record. Each committing + * compile with `binds` writes the record; a dry compile reports what it + * would leave out and writes no record. The record survives a restart. * * So once a branch has a record, a reply keeps its thinking at the compile * after it's stored only if the host has noted it: each later round of a @@ -454,10 +458,11 @@ export interface SelectOptions { * stored. A host that passes `thinkingBinding` and notes nothing loses * each reply's thinking at the compile after it's stored, and is never * refused for it. A note names the compile its stream was built from, so - * a committing compile while a stream is open makes that stream's later - * notes vouch for nothing, and its replies go without their thinking: - * while one of a branch's requests is streaming, compile for its branch - * only with `dryRun`, which writes no record. A preview prices a + * a committing compile with `binds` while a stream is open makes that + * stream's later notes vouch for nothing, and its replies go without + * their thinking: while one of a branch's requests is streaming, compile + * for its branch with `binds` only with `dryRun`, which writes no + * record. A preview prices a * summary's carriers by the option it's given (`previewContext`'s * `thinkingBinding`), as a compile does. * diff --git a/test/thinking-binding.test.ts b/test/thinking-binding.test.ts index 2f1d74fb..ccd7038d 100644 --- a/test/thinking-binding.test.ts +++ b/test/thinking-binding.test.ts @@ -448,9 +448,9 @@ describe('thinking binding: compile', () => { await manager.close(); }); - it('a note is for the stream of the compile that wrote the record: after another committing compile it writes nothing, and a dry compile changes nothing', async () => { + it('a note is for the stream of the compile that wrote the record: after another committing compile with binds it writes nothing, and a dry compile or one without binds changes nothing', async () => { // previewContext calls the strategy alone and never reaches the record. - for (const between of ['a committing compile', 'a dry compile', 'nothing'] as const) { + for (const between of ['a committing compile', 'a committing compile without binds', 'a dry compile', 'nothing'] as const) { cleanup(); const manager = await open(); manager.addMessage('User', [text('U1')]); @@ -458,6 +458,7 @@ describe('thinking binding: compile', () => { // Stored while C1's stream is open, so C1's reply was minted without it. manager.addMessage('User', [text('zz-X')]); if (between === 'a committing compile') await manager.compile(BIG, undefined, BIND); + if (between === 'a committing compile without binds') await manager.compile(BIG); if (between === 'a dry compile') await manager.compile(BIG, undefined, bind({ dryRun: true })); const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); const accepted = manager.noteRequestSent([a1], identity(), c1.provenance!.compileId); @@ -475,6 +476,8 @@ describe('thinking binding: compile', () => { manager.addMessage('User', [text('U1')]); const c1 = await manager.compile(BIG, undefined, BIND); await manager.compile(BIG, undefined, bind({ dryRun: true })); + // A compile without binds leaves the record, so C1's stream still notes. + await manager.compile(BIG); const a1 = manager.addMessage('Claude', [thought(1), text('A1')]); assert.equal(manager.noteRequestSent([a1], identity(), c1.provenance!.compileId), true); assert.equal(manager.noteRequestSent([a1], identity(), 'zz-another-compile'), false, 'another compile\u2019s id writes nothing'); @@ -1035,6 +1038,37 @@ describe('thinking binding: compile', () => { } }); + it('a compile whose record the store can\u2019t write leaves the record before it, which that compile\u2019s stream can\u2019t note onto', async () => { + cleanup(); + const manager = await open(); + seed(manager); + await manager.compile(BIG, undefined, BIND_DROP); + const recorded = JSON.stringify(recordOf(manager)); + manager.addMessage('User', [text('U3')]); + const store = manager.getStore() as unknown as { setStateJson(id: string, value: unknown): void }; + const set = store.setStateJson.bind(store); + store.setStateJson = (id: string, value: unknown) => { if (id === 'default/thinking-binding') throw new Error('zz-store-unwritable'); set(id, value); }; + const warn = console.warn; + const warned: string[] = []; + console.warn = (message: string) => { warned.push(message); }; + let c2: CompileResult; + try { + c2 = await manager.compile(BIG, undefined, BIND_DROP); + } finally { + console.warn = warn; + store.setStateJson = set; + } + assert.equal(warned.length, 1, 'the failed write is logged'); + assert.equal(JSON.stringify(recordOf(manager)), recorded, 'the record before it stands'); + const a3 = manager.addMessage('Claude', [thought(3), text('A3')]); + assert.equal(manager.noteRequestSent([a3], identity(), c2.provenance!.compileId), false, 'the failed compile\u2019s stream can\u2019t note onto it'); + // Compared against the first compile's record: U3 and A3 are past its end. + const next = await manager.compile(BIG, undefined, BIND_DROP); + assert.deepEqual(thinkingByText(next), { U1: false, A1: true, U2: false, A2: true, U3: false, A3: false }); + assert.deepEqual(next.thinkingStrip, { newly: [a3], cause: 'messages' }); + await manager.close(); + }); + it('a compile that fails after selecting writes no record and tells the strategy nothing', async () => { cleanup(); const told: ContentBlock[][] = []; @@ -1107,7 +1141,7 @@ describe('thinking binding: compile', () => { cleanup(); }); - it('thinkingRecordedAfter is the branch\u2019s sequence, at or above every message stored before, one the view filter hides included', async () => { + it('thinkingRecordedAfter is the branch\u2019s sequence where its record began, at or above every message stored before, one the view filter hides included', async () => { cleanup(); const manager = await ContextManager.open({ path: STORE, From 58a6884f686e61595efa89f7b894c0408f7b67e5 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Fri, 23 Oct 2026 21:40:16 -0400 Subject: [PATCH 50/67] fix(system-changes): the refusal's raw-pin tokens leave out messages of system changes main's over-budget refusal (rawPinsDiagnostics) totals the middle's positions in forceRawPinPositions. On this branch that set holds every message of system changes too, since resolvedPinBounds keeps them raw as a pin keeps a message. So a change in the middle was counted as tokens the pins keep raw, though unpinning frees none of it: in the new test, a 6,000-word mark's refusal reported 23,648 tokens against the mark's own 18,241. The total now counts only positions a pin keeps raw that aren't changes. The new test fails without the fix (receipt alongside the branch's other runs) and passes with it. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 4 +++- test/whole-body-pins.test.ts | 17 +++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 6de7717c..d2ac8940 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -13042,10 +13042,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (this.pins.length === 0) return undefined; const messages = store.getAll(); const raw = this.forceRawPinPositions(messages); + // A message of system changes stays raw whatever the pins (resolvedPinBounds), so unpinning frees none of it. + const changes = new Set(systemChangePositions(messages)); const headStart = this.getHeadWindowStartIndex(store); const headEnd = this.getHeadWindowEnd(store); const tailStart = Math.max(this.getRecentWindowStart(store), headEnd); - const kept = (i: number): boolean => raw.has(i) && i < tailStart && !(i >= headStart && i < headEnd); + const kept = (i: number): boolean => raw.has(i) && !changes.has(i) && i < tailStart && !(i >= headStart && i < headEnd); let tokens = 0; for (const i of raw) if (kept(i)) tokens += price(i); diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 712cf6d1..0605f5cf 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -695,6 +695,23 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th cm.close(); }); }); + + it('counts no message of system changes toward the pins\' tokens: it stays raw whatever the pins', async () => { + await withDir(async (dir) => { + const { cm } = await open(dir); + await history(cm); + const b = cm.addMessage('user', words('Bdoc', 6000)); + const rules = (words('rule', 1500)[0] as { text: string }).text; + cm.addMessage('Harness', [{ type: 'system_change', addSystem: [rules] } as unknown as ContentBlock]); + await traffic(cm, 360); + await ready(cm); + const mark = cm.markDocument(b); + const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); + assert.strictEqual(raw.tokens, raw.pins[0]!.tokens, 'the total is the mark\'s alone, not the change beside it'); + await cm.close(); + }); + }); }); describe('the refusal\'s words for the pins it names', () => { From b1a3dff79ee9b28c605e5e3592cc5ffc87e29e5a Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 21:39:18 -0400 Subject: [PATCH 51/67] docs(changelog): note that marks larger than the summarizer's budget can stop compression A message no chunk held before its pin is never summarized while the pin stands, and later compression requests carry it whole without pricing it against compressionContextBudgetTokens, as for a pinRange over the same parts (#332). Since whole-message pins, a mark made on a fresh long message is one, so the release that carries them should say what happens when such marks outgrow that budget, and that unpinning clears it. Co-Authored-By: Claude Opus 5.5 --- changelog.d/whole-body-pins.fixed.md | 1 + 1 file changed, 1 insertion(+) diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index 512ac6f0..de059b5b 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,3 +1,4 @@ - **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. + - **Marks the summarizer's budget can't hold.** A message no chunk held before its pin is never summarized while the pin stands, and the compression requests after it carry it whole without pricing it against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made on a long message as it arrives is one. Where such marks together come to more than that budget, later compression requests are larger than it, and a summarizer whose window is smaller would refuse them, so compression would stop making progress for that store. Unpinning the marks clears it. - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. From 0060b141d2ca7ea0902bb8f1082287a5b3d4d0fb Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Fri, 23 Oct 2026 21:45:01 -0400 Subject: [PATCH 52/67] docs, test(system-changes): the refusal's raw-pin doc names what it leaves out; a pin over a change counts nothing for it rawPinsDiagnostics' doc says the positions it counts are those pins keep raw less the messages of system changes. The test now also pins the per-pin side: a range over the mark's message and the change beside it is named with the mark's tokens alone. Co-Authored-By: Claude Opus 5.5 --- src/strategies/autobiographical.ts | 3 ++- test/whole-body-pins.test.ts | 8 +++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index d2ac8940..56bc9863 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -13030,7 +13030,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * head, and outside the head window; with the head window disabled after a * reset, the plan's tail holds the recent messages before the head) that * pins keep raw - * (`forceRawPinPositions`), each priced by `price`, the plan's own raw + * (`forceRawPinPositions`), less the messages of system changes, which + * stay raw whatever the pins, each priced by `price`, the plan's own raw * price. Every such position counts, a part locked with `lockChunk` among * them, so the tokens are what unpinning frees at most: a pin that keeps no * part raw alone is still named. Where bounds are read, a pin whose own diff --git a/test/whole-body-pins.test.ts b/test/whole-body-pins.test.ts index 0605f5cf..1c92b5c2 100644 --- a/test/whole-body-pins.test.ts +++ b/test/whole-body-pins.test.ts @@ -702,13 +702,19 @@ describe('a compile that can\'t hold a whole message refuses, naming the pins th await history(cm); const b = cm.addMessage('user', words('Bdoc', 6000)); const rules = (words('rule', 1500)[0] as { text: string }).text; - cm.addMessage('Harness', [{ type: 'system_change', addSystem: [rules] } as unknown as ContentBlock]); + const changeId = cm.addMessage('Harness', [{ type: 'system_change', addSystem: [rules] } as unknown as ContentBlock]); await traffic(cm, 360); await ready(cm); const mark = cm.markDocument(b); const raw = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; assert.deepStrictEqual(raw.pins.map((p) => p.pinId), [mark]); assert.strictEqual(raw.tokens, raw.pins[0]!.tokens, 'the total is the mark\'s alone, not the change beside it'); + // A pin over the mark's message and the change counts the change for nothing, too. + assert.ok(cm.unpin(mark)); + const range = cm.pinRange(b, changeId); + const ranged = (await refusalOf(cm, 30_000)).diagnostics.rawPins!; + assert.deepStrictEqual(ranged.pins.map((p) => [p.pinId, p.tokens]), [[range, raw.tokens]]); + assert.strictEqual(ranged.tokens, raw.tokens); await cm.close(); }); }); From 6feb9ead5b8b82f05e3f1b19b9f093c964c0ea81 Mon Sep 17 00:00:00 2001 From: Greta-2425 Date: Fri, 23 Oct 2026 21:57:59 -0400 Subject: [PATCH 53/67] docs(changelog): narrow the note on marks the summarizer's budget can't hold From the review of the first wording: a mark made as a long message arrives can find parts already chunked; before the head window's end a pinned message is charged and the fallback leaves it out; the source-only modes keep compression going; another pin's level can still let the parts fold; and a chunk the refusals quarantined can stay quarantined after the marks are unpinned, until clearCompressionRefusalQuarantine. The note now says each. Co-Authored-By: Claude Opus 5.5 --- changelog.d/whole-body-pins.fixed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md index de059b5b..c84eaded 100644 --- a/changelog.d/whole-body-pins.fixed.md +++ b/changelog.d/whole-body-pins.fixed.md @@ -1,4 +1,4 @@ - **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. - - **Marks the summarizer's budget can't hold.** A message no chunk held before its pin is never summarized while the pin stands, and the compression requests after it carry it whole without pricing it against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made on a long message as it arrives is one. Where such marks together come to more than that budget, later compression requests are larger than it, and a summarizer whose window is smaller would refuse them, so compression would stop making progress for that store. Unpinning the marks clears it. + - **Marks the summarizer's budget can't hold.** The parts of a marked message that no chunk held when it was marked stay out of compression while the mark keeps them raw. From the head window's end on, the compression requests after them carry them whole and don't price them against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made as a long message arrives leaves every part not yet chunked this way. Where such parts together come to more than that budget, a summarizer whose window is smaller would refuse those requests, and unless `compressionSourceOnly` or `compressionSourceOnlyFallback` is set, compression would stop making progress for that store. Unpinning the marks lets later requests leave them out; a chunk already refused in three request shapes can stay quarantined after that, and `clearCompressionRefusalQuarantine` clears it. - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. From 656b2e9803e9b57109aceb2a25682e3511cc9c9d Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Sat, 24 Oct 2026 04:39:35 +0200 Subject: [PATCH 54/67] chore(release): 0.21.0 Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 13 +++++++++++++ changelog.d/raw-pins-in-refusals.added.md | 1 - changelog.d/whole-body-pins.fixed.md | 4 ---- package-lock.json | 4 ++-- package.json | 2 +- 5 files changed, 16 insertions(+), 8 deletions(-) delete mode 100644 changelog.d/raw-pins-in-refusals.added.md delete mode 100644 changelog.d/whole-body-pins.fixed.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c6db09b..9585c430 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,19 @@ Releases up to and including 0.6.2 predate this file; for their contents see ## Unreleased +## 0.21.0 — 2026-10-24 + +### Added + +- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them), and the message ends with a sentence naming each pin and the tokens, and saying that whoever manages pins can unpin them, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent, and it comes last and under 140 characters, so a host that quotes a long message by its head and its last 140 characters, as agent-framework does, keeps it whole. Each pin's tokens are what it keeps raw, a part locked with `lockChunk` included, so they're what unpinning frees at most; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. + +### Fixed + +- **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. + - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. + - **Marks the summarizer's budget can't hold.** The parts of a marked message that no chunk held when it was marked stay out of compression while the mark keeps them raw. From the head window's end on, the compression requests after them carry them whole and don't price them against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made as a long message arrives leaves every part not yet chunked this way. Where such parts together come to more than that budget, a summarizer whose window is smaller would refuse those requests, and unless `compressionSourceOnly` or `compressionSourceOnlyFallback` is set, compression would stop making progress for that store. Unpinning the marks lets later requests leave them out; a chunk already refused in three request shapes can stay quarantined after that, and `clearCompressionRefusalQuarantine` clears it. + - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. + ## 0.20.1 — 2026-10-23 ### Added diff --git a/changelog.d/raw-pins-in-refusals.added.md b/changelog.d/raw-pins-in-refusals.added.md deleted file mode 100644 index b39bfa1a..00000000 --- a/changelog.d/raw-pins-in-refusals.added.md +++ /dev/null @@ -1 +0,0 @@ -- **`OverBudgetError` names the pins that keep raw tokens in its middle.** Under adaptive resolution, the refusal of a plan whose middle is over the budget ("Adaptive picker exhausted but …") sets `diagnostics.rawPins` (`{ tokens, pins }`, each pin an `OverBudgetRawPin`: `pinId`, `kind`, `name`, `firstMessageId`, `lastMessageId`, `parts` for a pin read as whole messages, and the raw `tokens` it keeps in that middle, priced as the plan priced them), and the message ends with a sentence naming each pin and the tokens, and saying that whoever manages pins can unpin them, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. It says who can act rather than telling its reader to, since a host can pass it on to an agent, and it comes last and under 140 characters, so a host that quotes a long message by its head and its last 140 characters, as agent-framework does, keeps it whole. Each pin's tokens are what it keeps raw, a part locked with `lockChunk` included, so they're what unpinning frees at most; a message two pins cover counts toward each, and once in the total. The message still begins as before and still says "exceed hard budget". Every other refusal is unchanged, a refusal with no pin keeping raw tokens in the middle among them. diff --git a/changelog.d/whole-body-pins.fixed.md b/changelog.d/whole-body-pins.fixed.md deleted file mode 100644 index c84eaded..00000000 --- a/changelog.d/whole-body-pins.fixed.md +++ /dev/null @@ -1,4 +0,0 @@ -- **Pins and document marks cover whole messages.** Under `adaptiveResolution`, a pin whose first or last id names one part of a message stored in parts (a long message that ingress splits) now covers that message from its first part to its last, in what the agent is sent, in compression and in the summarizer's prompt alike, as a `pinRange` over all its parts does where no part is locked. Before, it covered only the parts its ids named, so a document mark on a long message, made with the id of its first part as connectome-host's web panel makes it, kept that part raw and let the rest fold. A part locked with `lockChunk` outside the span the pin's ids name keeps its lock, which only the whole reading would have added; one between them is covered, as before. Under `kv-stable`, a pin with `level` (not `maxLevel`) holds whole each summary it reaches, so a locked part that shares a summary with a part the pin covers renders at the level the pin holds that summary at, as one sharing a summary with the part its ids name did before. - - **The summarizer's input.** The canonical L1 compression prompt shows a marked message raw, as it shows any raw pin, so compression requests after it carry the whole message where they carried its first part. In a test store, that took the summarizer's input after a mark to 2.3 times what it was for a 6,800-word message and 5.2 times for a 20,000-word one. - - **Marks the summarizer's budget can't hold.** The parts of a marked message that no chunk held when it was marked stay out of compression while the mark keeps them raw. From the head window's end on, the compression requests after them carry them whole and don't price them against `compressionContextBudgetTokens`, as for a `pinRange` over the same parts (#332). A mark made as a long message arrives leaves every part not yet chunked this way. Where such parts together come to more than that budget, a summarizer whose window is smaller would refuse those requests, and unless `compressionSourceOnly` or `compressionSourceOnlyFallback` is set, compression would stop making progress for that store. Unpinning the marks lets later requests leave them out; a chunk already refused in three request shapes can stay quarantined after that, and `clearCompressionRefusalQuarantine` clears it. - - **Upgrade.** A stored mark on part of a long message reads whole at the first compile after the upgrade. Where the whole message fits, that compile rewrites the request once, from that message on, or from earlier where the select folds older history further to make room for it. Where it doesn't fit, the compile refuses with `OverBudgetError`, as for any pin the budget can't hold; where the plan's middle is what's over, that refusal ("Adaptive picker exhausted but …") names the mark and the tokens it keeps raw, and says whoever manages pins can unpin it, freeing up to those tokens, or whoever sets the hard budget can raise it if the model's window allows. diff --git a/package-lock.json b/package-lock.json index 3d8f444b..f9c98d14 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@animalabs/context-manager", - "version": "0.20.1", + "version": "0.21.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@animalabs/context-manager", - "version": "0.20.1", + "version": "0.21.0", "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", diff --git a/package.json b/package.json index fe90bb8d..f164d71c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@animalabs/context-manager", - "version": "0.20.1", + "version": "0.21.0", "description": "Context management layer for LLM-based agents using Chronicle and Membrane", "type": "module", "main": "dist/src/index.js", From b089906f57db5557574e6ca41f0688a8e89b2365 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Fri, 23 Oct 2026 22:59:14 -0400 Subject: [PATCH 55/67] docs(changelog): the siblings' ranges after 0.21.0 agent-framework 0.26.1 and connectome-host 0.13.0 depend on ^0.20.0 (npm, 2026-10-24), which admits neither 0.21.0, released since, nor a later release that would carry this change. --- changelog.d/request-image-ceilings.breaking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index e6738206..f6b4afcc 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ - **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling. It doesn't say so for an image the host's own limits would strip anyway. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** agent-framework 0.26.0 and connectome-host 0.13.0 depend on `^0.20.0` (npm, 2026-10-23), which admits a 0.20.x release and not a 0.21.0 one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. + - **Siblings:** agent-framework 0.26.1 and connectome-host 0.13.0 depend on `^0.20.0` (npm, 2026-10-24), which admits a 0.20.x release and not 0.21.0 or a later one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. From 9aa00fb56be41f9ebb525c4d6aa3a5971587a935 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Fri, 23 Oct 2026 23:12:00 -0400 Subject: [PATCH 56/67] docs(changelog): connectome-host 0.13.1's range connectome-host 0.13.1, released since the previous commit, depends on ^0.20.1, which admits neither 0.21.0 nor a later release that would carry this change, as agent-framework 0.26.1's ^0.20.0 doesn't (npm, 2026-10-24). --- changelog.d/request-image-ceilings.breaking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index f6b4afcc..4c0488a1 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ - **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling. It doesn't say so for an image the host's own limits would strip anyway. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** agent-framework 0.26.1 and connectome-host 0.13.0 depend on `^0.20.0` (npm, 2026-10-24), which admits a 0.20.x release and not 0.21.0 or a later one. A sibling whose range doesn't admit this release takes these changes on when it raises its range. + - **Siblings:** agent-framework 0.26.1 depends on `^0.20.0` and connectome-host 0.13.1 on `^0.20.1` (npm, 2026-10-24), and neither range admits 0.21.0 or a later release. A sibling whose range doesn't admit this release takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. From 0b808673e1ccfdc783d7f359aab9611ecf2393a3 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Sat, 24 Oct 2026 02:02:14 -0400 Subject: [PATCH 57/67] fix(images): the rule's ceiling line is credited per trim (R2-1) Under imageStripHysteresisRatio the line credited an image the rule's set replaced by replaying the set counterfactually: with the ceilings' limits lifted, and with each ceiling alone. Once a ceiling's limit had taken part anywhere in the history, every later compile whose render replaced an image ran those replays, and where the images were ones the depth or the host's own limit strips, nothing was credited, the line stayed unsaid, and the next compile replayed again, for the life of the instance (review round 2, R2-1, with a probe of one extra replay per compile). The set's own replay now records, for each image it strips, which of the window's full limits were over where the trim that stripped it began, and the credit applies ceilingsDeciding to that: a ceiling decided the images a trim strips when the limit it sets was over there and neither the depth limit nor a limit the host set was. The record lives in the memoized set, so reporting replays nothing. `involved`, the shortcut that lifted only uninvolved limits, the counterfactual replays and the `wanted` filter (reportCeilingHits already skips a said line) go, and stripReplayHeld is folded back into the memoized entry. Judged at the trim alone, the credit has two consequences the option's documentation and the changed fragment now state: a trim a limit the host set, or the depth limit, also brought on says nothing, though the ceiling's lower target there can strip an image the host's own limits keep; and the line is said for an image the ceiling's trim stripped even where a later trim would have stripped it anyway. Tests: the replay-count tests give way to one that reporting runs no replay; the mixed-trim and later-trim cases are turned round; the place test credits by place in both orders; a trim the depth limit brings on with the count ceiling says nothing; and test comments that explained a verdict by the counterfactual now explain it by the trim. --- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 8 +- src/strategies/autobiographical.ts | 197 +++++----- src/types/strategy.ts | 15 +- test/request-image-ceilings.test.ts | 346 ++++++++++-------- 5 files changed, 290 insertions(+), 278 deletions(-) diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 96f67b8b..64fcde97 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips where that ceiling's limit took part in the set, and that the set without the ceilings keeps and the set with that ceiling alone over the host's own limits strips. Working that out replays the set: while a line is unsaid and its ceiling's limit takes part, a compile whose render the set stripped replays it once or twice more, and three times when both ceilings' limits take part and neither line is said. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips in a trim that ceiling's limit brought on, where neither the depth limit nor a limit the host set was over too. That's judged at the trim alone, with two consequences: a trim that a limit the host set, or the depth limit, also brought on says nothing, though the ceiling's lower target there can strip an image the host's own limits keep; and the line is said for an image the ceiling's trim stripped even where a later trim would have stripped it anyway. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 2684e744..884e72fa 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -165,11 +165,9 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b * warning at initialize says so. Only a ceiling that decides changes what * the window keeps, so only that one says its line. With the memory * strategy's `imageStripHysteresisRatio`, whose set decides in place of the - * limits, the strategy credits the set's images itself, not here: a ceiling - * decides an image the set strips when its limit took part in that set, the - * set without the ceilings keeps the image, and the set with that ceiling - * alone over the host's own limits strips it (the memory strategy's - * ruleCeilingHits). + * limits, the strategy applies this to each trim of its set: the limits over + * where the trim began, and the depth limit's, decide every image the trim + * strips (the memory strategy's ruleCeilingHits). */ export function ceilingsDeciding( over: { images: boolean; bytes: boolean }, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 95515bd7..b445296b 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -649,14 +649,18 @@ type CeilingNotes = { ruleStripped: Array<{ id: MessageId; nth: number }>; }; -/** imageStripHysteresisRatio's set for some limits (stripReplayHeld): how - * many of each message's images it strips, and whether the count limit, - * and the byte limit, took part in it. */ +/** imageStripHysteresisRatio's set at the window's limits + * (stripReplayAtLimits): how many of each message's images it strips, and + * for each stripped image, by its place among its message's images, which + * full limits were over where the trim that stripped it began. */ type StripReplay = { out: ReadonlyMap; - involved: { images: boolean; bytes: boolean }; + trims: ReadonlyMap>; }; +/** Which of the window's full limits were over where a trim began. */ +type TrimOver = { readonly images: boolean; readonly bytes: boolean; readonly depth: boolean }; + /** A tool call the conversation answers: its message's place in the listing, * the place of the result that first answered it, and every result that * answers it, by message and place in its content. */ @@ -17653,8 +17657,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { return this.stripReplayAtLimits(store)?.out; } - /** The rule's set at the window's limits (imageStripReplay), memoized, with - * which of the count and byte limits took part in it (stripReplayHeld). */ + /** The rule's set at the window's limits (imageStripReplay), memoized, + * with the limits that were over where the trim that stripped each image + * began (ruleCeilingHits reads them). */ private stripReplayAtLimits(store: MessageStoreView): StripReplay | undefined { const ratio = this.imageHysteresisRatio(); if (ratio === undefined) return undefined; @@ -17676,76 +17681,62 @@ export class AutobiographicalStrategy implements ResettableStrategy { memo.countCap === countCap && memo.bytesCap === bytesCap && memo.ratio === ratio && memo.pruning === pruning && memo.msgCap === msgCap ) return memo.replay; - const replay = this.stripReplayHeld(store, messages, depthCap, countCap, bytesCap, ratio); + // tokensBefore[i]: the tokens of every message before i, so the depth of + // an image in message j, with message i the newest, is + // tokensBefore[i + 1] - tokensBefore[j]. + const tokensBefore = new Array(messages.length + 1); + tokensBefore[0] = 0; + for (let i = 0; i < messages.length; i++) tokensBefore[i + 1] = tokensBefore[i]! + store.estimateTokens(messages[i]!); + const live: Array<{ at: number; bytes: number }> = []; // oldest first; live[first..] are live + let first = 0; + let liveBytes = 0; + const out = new Map(); + const trims = new Map(); + for (let i = 0; i < messages.length; i++) { + for (const image of imagesIn(this.cappedForSend(store, messages[i]!, msgCap).content)) { + const bytes = inlineImageBytes(image); + live.push({ at: i, bytes }); + liveBytes += bytes; + } + const overCount = (count: number): boolean => countCap > 0 && live.length - first > count; + const overBytes = (size: number): boolean => bytesCap > 0 && liveBytes > size; + const overDepth = (depth: number): boolean => + depthCap > 0 && first < live.length && tokensBefore[i + 1]! - tokensBefore[live[first]!.at]! > depth; + const overCountOrBytes = (count: number, size: number): boolean => overCount(count) || overBytes(size); + const over = (depth: number, count: number, size: number): boolean => + overCountOrBytes(count, size) || overDepth(depth); + if (!over(depthCap, countCap, bytesCap)) continue; + // What brought this trim on, at the full limits. + const trim: TrimOver = { images: overCount(countCap), bytes: overBytes(bytesCap), depth: overDepth(depthCap) }; + const depth = Math.floor(depthCap * ratio); + const countTarget = Math.floor(countCap * ratio); + const sizeTarget = Math.floor(bytesCap * ratio); + while (first < live.length && over(depth, countTarget, sizeTarget)) { + // The images left are one message's and within the full limits: the + // newest of them stays, and the rest stay too while only depth is over + // its target, since stripping images of one message can't lower the + // depth it's at. + if ( + live[first]!.at === live[live.length - 1]!.at && !over(depthCap, countCap, bytesCap) && + (live.length - first === 1 || !overCountOrBytes(countTarget, sizeTarget)) + ) break; + const oldest = live[first++]!; + liveBytes -= oldest.bytes; + const id = messages[oldest.at]!.id; + out.set(id, (out.get(id) ?? 0) + 1); + // Stripped oldest first, so its place in this list is its place + // among its message's images. + const ofMessage = trims.get(id); + if (ofMessage) ofMessage.push(trim); + else trims.set(id, [trim]); + } + } + const replay: StripReplay = { out, trims }; this._imageStripReplayMemo = { messages, calibration, depthCap, countCap, bytesCap, ratio, pruning, msgCap, replay }; return replay; }); } - /** The rule's set for these limits, as imageStripReplay describes it, under - * the tool pruning in use (withToolPruning). 0 is no limit; with none, it - * strips nothing. `involved` says whether the count limit, and the byte - * limit, was ever over its full or trimmed value where the replay asked: - * one that never was decided nothing, so lifting only such limits gives - * this same set (ruleCeilingHits). */ - private stripReplayHeld( - store: MessageStoreView, - messages: readonly StoredMessage[], - depthCap: number, - countCap: number, - bytesCap: number, - ratio: number, - ): StripReplay { - const msgCap = this.config.maxMessageTokens; - // tokensBefore[i]: the tokens of every message before i, so the depth of - // an image in message j, with message i the newest, is - // tokensBefore[i + 1] - tokensBefore[j]. - const tokensBefore = new Array(messages.length + 1); - tokensBefore[0] = 0; - for (let i = 0; i < messages.length; i++) tokensBefore[i + 1] = tokensBefore[i]! + store.estimateTokens(messages[i]!); - const live: Array<{ at: number; bytes: number }> = []; // oldest first; live[first..] are live - let first = 0; - let liveBytes = 0; - const out = new Map(); - const involved = { images: false, bytes: false }; - for (let i = 0; i < messages.length; i++) { - for (const image of imagesIn(this.cappedForSend(store, messages[i]!, msgCap).content)) { - const bytes = inlineImageBytes(image); - live.push({ at: i, bytes }); - liveBytes += bytes; - } - const overCountOrBytes = (count: number, size: number): boolean => { - const overCount = countCap > 0 && live.length - first > count; - const overBytes = bytesCap > 0 && liveBytes > size; - if (overCount) involved.images = true; - if (overBytes) involved.bytes = true; - return overCount || overBytes; - }; - const over = (depth: number, count: number, size: number): boolean => - overCountOrBytes(count, size) || - (depthCap > 0 && first < live.length && tokensBefore[i + 1]! - tokensBefore[live[first]!.at]! > depth); - if (!over(depthCap, countCap, bytesCap)) continue; - const depth = Math.floor(depthCap * ratio); - const count = Math.floor(countCap * ratio); - const size = Math.floor(bytesCap * ratio); - while (first < live.length && over(depth, count, size)) { - // The images left are one message's and within the full limits: the - // newest of them stays, and the rest stay too while only depth is over - // its target, since stripping images of one message can't lower the - // depth it's at. - if ( - live[first]!.at === live[live.length - 1]!.at && !over(depthCap, countCap, bytesCap) && - (live.length - first === 1 || !overCountOrBytes(count, size)) - ) break; - const oldest = live[first++]!; - liveBytes -= oldest.bytes; - const id = messages[oldest.at]!.id; - out.set(id, (out.get(id) ?? 0) + 1); - } - } - return { out, involved }; - } - private _imageStripReplayMemo: { messages: readonly StoredMessage[]; calibration: number; depthCap: number; countCap: number; bytesCap: number; ratio: number; pruning: object; msgCap: number | undefined; replay: StripReplay; @@ -17923,13 +17914,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { * an image past the ceiling that nothing else strips (see * ceilingsDeciding). With imageStripHysteresisRatio, the rule's set * decides in place of the limits, so an image it replaces is credited - * by replaying the rule without the ceilings (ruleCeilingHits). A dry + * to the ceiling whose limit brought on the trim that stripped it, with + * no limit the host set, and not the depth limit, over there + * (ruleCeilingHits). A dry * select, a preview, an observer's reconstruction or a refused select * says nothing, and the line waits for the first render returned. */ private reportCeilingHits(store: MessageStoreView): void { const said = this.ceilingBoundReported; const notes = this._ceilingNotes; - const rule = this.ruleCeilingHits(store, notes.ruleStripped, { images: !said.images, bytes: !said.bytes }); + const rule = this.ruleCeilingHits(store, notes.ruleStripped); for (const kind of ['images', 'bytes'] as const) { if (!(notes[kind] || rule[kind]) || said[kind]) continue; said[kind] = true; @@ -17938,53 +17931,31 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** - * Which of the ceilings in `wanted` decided an image the rule's set - * replaced (`stripped`, by message and place among its images), as - * ceilingsDeciding decides it under the limits: the ceiling sets a limit - * (ceilingSetLimits) that took part in the set at the window's limits - * (stripReplayHeld's `involved`), the set at the host's own limits, with - * every ceiling that sets a limit lifted and the depth limit kept, keeps - * the image, and the set with that ceiling alone over the host's limits - * strips it. Each lifted limit is none, since a limit the host gave above - * its ceiling isn't in force. A limit that took no part decided nothing, - * and lifting only such limits gives that same set, so the credit replays - * only for a ceiling that sets a limit, took part, is wanted and has an - * image to credit: once each line is said, or where no ceiling's limit - * takes part, it replays nothing. The replays are this class's rule; a - * subclass's override of imageStripReplay isn't consulted for them. + * Which of the ceilings decided an image the rule's set replaced + * (`stripped`, by message and place among its images): the ceiling whose + * limit brought on the trim that stripped it, with neither the depth limit + * nor a limit the host set over there. That's ceilingsDeciding's rule, + * given what was over where the trim began. It reads the set's own replay, + * memoized for the listing the render was made from (stripReplayAtLimits), + * and replays nothing. The set is this class's rule; a subclass's override + * of imageStripReplay isn't consulted for it. */ private ruleCeilingHits( store: MessageStoreView, stripped: ReadonlyArray<{ id: MessageId; nth: number }>, - wanted: { images: boolean; bytes: boolean }, ): { images: boolean; bytes: boolean } { const hits = { images: false, bytes: false }; - const ratio = this.imageHysteresisRatio(); - if (ratio === undefined || stripped.length === 0 || (!wanted.images && !wanted.bytes)) return hits; - const atLimits = this.stripReplayAtLimits(store); - if (!atLimits) return hits; + const replay = this.stripReplayAtLimits(store); + if (!replay) return hits; const set = ceilingSetLimits(this.configuredImageLimits()); - const kinds = (['images', 'bytes'] as const).filter((kind) => set[kind] && atLimits.involved[kind] && wanted[kind]); - if (kinds.length === 0) return hits; - const depthCap = this.config.imageStripDepthTokens ?? 0; - const { target } = this.resolvedImageLimits(); - return this.withToolPruning(store, () => { - const messages = store.getAll(); - /** The rule's set with each limit a ceiling sets lifted, unless kept. */ - const replay = (keep: { images: boolean; bytes: boolean }): ReadonlyMap => { - const lift = { images: set.images && !keep.images, bytes: set.bytes && !keep.bytes }; - if (!(lift.images && atLimits.involved.images) && !(lift.bytes && atLimits.involved.bytes)) return atLimits.out; - return this.stripReplayHeld( - store, messages, depthCap, lift.images ? 0 : target.images, lift.bytes ? 0 : target.bytes, ratio, - ).out; - }; - const own = replay({ images: false, bytes: false }); - for (const kind of kinds) { - const alone = replay({ images: kind === 'images', bytes: kind === 'bytes' }); - hits[kind] = stripped.some(({ id, nth }) => nth >= (own.get(id) ?? 0) && nth < (alone.get(id) ?? 0)); - } - return hits; - }); + for (const { id, nth } of stripped) { + const trim = replay.trims.get(id)?.[nth]; + if (!trim) continue; + const decided = ceilingsDeciding(trim, trim.depth, set); + if (decided.images) hits.images = true; + if (decided.bytes) hits.bytes = true; + } + return hits; } /** The image limits whose values are defaults: those the library's default diff --git a/src/types/strategy.ts b/src/types/strategy.ts index f22373a3..585ce16f 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1563,13 +1563,14 @@ export interface AutobiographicalConfig { * placeholders because of this ceiling, rather than because of a limit * the host set: an image past it that the host's own limits, the depth * limit included, don't strip anyway, or, with - * `imageStripHysteresisRatio`, an image the rule's set strips where this - * ceiling's limit took part in that set, and that the set without the - * ceilings keeps and the set with this ceiling alone over the host's own - * limits strips. Working that out replays the set: while the line is - * unsaid and this ceiling's limit takes part, a compile whose render the - * set stripped replays it once or twice more, and three times when both - * ceilings' limits take part and neither line is said. + * `imageStripHysteresisRatio`, an image the rule's set strips in a trim + * this ceiling's limit brought on, where neither the depth limit nor a + * limit the host set was over too. That's judged at the trim alone, with + * two consequences: a trim that a limit the host set, or the depth limit, + * also brought on says nothing, though this ceiling's lower target there + * can strip an image the host's own limits keep; and the line is said for + * an image this ceiling's trim stripped even where a later trim would have + * stripped it anyway. */ imageCeiling?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 541e2695..fdcef93c 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -858,10 +858,11 @@ describe('the line when an image ceiling first strips', () => { assert.deepEqual(lines, []); }); - it("the memory strategy's rule says nothing when the depth limit strips what the ceiling also would", async () => { - // Three 8-byte images under a 20-byte ceiling: the oldest is past the - // ceiling (24 bytes) and, two image messages deep, past the depth limit - // too, so the set strips it with or without the ceiling. + it("the memory strategy's rule says nothing when the depth limit's trims strip what the ceiling's would", async () => { + // Three 8-byte images under a 20-byte ceiling and a 2,500-token depth + // limit, each image message about 1,600 tokens: the depth limit brings + // a trim on at the second image and again at the third, each stripping + // the older one, so the bytes never pass the ceiling where a trim begins. const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500, imageStripHysteresisRatio: 0.5 }; const alone = await ceilingLines(memory({ ...depth, imageByteCeiling: 0 }), [{ add: 3, bytes: 8 }]); const both = await ceilingLines(memory({ ...depth, imageByteCeiling: 20 }), [{ add: 3, bytes: 8 }]); @@ -875,7 +876,7 @@ describe('the line when an image ceiling first strips', () => { // a 6,000-token depth limit: without a ceiling nothing binds and nothing // is stripped. A count ceiling of 2 binds at the third, and its trim to // half strips the two older images, which the depth limit alone keeps: - // the ceiling decided. + // the trim is the ceiling's alone. const options = { maxLiveImages: 0, imageStripDepthTokens: 6000, imageStripHysteresisRatio: 0.5 }; const alone = await ceilingLines(memory({ ...options, imageCeiling: 0 }), [3]); assert.equal(alone.last, 'III', 'the depth limit alone strips nothing'); @@ -884,8 +885,8 @@ describe('the line when an image ceiling first strips', () => { assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); assert.ok(ceiling.lines[0]!.includes('imageCeiling 2 holds'), ceiling.lines[0]); // Under a 3,000-token depth limit, the first image is past the depth once - // the second comes, before the count ceiling is passed, and the depth's - // trims strip the same images with or without the ceiling. + // the second comes, before the count ceiling is passed: the trims that + // strip are the depth's, and the ceiling's limit brings none on. const deeper = { maxLiveImages: 0, imageStripDepthTokens: 3000, imageStripHysteresisRatio: 0.5 }; const deepAlone = await ceilingLines(memory({ ...deeper, imageCeiling: 0 }), [3]); assert.equal(deepAlone.last, '..I', 'the depth limit alone strips the two older images'); @@ -914,13 +915,13 @@ describe('the line when an image ceiling first strips', () => { } } - it("the memory strategy's rule says nothing for an image the ceiling strips first that the depth limit strips later anyway", async () => { + it("the memory strategy's rule names the byte ceiling whose trim stripped an image first, though the depth limit strips it later", async () => { // A 16-byte image, an 8-byte image, then about 1,500 tokens of text. When // the second image comes, the first passes a 20-byte ceiling (24 bytes) // while it's within a 4,000-token depth limit (about 3,200 tokens deep), - // so the set strips it then. The text takes it past the depth (about - // 4,700), and the set without the ceiling strips it there: the render is - // the same, so the ceiling decided nothing in it. + // so the ceiling's trim strips it then. The text takes it past the depth + // (about 4,700), where the set without the ceiling strips it: the render + // is the same, but the ceiling's trim stripped it first. const text = [{ type: 'text', text: 'word '.repeat(870) }]; const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.9 }; const messages = [imageMessage('img-1', 16), imageMessage('img-2', 8), text]; @@ -928,32 +929,33 @@ describe('the line when an image ceiling first strips', () => { assert.equal(alone.kept, '.I', 'the depth limit alone strips the first image'); const ceiling = await ruleRun({ ...options, imageByteCeiling: 20 }, messages); assert.equal(ceiling.kept, '.I', 'the render is the same'); - assert.deepEqual(ceiling.lines, []); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('imageByteCeiling 20 bytes holds'), ceiling.lines[0]); }); - it("the memory strategy's rule names the count ceiling when the host's own byte limit binds and the ceiling's target strips more", async () => { - // A host byte limit of 40, in force, and a count ceiling of 4. When three - // 4-byte images come after a 32-byte one, the host's byte limit binds (44 - // bytes), and the trim goes to half of each limit: 20 bytes, and 2 images - // under the count ceiling. Without the ceiling the trim to 20 bytes strips - // only the 32-byte image; the count ceiling's target strips the next one - // too, so the ceiling decided that one, though no image passed it. - const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; + it("the memory strategy's rule says nothing for a trim the host's own byte limit brings on too, though the ceiling's target strips more, and names a trim the ceiling brings on alone", async () => { + // A 32-byte image, then three 4-byte ones, under a count ceiling of 3. + // At the fourth image the count passes the ceiling. With a host byte + // limit of 40, in force, the bytes (44) pass it too: the trim is the + // host's as well as the ceiling's, so the line isn't said, though the + // count ceiling's target, 1, strips more than the host's limit alone + // would. With a host byte limit of 60 the bytes don't pass it, and the + // trim is the ceiling's alone. const steps = [{ add: 1, bytes: 32 }, { add: 3, bytes: 4 }]; - const alone = await ceilingLines(memory({ ...options, imageCeiling: 0 }), steps); - assert.deepEqual(alone.patterns, ['I', '.III'], "the host's byte limit alone strips the 32-byte image"); - const { lines, patterns } = await ceilingLines(memory({ ...options, imageCeiling: 4 }), steps); - assert.deepEqual(patterns, ['I', '..II'], 'the count trim strips the second-oldest image too'); - assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('imageCeiling 4 holds'), lines[0]); + const host = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 0, imageStripHysteresisRatio: 0.5 }), steps); + const mixed = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); + assert.notEqual(mixed.last, host.last, `premise: the ceiling's target strips more (${mixed.last} against ${host.last})`); + assert.deepEqual(mixed.lines, [], JSON.stringify(mixed.lines)); + const alone = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 60, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); + assert.ok(alone.last.includes('.'), `premise: the ceiling's trim strips (${alone.last})`); + assert.equal(alone.lines.length, 1, JSON.stringify(alone.lines)); + assert.ok(alone.lines[0]!.includes('imageCeiling 3 holds'), alone.lines[0]); }); - it("the memory strategy's rule names the count ceiling it decided by, not a byte ceiling that strips nothing alone", async () => { + it("the memory strategy's rule names the count ceiling whose limit brought the trim on, not the byte ceiling", async () => { // Both window limits are set by their ceilings. The fifth 4-byte image // passes the count ceiling of 4, and 20 bytes are far under the byte - // ceiling, so the set without the ceilings keeps every image, the set - // under the count ceiling alone strips three, and the set under the byte - // ceiling alone strips none. + // ceiling, so the trim is the count ceiling's alone. const { lines, last } = await ceilingLines( memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), [5], @@ -963,11 +965,10 @@ describe('the line when an image ceiling first strips', () => { assert.ok(lines[0]!.includes('imageCeiling 4 holds'), lines[0]); }); - it("the memory strategy's rule names each ceiling whose set alone strips an image the set without them keeps", async () => { + it("the memory strategy's rule names both ceilings when both their limits bring the trim on", async () => { // Both window limits are set by their ceilings: 4 images and 16 bytes. - // Five 4-byte images pass both (5 images, 20 bytes) at the fifth, so the - // set under either ceiling alone strips the three oldest, and the set - // without the ceilings keeps them all: each ceiling decided them. + // Five 4-byte images pass both (5 images, 20 bytes) at the fifth, with no + // limit the host set, so each ceiling decided the images the trim strips. const { lines, last } = await ceilingLines( memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), [5], @@ -978,67 +979,67 @@ describe('the line when an image ceiling first strips', () => { assert.ok(lines.some((line) => line.includes('imageByteCeiling 16 bytes holds')), JSON.stringify(lines)); }); - it("the memory strategy's rule says nothing for images the ceiling strips that the host's own limit strips without it", async () => { + it("the memory strategy's rule names the count ceiling for images its trim stripped, though the host's own limit strips every image later", async () => { // A count ceiling of 4 and a host byte limit of 40, in force. The fifth - // image passes the count ceiling, and a 44-byte image then passes the - // host's byte limit on its own: the host's limit strips every image now, - // those the ceiling stripped too, so the render is the same with or - // without the ceiling. + // image passes the count ceiling with the bytes under the host's limit + // (36), so that trim is the ceiling's alone. A 44-byte image then passes + // the host's byte limit on its own: the host's limit strips every image + // now, so the render is the same with or without the ceiling, but the + // ceiling's trim stripped some of them first. const messages = [imageMessage('img-1', 20), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`)), imageMessage('img-6', 44)]; const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; const alone = await ruleRun({ ...options, imageCeiling: 0 }, messages); const ceiling = await ruleRun({ ...options, imageCeiling: 4 }, messages); assert.equal(ceiling.kept, '......'); assert.equal(ceiling.kept, alone.kept, 'the render is the same without the ceiling'); - assert.deepEqual(ceiling.lines, []); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('imageCeiling 4 holds'), ceiling.lines[0]); }); - it("the memory strategy's rule names the count ceiling for images its trim strips that the host's own limit keeps", async () => { + it("the memory strategy's rule says nothing for a trim the count ceiling and the host's own byte limit bring on together", async () => { // A count ceiling of 4 and a host byte limit of 40, in force. When the // fifth image comes (8 bytes, after a 24-byte one and three 4-byte ones), // it passes both: the count ceiling and 44 bytes. The host's limit alone // trims to 20 bytes, stripping the oldest; the count ceiling's trim, to - // 2, strips the two after it too, which the host's limit keeps. + // 2, strips the two after it too, which the host's limit keeps. The trim + // is the host's as well, so the line isn't said. const messages = [imageMessage('img-1', 24), ...[2, 3, 4].map((i) => imageMessage(`img-${i}`)), imageMessage('img-5', 8)]; const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; const alone = await ruleRun({ ...options, imageCeiling: 0 }, messages); assert.equal(alone.kept, '.IIII', "the host's limit alone strips the oldest"); const ceiling = await ruleRun({ ...options, imageCeiling: 4 }, messages); assert.equal(ceiling.kept, '...II'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageCeiling 4 holds'), ceiling.lines[0]); + assert.deepEqual(ceiling.lines, []); }); - it("the memory strategy's rule names the byte ceiling for an image its trim strips that the host's own limit keeps", async () => { + it("the memory strategy's rule says nothing for a trim the byte ceiling and the host's own count limit bring on together", async () => { // The mirror, with a byte ceiling of 20 and a host count limit of 4, in // force. When the fifth image comes (8 bytes, after three 4-byte ones and // an 8-byte one), it passes the host's count and the byte ceiling (28 // bytes). The host's count alone trims to 2, stripping the three oldest; // the byte ceiling's trim, to 10 bytes, strips the 8-byte fourth image - // too, which the host's count keeps. + // too, which the host's count keeps. The trim is the host's as well. const messages = [...[1, 2, 3].map((i) => imageMessage(`img-${i}`)), imageMessage('img-4', 8), imageMessage('img-5', 8)]; const options = { maxLiveImages: 4, maxLiveImageBytes: 0, imageStripHysteresisRatio: 0.5 }; const alone = await ruleRun({ ...options, imageByteCeiling: 0 }, messages); assert.equal(alone.kept, '...II', "the host's count alone strips the three oldest"); const ceiling = await ruleRun({ ...options, imageByteCeiling: 20 }, messages); assert.equal(ceiling.kept, '....I'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageByteCeiling 20 bytes holds'), ceiling.lines[0]); + assert.deepEqual(ceiling.lines, []); }); it("the memory strategy's rule credits by each image's place in its message", async () => { - // A count ceiling of 3 and a host byte limit of 16, in force. An 8-byte - // image, a message of two 4-byte images, and a 4-byte image: the last - // passes both (4 images, 20 bytes). The host's limit alone trims to 8 - // bytes, stripping the oldest and the first image of the pair; the count - // ceiling's trim, to 1, strips the pair's second image too. So the - // ceiling decided that one image, the second of its message. - const pair = [ - { type: 'image', source: { type: 'base64', data: 'A'.repeat(4), mediaType: 'image/png' }, tokenEstimate: 100 }, - { type: 'image', source: { type: 'base64', data: 'B'.repeat(4), mediaType: 'image/png' }, tokenEstimate: 100 }, - { type: 'text', text: 'pair' }, - ] as ContentBlock[]; - const messages = [imageMessage('img-1', 8), pair, imageMessage('img-3', 4)]; + // A host byte limit of 19, in force, and a count ceiling of 3. A message + // of three images, 12, 4 and 4 bytes, passes the host's limit (20 bytes) + // and not the count: that trim is the host's, and strips the message's + // first image. Two 4-byte images follow; the second passes the count + // ceiling (4 images, 16 bytes), and that trim, the ceiling's alone, + // strips the message's second and third images. So the ceiling decided + // those two, by their places in their message, and not its first. + const image = (data: string): ContentBlock => + ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' }, tokenEstimate: 100 }) as ContentBlock; + const triple = [image('A'.repeat(12)), image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'triple' }] as ContentBlock[]; + const messages = [triple, imageMessage('img-x'), imageMessage('img-y'), imageMessage('img-z')]; const kept = async (options: Record) => { const said: string[] = []; const original = console.warn; @@ -1051,20 +1052,61 @@ describe('the line when an image ceiling first strips', () => { const of = (label: string) => compiled.messages.find((m) => m.content.some((b) => b.type === 'text' && b.text === label)); const marks = (label: string) => of(label)!.content.filter((b) => b.type === 'image' || (b.type === 'text' && b.text === PLACEHOLDER)) .map((b) => b.type === 'image' ? 'I' : '.').join(''); - return { kept: [marks('img-1'), marks('pair'), marks('img-3')].join(' '), lines: said.filter((line) => line.includes(CEILING_LINE)) }; + return { kept: ['triple', 'img-x', 'img-y', 'img-z'].map(marks).join(' '), lines: said.filter((line) => line.includes(CEILING_LINE)) }; } finally { console.warn = original; await manager.close(); rmSync(dir, { recursive: true, force: true }); } }; - const options = { maxLiveImages: 0, maxLiveImageBytes: 16, imageStripHysteresisRatio: 0.5 }; + const options = { maxLiveImages: 0, maxLiveImageBytes: 19, imageStripHysteresisRatio: 0.9 }; const alone = await kept({ ...options, imageCeiling: 0 }); - assert.equal(alone.kept, '. .I I', "the host's limit alone strips the oldest and the pair's first image"); + assert.equal(alone.kept, '..I I I I', "the host's limit alone strips the message's first image, and its second at the last image"); const ceiling = await kept({ ...options, imageCeiling: 3 }); - assert.equal(ceiling.kept, '. .. I', "the count ceiling's trim strips the pair's second image too"); + assert.equal(ceiling.kept, '... I I I', "the count ceiling's trim strips the message's second and third images"); assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); assert.ok(ceiling.lines[0]!.includes('imageCeiling 3 holds'), ceiling.lines[0]); + + // The other order, under a host byte limit of 30: a message of four + // 4-byte images passes the count ceiling alone (4 images, 16 bytes), and + // that trim strips its first two. A 28-byte image then passes the host's + // limit alone (36 bytes), and that trim strips the other two. So the + // ceiling decided the message's first two images, and not its last two. + const quad = [...['A', 'B', 'C', 'D'].map((c) => image(c.repeat(4))), { type: 'text', text: 'triple' }] as ContentBlock[]; + messages.splice(0, messages.length, quad, imageMessage('img-x', 28), imageMessage('img-y'), imageMessage('img-z')); + const second = await kept({ maxLiveImages: 0, maxLiveImageBytes: 30, imageStripHysteresisRatio: 0.9, imageCeiling: 3 }); + assert.ok(second.kept.startsWith('....'), `every image of the message is stripped (${second.kept})`); + assert.equal(second.lines.length, 1, JSON.stringify(second.lines)); + assert.ok(second.lines[0]!.includes('imageCeiling 3 holds'), second.lines[0]); + }); + + it("the memory strategy's rule says nothing for a trim the depth limit brings on too", async () => { + // A count ceiling of 2 and a 4,000-token depth limit. An image, then a + // message of two images and about 5,000 tokens of text: there the count + // passes the ceiling (3 images) and the first image is past the depth, + // so the trim is the depth's as well, and the line isn't said. + const image = (data: string): ContentBlock => + ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' }, tokenEstimate: 100 }) as ContentBlock; + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ maxLiveImages: 0, imageCeiling: 2, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }), + }); + try { + manager.addMessage('user', imageMessage('img-1')); + manager.addMessage('user', [image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'pair ' + 'word '.repeat(2900) }]); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + const placeholders = compiled.messages.flatMap((m) => m.content).filter((b) => b.type === 'text' && b.text === PLACEHOLDER).length; + assert.ok(placeholders > 0, 'premise: the trim strips'); + assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), []); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } }); it("the memory strategy's rule says nothing for images the ceiling's trim strips that the host's own limit strips too, later", async () => { @@ -1078,15 +1120,17 @@ describe('the line when an image ceiling first strips', () => { assert.deepEqual(ceiling.lines, []); }); - it("the memory strategy's rule says nothing for an image the byte ceiling strips that the host's own count limit strips too, later", async () => { + it("the memory strategy's rule names the byte ceiling for an image its trim stripped, though the host's own count limit strips it later", async () => { // A byte ceiling of 20 and a host count limit of 4, in force. A 16-byte - // image and two 4-byte ones: the third passes the byte ceiling (24 bytes), - // and the set strips the 16-byte image. Without the ceiling the host's - // count strips it as well, with two more, when the fifth image comes. + // image and two 4-byte ones: the third passes the byte ceiling (24 bytes) + // with the count under the host's limit, and the ceiling's trim strips + // the 16-byte image. Without the ceiling the host's count strips it as + // well, with two more, when the fifth image comes. const messages = [imageMessage('img-1', 16), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`))]; const ceiling = await ruleRun({ maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 20, imageStripHysteresisRatio: 0.5 }, messages); assert.equal(ceiling.kept, '.IIII'); - assert.deepEqual(ceiling.lines, []); + assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); + assert.ok(ceiling.lines[0]!.includes('imageByteCeiling 20 bytes holds'), ceiling.lines[0]); }); for (const [kind, options, why] of [ @@ -1107,8 +1151,9 @@ describe('the line when an image ceiling first strips', () => { // A count ceiling of 4 and a 20,000-token depth limit. The fifth image's // message passes the ceiling, with nothing past the depth. About 9,000 // tokens of text before the newest image put the second-newest past half - // the depth, so the trim strips it by the depth, not the count. Without - // the ceiling nothing binds, so the ceiling decided all four. + // the depth, so the trim strips it by the depth's target, not the + // count's. The trim is the ceiling's alone, begun with nothing past the + // full depth, so the ceiling decided all four. const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -1131,13 +1176,11 @@ describe('the line when an image ceiling first strips', () => { } }); - it("the memory strategy's rule names only a ceiling whose limit took part in its set", async () => { + it("the memory strategy's rule names only the ceiling whose limit brings its trims on", async () => { // Both window limits are set by their ceilings: 2 images and 40 bytes. // Eleven 4-byte images: the count ceiling binds at every third, and its - // trims keep the live bytes under half the byte ceiling, so the byte - // ceiling takes no part. Under the byte ceiling alone, with no count, the - // eleventh image would bring a byte trim that strips the oldest ones too, - // but that isn't why they're stripped here. + // trims keep the live bytes under half the byte ceiling, so the byte limit + // is never over where a trim begins. const { lines, last } = await ceilingLines( memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 40, imageStripHysteresisRatio: 0.5 }), [11], @@ -1147,12 +1190,12 @@ describe('the line when an image ceiling first strips', () => { assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); }); - it("the memory strategy's rule names only the ceiling whose limit alone strips, though the other took part", async () => { + it("the memory strategy's rule names only the ceiling whose limit brought the trim on, though it trims to the other's target too", async () => { // Both window limits are set by their ceilings: 2 images and 16 bytes. // The third 4-byte image passes the count ceiling, and the trim, to 1 - // image and 8 bytes, strips the two older ones. The byte limit takes part - // in that trim (12 bytes are over 8), but the set under the byte ceiling - // alone, with no count, strips nothing, so only the count ceiling decided. + // image and 8 bytes, strips the two older ones. The bytes (12) are over + // the trim's 8 but weren't over 16 where it began, so only the count + // ceiling decided. const { lines, last } = await ceilingLines( memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), [3], @@ -1169,7 +1212,7 @@ describe('the line when an image ceiling first strips', () => { // byte limit alone strips the same three. Two 4-byte images then make four // live, 28 bytes. By the limits' own account, newest first, the third // oldest is now past the count ceiling and within the host's bytes, but - // the set decides it, and the set without the ceiling strips it too. + // the set decides it, and the trim that stripped it was the host's too. const { lines, patterns } = await ceilingLines( memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), [{ add: 4, bytes: 8 }, { add: 1, bytes: 12 }, { add: 2, bytes: 4 }], @@ -1178,50 +1221,79 @@ describe('the line when an image ceiling first strips', () => { assert.deepEqual(lines, [], JSON.stringify(lines)); }); - it("the memory strategy replays the rule for the credit only for a ceiling whose limit took part and is still to be said", async () => { - // The credit replays the rule with the ceilings' limits lifted. Once each - // line is said, a later compile replays nothing for it; nor does a host - // whose limits no ceiling sets, or one whose ceiling's limit takes no part - // in the set, as under a depth limit that strips first. + it("the memory strategy's credit reads the set's own trims and replays nothing, whether or not it says the line", async () => { + // The credit is kept with the set, as the replay builds it, so reporting + // reads it and replays nothing of its own: at the compile that says a + // line, and at each later compile whose render replaced images that no + // ceiling decided, as here, where the host's byte limit strips them. const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; - for (const [options, first] of [ - [{ maxLiveImages: 0, imageCeiling: 4 }, 1], - [{ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageByteCeiling: 16 }, 3], - [{ maxLiveImages: 4 }, 0], - [{ maxLiveImages: 0, imageStripDepthTokens: 3000 }, 0], - ] as const) { - const original = console.warn; - console.warn = () => {}; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const strategy = memory({ ...options, imageStripHysteresisRatio: 0.5 }); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - const s = strategy as unknown as { - stripReplayHeld: (...args: unknown[]) => unknown; - ruleCeilingHits: (...args: unknown[]) => unknown; - }; - let crediting = false; - let replays = 0; - const held = s.stripReplayHeld.bind(strategy); - s.stripReplayHeld = (...args: unknown[]) => { if (crediting) replays++; return held(...args); }; - const credit = s.ruleCeilingHits.bind(strategy); - s.ruleCeilingHits = (...args: unknown[]) => { - crediting = true; - try { return credit(...args); } finally { crediting = false; } - }; - try { - for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - const kept = pattern((await manager.compile(budget)).messages, 5); - assert.ok(kept.includes('.'), `the rule strips for ${JSON.stringify(options)}: ${kept}`); - assert.equal(replays, first, `the first compile's credit, for ${JSON.stringify(options)}`); - replays = 0; - for (let i = 6; i <= 8; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); + const s = strategy as unknown as { + cappedForSend: (...args: unknown[]) => unknown; + ruleCeilingHits: (store: unknown, stripped: unknown[]) => unknown; + }; + let crediting = false; + let cuts = 0; + const credited: number[] = []; + const cut = s.cappedForSend.bind(strategy); + s.cappedForSend = (...args: unknown[]) => { if (crediting) cuts++; return cut(...args); }; + const credit = s.ruleCeilingHits.bind(strategy); + s.ruleCeilingHits = (store: unknown, stripped: unknown[]) => { + crediting = true; + credited.push(stripped.length); + try { return credit(store, stripped); } finally { crediting = false; } + }; + try { + // Five 4-byte images: the fifth passes the count ceiling alone. + for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + await manager.compile(budget); + assert.equal(said.filter((line) => line.includes(CEILING_LINE)).length, 1, 'the line, at the first compile'); + // Then 30-byte images, each passing the host's byte limit. + for (let round = 0; round < 3; round++) { + manager.addMessage('user', imageMessage(`big-${round}`, 30)); await manager.compile(budget); - assert.equal(replays, 0, `nothing at the next compile, for ${JSON.stringify(options)}`); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); } + assert.ok(credited.length >= 4 && credited.every((n) => n > 0), `premise: every compile's render replaced images (${credited})`); + assert.equal(cuts, 0, 'reporting replayed nothing'); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("the memory strategy's credit isn't read from a preview's set under ceilings that set no limit", async () => { + // A preview with maxLiveImages 4 under a count ceiling of 10 limits the + // count to 4, as the configured 0 under a ceiling of 4 does, but there + // the host sets the limit, so its trims decide nothing for a ceiling. + // The compile after it, under the configured ceilings, still names the + // count ceiling its own trim brought on. + const said: string[] = []; + const original = console.warn; + console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; + const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); + const manager = await ContextManager.open({ + path: join(dir, 'store'), + strategy: memory({ hierarchical: false, adaptiveResolution: true, maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), + }); + try { + for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + assert.ok(manager.previewContext(budget, { maxLiveImages: 4, imageCeiling: 10 }), 'premise: the strategy previews'); + const compiled = await manager.compile(budget); + assert.equal(pattern(compiled.messages, 5), '...II'); + const ours = said.filter((line) => line.includes(CEILING_LINE)); + assert.equal(ours.length, 1, JSON.stringify(said)); + assert.ok(ours[0]!.includes('imageCeiling 4 holds'), ours[0]); + } finally { + console.warn = original; + await manager.close(); + rmSync(dir, { recursive: true, force: true }); } }); @@ -1294,36 +1366,6 @@ describe('the line when an image ceiling first strips', () => { }); } - it("the memory strategy's credit replays nothing for a render that replaced none of the set's images", async () => { - // A count ceiling of 4 takes part in the set of five images, so a render - // that replaced some would be credited by replay; asked for a render that - // replaced none, the credit replays nothing and names nothing. - const original = console.warn; - console.warn = () => {}; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - const s = strategy as unknown as { - stripReplayHeld: (...args: unknown[]) => unknown; - ruleCeilingHits: (store: unknown, stripped: unknown[], wanted: { images: boolean; bytes: boolean }) => { images: boolean; bytes: boolean }; - stripReplayAtLimits: (store: unknown) => { involved: { images: boolean; bytes: boolean } } | undefined; - }; - try { - for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - const view = (manager as unknown as { messageStore: { createView(): unknown } }).messageStore.createView(); - assert.equal(s.stripReplayAtLimits(view)?.involved.images, true, 'premise: the count limit took part'); - let replays = 0; - const held = s.stripReplayHeld.bind(strategy); - s.stripReplayHeld = (...args: unknown[]) => { replays++; return held(...args); }; - assert.deepEqual(s.ruleCeilingHits(view, [], { images: true, bytes: true }), { images: false, bytes: false }); - assert.equal(replays, 0); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } - }); - it('marks only defaults in force: an explicit undefined maxLiveImages is no limit in the memory strategy, and 6 in windowed passthrough', async () => { const auto = await ceilingLines(memory({ maxLiveImages: undefined, imageCeiling: 2 }), [3]); assert.equal(auto.lines.length, 1, JSON.stringify(auto.lines)); From b6bd6773aa1687e593f0e377481e3d1afd3cebeb Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Sat, 24 Oct 2026 02:02:23 -0400 Subject: [PATCH 58/67] fix(compression): the kept counts are filled with no cap set (R2-6) capCompressionImageBytes returned before filling its stats when neither the byte budget nor the count cap was set, so prepareCompressionRequest reported imagesKept 0 and imageBytesKept 0 while it kept every image (review round 2, R2-6). The walk now runs with no cap too, keeping every image and counting it; the 501-image test asserts the counts. --- src/strategies/autobiographical.ts | 3 +-- test/request-image-ceilings.test.ts | 2 ++ 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b445296b..38e1d280 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -18350,7 +18350,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { * uses, so the summarizer is told plainly that it is not seeing them. * A cap of 0 or less is none. * Returns the number of images replaced; `stats`, when given, receives the - * bytes and the images kept. + * bytes and the images kept, every image's where neither cap is set. */ protected capCompressionImageBytes( messages: Array<{ content: ContentBlock[] }>, @@ -18359,7 +18359,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { capImages = 0, ): number { const countCap = capImages > 0 ? capImages : 0; - if (capBytes <= 0 && countCap === 0) return 0; let kept = 0; let keptImages = 0; let dropped = 0; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index fdcef93c..319918a4 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -1497,5 +1497,7 @@ describe('compression prompts within the image ceilings', () => { ); assert.equal(prepared.imageReplacements, 0); assert.equal(prepared.request.messages.filter((m) => m.content[0]!.type === 'image').length, 501, 'all 501 are still images'); + assert.equal(prepared.imagesKept, 501, 'the counts say what was kept'); + assert.equal(prepared.imageBytesKept, 501 * 4); }); }); From 40c90e46facad3ba13ced034b0e04eb3f316fe3f Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Sat, 24 Oct 2026 02:02:23 -0400 Subject: [PATCH 59/67] docs(images): the shed note names what membrane 0.7.1 sheds, and the siblings line is undated (R2-3, R2-5) R2-3: membrane sheds and checks the messages the formatter built, and its shed replaces only membrane image blocks, top-level or in a tool_result. A request that sets shedOversizeImages and is built by the openai-responses formatter carries input_image parts, so none is shed and the transform doesn't check it against the cap, and that adapter doesn't check it again (membrane 0.7.1: transformRequest, shedImagesToFitByteBudget, the openai-responses formatter). The byte ceiling's doc and the byte-cap note say so. R2-5: the breaking fragment's siblings line no longer names versions and a date, which went stale when agent-framework 0.26.2 was published on the same range: no sibling's current range admits this release, and each takes these changes on when it raises its range. --- changelog.d/request-image-ceilings.breaking.md | 2 +- src/live-image-limits.ts | 7 ++++++- src/types/strategy.ts | 6 +++++- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index 4c0488a1..a87abaf3 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,4 @@ - **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling. It doesn't say so for an image the host's own limits would strip anyway. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** agent-framework 0.26.1 depends on `^0.20.0` and connectome-host 0.13.1 on `^0.20.1` (npm, 2026-10-24), and neither range admits 0.21.0 or a later release. A sibling whose range doesn't admit this release takes these changes on when it raises its range. + - **Siblings:** no sibling's current range admits this release; each takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 884e72fa..8942a9e7 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -37,7 +37,12 @@ export const DEFAULT_IMAGE_CEILING = 500; * as this package's compression requests do. Then it replaces images, * oldest first, until the request fits or no image is left, and passes it * on either way; the Anthropic adapter's own check, which never sheds, - * refuses one still over the cap. 24 MiB of base64 images leaves 4 MiB of that + * refuses one still over the cap. The shed works on the messages the + * formatter built, and replaces only membrane's `image` blocks, top-level or + * in a `tool_result`: a request that sets `shedOversizeImages` and is built + * by membrane's `openai-responses` formatter, whose images are + * `input_image` parts, has none shed and isn't checked against the cap + * there, and that adapter doesn't check it again. 24 MiB of base64 images leaves 4 MiB of that * default cap for everything else the messages carry: their text, their * JSON, and any documents or other media. */ diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 585ce16f..2529f162 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1593,7 +1593,11 @@ export interface AutobiographicalConfig { * request that asks membrane to shed images, as this strategy's compression * requests do, first has its oldest images replaced until it fits or none * is left, and then only the Anthropic adapter's own check refuses it if - * it's still over (`src/live-image-limits.ts` says where each check runs). This + * it's still over (`src/live-image-limits.ts` says where each check runs). + * That's for a formatter whose messages keep membrane's `image` blocks: + * such a request built with membrane's `openai-responses` formatter + * carries `input_image` parts, which aren't shed, and it isn't checked + * either. This * setting isn't read from `MEMBRANE_MAX_REQUEST_BYTES`: a host that changes * membrane's cap sets this too, keeping room under the cap for what this * doesn't count. From 186641bc3ee1356943c1ae8950637f2978e1f5b1 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Sat, 24 Oct 2026 02:38:43 -0400 Subject: [PATCH 60/67] refactor(images): the ceiling line says a ceiling's limit was over, without attributing each image The once-only line that names a ceiling holding a limit of 0, or above it, was said only for images the ceiling decided: under the limits, an image past the ceiling that nothing else strips (ceilingsDeciding), and under imageStripHysteresisRatio, an image whose trim the ceiling's limit alone brought on. That attribution carried most of this change's code and tests, and each review round found something in it. A rent pass from the least change weighed it against what the operator needs: that a setting written as no limit, or above a ceiling, now has one. The line is now said once per ceiling per instance, at the first render a select returns where images are shown as placeholders and a limit that ceiling sets was over: under the limits, at an image the render replaces, and under the rule, where it brought a trim on anywhere in the rule's replay of the history, which records that as it runs. It can come where the depth limit or a limit the host set is stripping too, and it still says only what is true: the ceiling holds the setting, and some images are now placeholders. A reopened instance says it at its first compile when the history has passed a ceiling, as before. ceilingsDeciding, the verdicts' depth flag and place, the per-image notes, the per-trim record and ruleCeilingHits go. ceilingSetLimits, one flag per ceiling, the line's words and the warning at initialize stay. A window held at a ceiling strips its oldest images as new ones come, as under any limit, but from deep in the request, so the prompt cache is rewritten from there at every new image. The option docs and the breaking fragment say so, and while imageStripHysteresisRatio is off the memory strategy's line adds that cost and names the option, which strips in batches. Windowed passthrough has no such option, so its line doesn't. Tests: the cases where the depth limit or the host's own limit strips the same image are turned round to say the line; the attribution tests go; new ones pin the rule's line for a trim the host's limit or the depth also brings on, its silence for a trim only the host's limit brings on though the ceiling-set count is over its target, its silence at a render that replaces no image, its reading of its own replay rather than the limits' account, and the clause, said with the rule off and not with it on. --- .../request-image-ceilings.breaking.md | 3 +- changelog.d/request-image-ceilings.changed.md | 2 +- src/live-image-limits.ts | 35 +- src/strategies/autobiographical.ts | 169 +++---- src/strategies/windowed-passthrough.ts | 19 +- src/types/strategy.ts | 30 +- test/request-image-ceilings.test.ts | 461 ++++-------------- 7 files changed, 184 insertions(+), 535 deletions(-) diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md index a87abaf3..bc2f7d49 100644 --- a/changelog.d/request-image-ceilings.breaking.md +++ b/changelog.d/request-image-ceilings.breaking.md @@ -1,4 +1,5 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling. It doesn't say so for an image the host's own limits would strip anyway. +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and that ceiling's limit was over. + - **Cost at a ceiling:** a window held at a ceiling strips its oldest images as new ones come, as under any limit, but from deep in the request, so the prompt cache is rewritten from there at every new image. At the store's default price of 1,600 tokens an image, a window held at the count ceiling rewrites about 800k tokens of it at each new image. The autobiographical strategy's `imageStripHysteresisRatio` strips in batches instead, so the cache is rewritten once a batch, and its line names that option when it's off. - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - **Siblings:** no sibling's current range admits this release; each takes these changes on when it raises its range. - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md index 64fcde97..97068ca5 100644 --- a/changelog.d/request-image-ceilings.changed.md +++ b/changelog.d/request-image-ceilings.changed.md @@ -1,3 +1,3 @@ - The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders because of that ceiling, rather than because of a limit the host set. That's an image past the ceiling, unless the depth limit or a limit the host has in force under its ceiling strips it anyway, or, with `imageStripHysteresisRatio`, an image the rule's set strips in a trim that ceiling's limit brought on, where neither the depth limit nor a limit the host set was over too. That's judged at the trim alone, with two consequences: a trim that a limit the host set, or the depth limit, also brought on says nothing, though the ceiling's lower target there can strip an image the host's own limits keep; and the line is said for an image the ceiling's trim stripped even where a later trim would have stripped it anyway. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and the limit that ceiling sets was over: at an image the render replaces, or, with `imageStripHysteresisRatio`, where it brought a trim on anywhere in the rule's replay of the history. The depth limit or a limit the host set may be stripping images there too. - `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index 8942a9e7..baa450d2 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -158,36 +158,11 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b } /** - * Which ceilings decide that an image is stripped: those whose limit it - * passes (`over`) where the ceiling sets the window's limit (`ceilingSet`, - * see ceilingSetLimits), when nothing else strips it. An image passes the - * count limit when the window already keeps that many, and the byte limit - * when it doesn't fit with what's kept, a single image bigger than the limit - * included. Something else strips it when `strippedOtherwise` says so, as - * the host's depth limit does, or when it also passes a count or byte limit - * the host set itself and has in force, under its ceiling. A limit the host - * set above its ceiling isn't in force: the ceiling holds it, and the - * warning at initialize says so. Only a ceiling that decides changes what - * the window keeps, so only that one says its line. With the memory - * strategy's `imageStripHysteresisRatio`, whose set decides in place of the - * limits, the strategy applies this to each trim of its set: the limits over - * where the trim began, and the depth limit's, decide every image the trim - * strips (the memory strategy's ruleCeilingHits). - */ -export function ceilingsDeciding( - over: { images: boolean; bytes: boolean }, - strippedOtherwise: boolean, - ceilingSet: { images: boolean; bytes: boolean }, -): { images: boolean; bytes: boolean } { - const otherwise = strippedOtherwise || (over.images && !ceilingSet.images) || (over.bytes && !ceilingSet.bytes); - return otherwise ? { images: false, bytes: false } : { images: over.images, bytes: over.bytes }; -} - -/** - * The line a strategy says once for each ceiling, the first time that - * ceiling decides that an image is stripped (see ceilingsDeciding): which - * ceiling, at what value, and the window limit of 0 or above it that the - * ceiling holds, so an operator who set no limit sees where one now applies. + * The line a strategy says once for each ceiling, at the first render a + * select returns where images are shown as placeholders and a limit that + * ceiling sets (ceilingSetLimits) was over: which ceiling, at what value, + * and the window limit of 0 or above it that the ceiling holds, so an + * operator who set no limit sees where one now applies. */ export function ceilingBoundLine( strategy: string, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 38e1d280..b1c5cad0 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -92,7 +92,6 @@ import { liveImageLimitsWarning, ceilingBoundLine, ceilingSetLimits, - ceilingsDeciding, resolveLiveImageLimits, type ResolvedImageLimits, } from '../live-image-limits.js'; @@ -625,42 +624,26 @@ type ToolPruningDecision = { /** The verdict postStripEstimates gives one image, which the strip applies * (applyImageStripping's asEstimated): the image's payload (imagePayload), - * whether it's kept, and what decided it, for the ceiling line: the count - * and byte limits it passes and the depth limit (noteCeilings), or, with - * imageStripHysteresisRatio, whose set decides in place of the limits, its - * place among its message's images as the cut sends them (`nth`, from 0), - * which reportCeilingHits credits. */ + * whether it's kept, and, for the ceiling line, the count and byte limits + * it passes. */ type ImageVerdict = { key: unknown; keep: boolean; - over: { images: boolean; bytes: boolean }; - beyondDepth: boolean; - nth?: number; + over: CeilingKinds; }; -/** What decided the images the strip of a render select returns replaced, - * for the ceiling line (reportCeilingHits): the ceilings that decided one - * under the limits (ceilingsDeciding), and, with imageStripHysteresisRatio, - * each image the rule's set replaced, by its message and its place among - * that message's images. */ -type CeilingNotes = { - images: boolean; - bytes: boolean; - ruleStripped: Array<{ id: MessageId; nth: number }>; -}; +/** The count limit and the byte limit, by the ceiling that can set each. */ +type CeilingKinds = { images: boolean; bytes: boolean }; /** imageStripHysteresisRatio's set at the window's limits * (stripReplayAtLimits): how many of each message's images it strips, and - * for each stripped image, by its place among its message's images, which - * full limits were over where the trim that stripped it began. */ + * which of the count and byte limits brought a trim on anywhere in the + * history it replays, for the ceiling line. */ type StripReplay = { out: ReadonlyMap; - trims: ReadonlyMap>; + over: Readonly; }; -/** Which of the window's full limits were over where a trim began. */ -type TrimOver = { readonly images: boolean; readonly bytes: boolean; readonly depth: boolean }; - /** A tool call the conversation answers: its message's place in the listing, * the place of the result that first answered it, and every result that * answers it, by message and place in its content. */ @@ -7971,7 +7954,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // What this select's strips record is what it reports. A dry select's // strips, and a refused one's, record too; each live select starts by // clearing what earlier ones left, so it reports its own render's. - this._ceilingNotes = { images: false, bytes: false, ruleStripped: [] }; + this._ceilingNotes = { images: false, bytes: false }; let entries: ContextEntry[]; this.thinkingBound = opts?.thinkingBinding?.binds === true; try { @@ -7987,7 +7970,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // refused threw above, and a dry select never comes here. Every select // path strips its render last (applyImageStripping, `returned`), and a // strip with no limits records nothing, so the hits are this render's. - this.reportCeilingHits(store); + this.reportCeilingHits(); return entries; } // A dry run commits nothing, and that includes what a select leaves for @@ -17479,11 +17462,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { * so counting what the entries send has nothing to add, and a tool result * pairing repair moved keeps its own message's count. * - * `returned`: select returns this render, so the strip leaves what - * decided the images it replaces for select to report once it has - * returned it (reportCeilingHits): the ceilings that decided an image - * under the limits (noteCeilings), or, with `imageStripHysteresisRatio`, - * each image the rule's set replaces, which reportCeilingHits credits. + * `returned`: select returns this render, so where it replaces an image + * the strip notes which ceiling-set limits were over, for select to + * report once it has returned it (reportCeilingHits): under the limits, + * those the replaced images pass, and with `imageStripHysteresisRatio`, + * those that brought a trim on anywhere in the rule's replay. * A layout rendered only to compare against, such as kv-unified's kept * one, and any other caller's render, leave it unset. */ protected applyImageStripping( @@ -17509,17 +17492,15 @@ export class AutobiographicalStrategy implements ResettableStrategy { .filter(({ idx }) => this.hasImageBlock(entries[idx].content)) .sort((a, b) => b.pos - a.pos); - // What decided the images this strip replaces, for the ceiling line: - // under the limits, a ceiling whose limit an image passes, where that - // ceiling sets the limit and nothing else strips the image - // (ceilingsDeciding); under the rule's set, each image it replaces, which - // reportCeilingHits credits once select has returned the render. + // For the ceiling line: which limits a ceiling sets were over where this + // strip replaces an image. Under the limits, those the image passes, + // whatever else strips it too; under the rule's set, those that brought + // a trim on anywhere in its replay (stripReplayAtLimits). const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); - const notes: CeilingNotes = { images: false, bytes: false, ruleStripped: [] }; - const noteCeilings = (over: { images: boolean; bytes: boolean }, tooDeep: boolean): void => { - const deciding = ceilingsDeciding(over, tooDeep, ceilingSet); - if (deciding.images) notes.images = true; - if (deciding.bytes) notes.bytes = true; + const notes: CeilingKinds = { images: false, bytes: false }; + const noteCeilings = (over: Readonly): void => { + if (over.images && ceilingSet.images) notes.images = true; + if (over.bytes && ceilingSet.bytes) notes.bytes = true; }; if (opts.asEstimated || this.imageHysteresisRatio() !== undefined) { @@ -17535,6 +17516,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const judged = verdicts.get(home)?.get(group)?.[n]; return judged !== undefined && judged.key === imagePayload(image) ? judged : undefined; }; + const rule = this.imageHysteresisRatio() !== undefined; + let ruleReplaced = false; for (const { idx, pos } of ordered) { const keeps: boolean[] = []; for (const block of entries[idx].content) { @@ -17546,9 +17529,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const image of imagesIn([block])) { const judged = verdictOf(image, home, group); if (judged && !judged.keep) { - // A verdict exists only for a message in the listing (home >= 0). - if (judged.nth !== undefined) notes.ruleStripped.push({ id: messages[home]!.id, nth: judged.nth }); - else noteCeilings(judged.over, judged.beyondDepth); + if (rule) ruleReplaced = true; + else noteCeilings(judged.over); } keeps.push(judged?.keep ?? false); } @@ -17558,6 +17540,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { entries[idx].content = replaceImages(entries[idx].content, () => !keeps[next++], AutobiographicalStrategy.IMAGE_PLACEHOLDER); } + if (ruleReplaced) { + const replay = this.stripReplayAtLimits(store); + if (replay) noteCeilings(replay.over); + } if (opts.returned) this._ceilingNotes = notes; return; } @@ -17592,7 +17578,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + blockBytes > maxLiveBytes; if (tooDeep || overCount || overBytes) { - noteCeilings({ images: overCount, bytes: overBytes }, tooDeep); + noteCeilings({ images: overCount, bytes: overBytes }); return false; } keptImages++; @@ -17658,8 +17644,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { } /** The rule's set at the window's limits (imageStripReplay), memoized, - * with the limits that were over where the trim that stripped each image - * began (ruleCeilingHits reads them). */ + * with the count and byte limits that brought a trim on (applyImageStripping + * notes them for the ceiling line). */ private stripReplayAtLimits(store: MessageStoreView): StripReplay | undefined { const ratio = this.imageHysteresisRatio(); if (ratio === undefined) return undefined; @@ -17691,7 +17677,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { let first = 0; let liveBytes = 0; const out = new Map(); - const trims = new Map(); + const brought: CeilingKinds = { images: false, bytes: false }; for (let i = 0; i < messages.length; i++) { for (const image of imagesIn(this.cappedForSend(store, messages[i]!, msgCap).content)) { const bytes = inlineImageBytes(image); @@ -17706,8 +17692,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { const over = (depth: number, count: number, size: number): boolean => overCountOrBytes(count, size) || overDepth(depth); if (!over(depthCap, countCap, bytesCap)) continue; - // What brought this trim on, at the full limits. - const trim: TrimOver = { images: overCount(countCap), bytes: overBytes(bytesCap), depth: overDepth(depthCap) }; + if (overCount(countCap)) brought.images = true; + if (overBytes(bytesCap)) brought.bytes = true; const depth = Math.floor(depthCap * ratio); const countTarget = Math.floor(countCap * ratio); const sizeTarget = Math.floor(bytesCap * ratio); @@ -17724,14 +17710,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { liveBytes -= oldest.bytes; const id = messages[oldest.at]!.id; out.set(id, (out.get(id) ?? 0) + 1); - // Stripped oldest first, so its place in this list is its place - // among its message's images. - const ofMessage = trims.get(id); - if (ofMessage) ofMessage.push(trim); - else trims.set(id, [trim]); } } - const replay: StripReplay = { out, trims }; + const replay: StripReplay = { out, over: brought }; this._imageStripReplayMemo = { messages, calibration, depthCap, countCap, bytesCap, ratio, pruning, msgCap, replay }; return replay; }); @@ -17853,9 +17834,6 @@ export class AutobiographicalStrategy implements ResettableStrategy { const cut = this.cappedForSend(store, messages[i]!, msgCap).content; const groups = new Map(); let ruleLeft = replay !== undefined ? replay.get(messages[i]!.id) ?? 0 : undefined; - // Each image's place among the message's images, for the ceiling - // line's credit of the rule's set (reportCeilingHits). - let nth = 0; for (const block of cut) { const group = imageGroup(block); for (const b of imagesIn([block])) { @@ -17878,11 +17856,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } let judged = groups.get(group); if (!judged) groups.set(group, judged = []); - judged.push({ - key: imagePayload(b), keep, over: { images: beyondCount, bytes: beyondBytes }, beyondDepth, - nth: replay !== undefined ? nth : undefined, - }); - nth++; + judged.push({ key: imagePayload(b), keep, over: { images: beyondCount, bytes: beyondBytes } }); } } if (groups.size > 0) verdicts.set(i, groups); @@ -17905,58 +17879,37 @@ export class AutobiographicalStrategy implements ResettableStrategy { /** Which ceilings have been said to strip the live window (once each per instance). */ private readonly ceilingBoundReported = { images: false, bytes: false }; - /** What decided the images in the last render select returned (applyImageStripping, `returned`). */ - private _ceilingNotes: CeilingNotes = { images: false, bytes: false, ruleStripped: [] }; + /** Which ceiling-set limits were over where the last render select + * returned replaced an image (applyImageStripping, `returned`). */ + private _ceilingNotes: CeilingKinds = { images: false, bytes: false }; /** Say once for each ceiling, when a select returns a render in which - * images are shown as placeholders because of that ceiling, rather than - * because of a limit the host set. Under the limits, that's - * an image past the ceiling that nothing else strips (see - * ceilingsDeciding). With imageStripHysteresisRatio, the rule's set - * decides in place of the limits, so an image it replaces is credited - * to the ceiling whose limit brought on the trim that stripped it, with - * no limit the host set, and not the depth limit, over there - * (ruleCeilingHits). A dry - * select, a preview, an observer's reconstruction or a refused select - * says nothing, and the line waits for the first render returned. */ - private reportCeilingHits(store: MessageStoreView): void { + * images are shown as placeholders and a limit that ceiling sets was over + * (applyImageStripping's notes): under the limits, at an image the render + * replaces, and with imageStripHysteresisRatio, at a trim anywhere in the + * rule's replay. The depth limit or a limit the host set may be stripping + * too. While the rule is off, the line adds the cost of a window held at + * the ceiling and names the option that spaces it. The rule's set is this + * class's; a subclass's override of imageStripReplay isn't consulted for + * the line. A dry select, a preview, an observer's reconstruction or a + * refused select says nothing, and the line waits for the first render + * returned. */ + private reportCeilingHits(): void { const said = this.ceilingBoundReported; const notes = this._ceilingNotes; - const rule = this.ruleCeilingHits(store, notes.ruleStripped); for (const kind of ['images', 'bytes'] as const) { - if (!(notes[kind] || rule[kind]) || said[kind]) continue; + if (!notes[kind] || said[kind]) continue; said[kind] = true; - this.diag.warn(ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted())); + const line = ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted()); + this.diag.warn(this.imageHysteresisRatio() === undefined ? line + AutobiographicalStrategy.CEILING_BATCH_CLAUSE : line); } } - /** - * Which of the ceilings decided an image the rule's set replaced - * (`stripped`, by message and place among its images): the ceiling whose - * limit brought on the trim that stripped it, with neither the depth limit - * nor a limit the host set over there. That's ceilingsDeciding's rule, - * given what was over where the trim began. It reads the set's own replay, - * memoized for the listing the render was made from (stripReplayAtLimits), - * and replays nothing. The set is this class's rule; a subclass's override - * of imageStripReplay isn't consulted for it. - */ - private ruleCeilingHits( - store: MessageStoreView, - stripped: ReadonlyArray<{ id: MessageId; nth: number }>, - ): { images: boolean; bytes: boolean } { - const hits = { images: false, bytes: false }; - const replay = this.stripReplayAtLimits(store); - if (!replay) return hits; - const set = ceilingSetLimits(this.configuredImageLimits()); - for (const { id, nth } of stripped) { - const trim = replay.trims.get(id)?.[nth]; - if (!trim) continue; - const decided = ceilingsDeciding(trim, trim.depth, set); - if (decided.images) hits.images = true; - if (decided.bytes) hits.bytes = true; - } - return hits; - } + /** What the ceiling line adds while imageStripHysteresisRatio is off: the + * cost of a window held at a ceiling, and the option that spaces it. */ + private static readonly CEILING_BATCH_CLAUSE = + '; at the ceiling each new image turns the oldest live ones into placeholders, which rewrites the prompt ' + + 'cache from there, and imageStripHysteresisRatio does that in batches instead'; /** The image limits whose values are defaults: those the library's default * layer gave or no layer gave, and those the host gave as undefined or diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 200d6565..0ca71b10 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -27,7 +27,6 @@ import { liveImageLimitsWarning, ceilingBoundLine, ceilingSetLimits, - ceilingsDeciding, resolveLiveImageLimits, type ResolvedImageLimits, } from '../live-image-limits.js'; @@ -226,9 +225,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { return resolveLiveImageLimits(this.configuredImageLimits); } - /** Say once that an image ceiling, rather than a limit the options set, - * has decided that an image is stripped from the live window (see - * ceilingsDeciding). */ + /** Say once that an image ceiling holds a limit the options set as none + * or above it, once a limit it sets was over where the window strips an + * image (liveImagePolicy's `ceilings`). */ private reportCeilingBoundOnce(kind: 'images' | 'bytes'): void { if (this.ceilingBoundReported[kind]) return; this.ceilingBoundReported[kind] = true; @@ -522,9 +521,10 @@ export class WindowedPassthroughStrategy implements ContextStrategy { * each message's estimate gives back for the images it loses (`saved`). * `select` prices the window by `saved` and applies `strips` to what it * emits (stripImages), so the verdicts it prices are the ones the strip - * applies. `ceilings` names, for each message, the image ceilings that - * decided one of its images is stripped (ceilingsDeciding), for the line - * a returned render says (reportCeilings). + * applies. `ceilings` names, for each message, the limits an image + * ceiling sets (ceilingSetLimits) that were over where one of its images + * is stripped, whatever else strips it too, for the line a returned + * render says (reportCeilings). * * That holds wherever the window starts, and the re-anchor walk reads them * as they are, as `renderAsNewest` does from the batch's oldest message: @@ -580,10 +580,9 @@ export class WindowedPassthroughStrategy implements ContextStrategy { const overCount = maxLive > 0 && keptImages >= maxLive; const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes; const strip = tooDeep || overCount || overBytes; - const deciding = ceilingsDeciding({ images: overCount, bytes: overBytes }, tooDeep, ceilingSet); const decided = (ceilings[i] ??= { images: false, bytes: false }); - decided.images ||= deciding.images; - decided.bytes ||= deciding.bytes; + decided.images ||= overCount && ceilingSet.images; + decided.bytes ||= overBytes && ceilingSet.bytes; if (strip) { tokens += strippedImageSaving(image, placeholderPrice, calibration); } else { diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 2529f162..4c801e6f 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -1558,19 +1558,19 @@ export interface AutobiographicalConfig { * for what this doesn't count. 0 = no ceiling. A value that isn't 0 or a * finite number above it takes the default, and the strategy's warning at * initialize names it. A fraction is floored, to at least 1. Under a - * window limit of 0 or above this ceiling, the strategy says so once, at - * the first render a select returns where images are shown as - * placeholders because of this ceiling, rather than because of a limit - * the host set: an image past it that the host's own limits, the depth - * limit included, don't strip anyway, or, with - * `imageStripHysteresisRatio`, an image the rule's set strips in a trim - * this ceiling's limit brought on, where neither the depth limit nor a - * limit the host set was over too. That's judged at the trim alone, with - * two consequences: a trim that a limit the host set, or the depth limit, - * also brought on says nothing, though this ceiling's lower target there - * can strip an image the host's own limits keep; and the line is said for - * an image this ceiling's trim stripped even where a later trim would have - * stripped it anyway. + * window limit of 0 or above this ceiling, the strategy says once that + * this ceiling holds it, at the first render a select returns where images + * are shown as placeholders and the limit this ceiling sets was over: at + * an image the render replaces, or, with `imageStripHysteresisRatio`, + * where it brought a trim on anywhere in the rule's replay of the history. + * The depth limit or a limit the host set may be stripping images there + * too. + * + * A window held at a ceiling strips its oldest images as new ones come, + * as under any limit, but from deep in the request, so the prompt cache is + * rewritten from there at every new image. The memory strategy's + * `imageStripHysteresisRatio` strips in batches instead, so the cache is + * rewritten once a batch, and while that option is off the line names it. */ imageCeiling?: number; @@ -1604,7 +1604,9 @@ export interface AutobiographicalConfig { * `imageCeilings()` reports it, as for `imageCeiling`. 0 = no ceiling. A * value that isn't 0 or a finite number above it takes the default, and a * fraction is floored, to at least 1. Under a window limit of 0 or above - * this ceiling, the strategy says so once, as for `imageCeiling`. + * this ceiling, the strategy says so once, and a window held at it rewrites + * the prompt cache from deep in the request at every new image, both as + * for `imageCeiling`. */ imageByteCeiling?: number; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 319918a4..33497a77 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -459,6 +459,10 @@ describe('the line when an image ceiling first strips', () => { assert.deepEqual(byStep.map((step) => step.length), [1, 0], `said once, at the first step, for ${JSON.stringify(options)}`); assert.ok(lines[0]!.startsWith(`[${strategy.name}] ${said}, so some images are now shown as placeholders`), lines[0]); assert.match(lines[0]!, / ns=/, 'through the namespaced route the warning takes'); + // The memory strategy names the option that strips in batches, which + // is off here; windowed passthrough has none to name. + assert.equal(lines[0]!.includes('; at the ceiling each new image turns the oldest live ones into placeholders, which rewrites the prompt cache from there, and imageStripHysteresisRatio does that in batches instead'), + name === 'autobiographical', lines[0]); } }); @@ -494,7 +498,7 @@ describe('the line when an image ceiling first strips', () => { it(`${name}: says it when a message's first image passes the byte ceiling and its second fits`, async () => { // One message: a 12-byte image, then a 4-byte one, under a byte // ceiling of 8. The first is past it on its own and the second fits, - // so the message holds one image the ceiling decided and one it kept. + // so the message holds one image past the ceiling and one it kept. const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -527,28 +531,31 @@ describe('the line when an image ceiling first strips', () => { assert.ok(byStep[0]![0]!.includes('imageByteCeiling 8 bytes holds maxLiveImageBytes 0 (no limit) to it'), byStep[0]![0]); }); - it(`${name}: says nothing when the depth limit strips the image the ceiling would, and says it when only the ceiling does`, async () => { + it(`${name}: says it when the ceiling's limit is over at an image the render replaces, whether or not the depth limit strips it too`, async () => { // Two 8-byte images under a 12-byte ceiling: the older one is past the // ceiling. With a depth limit that the older one is also past, the - // depth strips it with or without the ceiling, so the ceiling decided - // nothing (Anarchid's probe on #268, R1-4). + // depth strips it with or without the ceiling; the line still says the + // ceiling holds the setting, since its limit is over there. const depth = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 2500 }; const alone = await ceilingLines(make({ ...depth, imageByteCeiling: 0 }), [{ add: 2, bytes: 8 }]); assert.equal(alone.last, '.I', 'the depth limit alone strips the older image'); + assert.deepEqual(alone.lines, [], 'no ceiling, no line'); const both = await ceilingLines(make({ ...depth, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); assert.equal(both.last, '.I', 'the ceiling changes nothing the window keeps'); - assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); + assert.equal(both.lines.length, 1, JSON.stringify(both.lines)); const ceilingOnly = await ceilingLines(make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); }); - it(`${name}: says nothing when the host's own count limit strips the image the byte ceiling would`, async () => { + it(`${name}: says it for the byte ceiling when its limit is over at an image the host's own count limit strips too`, async () => { // The older image is past the host's maxLiveImages of 1 and past the - // 12-byte ceiling: the host's limit strips it whatever the ceiling. + // 12-byte ceiling: the host's limit strips it whatever the ceiling, and + // the byte ceiling's limit is over there, so its line is said. const { lines, last } = await ceilingLines(make({ maxLiveImages: 1, maxLiveImageBytes: 0, imageByteCeiling: 12 }), [{ add: 2, bytes: 8 }]); assert.equal(last, '.I'); - assert.deepEqual(lines, [], JSON.stringify(lines)); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageByteCeiling 12 bytes holds'), lines[0]); }); it(`${name}: says nothing from renderAsNewest, whose render a ceiling strips, and leaves the line for the next compile`, async () => { @@ -611,6 +618,7 @@ describe('the line when an image ceiling first strips', () => { assert.equal(last, '...II', 'one trim to half the ceiling'); assert.equal(lines.length, 1, JSON.stringify(lines)); assert.ok(lines[0]!.includes('imageCeiling 4 holds maxLiveImages 0 (no limit) to it'), lines[0]); + assert.ok(!lines[0]!.includes('imageStripHysteresisRatio'), `the option is on, so the line doesn't name it: ${lines[0]}`); }); it('the memory strategy says it from its adaptive selection too, and nothing from a preview', async () => { @@ -715,7 +723,7 @@ describe('the line when an image ceiling first strips', () => { } }); - it("the memory strategy's adaptive selection says nothing when the depth limit strips the image the ceiling would, and says it when only the ceiling does", async () => { + it("the memory strategy's adaptive selection says it when the ceiling's limit is over at an image it replaces, whether or not the depth limit strips it too", async () => { // The adaptive selection's strip applies the post-strip estimates' // verdicts, which carry the depth limit's own: two 8-byte images under a // 12-byte ceiling, the older one also past a 2,500-token depth limit. @@ -725,7 +733,7 @@ describe('the line when an image ceiling first strips', () => { }); const depth = await ceilingLines(adaptive({ imageStripDepthTokens: 2500 }), [{ add: 2, bytes: 8 }]); assert.equal(depth.last, '.I', 'the depth limit strips the older image'); - assert.deepEqual(depth.lines, [], JSON.stringify(depth.lines)); + assert.equal(depth.lines.length, 1, JSON.stringify(depth.lines)); const ceilingOnly = await ceilingLines(adaptive({}), [{ add: 2, bytes: 8 }]); assert.equal(ceilingOnly.last, '.I', 'without the depth limit, the ceiling strips the older image'); assert.equal(ceilingOnly.lines.length, 1, JSON.stringify(ceilingOnly.lines)); @@ -786,9 +794,9 @@ describe('the line when an image ceiling first strips', () => { } }); - it("the memory strategy credits the rule's set from the stored messages alone, so a reopened instance says it at its first compile", async () => { + it("the memory strategy's rule reads its set from the stored messages alone, so a reopened instance says it at its first compile", async () => { // The rule keeps no state: a second instance on the same store replays - // the same set, which the ceiling decided, so it says the line at the + // the same set, with the ceiling's trim in it, so it says the line at the // first compile it returns, as the first instance did. const said: string[] = []; const original = console.warn; @@ -822,7 +830,7 @@ describe('the line when an image ceiling first strips', () => { ['count', { maxLiveImages: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }, 'imageCeiling 4 holds'], ['byte', { maxLiveImages: 0, maxLiveImageBytes: 0, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }, 'imageByteCeiling 16 bytes holds'], ] as const) { - it(`the memory strategy credits the rule's set from the branch it compiles, under the ${kind} ceiling`, async () => { + it(`the memory strategy's rule reads the set of the branch it compiles, under the ${kind} ceiling`, async () => { const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; @@ -871,32 +879,8 @@ describe('the line when an image ceiling first strips', () => { assert.deepEqual(both.lines, [], JSON.stringify(both.lines)); }); - it("the memory strategy's rule says it when only the ceiling strips, and nothing when the depth limit strips the same anyway", async () => { - // Image messages price at about 1,600 tokens, so three of them are within - // a 6,000-token depth limit: without a ceiling nothing binds and nothing - // is stripped. A count ceiling of 2 binds at the third, and its trim to - // half strips the two older images, which the depth limit alone keeps: - // the trim is the ceiling's alone. - const options = { maxLiveImages: 0, imageStripDepthTokens: 6000, imageStripHysteresisRatio: 0.5 }; - const alone = await ceilingLines(memory({ ...options, imageCeiling: 0 }), [3]); - assert.equal(alone.last, 'III', 'the depth limit alone strips nothing'); - const ceiling = await ceilingLines(memory({ ...options, imageCeiling: 2 }), [3]); - assert.equal(ceiling.last, '..I', 'the ceiling strips what the depth limit alone keeps'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageCeiling 2 holds'), ceiling.lines[0]); - // Under a 3,000-token depth limit, the first image is past the depth once - // the second comes, before the count ceiling is passed: the trims that - // strip are the depth's, and the ceiling's limit brings none on. - const deeper = { maxLiveImages: 0, imageStripDepthTokens: 3000, imageStripHysteresisRatio: 0.5 }; - const deepAlone = await ceilingLines(memory({ ...deeper, imageCeiling: 0 }), [3]); - assert.equal(deepAlone.last, '..I', 'the depth limit alone strips the two older images'); - const deepCeiling = await ceilingLines(memory({ ...deeper, imageCeiling: 2 }), [3]); - assert.equal(deepCeiling.last, deepAlone.last, 'the ceiling changes nothing the window keeps'); - assert.deepEqual(deepCeiling.lines, [], JSON.stringify(deepCeiling.lines)); - }); - - /** Compile, after adding each message in turn, with a strategy built from - * `options`; the window kept and the ceiling lines said. */ + /** The memory strategy over `messages`, compiled once: the pattern its + * image messages kept, and the ceiling lines it said. */ async function ruleRun(options: Record, messages: unknown[][]) { const said: string[] = []; const original = console.warn; @@ -915,351 +899,87 @@ describe('the line when an image ceiling first strips', () => { } } - it("the memory strategy's rule names the byte ceiling whose trim stripped an image first, though the depth limit strips it later", async () => { - // A 16-byte image, an 8-byte image, then about 1,500 tokens of text. When - // the second image comes, the first passes a 20-byte ceiling (24 bytes) - // while it's within a 4,000-token depth limit (about 3,200 tokens deep), - // so the ceiling's trim strips it then. The text takes it past the depth - // (about 4,700), where the set without the ceiling strips it: the render - // is the same, but the ceiling's trim stripped it first. - const text = [{ type: 'text', text: 'word '.repeat(870) }]; - const options = { maxLiveImages: 0, maxLiveImageBytes: 0, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.9 }; - const messages = [imageMessage('img-1', 16), imageMessage('img-2', 8), text]; - const alone = await ruleRun({ ...options, imageByteCeiling: 0 }, messages); - assert.equal(alone.kept, '.I', 'the depth limit alone strips the first image'); - const ceiling = await ruleRun({ ...options, imageByteCeiling: 20 }, messages); - assert.equal(ceiling.kept, '.I', 'the render is the same'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageByteCeiling 20 bytes holds'), ceiling.lines[0]); - }); - - it("the memory strategy's rule says nothing for a trim the host's own byte limit brings on too, though the ceiling's target strips more, and names a trim the ceiling brings on alone", async () => { - // A 32-byte image, then three 4-byte ones, under a count ceiling of 3. - // At the fourth image the count passes the ceiling. With a host byte - // limit of 40, in force, the bytes (44) pass it too: the trim is the - // host's as well as the ceiling's, so the line isn't said, though the - // count ceiling's target, 1, strips more than the host's limit alone - // would. With a host byte limit of 60 the bytes don't pass it, and the - // trim is the ceiling's alone. + it("the memory strategy's rule says it when the ceiling's limit brings a trim on, though the host's own byte limit or the depth limit is over there too", async () => { + // A 32-byte image, then three 4-byte ones, under a count ceiling of 3 and + // a host byte limit of 40, in force: at the fourth image the count passes + // the ceiling and the bytes (44) pass the host's limit, so one trim has + // both over. The line is said, since the ceiling's limit brought it on. const steps = [{ add: 1, bytes: 32 }, { add: 3, bytes: 4 }]; - const host = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 0, imageStripHysteresisRatio: 0.5 }), steps); - const mixed = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); - assert.notEqual(mixed.last, host.last, `premise: the ceiling's target strips more (${mixed.last} against ${host.last})`); - assert.deepEqual(mixed.lines, [], JSON.stringify(mixed.lines)); - const alone = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 60, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); - assert.ok(alone.last.includes('.'), `premise: the ceiling's trim strips (${alone.last})`); - assert.equal(alone.lines.length, 1, JSON.stringify(alone.lines)); - assert.ok(alone.lines[0]!.includes('imageCeiling 3 holds'), alone.lines[0]); - }); - - it("the memory strategy's rule names the count ceiling whose limit brought the trim on, not the byte ceiling", async () => { - // Both window limits are set by their ceilings. The fifth 4-byte image - // passes the count ceiling of 4, and 20 bytes are far under the byte - // ceiling, so the trim is the count ceiling's alone. - const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageStripHysteresisRatio: 0.5 }), - [5], - ); - assert.equal(last, '...II'); - assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('imageCeiling 4 holds'), lines[0]); - }); - - it("the memory strategy's rule names both ceilings when both their limits bring the trim on", async () => { - // Both window limits are set by their ceilings: 4 images and 16 bytes. - // Five 4-byte images pass both (5 images, 20 bytes) at the fifth, with no - // limit the host set, so each ceiling decided the images the trim strips. - const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 4, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), - [5], - ); - assert.equal(last, '...II'); - assert.equal(lines.length, 2, JSON.stringify(lines)); - assert.ok(lines.some((line) => line.includes('imageCeiling 4 holds')), JSON.stringify(lines)); - assert.ok(lines.some((line) => line.includes('imageByteCeiling 16 bytes holds')), JSON.stringify(lines)); - }); - - it("the memory strategy's rule names the count ceiling for images its trim stripped, though the host's own limit strips every image later", async () => { - // A count ceiling of 4 and a host byte limit of 40, in force. The fifth - // image passes the count ceiling with the bytes under the host's limit - // (36), so that trim is the ceiling's alone. A 44-byte image then passes - // the host's byte limit on its own: the host's limit strips every image - // now, so the render is the same with or without the ceiling, but the - // ceiling's trim stripped some of them first. - const messages = [imageMessage('img-1', 20), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`)), imageMessage('img-6', 44)]; - const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; - const alone = await ruleRun({ ...options, imageCeiling: 0 }, messages); - const ceiling = await ruleRun({ ...options, imageCeiling: 4 }, messages); - assert.equal(ceiling.kept, '......'); - assert.equal(ceiling.kept, alone.kept, 'the render is the same without the ceiling'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageCeiling 4 holds'), ceiling.lines[0]); - }); - - it("the memory strategy's rule says nothing for a trim the count ceiling and the host's own byte limit bring on together", async () => { - // A count ceiling of 4 and a host byte limit of 40, in force. When the - // fifth image comes (8 bytes, after a 24-byte one and three 4-byte ones), - // it passes both: the count ceiling and 44 bytes. The host's limit alone - // trims to 20 bytes, stripping the oldest; the count ceiling's trim, to - // 2, strips the two after it too, which the host's limit keeps. The trim - // is the host's as well, so the line isn't said. - const messages = [imageMessage('img-1', 24), ...[2, 3, 4].map((i) => imageMessage(`img-${i}`)), imageMessage('img-5', 8)]; - const options = { maxLiveImages: 0, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }; - const alone = await ruleRun({ ...options, imageCeiling: 0 }, messages); - assert.equal(alone.kept, '.IIII', "the host's limit alone strips the oldest"); - const ceiling = await ruleRun({ ...options, imageCeiling: 4 }, messages); - assert.equal(ceiling.kept, '...II'); - assert.deepEqual(ceiling.lines, []); - }); - - it("the memory strategy's rule says nothing for a trim the byte ceiling and the host's own count limit bring on together", async () => { - // The mirror, with a byte ceiling of 20 and a host count limit of 4, in - // force. When the fifth image comes (8 bytes, after three 4-byte ones and - // an 8-byte one), it passes the host's count and the byte ceiling (28 - // bytes). The host's count alone trims to 2, stripping the three oldest; - // the byte ceiling's trim, to 10 bytes, strips the 8-byte fourth image - // too, which the host's count keeps. The trim is the host's as well. - const messages = [...[1, 2, 3].map((i) => imageMessage(`img-${i}`)), imageMessage('img-4', 8), imageMessage('img-5', 8)]; - const options = { maxLiveImages: 4, maxLiveImageBytes: 0, imageStripHysteresisRatio: 0.5 }; - const alone = await ruleRun({ ...options, imageByteCeiling: 0 }, messages); - assert.equal(alone.kept, '...II', "the host's count alone strips the three oldest"); - const ceiling = await ruleRun({ ...options, imageByteCeiling: 20 }, messages); - assert.equal(ceiling.kept, '....I'); - assert.deepEqual(ceiling.lines, []); - }); - - it("the memory strategy's rule credits by each image's place in its message", async () => { - // A host byte limit of 19, in force, and a count ceiling of 3. A message - // of three images, 12, 4 and 4 bytes, passes the host's limit (20 bytes) - // and not the count: that trim is the host's, and strips the message's - // first image. Two 4-byte images follow; the second passes the count - // ceiling (4 images, 16 bytes), and that trim, the ceiling's alone, - // strips the message's second and third images. So the ceiling decided - // those two, by their places in their message, and not its first. - const image = (data: string): ContentBlock => - ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' }, tokenEstimate: 100 }) as ContentBlock; - const triple = [image('A'.repeat(12)), image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'triple' }] as ContentBlock[]; - const messages = [triple, imageMessage('img-x'), imageMessage('img-y'), imageMessage('img-z')]; - const kept = async (options: Record) => { - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy: memory(options) }); - try { - for (const content of messages) manager.addMessage('user', content); - const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - const of = (label: string) => compiled.messages.find((m) => m.content.some((b) => b.type === 'text' && b.text === label)); - const marks = (label: string) => of(label)!.content.filter((b) => b.type === 'image' || (b.type === 'text' && b.text === PLACEHOLDER)) - .map((b) => b.type === 'image' ? 'I' : '.').join(''); - return { kept: ['triple', 'img-x', 'img-y', 'img-z'].map(marks).join(' '), lines: said.filter((line) => line.includes(CEILING_LINE)) }; - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } - }; - const options = { maxLiveImages: 0, maxLiveImageBytes: 19, imageStripHysteresisRatio: 0.9 }; - const alone = await kept({ ...options, imageCeiling: 0 }); - assert.equal(alone.kept, '..I I I I', "the host's limit alone strips the message's first image, and its second at the last image"); - const ceiling = await kept({ ...options, imageCeiling: 3 }); - assert.equal(ceiling.kept, '... I I I', "the count ceiling's trim strips the message's second and third images"); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageCeiling 3 holds'), ceiling.lines[0]); - - // The other order, under a host byte limit of 30: a message of four - // 4-byte images passes the count ceiling alone (4 images, 16 bytes), and - // that trim strips its first two. A 28-byte image then passes the host's - // limit alone (36 bytes), and that trim strips the other two. So the - // ceiling decided the message's first two images, and not its last two. - const quad = [...['A', 'B', 'C', 'D'].map((c) => image(c.repeat(4))), { type: 'text', text: 'triple' }] as ContentBlock[]; - messages.splice(0, messages.length, quad, imageMessage('img-x', 28), imageMessage('img-y'), imageMessage('img-z')); - const second = await kept({ maxLiveImages: 0, maxLiveImageBytes: 30, imageStripHysteresisRatio: 0.9, imageCeiling: 3 }); - assert.ok(second.kept.startsWith('....'), `every image of the message is stripped (${second.kept})`); - assert.equal(second.lines.length, 1, JSON.stringify(second.lines)); - assert.ok(second.lines[0]!.includes('imageCeiling 3 holds'), second.lines[0]); - }); - - it("the memory strategy's rule says nothing for a trim the depth limit brings on too", async () => { - // A count ceiling of 2 and a 4,000-token depth limit. An image, then a - // message of two images and about 5,000 tokens of text: there the count - // passes the ceiling (3 images) and the first image is past the depth, - // so the trim is the depth's as well, and the line isn't said. + const host = await ceilingLines(memory({ maxLiveImages: 0, maxLiveImageBytes: 40, imageCeiling: 3, imageStripHysteresisRatio: 0.5 }), steps); + assert.ok(host.last.includes('.'), `premise: the trim strips (${host.last})`); + assert.equal(host.lines.length, 1, JSON.stringify(host.lines)); + assert.ok(host.lines[0]!.includes('imageCeiling 3 holds'), host.lines[0]); + + // A count ceiling of 2 and a 4,000-token depth limit: an image, then a + // message of two images with about 5,000 tokens of text, where the count + // passes the ceiling and the first image is past the depth, both at once. const image = (data: string): ContentBlock => ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' }, tokenEstimate: 100 }) as ContentBlock; - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const manager = await ContextManager.open({ - path: join(dir, 'store'), - strategy: memory({ maxLiveImages: 0, imageCeiling: 2, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }), - }); - try { - manager.addMessage('user', imageMessage('img-1')); - manager.addMessage('user', [image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'pair ' + 'word '.repeat(2900) }]); - const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); - const placeholders = compiled.messages.flatMap((m) => m.content).filter((b) => b.type === 'text' && b.text === PLACEHOLDER).length; - assert.ok(placeholders > 0, 'premise: the trim strips'); - assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), []); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } - }); - - it("the memory strategy's rule says nothing for images the ceiling's trim strips that the host's own limit strips too, later", async () => { - // A count ceiling of 4 and a host byte limit of 40, in force. The fifth - // image passes both, and the set strips the three oldest. Without the - // ceiling the host's byte limit strips those three as well, once the - // 24-byte image comes, so the ceiling decided none of them. - const messages = [...[1, 2, 3].map((i) => imageMessage(`img-${i}`, 12)), imageMessage('img-4'), imageMessage('img-5'), imageMessage('img-6', 24)]; - const ceiling = await ruleRun({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }, messages); - assert.equal(ceiling.kept, '...III'); - assert.deepEqual(ceiling.lines, []); - }); - - it("the memory strategy's rule names the byte ceiling for an image its trim stripped, though the host's own count limit strips it later", async () => { - // A byte ceiling of 20 and a host count limit of 4, in force. A 16-byte - // image and two 4-byte ones: the third passes the byte ceiling (24 bytes) - // with the count under the host's limit, and the ceiling's trim strips - // the 16-byte image. Without the ceiling the host's count strips it as - // well, with two more, when the fifth image comes. - const messages = [imageMessage('img-1', 16), ...[2, 3, 4, 5].map((i) => imageMessage(`img-${i}`))]; - const ceiling = await ruleRun({ maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 20, imageStripHysteresisRatio: 0.5 }, messages); - assert.equal(ceiling.kept, '.IIII'); - assert.equal(ceiling.lines.length, 1, JSON.stringify(ceiling.lines)); - assert.ok(ceiling.lines[0]!.includes('imageByteCeiling 20 bytes holds'), ceiling.lines[0]); - }); - - for (const [kind, options, why] of [ - ['byte', { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 16 }, 'the host byte limit of 16 strips what the count ceiling of 4 does'], - ['count', { maxLiveImages: 4, maxLiveImageBytes: 0, imageByteCeiling: 16 }, 'the host count limit of 4 strips what the byte ceiling of 16 does'], - ] as const) { - it(`the memory strategy's rule says nothing when the host's own ${kind} limit strips what a ceiling does`, async () => { - // The fifth 4-byte image passes both the host's limit and the ceiling, - // and both trims, to 2 images and to 8 bytes, strip the three oldest: - // ${why}. - const { lines, last } = await ceilingLines(memory({ ...options, imageStripHysteresisRatio: 0.5 }), [5]); - assert.equal(last, '...II'); - assert.deepEqual(lines, [], why); - }); - } - - it("the memory strategy's rule credits the ceiling whose trim it is, where the trim stops at the depth", async () => { - // A count ceiling of 4 and a 20,000-token depth limit. The fifth image's - // message passes the ceiling, with nothing past the depth. About 9,000 - // tokens of text before the newest image put the second-newest past half - // the depth, so the trim strips it by the depth's target, not the - // count's. The trim is the ceiling's alone, begun with nothing past the - // full depth, so the ceiling decided all four. - const said: string[] = []; - const original = console.warn; - console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; - const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, imageStripDepthTokens: 20000, imageStripHysteresisRatio: 0.5 }); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - try { - for (let i = 1; i <= 4; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - manager.addMessage('user', [{ type: 'text', text: 'word '.repeat(5200) }]); - manager.addMessage('user', imageMessage('img-5')); - const kept = pattern((await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 })).messages, 5); - assert.equal(kept, '....I', 'the trim strips the second-newest image too'); - const ours = said.filter((line) => line.includes(CEILING_LINE)); - assert.equal(ours.length, 1, JSON.stringify(said)); - assert.ok(ours[0]!.includes('imageCeiling 4 holds'), ours[0]); - } finally { - console.warn = original; - await manager.close(); - rmSync(dir, { recursive: true, force: true }); - } + const depth = await ruleRun({ maxLiveImages: 0, imageCeiling: 2, imageStripDepthTokens: 4000, imageStripHysteresisRatio: 0.5 }, [ + imageMessage('img-1'), + [image('B'.repeat(4)), image('C'.repeat(4)), { type: 'text', text: 'img-2' }, { type: 'text', text: 'word '.repeat(2900) }], + ]); + assert.ok(depth.kept.startsWith('.'), `premise: the trim strips the first image (${depth.kept})`); + assert.equal(depth.lines.length, 1, JSON.stringify(depth.lines)); + assert.ok(depth.lines[0]!.includes('imageCeiling 2 holds'), depth.lines[0]); }); - it("the memory strategy's rule names only the ceiling whose limit brings its trims on", async () => { - // Both window limits are set by their ceilings: 2 images and 40 bytes. - // Eleven 4-byte images: the count ceiling binds at every third, and its - // trims keep the live bytes under half the byte ceiling, so the byte limit - // is never over where a trim begins. - const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 40, imageStripHysteresisRatio: 0.5 }), - [11], + it("the memory strategy's rule says it from its own replay, not from the limits' account of the images it strips", async () => { + // A host byte limit of 40, in force, and a count ceiling of 4. A 36-byte + // image and an 8-byte one pass the host's limit (44 bytes), and that + // trim, to 20 bytes, strips the 36-byte image. Three 4-byte images then + // make four live, 20 bytes, so no trim's start ever has the count over + // its ceiling. By the limits' own account, newest first, the 36-byte + // image is past the count ceiling (four kept before it), but the set + // decides it, and the replay's trims were the host's limit's alone. + const { lines, patterns } = await ceilingLines( + memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), + [{ add: 1, bytes: 36 }, { add: 1, bytes: 8 }, { add: 3, bytes: 4 }], ); - assert.equal(last.slice(0, 6), '......', 'the count ceiling strips the oldest images'); - assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); + assert.deepEqual(patterns, ['I', '.I', '.IIII']); + assert.deepEqual(lines, [], JSON.stringify(lines)); }); - it("the memory strategy's rule names only the ceiling whose limit brought the trim on, though it trims to the other's target too", async () => { - // Both window limits are set by their ceilings: 2 images and 16 bytes. - // The third 4-byte image passes the count ceiling, and the trim, to 1 - // image and 8 bytes, strips the two older ones. The bytes (12) are over - // the trim's 8 but weren't over 16 where it began, so only the count - // ceiling decided. + it("the memory strategy's rule says nothing for a trim the host's own byte limit brings on, though the ceiling-set count is over its target there", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force. Three + // 4-byte images, then a 32-byte one: the bytes (44) pass the host's + // limit, and the count, 4, isn't over its ceiling, though it's over the + // trim's target of 2. The trim is the host's, so no line. const { lines, last } = await ceilingLines( - memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 16, imageStripHysteresisRatio: 0.5 }), - [3], - ); - assert.equal(last, '..I'); - assert.equal(lines.length, 1, JSON.stringify(lines)); - assert.ok(lines[0]!.includes('imageCeiling 2 holds'), lines[0]); - }); - - it("the memory strategy's rule credits only its set, not the limits' account of the images it strips", async () => { - // A host byte limit of 40, in force, and a count ceiling of 4. Four 8-byte - // images, then a 12-byte one, which passes both (5 images, 44 bytes): the - // trim, to 2 images and 20 bytes, strips the three oldest, and the host's - // byte limit alone strips the same three. Two 4-byte images then make four - // live, 28 bytes. By the limits' own account, newest first, the third - // oldest is now past the count ceiling and within the host's bytes, but - // the set decides it, and the trim that stripped it was the host's too. - const { lines, patterns } = await ceilingLines( memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }), - [{ add: 4, bytes: 8 }, { add: 1, bytes: 12 }, { add: 2, bytes: 4 }], + [3, { add: 1, bytes: 32 }], ); - assert.deepEqual(patterns, ['IIII', '...II', '...IIII']); + assert.ok(last.includes('.'), `premise: the trim strips (${last})`); assert.deepEqual(lines, [], JSON.stringify(lines)); }); - it("the memory strategy's credit reads the set's own trims and replays nothing, whether or not it says the line", async () => { - // The credit is kept with the set, as the replay builds it, so reporting - // reads it and replays nothing of its own: at the compile that says a - // line, and at each later compile whose render replaced images that no - // ceiling decided, as here, where the host's byte limit strips them. - const budget = { maxTokens: 1_000_000, reserveForResponse: 0 }; + it("the memory strategy's rule says nothing at a render that replaces no image, though its replay has a ceiling's trim", async () => { + // Three images under a count ceiling of 2: the third brings a trim on + // that strips the first two. Then enough text that compression folds the + // image messages into summaries before the first compile: the replay + // still has the ceiling's trim, but the render shows no placeholder, so + // the line waits. + const membrane = { + complete: async () => ({ stopReason: 'end_turn', content: [{ type: 'text', text: '[mock summary] ' + 'x '.repeat(40) }], usage: { input_tokens: 100, output_tokens: 25 } }), + } as never; + const prose = (label: string, tokens: number): ContentBlock[] => [{ type: 'text', text: `${label} ` + 'abcd'.repeat(Math.round(tokens * 2.9 / 4)) }]; const said: string[] = []; const original = console.warn; console.warn = (...parts: unknown[]) => { said.push(parts.map(String).join(' ')); }; const dir = mkdtempSync(join(tmpdir(), 'cm-request-ceilings-')); - const strategy = memory({ maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }); - const manager = await ContextManager.open({ path: join(dir, 'store'), strategy }); - const s = strategy as unknown as { - cappedForSend: (...args: unknown[]) => unknown; - ruleCeilingHits: (store: unknown, stripped: unknown[]) => unknown; - }; - let crediting = false; - let cuts = 0; - const credited: number[] = []; - const cut = s.cappedForSend.bind(strategy); - s.cappedForSend = (...args: unknown[]) => { if (crediting) cuts++; return cut(...args); }; - const credit = s.ruleCeilingHits.bind(strategy); - s.ruleCeilingHits = (store: unknown, stripped: unknown[]) => { - crediting = true; - credited.push(stripped.length); - try { return credit(store, stripped); } finally { crediting = false; } - }; + const strategy = memory({ + compressionModel: 'mock', targetChunkTokens: 1500, headWindowTokens: 0, recentWindowTokens: 9000, hierarchical: true, + maxLiveImages: 0, imageCeiling: 2, imageStripHysteresisRatio: 0.5, + }); + const manager = await ContextManager.open({ path: join(dir, 'store'), strategy, membrane }); try { - // Five 4-byte images: the fifth passes the count ceiling alone. - for (let i = 1; i <= 5; i++) manager.addMessage('user', imageMessage(`img-${i}`)); - await manager.compile(budget); - assert.equal(said.filter((line) => line.includes(CEILING_LINE)).length, 1, 'the line, at the first compile'); - // Then 30-byte images, each passing the host's byte limit. - for (let round = 0; round < 3; round++) { - manager.addMessage('user', imageMessage(`big-${round}`, 30)); - await manager.compile(budget); - } - assert.ok(credited.length >= 4 && credited.every((n) => n > 0), `premise: every compile's render replaced images (${credited})`); - assert.equal(cuts, 0, 'reporting replayed nothing'); + for (let i = 1; i <= 3; i++) manager.addMessage('user', imageMessage(`img-${i}`)); + for (let k = 0; k < 20; k++) manager.addMessage('user', prose(`t${k}`, 800)); + for (let k = 0; k < 500 && !manager.isReady(); k++) await manager.tick(); + const compiled = await manager.compile({ maxTokens: 1_000_000, reserveForResponse: 0 }); + assert.equal(pattern(compiled.messages, 3), '---', 'premise: every image message is folded into a summary'); + assert.ok(!compiled.messages.some((m) => m.content.some((b) => b.type === 'text' && b.text === PLACEHOLDER)), 'premise: no placeholder'); + assert.deepEqual(said.filter((line) => line.includes(CEILING_LINE)), []); } finally { console.warn = original; await manager.close(); @@ -1267,10 +987,10 @@ describe('the line when an image ceiling first strips', () => { } }); - it("the memory strategy's credit isn't read from a preview's set under ceilings that set no limit", async () => { + it("the memory strategy's line isn't read from a preview's set under ceilings that set no limit", async () => { // A preview with maxLiveImages 4 under a count ceiling of 10 limits the // count to 4, as the configured 0 under a ceiling of 4 does, but there - // the host sets the limit, so its trims decide nothing for a ceiling. + // the host sets the limit, so its trims say nothing for a ceiling. // The compile after it, under the configured ceilings, still names the // count ceiling its own trim brought on. const said: string[] = []; @@ -1301,7 +1021,7 @@ describe('the line when an image ceiling first strips', () => { // Two image messages under a count ceiling of 1, about 2,000 tokens of // text between them, past a 3,000-token budget: the window re-anchors to // the newest message, whose image the ceiling keeps. The verdicts are - // decided from the old anchor, where the ceiling strips the older image, + // made from the old anchor, where the ceiling strips the older image, // which the render no longer holds. With room for both, the same images // say the line. const said: string[] = []; @@ -1328,12 +1048,11 @@ describe('the line when an image ceiling first strips', () => { }); for (const hierarchical of [true, false]) { - it(`the memory strategy's rule credits a moved tool result's image to its own message (${hierarchical ? 'hierarchical' : 'adaptive'})`, async () => { + it(`the memory strategy's rule strips a moved tool result's image as its own message's, and says the line (${hierarchical ? 'hierarchical' : 'adaptive'})`, async () => { // Result a holds image A, ahead of image B in its message. Under a count // ceiling of 1 the rule strips that message's first image, A, and keeps // B. Pairing repair moves result a up beside result b, into another - // message's entry; A is still its own message's first image, which the - // set without the ceiling keeps. + // message's entry; A is still its own message's first image. const image = (data: string): ContentBlock => ({ type: 'image', source: { type: 'base64', data, mediaType: 'image/png' } }) as ContentBlock; const said: string[] = []; const original = console.warn; From 70bdf1778a017cf04333dd00952e401d57af9553 Mon Sep 17 00:00:00 2001 From: Milo-2653 Date: Sat, 24 Oct 2026 03:03:07 -0400 Subject: [PATCH 61/67] fix(images): review round: the clause where the option can't space trims, both ceilings at once, and texts The memory strategy's ceiling line named imageStripHysteresisRatio as the remedy for a window held at a ceiling even under a count ceiling of 1, where the rule keeps the newest image at every trim and so can't space them. The clause is now said for the count ceiling only above 1. Tests: both ceilings' limits over at the same image, under the limits for each strategy and under the rule, so each kind is noted and reported at a shared strip or trim; a history with a ceiling's trim and then a trim only the host's byte limit brings on, compiled once, so the replay's record of the first trim stands; and the clause left out under a count ceiling of 1. Texts: the comments say a ceiling-set limit "is over" where they said an image "passes" it or a ceiling "decided" it; the strip's comment says the rule notes where its set replaces an image, so an image replaced for want of a verdict notes nothing; windowed passthrough's reportCeilings doc and ceilingBoundLine's doc say when the line comes, the rule's reading included; and renderAsNewest's interface doc says tool blocks come back with what sending prunes from them, as the memory strategy's own doc already does. --- src/live-image-limits.ts | 8 +++-- src/strategies/autobiographical.ts | 29 ++++++++++------ src/strategies/windowed-passthrough.ts | 2 +- src/types/strategy.ts | 5 +-- test/request-image-ceilings.test.ts | 46 ++++++++++++++++++++++++++ 5 files changed, 73 insertions(+), 17 deletions(-) diff --git a/src/live-image-limits.ts b/src/live-image-limits.ts index baa450d2..e0269b81 100644 --- a/src/live-image-limits.ts +++ b/src/live-image-limits.ts @@ -160,9 +160,11 @@ export function ceilingSetLimits(configured: ConfiguredImageLimits): { images: b /** * The line a strategy says once for each ceiling, at the first render a * select returns where images are shown as placeholders and a limit that - * ceiling sets (ceilingSetLimits) was over: which ceiling, at what value, - * and the window limit of 0 or above it that the ceiling holds, so an - * operator who set no limit sees where one now applies. + * ceiling sets (ceilingSetLimits) was over: at an image the render + * replaces, or, under the memory strategy's `imageStripHysteresisRatio`, at + * the start of a trim anywhere in the rule's replay. It says which ceiling, + * at what value, and the window limit of 0 or above it that the ceiling + * holds, so an operator who set no limit sees where one now applies. */ export function ceilingBoundLine( strategy: string, diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index b1c5cad0..8063cff1 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -17465,8 +17465,9 @@ export class AutobiographicalStrategy implements ResettableStrategy { * `returned`: select returns this render, so where it replaces an image * the strip notes which ceiling-set limits were over, for select to * report once it has returned it (reportCeilingHits): under the limits, - * those the replaced images pass, and with `imageStripHysteresisRatio`, - * those that brought a trim on anywhere in the rule's replay. + * those a replaced image is over, and with `imageStripHysteresisRatio`, + * where the rule's set replaces an image, those that brought a trim on + * anywhere in the rule's replay. * A layout rendered only to compare against, such as kv-unified's kept * one, and any other caller's render, leave it unset. */ protected applyImageStripping( @@ -17493,9 +17494,11 @@ export class AutobiographicalStrategy implements ResettableStrategy { .sort((a, b) => b.pos - a.pos); // For the ceiling line: which limits a ceiling sets were over where this - // strip replaces an image. Under the limits, those the image passes, - // whatever else strips it too; under the rule's set, those that brought - // a trim on anywhere in its replay (stripReplayAtLimits). + // strip replaces an image. Under the limits, those the image is over, + // whatever else strips it too; where the rule's set replaces an image, + // those that brought a trim on anywhere in its replay + // (stripReplayAtLimits). An image replaced for want of a verdict isn't + // the set's strip, and notes nothing. const ceilingSet = ceilingSetLimits(this.configuredImageLimits()); const notes: CeilingKinds = { images: false, bytes: false }; const noteCeilings = (over: Readonly): void => { @@ -17602,10 +17605,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { return true; }, AutobiographicalStrategy.IMAGE_PLACEHOLDER); } - // A render select returns (`returned`) leaves what its ceilings decided - // for select to report once it has returned it; the last such strip is - // the render it returns. Any other strip, such as renderAsNewest's, - // leaves nothing. + // A render select returns (`returned`) leaves which ceiling-set limits + // were over for select to report once it has returned it; the last such + // strip is the render it returns. Any other strip, such as + // renderAsNewest's, leaves nothing. if (opts.returned) this._ceilingNotes = notes; } @@ -17889,7 +17892,8 @@ export class AutobiographicalStrategy implements ResettableStrategy { * replaces, and with imageStripHysteresisRatio, at a trim anywhere in the * rule's replay. The depth limit or a limit the host set may be stripping * too. While the rule is off, the line adds the cost of a window held at - * the ceiling and names the option that spaces it. The rule's set is this + * the ceiling and names the option that spaces it, except under a count + * ceiling of 1, which leaves the option no room. The rule's set is this * class's; a subclass's override of imageStripReplay isn't consulted for * the line. A dry select, a preview, an observer's reconstruction or a * refused select says nothing, and the line waits for the first render @@ -17901,7 +17905,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (!notes[kind] || said[kind]) continue; said[kind] = true; const line = ceilingBoundLine(this.name, kind, this.configuredImageLimits(), this.imageLimitsDefaulted()); - this.diag.warn(this.imageHysteresisRatio() === undefined ? line + AutobiographicalStrategy.CEILING_BATCH_CLAUSE : line); + // The option can space the trims only where the ceiling leaves it room: + // under a count ceiling of 1 the newest image stays at every trim. + const spaces = kind === 'bytes' || this.resolvedImageLimits().ceiling.images > 1; + this.diag.warn(this.imageHysteresisRatio() === undefined && spaces ? line + AutobiographicalStrategy.CEILING_BATCH_CLAUSE : line); } } diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 0ca71b10..8a8883b7 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -610,7 +610,7 @@ export class WindowedPassthroughStrategy implements ContextStrategy { } } - /** Says a ceiling's line once, for a ceiling that decided an image of an entry the render holds. */ + /** Says a ceiling's line once, for a ceiling whose limit was over where the window strips an image of an entry the render holds. */ private reportCeilings(entries: ContextEntry[], policy: ImagePolicy): void { for (const entry of entries) { const decided = policy.ceilings[entry.index]; diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 4c801e6f..f7708c05 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -185,8 +185,9 @@ export interface ContextStrategy { * * What comes back is made of the stored arrays and blocks, as a compile's * messages are, so a caller that edits what comes back copies it first. - * Tool blocks come back as the cap leaves them, since how a compile pairs - * and prunes them depends on the window around them. Rendering doesn't + * Tool blocks come back as the cap leaves them, with what sending prunes + * from them, since how a compile pairs them depends on the window around + * them. Rendering doesn't * change what a later compile sends. */ renderAsNewest?(store: MessageStoreView, ids: readonly MessageId[]): Array; diff --git a/test/request-image-ceilings.test.ts b/test/request-image-ceilings.test.ts index 33497a77..e0715636 100644 --- a/test/request-image-ceilings.test.ts +++ b/test/request-image-ceilings.test.ts @@ -524,6 +524,18 @@ describe('the line when an image ceiling first strips', () => { } }); + it(`${name}: says both lines when both ceilings' limits are over at the same image`, async () => { + // Three 4-byte images under ceilings of 2 images and 8 bytes: the + // oldest is past both, so one strip notes both ceilings. + const { byStep, last } = await ceilingLines( + make({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 8 }), [3], + ); + assert.equal(last, '.II'); + assert.equal(byStep[0]!.length, 2, JSON.stringify(byStep)); + assert.ok(byStep[0]!.some((line) => line.includes('imageCeiling 2 holds')), JSON.stringify(byStep)); + assert.ok(byStep[0]!.some((line) => line.includes('imageByteCeiling 8 bytes holds')), JSON.stringify(byStep)); + }); + it(`${name}: says it for a single image bigger than the byte ceiling`, async () => { const { byStep, patterns } = await ceilingLines(make({ maxLiveImageBytes: 0, imageByteCeiling: 8 }), [{ add: 1, bytes: 12 }]); assert.deepEqual(patterns, ['.'], 'the one image is past the ceiling on its own'); @@ -987,6 +999,40 @@ describe('the line when an image ceiling first strips', () => { } }); + it("the memory strategy's rule says both lines when both ceilings' limits bring the same trim on", async () => { + // Three 4-byte images under ceilings of 2 images and 8 bytes: at the + // third, the count (3) and the bytes (12) are both over, at one trim. + const { lines } = await ceilingLines( + memory({ maxLiveImages: 0, maxLiveImageBytes: 0, imageCeiling: 2, imageByteCeiling: 8, imageStripHysteresisRatio: 0.5 }), [3], + ); + assert.equal(lines.length, 2, JSON.stringify(lines)); + assert.ok(lines.some((line) => line.includes('imageCeiling 2 holds')), JSON.stringify(lines)); + assert.ok(lines.some((line) => line.includes('imageByteCeiling 8 bytes holds')), JSON.stringify(lines)); + }); + + it("the memory strategy's rule keeps a ceiling's trim in its record past a later trim only the host's limit brings on", async () => { + // A count ceiling of 4 and a host byte limit of 40, in force, compiled + // once, at the end. Five 4-byte images bring the count ceiling's trim + // on; a 36-byte image then brings a trim on that only the host's byte + // limit is over at. The replay's record of the first trim stands. + const { kept, lines } = await ruleRun( + { maxLiveImages: 0, imageCeiling: 4, maxLiveImageBytes: 40, imageStripHysteresisRatio: 0.5 }, + [1, 2, 3, 4, 5].map((i) => imageMessage(`img-${i}`)).concat([imageMessage('img-6', 36)]), + ); + assert.equal(kept, '.....I', 'premise: both trims strip'); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 4 holds'), lines[0]); + }); + + it("the memory strategy's line doesn't name imageStripHysteresisRatio under a count ceiling of 1, which leaves it no room", async () => { + // Under a count ceiling of 1 the rule keeps the newest image at every + // trim, so it strips at every image as the limits do. + const { lines } = await ceilingLines(memory({ maxLiveImages: 0, imageCeiling: 1 }), [2]); + assert.equal(lines.length, 1, JSON.stringify(lines)); + assert.ok(lines[0]!.includes('imageCeiling 1 holds maxLiveImages 0 (no limit) to it, so some images are now shown as placeholders'), lines[0]); + assert.ok(!lines[0]!.includes('imageStripHysteresisRatio'), lines[0]); + }); + it("the memory strategy's line isn't read from a preview's set under ceilings that set no limit", async () => { // A preview with maxLiveImages 4 under a count ceiling of 10 limits the // count to 4, as the configured 0 under a ceiling of 4 does, but there From 9a147275e3002be90008321f1d4bccfad62b609f Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Sat, 24 Oct 2026 17:33:20 -0400 Subject: [PATCH 62/67] feat(observer): tell a host's observer of each compile, compression refusal and calibration sample A host couldn't see what its context managers do as it happens: a refused compile reached it only as agent-framework's classification, a refusal's stage only as message text, and the library's own compression refusals and calibration samples only on stderr and in the compression log. A host's telemetry needs these facts; the library tells an observer the host gives it, on its own terms, and learns no telemetry schema. - ContextManagerConfig.observer (ContextObserver, src/observer.ts), with three optional callbacks, handed to the strategy in StrategyContext.observer. isolateObserver reads each callback once, calls it synchronously, and keeps what it throws, or a promise or thenable it returns that rejects, out of the work: what it returns is adopted through Promise.resolve and given its handler through the intrinsic then. The manager logs a callback's first failure once through its namespaced diagnostics and drops the rest. - compile(record), once per compile(): strategy, dryRun, startedAt on the wall clock (Date.now(), so an observer adds no read of the injectable clock), the whole compile's durationMs, outcome ('compiled', 'refused' for an error named in COMPILE_REFUSALS, or 'failed'), the thrown error itself and a compiled compile's renderStats. compile() awaits nothing, so its body moves unchanged into a synchronous compileNow(), which compile() times; with no compile callback it returns compileNow()'s result directly. isCompileRefusal reads a thrown value's name without ever throwing. - OverBudgetError.stageId, from OVER_BUDGET_STAGES. The library's sites pass a new stageId option typed as their ids; the constructor's stage option is exactly 0.21.0's, words that open the message as given. Words alone read 'strategy-defined', a stageId beside them names the stage, neither is 'picker-exhausted', and an id counts only as the words map's own key, as a string. Every call 0.21.0 accepted builds the same message byte for byte. AutobiographicalStrategy's seven sites pass the id and its words through overBudgetStage(id), so an override of the protected overBudgetError hook is given stage as before. - compression(record), for the refusals the library makes itself, by a reason from COMPRESSION_REFUSALS: 'admission-rejected' and 'topology-violation' before any call; 'empty', 'incomplete' and 'malformed' of a response; the split rung's caps, 'chunk-call-cap' and 'window-call-cap'. Each is reported where it's decided: the L1 ladder's record of a rung's answer (a rung reused from saved progress isn't reported again), its admission refusals (decided, and reported, by each ladder run that reaches them), the merge gate and its check for an answer that strips to nothing, the crossed-merge refusal, each split piece's empty or incomplete answer, and a cap's trip, kept in a variable typed as the two cap reasons. ownCompressionRefusal decides which outcomes are the library's own, leaving a provider's refusal or error to membrane, and fails to compile until a new outcome is decided. - calibration(record), once per sample reportRealInputTokens takes: outcome from CALIBRATION_SAMPLE_OUTCOMES, ratio, and the multiplier after. - README section "Observing a manager's work"; changelog.d added fragment. - test/context-observer.test.ts pins each record, each reason at each site and each piece of isolation, and checks under each of the four strategies that a compile's result is the same with no observer, a recording one and one that throws or rejects; test/overbudget-diagnostics.test.ts and test/system-changes.test.ts check each site's stageId beside the message they pin, and the latter reaches passthrough's refusal of its system changes and the newest message, which no test did. Co-Authored-By: Claude Opus 5.5 --- README.md | 4 + changelog.d/context-observer.added.md | 33 + src/adaptive/picker.ts | 107 ++- src/context-manager.ts | 57 ++ src/index.ts | 16 +- src/observer.ts | 216 +++++ src/strategies/autobiographical.ts | 71 +- src/strategies/passthrough.ts | 4 +- src/strategies/windowed-passthrough.ts | 10 +- src/types/strategy.ts | 7 + test/context-observer.test.ts | 1124 ++++++++++++++++++++++++ test/overbudget-diagnostics.test.ts | 22 + test/system-changes.test.ts | 23 +- 13 files changed, 1669 insertions(+), 25 deletions(-) create mode 100644 changelog.d/context-observer.added.md create mode 100644 src/observer.ts create mode 100644 test/context-observer.test.ts diff --git a/README.md b/README.md index 8253b942..61f1e62e 100644 --- a/README.md +++ b/README.md @@ -98,6 +98,10 @@ Every option is documented on `AutobiographicalConfig` in `src/types/strategy.ts Chronicle keeps the store's history, so a conversation can branch. `branchAt(messageId)` creates a branch as of a message, and `switchBranch(name)` moves to it. `fork()` creates a branch at the current head and switches to it. The strategy's state, summaries included, follows the branch. +## Observing a manager's work + +Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. + ## Inspecting and repairing stores Among other tools, `scripts/` holds ones for inspecting and repairing stores. For example, `inspect-store.ts` writes out a store's messages, summary tree and resolutions, `audit-topology.ts` checks that each summary covers a contiguous run of messages, and `repair-topology.ts` plans a repair for one that doesn't, which it applies with `--apply`. Opening a store takes its lock and can write to it, so run them on a stopped agent's store or on a copy. `scripts/README.md` shows how to compile a script and describes several of them. diff --git a/changelog.d/context-observer.added.md b/changelog.d/context-observer.added.md new file mode 100644 index 00000000..12d6715e --- /dev/null +++ b/changelog.d/context-observer.added.md @@ -0,0 +1,33 @@ +- `ContextManagerConfig.observer` (`ContextObserver`): a host is told, as each + happens, of every compile, of each refusal context-manager itself makes of + compression, and of each calibration sample. The manager hands the observer + to its strategy in `StrategyContext.observer`. Each callback is optional and + called synchronously. What it throws or rejects never reaches the work: the + manager logs a callback's first failure once through its diagnostics and + drops the rest, so a compile's result is the same with an observer as + without one. A record's error and render stats are the objects the caller + gets, so an observer treats them as read-only. + - `compile(record)`, once per `compile()`: the strategy, `dryRun`, + `startedAt` on the wall clock (`Date.now()`), the whole compile's + `durationMs` on the monotonic clock, `outcome` (`'compiled'`, `'refused'` + for one of `COMPILE_REFUSALS`, or `'failed'`), the thrown `error` itself, + and a compiled compile's `renderStats`, the object its result holds. + - `compression(record)`: a `reason` from `COMPRESSION_REFUSALS` + (`'admission-rejected'`, `'topology-violation'`, `'empty'`, `'incomplete'`, + `'malformed'`, and the split rung's caps, `'chunk-call-cap'` and + `'window-call-cap'`). A provider's refusal or error on a compression call + isn't one of these. + - `calibration(record)`, once per sample `reportRealInputTokens` takes: an + `outcome` from `CALIBRATION_SAMPLE_OUTCOMES` (`'out-of-band'`, `'held'`, + `'confirmed'`, `'replaced'`, `'stepped'`), the `ratio`, and the + `multiplier` after. +- `OverBudgetError.stageId` names which stage refused, from the exported + `OVER_BUDGET_STAGES`, so a host needn't parse the message. The library's own + strategies give it through the constructor's new `stageId` option, whose + words open the message when no `stage` is given. The constructor's `stage` + is unchanged: a strategy's own words, which open the message exactly as + before. Given words alone, `stageId` reads `'strategy-defined'`, a new id; + given a `stageId` beside them, it reads that id; given neither, it's + `'picker-exhausted'`. Every message reads as before, and + `AutobiographicalStrategy`'s protected `overBudgetError` is still given each + stage's words as `stage`. diff --git a/src/adaptive/picker.ts b/src/adaptive/picker.ts index 43eee8ed..075b7b03 100644 --- a/src/adaptive/picker.ts +++ b/src/adaptive/picker.ts @@ -176,6 +176,83 @@ export interface OverBudgetRawPin { */ export type OverBudgetClearedBy = 'compression' | 'newer-message' | 'smaller-recent-window'; +/** + * Which stage of a compile refused (`OverBudgetError.stageId`), from a closed + * list a host can read without parsing the message: + * - `'picker-exhausted'`: the adaptive picker folded everything it could. + * - `'head-tail-reservation'`: the head and the recent window alone exceed + * the budget. + * - `'emission-overran-plan'`: the emitter spent more than the plan priced + * (planner/emitter estimator drift). + * - `'tail-emission-dropped'`: tail emission would have dropped messages + * reserved for the recent window. + * - `'repair-lost-newest-turn'`: structural repair didn't keep the newest + * turn. + * - `'hierarchical-emission-overran'`: the hierarchical emitter overran the + * budget. + * - `'hierarchical-system-changes'`: the hierarchical emitter's system + * changes alone exceed the budget. + * - `'passthrough-system-changes'`: `PassthroughStrategy`'s system changes + * alone exceed the budget. + * - `'passthrough-changes-and-newest-message'`: its system changes and the + * newest conversation message together exceed the budget. + * - `'windowed-newest-message'`: `WindowedPassthroughStrategy`'s newest + * message alone exceeds the budget. + * - `'windowed-changes-and-newest-message'`: its system changes and the + * newest conversation message together exceed the budget. + * - `'windowed-system-changes'`: its system changes alone exceed the budget. + * - `'strategy-defined'`: a strategy named its stage only in words of its + * own (the constructor's `stage`), which open the message. + */ +export const OVER_BUDGET_STAGES = [ + 'picker-exhausted', + 'head-tail-reservation', + 'emission-overran-plan', + 'tail-emission-dropped', + 'repair-lost-newest-turn', + 'hierarchical-emission-overran', + 'hierarchical-system-changes', + 'passthrough-system-changes', + 'passthrough-changes-and-newest-message', + 'windowed-newest-message', + 'windowed-changes-and-newest-message', + 'windowed-system-changes', + 'strategy-defined', +] as const; + +/** A stage of {@link OVER_BUDGET_STAGES}. */ +export type OverBudgetStage = typeof OVER_BUDGET_STAGES[number]; + +/** A stage context-manager's own strategies refuse at, which has words of its own: what the constructor's `stageId` takes. */ +type LibraryStage = Exclude; + +/** How each of context-manager's own stages opens the refusal's message. Every such stage has words, and nothing else does. */ +const OVER_BUDGET_STAGE_TEXT: Record = { + 'picker-exhausted': 'Adaptive picker exhausted', + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', + 'passthrough-system-changes': 'Passthrough kept only its system changes', + 'passthrough-changes-and-newest-message': 'Passthrough: its system changes and the newest conversation message', + 'windowed-newest-message': 'Windowed passthrough: the newest message alone', + 'windowed-changes-and-newest-message': 'Windowed passthrough: its system changes and the newest conversation message', + 'windowed-system-changes': 'Windowed passthrough: its system changes alone', +}; + +/** + * The options a library site refusing at `stageId` gives: the id, and its + * words as `stage`. AutobiographicalStrategy's sites go through its + * protected `overBudgetError` hook, whose override received the words as + * `stage` in 0.21.0 and still does; the message and the field are as for + * the id alone. + */ +export function overBudgetStage(stageId: LibraryStage): { stageId: LibraryStage; stage: string } { + return { stageId, stage: OVER_BUDGET_STAGE_TEXT[stageId] }; +} + /** The sentence naming the pins that keep raw tokens in a refusal's middle, or nothing when none do. */ function rawPinsText(raw: OverBudgetDiagnostics['rawPins']): string { if (!raw || raw.pins.length === 0) return ''; @@ -229,6 +306,13 @@ export class OverBudgetError extends Error { * {@link OverBudgetClearedBy}). Empty when nothing else is known to clear it. */ readonly clearedBy: readonly OverBudgetClearedBy[]; + /** + * Which stage refused (see {@link OVER_BUDGET_STAGES}): the `stageId` the + * thrower gave, `'strategy-defined'` when it gave only words of its own + * (`stage`), or `'picker-exhausted'` when it gave neither. A `stageId` that + * isn't one of the library's counts as none given. + */ + readonly stageId: OverBudgetStage; constructor(opts: { budget: number; @@ -236,9 +320,15 @@ export class OverBudgetError extends Error { diagnostics: OverBudgetError['diagnostics']; /** What clears the refusal besides a larger budget; empty when left out. */ clearedBy?: readonly OverBudgetClearedBy[]; - /** Which stage refused — defaults to the plan-stage wording. Emission - * sites pass their own so a planner/emitter drift refusal doesn't - * masquerade as picker exhaustion (it cost a debugging session once). */ + /** Which of the library's stages refused — defaults to + * `'picker-exhausted'`. Emission sites pass their own so a + * planner/emitter drift refusal doesn't masquerade as picker exhaustion + * (it cost a debugging session once). Its words open the message unless + * `stage` gives words. */ + stageId?: LibraryStage; + /** A strategy's own words for the stage that refused, which open the + * message exactly as given, as in 0.21.0; `stageId` then reads + * `'strategy-defined'` unless the thrower also gave a `stageId`. */ stage?: string; }) { const countLabels = { @@ -258,8 +348,16 @@ export class OverBudgetError extends Error { const depth = opts.diagnostics.deepestLevelKind === 'unplanned' ? 'fold plan=not computed' : `${depthLabel}=L${opts.diagnostics.deepestLevel}`; + // Words open the message exactly as 0.21.0 built it (`stage ?? default`), + // whatever they say. An id names a stage only as the map's own key, so a + // caller past the type who gives another value gives no id. + const words: unknown = opts.stage; + const named: unknown = opts.stageId; + const id = typeof named === 'string' && Object.hasOwn(OVER_BUDGET_STAGE_TEXT, named) ? named as LibraryStage : undefined; + const stageId: OverBudgetStage = id ?? (words != null ? 'strategy-defined' : 'picker-exhausted'); + const opening = words != null ? `${words}` : OVER_BUDGET_STAGE_TEXT[id ?? 'picker-exhausted']; super( - `${opts.stage ?? 'Adaptive picker exhausted'} but ${opts.actual} tokens still exceed hard budget ${opts.budget}` + + `${opening} but ${opts.actual} tokens still exceed hard budget ${opts.budget}` + ` (head=${opts.diagnostics.headTokens}, tail=${opts.diagnostics.tailTokens},` + ` middle=${opts.diagnostics.middleTokens}${middleCount}, ${depth}` + (opts.diagnostics.deepestAvailableLevel === undefined ? '' : @@ -280,6 +378,7 @@ export class OverBudgetError extends Error { this.actual = opts.actual; this.diagnostics = opts.diagnostics; this.clearedBy = Object.freeze([...(opts.clearedBy ?? [])]); + this.stageId = stageId; } } diff --git a/src/context-manager.ts b/src/context-manager.ts index 5b714399..c5fdbcaa 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -58,6 +58,7 @@ import { markStoreBranchSwitch, observeStoreBranch } from './branch-generation.j import type { StoreBranchGeneration } from './branch-generation.js'; import { randomUUID } from 'node:crypto'; import { attributeEntries, buildRenderedLayout, rawBodiesOf } from './compile-provenance.js'; +import { isCompileRefusal, isolateObserver, type ContextObserver } from './observer.js'; import { FoldJournal, branchRefOf, cachedStoreIdentity, existingStoreIdentity } from './fold-journal.js'; import { describeThrown, logThrown } from './describe-thrown.js'; import { @@ -141,6 +142,16 @@ interface ContextManagerBaseConfig { * inject a fake clock. */ now?: () => number; + /** + * Told of each compile, of each refusal context-manager itself makes of + * compression, and of each calibration sample, as each happens (see + * `ContextObserver`). The manager hands it to its strategy in + * `StrategyContext`. It's called synchronously, and what it throws or + * rejects never reaches the work: the manager logs a callback's first + * failure through its diagnostics and drops the rest. A record's error and + * render stats are the objects the caller gets, so treat them as read-only. + */ + observer?: ContextObserver; /** * Strategy-facing exclusion predicate: return true to keep a message * visible. When set, the view handed to the strategy — for compile, @@ -303,6 +314,8 @@ export class ContextManager { /** heldAt / expiresAt per held id (expiresAt absent = no timeout). */ private compressionHoldInfo = new Map(); private now: () => number = Date.now; + /** The host's observer, isolated (`ContextManagerConfig.observer`), or none. */ + private observer: ContextObserver | undefined; /** Hold options + shared heldAt for the addMessage currently appending. */ private holdingAddOptions: CompressionHoldOptions | undefined; private holdingAddAt = 0; @@ -529,6 +542,14 @@ export class ContextManager { auxiliaryStores, ); if (config.now) manager.now = config.now; + if (config.observer) { + manager.observer = isolateObserver(config.observer, (callback, error) => logThrown( + `[context-manager] the observer's ${callback} callback threw or rejected; the work it observes goes on, ` + + `and its later failures aren't logged:`, + error, + manager.diag, + )); + } try { await manager.initializeStrategy(); @@ -1198,6 +1219,9 @@ export class ContextManager { * model that binds thinking can no longer take, injections placed (see * `SelectOptions.thinkingBinding`). * + * With `ContextManagerConfig.observer`, it tells the observer of the + * compile as it returns or throws (`ContextObserver.compile`). + * * May block if strategy has pending work. */ async compile( @@ -1205,6 +1229,38 @@ export class ContextManager { injections?: ContextInjection[], opts?: SelectOptions ): Promise { + const observer = this.observer; + if (!observer?.compile) return this.compileNow(budget, injections, opts); + // A compile awaits nothing, so it's timed whole on one clock reading + // each side, and the observer hears of it as it returns or throws. The + // wall clock, not the manager's own, so an observer adds no read of now(). + const startedAt = Date.now(); + const start = performance.now(); + const strategy = this.strategy.name; + const dryRun = opts?.dryRun === true; + let result: CompileResult; + try { + result = this.compileNow(budget, injections, opts); + } catch (error) { + observer.compile({ + strategy, dryRun, startedAt, durationMs: performance.now() - start, + outcome: isCompileRefusal(error) ? 'refused' : 'failed', error, + }); + throw error; + } + observer.compile({ + strategy, dryRun, startedAt, durationMs: performance.now() - start, outcome: 'compiled', + ...(result.renderStats ? { renderStats: result.renderStats } : {}), + }); + return result; + } + + /** compile()'s work, which awaits nothing. */ + private compileNow( + budget?: TokenBudget, + injections?: ContextInjection[], + opts?: SelectOptions + ): CompileResult { this.requireInitialized('compile'); // Before anything is selected or committed: a compile refused here leaves // nothing behind. @@ -2319,6 +2375,7 @@ export class ContextManager { messageStore: this.strategyMessageView(), contextLog: this.contextLog.createView(), membrane: this.membrane, + ...(this.observer ? { observer: this.observer } : {}), currentSequence: this.store.currentSequence(), store: this.store, namespace: this.strategyNamespace, diff --git a/src/index.ts b/src/index.ts index f95ad2a8..90d0008d 100644 --- a/src/index.ts +++ b/src/index.ts @@ -38,8 +38,20 @@ export type { ConfigLayer, ConfigResolutionSemantics, EffectiveConfigReport } fr // OverBudget drain breaker on these errors (AF PR #58, framework.ts // classifyInferenceError); exporting them from the root gives consumers a // real `instanceof` instead of stringly-typed `err.name` matching. -export { OverBudgetError, UncoveredDropError } from './adaptive/picker.js'; -export type { OverBudgetClearedBy } from './adaptive/picker.js'; +export { OverBudgetError, UncoveredDropError, OVER_BUDGET_STAGES } from './adaptive/picker.js'; +export type { OverBudgetClearedBy, OverBudgetStage } from './adaptive/picker.js'; + +// What a host observes as it happens (ContextManagerConfig.observer). +export { COMPILE_REFUSALS, COMPRESSION_REFUSALS, CALIBRATION_SAMPLE_OUTCOMES } from './observer.js'; +export type { + ContextObserver, + CompileRecord, + CompressionRefusalRecord, + CalibrationSampleRecord, + CompileRefusal, + CompressionRefusal, + CalibrationSampleOutcome, +} from './observer.js'; export { CompressionWithdrawn } from './compression-withdrawn.js'; export { StoreTopologyError, type TopologyViolation } from './strategies/autobiographical.js'; export { planTopologyRepair, type RepairPlan, type RepairInputs, type RepairOptions } from './repair/topology.js'; diff --git a/src/observer.ts b/src/observer.ts new file mode 100644 index 00000000..387c811b --- /dev/null +++ b/src/observer.ts @@ -0,0 +1,216 @@ +/** + * What a host can observe of a context manager's work as it happens: each + * compile, each refusal context-manager itself makes of compression, and each + * calibration sample. A host passes one `ContextObserver` to + * `ContextManager.open` (`observer`), and the manager hands it to its strategy + * in `StrategyContext`. + * + * The records are facts on context-manager's own terms. What a host keeps or + * sends, and in what shape, is the host's to decide: a record can hold text + * from the store, such as a refusal's message, which names pins. + */ + +import type { RenderStats } from './types/strategy.js'; + +/** + * The thrown errors that are a compile's refusals, by `Error.name`: a + * designed answer that the compile can't fit or can't cover its messages, as + * opposed to a defect. The name survives two copies of context-manager in one + * process, where `instanceof` doesn't. + */ +export const COMPILE_REFUSALS = ['OverBudgetError', 'UncoveredDropError'] as const; + +/** A name of {@link COMPILE_REFUSALS}. */ +export type CompileRefusal = typeof COMPILE_REFUSALS[number]; + +/** + * Context-manager's own refusals of compression, from a closed list. A + * provider's refusal or error on a compression call is membrane's fact, not + * one of these: + * - `'admission-rejected'`: the request didn't fit `compressionContextBudgetTokens`, + * so no call was made. + * - `'topology-violation'`: a merge's sources don't make a group it can + * mint over (fewer than two, not one level below its target, or not + * adjacent in store order), so no call was made and the group went into the + * merge quarantine. + * - `'empty'`: the response held no text to keep, such as text that was a + * literal `` preamble alone. + * - `'incomplete'`: the response stopped other than at `end_turn` and wasn't + * a refusal (a `max_tokens` cut, a tool call, a stop sequence), so its text + * isn't kept. + * - `'malformed'`: the response, or a block in it, isn't the shape a response + * has. + * - `'chunk-call-cap'`: the split rung (`compressionSplitFallback`) reached + * its cap on calls in one attempt at a chunk + * (`compressionSplitMaxCallsPerChunk`) before it could finish, so it ended + * before another call. + * - `'window-call-cap'`: the split rung's attempt alone used every call its + * ten-minute window allows (`compressionSplitMaxCallsPer10Min`), so it ended + * before another call. + * + * Each is reported once per decision. A verdict on an answer the L1 ladder + * keeps in its saved progress isn't decided again when the ladder resumes, + * so it isn't reported again; an admission rejection makes no call, so + * progress keeps none, and each ladder run that reaches it decides and + * reports it anew. In the split rung, each piece's empty or incomplete answer + * is reported as it comes, a cap's trip when the rung ends, and a malformed + * piece ends the rung as `'malformed'`. + */ +export const COMPRESSION_REFUSALS = [ + 'admission-rejected', + 'topology-violation', + 'empty', + 'incomplete', + 'malformed', + 'chunk-call-cap', + 'window-call-cap', +] as const; + +/** A reason of {@link COMPRESSION_REFUSALS}. */ +export type CompressionRefusal = typeof COMPRESSION_REFUSALS[number]; + +/** + * What a calibration sample did, from a closed list: + * - `'out-of-band'`: rejected, since neither its ratio nor the multiplier it + * implies is within the sanity band; the multiplier stays. + * - `'held'`: within the dead band of the multiplier in force, which stays. + * - `'confirmed'`: within the dead band of a provisional multiplier, which + * stays and is no longer provisional. + * - `'replaced'`: a provisional multiplier replaced by what the sample + * implies, within the clamp. + * - `'stepped'`: the multiplier moved a step toward what the sample implies, + * within the clamp. + */ +export const CALIBRATION_SAMPLE_OUTCOMES = ['out-of-band', 'held', 'confirmed', 'replaced', 'stepped'] as const; + +/** An outcome of {@link CALIBRATION_SAMPLE_OUTCOMES}. */ +export type CalibrationSampleOutcome = typeof CALIBRATION_SAMPLE_OUTCOMES[number]; + +/** One `ContextManager.compile()`, refused or failed ones included. */ +export interface CompileRecord { + /** The strategy's `name`. */ + strategy: string; + /** Whether it was a dry compile (`SelectOptions.dryRun`). */ + dryRun: boolean; + /** When it started, on the wall clock (`Date.now()`, ms since epoch). */ + startedAt: number; + /** How long the whole `compile()` took, on the monotonic clock, in ms. */ + durationMs: number; + /** + * `'compiled'`; `'refused'`, when it threw one of {@link COMPILE_REFUSALS}; + * or `'failed'`, when it threw anything else. + */ + outcome: 'compiled' | 'refused' | 'failed'; + /** + * What a refused or failed compile threw, as the caller receives it, such as + * an `OverBudgetError` with its `stageId`, `budget`, `actual`, `clearedBy` and + * `diagnostics`. Treat it as read-only. + */ + error?: unknown; + /** + * A compiled compile's render stats, the object its `CompileResult.renderStats` + * holds (calibration in force, solver, the view's message count), when its + * strategy records them. Treat them as read-only. + */ + renderStats?: RenderStats; +} + +/** One refusal context-manager made of compression (see {@link COMPRESSION_REFUSALS}). */ +export interface CompressionRefusalRecord { + reason: CompressionRefusal; +} + +/** One calibration sample: the provider report a compile's estimate is compared with. */ +export interface CalibrationSampleRecord { + outcome: CalibrationSampleOutcome; + /** The provider's input total over the compile's estimate. */ + ratio: number; + /** The multiplier in force after the sample. */ + multiplier: number; +} + +/** + * Called synchronously as each fact happens, each callback optional. What + * it throws, or a promise it returns that rejects, never reaches the work it + * observes: the manager logs a callback's first failure through its + * diagnostics, and drops the rest. A record's error and render stats are the + * objects the caller gets, so an observer treats them as read-only. + */ +export interface ContextObserver { + compile?(record: CompileRecord): void; + compression?(record: CompressionRefusalRecord): void; + calibration?(record: CalibrationSampleRecord): void; +} + +/** + * `observer` with each callback it has wrapped, so that nothing it throws or + * rejects reaches the caller. `failed` hears of a callback's first throw or + * rejection, once per callback, so a broken observer shows without a line per + * record. Each callback is read once, here, and that function is the one + * called, so replacing or removing it later changes nothing; one that can't + * even be read is treated as absent, and told of the same way. + */ +export function isolateObserver( + observer: ContextObserver, + failed: (callback: keyof ContextObserver, error: unknown) => void = () => {}, +): ContextObserver { + const isolated: ContextObserver = {}; + const told = new Set(); + const fail = (callback: keyof ContextObserver, error: unknown): void => { + if (told.has(callback)) return; + told.add(callback); + try { + failed(callback, error); + } catch { + // Telling of a failure can't end the work either. + } + }; + for (const name of ['compile', 'compression', 'calibration'] as const) { + let callback: unknown; + try { + callback = observer[name]; + } catch (error) { + fail(name, error); + } + if (typeof callback !== 'function') continue; + const method = callback; + isolated[name] = (record: never) => { + try { + const returned: unknown = Reflect.apply(method, observer, [record]); + // Any object or function may be a thenable. Promise.resolve returns a + // native promise as it is, and adopts anything else with resolve + // functions of its own: it reads its `then` at once and calls it in a + // job of its own, so a `then` getter that throws, or a `then` that + // throws before it settles, rejects the promise handled here. The + // handler goes on through the intrinsic `then`, so a promise's own + // `then` can't skip it, and the fulfilment handler returns nothing, + // so nothing returned is read again. (A native promise whose own + // `constructor` getter throws can't be given a handler from + // JavaScript at all: every way reads it first. That throw is caught + // below and told, and if that promise rejects, its rejection is + // unhandled, as the host's `unhandledRejection` handling decides.) + if ((typeof returned === 'object' && returned !== null) || typeof returned === 'function') { + Reflect.apply(Promise.prototype.then, Promise.resolve(returned), [() => undefined, (error: unknown) => fail(name, error)]); + } + } catch (error) { + // An observer can't end a compile, a compression or a sample. + fail(name, error); + } + }; + } + return isolated; +} + +/** + * Whether `error` is one of {@link COMPILE_REFUSALS}. A value whose `name` + * can't be read, such as a revoked proxy, isn't, and reading it never throws, + * so the caller still gets what the compile threw. + */ +export function isCompileRefusal(error: unknown): boolean { + try { + const name = error !== null && typeof error === 'object' ? (error as { name?: unknown }).name : undefined; + return typeof name === 'string' && (COMPILE_REFUSALS as readonly string[]).includes(name); + } catch { + return false; + } +} diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index f564b9cb..e5f6586f 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -55,7 +55,7 @@ import { persistMintRequestPreimage } from '../mint-preimage.js'; import { appendFileSync, mkdirSync } from 'node:fs'; import { dirname } from 'node:path'; import { createHash } from 'node:crypto'; -import { Picker, OverBudgetError, UncoveredDropError, type OverBudgetClearedBy, type OverBudgetDiagnostics, type OverBudgetRawPin, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; +import { Picker, OverBudgetError, UncoveredDropError, overBudgetStage, type OverBudgetClearedBy, type OverBudgetDiagnostics, type OverBudgetRawPin, type PickerChunk, type PickerInputs, type PickerResult } from '../adaptive/picker.js'; import { FlatProfileStrategy } from '../adaptive/strategies/flat-profile.js'; import { KvStableStrategy } from '../adaptive/strategies/kv-stable.js'; import { reportOverlap } from '../adaptive/kv-control.js'; @@ -97,6 +97,7 @@ import { } from '../live-image-limits.js'; import type { MessageId } from '../types/message.js'; import type { CarrierPolicy, IngressChunkResult } from '../types/strategy.js'; +import type { CompressionRefusal } from '../observer.js'; import { type Diagnostics, namespaceDiagnostics } from '../diagnostics.js'; import { holdsSystemChange, isSystemChange, systemChangePositions } from '../system-changes.js'; import { storedJson } from '../stored-json.js'; @@ -1124,6 +1125,26 @@ type CompressionAttemptOutcome = */ | 'incomplete'; +/** + * The refusal an outcome is when context-manager itself made it (see + * `COMPRESSION_REFUSALS`), or nothing. A provider's refusal, and an error it + * returned or a run of them decided, are membrane's to report; a response's + * shape is context-manager's verdict (`assessFallbackCompressionResponse`'s + * `malformed_*`). + */ +function ownCompressionRefusal(outcome: CompressionAttemptOutcome, errorType?: string): CompressionRefusal | undefined { + switch (outcome) { + case 'admission_rejected': return 'admission-rejected'; + case 'topology_violation': return 'topology-violation'; + case 'unusable_empty': return 'empty'; + case 'incomplete': return 'incomplete'; + case 'provider_error': return errorType?.startsWith('malformed_') ? 'malformed' : undefined; + case 'refusal': return undefined; + // A new outcome has to be decided here before this compiles. + default: { const undecided: never = outcome; return undecided; } + } +} + /** * Typed rejection thrown by `executeMerge` when the merge response fails * the terminal-disposition gate. `tick()` catches this specifically to run @@ -2813,6 +2834,12 @@ export class AutobiographicalStrategy implements ResettableStrategy { for (const call of this.callsOut) call.abort(); } + /** Tell the host's observer of a compression outcome that is a refusal context-manager made itself (ownCompressionRefusal). */ + private reportCompressionRefusal(outcome: CompressionAttemptOutcome, errorType?: string): void { + const reason = ownCompressionRefusal(outcome, errorType); + if (reason !== undefined) this.hostContext?.observer?.compression?.({ reason }); + } + /** * `membrane.complete()` for the current generation's work, with a * controller of its own kept in callsOut while the call is out, so that @@ -6737,6 +6764,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { }; this.mergeQuarantine.set(record.key, record); this.persistMergeQuarantine(); + this.reportCompressionRefusal('topology_violation'); this.diag.error( `[merge-topology] ⛔ refused L${targetLevel} merge over ${sourceIds.length} source(s) ` + `(${sourceIds.join(', ')}): ${reason}. The node was NOT minted; entry quarantined ` + @@ -10271,6 +10299,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { familyRecord.push(outcome); endRungRuns(outcome.curveLabel); persistProgress(); + this.reportCompressionRefusal(outcome.outcome, outcome.errorType); }; /** * A failure that is not a rejection says nothing about the request by its @@ -10601,6 +10630,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { admittedTokens: admission.admittedTokens, budgetTokens: admission.budgetTokens, }); + this.reportCompressionRefusal('admission_rejected'); logCompressionCall({ event: 'compression:curve-attempt', operation: 'compress_l1', metadata: trace }, this.ns); return; } @@ -10696,6 +10726,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { admittedTokens, budgetTokens: contextBudget, }); + this.reportCompressionRefusal('admission_rejected'); logCompressionCall({ event: 'compression:curve-attempt', operation: 'compress_l1', metadata: trace, }, this.ns); @@ -10918,6 +10949,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { let modelReportedLeaves = 0; const splitStart = Date.now(); let abortReason: string | undefined; + // Which cap ended the rung, typed as the reasons the host is told, so + // a cap spelled otherwise here fails to compile rather than going + // unreported. + let capTripped: Extract | undefined; const MAX_SPLIT_CALLS = this.config.compressionSplitMaxCallsPerChunk ?? 40; const WINDOW_MS = 10 * 60_000; const MAX_WINDOW_CALLS = this.config.compressionSplitMaxCallsPer10Min ?? 80; @@ -10932,13 +10967,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { if (abortReason) return false; const subset = chunk.messages.slice(a, b + 1); if (attempt) { - if (calls >= MAX_SPLIT_CALLS) { abortReason = 'chunk-call-cap'; return false; } + if (calls >= MAX_SPLIT_CALLS) { abortReason = capTripped = 'chunk-call-cap'; return false; } // This attempt alone has used the whole window: under these caps // the rung can't finish, which is its outcome, like the per-attempt // cap. Otherwise the window is shared with earlier calls (reached // only when the per-attempt cap exceeds the window), and the rest // of the attempt waits for it like a rung's start. - if (calls >= MAX_WINDOW_CALLS) { abortReason = 'window-call-cap'; return false; } + if (calls >= MAX_WINDOW_CALLS) { abortReason = capTripped = 'window-call-cap'; return false; } const windowWait = this.splitWindowWait(1, MAX_WINDOW_CALLS, WINDOW_MS); if (windowWait) throw windowWait; const partTokens = sentTokens.slice(a, b + 1).reduce((n, c) => n + c, 0); @@ -11022,6 +11057,10 @@ export class AutobiographicalStrategy implements ResettableStrategy { } else { trace.outcome = assessment.outcome; if (assessment.outcome === 'provider_error') { abortReason = `provider-error:${assessment.errorType ?? 'unclassified'}`; return false; } + // Each piece's empty or incomplete answer is the library's verdict + // on a call, reported as it's made, whether the rung goes on to + // split the piece or ends here. A refusal reports nothing. + this.reportCompressionRefusal(assessment.outcome); } } // Refused (or empty): split at a lawful boundary, never inside a tool round. @@ -11143,6 +11182,13 @@ export class AutobiographicalStrategy implements ResettableStrategy { ...(abortReason ? { errorType: rejectedAs ?? abortReason } : {}), }; noteOutcome(splitOutcome); + // A cap's trip ends the rung before a call, a refusal the library + // makes itself; its record stays a refusal, which reports nothing. + // Each piece's empty or incomplete answer was reported as it came, + // and a malformed piece is reported through noteOutcome. + if (capTripped !== undefined) { + this.hostContext?.observer?.compression?.({ reason: capTripped }); + } this.diag.error(`[autobiographical] split-stitch abandoned for chunk ${chunk.index}: ${reason} after ${calls} call(s), ${parts.length} piece(s) discarded (receipts only)`); logCompressionCall({ event: 'compression:split-stitch-abandoned', operation: 'compress_l1', metadata: { quarantine_key: quarantineRecord.key, leaf_hash: leafHash, reason, calls, attempted, piecesDiscarded: parts.length } }, this.ns); } @@ -12510,6 +12556,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { model: this.requireCompressionModel(), }, }, this.ns); + this.reportCompressionRefusal(assessment.outcome, errorType); throw new MergeDispositionRejection( targetLevel, assessment.outcome, requestHash, stopReason, errorType, ); @@ -12533,6 +12580,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { `[autobiographical] L${targetLevel} merge generation was all preamble — ` + `rejecting for retry (${sources.length} sources left unmerged)`, ); + this.reportCompressionRefusal('unusable_empty'); throw new MergeDispositionRejection( targetLevel, 'unusable_empty', requestHash, 'end_turn', ); @@ -13242,7 +13290,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } } throw this.overBudgetError(budget, { - stage: 'Head/tail reservation exceeded budget', + ...overBudgetStage('head-tail-reservation'), budget: rejectionBudget, actual: headTokens + tailTokens, clearedBy: headTokens + floorTokens <= rejectionBudget ? ['smaller-recent-window'] : [], @@ -13766,7 +13814,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const middleChunkCountDiag = middleChunkCount; const emissionOverBudget = (attempted: number): OverBudgetError => this.overBudgetError(budget, { - stage: 'Emission overran the plan (planner/emitter estimator drift)', + ...overBudgetStage('emission-overran-plan'), budget: rejectionBudget, actual: attempted + tailTokens, clearedBy: planClearedBy(), @@ -13975,7 +14023,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const tailStats = this.emitRecentNewestFirst(entries, store, messages, effectiveRecentStart, msgCap, rejectionBudget, totalTokens, pse); if (tailStats.messages !== messages.length - effectiveRecentStart) { throw this.overBudgetError(budget, { - stage: 'Tail emission dropped reserved recent-window messages', + ...overBudgetStage('tail-emission-dropped'), budget: rejectionBudget, actual: totalTokens + tailTokens, clearedBy: planClearedBy(), @@ -14036,7 +14084,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { ); if (!newestRetained) { throw this.overBudgetError(budget, { - stage: 'Structural repair did not retain the newest turn', + ...overBudgetStage('repair-lost-newest-turn'), budget: rejectionBudget, actual: totalTokens + tailTokens, // The newest turn is what's missing: the repair trimmed it as an @@ -14194,6 +14242,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { `(est=${Math.round(est / 1000)}k real=${Math.round(realTotal / 1000)}k) — ` + `not a window-shaped request; multiplier stays ${this._calibration.toFixed(2)}`, ); + this.hostContext?.observer?.calibration?.({ outcome: 'out-of-band', ratio, multiplier: this._calibration }); return; } @@ -14225,6 +14274,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { ); this.persistCalibration(); } + this.hostContext?.observer?.calibration?.({ outcome: provisional ? 'confirmed' : 'held', ratio, multiplier: current }); return; } this.diag.error( @@ -14235,6 +14285,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { this._calibration = clamped; this.applyCalibration(); this.persistCalibration(); + this.hostContext?.observer?.calibration?.({ outcome: provisional ? 'replaced' : 'stepped', ratio, multiplier: clamped }); } /** Write the multiplier in force under the current pricing stamp, with the @@ -16092,7 +16143,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { } if (changeTokens > graceLimit) { throw this.overBudgetError(budget, { - stage: 'Hierarchical: its system changes alone exceed the budget', + ...overBudgetStage('hierarchical-system-changes'), budget: graceLimit, actual: changeTokens, clearedBy: [], @@ -16122,7 +16173,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // messages no summary covers. Refuse honestly beyond grace. if (totalTokens + tokens > graceLimit) { throw this.overBudgetError(budget, { - stage: 'Hierarchical emission overran the budget', + ...overBudgetStage('hierarchical-emission-overran'), budget: graceLimit, actual: totalTokens + tokens, // The head is sent whole by definition, and its size is cold @@ -16187,7 +16238,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const prefix = new Set(refused.raw.map((m) => m.id)); for (const s of refused.summaries) this.expandSummaryToLeafMessageIds(s, summariesById, prefix); return this.overBudgetError(budget, { - stage: 'Hierarchical emission overran the budget', + ...overBudgetStage('hierarchical-emission-overran'), budget: graceLimit, actual: refused.actual, clearedBy: this.compressionWorkQueued(store, 'hierarchical', prefix) ? ['compression'] : [], diff --git a/src/strategies/passthrough.ts b/src/strategies/passthrough.ts index 7517dccf..8ad84472 100644 --- a/src/strategies/passthrough.ts +++ b/src/strategies/passthrough.ts @@ -102,7 +102,7 @@ export class PassthroughStrategy implements ContextStrategy { throw new OverBudgetError({ budget: maxTokens, actual: totalTokens, - stage: 'Passthrough kept only its system changes', + stageId: 'passthrough-system-changes', clearedBy: [], diagnostics: { headTokens: totalTokens, tailTokens: 0, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, }); @@ -132,7 +132,7 @@ export class PassthroughStrategy implements ContextStrategy { throw new OverBudgetError({ budget: maxTokens, actual: changeTokens + newestTokens, - stage: 'Passthrough: its system changes and the newest conversation message', + stageId: 'passthrough-changes-and-newest-message', clearedBy: changeTokens < maxTokens ? ['newer-message'] : [], diagnostics: { headTokens: changeTokens, tailTokens: newestTokens, middleTokens: 0, middleChunkCount: 0, deepestLevel: 0 }, }); diff --git a/src/strategies/windowed-passthrough.ts b/src/strategies/windowed-passthrough.ts index 8b741436..75de23a5 100644 --- a/src/strategies/windowed-passthrough.ts +++ b/src/strategies/windowed-passthrough.ts @@ -441,11 +441,13 @@ export class WindowedPassthroughStrategy implements ContextStrategy { throw new OverBudgetError({ budget: maxTokens, actual: total, - stage: changeTokens === 0 - ? 'Windowed passthrough: the newest message alone' + // Which part is over: the newest message alone, or the system + // changes sent beside it, with it or alone. + stageId: changeTokens === 0 + ? 'windowed-newest-message' : total > changeTokens - ? 'Windowed passthrough: its system changes and the newest conversation message' - : 'Windowed passthrough: its system changes alone', + ? 'windowed-changes-and-newest-message' + : 'windowed-system-changes', // No compression here: the window moves past this message once a // newer one that fits arrives (the progress clause above). System // changes go out wherever the window starts, so once they leave no diff --git a/src/types/strategy.ts b/src/types/strategy.ts index d6087884..874c71c2 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -8,6 +8,7 @@ import type { PendingWork, } from './context.js'; import type { OverBudgetClearedBy } from '../adaptive/picker.js'; +import type { ContextObserver } from '../observer.js'; import { DEFAULT_IMAGE_BYTE_CEILING, DEFAULT_IMAGE_CEILING } from '../live-image-limits.js'; /** @@ -114,6 +115,12 @@ export interface StrategyContext { * CompressionWithdrawn). */ membrane?: Membrane; + /** + * The host's observer (`ContextManagerConfig.observer`), already isolated: + * a strategy reports its compression refusals and calibration samples to it + * and needs no guard of its own. Absent when the host gave none. + */ + observer?: ContextObserver; /** Current sequence number */ currentSequence: Sequence; /** diff --git a/test/context-observer.test.ts b/test/context-observer.test.ts new file mode 100644 index 00000000..5cbbd5f6 --- /dev/null +++ b/test/context-observer.test.ts @@ -0,0 +1,1124 @@ +/** + * ContextManagerConfig.observer: what a host is told as it happens. Each + * compile, refused and failed ones included; each refusal context-manager + * itself makes of compression, and none of a provider's; each calibration + * sample. Nothing an observer throws or rejects reaches the work it observes, + * so a compile's result is the same with one, with none, and with one that + * throws or rejects, under each strategy. + */ + +import { after, describe, it } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock, NormalizedRequest, ToolDefinition } from '@animalabs/membrane'; +import { + ContextManager, + PassthroughStrategy, + WindowedPassthroughStrategy, + AutobiographicalStrategy, + KnowledgeStrategy, + OverBudgetError, + UncoveredDropError, + OVER_BUDGET_STAGES, + COMPILE_REFUSALS, + COMPRESSION_REFUSALS, + CALIBRATION_SAMPLE_OUTCOMES, + type ContextObserver, + type CompileRecord, + type CompressionRefusalRecord, + type CalibrationSampleRecord, + type ContextStrategy, +} from '../src/index.js'; +import type { Chunk } from '../src/strategies/autobiographical.js'; +import type { StrategyContext, SummaryEntry } from '../src/types/index.js'; + +const dir = mkdtempSync(join(tmpdir(), 'context-observer-')); +after(() => rmSync(dir, { recursive: true, force: true })); +let stores = 0; +const freshPath = (): string => join(dir, `store-${stores++}`); +const text = (t: string): ContentBlock => ({ type: 'text', text: t }); + +function recorder() { + const compiles: CompileRecord[] = []; + const compressions: CompressionRefusalRecord[] = []; + const samples: CalibrationSampleRecord[] = []; + const observer: ContextObserver = { + compile: (record) => { compiles.push(record); }, + compression: (record) => { compressions.push(record); }, + calibration: (record) => { samples.push(record); }, + }; + return { observer, compiles, compressions, samples }; +} + +/** An observer each of whose callbacks throws, or returns a promise that rejects. */ +function hostile(kind: 'throws' | 'rejects', calls: string[] = []): ContextObserver { + const act = (name: string) => () => { + calls.push(name); + if (kind === 'throws') throw new Error(`observer ${name} threw`); + return Promise.reject(new Error(`observer ${name} rejected`)) as unknown as void; + }; + return { compile: act('compile'), compression: act('compression'), calibration: act('calibration') }; +} + +function strategyContext(manager: ContextManager): StrategyContext { + return (manager as unknown as { createStrategyContext(): StrategyContext }).createStrategyContext(); +} + +/** Rejections no handler took while `run` ran and for a few turns after it. */ +async function unhandledRejections(run: () => Promise): Promise { + const seen: unknown[] = []; + const listener = (reason: unknown) => { seen.push(reason); }; + process.on('unhandledRejection', listener); + try { + await run(); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + } finally { + process.off('unhandledRejection', listener); + } + return seen; +} + +describe('compile records', () => { + it('a compile is recorded once, timed whole, on the wall clock, with its dry flag, and reads the manager clock no more', async () => { + const rec = recorder(); + const reads = { observed: 0, bare: 0 }; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: rec.observer, now: () => { reads.observed++; return 1234; } }); + manager.addMessage('User', [text('hello')]); + manager.addMessage('Claude', [text('hi')]); + const bare = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), now: () => { reads.bare++; return 1234; } }); + bare.addMessage('User', [text('hello')]); + bare.addMessage('Claude', [text('hi')]); + reads.observed = reads.bare = 0; + await bare.compile(); + const before = Date.now(); + const result = await manager.compile(); + const after = Date.now(); + assert.equal(reads.observed, reads.bare, 'a compile with an observer reads the manager clock as often as one without'); + bare.close(); + assert.equal(rec.compiles.length, 1); + const [record] = rec.compiles; + assert.equal(record!.strategy, 'passthrough'); + assert.equal(record!.dryRun, false); + assert.ok(record!.startedAt >= before && record!.startedAt <= after, `${before} <= ${record!.startedAt} <= ${after}`); + assert.equal(record!.outcome, 'compiled'); + assert.ok(Number.isFinite(record!.durationMs) && record!.durationMs >= 0, String(record!.durationMs)); + assert.equal('error' in record!, false); + // Passthrough records no render stats, so none are given. + assert.equal(result.renderStats, undefined); + assert.equal('renderStats' in record!, false); + + await manager.compile(undefined, undefined, { dryRun: true }); + assert.equal(rec.compiles.length, 2); + assert.equal(rec.compiles[1]!.dryRun, true); + manager.close(); + }); + + it('an observer without a compile callback is told nothing of compiles, and the compile goes on', async () => { + const compressions: CompressionRefusalRecord[] = []; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compression: (r) => { compressions.push(r); } } }); + manager.addMessage('User', [text('hello')]); + const result = await manager.compile(); + assert.equal(result.messages.length, 1); + assert.deepEqual(compressions, []); + manager.close(); + }); + + it('the duration covers the whole compile, the strategy\'s select included', async () => { + class Slow extends PassthroughStrategy { + override select(...args: Parameters): ReturnType { + const until = performance.now() + 40; + while (performance.now() < until) { /* a select that takes 40 ms */ } + return super.select(...args); + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Slow(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('the duration covers work the compile does after the strategy\'s select', async () => { + class SlowAfter extends PassthroughStrategy { + takeSelectionCause(): undefined { + const until = performance.now() + 40; + while (performance.now() < until) { /* 40 ms the compile spends after select */ } + return undefined; + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new SlowAfter(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('a compiled record holds the render stats its result holds, the same object', async () => { + const rec = recorder(); + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer: rec.observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const result = await manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + assert.ok(result.renderStats, 'the adaptive select records render stats'); + assert.equal(rec.compiles[0]!.strategy, 'autobiographical'); + assert.equal(rec.compiles[0]!.renderStats, result.renderStats); + manager.close(); + }); + + it('a refused compile is recorded with the error the caller gets, its stage readable', async () => { + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new WindowedPassthroughStrategy(), observer: rec.observer }); + manager.addMessage('User', [text('words '.repeat(2_000))]); + const before = Date.now(); + const thrown = await manager.compile({ maxTokens: 200, reserveForResponse: 20 }).then( + () => assert.fail('the newest message alone is over the budget'), + (error: unknown) => error, + ); + const after = Date.now(); + assert.ok(thrown instanceof OverBudgetError); + assert.equal(thrown.stageId, 'windowed-newest-message'); + assert.equal(rec.compiles.length, 1); + assert.equal(rec.compiles[0]!.outcome, 'refused'); + assert.equal(rec.compiles[0]!.strategy, 'windowed-passthrough'); + assert.equal(rec.compiles[0]!.error, thrown); + assert.equal('renderStats' in rec.compiles[0]!, false); + assert.equal(rec.compiles[0]!.dryRun, false); + assert.ok(rec.compiles[0]!.startedAt >= before && rec.compiles[0]!.startedAt <= after, `${before} <= ${rec.compiles[0]!.startedAt} <= ${after}`); + assert.ok(Number.isFinite(rec.compiles[0]!.durationMs) && rec.compiles[0]!.durationMs >= 0, String(rec.compiles[0]!.durationMs)); + manager.close(); + }); + + it('an override of the protected overBudgetError hook is still given the stage\'s words, as 0.21.0 gave them', async () => { + const given: Array<{ stage?: string; stageId?: string }> = []; + class Decorating extends AutobiographicalStrategy { + protected override overBudgetError(...args: Parameters): OverBudgetError { + given.push({ stage: args[1].stage, stageId: args[1].stageId }); + return super.overBudgetError(args[0], { ...args[1], stage: `[custom] ${args[1].stage ?? 'Adaptive picker exhausted'}` }); + } + } + const strategy = new Decorating({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const thrown = await manager.compile({ maxTokens: 501, reserveForResponse: 500 }).then( + () => assert.fail('one token of room fits nothing'), + (error: unknown) => error, + ); + assert.deepEqual(given, [{ stage: 'Head/tail reservation exceeded budget', stageId: 'head-tail-reservation' }]); + assert.ok(thrown instanceof OverBudgetError); + assert.match(thrown.message, /^\[custom\] Head\/tail reservation exceeded budget but /); + assert.equal(thrown.stageId, 'head-tail-reservation'); + manager.close(); + }); + + it('an autobiographical refusal names its stage, and its message opens as before', async () => { + const rec = recorder(); + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer: rec.observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const thrown = await manager.compile({ maxTokens: 501, reserveForResponse: 500 }).then( + () => assert.fail('one token of room fits nothing'), + (error: unknown) => error, + ); + assert.ok(thrown instanceof OverBudgetError); + assert.equal(thrown.stageId, 'head-tail-reservation'); + assert.match(thrown.message, /^Head\/tail reservation exceeded budget but \d+ tokens still exceed hard budget 1 /); + assert.equal(rec.compiles[0]!.outcome, 'refused'); + assert.equal(rec.compiles[0]!.error, thrown); + manager.close(); + }); + + it('a thrown value whose name can\'t be read is a failed compile, and the caller still gets it', async () => { + const { proxy: thrown, revoke } = Proxy.revocable({}, {}); + revoke(); + class Throwing extends PassthroughStrategy { + override select(): never { throw thrown; } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Throwing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + // Caught by hand: assert.rejects reads what it gets, which a revoked proxy refuses. + let caught: unknown = 'nothing thrown'; + try { await manager.compile(); } catch (error) { caught = error; } + assert.ok(caught === thrown, 'the caller gets the value thrown'); + assert.equal(rec.compiles.length, 1); + assert.equal(rec.compiles[0]!.outcome, 'failed'); + assert.ok(rec.compiles[0]!.error === thrown); + manager.close(); + }); + + it('anything else thrown is a failed compile, null and strings included; a refusal from another copy of the package is still a refusal', async () => { + const defect = new Error('a defect in select'); + const foreignRefusal = Object.assign(new Error('refused by another copy'), { name: 'UncoveredDropError' }); + for (const [thrown, outcome] of [[defect, 'failed'], [foreignRefusal, 'refused'], [null, 'failed'], ['a thrown string', 'failed']] as const) { + class Throwing extends PassthroughStrategy { + override select(): never { throw thrown; } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Throwing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + for (const dryRun of [false, true]) { + let caught: unknown = 'nothing thrown'; + try { await manager.compile(undefined, undefined, { dryRun }); } catch (error) { caught = error; } + assert.ok(caught === thrown, `the caller gets ${String(thrown)}`); + } + assert.equal(rec.compiles.length, 2); + assert.deepEqual(rec.compiles.map((r) => [r.outcome, r.dryRun]), [[outcome, false], [outcome, true]]); + assert.ok(rec.compiles.every((r) => r.error === thrown && 'error' in r)); + manager.close(); + } + }); + + it('an observer\'s methods are called on the observer itself', async () => { + class Recorder implements ContextObserver { + readonly records: CompileRecord[] = []; + compile(record: CompileRecord): void { this.records.push(record); } + } + const observer = new Recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + assert.equal(observer.records.length, 1); + manager.close(); + }); + + it('a callback that throws or rejects is logged once through the manager\'s diagnostics, and one that can\'t be read is treated as absent', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + const throwing = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('throws') }); + throwing.addMessage('User', [text('hello')]); + await throwing.compile(); + await throwing.compile(); + throwing.close(); + const rejecting = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('rejects') }); + rejecting.addMessage('User', [text('hello')]); + await rejecting.compile(); + await rejecting.compile(); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + rejecting.close(); + const compressions: CompressionRefusalRecord[] = []; + const unreadable = { + get compile(): never { throw new Error('a getter that throws'); }, + compression: (r: CompressionRefusalRecord) => { compressions.push(r); }, + }; + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const opened = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: unreadable as never }); + opened.addMessage('User', [text('hello')]); + const result = await opened.compile(); + assert.equal(result.messages.length, 1, 'the manager opened and compiled'); + assert.equal(handed!.compile, undefined); + handed!.compression!({ reason: 'empty' }); + assert.deepEqual(compressions, [{ reason: 'empty' }]); + opened.close(); + } finally { + console.error = original; + } + const logged = lines.filter((line) => line.includes("the observer's compile callback threw or rejected")); + assert.equal(logged.length, 3, `once each for the throwing, the rejecting and the unreadable observer: ${logged.join(' | ')}`); + // Each line carries what failed and the manager's namespace. + assert.ok(logged[0]!.includes(' ns=') && logged[0]!.includes('observer compile threw'), logged[0]); + assert.ok(logged[1]!.includes(' ns=') && logged[1]!.includes('observer compile rejected'), logged[1]); + assert.ok(logged[2]!.includes(' ns=') && logged[2]!.includes('a getter that throws'), logged[2]); + }); + + it('a failure is logged once per callback and once per manager, under that manager\'s namespace', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + const observer = hostile('throws'); + for (const namespace of ['first-probe', 'second-probe']) { + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer, namespace }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + await manager.compile(); + handed!.compression!({ reason: 'empty' }); + handed!.compression!({ reason: 'empty' }); + manager.close(); + } + } finally { + console.error = original; + } + const count = (callback: string, namespace: string) => lines.filter((line) => line.includes(`the observer's ${callback} callback threw or rejected`) && line.includes(` ns=${namespace} `)).length; + for (const namespace of ['first-probe', 'second-probe']) { + assert.equal(count('compile', namespace), 1, `once in ${namespace}`); + assert.equal(count('compression', namespace), 1, `once in ${namespace}, apart from compile`); + } + assert.equal(lines.length, 4, lines.join(' | ')); + }); + + it('a log that throws as it tells of a failure changes nothing either', async () => { + const original = console.error; + console.error = () => { throw new Error('the log broke'); }; + try { + const throwing = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('throws') }); + throwing.addMessage('User', [text('hello')]); + assert.equal((await throwing.compile()).messages.length, 1); + throwing.close(); + const unreadable = { get compile(): never { throw new Error('a getter that throws'); } }; + const opened = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: unreadable as never }); + opened.addMessage('User', [text('hello')]); + assert.equal((await opened.compile()).messages.length, 1); + opened.close(); + } finally { + console.error = original; + } + }); + + it('a returned thenable or promise that rejects, or whose then fails before it settles, is handled and told; a callback is read once', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + const fulfilling = { then(resolve: (value: unknown) => void) { setImmediate(() => resolve('fine')); } }; + const shapes: Record unknown> = { + 'a then that throws': () => ({ then() { throw new Error('then threw'); } }), + 'a then getter that throws': () => ({ get then(): never { throw new Error('then getter threw'); } }), + 'a callable thenable that rejects': () => Object.assign(() => {}, { then(_: unknown, reject: (error: unknown) => void) { reject(new Error('callable rejected')); } }), + 'a native promise with a no-op then of its own': () => Object.assign(Promise.reject(new Error('own-then promise rejected')), { then() {} }), + 'a native promise with a then of its own that isn\'t callable': () => Object.assign(Promise.reject(new Error('uncallable-then promise rejected')), { then: 42 }), + 'a resolved native promise whose constructor getter throws': () => Object.defineProperty(Promise.resolve(), 'constructor', { get(): never { throw new Error('constructor getter threw'); } }), + 'a thenable that fulfils': () => fulfilling, + 'an object whose then getter throws only when read again': () => { let reads = 0; return { get then() { if (reads++ > 0) throw new Error('read again'); return undefined; } }; }, + }; + let stray: unknown[] = []; + try { + stray = await unhandledRejections(async () => { + for (const [shape, returned] of Object.entries(shapes)) { + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compile: () => returned() as void } }); + manager.addMessage('User', [text('hello')]); + assert.equal((await manager.compile()).messages.length, 1, shape); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + manager.close(); + } + }); + } finally { + console.error = original; + } + assert.deepEqual(stray, []); + const told = lines.filter((line) => line.includes("the observer's compile callback threw or rejected")); + const expected = ['then threw', 'then getter threw', 'callable rejected', 'own-then promise rejected', 'uncallable-then promise rejected', 'constructor getter threw']; + assert.equal(told.length, expected.length, `each failing shape told once, the others not: ${told.join(' | ')}`); + expected.forEach((words, i) => assert.ok(told[i]!.includes(words), `${words}: ${told[i]}`)); + + // The callback is the function read at open, read once: replacing or removing it later changes nothing. + const records: CompileRecord[] = []; + let reads = 0; + const record = (r: CompileRecord) => { records.push(r); }; + const observer = { get compile() { reads++; return record; } } as ContextObserver; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer }); + manager.addMessage('User', [text('hello')]); + await manager.compile(); + Object.defineProperty(observer, 'compile', { value: undefined }); + await manager.compile(); + assert.equal(records.length, 2); + assert.equal(reads, 1, 'read once, when the observer was isolated'); + manager.close(); + + // A callback that isn't a function is absent: nothing called, nothing told. + const lines2: string[] = []; + console.error = (...args: unknown[]) => { lines2.push(args.map(String).join(' ')); }; + try { + const odd = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compile: 'yes' } as never }); + odd.addMessage('User', [text('hello')]); + assert.equal((await odd.compile()).messages.length, 1); + odd.close(); + } finally { + console.error = original; + } + assert.deepEqual(lines2, []); + }); +}); + +describe('what an observer throws or rejects never reaches the work', () => { + const STRATEGIES: Record ContextStrategy> = { + passthrough: () => new PassthroughStrategy(), + windowed: () => new WindowedPassthroughStrategy(), + autobiographical: () => new AutobiographicalStrategy({ + targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, hierarchical: true, adaptiveResolution: true, + l1HoldbackChunks: 0, minChunkCharsForLLM: 0, autoTickOnNewMessage: false, compressionModel: 'test-model', + }), + knowledge: () => new KnowledgeStrategy({ + targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 400, hierarchical: true, + l1HoldbackChunks: 0, minChunkCharsForLLM: 0, autoTickOnNewMessage: false, compressionModel: 'test-model', + } as never), + }; + + /** `value` as JSON without the fields that differ between any two compiles: timings and ids minted per compile. */ + function withoutVolatile(value: unknown): unknown { + return JSON.parse(JSON.stringify(value ?? null, (key, v) => (['compileMs', 'solveMs', 'compileId'].includes(key) ? undefined : v))); + } + + /** One store's compiled request, and what the compile left, after compression has had its turn. */ + async function compiled(strategy: ContextStrategy, observer?: ContextObserver) { + let summaries = 0; + const membrane = { + complete: async () => ({ stopReason: 'end_turn', content: [text(`Summary ${summaries++} of the chunk`)], usage: { inputTokens: 100, outputTokens: 20 } }), + }; + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, ...(observer ? { observer } : {}) }); + for (let i = 0; i < 24; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'substantive words '.repeat(20))]); + const tick = (strategy as { tick?: (ctx: StrategyContext) => Promise }).tick; + for (let i = 0; i < 12 && tick; i++) await tick.call(strategy, strategyContext(manager)); + const budget = { maxTokens: 3_000, reserveForResponse: 200 }; + const injections = [ + { namespace: 'probe', position: 'afterUser' as const, content: [text('injected')] }, + { namespace: 'probe', position: 'system' as const, content: [text('a system note')] }, + ]; + const calibration = strategy as unknown as { _calibration?: number; _calibrationArmed?: boolean; _lastCompileEstimate?: number }; + const outcome = await manager.compile(budget, injections).then( + (result) => ({ + messages: result.messages, + systemInjections: result.systemInjections, + renderStats: withoutVolatile(result.renderStats), + provenance: withoutVolatile({ messages: result.provenance?.messages, layout: result.provenance?.layout, strategy: result.provenance?.strategy }), + calibration: [calibration._calibration, calibration._calibrationArmed, calibration._lastCompileEstimate], + }), + (error: unknown) => ({ error: String(error) }), + ); + manager.close(); + return { outcome, summaries }; + } + + for (const [name, make] of Object.entries(STRATEGIES)) { + it(`under ${name}, a compile's result is the same with an observer, without one, and with one that throws or rejects`, async () => { + const bare = await compiled(make()); + const rec = recorder(); + const observed = await compiled(make(), rec.observer); + const calls: string[] = []; + let thrown: Awaited> | undefined; + let rejected: Awaited> | undefined; + const stray = await unhandledRejections(async () => { + thrown = await compiled(make(), hostile('throws', calls)); + rejected = await compiled(make(), hostile('rejects', calls)); + }); + assert.ok(!('error' in bare.outcome), `the fixture compiles under ${name}: ${JSON.stringify(bare.outcome)}`); + assert.ok(bare.outcome.systemInjections.length > 0, 'a system injection to compare'); + assert.deepEqual(observed.outcome, bare.outcome); + assert.deepEqual(thrown!.outcome, bare.outcome); + assert.deepEqual(rejected!.outcome, bare.outcome); + assert.deepEqual(stray, [], 'an observer\'s rejection is no unhandled rejection'); + // The observers were told, so the comparison is with observed compiles. + assert.equal(rec.compiles.length, 1); + assert.ok(calls.filter((call) => call === 'compile').length === 2, calls.join(',')); + if (name === 'autobiographical' || name === 'knowledge') { + assert.ok(bare.summaries > 0, 'compression ran, so the compile folds summaries'); + } + }); + } + + it('a fork initializes the strategy again with the same observer', async () => { + const handed: Array = []; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed.push(ctx.observer); } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + await manager.fork('probe-branch'); + assert.ok(handed.length >= 2, `initialized ${handed.length} time(s)`); + assert.ok(handed.every((observer) => observer !== undefined && observer === handed[0]), 'every initialization gets the one isolated observer'); + manager.close(); + }); + + it('a strategy is handed the isolated observer: what the host\'s throws never reaches the strategy', async () => { + let handed: StrategyContext['observer']; + class Capturing extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { handed = ctx.observer; } + } + const calls: string[] = []; + const manager = await ContextManager.open({ path: freshPath(), strategy: new Capturing(), observer: hostile('throws', calls) }); + assert.ok(handed?.compression, 'the strategy has the observer'); + handed!.compression!({ reason: 'empty' }); + handed!.calibration!({ outcome: 'held', ratio: 1, multiplier: 1 }); + assert.deepEqual(calls, ['compression', 'calibration'], 'the host\'s callbacks ran'); + manager.close(); + + let none: StrategyContext['observer'] | 'unset' = 'unset'; + class Bare extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { none = ctx.observer; } + } + const without = await ContextManager.open({ path: freshPath(), strategy: new Bare() }); + assert.equal(none, undefined, 'no observer, none handed'); + without.close(); + + let partial: StrategyContext['observer']; + class Partial extends PassthroughStrategy { + async initialize(ctx: StrategyContext): Promise { partial = ctx.observer; } + } + const onlyCompression = await ContextManager.open({ path: freshPath(), strategy: new Partial(), observer: { compression: () => {} } }); + assert.deepEqual(Object.keys(partial!), ['compression'], 'only the callbacks the host gave'); + onlyCompression.close(); + }); +}); + +describe('the exported lists', () => { + const diagnostics = { headTokens: 1, tailTokens: 2, middleTokens: 3, middleChunkCount: 1, deepestLevel: 1 }; + + it('words given as stage build the message 0.21.0 built from them, byte for byte, and read as strategy-defined', () => { + // 0.21.0 opened the message with `${opts.stage ?? 'Adaptive picker exhausted'}` + // and left the rest as it is; the rest is pinned by the message tests. + const rest = new OverBudgetError({ budget: 1, actual: 2, diagnostics }).message.slice('Adaptive picker exhausted'.length); + const released = (stage: unknown) => `${stage ?? 'Adaptive picker exhausted'}${rest}`; + const words = [ + // The library's own ids and words, given as a strategy's words. + ...OVER_BUDGET_STAGES, + 'Head/tail reservation exceeded budget', + 'Adaptive picker exhausted', + 'My strategy: the newest turn alone', + // Words that name a property every object has, and none. + 'constructor', 'toString', '__proto__', 'hasOwnProperty', '', + ]; + for (const stage of words) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stage }); + assert.equal(error.message, released(stage), stage); + assert.equal(error.stageId, 'strategy-defined', stage); + } + // Values a caller past the type passes build what 0.21.0 built from them. + for (const stage of [null, undefined, 42]) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stage } as never); + assert.equal(error.message, released(stage), String(stage)); + assert.equal(error.stageId, stage == null ? 'picker-exhausted' : 'strategy-defined', String(stage)); + } + }); + + it('a stageId names the library\'s stage, and words given beside it still open the message', () => { + const both = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: 'head-tail-reservation', stage: 'My words' }); + assert.equal(both.stageId, 'head-tail-reservation'); + assert.match(both.message, /^My words but 2 tokens still exceed hard budget 1 /); + // An id past the type that isn't one of the library's own keys, as a string, is no id. + for (const stageId of ['strategy-defined', 'constructor', '__proto__', 'no-such-stage', { toString: () => 'head-tail-reservation' }, ['head-tail-reservation']]) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId } as never); + assert.equal(error.stageId, 'picker-exhausted', String(stageId)); + assert.match(error.message, /^Adaptive picker exhausted but /, String(stageId)); + } + }); + + it('each of the library\'s stages opens its message with the words `main` gave it', () => { + const released: Record = { + 'picker-exhausted': 'Adaptive picker exhausted', + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', + 'passthrough-system-changes': 'Passthrough kept only its system changes', + 'passthrough-changes-and-newest-message': 'Passthrough: its system changes and the newest conversation message', + 'windowed-newest-message': 'Windowed passthrough: the newest message alone', + 'windowed-changes-and-newest-message': 'Windowed passthrough: its system changes and the newest conversation message', + 'windowed-system-changes': 'Windowed passthrough: its system changes alone', + }; + assert.deepEqual(OVER_BUDGET_STAGES.filter((s) => s !== 'strategy-defined').sort(), Object.keys(released).sort()); + for (const [stageId, words] of Object.entries(released)) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: stageId as never }); + assert.ok(error.message.startsWith(`${words} but 2 tokens still exceed hard budget 1 (`), error.message); + } + }); + + it('each stage opens its refusal\'s message with words of its own, and a refusal that names none is the picker\'s', () => { + const openings = new Set(); + const library = OVER_BUDGET_STAGES.filter((s) => s !== 'strategy-defined'); + for (const stageId of library) { + const error = new OverBudgetError({ budget: 1, actual: 2, diagnostics, stageId: stageId as never }); + assert.equal(error.stageId, stageId); + const opening = error.message.slice(0, error.message.indexOf(' but 2 tokens still exceed hard budget 1')); + assert.ok(opening.length > 0, error.message); + openings.add(opening); + } + assert.equal(openings.size, library.length, [...openings].join(' | ')); + const unnamed = new OverBudgetError({ budget: 1, actual: 2, diagnostics }); + assert.equal(unnamed.stageId, 'picker-exhausted'); + assert.match(unnamed.message, /^Adaptive picker exhausted but 2 tokens still exceed hard budget 1 /); + }); + + it('the compile refusals are the two refusal errors by name, and nothing else', () => { + const names = [ + new OverBudgetError({ budget: 1, actual: 2, diagnostics }).name, + new UncoveredDropError({ droppedIds: [], site: 'test', diagnostics: { budget: 1, totalTokens: 2 } }).name, + ]; + assert.deepEqual([...COMPILE_REFUSALS].sort(), names.sort()); + }); + + it('the compression reasons and sample outcomes are as documented', () => { + assert.deepEqual([...COMPRESSION_REFUSALS], ['admission-rejected', 'topology-violation', 'empty', 'incomplete', 'malformed', 'chunk-call-cap', 'window-call-cap']); + assert.deepEqual([...CALIBRATION_SAMPLE_OUTCOMES], ['out-of-band', 'held', 'confirmed', 'replaced', 'stepped']); + }); +}); + +// ---- Compression: an L1 chunk's ladder ---- + +class LadderProbe extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { this.pushSummary(entry); } + run(chunk: Chunk, ctx: StrategyContext): Promise { return this.compressChunkHierarchical(chunk, ctx); } +} + +const DIARY = 'I weighed the thread and decided to stay quiet. '.repeat(12); +const toolUse = (id: string, name: string, input: Record): ContentBlock => ({ type: 'tool_use', id, name, input } as ContentBlock); +const toolResult = (id: string, content: string): ContentBlock => ({ type: 'tool_result', toolUseId: id, content } as ContentBlock); +const tool = (name: string): ToolDefinition => ({ name, description: 'd', inputSchema: { type: 'object', properties: {} } } as never); + +type Answer = Record | { error: Error }; +const REFUSAL = (inputTokens = 100): Answer => ({ content: [], stopReason: 'refusal', usage: { inputTokens, outputTokens: 0 }, raw: { response: { stop_details: { category: 'reasoning_extraction' } } } }); +const ANSWER = (t: string, stopReason = 'end_turn'): Answer => ({ content: t ? [text(t)] : [], stopReason, usage: { inputTokens: 80, outputTokens: 20 } }); + +function scripted(answers: Answer[]) { + const calls: NormalizedRequest[] = []; + return { + calls, + membrane: { + complete: async (request: NormalizedRequest) => { + const answer = answers[Math.min(calls.length, answers.length - 1)]!; + calls.push(structuredClone(request)); + if ('error' in answer && answer.error instanceof Error) throw answer.error; + return answer; + }, + } as never, + }; +} + +/** A chunk whose last turn holds a long private-reasoning tool argument, after one seeded L1. */ +async function ladder(membrane: unknown, options: Record, path = freshPath(), from = 10) { + const rec = recorder(); + const strategy = new LadderProbe({ + compressionModel: 'same-model', targetChunkTokens: 100, recentWindowTokens: 0, headWindowTokens: 100_000, + autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, compressionRefusalCurveFallbacks: 0, + ...options, + } as never); + const manager = await ContextManager.open({ path, strategy, membrane: membrane as never, observer: rec.observer }); + manager.setToolDefinitions(['skip_reply', 'journal'].map(tool)); + const ids: string[] = []; + for (let i = 0; i < 10; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substantive '.repeat(12))])); + ids.push(manager.addMessage('User', [text('raw-10 ambient chatter ' + 'substantive '.repeat(12))])); + ids.push(manager.addMessage('Claude', [toolUse('skip1', 'skip_reply', { reason: DIARY, wake_in_seconds: 1500 })])); + ids.push(manager.addMessage('User', [toolResult('skip1', '{"skipped":true}')])); + strategy.seed({ id: 'L1-100', level: 1, content: 'authored L1-100', tokens: 20, sourceLevel: 0, sourceIds: [ids[0]!, ids[1]!], sourceRange: { first: ids[0]!, last: ids[1]! }, created: 100 }); + const want = new Set(ids.slice(from)); + const target = (): Chunk => ({ index: 999, startIndex: from, endIndex: 13, messages: strategyContext(manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }); + return { manager, strategy, target, rec }; +} + +const HOIST = { compressionToolProseFallback: { intoTool: 'journal', fromTools: ['skip_reply'] } }; + +describe('compression refusals: an L1 ladder', () => { + it('a provider\'s refusal is membrane\'s to report, not one of these', async () => { + const { calls, membrane } = scripted([REFUSAL()]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a provider\'s rejection of the request is membrane\'s to report too', async () => { + const rejected = Object.assign(new Error('invalid request'), { type: 'invalid_request', httpStatus: 400, retryable: false }); + const { calls, membrane } = scripted([{ error: rejected }]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a canonical answer cut short is incomplete', async () => { + const { calls, membrane } = scripted([ANSWER('a summary cut', 'max_tokens')]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }]); + fx.manager.close(); + }); + + it('a canonical answer with no text is empty', async () => { + const { membrane } = scripted([ANSWER('')]); + const fx = await ladder(membrane, {}); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + // The canonical answer, then the plain-prose retry's, each empty. + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }, { reason: 'empty' }]); + fx.manager.close(); + }); + + it('a later rung\'s answer that isn\'t a response\'s shape is malformed', async () => { + const { calls, membrane } = scripted([REFUSAL(), { stopReason: 'end_turn', content: 'not a list of blocks', usage: { inputTokens: 80, outputTokens: 20 } }]); + const fx = await ladder(membrane, HOIST); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 2, 'canonical, then the hoisted retry'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'malformed' }]); + fx.manager.close(); + }); + + it('a hoisted retry over the compression budget is refused before any call', async () => { + const { calls, membrane } = scripted([REFUSAL(90_000), ANSWER('must never be called')]); + const fx = await ladder(membrane, { ...HOIST, compressionContextBudgetTokens: 50_000 }); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 1, 'the hoisted retry is never sent'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }]); + fx.manager.close(); + }); + + describe('the split rung, by what ended it', () => { + const SPLIT = { compressionSourceOnlyFallback: true, compressionSplitFallback: true }; + // The canonical request and source-only refuse, so the split rung runs: its + // first piece is the chunk's first message, its second the tool round, + // which it can't split further. + const cases: Array<[string, Record, Answer[], CompressionRefusalRecord[]]> = [ + ['its cap on calls in one attempt', { compressionSplitMaxCallsPerChunk: 1 }, [ANSWER('the first piece summarized ' + 'detail '.repeat(20))], [{ reason: 'chunk-call-cap' }]], + ['its window\'s calls used by the attempt alone', { compressionSplitMaxCallsPer10Min: 1 }, [ANSWER('the first piece summarized ' + 'detail '.repeat(20))], [{ reason: 'window-call-cap' }]], + ['a piece it can\'t split that answers with no text', {}, [ANSWER('')], [{ reason: 'empty' }]], + ['a piece it can\'t split that answers cut short', {}, [ANSWER('a piece summary cut', 'max_tokens')], [{ reason: 'incomplete' }]], + ['a piece it can\'t split that the provider refuses', {}, [REFUSAL()], []], + ['a piece whose answer is malformed', {}, [{ stopReason: 'end_turn', content: 'not a list', usage: { inputTokens: 80, outputTokens: 20 } }], [{ reason: 'malformed' }]], + ]; + it('a piece that answers empty and is split further is reported once, and its halves fold', async () => { + // Five messages: raw-8, raw-9, raw-10, then the tool round. The root splits + // after raw-10, so the first piece is the three plain messages, which + // answer empty and split into raw-8..9 and raw-10. + const pieces = ['raw-8 and raw-9', 'raw-10', 'the tool round'].map((p) => ANSWER(`${p} summarized ` + 'detail '.repeat(20))); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), ...pieces]); + const fx = await ladder(membrane, SPLIT, freshPath(), 8); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 6, 'canonical, source-only, the empty piece, its two halves, and the tool round'); + assert.equal(chunk.compressed, true, 'the stitch landed'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + + it('a split interrupted and run again reports its piece once per call', async () => { + const transient = new Error('a socket hang up the split resumes after'); + const tool = ANSWER('the tool round summarized ' + 'detail '.repeat(20)); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), { error: transient }, ANSWER(''), tool]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitPlaceholder: true }); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.equal(calls.length, 4); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 6, 'only the split rung runs again, asking both pieces'); + assert.equal(chunk.compressed, true); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }, { reason: 'empty' }], 'the first piece answered empty on each call'); + fx.manager.close(); + }); + + it('a split its cap ended isn\'t reported again on a later run, which makes no call', async () => { + const path = freshPath(); + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER('the first piece summarized ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitMaxCallsPerChunk: 1 }, path); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3); + assert.deepEqual(fx.rec.compressions, [{ reason: 'chunk-call-cap' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'the chunk is quarantined, so nothing is asked again'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'chunk-call-cap' }]); + fx.manager.close(); + }); + + it('an empty piece is reported even when the stitch succeeds', async () => { + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), ANSWER('the tool round summarized ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...SPLIT, compressionSplitPlaceholder: true }); + const chunk = fx.target(); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 4, 'canonical, source-only, and both pieces'); + assert.equal(chunk.compressed, true, 'the stitch landed'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + for (const [name, options, pieces, expected] of cases) { + it(`${name}: ${expected.length ? expected[0]!.reason : 'nothing reported'}`, async () => { + const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ...pieces]); + const fx = await ladder(membrane, { ...SPLIT, ...options }); + await fx.strategy.run(fx.target(), strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'canonical, source-only, and one piece'); + assert.deepEqual(fx.rec.compressions, expected); + fx.manager.close(); + }); + } + }); + + it('a verdict is reported once: a resumed ladder reuses it without a second report', async () => { + const transient = new Error('a socket hang up the ladder resumes after'); + const path = freshPath(); + const first = scripted([ANSWER('a summary cut', 'max_tokens'), { error: transient }, ANSWER('the source-only summary ' + 'detail '.repeat(20))]); + const fx = await ladder(first.membrane, { compressionSourceOnlyFallback: true }, path); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.equal(first.calls.length, 2, 'canonical, then source-only, which fails'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(first.calls.length, 3, 'the resumed ladder sends only source-only again, so it reached the reused canonical verdict'); + assert.equal(chunk.compressed, true, 'and the resumed ladder compressed the chunk'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'incomplete' }], 'the canonical verdict is reused, not reported again'); + fx.manager.close(); + }); + + it('an admission rejection is decided again by a resumed ladder, and reported again', async () => { + const transient = new Error('a socket hang up the ladder resumes after'); + const { calls, membrane } = scripted([REFUSAL(90_000), { error: transient }, ANSWER('the source-only summary ' + 'detail '.repeat(20))]); + const fx = await ladder(membrane, { ...HOIST, compressionContextBudgetTokens: 50_000, compressionSourceOnlyFallback: true }); + const chunk = fx.target(); + await assert.rejects(fx.strategy.run(chunk, strategyContext(fx.manager)), (error: unknown) => error === transient); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }]); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 3, 'canonical and source-only, then source-only again; the hoisted retry is never sent'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'admission-rejected' }, { reason: 'admission-rejected' }]); + fx.manager.close(); + }); +}); + +// ---- Compression: the recall variants' plan ---- + +class RecallProbe extends LadderProbe { + linkChild(child: SummaryEntry, parentId: string): void { this.setMergedInto(child, parentId); } +} + +const summaryOf = (id: string, level: number, sourceIds: string[], first: string, last: string): SummaryEntry => ({ + id, level, content: `authored ${id}`, tokens: 20, sourceLevel: level - 1, sourceIds, sourceRange: { first, last }, + created: Number(id.replace(/\D/g, '')) || 1, +}); + +describe('compression refusals: the recall variants', () => { + it('each variant over the compression budget is refused before any call', async () => { + const rec = recorder(); + const { calls, membrane } = scripted([REFUSAL(90_000), ANSWER('must never be called')]); + const strategy = new RecallProbe({ + compressionModel: 'same-model', targetChunkTokens: 100, recentWindowTokens: 0, headWindowTokens: 0, + autoTickOnNewMessage: false, minChunkCharsForLLM: 0, mergeThreshold: 99, + compressionRefusalCurveFallbacks: 3, compressionContextBudgetTokens: 50_000, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, observer: rec.observer }); + const ids: string[] = []; + for (let i = 0; i < 14; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substantive '.repeat(18))])); + const children = [0, 2, 4, 6].map((a, k) => summaryOf(`L1-10${k}`, 1, [ids[a]!, ids[a + 1]!], ids[a]!, ids[a + 1]!)); + for (const child of children) strategy.seed(child); + const older = summaryOf('L2-200', 2, [children[1]!.id, children[0]!.id], ids[0]!, ids[3]!); + const newer = summaryOf('L2-201', 2, [children[3]!.id, children[2]!.id], ids[4]!, ids[7]!); + strategy.seed(older); + strategy.seed(newer); + for (const child of children.slice(0, 2)) strategy.linkChild(child, older.id); + for (const child of children.slice(2)) strategy.linkChild(child, newer.id); + const want = new Set(ids.slice(10, 12)); + const target: Chunk = { index: 999, startIndex: 10, endIndex: 12, messages: strategyContext(manager).messageStore.getAll().filter((m) => want.has(m.id)), tokens: 100, compressed: false }; + await strategy.run(target, strategyContext(manager)); + assert.equal(calls.length, 1, 'no variant is sent'); + // One for each variant the plan holds: each of the two parents expanded. + assert.deepEqual(rec.compressions, [{ reason: 'admission-rejected' }, { reason: 'admission-rejected' }]); + manager.close(); + }); +}); + +// ---- Compression: merges ---- + +class MergeProbe extends AutobiographicalStrategy { + seed(entry: SummaryEntry): void { this.pushSummary(entry); } + qMerge(level: number, sourceIds: string[]): void { this.enqueueMerge({ level, sourceIds }); } + summariesView(): SummaryEntry[] { return [...this.summaries]; } +} + +/** Two seeded L1s over real messages, and a merge of them queued at `level`. */ +async function merge(membrane: unknown, level = 2, observe: ContextObserver | 'none' | 'record' = 'record') { + const rec = recorder(); + const observer = observe === 'record' ? rec.observer : observe === 'none' ? undefined : observe; + const strategy = new MergeProbe({ + compressionModel: 'same-model', hierarchical: true, targetChunkTokens: 100_000, recentWindowTokens: 0, headWindowTokens: 0, + autoTickOnNewMessage: false, mergeThreshold: 99, quarantineAlarmIntervalMs: 0, mergeAttemptLimit: 5, + } as never); + const manager = await ContextManager.open({ path: freshPath(), strategy, membrane: membrane as never, observer }); + const ids: string[] = []; + for (let i = 0; i < 8; i++) ids.push(manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`raw-${i} ` + 'substance '.repeat(10))])); + strategy.seed(summaryOf('L1-100', 1, [ids[0]!, ids[1]!], ids[0]!, ids[1]!)); + strategy.seed(summaryOf('L1-101', 1, [ids[2]!, ids[3]!], ids[2]!, ids[3]!)); + strategy.qMerge(level, ['L1-100', 'L1-101']); + return { manager, strategy, rec }; +} + +describe('compression refusals: merges', () => { + const cases: Array<[string, Answer, CompressionRefusalRecord[]]> = [ + ['a refusal, membrane\'s', REFUSAL(), []], + ['a provider error, membrane\'s', { error: Object.assign(new Error('bad request'), { type: 'invalid_request', httpStatus: 400, retryable: false }) }, []], + ['an answer cut short', ANSWER('a partial consolidation', 'max_tokens'), [{ reason: 'incomplete' }]], + ['an answer with no text', ANSWER(''), [{ reason: 'empty' }]], + ['an answer of no response\'s shape', { stopReason: 'end_turn', content: 'not a list', usage: { inputTokens: 80, outputTokens: 20 } }, [{ reason: 'malformed' }]], + ]; + for (const [name, answer, expected] of cases) { + it(`${name}: ${expected.length ? expected[0]!.reason : 'nothing reported'}`, async () => { + const { calls, membrane } = scripted([answer]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)).catch(() => {}); + assert.equal(calls.length, 1, 'the merge was asked once'); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 0, 'nothing minted'); + assert.deepEqual(fx.rec.compressions, expected); + fx.manager.close(); + }); + } + + it('a merge answer that is a thinking preamble alone is empty', async () => { + const { calls, membrane } = scripted([ANSWER('only my own scratch, nothing to keep')]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)).catch(() => {}); + assert.equal(calls.length, 1); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 0, 'nothing minted'); + assert.deepEqual(fx.rec.compressions, [{ reason: 'empty' }]); + fx.manager.close(); + }); + + it('an observer that throws or rejects at the merge gate changes nothing the merge does', async () => { + const outcomes: unknown[] = []; + const told: string[][] = [[], []]; + const stray = await unhandledRejections(async () => { + for (const observe of ['none', hostile('throws', told[0]), hostile('rejects', told[1])] as const) { + const { calls, membrane } = scripted([ANSWER('a partial consolidation', 'max_tokens')]); + const fx = await merge(membrane, 2, observe); + await fx.strategy.tick(strategyContext(fx.manager)); + outcomes.push([calls.length, fx.strategy.summariesView().filter((s) => s.level === 2).length, fx.manager.getStore().getStateJson('default/autobio:mergeQueue')]); + fx.manager.close(); + } + }); + assert.equal((outcomes[0] as unknown[])[0], 1, 'the merge made its call'); + assert.deepEqual(outcomes[1], outcomes[0]); + assert.deepEqual(outcomes[2], outcomes[0]); + for (const calls of told) assert.ok(calls.includes('compression'), `the gate's refusal reached the hostile observer: ${calls.join(', ')}`); + assert.deepEqual(stray, []); + }); + + it('an accepted merge reports nothing', async () => { + const { membrane } = scripted([ANSWER('A real consolidation.')]); + const fx = await merge(membrane); + await fx.strategy.tick(strategyContext(fx.manager)); + assert.equal(fx.strategy.summariesView().filter((s) => s.level === 2).length, 1); + assert.deepEqual(fx.rec.compressions, []); + fx.manager.close(); + }); + + it('a merge over sources of the wrong level is refused before any call', async () => { + const { calls, membrane } = scripted([ANSWER('must never be called')]); + const fx = await merge(membrane, 3); + await fx.strategy.tick(strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.deepEqual(fx.rec.compressions, [{ reason: 'topology-violation' }]); + fx.manager.close(); + }); +}); + +// ---- Calibration samples ---- + +type CalibrationInternals = { + _calibration: number; + _calibrationArmed: boolean; + _lastCompileEstimate: number; + _calibrationProvisional: { fromEpoch: number | null } | null; +}; + +async function calibrated(observe: ContextObserver | 'none' | 'record' = 'record') { + const rec = recorder(); + const observer = observe === 'record' ? rec.observer : observe === 'none' ? undefined : observe; + const strategy = new AutobiographicalStrategy({ targetChunkTokens: 5_000, recentWindowTokens: 50_000, adaptiveResolution: true, autoTickOnNewMessage: false }); + const manager = await ContextManager.open({ path: freshPath(), strategy, observer }); + for (let i = 0; i < 6; i++) manager.addMessage(i % 2 ? 'Claude' : 'User', [text(`message ${i} ` + 'words '.repeat(40))]); + const internals = strategy as unknown as CalibrationInternals; + /** Compile, then report `ratio` times the compile's estimate as the provider's total, with a provisional multiplier if asked. */ + const sample = async (ratio: number, provisional = false) => { + await manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + if (provisional) internals._calibrationProvisional = { fromEpoch: null }; + const estimate = internals._lastCompileEstimate; + assert.ok(estimate > 0); + strategy.reportRealInputTokens(estimate * ratio); + }; + return { manager, strategy, internals, rec, sample }; +} + +describe('calibration samples', () => { + it('one record per sample, with its outcome, ratio and the multiplier after', async () => { + const fx = await calibrated(); + await fx.sample(3); + await fx.sample(1.01); + await fx.sample(1.5); + // A second report after one compile is no sample. + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 1.5); + const outcomes = fx.rec.samples.map((s) => s.outcome); + assert.deepEqual(outcomes, ['out-of-band', 'held', 'stepped']); + for (const [record, ratio] of fx.rec.samples.map((r, i) => [r, [3, 1.01, 1.5][i]!] as const)) { + assert.ok(Math.abs(record.ratio - ratio) < 1e-9, `${record.outcome}: ratio ${record.ratio}, given ${ratio}`); + } + assert.equal(fx.rec.samples[0]!.multiplier, 1); + assert.equal(fx.rec.samples[1]!.multiplier, 1); + // The held sample moved the average to 1.002, and the step goes a fifth of the way from there to 1.5. + assert.ok(Math.abs(fx.rec.samples[2]!.multiplier - (1.002 + 0.2 * (1.5 - 1.002))) < 1e-9, String(fx.rec.samples[2]!.multiplier)); + assert.equal(fx.rec.samples[2]!.multiplier, fx.internals._calibration); + fx.manager.close(); + }); + + it('a provisional multiplier is confirmed in the dead band, and replaced outside it', async () => { + const confirm = await calibrated(); + await confirm.sample(1.01, true); + assert.deepEqual(confirm.rec.samples.map((s) => [s.outcome, s.multiplier]), [['confirmed', 1]]); + assert.ok(Math.abs(confirm.rec.samples[0]!.ratio - 1.01) < 1e-9); + confirm.manager.close(); + + const replace = await calibrated(); + await replace.sample(1.3, true); + assert.equal(replace.rec.samples.length, 1); + assert.equal(replace.rec.samples[0]!.outcome, 'replaced'); + assert.ok(Math.abs(replace.rec.samples[0]!.multiplier - 1.3) < 1e-9, String(replace.rec.samples[0]!.multiplier)); + assert.ok(Math.abs(replace.rec.samples[0]!.ratio - 1.3) < 1e-9); + replace.manager.close(); + }); + + it('a ratio out of band whose implied multiplier is in band is learned, not out-of-band', async () => { + // At the 1.8 clamp, an honest sample reads about 0.56 of the estimate: out of band as a ratio, but it implies 1.008. + const fx = await calibrated(); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + fx.internals._calibration = 1.8; + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 0.56); + assert.equal(fx.rec.samples.length, 1); + assert.equal(fx.rec.samples[0]!.outcome, 'stepped'); + assert.ok(Math.abs(fx.rec.samples[0]!.ratio - 0.56) < 1e-9); + assert.ok(Math.abs(fx.rec.samples[0]!.multiplier - (1.8 + 0.2 * (0.56 * 1.8 - 1.8))) < 1e-9, String(fx.rec.samples[0]!.multiplier)); + fx.manager.close(); + }); + + it('a report that is no sample makes no record and leaves the sample armed', async () => { + const fresh = await calibrated(); + fresh.strategy.reportRealInputTokens(5_000); + assert.deepEqual(fresh.rec.samples, [], 'no compile, so no estimate to compare'); + fresh.manager.close(); + + const fx = await calibrated(); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + for (const total of [0, -5, Number.NaN, Number.POSITIVE_INFINITY]) fx.strategy.reportRealInputTokens(total); + assert.equal(fx.rec.samples.length, 0); + assert.equal(fx.internals._calibrationArmed, true, 'the compile\'s sample is still to come'); + fx.strategy.reportRealInputTokens(fx.internals._lastCompileEstimate * 1.01); + assert.deepEqual(fx.rec.samples.map((r) => r.outcome), ['held']); + fx.manager.close(); + }); + + it('an observer that throws or rejects changes nothing a sample does', async () => { + const multipliers: number[] = []; + const told: string[][] = [[], []]; + const stray = await unhandledRejections(async () => { + for (const observe of ['none', hostile('throws', told[0]), hostile('rejects', told[1])] as const) { + const fx = await calibrated(observe); + await fx.sample(3); + await fx.sample(1.5); + multipliers.push(fx.internals._calibration); + fx.manager.close(); + } + }); + assert.equal(multipliers[1], multipliers[0]); + assert.equal(multipliers[2], multipliers[0]); + assert.notEqual(multipliers[0], 1, 'the stepped sample moved it'); + for (const calls of told) assert.equal(calls.filter((c) => c === 'calibration').length, 2, `both samples reached the hostile observer: ${calls.join(', ')}`); + assert.deepEqual(stray, []); + }); +}); diff --git a/test/overbudget-diagnostics.test.ts b/test/overbudget-diagnostics.test.ts index c32186ba..ab0f4cdf 100644 --- a/test/overbudget-diagnostics.test.ts +++ b/test/overbudget-diagnostics.test.ts @@ -503,6 +503,7 @@ async function refusalOfWholeMiddle(manager: ContextManager): Promise { assert.doesNotMatch(error.message, /deepest planned level/); } else { assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.match(error.message, /attempted representation level=L0/); } }); @@ -650,6 +652,7 @@ describe('OverBudgetError diagnostics', () => { strategy.seedPyramid([id], 5); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); + assert.equal((error as OverBudgetError).stageId, 'emission-overran-plan'); checkBudget(error); assert.deepEqual(error.clearedBy, [], 'the plan has no summaries left to produce'); assert.equal(error.diagnostics.deepestLevel, 5, 'the first raw emission fails, but the complete plan reaches L5'); @@ -1022,6 +1025,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); + assert.equal((error as OverBudgetError).stageId, 'emission-overran-plan'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -1043,10 +1047,12 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Tail emission dropped reserved recent-window messages/); + assert.equal((error as OverBudgetError).stageId, 'tail-emission-dropped'); assert.deepEqual(error.clearedBy, ['compression']); strategy.produce = false; const without = await refusal(manager); assert.match(without.message, /^Tail emission dropped reserved recent-window messages/); + assert.equal((without as OverBudgetError).stageId, 'tail-emission-dropped'); assert.deepEqual(without.clearedBy, [], 'a plan with nothing to produce names nothing'); }); }); @@ -1112,6 +1118,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Structural repair did not retain the newest turn/); + assert.equal((dry as OverBudgetError).stageId, 'repair-lost-newest-turn'); assert.deepEqual(dry.clearedBy, []); // The live compile refuses the plan at its gate first, so the preview // names what that refusal names. @@ -1223,9 +1230,11 @@ describe('OverBudgetError.clearedBy', () => { for (let i = 0; i < 6; i++) manager.addMessage('user', [{ type: 'text', text: `message ${i} ` + 'word '.repeat(30) }]); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Emission overran the plan/); + assert.equal((dry as OverBudgetError).stageId, 'emission-overran-plan'); assert.deepEqual(dry.clearedBy, []); const live = await refusal(manager); assert.match(live.message, /^Emission overran the plan/); + assert.equal((live as OverBudgetError).stageId, 'emission-overran-plan'); assert.deepEqual(live.clearedBy, ['compression'], 'the live compile demands the chunk first'); }); }); @@ -1246,6 +1255,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are waiting for compression'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -2034,6 +2044,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.seedL1('L1-covering', ids); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // Tick drops each one without compressing it: the stub membrane has no // model to call, and no L1 is added. @@ -2089,6 +2100,7 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(!strategy.summaryIds().some((id) => id.startsWith('L1-stored-')), 'no exact L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2108,6 +2120,7 @@ describe('OverBudgetError.clearedBy', () => { }); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, [], 'adoption doesn\'t persist, so the stored L1s shrink nothing'); }, { membrane: true }); }); @@ -2122,6 +2135,7 @@ describe('OverBudgetError.clearedBy', () => { assert.deepEqual(strategy.summaryIds(), [], 'no L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2153,6 +2167,7 @@ describe('OverBudgetError.clearedBy', () => { queued.forEach((ids, i) => strategy.seedL1(`L1-exact-${i}`, ids)); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(strategy.queuedChunkIds(), queued, 'the compile closed no new chunk'); assert.deepEqual(error.clearedBy, persistChunks ? ['compression'] : []); if (!persistChunks) return; @@ -2449,6 +2464,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'the merge is queued'); const error = await refusal(manager, undefined, positionedRecallPairs ? undefined : PAIR_FITS); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.equal(Reflect.get(error.diagnostics, 'middleCountUnit'), positionedRecallPairs ? 'selection-items' : 'raw-messages', 'the pair, or with the combined pair fitting, the raw message after it, refuses'); assert.deepEqual(error.clearedBy, [], 'tick stops at a blocked merge until the hold is released'); @@ -2493,6 +2509,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are queued'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, merge === 'blocked' ? [] : ['compression']); }); }); @@ -2636,6 +2653,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: tick still takes them, adopting each stored L1. await adoptAll(manager, strategy); @@ -2653,6 +2671,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: adopting the exact L1s leaves the render as it was, where unpinned chunks' adoption shrinks it. const before = (await manager.compile({ maxTokens: 100_000, reserveForResponse: 20 })).messages.length; @@ -2777,6 +2796,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2791,6 +2811,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2804,6 +2825,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); + assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); diff --git a/test/system-changes.test.ts b/test/system-changes.test.ts index 70964238..496fbcf7 100644 --- a/test/system-changes.test.ts +++ b/test/system-changes.test.ts @@ -849,23 +849,27 @@ describe('the passthrough strategies and a system change before their window', ( assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); assert.deepEqual(err.clearedBy, [], `${label}: only consolidating clears it`); if (label === 'passthrough') assert.match(err.message, /^Passthrough kept only its system changes/, label); + if (label === 'passthrough') assert.equal(err.stageId, 'passthrough-system-changes', label); // Its window still holds the newest conversation message, which its numbers count. if (label === 'windowed-passthrough') assert.match(err.message, /^Windowed passthrough: its system changes and the newest conversation message/, label); + if (label === 'windowed-passthrough') assert.equal(err.stageId, 'windowed-changes-and-newest-message', label); if (label === 'hierarchical') assert.match(err.message, /system changes/, label); + if (label === 'hierarchical') assert.equal(err.stageId, 'hierarchical-system-changes', label); return true; }); } // A store of changes alone: over the budget, each passthrough strategy names them alone; // filling it exactly, each sends them. - for (const [label, make, stage] of [ - ['passthrough', () => new PassthroughStrategy(), /^Passthrough kept only its system changes/], - ['windowed-passthrough', () => new WindowedPassthroughStrategy(), /^Windowed passthrough: its system changes alone/], + for (const [label, make, stage, stageId] of [ + ['passthrough', () => new PassthroughStrategy(), /^Passthrough kept only its system changes/, 'passthrough-system-changes'], + ['windowed-passthrough', () => new WindowedPassthroughStrategy(), /^Windowed passthrough: its system changes alone/, 'windowed-system-changes'], ] as const) { const alone = await open(make()); const id = alone.addMessage('Harness', [huge]); await assert.rejects(alone.compile(budget), (err: unknown) => { assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); assert.match(err.message, stage, label); + assert.equal(err.stageId, stageId, label); return true; }); assert.deepEqual(rawIds(await alone.compile({ maxTokens: estimateOf(alone, id), reserveForResponse: 0 })), [id], label); @@ -1049,6 +1053,19 @@ describe('the passthrough strategies and a system change before their window', ( await assert.rejects(windowed.compile({ maxTokens: estimateOf(windowed, big2) - 1, reserveForResponse: 0 }), (err: unknown) => { assert.ok(err instanceof OverBudgetError, String(err)); assert.match(err.message, /^Windowed passthrough: the newest message alone/); + assert.equal(err.stageId, 'windowed-newest-message'); + return true; + }); + // Passthrough's changes fit, but no conversation message fits beside them: + // it names both, and the newest message's tokens. + const both = await open(new PassthroughStrategy()); + const rule = both.addMessage('Harness', [change({ addSystem: ['Rule. '.repeat(40)] })]); + const newest = both.addMessage('User', [text('big ' + 'word '.repeat(400))]); + await assert.rejects(both.compile({ maxTokens: estimateOf(both, rule) + 10, reserveForResponse: 0 }), (err: unknown) => { + assert.ok(err instanceof OverBudgetError, String(err)); + assert.match(err.message, /^Passthrough: its system changes and the newest conversation message/); + assert.equal(err.stageId, 'passthrough-changes-and-newest-message'); + assert.equal(err.diagnostics.tailTokens, estimateOf(both, newest)); return true; }); }); From 879693729f12adbfc63db029cf0b22ec0b43c804 Mon Sep 17 00:00:00 2001 From: Oscar-1005 Date: Sun, 25 Oct 2026 00:17:56 +0200 Subject: [PATCH 63/67] chore(release): 0.22.0 Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 109 ++++++++++++++++++ changelog.d/155-thinking-binding.added.md | 14 --- changelog.d/request-image-ceilings.added.md | 2 - .../request-image-ceilings.breaking.md | 5 - changelog.d/request-image-ceilings.changed.md | 3 - changelog.d/request-image-ceilings.fixed.md | 1 - changelog.d/system-changes.added.md | 49 -------- changelog.d/system-changes.fixed.md | 18 --- package-lock.json | 4 +- package.json | 2 +- 10 files changed, 112 insertions(+), 95 deletions(-) delete mode 100644 changelog.d/155-thinking-binding.added.md delete mode 100644 changelog.d/request-image-ceilings.added.md delete mode 100644 changelog.d/request-image-ceilings.breaking.md delete mode 100644 changelog.d/request-image-ceilings.changed.md delete mode 100644 changelog.d/request-image-ceilings.fixed.md delete mode 100644 changelog.d/system-changes.added.md delete mode 100644 changelog.d/system-changes.fixed.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 9585c430..ee68e811 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,115 @@ Releases up to and including 0.6.2 predate this file; for their contents see ## Unreleased +## 0.22.0 — 2026-10-24 + +### Breaking + +- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and that ceiling's limit was over. + - **Cost at a ceiling:** a window held at a ceiling strips its oldest images as new ones come, as under any limit, but from deep in the request, so the prompt cache is rewritten from there at every new image. At the store's default price of 1,600 tokens an image, a window held at the count ceiling rewrites about 800k tokens of it at each new image. The autobiographical strategy's `imageStripHysteresisRatio` strips in batches instead, so the cache is rewritten once a batch, and its line names that option when it's off. + - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. + - **Siblings:** no sibling's current range admits this release; each takes these changes on when it raises its range. + - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. + +### Added + +- `compile()` takes `thinkingBinding` and `identity` (`SelectOptions`) for a model that binds each signed thinking block to the request it was minted in (#155). On an account that enforces the binding, such a model refuses a request carrying a block whose request has changed since, with a 400 ("Invalid `signature` in `thinking` block. The block is bound to a different conversation…"), and with the provider's drop control on it sets the block aside instead. With `thinkingBinding.binds`, a compile leaves such blocks out. + - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. + - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message its last committing compile with `binds` sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. + - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. + - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. + - **A dry compile** reports what it would leave out and writes no record. + - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent and its stream noted keeps its thinking, and nothing past the record's end does. + - **A record the store can't read or write** is logged and never fails a compile or a note. One it can't read is read as none, so a committing compile with `binds` writes its own in its place; when it can't write a compile's record, the record before it, if any, stands. + - **While a request is streaming,** compile for its branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing, so its replies go without their thinking. + - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. +- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until a compile writes it again; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile with `binds` (a compile without it leaves the record, so a later note still writes), and on a branch with no record or one it can't read, or when the store can't write it. +- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the compile that wrote the branch's record, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. +- `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. +- On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. + +- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They count images as the window's limits do. They don't bound a whole request: an image inside an opaque provider item, and anything a caller injects at compile time or appends to a compiled request, aren't counted. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. +- `ContextStrategy.imageCeilings()`, which both strategies implement, reports the two ceilings as the strategy applies them, a malformed one at its default. It reports configuration, not usage: a caller that wants a request within the ceilings counts that request's images itself. `ImageCeilings` is exported. + +- System changes: a message holding only membrane's `system_change` blocks + (`SystemChangeContent`, membrane's release that adds them) changes the tools + and system text the model is offered from its place on, and every strategy + compiles it unaltered, in its place among the raw messages. (Under the + autobiographical strategy's `positionedRecallPairs: false`, the middle's + summaries go ahead of its raw messages, as they always have.) The + autobiographical strategy, and with it `KnowledgeStrategy`, keeps it raw as + a pin keeps a message, so no fold takes it and no chunk holds or spans it; + after a head-window reset that leaves messages in the head window, the + changes stored before the new head are part of the head, ahead of the head + window, in order. `PassthroughStrategy` and + `WindowedPassthroughStrategy` keep one that falls before their window, + ahead of the window and priced into it, and the windowed strategy sizes a + re-anchored window as its fraction of the room the changes leave. None is + dropped: when the changes alone exceed the budget (beyond the + autobiographical strategy's grace, and on its hierarchical path only where + `enforceBudget` is on), every strategy refuses with `OverBudgetError`, + `clearedBy` empty, until the caller consolidates them into `tools` and + `system`; a dry compile reports where the strategy's other refusals do, + which for the passthrough strategies is nowhere. They also refuse when the + newest conversation message doesn't fit beside the changes, naming + `newer-message` while they leave room for one. + Compression never reads one. It's priced from its JSON. A write refuses one + beside other blocks or inside a tool result, and one whose `removeTools` or + `addSystem` isn't an array of strings, or whose `defineTools` isn't an array + of tool definitions, each with a non-empty string `name`, a string + `description` and an object `inputSchema` whose `type` is `'object'`; a read + leaves such a change out. A write also refuses a change while one of the + nearest calls before it has no result between it and the change (counting + back over other changes and over results, by id, to the nearest message + holding a call, every shard of it), + a tool result answering a call that a change already follows, and an edit + giving a message a `tool_use` that a change follows before its results: + membrane sends a change there after the result anyway, so write it after + the result. Ingress sharding never splits a message of them, since it holds + no text block. + Only membrane's `NativeFormatter`, on an adapter that sends system changes, + sends one; elsewhere membrane refuses the request. Rolling back to a + release without them isn't supported once a store holds one. +- `CompileResult.firstChanged`, optional: on a committing compile, the index + of the first compiled message, injections included, whose participant or + content bytes differ from the last committing compile's message at that + index, or that has none there, so after appends alone it is the first + appended index; an injection placed beside the newest user message moves + with each new user message, so after one it's that injection's old index. + It compares bytes, so it holds across a branch switch or a new strategy, + and it's `null` only for the manager's first committing compile. A dry + compile leaves it out and doesn't move it. It's compared when read, + so a compile whose caller never reads it serializes only its injections. + +### Changed + +- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. +- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and the limit that ceiling sets was over: at an image the render replaces, or, with `imageStripHysteresisRatio`, where it brought a trim on anywhere in the rule's replay of the history. The depth limit or a limit the host set may be stripping images there too. +- `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. + +### Fixed + +- A compression prompt carries at most `imageCeiling` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `imageByteCeiling`, and the line it logs when it replaces images names the count ceiling and the images kept. + +- A summarizer request no longer carries an empty turn, or a `---` separator + with nothing after it, where a message left empty for it sat, as one of + thinking alone was: merging consecutive messages passes over an empty one, + and its neighbours join as they would without it. +- With the head window disabled (`headWindowTokens: 0`), a compile after + `resetHeadWindow` under `adaptiveResolution` no longer sends history from + before the reset twice, and under kv-unified no longer throws + `CanonicalForestError` (`duplicate-chunk-id`) for it: the middle's ranges + overlapped each other, and the recent window when it starts before the + reset. +- A `WindowedPassthroughStrategy` compile that refuses no longer moves the + window's anchor; only a compile that sends does. So a retry with a larger + budget still sends the older turns that fit. +- A `KnowledgeStrategy` chunk no longer runs into or across an L1 summary's + span once a removal moves where its walk closes, which left the chunk after + it uncompressed for good and its messages sent raw beside the summary: a + chunk now closes where the L1 summary covering its messages changes, except + between a call and the result right after it. + ## 0.21.0 — 2026-10-24 ### Added diff --git a/changelog.d/155-thinking-binding.added.md b/changelog.d/155-thinking-binding.added.md deleted file mode 100644 index e0803f1d..00000000 --- a/changelog.d/155-thinking-binding.added.md +++ /dev/null @@ -1,14 +0,0 @@ -- `compile()` takes `thinkingBinding` and `identity` (`SelectOptions`) for a model that binds each signed thinking block to the request it was minted in (#155). On an account that enforces the binding, such a model refuses a request carrying a block whose request has changed since, with a 400 ("Invalid `signature` in `thinking` block. The block is bound to a different conversation…"), and with the provider's drop control on it sets the block aside instead. With `thinkingBinding.binds`, a compile leaves such blocks out. - - **What it takes:** `thinkingBinding: { binds, dropBlock, participant }`, with `participant` the name the model's own replies are stored under, and `identity: { system, tools, model, request }`, digests of the request's head. With `binds`, `participant` and `identity` are required, and a compile without them, or with a digest that isn't a string, is refused before anything is selected. - - **The record:** the manager keeps one per branch, in its namespace, of what the branch last sent: each message its last committing compile with `binds` sent (injections included, cache breakpoints aside), the rows a host noted after them, the stored replies the compile sent without their thinking, the compile's head digests, and the branch's sequence where the record began. It survives a restart, and a branch made from another starts from the parent's as of its branch point. - - **What a compile keeps:** from the first message that differs from the record, or from the first message when an `identity` digest differs, every message goes without its thinking. Past the record's end nothing keeps its thinking: only the host knows which stored reply answered which request. A reply sent without its thinking stays without it while it renders. Another participant's message, and one with no stored body, such as a summary's recall answer or an injection, never keeps thinking. - - **Replies from before the record:** without `dropBlock`, a reply stored at or before the sequence where the branch's record began never keeps its thinking. So a branch's first compile without the drop control leaves out every reply's thinking; with the drop control it leaves them for the provider to judge. Either way it starts the record. - - **A dry compile** reports what it would leave out and writes no record. - - **Without `binds`,** a compile is as before: it neither reads nor writes the branch's record. A record an earlier binding compile left is compared at the next binding compile like any other, so only a prefix that matches what that compile sent and its stream noted keeps its thinking, and nothing past the record's end does. - - **A record the store can't read or write** is logged and never fails a compile or a note. One it can't read is read as none, so a committing compile with `binds` writes its own in its place; when it can't write a compile's record, the record before it, if any, stands. - - **While a request is streaming,** compile for its branch with `binds` only with `dryRun`: a committing compile with `binds` makes that stream's later notes vouch for nothing, so its replies go without their thinking. - - **A host that passes `thinkingBinding` and notes nothing** loses each reply's thinking at the compile after it's stored, once the branch has a record, and is never refused for it. -- `ContextManager.noteRequestSent(rows, identity, compileId)` (new): a host calls it, with the `provenance.compileId` of the compile its stream was built from, as it sends each later round of that stream, with every row the stream's request carried after the compiled messages so far, and once the stream has ended, with those rows and its last reply. Each row, in order, is the id of a stored message, or `{ unstored: true }` for one the host can't vouch was carried or returned as stored. The record then holds them, so the next compile keeps their thinking while nothing up to them differs. Each stored row is digested as `renderAsNewest` renders it and as the compile sends it, a bundled tool cycle as each of its parts, and an id the store no longer holds vouches for nothing, as the marker does. A request-only row the host puts back in the same place at every later request needs no row. Call it on the branch the stream's compile was on. A note sent with another head than the record's leaves the record vouching for nothing until a compile writes it again; a compile under the new head reports the head change, and one under the record's own head reports `'messages'`. It returns false, writing nothing, when `compileId` isn't the compile that wrote the branch's record, as after another committing compile with `binds` (a compile without it leaves the record, so a later note still writes), and on a branch with no record or one it can't read, or when the store can't write it. -- `CompileResult.thinkingStrip` says what a binding compile newly leaves out of the participant's replies (`newly`, the stored bodies, against the compile that wrote the branch's record, sent or not, so a reply that compile didn't send can be reported again) and why (`cause`: `'head'` with the `changed` digests, `'first'`, or `'messages'`). `CompileResult.thinkingRecordedAfter` gives, on every binding compile, the branch's sequence when its record began. A reply at or below it was stored before the record, so the provider dropping its thinking isn't a strip that missed. A body sent without its thinking reads as incomplete in provenance (`missing: ['content']`), and the rendered layout prices what was sent: thinking only where a stored reply kept it. New exports: `ThinkingBinding`, `RequestIdentity`, `ThinkingStrip`. -- `ContextStrategy.noteThinkingLeftOut(blocks)`: after a committing compile, the manager hands the strategy the stored thinking it left out. `AutobiographicalStrategy` takes their price off its calibration sample's estimate, so the sample compares the provider's count with what was sent. -- On a binding compile, `AutobiographicalStrategy` renders and prices a summary's carriers as under `carrierPolicy: 'live-strip'`, whatever the configured policy: a carrier was minted in its summarizer's request, so such a model never takes it in the live window. A select prices them by its own option, so `previewContext` takes `thinkingBinding` too, and a preview without it prices them as configured. Mint and merge requests are unchanged. diff --git a/changelog.d/request-image-ceilings.added.md b/changelog.d/request-image-ceilings.added.md deleted file mode 100644 index 968a414a..00000000 --- a/changelog.d/request-image-ceilings.added.md +++ /dev/null @@ -1,2 +0,0 @@ -- `imageCeiling` and `imageByteCeiling`, on `AutobiographicalConfig` and `WindowedPassthroughOptions`: the most images, and the most of their inline image bytes, a strategy's live window keeps, whatever `maxLiveImages` and `maxLiveImageBytes` say, and the most images and image bytes a compression prompt carries. They count images as the window's limits do. They don't bound a whole request: an image inside an opaque provider item, and anything a caller injects at compile time or appends to a compiled request, aren't counted. The defaults are 500 images, since the provider refuses a request with more than 600 images and documents together, and 24 MiB of base64, under membrane's 28 MiB request byte cap. They're in `DEFAULT_AUTOBIOGRAPHICAL_CONFIG`, so `config:effective` reports them. 0 means no ceiling, and a value that isn't 0 or a finite number above it takes the default. -- `ContextStrategy.imageCeilings()`, which both strategies implement, reports the two ceilings as the strategy applies them, a malformed one at its default. It reports configuration, not usage: a caller that wants a request within the ceilings counts that request's images itself. `ImageCeilings` is exported. diff --git a/changelog.d/request-image-ceilings.breaking.md b/changelog.d/request-image-ceilings.breaking.md deleted file mode 100644 index bc2f7d49..00000000 --- a/changelog.d/request-image-ceilings.breaking.md +++ /dev/null @@ -1,5 +0,0 @@ -- **Host operators:** a `maxLiveImages` or `maxLiveImageBytes` of 0, documented as no limit, now means up to `imageCeiling` (500 images by default) or `imageByteCeiling` (24 MiB), and a `maxCompressionImageBytes` of 0, which meant no budget, now means up to `imageByteCeiling`. So does any other value of these that isn't a positive finite number: a negative or `Infinity`, which had also meant no limit, or `NaN`, which had meant no limit in the window and had replaced every image in a compression prompt. A value above its ceiling, such as a `maxLiveImageBytes` of 30 MiB, is now held to the ceiling, and the strategy warns about it when it initializes. A host that set one of these to 0 to keep every image, or above a ceiling, now keeps up to the ceiling's worth, and its strategy says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and that ceiling's limit was over. - - **Cost at a ceiling:** a window held at a ceiling strips its oldest images as new ones come, as under any limit, but from deep in the request, so the prompt cache is rewritten from there at every new image. At the store's default price of 1,600 tokens an image, a window held at the count ceiling rewrites about 800k tokens of it at each new image. The autobiographical strategy's `imageStripHysteresisRatio` strips in batches instead, so the cache is rewritten once a batch, and its line names that option when it's off. - - **Migration:** to keep no limit, or a limit above a ceiling, set that ceiling to 0 or raise it. A host that raised membrane's byte cap (`MEMBRANE_MAX_REQUEST_BYTES`) raises `imageByteCeiling` with it, leaving room under the cap for what the ceiling doesn't count. - - **Siblings:** no sibling's current range admits this release; each takes these changes on when it raises its range. - - **Unchanged:** for a host that sets none of these, the live window, whose defaults of 6 images and 20 MiB are under the default ceilings. A compression prompt keeps its default 12 MiB budget, though it now carries at most 500 images (under Fixed). A whole-number limit above 0 and under its ceiling works as before; a fractional one is now floored, to at least 1. diff --git a/changelog.d/request-image-ceilings.changed.md b/changelog.d/request-image-ceilings.changed.md deleted file mode 100644 index 97068ca5..00000000 --- a/changelog.d/request-image-ceilings.changed.md +++ /dev/null @@ -1,3 +0,0 @@ -- The live window's image limits stay within the image ceilings, in both strategies' strip, the autobiographical strategy's `imageStripHysteresisRatio` set and its post-strip estimates. They bind the images the window counts, as its limits count them. A limit above its ceiling keeps the ceiling's worth, and a fractional limit is floored, to at least 1. The default window, 6 images and 20 MiB, is under the default ceilings and doesn't change. -- Each strategy warns once, when it initializes, when a limit is above its ceiling (the compression budget included), or when a ceiling isn't 0 or a finite number above it and takes its default. A setting the host didn't give is marked as the default. Under a window limit of 0 or above a ceiling, it also says once which ceiling it was and what it holds, at the first render a select returns where images are shown as placeholders and the limit that ceiling sets was over: at an image the render replaces, or, with `imageStripHysteresisRatio`, where it brought a trim on anywhere in the rule's replay of the history. The depth limit or a limit the host set may be stripping images there too. -- `maxLiveImageBytes`' documentation no longer says the API's request cap is 32 MB. It names membrane's request byte cap, which the image byte ceiling leaves room under. diff --git a/changelog.d/request-image-ceilings.fixed.md b/changelog.d/request-image-ceilings.fixed.md deleted file mode 100644 index 83838a43..00000000 --- a/changelog.d/request-image-ceilings.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- A compression prompt carries at most `imageCeiling` images, counted newest message first in the walk that keeps its byte budget, images nested in tool results and generated images included. It had no count limit, so a chunk with more than 600 images made a prompt the provider refuses. Its byte budget, `maxCompressionImageBytes` (default 12 MiB), is kept within `imageByteCeiling`, and the line it logs when it replaces images names the count ceiling and the images kept. diff --git a/changelog.d/system-changes.added.md b/changelog.d/system-changes.added.md deleted file mode 100644 index 73c56596..00000000 --- a/changelog.d/system-changes.added.md +++ /dev/null @@ -1,49 +0,0 @@ -- System changes: a message holding only membrane's `system_change` blocks - (`SystemChangeContent`, membrane's release that adds them) changes the tools - and system text the model is offered from its place on, and every strategy - compiles it unaltered, in its place among the raw messages. (Under the - autobiographical strategy's `positionedRecallPairs: false`, the middle's - summaries go ahead of its raw messages, as they always have.) The - autobiographical strategy, and with it `KnowledgeStrategy`, keeps it raw as - a pin keeps a message, so no fold takes it and no chunk holds or spans it; - after a head-window reset that leaves messages in the head window, the - changes stored before the new head are part of the head, ahead of the head - window, in order. `PassthroughStrategy` and - `WindowedPassthroughStrategy` keep one that falls before their window, - ahead of the window and priced into it, and the windowed strategy sizes a - re-anchored window as its fraction of the room the changes leave. None is - dropped: when the changes alone exceed the budget (beyond the - autobiographical strategy's grace, and on its hierarchical path only where - `enforceBudget` is on), every strategy refuses with `OverBudgetError`, - `clearedBy` empty, until the caller consolidates them into `tools` and - `system`; a dry compile reports where the strategy's other refusals do, - which for the passthrough strategies is nowhere. They also refuse when the - newest conversation message doesn't fit beside the changes, naming - `newer-message` while they leave room for one. - Compression never reads one. It's priced from its JSON. A write refuses one - beside other blocks or inside a tool result, and one whose `removeTools` or - `addSystem` isn't an array of strings, or whose `defineTools` isn't an array - of tool definitions, each with a non-empty string `name`, a string - `description` and an object `inputSchema` whose `type` is `'object'`; a read - leaves such a change out. A write also refuses a change while one of the - nearest calls before it has no result between it and the change (counting - back over other changes and over results, by id, to the nearest message - holding a call, every shard of it), - a tool result answering a call that a change already follows, and an edit - giving a message a `tool_use` that a change follows before its results: - membrane sends a change there after the result anyway, so write it after - the result. Ingress sharding never splits a message of them, since it holds - no text block. - Only membrane's `NativeFormatter`, on an adapter that sends system changes, - sends one; elsewhere membrane refuses the request. Rolling back to a - release without them isn't supported once a store holds one. -- `CompileResult.firstChanged`, optional: on a committing compile, the index - of the first compiled message, injections included, whose participant or - content bytes differ from the last committing compile's message at that - index, or that has none there, so after appends alone it is the first - appended index; an injection placed beside the newest user message moves - with each new user message, so after one it's that injection's old index. - It compares bytes, so it holds across a branch switch or a new strategy, - and it's `null` only for the manager's first committing compile. A dry - compile leaves it out and doesn't move it. It's compared when read, - so a compile whose caller never reads it serializes only its injections. diff --git a/changelog.d/system-changes.fixed.md b/changelog.d/system-changes.fixed.md deleted file mode 100644 index f4645acd..00000000 --- a/changelog.d/system-changes.fixed.md +++ /dev/null @@ -1,18 +0,0 @@ -- A summarizer request no longer carries an empty turn, or a `---` separator - with nothing after it, where a message left empty for it sat, as one of - thinking alone was: merging consecutive messages passes over an empty one, - and its neighbours join as they would without it. -- With the head window disabled (`headWindowTokens: 0`), a compile after - `resetHeadWindow` under `adaptiveResolution` no longer sends history from - before the reset twice, and under kv-unified no longer throws - `CanonicalForestError` (`duplicate-chunk-id`) for it: the middle's ranges - overlapped each other, and the recent window when it starts before the - reset. -- A `WindowedPassthroughStrategy` compile that refuses no longer moves the - window's anchor; only a compile that sends does. So a retry with a larger - budget still sends the older turns that fit. -- A `KnowledgeStrategy` chunk no longer runs into or across an L1 summary's - span once a removal moves where its walk closes, which left the chunk after - it uncompressed for good and its messages sent raw beside the summary: a - chunk now closes where the L1 summary covering its messages changes, except - between a call and the result right after it. diff --git a/package-lock.json b/package-lock.json index f9c98d14..8d141d99 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@animalabs/context-manager", - "version": "0.21.0", + "version": "0.22.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@animalabs/context-manager", - "version": "0.21.0", + "version": "0.22.0", "license": "MIT", "dependencies": { "@animalabs/chronicle": "^0.4.0", diff --git a/package.json b/package.json index f164d71c..de4c1c2e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@animalabs/context-manager", - "version": "0.21.0", + "version": "0.22.0", "description": "Context management layer for LLM-based agents using Chronicle and Membrane", "type": "module", "main": "dist/src/index.js", From 292f724a31f2db7d3d3a2248da2fe2e612c64264 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Sat, 24 Oct 2026 19:03:41 -0400 Subject: [PATCH 64/67] fix(observer): answer the automated review of 9a14727 - The L1 overlap guard's refusal is reported as 'overlap-guard', a new reason in COMPRESSION_REFUSALS, once per chunk as its log line is: a refusal the library makes on its own judgment, which a host watching for store damage wants to see. - Text: a provider's refusal or error on a compression call is the provider's decision, which context-manager doesn't report (the comments no longer say membrane reports it). The README, the option's doc, the ContextObserver doc and the changelog name the one case isolation can't cover: a returned native promise whose constructor or species getter throws, which no JavaScript can attach a handler to. isReady() and compile() no longer say compile() waits for background work. - isolateObserver's failure sink is required; its silent default was unused. - Tests: an observed preview leaves the strategy as an unobserved one does, so the options reach the work; a compile that throws is timed whole; each record is taken before the caller awaits; the throwing-log test checks the log was asked; the overlap guard's report; and a shared check (test/fixtures/over-budget-hook.ts) that every refusal through the protected overBudgetError hook gives it the stage's words, or none for the picker's own refusal, beside each stageId the existing tests check, calibration-band's picker refusal included. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/context-observer.added.md | 14 ++++-- src/context-manager.ts | 12 +++-- src/observer.ts | 23 ++++++--- src/strategies/autobiographical.ts | 7 +-- test/calibration-band.test.ts | 7 ++- test/context-observer.test.ts | 67 +++++++++++++++++++++++++-- test/fixtures/over-budget-hook.ts | 45 ++++++++++++++++++ test/overbudget-diagnostics.test.ts | 45 +++++++++--------- test/system-changes.test.ts | 13 +++--- 10 files changed, 182 insertions(+), 53 deletions(-) create mode 100644 test/fixtures/over-budget-hook.ts diff --git a/README.md b/README.md index 61f1e62e..07b40161 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ Chronicle keeps the store's history, so a conversation can branch. `branchAt(mes ## Observing a manager's work -Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. +Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. The one exception is a returned native promise whose `constructor` or species getter throws, which no JavaScript can handle: that throw is logged, and the promise's rejection goes unhandled. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. ## Inspecting and repairing stores diff --git a/changelog.d/context-observer.added.md b/changelog.d/context-observer.added.md index 12d6715e..519a7301 100644 --- a/changelog.d/context-observer.added.md +++ b/changelog.d/context-observer.added.md @@ -5,8 +5,11 @@ called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics and drops the rest, so a compile's result is the same with an observer as - without one. A record's error and render stats are the objects the caller - gets, so an observer treats them as read-only. + without one. The one exception is a returned native promise whose + `constructor` or species getter throws, which no JavaScript can handle: that + throw is logged, and the promise's rejection goes unhandled. A record's + error and render stats are the objects the caller gets, so an observer + treats them as read-only. - `compile(record)`, once per `compile()`: the strategy, `dryRun`, `startedAt` on the wall clock (`Date.now()`), the whole compile's `durationMs` on the monotonic clock, `outcome` (`'compiled'`, `'refused'` @@ -14,9 +17,10 @@ and a compiled compile's `renderStats`, the object its result holds. - `compression(record)`: a `reason` from `COMPRESSION_REFUSALS` (`'admission-rejected'`, `'topology-violation'`, `'empty'`, `'incomplete'`, - `'malformed'`, and the split rung's caps, `'chunk-call-cap'` and - `'window-call-cap'`). A provider's refusal or error on a compression call - isn't one of these. + `'malformed'`, the split rung's caps, `'chunk-call-cap'` and + `'window-call-cap'`, and `'overlap-guard'`, a chunk refused because an L1 + already covers part of it). A provider's refusal or error on a compression + call isn't one of these. - `calibration(record)`, once per sample `reportRealInputTokens` takes: an `outcome` from `CALIBRATION_SAMPLE_OUTCOMES` (`'out-of-band'`, `'held'`, `'confirmed'`, `'replaced'`, `'stepped'`), the `ratio`, and the diff --git a/src/context-manager.ts b/src/context-manager.ts index c5fdbcaa..5bf5c8fe 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -148,8 +148,10 @@ interface ContextManagerBaseConfig { * `ContextObserver`). The manager hands it to its strategy in * `StrategyContext`. It's called synchronously, and what it throws or * rejects never reaches the work: the manager logs a callback's first - * failure through its diagnostics and drops the rest. A record's error and - * render stats are the objects the caller gets, so treat them as read-only. + * failure through its diagnostics and drops the rest. (A returned native + * promise whose `constructor` or species getter throws can't be handled + * from JavaScript; see `ContextObserver`.) A record's error and render + * stats are the objects the caller gets, so treat them as read-only. */ observer?: ContextObserver; /** @@ -1181,7 +1183,9 @@ export class ContextManager { // ========================================================================== /** - * Check if compile() will block waiting for background work. + * Whether the strategy has no background work pending (its + * `checkReadiness()`). compile() doesn't wait for that work: it compiles + * from what's there. */ isReady(): boolean { this.requireInitialized('isReady'); @@ -1222,7 +1226,7 @@ export class ContextManager { * With `ContextManagerConfig.observer`, it tells the observer of the * compile as it returns or throws (`ContextObserver.compile`). * - * May block if strategy has pending work. + * It awaits nothing, so it doesn't wait for the strategy's background work. */ async compile( budget?: TokenBudget, diff --git a/src/observer.ts b/src/observer.ts index 387c811b..6a09b7b0 100644 --- a/src/observer.ts +++ b/src/observer.ts @@ -25,8 +25,8 @@ export type CompileRefusal = typeof COMPILE_REFUSALS[number]; /** * Context-manager's own refusals of compression, from a closed list. A - * provider's refusal or error on a compression call is membrane's fact, not - * one of these: + * provider's refusal or error on a compression call is the provider's + * decision, not one of these: * - `'admission-rejected'`: the request didn't fit `compressionContextBudgetTokens`, * so no call was made. * - `'topology-violation'`: a merge's sources don't make a group it can @@ -47,8 +47,14 @@ export type CompileRefusal = typeof COMPILE_REFUSALS[number]; * - `'window-call-cap'`: the split rung's attempt alone used every call its * ten-minute window allows (`compressionSplitMaxCallsPer10Min`), so it ended * before another call. + * - `'overlap-guard'`: a chunk's messages are partly covered by an L1 that + * already exists, which only damaged bookkeeping or store surgery leaves, + * so the strategy refuses to compress it rather than mint a duplicate + * memory. It's refused on every attempt while the overlap stands, and + * reported once per chunk until the strategy loads its state again (an + * open, a branch switch), as its log line is. * - * Each is reported once per decision. A verdict on an answer the L1 ladder + * The rest are reported once per decision. A verdict on an answer the L1 ladder * keeps in its saved progress isn't decided again when the ladder resumes, * so it isn't reported again; an admission rejection makes no call, so * progress keeps none, and each ladder run that reaches it decides and @@ -64,6 +70,7 @@ export const COMPRESSION_REFUSALS = [ 'malformed', 'chunk-call-cap', 'window-call-cap', + 'overlap-guard', ] as const; /** A reason of {@link COMPRESSION_REFUSALS}. */ @@ -133,8 +140,12 @@ export interface CalibrationSampleRecord { * Called synchronously as each fact happens, each callback optional. What * it throws, or a promise it returns that rejects, never reaches the work it * observes: the manager logs a callback's first failure through its - * diagnostics, and drops the rest. A record's error and render stats are the - * objects the caller gets, so an observer treats them as read-only. + * diagnostics, and drops the rest. The one exception is a returned native + * promise whose `constructor` or species getter throws: no JavaScript can + * attach a handler to it, so that throw is logged and the promise's own + * rejection, if it rejects, goes unhandled. A record's error and render + * stats are the objects the caller gets, so an observer treats them as + * read-only. */ export interface ContextObserver { compile?(record: CompileRecord): void; @@ -152,7 +163,7 @@ export interface ContextObserver { */ export function isolateObserver( observer: ContextObserver, - failed: (callback: keyof ContextObserver, error: unknown) => void = () => {}, + failed: (callback: keyof ContextObserver, error: unknown) => void, ): ContextObserver { const isolated: ContextObserver = {}; const told = new Set(); diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index e5f6586f..e34ec920 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -1128,9 +1128,9 @@ type CompressionAttemptOutcome = /** * The refusal an outcome is when context-manager itself made it (see * `COMPRESSION_REFUSALS`), or nothing. A provider's refusal, and an error it - * returned or a run of them decided, are membrane's to report; a response's - * shape is context-manager's verdict (`assessFallbackCompressionResponse`'s - * `malformed_*`). + * returned or a run of them decided, are the provider's decisions, which + * context-manager doesn't report; a response's shape is context-manager's + * verdict (`assessFallbackCompressionResponse`'s `malformed_*`). */ function ownCompressionRefusal(outcome: CompressionAttemptOutcome, errorType?: string): CompressionRefusal | undefined { switch (outcome) { @@ -9155,6 +9155,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { const key = chunk.recordId ?? chunkIdKey; if (!this._overlapBlocked.has(key)) { this._overlapBlocked.add(key); + this.hostContext?.observer?.compression?.({ reason: 'overlap-guard' }); this.diag.error( `[autobiographical] OVERLAP GUARD: refusing to compress chunk ` + `${chunk.recordId ?? `#${chunk.index}`} — ${overlapIds.length}/${chunk.messages.length} ` + diff --git a/test/calibration-band.test.ts b/test/calibration-band.test.ts index 0c108c8c..c2353048 100644 --- a/test/calibration-band.test.ts +++ b/test/calibration-band.test.ts @@ -14,6 +14,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { Membrane, MockAdapter, NativeFormatter, type ContentBlock } from '@animalabs/membrane'; import { ContextManager, AutobiographicalStrategy } from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; import type { MessageStoreView, StoredMessage } from '../src/types/index.js'; const dir = mkdtempSync(join(tmpdir(), 'calibration-band-')); @@ -739,7 +740,11 @@ describe('estimator calibration band', () => { control.cm.close(); const controlReopened = await openSharded('picker-control', false); assert.equal(controlReopened.cm.previewContext(tight)!.fits, false); - await assert.rejects(controlReopened.cm.compile(tight), /Adaptive picker exhausted/); + await assert.rejects(controlReopened.cm.compile(tight), (err: unknown) => { + assert.match(String((err as Error).message), /Adaptive picker exhausted/); + assertStage(err, 'picker-exhausted'); + return true; + }); controlReopened.cm.close(); const first = await openSharded('picker', true); diff --git a/test/context-observer.test.ts b/test/context-observer.test.ts index 5cbbd5f6..51ae9ac8 100644 --- a/test/context-observer.test.ts +++ b/test/context-observer.test.ts @@ -116,12 +116,10 @@ describe('compile records', () => { }); it('an observer without a compile callback is told nothing of compiles, and the compile goes on', async () => { - const compressions: CompressionRefusalRecord[] = []; - const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compression: (r) => { compressions.push(r); } } }); + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { compression: () => {} } }); manager.addMessage('User', [text('hello')]); const result = await manager.compile(); assert.equal(result.messages.length, 1); - assert.deepEqual(compressions, []); manager.close(); }); @@ -250,6 +248,33 @@ describe('compile records', () => { manager.close(); }); + it('a compile that throws is timed whole too, and every record is taken before the caller awaits', async () => { + class SlowThenThrow extends PassthroughStrategy { + override select(): never { + const until = performance.now() + 40; + while (performance.now() < until) { /* 40 ms of work before the refusal */ } + throw new Error('a defect after some work'); + } + } + const rec = recorder(); + const manager = await ContextManager.open({ path: freshPath(), strategy: new SlowThenThrow(), observer: rec.observer }); + manager.addMessage('User', [text('hello')]); + const pending = manager.compile(); + assert.equal(rec.compiles.length, 1, 'told before the caller awaits'); + await pending.catch(() => {}); + assert.equal(rec.compiles[0]!.outcome, 'failed'); + assert.ok(rec.compiles[0]!.durationMs >= 40, String(rec.compiles[0]!.durationMs)); + manager.close(); + + const ok = recorder(); + const fine = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: ok.observer }); + fine.addMessage('User', [text('hello')]); + const compiled = fine.compile(); + assert.equal(ok.compiles.length, 1, 'told before the caller awaits'); + await compiled; + fine.close(); + }); + it('anything else thrown is a failed compile, null and strings included; a refusal from another copy of the package is still a refusal', async () => { const defect = new Error('a defect in select'); const foreignRefusal = Object.assign(new Error('refused by another copy'), { name: 'UncoveredDropError' }); @@ -361,14 +386,18 @@ describe('compile records', () => { it('a log that throws as it tells of a failure changes nothing either', async () => { const original = console.error; - console.error = () => { throw new Error('the log broke'); }; + let entered = 0; + console.error = () => { entered++; throw new Error('the log broke'); }; try { const throwing = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: hostile('throws') }); throwing.addMessage('User', [text('hello')]); assert.equal((await throwing.compile()).messages.length, 1); + assert.ok(entered > 0, 'the log was asked to tell of the throw'); throwing.close(); + entered = 0; const unreadable = { get compile(): never { throw new Error('a getter that throws'); } }; const opened = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: unreadable as never }); + assert.ok(entered > 0, 'the log was asked to tell of the unreadable callback'); opened.addMessage('User', [text('hello')]); assert.equal((await opened.compile()).messages.length, 1); opened.close(); @@ -651,7 +680,7 @@ describe('the exported lists', () => { }); it('the compression reasons and sample outcomes are as documented', () => { - assert.deepEqual([...COMPRESSION_REFUSALS], ['admission-rejected', 'topology-violation', 'empty', 'incomplete', 'malformed', 'chunk-call-cap', 'window-call-cap']); + assert.deepEqual([...COMPRESSION_REFUSALS], ['admission-rejected', 'topology-violation', 'empty', 'incomplete', 'malformed', 'chunk-call-cap', 'window-call-cap', 'overlap-guard']); assert.deepEqual([...CALIBRATION_SAMPLE_OUTCOMES], ['out-of-band', 'held', 'confirmed', 'replaced', 'stepped']); }); }); @@ -824,6 +853,20 @@ describe('compression refusals: an L1 ladder', () => { fx.manager.close(); }); + it('a chunk an existing L1 partly covers is refused by the overlap guard, reported once', async () => { + const { calls, membrane } = scripted([ANSWER('never asked')]); + const fx = await ladder(membrane, SPLIT); + const chunk = fx.target(); + const first = chunk.messages[0]!.id; + fx.strategy.seed({ id: 'L1-200', level: 1, content: 'covers raw-10 alone', tokens: 20, sourceLevel: 0, sourceIds: [first], sourceRange: { first, last: first }, created: 200 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0, 'nothing is asked'); + assert.equal(chunk.compressed, false); + assert.deepEqual(fx.rec.compressions, [{ reason: 'overlap-guard' }], 'refused twice, reported once'); + fx.manager.close(); + }); + it('an empty piece is reported even when the stitch succeeds', async () => { const { calls, membrane } = scripted([REFUSAL(), REFUSAL(), ANSWER(''), ANSWER('the tool round summarized ' + 'detail '.repeat(20))]); const fx = await ladder(membrane, { ...SPLIT, compressionSplitPlaceholder: true }); @@ -1087,6 +1130,20 @@ describe('calibration samples', () => { fx.manager.close(); }); + it('an observed preview leaves the strategy as an unobserved one does: its options reach the work', async () => { + for (const observe of ['none', 'record'] as const) { + const fx = await calibrated(observe); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }, undefined, { dryRun: true }); + assert.equal(fx.internals._calibrationArmed, false, `${observe}: a preview arms no sample`); + fx.strategy.reportRealInputTokens(5_000); + assert.deepEqual(fx.rec.samples, [], `${observe}: so a report after it is no sample`); + if (observe === 'record') assert.equal(fx.rec.compiles.at(-1)!.dryRun, true); + await fx.manager.compile({ maxTokens: 100_000, reserveForResponse: 2_000 }); + assert.equal(fx.internals._calibrationArmed, true, `${observe}: a live compile arms one`); + fx.manager.close(); + } + }); + it('a report that is no sample makes no record and leaves the sample armed', async () => { const fresh = await calibrated(); fresh.strategy.reportRealInputTokens(5_000); diff --git a/test/fixtures/over-budget-hook.ts b/test/fixtures/over-budget-hook.ts new file mode 100644 index 00000000..ffcf288c --- /dev/null +++ b/test/fixtures/over-budget-hook.ts @@ -0,0 +1,45 @@ +/** + * Records what each refusal through `AutobiographicalStrategy`'s protected + * `overBudgetError` hook was given, so a test can check that an override of + * the hook still gets a library stage's words as `stage`, as it did before + * `stageId`. Importing this wraps the hook for the importing test file. + */ +import assert from 'node:assert/strict'; +import { AutobiographicalStrategy, OverBudgetError } from '../../src/index.js'; + +type HookOptions = { stage?: string; stageId?: string }; + +const given = new WeakMap(); +const proto = AutobiographicalStrategy.prototype as unknown as { + overBudgetError(budget: unknown, opts: HookOptions): OverBudgetError; +}; +const original = proto.overBudgetError; +proto.overBudgetError = function (this: unknown, budget: unknown, opts: HookOptions): OverBudgetError { + const error = original.call(this, budget, opts); + given.set(error, { stage: opts.stage, stageId: opts.stageId }); + return error; +}; + +/** + * `error` is an `OverBudgetError` naming `stageId`. A refusal of + * `AutobiographicalStrategy`'s (any but passthrough's and windowed + * passthrough's) came through the hook, which was given that id and the + * words that open its message, or, for the picker's own refusal, neither, as + * before. + */ +export function assertStage(error: unknown, stageId: string): void { + assert.ok(error instanceof OverBudgetError, String(error)); + assert.equal(error.stageId, stageId); + const hook = given.get(error); + if (stageId.startsWith('windowed-') || stageId.startsWith('passthrough-')) return; + assert.ok(hook !== undefined, `${stageId}: the refusal came through the hook`); + if (stageId === 'picker-exhausted') { + assert.deepEqual(hook, { stage: undefined, stageId: undefined }, 'the picker\'s refusal gives the hook no stage'); + return; + } + assert.equal(hook.stageId, stageId); + assert.ok( + typeof hook.stage === 'string' && error.message.startsWith(`${hook.stage} but `), + `the hook was given the words that open the message: ${String(hook.stage)} / ${error.message.slice(0, 80)}`, + ); +} diff --git a/test/overbudget-diagnostics.test.ts b/test/overbudget-diagnostics.test.ts index ab0f4cdf..b8c7c0b1 100644 --- a/test/overbudget-diagnostics.test.ts +++ b/test/overbudget-diagnostics.test.ts @@ -4,6 +4,7 @@ import { mkdtempSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { AutobiographicalStrategy, ContextManager, OverBudgetError } from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; import type { StoredMessage, SummaryEntry, SummaryLevel } from '../src/types/index.js'; import { Picker, type PickerInputs } from '../src/adaptive/picker.js'; import type { FoldingBudget } from '../src/adaptive/folding-strategy.js'; @@ -503,7 +504,7 @@ async function refusalOfWholeMiddle(manager: ContextManager): Promise { assert.doesNotMatch(error.message, /deepest planned level/); } else { assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.match(error.message, /attempted representation level=L0/); } }); @@ -652,7 +653,7 @@ describe('OverBudgetError diagnostics', () => { strategy.seedPyramid([id], 5); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); - assert.equal((error as OverBudgetError).stageId, 'emission-overran-plan'); + assertStage(error, 'emission-overran-plan'); checkBudget(error); assert.deepEqual(error.clearedBy, [], 'the plan has no summaries left to produce'); assert.equal(error.diagnostics.deepestLevel, 5, 'the first raw emission fails, but the complete plan reaches L5'); @@ -1025,7 +1026,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Emission overran the plan/); - assert.equal((error as OverBudgetError).stageId, 'emission-overran-plan'); + assertStage(error, 'emission-overran-plan'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -1047,12 +1048,12 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const error = await refusal(manager); assert.match(error.message, /^Tail emission dropped reserved recent-window messages/); - assert.equal((error as OverBudgetError).stageId, 'tail-emission-dropped'); + assertStage(error, 'tail-emission-dropped'); assert.deepEqual(error.clearedBy, ['compression']); strategy.produce = false; const without = await refusal(manager); assert.match(without.message, /^Tail emission dropped reserved recent-window messages/); - assert.equal((without as OverBudgetError).stageId, 'tail-emission-dropped'); + assertStage(without, 'tail-emission-dropped'); assert.deepEqual(without.clearedBy, [], 'a plan with nothing to produce names nothing'); }); }); @@ -1118,7 +1119,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.startCompressing(); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Structural repair did not retain the newest turn/); - assert.equal((dry as OverBudgetError).stageId, 'repair-lost-newest-turn'); + assertStage(dry, 'repair-lost-newest-turn'); assert.deepEqual(dry.clearedBy, []); // The live compile refuses the plan at its gate first, so the preview // names what that refusal names. @@ -1230,11 +1231,11 @@ describe('OverBudgetError.clearedBy', () => { for (let i = 0; i < 6; i++) manager.addMessage('user', [{ type: 'text', text: `message ${i} ` + 'word '.repeat(30) }]); const dry = await refusal(manager, { dryRun: true }); assert.match(dry.message, /^Emission overran the plan/); - assert.equal((dry as OverBudgetError).stageId, 'emission-overran-plan'); + assertStage(dry, 'emission-overran-plan'); assert.deepEqual(dry.clearedBy, []); const live = await refusal(manager); assert.match(live.message, /^Emission overran the plan/); - assert.equal((live as OverBudgetError).stageId, 'emission-overran-plan'); + assertStage(live, 'emission-overran-plan'); assert.deepEqual(live.clearedBy, ['compression'], 'the live compile demands the chunk first'); }); }); @@ -1255,7 +1256,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are waiting for compression'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); @@ -2044,7 +2045,7 @@ describe('OverBudgetError.clearedBy', () => { strategy.seedL1('L1-covering', ids); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // Tick drops each one without compressing it: the stub membrane has no // model to call, and no L1 is added. @@ -2100,7 +2101,7 @@ describe('OverBudgetError.clearedBy', () => { assert.ok(!strategy.summaryIds().some((id) => id.startsWith('L1-stored-')), 'no exact L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2120,7 +2121,7 @@ describe('OverBudgetError.clearedBy', () => { }); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, [], 'adoption doesn\'t persist, so the stored L1s shrink nothing'); }, { membrane: true }); }); @@ -2135,7 +2136,7 @@ describe('OverBudgetError.clearedBy', () => { assert.deepEqual(strategy.summaryIds(), [], 'no L1 is in memory'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); for (let i = 0; i < 20 && strategy.queuedChunkIds().length > 0; i++) await manager.tick(); assert.deepEqual(strategy.queuedChunkIds(), [], 'tick took every queued chunk'); @@ -2167,7 +2168,7 @@ describe('OverBudgetError.clearedBy', () => { queued.forEach((ids, i) => strategy.seedL1(`L1-exact-${i}`, ids)); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(strategy.queuedChunkIds(), queued, 'the compile closed no new chunk'); assert.deepEqual(error.clearedBy, persistChunks ? ['compression'] : []); if (!persistChunks) return; @@ -2464,7 +2465,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'the merge is queued'); const error = await refusal(manager, undefined, positionedRecallPairs ? undefined : PAIR_FITS); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.equal(Reflect.get(error.diagnostics, 'middleCountUnit'), positionedRecallPairs ? 'selection-items' : 'raw-messages', 'the pair, or with the combined pair fitting, the raw message after it, refuses'); assert.deepEqual(error.clearedBy, [], 'tick stops at a blocked merge until the hold is released'); @@ -2509,7 +2510,7 @@ describe('OverBudgetError.clearedBy', () => { assert.equal(strategy.checkReadiness().ready, false, 'chunks are queued'); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, merge === 'blocked' ? [] : ['compression']); }); }); @@ -2653,7 +2654,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: tick still takes them, adopting each stored L1. await adoptAll(manager, strategy); @@ -2671,7 +2672,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued); const error = await refusal(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, []); // The premise: adopting the exact L1s leaves the render as it was, where unpinned chunks' adoption shrinks it. const before = (await manager.compile({ maxTokens: 100_000, reserveForResponse: 20 })).messages.length; @@ -2796,7 +2797,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2811,7 +2812,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }, { membrane: true }); }); @@ -2825,7 +2826,7 @@ describe('OverBudgetError.clearedBy', () => { pinChunks(manager, queued.slice(0, 1)); const error = await refusalOfWholeMiddle(manager); assert.match(error.message, /^Hierarchical emission overran the budget/); - assert.equal((error as OverBudgetError).stageId, 'hierarchical-emission-overran'); + assertStage(error, 'hierarchical-emission-overran'); assert.deepEqual(error.clearedBy, ['compression']); }); }); diff --git a/test/system-changes.test.ts b/test/system-changes.test.ts index 496fbcf7..2ee82577 100644 --- a/test/system-changes.test.ts +++ b/test/system-changes.test.ts @@ -31,6 +31,7 @@ import { WindowedPassthroughStrategy, malformedContentPlaces, } from '../src/index.js'; +import { assertStage } from './fixtures/over-budget-hook.js'; import { jsonTokenEstimator } from '../src/message-store.js'; import { wellFormedContent } from '../src/well-formed-content.js'; import { systemChangePositions } from '../src/system-changes.js'; @@ -849,12 +850,12 @@ describe('the passthrough strategies and a system change before their window', ( assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); assert.deepEqual(err.clearedBy, [], `${label}: only consolidating clears it`); if (label === 'passthrough') assert.match(err.message, /^Passthrough kept only its system changes/, label); - if (label === 'passthrough') assert.equal(err.stageId, 'passthrough-system-changes', label); + if (label === 'passthrough') assertStage(err, 'passthrough-system-changes'); // Its window still holds the newest conversation message, which its numbers count. if (label === 'windowed-passthrough') assert.match(err.message, /^Windowed passthrough: its system changes and the newest conversation message/, label); - if (label === 'windowed-passthrough') assert.equal(err.stageId, 'windowed-changes-and-newest-message', label); + if (label === 'windowed-passthrough') assertStage(err, 'windowed-changes-and-newest-message'); if (label === 'hierarchical') assert.match(err.message, /system changes/, label); - if (label === 'hierarchical') assert.equal(err.stageId, 'hierarchical-system-changes', label); + if (label === 'hierarchical') assertStage(err, 'hierarchical-system-changes'); return true; }); } @@ -869,7 +870,7 @@ describe('the passthrough strategies and a system change before their window', ( await assert.rejects(alone.compile(budget), (err: unknown) => { assert.ok(err instanceof OverBudgetError, `${label}: ${String(err)}`); assert.match(err.message, stage, label); - assert.equal(err.stageId, stageId, label); + assertStage(err, stageId); return true; }); assert.deepEqual(rawIds(await alone.compile({ maxTokens: estimateOf(alone, id), reserveForResponse: 0 })), [id], label); @@ -1053,7 +1054,7 @@ describe('the passthrough strategies and a system change before their window', ( await assert.rejects(windowed.compile({ maxTokens: estimateOf(windowed, big2) - 1, reserveForResponse: 0 }), (err: unknown) => { assert.ok(err instanceof OverBudgetError, String(err)); assert.match(err.message, /^Windowed passthrough: the newest message alone/); - assert.equal(err.stageId, 'windowed-newest-message'); + assertStage(err, 'windowed-newest-message'); return true; }); // Passthrough's changes fit, but no conversation message fits beside them: @@ -1064,7 +1065,7 @@ describe('the passthrough strategies and a system change before their window', ( await assert.rejects(both.compile({ maxTokens: estimateOf(both, rule) + 10, reserveForResponse: 0 }), (err: unknown) => { assert.ok(err instanceof OverBudgetError, String(err)); assert.match(err.message, /^Passthrough: its system changes and the newest conversation message/); - assert.equal(err.stageId, 'passthrough-changes-and-newest-message'); + assertStage(err, 'passthrough-changes-and-newest-message'); assert.equal(err.diagnostics.tailTokens, estimateOf(both, newest)); return true; }); From 2c11a7c5917eea2c5371305448875700e6d5ba39 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Sat, 24 Oct 2026 19:49:15 -0400 Subject: [PATCH 65/67] fix(observer): native promises take the handler on themselves; the overlap guard's report documented as it runs - A native promise an observer callback returns, a subclass's included, now takes the rejection handler on itself through the intrinsic then, so a then of its own can't skip it; Promise.resolve adopts only what isn't one. The docs name the limit as a kind: a promise no JavaScript can give a handler (one whose constructor or species can't be used, or a proxy around one) is logged, and its rejection goes unhandled, which ends a Node process by default. - The overlap guard's doc: KnowledgeStrategy leaves a partial overlap in ordinary chunking, not only damaged bookkeeping; a chunk with a record isn't queued again while refused, one without is refused again each time it's queued; it's reported once per chunk until the strategy loads its state again, which also happens when it takes up what another instance sharing the store wrote or sets a branch's state up again. - ReadinessState and checkReadiness no longer say compile() waits for background work; _overlapBlocked's comment says what it holds. - Tests: a rejecting subclass promise with a then of its own; a proxy around a promise, logged, the compile unchanged; the overlap guard reports again after a state load and not for a chunk an L1 covers whole; the hook check pins each stage's words independently; the provider-refusal test names no longer credit membrane. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/context-observer.added.md | 7 +-- src/context-manager.ts | 6 +-- src/observer.ts | 72 +++++++++++++++++---------- src/strategies/autobiographical.ts | 2 +- src/types/strategy.ts | 7 +-- test/context-observer.test.ts | 59 +++++++++++++++++++--- test/fixtures/over-budget-hook.ts | 22 +++++--- 8 files changed, 125 insertions(+), 52 deletions(-) diff --git a/README.md b/README.md index 07b40161..abf12eb1 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ Chronicle keeps the store's history, so a conversation can branch. `branchAt(mes ## Observing a manager's work -Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. The one exception is a returned native promise whose `constructor` or species getter throws, which no JavaScript can handle: that throw is logged, and the promise's rejection goes unhandled. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. +Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. Outside that is a returned promise no JavaScript can give a handler, such as one whose `constructor` or species can't be used, or a proxy around one: what's thrown on the way is logged, and its rejection goes unhandled, which ends a Node process by default. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. ## Inspecting and repairing stores diff --git a/changelog.d/context-observer.added.md b/changelog.d/context-observer.added.md index 519a7301..ee73f2e2 100644 --- a/changelog.d/context-observer.added.md +++ b/changelog.d/context-observer.added.md @@ -5,9 +5,10 @@ called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics and drops the rest, so a compile's result is the same with an observer as - without one. The one exception is a returned native promise whose - `constructor` or species getter throws, which no JavaScript can handle: that - throw is logged, and the promise's rejection goes unhandled. A record's + without one. Outside that is a returned promise no JavaScript can give a + handler, such as one whose `constructor` or species can't be used, or a + proxy around one: what's thrown on the way is logged, and its rejection + goes unhandled, which ends a Node process by default. A record's error and render stats are the objects the caller gets, so an observer treats them as read-only. - `compile(record)`, once per `compile()`: the strategy, `dryRun`, diff --git a/src/context-manager.ts b/src/context-manager.ts index 5bf5c8fe..65860ca0 100644 --- a/src/context-manager.ts +++ b/src/context-manager.ts @@ -148,9 +148,9 @@ interface ContextManagerBaseConfig { * `ContextObserver`). The manager hands it to its strategy in * `StrategyContext`. It's called synchronously, and what it throws or * rejects never reaches the work: the manager logs a callback's first - * failure through its diagnostics and drops the rest. (A returned native - * promise whose `constructor` or species getter throws can't be handled - * from JavaScript; see `ContextObserver`.) A record's error and render + * failure through its diagnostics and drops the rest. (A returned promise + * no JavaScript can give a handler is outside that; see + * `ContextObserver`.) A record's error and render * stats are the objects the caller gets, so treat them as read-only. */ observer?: ContextObserver; diff --git a/src/observer.ts b/src/observer.ts index 6a09b7b0..36cf4ebc 100644 --- a/src/observer.ts +++ b/src/observer.ts @@ -48,11 +48,16 @@ export type CompileRefusal = typeof COMPILE_REFUSALS[number]; * ten-minute window allows (`compressionSplitMaxCallsPer10Min`), so it ended * before another call. * - `'overlap-guard'`: a chunk's messages are partly covered by an L1 that - * already exists, which only damaged bookkeeping or store surgery leaves, - * so the strategy refuses to compress it rather than mint a duplicate - * memory. It's refused on every attempt while the overlap stands, and - * reported once per chunk until the strategy loads its state again (an - * open, a branch switch), as its log line is. + * already exists, so the strategy refuses to compress it rather than mint + * a duplicate memory. In the autobiographical strategy's own chunking only + * damaged bookkeeping or store surgery leaves that; `KnowledgeStrategy` + * leaves it when a summary ends at a tool call whose result comes after + * it. A chunk with a record isn't queued again while it's refused, and + * one without is refused again each time it's queued. Either way it's + * reported once per chunk (its record id, or else its messages), as the + * guard's log line is, until the strategy loads its state again: an open, + * a branch switch, a branch's state set up again, or taking up what + * another instance sharing the store wrote. * * The rest are reported once per decision. A verdict on an answer the L1 ladder * keeps in its saved progress isn't decided again when the ladder resumes, @@ -140,12 +145,12 @@ export interface CalibrationSampleRecord { * Called synchronously as each fact happens, each callback optional. What * it throws, or a promise it returns that rejects, never reaches the work it * observes: the manager logs a callback's first failure through its - * diagnostics, and drops the rest. The one exception is a returned native - * promise whose `constructor` or species getter throws: no JavaScript can - * attach a handler to it, so that throw is logged and the promise's own - * rejection, if it rejects, goes unhandled. A record's error and render - * stats are the objects the caller gets, so an observer treats them as - * read-only. + * diagnostics, and drops the rest. Outside that is a returned promise no + * JavaScript can give a handler, such as one whose `constructor` or species + * can't be used, or a proxy around one: what's thrown on the way is logged, + * and the promise's own rejection goes unhandled, which ends a Node process + * by default. A record's error and render stats are the objects the caller + * gets, so an observer treats them as read-only. */ export interface ContextObserver { compile?(record: CompileRecord): void; @@ -155,9 +160,10 @@ export interface ContextObserver { /** * `observer` with each callback it has wrapped, so that nothing it throws or - * rejects reaches the caller. `failed` hears of a callback's first throw or - * rejection, once per callback, so a broken observer shows without a line per - * record. Each callback is read once, here, and that function is the one + * rejects reaches the caller, but for a returned promise no JavaScript can + * give a handler (see {@link ContextObserver}). `failed` hears of a + * callback's first throw or rejection, once per callback, so a broken + * observer shows without a line per record. Each callback is read once, here, and that function is the one * called, so replacing or removing it later changes nothing; one that can't * even be read is treated as absent, and told of the same way. */ @@ -188,20 +194,32 @@ export function isolateObserver( isolated[name] = (record: never) => { try { const returned: unknown = Reflect.apply(method, observer, [record]); - // Any object or function may be a thenable. Promise.resolve returns a - // native promise as it is, and adopts anything else with resolve - // functions of its own: it reads its `then` at once and calls it in a - // job of its own, so a `then` getter that throws, or a `then` that - // throws before it settles, rejects the promise handled here. The - // handler goes on through the intrinsic `then`, so a promise's own - // `then` can't skip it, and the fulfilment handler returns nothing, - // so nothing returned is read again. (A native promise whose own - // `constructor` getter throws can't be given a handler from - // JavaScript at all: every way reads it first. That throw is caught - // below and told, and if that promise rejects, its rejection is - // unhandled, as the host's `unhandledRejection` handling decides.) if ((typeof returned === 'object' && returned !== null) || typeof returned === 'function') { - Reflect.apply(Promise.prototype.then, Promise.resolve(returned), [() => undefined, (error: unknown) => fail(name, error)]); + // The fulfilment handler returns nothing, so nothing returned is + // read again. + const handlers = [() => undefined, (error: unknown) => fail(name, error)]; + let handled = false; + try { + // A native promise, a subclass's included, takes the handler on + // itself through the intrinsic `then`, so a `then` of its own + // can't skip it. The intrinsic `then` refuses anything else + // before reading it. + Reflect.apply(Promise.prototype.then, returned, handlers); + handled = true; + } catch { + // Not a native promise, or one whose constructor or species + // can't be used; the adoption below tells of the latter. + } + if (!handled) { + // Anything else may be a thenable. Promise.resolve reads its + // `then` at once and calls it in a job of its own, so a `then` + // getter that throws, or a `then` that throws before it settles, + // rejects the promise handled here. (A promise no JavaScript can + // give a handler, such as one whose constructor or species can't + // be used, or a proxy around one, is told of here or below, and + // its own rejection goes unhandled.) + Reflect.apply(Promise.prototype.then, Promise.resolve(returned), handlers); + } } } catch (error) { // An observer can't end a compile, a compression or a sample. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index e34ec920..c5a52a26 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -2007,7 +2007,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { */ protected chunkRecordsOrphaned = false; private _orphanWarned = false; - /** Record ids whose compression was refused by the L1 overlap guard. */ + /** Chunks the L1 overlap guard refused: a chunk's record id, or its message-id key (chunkKey) when it has none. */ private _overlapBlocked = new Set(); /** diff --git a/src/types/strategy.ts b/src/types/strategy.ts index 874c71c2..16b0fb48 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -145,7 +145,8 @@ export interface StrategyContext { * Result of readiness check. */ export interface ReadinessState { - /** Whether compile() can proceed immediately */ + /** Whether the strategy has no background work pending. compile() doesn't + * wait for that work either way: it compiles from what's there. */ ready: boolean; /** Promise that resolves when ready (if not ready) */ pendingWork?: Promise; @@ -255,8 +256,8 @@ export interface ContextStrategy { onCompressionHoldsReleased?(ctx: StrategyContext): void; /** - * Check if strategy is ready to compile. - * Returns pending work info if not ready. + * Whether the strategy has background work pending, with its info if so. + * compile() doesn't wait for it. */ checkReadiness(): ReadinessState; diff --git a/test/context-observer.test.ts b/test/context-observer.test.ts index 51ae9ac8..f7636cfa 100644 --- a/test/context-observer.test.ts +++ b/test/context-observer.test.ts @@ -2,9 +2,10 @@ * ContextManagerConfig.observer: what a host is told as it happens. Each * compile, refused and failed ones included; each refusal context-manager * itself makes of compression, and none of a provider's; each calibration - * sample. Nothing an observer throws or rejects reaches the work it observes, - * so a compile's result is the same with one, with none, and with one that - * throws or rejects, under each strategy. + * sample. Nothing an observer throws or rejects reaches the work it observes + * (but for a returned promise no JavaScript can give a handler), so a + * compile's result is the same with one, with none, and with one that throws + * or rejects, under each strategy. */ import { after, describe, it } from 'node:test'; @@ -384,6 +385,29 @@ describe('compile records', () => { assert.equal(lines.length, 4, lines.join(' | ')); }); + it('a proxy around a promise, which no JavaScript can give a handler, is told, and the compile goes on', async () => { + const lines: string[] = []; + const original = console.error; + console.error = (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }; + try { + let inner: Promise | undefined; + const manager = await ContextManager.open({ path: freshPath(), strategy: new PassthroughStrategy(), observer: { + compile: () => { inner = Promise.reject(new Error('the proxied promise rejected')); return new Proxy(inner, {}) as never; }, + } }); + manager.addMessage('User', [text('hello')]); + const pending = manager.compile(); + // The limit the docs name: nothing reaches the proxied promise, so the + // test handles it, as only the observer could. + inner!.catch(() => {}); + assert.equal((await pending).messages.length, 1, 'the compile is unchanged'); + for (let i = 0; i < 5; i++) await new Promise((resolve) => setImmediate(resolve)); + manager.close(); + } finally { + console.error = original; + } + assert.equal(lines.filter((line) => line.includes("the observer's compile callback threw or rejected")).length, 1, lines.join(' | ')); + }); + it('a log that throws as it tells of a failure changes nothing either', async () => { const original = console.error; let entered = 0; @@ -418,6 +442,7 @@ describe('compile records', () => { 'a native promise with a no-op then of its own': () => Object.assign(Promise.reject(new Error('own-then promise rejected')), { then() {} }), 'a native promise with a then of its own that isn\'t callable': () => Object.assign(Promise.reject(new Error('uncallable-then promise rejected')), { then: 42 }), 'a resolved native promise whose constructor getter throws': () => Object.defineProperty(Promise.resolve(), 'constructor', { get(): never { throw new Error('constructor getter threw'); } }), + 'a rejecting subclass promise with a then of its own': () => { class Sub extends Promise {} return Object.assign(Sub.reject(new Error('subclass promise rejected')), { then: undefined }); }, 'a thenable that fulfils': () => fulfilling, 'an object whose then getter throws only when read again': () => { let reads = 0; return { get then() { if (reads++ > 0) throw new Error('read again'); return undefined; } }; }, }; @@ -437,7 +462,7 @@ describe('compile records', () => { } assert.deepEqual(stray, []); const told = lines.filter((line) => line.includes("the observer's compile callback threw or rejected")); - const expected = ['then threw', 'then getter threw', 'callable rejected', 'own-then promise rejected', 'uncallable-then promise rejected', 'constructor getter threw']; + const expected = ['then threw', 'then getter threw', 'callable rejected', 'own-then promise rejected', 'uncallable-then promise rejected', 'constructor getter threw', 'subclass promise rejected']; assert.equal(told.length, expected.length, `each failing shape told once, the others not: ${told.join(' | ')}`); expected.forEach((words, i) => assert.ok(told[i]!.includes(words), `${words}: ${told[i]}`)); @@ -740,7 +765,7 @@ async function ladder(membrane: unknown, options: Record, path const HOIST = { compressionToolProseFallback: { intoTool: 'journal', fromTools: ['skip_reply'] } }; describe('compression refusals: an L1 ladder', () => { - it('a provider\'s refusal is membrane\'s to report, not one of these', async () => { + it('a provider\'s refusal is the provider\'s decision, not one of these', async () => { const { calls, membrane } = scripted([REFUSAL()]); const fx = await ladder(membrane, {}); await fx.strategy.run(fx.target(), strategyContext(fx.manager)); @@ -749,7 +774,7 @@ describe('compression refusals: an L1 ladder', () => { fx.manager.close(); }); - it('a provider\'s rejection of the request is membrane\'s to report too', async () => { + it('a provider\'s rejection of the request is the provider\'s decision too', async () => { const rejected = Object.assign(new Error('invalid request'), { type: 'invalid_request', httpStatus: 400, retryable: false }); const { calls, membrane } = scripted([{ error: rejected }]); const fx = await ladder(membrane, {}); @@ -853,10 +878,12 @@ describe('compression refusals: an L1 ladder', () => { fx.manager.close(); }); - it('a chunk an existing L1 partly covers is refused by the overlap guard, reported once', async () => { + it('a chunk an existing L1 partly covers is refused by the overlap guard, reported once until the strategy loads its state again', async () => { + // The chunk has no record, so the guard keys it by its messages. const { calls, membrane } = scripted([ANSWER('never asked')]); const fx = await ladder(membrane, SPLIT); const chunk = fx.target(); + assert.equal(chunk.recordId, undefined); const first = chunk.messages[0]!.id; fx.strategy.seed({ id: 'L1-200', level: 1, content: 'covers raw-10 alone', tokens: 20, sourceLevel: 0, sourceIds: [first], sourceRange: { first, last: first }, created: 200 }); await fx.strategy.run(chunk, strategyContext(fx.manager)); @@ -864,6 +891,24 @@ describe('compression refusals: an L1 ladder', () => { assert.equal(calls.length, 0, 'nothing is asked'); assert.equal(chunk.compressed, false); assert.deepEqual(fx.rec.compressions, [{ reason: 'overlap-guard' }], 'refused twice, reported once'); + (fx.strategy as unknown as { loadPersistedState(): void }).loadPersistedState(); + fx.strategy.seed({ id: 'L1-201', level: 1, content: 'covers raw-10 alone', tokens: 20, sourceLevel: 0, sourceIds: [first], sourceRange: { first, last: first }, created: 201 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.deepEqual(fx.rec.compressions, [{ reason: 'overlap-guard' }, { reason: 'overlap-guard' }], 'reported again after the load'); + fx.manager.close(); + }); + + it('a chunk an existing L1 covers whole is dropped, with no report', async () => { + const { calls, membrane } = scripted([ANSWER('never asked')]); + const fx = await ladder(membrane, SPLIT); + const chunk = fx.target(); + const ids = chunk.messages.map((m) => m.id); + fx.strategy.seed({ id: 'L1-200', level: 1, content: 'covers the whole chunk', tokens: 20, sourceLevel: 0, sourceIds: ids, sourceRange: { first: ids[0]!, last: ids.at(-1)! }, created: 200 }); + await fx.strategy.run(chunk, strategyContext(fx.manager)); + assert.equal(calls.length, 0); + assert.equal(chunk.compressed, true, 'dropped as covered'); + assert.deepEqual(fx.rec.compressions, []); fx.manager.close(); }); diff --git a/test/fixtures/over-budget-hook.ts b/test/fixtures/over-budget-hook.ts index ffcf288c..774c2033 100644 --- a/test/fixtures/over-budget-hook.ts +++ b/test/fixtures/over-budget-hook.ts @@ -9,6 +9,16 @@ import { AutobiographicalStrategy, OverBudgetError } from '../../src/index.js'; type HookOptions = { stage?: string; stageId?: string }; +/** The words each of AutobiographicalStrategy's stages gave the hook before stageId, as `main` gave them. */ +const HOOK_WORDS: Record = { + 'head-tail-reservation': 'Head/tail reservation exceeded budget', + 'emission-overran-plan': 'Emission overran the plan (planner/emitter estimator drift)', + 'tail-emission-dropped': 'Tail emission dropped reserved recent-window messages', + 'repair-lost-newest-turn': 'Structural repair did not retain the newest turn', + 'hierarchical-emission-overran': 'Hierarchical emission overran the budget', + 'hierarchical-system-changes': 'Hierarchical: its system changes alone exceed the budget', +}; + const given = new WeakMap(); const proto = AutobiographicalStrategy.prototype as unknown as { overBudgetError(budget: unknown, opts: HookOptions): OverBudgetError; @@ -23,9 +33,9 @@ proto.overBudgetError = function (this: unknown, budget: unknown, opts: HookOpti /** * `error` is an `OverBudgetError` naming `stageId`. A refusal of * `AutobiographicalStrategy`'s (any but passthrough's and windowed - * passthrough's) came through the hook, which was given that id and the - * words that open its message, or, for the picker's own refusal, neither, as - * before. + * passthrough's) came through the hook, which was given that id and exactly + * the words `main` gave it, which open its message, or, for the picker's own + * refusal, neither, as before. */ export function assertStage(error: unknown, stageId: string): void { assert.ok(error instanceof OverBudgetError, String(error)); @@ -38,8 +48,6 @@ export function assertStage(error: unknown, stageId: string): void { return; } assert.equal(hook.stageId, stageId); - assert.ok( - typeof hook.stage === 'string' && error.message.startsWith(`${hook.stage} but `), - `the hook was given the words that open the message: ${String(hook.stage)} / ${error.message.slice(0, 80)}`, - ); + assert.equal(hook.stage, HOOK_WORDS[stageId], `${stageId}: the hook was given the words main gave it`); + assert.ok(error.message.startsWith(`${hook.stage} but `), `the words open the message: ${error.message.slice(0, 80)}`); } From 55033a619a8e9f461a3bad2c59cd01c0eb8ae443 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Sat, 24 Oct 2026 20:05:21 -0400 Subject: [PATCH 66/67] docs(observer): the isolation limit says only what holds for every such promise A returned promise no JavaScript can give a handler keeps its own rejection unhandled; the public docs (ContextObserver, the README, the changelog) no longer say what it throws on the way is logged, since for one shape nothing is logged: a promise whose species can't be used and whose own then is undefined is adopted as a plain value. The code comment names that shape. Comments and docs only. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- changelog.d/context-observer.added.md | 4 ++-- src/observer.ts | 14 ++++++++------ 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index abf12eb1..ff9d1470 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ Chronicle keeps the store's history, so a conversation can branch. `branchAt(mes ## Observing a manager's work -Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. Outside that is a returned promise no JavaScript can give a handler, such as one whose `constructor` or species can't be used, or a proxy around one: what's thrown on the way is logged, and its rejection goes unhandled, which ends a Node process by default. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. +Pass `observer` to `ContextManager.open` to be told, as it happens, of each compile (`compile`: compiled, refused or failed, its whole duration, the thrown error, and the compile's render stats), of each refusal context-manager itself makes of compression (`compression`: by a reason from `COMPRESSION_REFUSALS`), and of each calibration sample (`calibration`: by an outcome from `CALIBRATION_SAMPLE_OUTCOMES`). The manager hands the observer to its strategy too. Each callback is optional and called synchronously. What it throws or rejects never reaches the work: the manager logs a callback's first failure once through its diagnostics, under its namespace, and drops the rest, so a compile's result is the same with an observer as without one. Outside that is a returned promise no JavaScript can give a handler, such as one whose `constructor` or species can't be used, or a proxy around one: its rejection goes unhandled, which ends a Node process by default. A record's error and render stats are the objects the caller gets, so treat them as read-only. A refused compile's `OverBudgetError` names its stage in `stageId`, from `OVER_BUDGET_STAGES`. A record can hold text from the store, such as a refusal's message, which names pins, so choose what you keep or send from it. ## Inspecting and repairing stores diff --git a/changelog.d/context-observer.added.md b/changelog.d/context-observer.added.md index ee73f2e2..30df77e6 100644 --- a/changelog.d/context-observer.added.md +++ b/changelog.d/context-observer.added.md @@ -7,8 +7,8 @@ drops the rest, so a compile's result is the same with an observer as without one. Outside that is a returned promise no JavaScript can give a handler, such as one whose `constructor` or species can't be used, or a - proxy around one: what's thrown on the way is logged, and its rejection - goes unhandled, which ends a Node process by default. A record's + proxy around one: its rejection goes unhandled, which ends a Node process + by default. A record's error and render stats are the objects the caller gets, so an observer treats them as read-only. - `compile(record)`, once per `compile()`: the strategy, `dryRun`, diff --git a/src/observer.ts b/src/observer.ts index 36cf4ebc..1aba516b 100644 --- a/src/observer.ts +++ b/src/observer.ts @@ -147,10 +147,9 @@ export interface CalibrationSampleRecord { * observes: the manager logs a callback's first failure through its * diagnostics, and drops the rest. Outside that is a returned promise no * JavaScript can give a handler, such as one whose `constructor` or species - * can't be used, or a proxy around one: what's thrown on the way is logged, - * and the promise's own rejection goes unhandled, which ends a Node process - * by default. A record's error and render stats are the objects the caller - * gets, so an observer treats them as read-only. + * can't be used, or a proxy around one: its rejection goes unhandled, which + * ends a Node process by default. A record's error and render stats are the + * objects the caller gets, so an observer treats them as read-only. */ export interface ContextObserver { compile?(record: CompileRecord): void; @@ -216,8 +215,11 @@ export function isolateObserver( // getter that throws, or a `then` that throws before it settles, // rejects the promise handled here. (A promise no JavaScript can // give a handler, such as one whose constructor or species can't - // be used, or a proxy around one, is told of here or below, and - // its own rejection goes unhandled.) + // be used, or a proxy around one, keeps its own rejection + // unhandled. What it throws is told where the adoption or the + // catch below meets it, if either does: one whose species can't + // be used and whose own `then` is undefined is adopted as a plain + // value, and nothing is told.) Reflect.apply(Promise.prototype.then, Promise.resolve(returned), handlers); } } From 57643cbe0e4b93f59e095a155085d954003c3e43 Mon Sep 17 00:00:00 2001 From: Esther-2584 Date: Sat, 24 Oct 2026 21:34:27 -0400 Subject: [PATCH 67/67] docs(calibration): the exceptions to learned and provisional, and what a declared caller paid - RenderStats.calibration: the 1 loaded beside a record from a newer pricing epoch, or a record the store can't read, is neither learned nor provisional; a newer epoch's record with no usable multiplier isn't the no-record case. - The fragment for anima-research/context-manager#328: a learned 1 took nine applied changes per undeclared caller, and two samples, the first held, for a declared one. - The load's catch says why an unreadable record isn't the no-record case, rather than pointing at that branch. Comments, docs and the changelog only. Co-Authored-By: Claude Opus 5.5 --- changelog.d/328-fresh-calibration-provisional.fixed.md | 7 ++++--- src/strategies/autobiographical.ts | 2 +- src/types/strategy.ts | 7 +++++-- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/changelog.d/328-fresh-calibration-provisional.fixed.md b/changelog.d/328-fresh-calibration-provisional.fixed.md index 1c7c4836..55485a4b 100644 --- a/changelog.d/328-fresh-calibration-provisional.fixed.md +++ b/changelog.d/328-fresh-calibration-provisional.fixed.md @@ -1,9 +1,10 @@ - A store with no calibration record, or a record with no finite multiplier, starts from a provisional 1, as a discard's restart does (#328). Its first sample inside the calibration band sets the multiplier, - declared or not: one sample, without the two-sample hold, where it - stepped a fifth of the way per sample (nine applied changes, each a - reprice of every message and summary, to reach a true 1.4). Until that + declared or not: one sample, without the two-sample hold. Learned, a 1 + stepped a fifth of the way per undeclared sample (nine applied changes, + each a reprice of every message and summary, to reach a true 1.4), and + took two declared samples, the first held. Until that sample, render stats report `calibration.provisional: true`. A record from a newer pricing epoch, or a read of the record that throws, still loads 1 as before, not provisional. diff --git a/src/strategies/autobiographical.ts b/src/strategies/autobiographical.ts index 537c5055..d02ba5d1 100644 --- a/src/strategies/autobiographical.ts +++ b/src/strategies/autobiographical.ts @@ -14942,7 +14942,7 @@ export class AutobiographicalStrategy implements ResettableStrategy { // write over even when it only confirms the 1. this._calibrationProvisional = { fromEpoch: null }; } - } catch { /* a record the store can't read loads 1, not provisional: see the no-record branch above */ } + } catch { /* a record the store can't read loads 1, not provisional: unlike no record, it may hold what the store keeps (see the no-record branch above) */ } } // A dry select changes the store's multiplier only where select() read it // first and can put it back (see MessageStoreView). diff --git a/src/types/strategy.ts b/src/types/strategy.ts index cf3ae88e..b6b9b8c7 100644 --- a/src/types/strategy.ts +++ b/src/types/strategy.ts @@ -900,13 +900,16 @@ export interface RenderStats { * that isn't provisional was learned under that epoch's pricing, or under * an older epoch's and kept, re-stamped, because nothing this store holds * was repriced since; either way it's reported under the epoch in force. + * The exception is the 1 loaded beside a record from a newer pricing + * epoch, or a record the store can't read: neither is used, and that 1 + * is neither learned nor provisional. * `provisional` is true while the multiplier is a guess, which the first * provider sample the calibration's band accepts replaces outright, rather * than stepping toward it (see * `AutobiographicalStrategy.CALIBRATION_PRICING_EPOCH`): one kept from a * record without a stamp, the 1 a discard restarts from, or the 1 a store - * with no calibration record, or none with a multiplier to use, starts - * from. A discard's 1 stays + * with no calibration record, or none with a multiplier to use other than + * a newer epoch's, starts from. A discard's 1 stays * provisional through a later epoch's load, which re-stamps it whatever * that epoch repriced. * `reset` is present once a multiplier from an older or unknown epoch has