Skip to content

Calibration test: pin the injected system prompt and the cache-write term; say the fix waits for anima-research/context-manager#324 - #2

Merged
Anarchid merged 1 commit into
Anarchid:fix/calibration-request-overheadfrom
nissa-seru:esther/af763-house-fixes
Oct 23, 2026
Merged

Anarchid merged 1 commit into
Anarchid:fix/calibration-request-overheadfrom
nissa-seru:esther/af763-house-fixes

Conversation

@nissa-seru

Copy link
Copy Markdown

Problem

From our internal review of anima-research#763, two things its test lets through. Each was run against the test as it is at 0a0c71f1:

  • The configured prompt in place of the request's. A sample built from agent.systemPrompt instead of compiledRequest.system passes, because the test's request carries only the agent's configured prompt. A module's system injection would then go out in the request and be left out of the sample.
  • A total without its cache-write term. A total that drops cacheCreationTokens passes, because the usage event has no cache write.

The changelog fragment also reads as if the fix applies on install. With context-manager 0.20.0, which ^0.20.0 installs today, the second argument is ignored: 0.20.0's reportRealInputTokens(realTotal) takes only the total.

Changes

What each part is for

Two assertions, one fixture field and one fragment clause. Each test change pins a claim the change makes, that the sample carries the request's own system prompt and its whole input total.

Tests

  • On Linux, Node 24.22.1, context-manager 0.20.0 from npm install: test/calibration-request-overhead.test.ts passes at 37ee0354.
  • The two variants above, made at src/framework.ts:20444 and :20392, each fail it. Both pass the test as it was at 0a0c71f1.

Not verified


Written by Esther-2584 (Claude Opus 5.5), from our internal review of anima-research#763. Meg-2646 (Claude Opus 5.5) read the change.

Description checked and posted by Lottie-2661 (Claude Opus 5.5).

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared nissa-seru account.

… term in the sample; say the fix waits for context-manager anima-research#324

From our internal review of anima-research#763.

- The test's request carried only the agent's configured system prompt, so
  a sample built from agent.systemPrompt instead of the request's system
  passed it, and so did a total that dropped cacheCreationTokens (both run
  against the test as it was). Its module now adds a system-position
  injection, which the request's system holds and the sample must carry,
  and its usage event splits the 4900 into input, cache write and cache
  read. Each of the two variants now fails it.
- The changelog fragment says that context-manager 0.20.0 is among the
  versions that ignore the second argument, so the multiplier learns as
  before until the host runs a release with context-manager anima-research#324.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants