Calibration: price message-position injections into the sample, move a store that learned the overhead as a rate in two samples, and start a store with no record provisional

changelog.d/323-calibration-request-overhead.fixed.md

@@ -5,16 +5,34 @@
   estimate doesn't, so a tool-heavy agent's multiplier learned its schemas
   as a per-token rate and folded history the budget had room for, or sat at
   1.00 rejecting every sample as out of band. A caller that passes nothing
-  gets the old comparison.
+  gets the old comparison, plus the compile's message-position injections
+  (below).
 - With the overhead declared, three out-of-band calibration samples in a row
-  whose implied multipliers agree within 10% are learned, outright, at the
-  clamp edge, instead of being rejected for good. At 1.00 the ratio and the
-  implied multiplier are the same number, so a store whose true ratio was
-  past 1.8, where every pricing-epoch discard restarts, could never learn.
-  Undeclared samples are rejected as before, and the rejection line says the
-  caller may be leaving out the system prompt and tools it sends.
+  whose implied multipliers agree within 10% of the first are learned,
+  outright, at the clamp edge, instead of being rejected for good. At 1.00
+  the ratio and the implied multiplier are the same number, so a store whose
+  true ratio was past 1.8, where every pricing-epoch discard restarts, could
+  never learn. Undeclared samples are rejected as before, and the rejection
+  line says the caller may be leaving out the system prompt and tools it
+  sends.
+- The sample also counts the compile's message-position injections
+  (`beforeUser` and `afterUser`), which the request carries and the window
+  estimate doesn't: `ContextManager.compile` passes their content to the
+  select as `SelectOptions.messageInjections`, and the select prices it with
+  the window. Left out, an agent whose injections are large beside its window
+  learned them as a per-token rate, or, with the overhead declared, at the
+  clamp.
 - **Host operators:** a multiplier learned before the caller declared its
-  overhead holds that overhead, and steps back down over about ten compiles
-  once samples declare it. To relearn in one sample, seed it as provisional
-  on the stopped store: `setStateJson('<ns>/autobio:calibration',
-  { multiplier, at, pricing, provisional: true })`.
+  overhead holds that overhead. Inside the calibration band, the first two
+  samples that declare it replace the multiplier with their mean when they
+  agree within 10%, and the EMA steps as before when they don't; either way
+  the record then carries `declaredOverhead: true`, so later declared
+  samples step by the EMA. (Out of the band, the three-sample run above
+  applies.) The first of the two is held in memory, so a restart between
+  them costs one more sample. An undeclared sample that moves the multiplier
+  writes the record without the flag. A store whose multiplier, above 1,
+  puts every compile over budget gets no sample to move it, as before: to
+  unwedge it, seed the record as provisional on the stopped store, and the
+  refused compile drops it to 1: `setStateJson('<ns>/autobio:calibration',
+  { multiplier, at, pricing: <current epoch>, provisional: true })`.
+  `RequestOverhead` is exported from the package.

changelog.d/328-fresh-calibration-provisional.fixed.md

@@ -0,0 +1,10 @@
+- A store with no calibration record, or a record with no finite
+  multiplier, starts from a provisional 1, as a discard's restart does
+  (#328). Its first sample inside the calibration band sets the multiplier,
+  declared or not: one sample, without the two-sample hold. Learned, a 1
+  stepped a fifth of the way per undeclared sample (nine applied changes,
+  each a reprice of every message and summary, to reach a true 1.4), and
+  took two declared samples, the first held. Until that
+  sample, render stats report `calibration.provisional: true`. A record
+  from a newer pricing epoch, or a read of the record that throws, still
+  loads 1 as before, not provisional.

src/context-manager.ts

@@ -1179,11 +1179,14 @@ export class ContextManager {
     const view = this.strategyMessageView();
 
     // Get selected entries from strategy
+    const messageInjections = (injections ?? [])
+      .filter((inj) => inj.position === 'beforeUser' || inj.position === 'afterUser')
+      .map((inj) => inj.content);
     const entries = this.strategy.select(
       view,
       this.contextLog.createView(),
       effectiveBudget,
-      opts
+      messageInjections.length > 0 ? { ...opts, messageInjections } : opts
     );
     const selectionCause = this.strategy.takeSelectionCause?.();
     // Read now, before anything else selects: these are this select's own.

src/index.ts

@@ -81,6 +81,7 @@ export type {
   RenderedLayout,
   LayoutUnit,
   // Strategy types
+  RequestOverhead,
   MessageStoreView,
   ContextLogView,
   StrategyContext,