fix(adapters,server): output budgets, adapter queue memory, and per-key model policy

devlog/_plan/260920_meaning_preservation_batch/050_lane_e.md

@@ -0,0 +1,128 @@
+# Lane E — output budgets, queue memory, per-key policy
+
+Status: OPEN. Branch `codex/260920-lane-e-budgets-key-policy` against `dev`, one pull request.
+Covers phase 2 bundles 10, 11 and 12 from [010_phase2.md](010_phase2.md).
+
+## What each bundle turned out to be
+
+### 10 — Devin output budget and the history ceiling
+
+Two defects, not one. The adapter forwarded only a caller-supplied
+`max_output_tokens`, and Codex never sends one, so every `devin/*` turn was capped at
+the cloud-direct encoder's 8192 fallback however the provider was configured. The
+escape hatch was closed too: no OAuth preset declares `defaultMaxOutputTokens` or
+`modelMaxOutputTokens`, so the delete-when-preset-undefined branch in
+`applyOAuthPresetCatalog` was the only branch either field ever took and a
+hand-edited value was gone before the next startup finished. Both had to move, or
+wiring the adapter alone would have been unreachable in practice.
+
+The resolver reads the caller's explicit value, then the configured per-model cap,
+then the provider default, then nothing — leaving the encoder fallback. It never
+reads `contextWindow` or `modelContextWindows`: CompletionConfiguration #2 is the
+output cap and #3 is the context window, and collapsing them would ask Cognition to
+generate a whole window of output.
+
+The history ceiling is the other half and stays a separate quantity. #5189 is
+carried with attribution: it derives the coding-agent projected-history bound from
+the declared context window in characters. That bounds replayed history memory;
+nothing there decides how long a reply may run.
+
+No retry change was needed. Source review of `stated-reset-retry.ts` and
+`upstream-retry.ts` confirms an upstream `incomplete / max_output_tokens` is a
+successful streaming response that has already emitted events, so it matches none
+of the replay conditions. The repeated identical attempts in #5190 are the client's.
+
+### 11 — adapter event queue memory
+
+PR #5182 had the right idea and the wrong number. Its 1 MiB aggregate default
+aborts a legitimate turn: a synchronous producer fills the queue before its
+consumer is scheduled, and the image loop does exactly that with over a million
+one-character deltas that coalesce into roughly 1.2 MB of retained text. Its own CI
+proved it, which is why it sits at `CHANGES_REQUESTED`.
+
+The work is carried with attribution and reshaped around two budgets rather than
+one, because a stalled consumer and a malformed event are different failures and an
+operator reading the terminal error should learn which happened. Accounting is now
+exact by construction: each queued item records what it was charged, so a merge
+pays only for appended text, a refused event is priced before anything is retained
+and never charged, and the terminal record explaining a refusal is admitted past
+the budget it reports but still charged and released. `retainedCodeUnits()` exposes
+the counter so the regressions assert it reaches zero rather than inferring it from
+an abort that happened to fire.
+
+Retention is measured by a bounded walk of own enumerable properties rather than a
+per-variant table. A table would be exhaustive over `AdapterEvent`, which is the
+union class `AGENTS.md` records: a member added on another branch would silently
+stop being counted.
+
+### 12 — per-admission-key model and provider scope
+
+The security question is where the check goes, not what it compares. A scope
+evaluated against the client's string authorizes one destination and reaches
+another, because alias resolution, policy and combo selection, subagent fallback
+and compaction override all rewrite that string. So the scope names destinations
+and is applied to the resolved route.
+
+On the Responses path every route produced by the request — direct name, alias,
+policy, combo child, shadow-intercept target and both subagent-fallback re-routes —
+passes through one capture point, which is where the check sits. Chat and Messages
+translate into that path; their native lanes and the compaction route send without
+re-entering it, so each applies the same predicate itself. `/v1/models` filters by
+the same predicate, and that filter is explicitly not the boundary.
+
+A malformed scope drops the key rather than degrading to `undefined` like every
+other field on the record, because degrading a permission field reads as "allowed
+everything".
+
+Out of scope and deliberately not started: Redis, a full multi-tenant conversion,
+and any budget or RPM/TPM system.
+
+#### What the scope does not cover, stated rather than implied
+
+An adversarial review of the branch found authenticated data-plane endpoints that
+spend provider quota without resolving a model through the router, so the scope
+does not reach them:
+
+- `/v1/images/generations` and `/v1/images/edits`,
+- `/v1/audio/transcriptions` and its streaming form,
+- `/v1/live`, `/v1/realtime/calls` and the standalone realtime sockets,
+- the non-account-qualified branch of `/v1/alpha/search`, which forwards the caller's
+  model to a search sidecar without routing it.
+
+The account-qualified search branch does route a model and is checked. The rest
+need a destination definition this lane does not own — an image or audio endpoint
+has a fixed-purpose model rather than a routed one — and inventing one here would
+be the multi-tenant expansion this batch rules out. They are recorded so the
+contract is not read as broader than it is.
+
+The review also found that resolving an OpenAI virtual model rewrites
+`route.modelId` to the wire id after the initial check. That one was a real hole in
+the stated contract and is fixed: the settled route is re-checked after
+normalization, so the id that is billed is the id that was authorized.
+
+## Verification
+
+Static source review plus exact-head hosted CI. No local suite, individual test,
+typecheck, build, install or live `ocx` execution was run — those are NOT RUN, not
+passing.
+
+Union-defect classes checked before pushing. No file in the touched set carries a
+`file-size-baseline.json` cap; the largest, `src/oauth/index.ts` and
+`src/server/index/serve-options.ts`, stay under the 2000-line new-file threshold.
+The two new test files are registered in both `scripts/test-layout/layout.json` and
+`tests/fixtures/test-layout-expected.json`. Nothing here restates a count or
+enumerates a union.
+
+## Ownership
+
+Lane E owns the adapter event queue budget and the Devin and coding-agent limits.
+Retry classification — `sendCount`, the send budget, the stage and cause vocabulary —
+is lane C's and is untouched.
+
+## Carried work
+
+- #5182 (luvs01) — adapter event queue backlog budget.
+- #5189 (mdwsk88) — coding-agent projected-history ceiling.
+
+Both carry a `Co-authored-by` trailer in the branch commit. Neither original pull
+request is closed here; the coordinator handles that after this lane lands.

scripts/test-layout/layout.json

@@ -1566,7 +1566,9 @@
     "web-search-sidecar-429.test.ts": "web-search",
     "management-google-tool-schema-policy.test.ts": "server",
     "codex-shim-destroyed-probe.test.ts": "codex-integration",
-    "client-runtime.test.ts": "clients"
+    "client-runtime.test.ts": "clients",
+    "devin-output-budget.test.ts": "providers",
+    "api-key-model-scope.test.ts": "server"
   },
   "migrated": [
     "adapters",

src/adapters/coding-agent/protocol.ts

@@ -21,6 +21,30 @@ export const MAX_STREAM_LINE_BYTES = 8 * 1024 * 1024;
 export const MAX_STREAM_TOTAL_BYTES = 64 * 1024 * 1024;
 /** Hard ceiling on projected conversation history text (characters) to prevent runaway memory. */
 export const MAX_PROJECTED_HISTORY_CHARS = 200_000;
+/**
+ * Chars-per-token ratio for deriving the projected-history ceiling from the model context
+ * window. Sits between the English/code (~4 chars per token) and CJK (~1.5) extremes: the
+ * ceiling is a runaway-memory bound and a coarse guard against cutting history the window can
+ * hold, not a token accounting - the caller-side compaction line stays the token authority.
+ */
+const PROJECTED_HISTORY_CHARS_PER_TOKEN = 3;
+/** Absolute ceiling on a window-derived history cap, so runaway metadata cannot unbound stdin. */
+const MAX_PROJECTED_HISTORY_DERIVED_CHARS = 4_000_000;
+
+/**
+ * Projected-history character ceiling for a turn, derived from the declared model context
+ * window. A missing or non-finite window keeps the legacy flat cap, and the derivation never
+ * lowers the cap below it: small windows change nothing, while large windows scale (a 1M-token
+ * model keeps 3M characters) until the hard ceiling. The flat 200k cap predates window
+ * metadata and cut long replays to roughly 50k-130k tokens of content regardless of the model.
+ */
+export function projectedHistoryCharLimit(contextWindowTokens: number | undefined): number {
+  if (typeof contextWindowTokens !== "number" || !Number.isFinite(contextWindowTokens) || contextWindowTokens <= 0) {
+    return MAX_PROJECTED_HISTORY_CHARS;
+  }
+  const derived = contextWindowTokens * PROJECTED_HISTORY_CHARS_PER_TOKEN;
+  return Math.min(Math.max(derived, MAX_PROJECTED_HISTORY_CHARS), MAX_PROJECTED_HISTORY_DERIVED_CHARS);
+}
 
 export class CodingAgentStreamLimitError extends Error {
   constructor(message: string) {
@@ -395,7 +419,7 @@ export function buildSystemPrompt(parsed: OcxParsedRequest): string | undefined
  * prior conversation turns are structured as bounded context text with tool results as text,
  * clearly demarcated from the current user request. Codex retains tool control; vendor tools are never invoked.
  */
-export function buildConversationInput(parsed: OcxParsedRequest): string[] {
+export function buildConversationInput(parsed: OcxParsedRequest, options: { maxHistoryChars?: number } = {}): string[] {
   const nonDev = parsed.context.messages.filter(m => m.role !== "developer");
   if (nonDev.length === 0) {
     return [JSON.stringify({ type: "user", message: { role: "user", content: [{ type: "text", text: "" }] } })];
@@ -475,10 +499,11 @@ export function buildConversationInput(parsed: OcxParsedRequest): string[] {
 
   const imageBlocks: WireContentPart[] = [...historyImageBlocks, ...currentImageBlocks];
 
+  const maxHistoryChars = options.maxHistoryChars ?? MAX_PROJECTED_HISTORY_CHARS;
   let historyText = historyMessages.map(formatMessageForHistory).filter(Boolean).join("\n\n");
-  if (historyText.length > MAX_PROJECTED_HISTORY_CHARS) {
+  if (historyText.length > maxHistoryChars) {
     historyText = `[Earlier conversation history truncated for length...]\n\n` +
-      historyText.slice(historyText.length - MAX_PROJECTED_HISTORY_CHARS);
+      historyText.slice(historyText.length - maxHistoryChars);
   }
 
   const combinedText = `Prior conversation context:\n\n${historyText}\n\nCurrent user request:\n\n${currentRequestText}`;

src/adapters/coding-agent/turn.ts

@@ -1,8 +1,9 @@
 import { execFileSync, spawn as nodeSpawn, type ChildProcess, type SpawnOptions } from "node:child_process";
 import type { AdapterEvent, OcxParsedRequest, OcxProviderConfig } from "../../types";
 import { commandInvocation } from "../../lib/win-exec";
+import { modelRecordValue } from "../../reasoning-effort";
 import type { IncomingMeta } from "../base";
-import { buildConversationInput, CodingAgentProtocolError, mapStreamMessageToEvents, readJsonLines, type StreamParseState } from "./protocol";
+import { buildConversationInput, CodingAgentProtocolError, mapStreamMessageToEvents, projectedHistoryCharLimit, readJsonLines, type StreamParseState } from "./protocol";
 import { resolveCodingAgentBinary, resolveProfileByBaseUrl, type CodingAgentProviderProfile, type WhichFn } from "./profile";
 
 /** Injectable spawn for tests; production uses node:child_process. */
@@ -264,7 +265,14 @@ export async function runCodingAgentTurn(input: CodingAgentTurnInput): Promise<v
     const stdin = child.stdin;
     if (stdin) {
       stdin.on("error", () => { /* EPIPE if the CLI exits early; surfaced via close/stderr */ });
-      for (const line of buildConversationInput(parsed)) stdin.write(`${line}\n`);
+      // The projected history scales with the model context window on the routed provider row
+      // (catalog and config metadata merged): a 1M-token model keeps 3M characters of replay
+      // where the flat cap cut it near 50k-130k tokens of content. Absent metadata keeps the
+      // flat cap.
+      const historyCharLimit = projectedHistoryCharLimit(
+        modelRecordValue(provider.modelContextWindows, parsed.modelId) ?? provider.contextWindow,
+      );
+      for (const line of buildConversationInput(parsed, { maxHistoryChars: historyCharLimit })) stdin.write(`${line}\n`);
       stdin.end();
     }
     const stdout = child.stdout;

src/adapters/devin.ts

@@ -192,6 +192,35 @@ async function resolveWireModelUid(
  */
 export const resolveWireModelUidForTests = resolveWireModelUid;
 
+const positiveTokenCount = (value: unknown): number | undefined =>
+  typeof value === "number" && Number.isSafeInteger(value) && value > 0 ? value : undefined;
+
+/**
+ * Read a per-model token count for the exact UID selected for this turn.
+ *
+ * Tries the selected UID and then its collapsed base id, preferring the
+ * canonical spelling and accepting dotted or case-folded saved hints — the same
+ * normalization the inference request applies to the model id. Where several
+ * spellings match one id, the smallest wins: a ceiling stated twice is
+ * satisfied by the lower statement.
+ */
+function devinModelTokenHint(
+  record: Record<string, number> | undefined,
+  modelUid: string,
+): number | undefined {
+  if (!record) return undefined;
+  for (const id of [modelUid, collapseDevinModelUid(modelUid)]) {
+    const exact = Object.hasOwn(record, id) ? positiveTokenCount(record[id]) : undefined;
+    if (exact !== undefined) return exact;
+    const matches = Object.entries(record)
+      .filter(([key]) => normalizeDevinModelId(key).toLowerCase() === id.toLowerCase())
+      .map(([, value]) => positiveTokenCount(value))
+      .filter((value): value is number => value !== undefined);
+    if (matches.length > 0) return Math.min(...matches);
+  }
+  return undefined;
+}
+
 /**
  * Resolve the INPUT ceiling for the exact UID selected for this turn. Catalog
  * ClientModelConfig #18 and CompletionConfiguration #3 both carry input tokens;
@@ -204,33 +233,50 @@ function resolveDevinMaxInputTokens(
   modelUid: string,
   liveWindow?: number,
 ): number | undefined {
-  const positive = (value: unknown): number | undefined =>
-    typeof value === "number" && Number.isSafeInteger(value) && value > 0 ? value : undefined;
-  const baseId = collapseDevinModelUid(modelUid);
-  const configured = (record: Record<string, number> | undefined): number | undefined => {
-    if (!record) return undefined;
-    for (const id of [modelUid, baseId]) {
-      // Prefer the canonical spelling; retain dotted/case-folded saved hints,
-      // matching the model-id normalization used for the inference request.
-      const exact = Object.hasOwn(record, id) ? positive(record[id]) : undefined;
-      if (exact !== undefined) return exact;
-      const matches = Object.entries(record)
-        .filter(([key]) => normalizeDevinModelId(key).toLowerCase() === id.toLowerCase())
-        .map(([, value]) => positive(value))
-        .filter((value): value is number => value !== undefined);
-      if (matches.length > 0) return Math.min(...matches);
-    }
-    return undefined;
-  };
-  const contextHint = configured(provider.modelContextWindows) ?? positive(provider.contextWindow);
-  const inputHint = configured(provider.modelMaxInputTokens);
-  const ceilings = [positive(liveWindow), contextHint, inputHint]
+  const contextHint = devinModelTokenHint(provider.modelContextWindows, modelUid)
+    ?? positiveTokenCount(provider.contextWindow);
+  const inputHint = devinModelTokenHint(provider.modelMaxInputTokens, modelUid);
+  const ceilings = [positiveTokenCount(liveWindow), contextHint, inputHint]
     .filter((value): value is number => value !== undefined);
   return ceilings.length > 0 ? Math.min(...ceilings) : undefined;
 }
 
-/** Pure test seam; runtime uses the same resolver immediately before dispatch. */
+/**
+ * Resolve the OUTPUT ceiling for this turn, highest authority first:
+ *
+ * 1. the caller's explicit `max_output_tokens`, forwarded unchanged — an
+ *    explicit cap is a request, so a small one is never widened into a
+ *    configured larger one;
+ * 2. the configured per-model cap (`modelMaxOutputTokens`), read through the
+ *    same UID-aware hint lookup the input ceiling uses;
+ * 3. the provider-wide `defaultMaxOutputTokens`;
+ * 4. undefined, which leaves the cloud-direct encoder's own 8192 fallback in
+ *    place for a provider that configured nothing.
+ *
+ * This is NOT the history ceiling, and the two must not collapse into one
+ * number. CompletionConfiguration #2 is the output cap and #3 is the context
+ * window, so feeding a context window into this resolver would ask Cognition to
+ * generate a whole window's worth of output. Nothing here reads
+ * `contextWindow` or `modelContextWindows` for that reason.
+ *
+ * Step 1 keeps the caller's raw value rather than `positiveTokenCount`: the
+ * inbound parser owns what a caller may send, and re-filtering here would
+ * silently promote a rejected value to a configured cap the caller never asked
+ * for.
+ */
+function resolveDevinMaxOutputTokens(
+  provider: OcxProviderConfig,
+  modelUid: string,
+  requested: number | undefined,
+): number | undefined {
+  if (typeof requested === "number") return requested;
+  return devinModelTokenHint(provider.modelMaxOutputTokens, modelUid)
+    ?? positiveTokenCount(provider.defaultMaxOutputTokens);
+}
+
+/** Pure test seams; runtime uses the same resolvers immediately before dispatch. */
 export const resolveDevinMaxInputTokensForTests = resolveDevinMaxInputTokens;
+export const resolveDevinMaxOutputTokensForTests = resolveDevinMaxOutputTokens;
 
 export class DevinMissingCredentialError extends Error {
   constructor() {
@@ -590,6 +636,9 @@ export function createDevinAdapter(
         const maxInputTokens = resolveDevinMaxInputTokens(
           provider, modelUid, catalog?.byUid.get(modelUid)?.contextWindow,
         );
+        const maxOutputTokens = resolveDevinMaxOutputTokens(
+          provider, modelUid, parsed.options.maxOutputTokens,
+        );
         // The reset-retry wrapper waits out a 429 that states its own recovery
         // delay ("limit will reset in 35 seconds") and replays the identical
         // request — but only while zero events have been yielded, so a
@@ -606,7 +655,7 @@ export function createDevinAdapter(
           // input hint used to force every model through the 128k default.
           completionOpts: {
             ...(maxInputTokens !== undefined ? { maxInputTokens } : {}),
-            ...(typeof parsed.options.maxOutputTokens === "number" ? { maxOutputTokens: parsed.options.maxOutputTokens } : {}),
+            ...(maxOutputTokens !== undefined ? { maxOutputTokens } : {}),
             ...(typeof parsed.options.temperature === "number" ? { temperature: parsed.options.temperature } : {}),
             ...(typeof parsed.options.topP === "number" ? { topP: parsed.options.topP } : {}),
           },