diff --git a/devlog/_plan/260926_claude_input_estimate/000_overview.md b/devlog/_plan/260926_claude_input_estimate/000_overview.md new file mode 100644 index 00000000000..9475efddc24 --- /dev/null +++ b/devlog/_plan/260926_claude_input_estimate/000_overview.md @@ -0,0 +1,16 @@ +# A Claude input estimate the settled route actually sends + +Unit opened 2026-09-26. + +- [010_estimation.md](010_estimation.md) — the Messages ingress. **DONE.** + +Origin: a Paseo agent on a DeepSeek V4.1 Flash conversation through this proxy drew its context +meter at 221%. The rate implied the agent had run far past the window without compacting, so the +report read as a broken compaction loop. Compaction was healthy; the number above it was not. Same +body, same upstream: the proxy published 432,068 input tokens on `message_start` for a prompt the +upstream billed at 131,907 — **3.28x**. + +This is the `#4857` family (the floor `message_start` publishes when the upstream has sent no +confirmed usage before the first frame), but not a recurrence: `#4891` and `#5057` fixed *when* the +floor is used and *whose* count it reports. The floor faithfully reports +`estimateClaudeRequestTokens`, and that estimate was measuring the wrong body. diff --git a/devlog/_plan/260926_claude_input_estimate/010_estimation.md b/devlog/_plan/260926_claude_input_estimate/010_estimation.md new file mode 100644 index 00000000000..84933550b3c --- /dev/null +++ b/devlog/_plan/260926_claude_input_estimate/010_estimation.md @@ -0,0 +1,65 @@ +# A Claude input estimate the settled route actually sends + +Defect: `src/server/claude-messages.ts` `estimateClaudeRequestTokens`. The estimator measured the +body the **caller sent**, while `message_start` publishes it as the floor for the prompt this proxy +**actually forwarded**. On the Anthropic wire those are the same body. On every other wire they are +not, and the gap is whatever the target adapter drops. + +Measured on the live path with a real Claude Code conversation replayed byte-identical (260 +messages, 23 tools, 1,734,433 B) to `thehive/deepseek-ai/deepseek-v4.1-flash` +(`adapter: openai-chat`): `message_start` **432,068** against the upstream's `message_delta` +**131,907** — **3.28x**. The contract the estimator's own doc cites allows `>2x` drift +(`devlog/_fin/260711_claude_inbound/040_phase4_hardening.md` §3); this is outside it. + +Why the two differ. Claude Code replays its own thinking blocks, and they dominate a long body: +80 blocks, 550,930 thinking chars and 750,284 `signature` chars. In the captured request the +thinking JSON is **78.8%** of all message JSON and the base64 `signature` is **56.7%** of the +thinking JSON. The `openai-chat` wire forwards that text only when the model is listed in +`preserveReasoningContentModels`, and it has no `signature` field at all — zero `signature` +references exist anywhere under `src/adapters/openai-chat*`. The `thehive` provider config declares +no reasoning policy keys, so it drops both, and upstreams do not bill replayed reasoning they never +receive. The estimator was counting 432,068 tokens of a prompt whose real size was 131,907. + +The gap is a property of the route, not a constant, so no divisor can absorb it: a route that does +replay thinking must keep counting it. Char-per-token calibration and the `thehive/` alias prefix +were both ruled out as causes — CJK is 0.07% of the body, and moving 4 to 3.5 chars/token is ±14%, +while dropping the alias alone makes the number 14% *worse*. + +Change: + +- New leaf `src/lib/claude-request-projection.ts`: `ClaudeThinkingProjection`, the native + `{text:true, signature:true}`, `projectBlock`, and `projectClaudeRequest` — a pure, idempotent + projection that returns message content with the blocks a wire does not carry emptied. It never + mutates its input and survives a message whose content is not an array. +- `src/adapters/openai-chat/messages.ts`: new exported `openAIChatSerializesThinking(provider, + modelId)`, the single source of truth for whether that wire carries a replayed thinking block. + The `messagesToChatFormat` conversion reads the same answer once per request + (`wireSerializesThinking`) instead of re-deriving it per message, so the estimator and the + serializer cannot drift apart — they are one rule. +- `src/server/claude-messages.ts`: `estimateClaudeRequestTokens` takes an optional + `thinking: ClaudeThinkingProjection` defaulting to the native pair, and projects the messages + before measuring. `claudeRequestTokenFloor` passes the projection for the route the turn settled + on (`settledRoute`, recorded where routing resolves), and `handleClaudeCountTokens` resolves its + own route through the read-only `previewRouteModel`. A route that is not `openai-chat` keeps the + native projection, so the Anthropic-native lane stays byte-exact and pre-existing two-argument + callers keep their behavior. + +The measurement only. The caller's body is never rewritten on its way to the adapter — the +projection exists to price the prompt, not to alter it, and every reader of the floor shares the one +estimate, so no number is double-counted. + +Rejected: excluding replayed thinking unconditionally (wrong for the native lane and any adapter +that does replay it); a `billsReplayedReasoning?: boolean` flag on `ProviderAdapter` (states a +billing policy on an interface whose business is wire shape, and cannot express "text yes, +signature no", which is the shape that actually occurs). + +Tests (new `tests/claude-integration/claude-estimate-projection.test.ts`): a replayed-thinking body +projects away exactly the unserialized fields; `projectClaudeRequest` is pure, idempotent, and keeps +a message it emptied; a body with no thinking is projection-invariant; and end-to-end, the +`message_start` floor for a real captured body lands within the confirmed usage instead of 3.28x +above it. Verified to fail when the projection is disabled — the end-to-end case reports 6946 where +it expects <42, so the assertion is load-bearing rather than decorative. + +Live verification, same captured body against the real upstream (104,318 / 131,907 = **0.79x**, and +`message_delta` byte-identical at 131,907, so the upstream result is untouched and only the +published estimate moved). The Paseo meter that opened this unit reads 240% before and 58% after. diff --git a/docs-site/src/content/docs/reference/adapters.md b/docs-site/src/content/docs/reference/adapters.md index 214784f2f9c..0d4b4719233 100644 --- a/docs-site/src/content/docs/reference/adapters.md +++ b/docs-site/src/content/docs/reference/adapters.md @@ -528,16 +528,20 @@ configuration that names the old id is rewritten at startup. `CompletionConfiguration`, #2 is the output cap and #3 is the context window; swapping those two makes every turn fail with an opaque `invalid_argument`. A temperature of exactly 0 is refused, so it is clamped to the smallest accepted value. -- A pre-output 429 that states a recovery delay is retried in place only when the full stated - delay fits within the remaining cumulative wait allowance. The adapter waits that full delay - and replays the request up to twice; the default cumulative allowance is 30 minutes - (`OPENCODEX_DEVIN_STATED_RESET_WAIT_MS`, hard ceiling one hour). If the delay exceeds the - remaining allowance, the original 429 is surfaced without waiting or replaying. Retrying - earlier than the stated delay is deliberately not attempted — the hint is the provider's best - estimate of its own window, and each replay slot is finite. If the limit still refuses, the - final 429 surfaces to the client with the stated delay preserved as its cooldown hint. A `~` - in the surfaced message marks a delay recovered from a secondhand trailer sentence rather - than an exact header value; clients still receive the parsed number itself. +- A pre-output 429 with a stated recovery delay is surfaced immediately by default, releasing the + admitted turn's shared capacity. Set `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` to a positive cumulative + allowance in milliseconds to wait for the full stated delay and replay the same request up to twice. + The allowance has a one-hour ceiling; an absent, empty, invalid, or negative value disables waiting. + An opted-in standalone wait keeps the HTTP turn and its shared active-turn slot open throughout the delay. + Streaming turns start SSE on a safe cooldown heartbeat, then schedule heartbeats every 500 ms or less + during the wait so the stall watchdog stays fed. A later pre-output 429 may still rotate to another + eligible OAuth account; without one it is reported inside the already-open stream. Buffered Grok + turns retain an HTTP 429 and `Retry-After` on a final refusal. + Combo children surface the pre-output 429 immediately, even when waiting is enabled, so the combo + can try its next target without holding an uncommitted response. Delays exceeding the remaining + allowance on standalone turns surface the original 429 without an early retry. The + final 429 preserves the stated delay as a cooldown hint. A `~` in its message marks a delay recovered + from a secondhand trailer sentence rather than an exact header value. - Experimental unofficial bridge; not shown in the dashboard preset by default. See the [provider guide](/guides/providers/) for login instructions. diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index e45d68b4bbb..988c03f45af 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -286,12 +286,21 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity | `directGeminiWireRenames?` | `boolean` | Google only. Applies only to direct AI Studio requests. Omitted or `true` keeps the `-tiered` wire rename for Gemini Flash ids (`gemini-3.7-flash` -> `gemini-3.7-flash-tiered`); `false` sends the requested bare ids to the wire unchanged. Vertex preserves the requested model ID, and Cloud Code Assist routing is unchanged. Set `false` when the configured upstream still serves the bare ids. | | `project?` | `string` | Vertex or Antigravity Cloud Code Assist project id. | | — | — | Antigravity account quota probes (`retrieveUserQuota` and `retrieveUserQuotaSummary`) always go to Google's own Cloud Code host through the pinned outbound transport, regardless of a configured `baseUrl`; the account bearer is never sent to an operator-configured endpoint and a redirect aborts the probe. Only the model-list fallback still honors `baseUrl`. | +| — | — | If Antigravity quota summary returns 403 for a valid OAuth account, OpenCodex retries that endpoint once with the legacy `antigravity/1.0` User-Agent and the same token and project. A 401 is not retried. Inference and model discovery retain the IDE User-Agent. | | `location?` | `string` | Vertex location; environment fallback is `GOOGLE_CLOUD_LOCATION`. | | `mcpServers?` | `Record` | Cursor only: stdio or Streamable HTTP MCP servers. | | `desktopExecutor?` | `DesktopExecutorConfig` | Cursor only: external computer-use and record-screen commands. | | `unsafeAllowNativeLocalExec?` | `boolean` | Cursor legacy boolean, equivalent to `nativeLocalExec: "on"` only when the newer field is unset. | | `nativeLocalExec?` | `"off" \| "codex-sandbox" \| "on"` | Cursor local-exec policy. `off` is default; `codex-sandbox` currently fails closed like `off`. | +Command Code's shipped per-model effort defaults include live API measurements. DeepSeek +v4/v4.1 Flash (including v4 Flash Vision), GLM-5.3 and GLM-5.3-Flash, Qwen3.8-Flash, +and Gemini-3.7-Flash support `low`, `medium`, `high`, `xhigh`, and `max`. Ladders remain +model-specific: `poolside/laguna-s-2.1-free` offers only `medium`, while Gemini-3.8-Flash +and MiMo-v2.5-Pro offer `low`, `medium`, and `high`. To override a pinned Command Code +row, set `modelReasoningEffortsAuthoritative: true` together with that model's +`modelReasoningEfforts` list. + Provider registration and replacement (`POST /api/providers`) validate `responsesPath` and `chatCompletionsPath` before changing live configuration or disk state. `PATCH /api/providers?name=` merges the request body with the stored provider; updates touching fields beyond `disabled` — except `requestPacing`-only updates — validate the merged provider's paths the same way before saving, and an invalid retained path returns `400` with the configuration unchanged. The same path rules apply when loading a configuration file. ### What a provider save keeps diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index cc5dd7842a5..a1d48542a31 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -168,6 +168,7 @@ } }, "explicit": { + "provider-antigravity-quota-retry.test.ts": "providers", "deepseek-artifact-tool-schema.test.ts": "providers", "client-config-export-output-limit.test.ts": "config", "openai-chat-serialized-tool-call-scaling.test.ts": "adapters/openai", @@ -466,6 +467,7 @@ "claude-picker-trust.test.ts": "claude-integration", "claude-desktop-remote-hub.test.ts": "claude-integration", "claude-dotenv-provenance-transport.test.ts": "claude-integration", + "claude-estimate-projection.test.ts": "claude-integration", "claude-first-party-union.test.ts": "claude-integration", "claude-gateway-cache.test.ts": "claude-integration", "claude-inbound-cache-stabilize.test.ts": "claude-integration", @@ -750,6 +752,7 @@ "combo-workspace-data.test.ts": "gui", "combos.test.ts": "codex-integration", "command-code-error-finish.test.ts": "providers", + "command-code-efforts.test.ts": "providers", "command-code-provider.test.ts": "providers", "command-code-quota.test.ts": "providers", "command-code-tool-text.test.ts": "providers", @@ -1635,6 +1638,7 @@ "server-auth.test.ts": "server", "server-background-lifecycle.test.ts": "server", "server-clickjacking-headers.test.ts": "server", + "server-combo-cooldown-fallback.test.ts": "server", "server-combo-failover-e2e.test.ts": "server", "server-combo-held-response.test.ts": "server", "server-combo-reasoning-replay-eligibility.test.ts": "server", diff --git a/src/adapters/base.ts b/src/adapters/base.ts index 0efce92e5b7..e56c9c69cf0 100644 --- a/src/adapters/base.ts +++ b/src/adapters/base.ts @@ -9,6 +9,8 @@ export interface IncomingMeta { headers: Headers; translatorBudget: TranslatorBudget; abortSignal?: AbortSignal; + /** Combo children must surface a pre-output refusal so the selector can try the next target. */ + comboAttempt?: boolean; /** * Provider-scoped fetch prepared by the Responses router. Stateful transports that emit more * than one physical HTTP request per logical turn must reuse it so every request participates in diff --git a/src/adapters/devin.ts b/src/adapters/devin.ts index 2fb7f567c09..6c7c8e3ff6d 100644 --- a/src/adapters/devin.ts +++ b/src/adapters/devin.ts @@ -9,7 +9,7 @@ import type { AdapterEvent, OcxAssistantMessage, OcxContentPart, OcxMessage, OcxParsedRequest, OcxProviderConfig, OcxTool, OcxToolCall, OcxToolResultMessage, OcxUsage } from "../types"; import { namespacedToolName } from "../types"; import type { IncomingMeta, ProviderAdapter } from "./base"; -import { streamChatEventsWithResetRetry, allocateCascadeId, CloudChatError, type ChatHistoryItem, type ToolDef } from "./devin/cloud-direct"; +import { streamChatEventsWithResetRetry, devinStatedResetWaitMs, allocateCascadeId, CloudChatError, type ChatHistoryItem, type ToolDef } from "./devin/cloud-direct"; import type { ContentPart } from "./devin/cloud-direct/chat"; import { getCachedCatalog, type CacheEntry } from "./devin/cloud-direct/catalog"; import { collapseDevinModelUid } from "./devin/live-models"; @@ -642,11 +642,12 @@ export function createDevinAdapter( const maxOutputTokens = resolveDevinMaxOutputTokens( provider, modelUid, parsed.options.maxOutputTokens, ); + // A combo child has not committed an outer response yet. Holding its preflight through + // a reset wait would also hold the next-target fallback with no client keepalive. + const resetWaitMs = incoming.comboAttempt ? 0 : devinStatedResetWaitMs(); // An admitted HTTP turn owns globally shared capacity until this call - // emits. Never retain that capacity while waiting out a provider 429: - // preserve the typed reset delay in generated diagnostic wording, - // never the raw trailer text that may reflect a credential. The - // refusal returns immediately so the caller can release its slot. + // emits. Without an explicit wait allowance, preserve the typed reset + // delay in generated diagnostic wording and return immediately. for await (const event of streamChatEventsWithResetRetry({ apiKey, apiServerUrl: host, @@ -665,7 +666,9 @@ export function createDevinAdapter( }, signal: incoming.abortSignal, }, { - maxWaitMs: 0, + maxWaitMs: resetWaitMs, + onWaitHeartbeat: resetWaitMs > 0 && parsed.stream + ? () => emit({ type: "heartbeat", preflightReady: true }) : undefined, execution: { executor: incoming.providerFetch, sendBudget: incoming.sendBudget, diff --git a/src/adapters/devin/cloud-direct/index.ts b/src/adapters/devin/cloud-direct/index.ts index f4dbab563f6..a07b90b0350 100644 --- a/src/adapters/devin/cloud-direct/index.ts +++ b/src/adapters/devin/cloud-direct/index.ts @@ -51,6 +51,7 @@ export { export { streamChatEventsWithResetRetry, + devinStatedResetWaitMs, STATED_RESET_MAX_REPLAYS, STATED_RESET_MAX_WAIT_MS, type StatedResetRetryOptions, diff --git a/src/adapters/devin/cloud-direct/stated-reset-retry.ts b/src/adapters/devin/cloud-direct/stated-reset-retry.ts index 1978e841ff5..eed3fa56d87 100644 --- a/src/adapters/devin/cloud-direct/stated-reset-retry.ts +++ b/src/adapters/devin/cloud-direct/stated-reset-retry.ts @@ -17,21 +17,26 @@ export const STATED_RESET_MAX_WAIT_MS = 1_800_000; /** Absolute maximum cumulative allowance, including explicit overrides. */ export const STATED_RESET_WAIT_CEILING_MS = 3_600_000; -function statedResetMaxWaitMs(): number { +function statedResetMaxWaitMs(defaultMs = STATED_RESET_MAX_WAIT_MS): number { const raw = process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS?.trim(); - if (!raw) return STATED_RESET_MAX_WAIT_MS; + if (!raw) return defaultMs; const parsed = Number(raw); - if (!Number.isFinite(parsed) || parsed < 0) return STATED_RESET_MAX_WAIT_MS; + if (!Number.isFinite(parsed) || parsed < 0) return defaultMs; // Zero explicitly disables local waiting. Values above one hour are capped. return Math.min(Math.floor(parsed), STATED_RESET_WAIT_CEILING_MS); } export const statedResetMaxWaitMsForTests = statedResetMaxWaitMs; +export function devinStatedResetWaitMs(): number { + return statedResetMaxWaitMs(0); +} + export interface StatedResetRetryOptions { /** Test seam: defaults to the real cloud stream. */ stream?: (req: CloudChatRequest) => AsyncGenerator; /** Test seam: must either honour the whole delay or reject on cancellation. */ sleep?: (ms: number, signal?: AbortSignal) => Promise; + onWaitHeartbeat?: () => void; maxReplays?: number; /** CUMULATIVE wait allowance, not a fresh allowance on every failure. */ maxWaitMs?: number; @@ -136,7 +141,16 @@ export async function* streamChatEventsWithResetRetry( // scheduling: waking a few milliseconds late must not reject an already // approved one-hour retry. No later wait can spend this allowance again. waitedMs += waitMs; - await sleep(waitMs, req.signal); + const heartbeat = options?.onWaitHeartbeat; + if (waitMs > 0) heartbeat?.(); + const beat = heartbeat && waitMs > 0 + ? setInterval(heartbeat, Math.min(500, Math.max(100, Math.floor(waitMs / 2)))) + : undefined; + try { + await sleep(waitMs, req.signal); + } finally { + if (beat !== undefined) clearInterval(beat); + } if (req.signal?.aborted) throw abortError(req.signal); } } diff --git a/src/adapters/openai-chat/messages.ts b/src/adapters/openai-chat/messages.ts index a712c83158a..bce315a2a19 100644 --- a/src/adapters/openai-chat/messages.ts +++ b/src/adapters/openai-chat/messages.ts @@ -7,6 +7,7 @@ import { EMPTY_TOOL_OUTPUT_ANNOTATION, isWhitespaceOnlyTextPartArray } from "../ import { identifyRoutedModel } from "../identity"; import { buildNonOpenAIToolCatalogNudgeForTools, shouldInjectNonOpenAIToolCatalogNudge } from "../tool-catalog-nudge"; import { peekReasoningForCall } from "../../responses/reasoning-replay-cache"; +import type { ClaudeThinkingProjection } from "../../lib/claude-request-projection"; import { inlineDocumentDataUrl } from "../../responses/inline-document"; import type { OcxAssistantMessage, OcxContentPart, OcxParsedRequest, OcxProviderConfig, OcxTextContent, OcxThinkingContent, OcxToolCall } from "../../types"; import { modelInList, namespacedToolName } from "../../types"; @@ -61,10 +62,36 @@ export function toolResultImageChatParts(content: string | OcxContentPart[]): un return parts; } +/** + * Whether this wire serializes a replayed thinking block for the given model. + * + * The single source of truth for that question on the Chat wire: the assistant branch below uses + * it to decide `reasoning_content`, and the Messages ingress uses it to decide how much of a + * prompt is worth counting. Two copies of this rule would drift, and the count would then + * describe a body this adapter does not send. + */ +export function openAIChatSerializesThinking( + provider: OcxProviderConfig, + modelId: string, +): ClaudeThinkingProjection { + return { + text: modelInList(provider.preserveReasoningContentModels, modelId), + // The wire has no `signature` field — base64 replay tokens are Anthropic-wire-only. + signature: false, + // A `redacted_thinking` block is opaque provider data with no Chat representation: the + // assistant branch below reads only `type: "thinking"` parts, so the encrypted form is + // dropped whatever the preserve list says. + redacted: false, + }; +} + export function messagesToChatFormat(parsed: OcxParsedRequest, provider: OcxProviderConfig): unknown[] { const out: unknown[] = []; const { context, options } = parsed; const replayCacheScope = parsed._reasoningReplayScope; + // One question, one answer for the whole conversion: does this wire carry replayed thinking + // back for this model? Asked once so a long history cannot pay for it per message. + const wireSerializesThinking = openAIChatSerializesThinking(provider, parsed.modelId).text; interface PendingToolCall { id: string; name: string } let pendingToolCalls: PendingToolCall[] = []; @@ -225,7 +252,7 @@ export function messagesToChatFormat(parsed: OcxParsedRequest, provider: OcxProv if ( reasoningContent.length === 0 && (toolCalls.length > 0 || thinkingParts.length > 0) - && modelInList(provider.preserveReasoningContentModels, parsed.modelId) + && wireSerializesThinking ) { const cached = toolCalls .map(tc => (tc.id ? peekReasoningForCall(tc.id, replayCacheScope) : undefined)) @@ -248,7 +275,7 @@ export function messagesToChatFormat(parsed: OcxParsedRequest, provider: OcxProv reasoningContent = " "; } } - if (reasoningContent.length > 0 && modelInList(provider.preserveReasoningContentModels, parsed.modelId)) { + if (reasoningContent.length > 0 && wireSerializesThinking) { // MiniMax's interleaved-thinking contract requires the structured // reasoning_details array back on the next turn; a reasoning_content // string is the native-format pass-back the docs mark unsupported. @@ -302,7 +329,7 @@ export function messagesToChatFormat(parsed: OcxParsedRequest, provider: OcxProv flushPendingToolCalls(); const name = safeToolName(msg.toolName); const cachedReasoning = - toolCallId && modelInList(provider.preserveReasoningContentModels, parsed.modelId) + toolCallId && wireSerializesThinking ? peekReasoningForCall(toolCallId, replayCacheScope) : undefined; // Same fallback as the main-assistant path: never emit a bare orphan @@ -316,7 +343,7 @@ export function messagesToChatFormat(parsed: OcxParsedRequest, provider: OcxProv // falsy hit as a miss so the placeholder still fires. const orphanReasoning = cachedReasoning - || (modelInList(provider.preserveReasoningContentModels, parsed.modelId) + || (wireSerializesThinking && modelInList(provider.requiresReasoningPlaceholderModels ?? provider.preserveReasoningContentModels, parsed.modelId) ? " " : undefined); diff --git a/src/adapters/run-turn-queue.ts b/src/adapters/run-turn-queue.ts index 6c706aff55f..1bafcbe545d 100644 --- a/src/adapters/run-turn-queue.ts +++ b/src/adapters/run-turn-queue.ts @@ -132,6 +132,7 @@ export interface AdapterEventPreflight { error?: Extract; empty: boolean; replayUnsafe: boolean; + ready?: boolean; timedOut?: boolean; } @@ -160,7 +161,7 @@ async function* replay( export async function preflightAdapterEvents( source: AsyncIterable, classifyFirstEvent?: (event: AdapterEvent) => Extract | undefined, - options?: { maxWaitMs?: number }, + options?: { maxWaitMs?: number; honorReady?: boolean }, ): Promise { const iterator = source[Symbol.asyncIterator](); const buffered: AdapterEvent[] = []; @@ -194,6 +195,9 @@ export async function preflightAdapterEvents( replayUnsafe ||= next.value.replayUnsafe === true; // Preserve the latch in replay even after the original unsafe heartbeat is evicted. buffered.push(replayUnsafe ? { ...next.value, replayUnsafe: true } : next.value); + if (next.value.preflightReady === true && options?.honorReady !== false) { + return { stream: replay(buffered, iterator), empty: false, replayUnsafe, ready: true }; + } if (buffered.length > PREFLIGHT_HEARTBEAT_RETAIN_LIMIT) buffered.shift(); continue; } @@ -255,10 +259,12 @@ export function createAdapterEventQueue(opts?: { // marker is not ordering — it is a latch. Dropping the incoming event // would discard the only record that Cursor already performed a local // side effect, and preflight would then permit an OAuth replay of it. - if (event.replayUnsafe === true && tail.replayUnsafe !== true) { - return { type: "heartbeat", replayUnsafe: true }; - } - return tail; + if (event.replayUnsafe !== true && event.preflightReady !== true) return tail; + return { + type: "heartbeat", + ...(tail.replayUnsafe === true || event.replayUnsafe === true ? { replayUnsafe: true as const } : {}), + ...(tail.preflightReady === true || event.preflightReady === true ? { preflightReady: true as const } : {}), + }; } if (event.type === "text_delta" && tail.type === "text_delta" && tail.phase === event.phase) { if (tail.text.length + event.text.length > COALESCE_MAX_CHUNK_LENGTH) return null; diff --git a/src/lib/claude-request-projection.ts b/src/lib/claude-request-projection.ts new file mode 100644 index 00000000000..cb10fb5ae03 --- /dev/null +++ b/src/lib/claude-request-projection.ts @@ -0,0 +1,101 @@ +/** + * Project a Messages request body onto the content a settled route actually serializes. + * + * `estimateClaudeRequestTokens` measures the caller's body as JSON. That is exactly right for the + * Anthropic-native wire, where a replayed `thinking` block — signature included — is forwarded + * verbatim. A routed wire may serialize far less, and on the Chat wire it does: replayed thinking + * becomes an optional `reasoning_content` string, the signature is never sent at all + * (`src/adapters/openai-chat/messages.ts` has no `signature` reference), and the text is dropped + * outright unless the model is on the provider's `preserveReasoningContentModels` list. + * + * The gap that opens is not a rounding error. On a captured 260-message Claude Code turn, replayed + * thinking blocks were 78.8% of the body's JSON characters and 56.7% of those characters were + * base64 signatures — bytes that are not prompt text under any tokenizer. Counting them made the + * published `message_start.usage.input_tokens` 3.28x the count the upstream actually reported + * (#4857 + the Paseo context meter it feeds), well past the >2x drift bound this estimator is held + * to (devlog 260711_claude_inbound 040 §3). + * + * The projection is therefore applied to the ESTIMATE only. The body the caller sent is never + * rewritten; this is a measurement that describes the route, not a transformation of the request. + */ + +/** Which replayed reasoning blocks and fields a wire serializes. */ +export interface ClaudeThinkingProjection { + /** Serialize `thinking.thinking`, the model's own replayed text. */ + text: boolean; + /** + * Serialize `thinking.signature`, the provider's base64 replay token. Real prompt size for + * wires that carry it; pure overhead for wires that do not. + */ + signature: boolean; + /** + * Serialize a `redacted_thinking` block, whose `data` is an opaque provider blob. It rides + * alongside `thinking` rather than inside it: a wire can reconstruct the model's reasoning + * without carrying the encrypted form, and the Chat wire does exactly that. + */ + redacted: boolean; +} + +/** + * The Anthropic-native wire forwards a replayed thinking block verbatim, signature included, so + * nothing is projected away. This is the default: an unknown route keeps the measured body. + */ +export const CLAUDE_NATIVE_THINKING: ClaudeThinkingProjection = { text: true, signature: true, redacted: true }; + +interface ProjectableBody { + system?: unknown; + messages?: unknown; + tools?: unknown; +} + +/** + * One content block as the given wire would carry it. + * + * `undefined` means the wire sends nothing for it, and the caller drops the entry. A `thinking` + * block that keeps its text but not its signature is returned as a copy rather than edited: the + * block object is shared with the outbound request builder, which must stay untouched. + */ +function projectBlock(block: unknown, thinking: ClaudeThinkingProjection): unknown { + if (!block || typeof block !== "object") return block; + if (!("type" in block)) return block; + if (block.type === "redacted_thinking") return thinking.redacted ? block : undefined; + if (block.type !== "thinking") return block; + if (!thinking.text) return undefined; + if (thinking.signature || !("signature" in block)) return block; + // Copy without the signature: the caller's block object is shared with the outbound request + // builder, so it must never be edited in place. + const { signature: _dropped, ...rest } = block; + return rest; +} + +/** + * A copy of `raw` whose message content carries only the replayed thinking this route serializes. + * + * Pure and idempotent; the input is never mutated. Blocks are dropped only inside array-valued + * message `content`, which is the sole protocol position a replayed thinking block occupies. + * A message whose content array empties out is left as an empty array rather than deleted: the + * adapter's own "nothing left to send" rule is keyed on text, tool calls and reasoning together, + * and re-deriving it here would be a second copy of that rule to keep in sync. + */ +export function projectClaudeRequest( + raw: ProjectableBody, + thinking: ClaudeThinkingProjection, +): ProjectableBody { + if (thinking.text && thinking.signature && thinking.redacted) return raw; + if (!Array.isArray(raw.messages)) return raw; + const messages = raw.messages as unknown[]; + return { + ...raw, + messages: messages.map(message => { + if (!message || typeof message !== "object") return message; + if (!("content" in message) || !Array.isArray(message.content)) return message; + const content = message.content as unknown[]; + return { + ...message, + content: content + .map(block => projectBlock(block, thinking)) + .filter(block => block !== undefined), + }; + }), + }; +} diff --git a/src/providers/command-code-efforts.ts b/src/providers/command-code-efforts.ts index 64919d6d95d..d4a40ff051f 100644 --- a/src/providers/command-code-efforts.ts +++ b/src/providers/command-code-efforts.ts @@ -22,7 +22,8 @@ import { readBoundedResponseBody } from "../lib/bounded-body"; * `ultra` returned 400. A correction therefore adds the rungs a profile newly lists and keeps the * rungs the upstream measurably accepts; dropping them would strip an effort that works today, * including Codex's default `high`. The refresh path decodes the same payload, so it narrows a row - * only after the upstream actually rejects a rung. + * only after the upstream actually rejects a rung. Issue #5096 supplies additional live API + * measurements that widen the DeepSeek Flash, GLM-5.3, Gemini-3.7-Flash and HY4 rows. */ const COMMAND_CODE_MODEL_EFFORTS = { // Captured profile payload 2026-09-23: claude-fable-5-1.html. @@ -36,11 +37,11 @@ const COMMAND_CODE_MODEL_EFFORTS = { profileUrl: "https://commandcode.ai/models/claude-opus-5-5", }, "deepseek/deepseek-v4-flash": { - efforts: ["high", "max"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/deepseek-v4-flash", }, "deepseek/deepseek-v4-flash-vision-exp": { - efforts: ["high", "max"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/deepseek-v4-flash-vision-exp", }, // Captured profile payload 2026-09-23: deepseek-v4-flash-fast.html. @@ -50,7 +51,7 @@ const COMMAND_CODE_MODEL_EFFORTS = { }, // Captured profile payload 2026-09-23: deepseek-v4-1-flash.html. "deepseek/deepseek-v4.1-flash": { - efforts: ["low", "high", "max"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/deepseek-v4-1-flash", }, "gpt-5.6-luna": { @@ -58,7 +59,7 @@ const COMMAND_CODE_MODEL_EFFORTS = { profileUrl: "https://commandcode.ai/models/gpt-5-6-luna", }, "google/gemini-3.7-flash": { - efforts: ["low", "medium", "high"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/gemini-3-7-flash", }, // Captured profile payload 2026-09-23: gemini-3-8-flash.html. @@ -83,11 +84,11 @@ const COMMAND_CODE_MODEL_EFFORTS = { profileUrl: "https://commandcode.ai/models/glm-5-2-fast", }, "zai-org/GLM-5.3": { - efforts: ["low", "high", "max"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/glm-5-3", }, "z-ai/glm-5.3-flash": { - efforts: ["low", "high", "max"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/glm-5-3-flash", }, // Captured profile payload 2026-09-23: glm-5-3-flashx.html. @@ -143,7 +144,7 @@ const COMMAND_CODE_MODEL_EFFORTS = { }, // Captured profile payload 2026-09-23: hy4-preview.html. "tencent/hy4-preview": { - efforts: ["low", "medium", "high"], + efforts: ["low", "medium", "high", "xhigh", "max"], profileUrl: "https://commandcode.ai/models/hy4-preview", }, // Captured profile payload 2026-09-23: grok-4-7.html. @@ -151,10 +152,31 @@ const COMMAND_CODE_MODEL_EFFORTS = { efforts: ["low", "medium", "high", "xhigh"], profileUrl: "https://commandcode.ai/models/grok-4-7", }, + // Live API measurements supplied in #5096; no verified public profile URL. + "moonshotai/Kimi-K3": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "MiniMaxAI/MiniMax-M3": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "xiaomi/mimo-v2.5": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "xai/grok-4.5": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "xai/grok-4.6": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "tencent/hy3-paid": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "stepfun/Step-3.7-Flash": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "Qwen/Qwen3.8-Max": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "Qwen/Qwen3.8-27B": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "nvidia/nemotron-3-ultra-550b-a55b": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "meituan/LongCat-2.0:free": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "inclusionai/ling-3.0-flash-sante:free": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "thinkingmachines/inkling-small": { efforts: ["low", "medium", "high", "xhigh", "max"] }, + "moonshotai/Kimi-K2.7-Code": { efforts: ["low", "medium", "high", "xhigh"] }, + "moonshotai/Kimi-K2.7-Code-Highspeed": { efforts: ["low", "high", "xhigh", "max"] }, + "xiaomi/mimo-v2.5-pro": { efforts: ["low", "medium", "high"] }, + "Qwen/Qwen3.7-32B": { efforts: ["low", "medium", "high", "xhigh"] }, + "Qwen/Qwen3.7-72B": { efforts: ["low", "medium", "high", "xhigh"] }, + "Qwen/Qwen3.6-35B-A22B": { efforts: ["low", "medium", "high", "xhigh"] }, + "poolside/laguna-s-2.1-free": { efforts: ["medium"] }, } as const; /** - * Official Command Code model-profile facts, not a model catalog. Models remain + * Command Code profile facts and live API measurements (#5096), not a model catalog. Models remain * account-scoped and come exclusively from the authenticated /provider/v1/models endpoint. */ export const COMMAND_CODE_MODEL_REASONING_EFFORTS: Record = Object.fromEntries( @@ -273,7 +295,7 @@ export async function refreshCommandCodeReasoningEfforts( destination = DEFAULT_EFFORT_DESTINATION, ): Promise { const key = cacheKey(modelId, destination); - let profile: { efforts: readonly string[]; profileUrl: string } | undefined; + let profile: { efforts: readonly string[]; profileUrl?: string } | undefined; for (const [id, row] of Object.entries(COMMAND_CODE_MODEL_EFFORTS)) { if (keyFor(id) === keyFor(modelId)) { profile = row; @@ -286,6 +308,7 @@ export async function refreshCommandCodeReasoningEfforts( rejected.add(rejectedEffort); rejectedEfforts.set(key, rejected); } + if (!profile.profileUrl) return commandCodeReasoningEfforts(modelId, destination); try { const response = await fetchFn(profile.profileUrl, { headers: { Accept: "text/html" }, diff --git a/src/providers/quota/antigravity.ts b/src/providers/quota/antigravity.ts index 928b10e2570..8e8c2428d0a 100644 --- a/src/providers/quota/antigravity.ts +++ b/src/providers/quota/antigravity.ts @@ -253,16 +253,21 @@ function antigravityUnavailableFailure( } export async function probeAntigravityUsageQuota(accessToken: string, projectId: string): Promise { - const fetchQuota = (url: string) => providerOutboundPost("google-antigravity", { baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE }, url, { + const fetchQuota = (url: string, userAgent = antigravityUserAgent()) => providerOutboundPost("google-antigravity", { baseUrl: ANTIGRAVITY_ACCOUNT_QUOTA_BASE }, url, { headers: { Accept: "application/json", "Content-Type": "application/json", - "User-Agent": antigravityUserAgent(), Authorization: `Bearer ${accessToken}`, + "User-Agent": userAgent, Authorization: `Bearer ${accessToken}`, }, body: JSON.stringify({ project: projectId }), signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS), }, antigravityOutboundDependencies); let summaryFailure: QuotaFailureCode | undefined; try { - const response = await fetchQuota(ANTIGRAVITY_QUOTA_SUMMARY_URL); + let response = await fetchQuota(ANTIGRAVITY_QUOTA_SUMMARY_URL); + if (response.status === 403) { + // Some valid accounts reject the IDE fingerprint only for quota accounting. + try { await response.body?.cancel(); } catch { /* Best-effort release before retry. */ } + response = await fetchQuota(ANTIGRAVITY_QUOTA_SUMMARY_URL, "antigravity/1.0"); + } if (await providerRedirectError(response, ANTIGRAVITY_QUOTA_SUMMARY_URL)) return unavailableAntigravityQuota("redirect_blocked"); if (response.status === 401 || response.status === 403) return unavailableAntigravityQuota("access_denied"); if (response.ok) { diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index 4bcb558cde3..33e8f36d776 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -18,6 +18,7 @@ import { sseFieldValue } from "../lib/sse-decoder"; import { enforceAnthropicImageLimits, sniffImageDimensions } from "../adapters/anthropic-image-guard"; import { normalizeAnthropicImages } from "../adapters/anthropic-image-normalize"; import { createToolCallIdAllocator } from "../adapters/tool-call-id"; +import { openAIChatSerializesThinking } from "../adapters/openai-chat/messages"; import { messagesToResponsesTranslation } from "../protocols/codecs/messages"; import { AnthropicRequestError, DesktopModelMappingUnavailableError, extractOcxEffortDirective, extractOcxRouteDirective, resolveInboundModel, type ClaudeCacheKeySource } from "../claude/inbound"; import { isKnownDesktop3pModelId, resolveDesktop3pAlias } from "../claude/desktop-3p"; @@ -45,7 +46,12 @@ import { } from "../claude/outbound"; import { clearableDeadline, idleDeadline } from "../lib/abort"; import { estimateTokens } from "../lib/token-estimate"; -import { captureRouteStaticPolicy, NoEligiblePolicyCandidateError, UnknownRoutingPolicyError, routeModel } from "../router"; +import { + CLAUDE_NATIVE_THINKING, + projectClaudeRequest, + type ClaudeThinkingProjection, +} from "../lib/claude-request-projection"; +import { captureRouteStaticPolicy, NoEligiblePolicyCandidateError, previewRouteModel, routedProviderConfig, UnknownRoutingPolicyError, routeModel, type RouteResult } from "../router"; import { evidenceFromBody } from "../routing/request-evidence"; import { resolveWireProtocolOverride } from "./adapter-resolve"; import type { OcxConfig } from "../types"; @@ -904,7 +910,7 @@ async function handleClaudeMessagesWithBudget( if (!requestedModel) requestedModel = (anthropicBody as Rec).model as string; const stream = internalBody.stream === true; /** - * This proxy's count of the prompt it is about to forward, computed at most once. + * This proxy's count of the prompt it is about to forward, computed at most once per wire. * * Two readers want it and they want it under different rules. The usage log takes it as a * floor only for estimated-usage adapters, because its merge is `max(reported, estimate)` and @@ -913,12 +919,17 @@ async function handleClaudeMessagesWithBudget( * `message_delta` still corrects it (#4857). */ let requestTokenFloor: number | undefined; - const claudeRequestTokenFloor = (): number => { - if (requestTokenFloor === undefined) { - requestTokenFloor = estimateClaudeRequestTokens(anthropicBody as Rec, requestedModel); - } - return requestTokenFloor; - }; + /** + * The `text|signature|redacted` triple `requestTokenFloor` was measured under. + * + * The count is not a constant for the request: a combo re-picks its child at dispatch and a + * retry can rotate the wire, so the pre-dispatch callers and the post-dispatch translator can + * legitimately want different projections. Keying the memo re-measures when they disagree + * instead of handing the translator the pre-dispatch measurement, and the estimator still runs + * at most twice for one request. At most, because a request has one settled wire per phase and + * the key collapses every repeat of the same answer. + */ + let requestTokenFloorKey: string | undefined; // Routed adapters only support streamed turns; always stream internally and fold // the translated Anthropic SSE into a message JSON for non-streaming clients. internalBody.stream = true; @@ -927,7 +938,25 @@ async function handleClaudeMessagesWithBudget( // Native ChatGPT passthrough (openai-responses forward) accepts only Codex-shaped // bodies: it 400s on sampling params ("Unsupported parameter: max_output_tokens", // verified live 2026-07-11). Strip them for that route; routed providers keep them. - let settledRoute: ReturnType | undefined; + let settledRoute: RouteResult | undefined; + /** + * Which replayed thinking fields the send that actually happened will serialize. + * + * Read lazily: routing settles the ingress wire, core may then re-pick it (a combo child, a + * rotated retry), and this is called both before and after that happens. It therefore reads the + * physical attempt when one exists and the ingress route only until then. + */ + const claudeThinkingProjection = (): ClaudeThinkingProjection => + thinkingProjectionForDispatch(config, settledRoute, logCtx); + const claudeRequestTokenFloor = (): number => { + const thinking = claudeThinkingProjection(); + const key = `${thinking.text}|${thinking.signature}|${thinking.redacted}`; + if (requestTokenFloor === undefined || requestTokenFloorKey !== key) { + requestTokenFloor = estimateClaudeRequestTokens(anthropicBody as Rec, requestedModel, thinking); + requestTokenFloorKey = key; + } + return requestTokenFloor; + }; try { const route = routeModel(config, internalBody.model as string, evidenceFromBody(internalBody)); // Same reason as the native Chat lane: this route can be sent from here, so @@ -1330,12 +1359,20 @@ function estimateBase64AttachmentTokens(data: string): number { * characters: one 2MB screenshot is ~2.7M base64 chars, which the plain chars/token * divide reports as hundreds of thousands of tokens versus a real cost around 1.6k. * That breaks the >2x drift bound the estimator is held to (devlog 260711_claude_inbound - * 040 §3). Text and url sources are left in place and counted as characters, as is + * 040 §3); a live 260-message turn whose replayed thinking was 78.8% of the body breached + * it at 3.28x, which is why the estimate is projected onto the settled route. Text and url + * sources are left in place and counted as characters, as is * anything outside protocol content positions (tool_use.input, tool schemas). + * + * `thinking` selects which replayed thinking fields the SETTLED route serializes, so the measure + * describes the prompt this proxy forwards rather than the one the caller typed. Omitted, the + * whole body counts — correct for the Anthropic-native wire, where nothing is projected away. + * See `claude-request-projection.ts` for why a routed wire must project it out. */ export function estimateClaudeRequestTokens( raw: { system?: unknown; messages?: unknown; tools?: unknown }, modelId: string | undefined, + thinking: ClaudeThinkingProjection = CLAUDE_NATIVE_THINKING, ): number { let attachmentTokens = 0; // Blank base64 payloads ONLY in protocol content positions: message content blocks and @@ -1370,11 +1407,93 @@ export function estimateClaudeRequestTokens( : messages; const parts: string[] = []; if (raw.system !== undefined) parts.push(typeof raw.system === "string" ? raw.system : JSON.stringify(raw.system)); - if (raw.messages !== undefined) parts.push(JSON.stringify(sanitizedMessages(raw.messages))); + if (raw.messages !== undefined) { + const projected = projectClaudeRequest(raw, thinking); + parts.push(JSON.stringify(sanitizedMessages(projected.messages))); + } if (raw.tools !== undefined) parts.push(JSON.stringify(raw.tools)); return Math.max(1, estimateTokens(parts.join("\n"), modelId) + attachmentTokens); } +/** + * The projection for a route that has already settled. + * + * Only the OpenAI-shaped Chat adapter discards replayed thinking; every other settled wire + * forwards the body it was given. An unknown route keeps the full body. + */ +function thinkingProjectionForRoute(route: RouteResult | undefined): ClaudeThinkingProjection { + if (!route || route.provider.adapter !== "openai-chat") return CLAUDE_NATIVE_THINKING; + return openAIChatSerializesThinking(route.provider, route.modelId); +} + +/** + * The projection for the wire that will physically carry the request. + * + * The ingress route is not the last word on that wire. A combo re-picks its child at dispatch, + * and a retry can rotate the adapter mid-turn, so the ingress pick can name a different provider + * — and therefore a different body — than the one that is sent. `logCtx.activeAttempt` is the + * send that actually happened (`sealRequestAttemptIdentity` keeps its adapter current), so it + * wins once it exists; before the first send the ingress route is the only authority there is. + * + * A Chat identity is re-derived through `routedProviderConfig`, not read off the raw config row: + * `preserveReasoningContentModels` is registry-merged, so a row that omits it would otherwise + * price a preserve-listed model as if the wire dropped its reasoning. + */ +function thinkingProjectionForDispatch( + config: OcxConfig, + route: RouteResult | undefined, + logCtx: Pick, +): ClaudeThinkingProjection { + const attempt = logCtx.activeAttempt; + const adapter = attempt?.adapter ?? logCtx.providerAdapter; + // No send to describe yet: the ingress route is the best available answer, and for the + // Anthropic-native wire (which drops nothing) it is already the right one. + if (adapter === undefined) return thinkingProjectionForRoute(route); + if (adapter !== "openai-chat") return CLAUDE_NATIVE_THINKING; + const providerName = attempt?.provider; + const modelId = attempt?.model ?? route?.modelId; + const provider = providerName !== undefined && Object.hasOwn(config.providers, providerName) + ? config.providers[providerName] + : undefined; + // A Chat wire whose destination cannot be named keeps the route's own answer: over-counting on + // a path that publishes nothing is harmless, under-counting a real prompt is not. + if (providerName === undefined || modelId === undefined || provider === undefined) { + return thinkingProjectionForRoute(route); + } + try { + return openAIChatSerializesThinking(routedProviderConfig(providerName, provider), modelId); + } catch { + return thinkingProjectionForRoute(route); + } +} + +/** + * The projection for a route resolved only to MEASURE a body this handler never sends. + * + * `previewRouteModel` is the read-only resolver: it advances no combo round-robin state, so a + * count request cannot steer where the next real turn goes. An unresolvable model keeps the + * full body, matching the old behavior for models routing cannot place. + * + * The wire is settled exactly as the turn path settles it. Routing fills in the provider's + * registry adapter, and a per-model `modelAdapters` override or a pinned wire is applied later by + * `resolveWireProtocolOverride` — so skipping it here would price the count against a body the + * routed adapter never sends. + */ +function thinkingProjectionForPreview(config: OcxConfig, modelId: string): ClaudeThinkingProjection { + try { + const route = previewRouteModel(config, modelId); + route.staticPolicy = captureRouteStaticPolicy( + route.providerName, route.modelId, route.provider, route.staticPolicy.effectiveAlias, "anthropic", + ); + route.provider = resolveWireProtocolOverride( + route.providerName, route.modelId, route.provider, "anthropic", route.staticPolicy, + ); + return thinkingProjectionForRoute(route); + } catch { + return CLAUDE_NATIVE_THINKING; + } +} + export async function handleClaudeCountTokens( req: Request, config: OcxConfig, @@ -1435,7 +1554,14 @@ export async function handleClaudeCountTokens( const nativeCountBody = resolveProtocolSettings(config).rollout.managedMessagesNative ? (await import("./messages-native")).nativeMessagesCountBody(config, cc, raw, { fastRow: countFastRow !== null }) : undefined; - const inputTokens = estimateClaudeRequestTokens(nativeCountBody ?? raw, model); + // A count answers for the prompt a real turn from this model would forward, so it projects + // the same unserialized content that turn's `message_start` floor does. Counting the raw + // caller body instead reported replayed thinking this route never sends (#4857 family). + const inputTokens = estimateClaudeRequestTokens( + nativeCountBody ?? raw, + model, + thinkingProjectionForPreview(config, model), + ); return new Response(JSON.stringify({ input_tokens: inputTokens }), { status: 200, headers: { "Content-Type": "application/json" }, diff --git a/src/server/index/link-listener.ts b/src/server/index/link-listener.ts index c6d9c017c08..f3aeb2a7626 100644 --- a/src/server/index/link-listener.ts +++ b/src/server/index/link-listener.ts @@ -29,6 +29,10 @@ export type LinkListenerStatus = { reason: string | null; }; +export function linkListenerOwnsTarget(status: LinkListenerStatus): boolean { + return status.state === "listening" && status.port !== null; +} + export interface LinkListenerLifecycle { ownsListener(server: Server): boolean; start(ctx: LinkListenerStartContext): void; diff --git a/src/server/index/optional-listeners.ts b/src/server/index/optional-listeners.ts index 9542585bfa3..6b3d9e56e89 100644 --- a/src/server/index/optional-listeners.ts +++ b/src/server/index/optional-listeners.ts @@ -6,6 +6,7 @@ import { } from "./claude-intercept-lifecycle"; import { createLinkListenerLifecycle, + linkListenerOwnsTarget, linkRouteAllowed, type LinkListenerDeps, type LinkListenerLifecycle, @@ -82,7 +83,9 @@ export function createOptionalListenerSet(linkDeps: LinkListenerDeps = {}): O activeConfig = ctx.config; linkListener.start({ dispatch: ctx.dispatch, maxRequestBodySize: ctx.maxRequestBodySize }); unregisterSupervisorAdmission ??= linkListener.onAuthenticatedCatalog(apiKeyId => supervisor.notifyAuthenticatedRequest?.(apiKeyId)); - supervisor.start(); + if (linkListenerOwnsTarget(linkListener.status())) { + supervisor.start(); + } supervisorStop = () => supervisor.stop(); claudeIntercept.start({ config: ctx.config, diff --git a/src/server/index/serve-options.ts b/src/server/index/serve-options.ts index 682c2d8fe58..97a7c674b70 100644 --- a/src/server/index/serve-options.ts +++ b/src/server/index/serve-options.ts @@ -1398,7 +1398,7 @@ export function createServeOptions(ctx: ServeOptionsContext) { }; return runAdmittedHttpTurn(req, policy, async turnAdmissionLease => { const response = await handleContextHistory(req, config, logCtx, contextEndpoint(url.pathname)!, - turnAdmissionLease, admission, () => resolveApiAuth(req, policy)); + turnAdmissionLease, admission, () => resolveApiAuth(req, ingress === "hub-link" ? linkPolicy() : policy)); addFinalRequestLog(requestId, start, logCtx, response.status, response.status === 499 ? { closeReason: "client_cancel" } : undefined); return withCors(response, req, policy); diff --git a/src/server/management/link-routes.ts b/src/server/management/link-routes.ts index 12c6eb9bed2..fae3201f14f 100644 --- a/src/server/management/link-routes.ts +++ b/src/server/management/link-routes.ts @@ -362,6 +362,7 @@ async function issue(ctx: ManagementContext): Promise { failureCode = "listener_unavailable"; throw new Error("link listener unavailable"); } + await state.supervisor.ensureStarted(); const boundStore = readStoreFor(ctx); if (!port(boundStore.listenerPort)) throw new Error("link listener did not bind"); return Response.json({ linkId: id, apiKeyId: issued.id, key: issued.key, listenerPort: boundStore.listenerPort }); diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts index 20b4cc20994..26d4b731c3f 100644 --- a/src/server/responses/core-combo.ts +++ b/src/server/responses/core-combo.ts @@ -667,7 +667,11 @@ export async function executeComboResponses( const attempt = beginRequestAttempt( (logCtx.attempts?.length ?? 0) + 1, pick.target.provider, - pick.target.model, + // The id the child wire will actually send, not the selector the combo named. A target may + // be an alias, and `routeConcreteModel` above is where it becomes the provider's native id; + // recording the alias here would describe a request that never left (the adapter resolves + // the id before it reads any per-model list). + targetRoute.modelId, config.providers[pick.target.provider]!.adapter, ); childLog.activeAttempt = attempt; diff --git a/src/server/responses/run-turn-execution.ts b/src/server/responses/run-turn-execution.ts index 5659c0970b9..8b0129b2212 100644 --- a/src/server/responses/run-turn-execution.ts +++ b/src/server/responses/run-turn-execution.ts @@ -251,6 +251,7 @@ export async function executeResponsesRunTurn( { headers: requestState.selectedForwardHeaders, abortSignal: runTurnAbort.signal, + comboAttempt: options.comboAttempt === true, translatorBudget, providerFetch: runTurnProviderFetch, // The only way the request budget reaches a transport the adapter owns. Without it @@ -435,6 +436,47 @@ export async function executeResponsesRunTurn( return false; } }; + const streamAfterPreflight = ( + initialSource: AsyncIterable, + replayParsed: PreparedResponsesRequest["parsed"], + initiallyReplayUnsafe: boolean, + ): AsyncIterable => (async function* () { + let source = initialSource; + let replayUnsafe = initiallyReplayUnsafe; + let firstMeaningfulSeen = false; + while (true) { + let rotated = false; + for await (const event of source) { + if (!firstMeaningfulSeen && event.type === "heartbeat") { + replayUnsafe ||= event.replayUnsafe === true; + yield event; + continue; + } + if (!firstMeaningfulSeen && !replayUnsafe && event.type === "error" + && await rotateRunTurnAdapterOnPreflight429(event)) { + const retryQueue = createAdapterEventQueue({ + onBacklogExceeded: () => runTurnAbort.abort(), + }); + const pendingPermit = sendBudgetState.pendingHopPermit; + const retryAttempt = runTurnAttempt(retryQueue, "oauth-account-429", false, replayParsed); + if (pendingPermit) { + const releaseIfUnclaimed = () => { + if (sendBudgetState.pendingHopPermit !== pendingPermit) return; + sendBudgetState.pendingHopPermit = undefined; + pendingPermit.release(); + }; + void retryAttempt.then(releaseIfUnclaimed, releaseIfUnclaimed); + } + source = retryQueue.stream(); + rotated = true; + break; + } + firstMeaningfulSeen = true; + yield event; + } + if (!rotated) return; + } + })(); const preflightRunTurnFailover = async ( firstSource: AsyncIterable, // LOCAL PATCH (runturn-websearch): replayed attempts re-dispatch this @@ -448,9 +490,10 @@ export async function executeResponsesRunTurn( let deferPendingPermitCleanup = false; try { while (true) { - const preflight = await preflightAdapterEvents(source, undefined, deadlineAt === undefined - ? undefined - : { maxWaitMs: deadlineAt - Date.now() }); + const preflight = await preflightAdapterEvents(source, undefined, { + ...(deadlineAt === undefined ? {} : { maxWaitMs: deadlineAt - Date.now() }), + honorReady: replayParsed.stream, + }); if (preflight.timedOut) { const pendingPermit = sendBudgetState.pendingHopPermit; if (pendingPermit && latestRetryAttempt) { @@ -465,8 +508,9 @@ export async function executeResponsesRunTurn( }; void latestRetryAttempt.then(releaseIfUnclaimed, releaseIfUnclaimed); } - return preflight.stream; + return streamAfterPreflight(preflight.stream, replayParsed, preflight.replayUnsafe); } + if (preflight.ready) return streamAfterPreflight(preflight.stream, replayParsed, preflight.replayUnsafe); if (preflight.replayUnsafe || !preflight.error || !(await rotateRunTurnAdapterOnPreflight429(preflight.error))) { @@ -553,7 +597,9 @@ export async function executeResponsesRunTurn( if (refusal) return refusal; } if (options.comboAttempt) { - const preflight = await preflightAdapterEvents(eventSource, classifyUndeclaredFirstTool); + const preflight = await preflightAdapterEvents( + eventSource, classifyUndeclaredFirstTool, { honorReady: false }, + ); if (preflight.error || preflight.empty) { runTurnAbort.abort(); queue.close(); @@ -672,7 +718,9 @@ export async function executeResponsesRunTurn( )) runTurnEvents.push(event); } if (grokDevinPreflight) { - const preflight = await preflightAdapterEvents((async function* () { yield* runTurnEvents; })()); + const preflight = await preflightAdapterEvents( + (async function* () { yield* runTurnEvents; })(), undefined, { honorReady: false }, + ); const refusal = grokRateLimitResponse(preflight); if (refusal) return refusal; } diff --git a/src/types/request.ts b/src/types/request.ts index c365826de94..1c0df11b3cb 100644 --- a/src/types/request.ts +++ b/src/types/request.ts @@ -361,7 +361,7 @@ export interface OcxProviderContinuationState { } export type AdapterEvent = - | { type: "heartbeat"; replayUnsafe?: true } + | { type: "heartbeat"; replayUnsafe?: true; preflightReady?: true } | { type: "text_delta"; text: string; phase?: OcxMessagePhase } | { type: "thinking_delta"; thinking: string } // Anthropic extended-thinking round-trip: signature_delta for the current thinking block, and diff --git a/structure/providers-and-adapters.md b/structure/providers-and-adapters.md index 94dc5987050..0dd25185c04 100644 --- a/structure/providers-and-adapters.md +++ b/structure/providers-and-adapters.md @@ -102,7 +102,7 @@ rewrite rules and the routed-id settlement. | `src/adapters/declaration-carrier.ts`, `src/adapters/input-media-guard.ts` | Default-deny allowlists for constraints the normalized request carries but a wire may not be able to express: `tools[*].allowed_callers`, which fences a tool off from callers, and inline document bytes. Both are refused with a 400 at the single guard every registered adapter passes through, rather than left to each adapter, because an adapter that never learned about the carrier rebuilds without it and answers normally. `allowed_callers` reaches the `anthropic` wire; document bytes reach `anthropic`, `openai-chat` and `google`; the `openai-responses` wire is exempt from the whole guard because it forwards the original body. Adding an `AdapterWire` member makes the omission visible in these lists instead of at a customer's upstream. The unrestricted `["direct"]` caller default is not a restriction. | | `src/adapters/azure.ts` | Azure OpenAI bridge. | | `src/adapters/cursor.ts`, `src/adapters/cursor/` | Cursor protobuf transport: discovery, request builder, event decoding, MCP, thread continuity, native-exec policy. | -| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated 429 reset delays are surfaced to the client rather than slept inside an admitted turn, so they cannot retain shared active-turn capacity. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. | +| `src/adapters/devin.ts`, `src/adapters/devin/cloud-direct/` | Devin runTurn transport over Cognition Connect-RPC. `GetChatMessage` uses the Responses provider executor and shared physical-send budget; catalog, JWT, and `src/web-search/devin-executor.ts` native search support RPCs remain outside inference-send accounting. Provider-stated pre-output 429 reset delays are surfaced immediately by default, releasing shared active-turn capacity. A positive `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` explicitly enables bounded waiting and up to two replays on standalone turns, which hold that capacity until completion or cancellation. Combo children bypass that wait and surface a pre-output 429 so the next target can run. During an opted-in standalone wait, safe heartbeats commit the response preflight and keep the stream's stall watchdog fed. Invalid values fail closed to the immediate-refusal behavior. A recorded tenant host is used only for the stored account whose credential owns the transmitted key, searched in the configured provider id and then its deprecated alias; a configured, forwarded, or unmatched key uses the configured base URL or the US default. Native search previews the current route by effective adapter without mutating combo selection state, pins one admitted active-account snapshot for the request, and calls `GetWebSearchResults`, so it starts no CLI or second model. | | `src/adapters/kiro.ts` and `src/adapters/kiro/` | Kiro event/tool/thinking/truncation/retry handling, including an egress-aware completion fallback, fixed public HTTP 5xx text, and closed-set status/code diagnostics. The original path is a facade over leaves for wire identity, reasoning, conversation state, token estimation, payload assembly, streaming, and the adapter. | | `src/adapters/mimo-free.ts` | Mimo Free transport (client identity + JWT). Concurrent requests share one JWT bootstrap bound only to its timeout; each request stops waiting on its own abort without cancelling the others. | | `src/adapters/command-code.ts`, `src/adapters/command-code-tool-text.ts`, `src/adapters/command-code-restored-schema.ts` | Command Code OAuth NDJSON translation. For every `xiaomi/mimo-` model, text, native calls, reasoning, and terminal decisions share one byte-bounded queue with linear queue visits. Markup is deduplicated against matching native calls; text-only restoration requires one contiguous text run, a clean finish, a declared tool, and arguments validated against supported schema constraints. A parameter-free (freeform) block may omit `` but must end with ``; parameter blocks keep the canonical close. Markup appended after prose in the same delta is split off at the marker and held like a block that opens with ``; a marker split across deltas after prose is still released as text. Native, reasoning, and other intervening events interrupt a still-probing block but leave a held block held in arrival order, and the queued byte bound still flushes an unresolved envelope as text. An envelope the strict parser rejects but that opens with ``, closes with ``, and names a declared function is dropped when a native call for that same function arrives and on a clean finish; markup that parses but fits no supported schema is still released as text. Regex patterns, other unsupported constraints, and abnormal finishes fail closed. `tests/providers/command-code-tool-text-prose-split.test.ts` covers the split, the interleaved-event hold, and both drop paths. | @@ -139,6 +139,14 @@ OAuth presets resolve discovery against the same canonical registry transport as before any adapter-specific transport override, so a stale configured `baseUrl` cannot receive an OAuth bearer token. +Command Code effort defaults in `src/providers/command-code-efforts.ts` combine public-profile +facts with the live API measurements from #5096. Both presets share the exact per-model rows; +`xhigh` is preserved when accepted, and narrow ladders such as Laguna's `medium`-only row remain +narrow. Rows without a verified profile URL still record rejected efforts but skip profile fetching. +An explicit `modelReasoningEffortsAuthoritative` model row overrides the shipped ladder; seeded +rows without that flag do not. `tests/providers/command-code-efforts.test.ts` covers the measured +rows and wire values; `tests/providers/command-code-provider.test.ts` covers operator overrides. + ## TypeSafe JEV decision provider `src/providers/registry/entries-extended.ts` owns the canonical `jev` key preset at diff --git a/structure/remote-link.md b/structure/remote-link.md index 70e6022dcd5..661d1553eb5 100644 --- a/structure/remote-link.md +++ b/structure/remote-link.md @@ -26,7 +26,7 @@ The client tunnel pidfile is `/link/client-tunnel.pid` with `{ versio ## Tunnels, management and CLI -`src/link/ssh-runner.ts` runs every OpenSSH and `ssh-keygen` argv without a shell and caps captured output. `src/link/supervisor.ts` keeps one `ssh -R` child per hub-initiated link, drives it with the tunnel reducer, coalesces reloads without dropping a later request, reconciles unowned `link:` API keys at startup, and stops children before the hub-link listener on shutdown. It reaps a leftover tunnel only when Linux `/proc//cmdline` matches the recorded argv exactly; on other platforms a leftover is reported, never killed. `src/link/status-projection.ts` builds the status document the dashboard and `ocx link status` read, including persisted compensation failures, and `src/link/admission-wait.ts` waits for the first key-authenticated `/v1/catalog` read that proves a new link works. +`src/link/ssh-runner.ts` runs every OpenSSH and `ssh-keygen` argv without a shell and caps captured output. `src/link/supervisor.ts` keeps one `ssh -R` child per hub-initiated link, drives it with the tunnel reducer, coalesces reloads without dropping a later request, reconciles unowned `link:` API keys at startup, and stops children before the hub-link listener on shutdown. Automatic startup begins the supervisor only after the hub-link listener owns its socket; a bind failure must never leave a reverse forward targeting the persisted port. It reaps a leftover tunnel only when Linux `/proc//cmdline` matches the recorded argv exactly; on other platforms a leftover is reported, never killed. `src/link/status-projection.ts` builds the status document the dashboard and `ocx link status` read, including persisted compensation failures, and `src/link/admission-wait.ts` waits for the first key-authenticated `/v1/catalog` read that proves a new link works. Applying a link probes the host key into a temporary file, waits for the operator to confirm the fingerprint, issues a data key, records the link, starts the tunnel and runs the client's `ocx connect --link --key-stdin` over SSH with the key on standard input. A failed step revokes the new key first and removes the record only after revocation succeeds; if revocation fails the `src/link/` state persists a `compensation_failed` marker, and status reports the failed compensation after restart. A listener that is not listening fails the request instead of handing out a key. Removing a link stops its tunnel, disconnects the client, revokes the key and deletes the record; a failed client disconnect restarts the tunnel and keeps the record and key unless removal is forced. `src/cli/link.ts` provides `ocx link port|issue|revoke|status`. `ocx link revoke` is idempotent: a `404 link_not_found` answer exits 0, because removal revokes the key before it deletes the record, so a missing record means the key is already gone. A 404 without that code still fails. diff --git a/structure/runtime.md b/structure/runtime.md index 84c3c0d7095..8d4c96f542f 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -232,7 +232,7 @@ GUI, session bootstrap/exchange, and `/api/*`. The `hub-link` socket is HTTP-only and default-denies all but the fixed data routes, catalog, hub-state, usage, and `GET /readyz`; every `Upgrade` header, management, GUI, session, health, and unknown `/v1/*` route is rejected before dispatch. Its `opencodex-link.invalid` policy admits only configured -key ids recorded by `links.json`, never the environment token. `ensureStarted()` is single-flight, +key ids recorded by `links.json`, never the environment token. Context relays rebuild that policy for their post-body admission check, so key revocation stops an in-flight request before dispatch. `ensureStarted()` is single-flight, final deletion closes the listener, and `src/server/index/optional-listeners.ts` runs supervisor teardown before closing this listener and the Claude intercept pair. ### Claude intercept pair diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index 944c2eaebad..9ad2992da13 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -253,6 +253,8 @@ Pool quota producers and account commands follow the [bounded raw-observation co ## Account quota failure diagnostics +`src/providers/quota/antigravity.ts` retries a quota-summary 403 once with `User-Agent: antigravity/1.0`, releasing the first response body and preserving the bearer, project and pinned accounting endpoint. A 401 is not retried; redirects remain blocked, and a repeated 403 remains unavailable with `access_denied`. Other retry failures retain the models fallback, whose User-Agent remains the IDE fingerprint, as do discovery and inference. + Antigravity account quota probes expose only a closed `quotaFailure` category when the read is unavailable. Typed transport failures, rejected destinations, redirects, denied access, rate limits and unusable bodies are distinguished; successful fallback clears the earlier failure. The last attempted endpoint determines the diagnosis. A 401/403 category does not change account health, entitlement or routing eligibility. `src/providers/quota.ts` binds diagnoses to the probed credential/project and rechecks before cache reads and API projection. Reauthentication invalidates an old diagnosis independently of last-good quota bars. Private digests, callbacks and upstream error values are not serialized. The CLI and current/all-account dashboard views consume the same closed code; unknown codes and local management-read failures retain generic unavailable text. Codes are transient, never persisted quota evidence. Authenticated TUN field acceptance remains separate from deterministic transport coverage. diff --git a/structure/transports/responses-failover.md b/structure/transports/responses-failover.md index 8d4cbb7c3b2..918eb0e2305 100644 --- a/structure/transports/responses-failover.md +++ b/structure/transports/responses-failover.md @@ -200,11 +200,19 @@ streaming Response. A first-event 429 without a replay-unsafe heartbeat becomes error through the shared error formatter and client Retry-After resolver. The buffered first event is replayed for every other outcome. The preflight is bounded by the configured stall timeout, including any earlier OAuth failover preflight on this path. On expiry, its pending iterator read is -handed to SSE replay exactly once; timeout therefore starts a 200 SSE response, and any later 429 -is an SSE failure. Text, reasoning, and tool output commit the stream. This boundary neither retries -the turn nor changes combo failover policy. Buffered Responses turns apply the same refusal -formatter to their collected first event after OAuth failover. Other buffered results retain -the original event list, including output preceding a late error. +handed to SSE replay exactly once; timeout therefore starts a 200 SSE response. An opted-in Devin +cooldown heartbeat also starts SSE before its wait ends. Once SSE begins, the stream forwards safe +heartbeats and checks the first meaningful event: a pre-output 429 may rotate to an eligible OAuth +account and replay the unchanged request. Without an eligible account it remains an in-stream +failure, since HTTP status is already committed. Text, reasoning, and tool output commit the stream +and prevent later rotation. Buffered Responses turns ignore the cooldown-ready heartbeat during +preflight and apply the same HTTP 429 formatter to a final refusal after OAuth failover. Other +buffered results retain the original event list, including output preceding a late error. Combo +children ignore cooldown readiness during their own preflight, so a final 429 without output can +still move to the next combo target. Devin combo children bypass opted-in stated-reset waiting and +surface the pre-output 429 immediately, because the outer response cannot forward their wait +heartbeats while it is choosing a target. An earlier replay-unsafe heartbeat or meaningful output keeps +the failure on the current target. ## Optional client transport hints diff --git a/structure/transports/streaming-health.md b/structure/transports/streaming-health.md index 61411dd0149..ca635c76245 100644 --- a/structure/transports/streaming-health.md +++ b/structure/transports/streaming-health.md @@ -40,6 +40,11 @@ public upstreams too. A disabled budget resolves to `0`, and the watchdog kill i `> 0`, so a `0` never mis-arms a kill on the first beat; keep-alives keep flowing regardless, so a silent-but-healthy local model (CPU-bound thinking or a long time-to-first-token) stays connected. Adapter-yielded `{ type: "heartbeat" }` events DO reset the watchdog. +During an opted-in standalone Devin stated-reset wait, `src/adapters/devin/cloud-direct/stated-reset-retry.ts` +emits a safe adapter heartbeat immediately and schedules the next ones at intervals no greater than +500 ms, below the shortest positive +stall budget of one second. The cooldown-ready marker opens SSE before the wait ends; a later +pre-output 429 still reaches the OAuth rotation check before any model output is committed. The Anthropic adapter maps both SSE comments and `ping` events to that heartbeat (#5707), so an upstream that only pings while a long thinking block is silent still counts as live. When the Responses-to-Chat converter receives that typed heartbeat, it emits the same bounded SSE diff --git a/tests/adapters/run-turn-queue.test.ts b/tests/adapters/run-turn-queue.test.ts index 50abe61aaab..12e5d4cda0b 100644 --- a/tests/adapters/run-turn-queue.test.ts +++ b/tests/adapters/run-turn-queue.test.ts @@ -267,6 +267,43 @@ describe("run-turn adapter event queue", () => { }); describe("run-turn adapter event preflight", () => { + test("a cooldown heartbeat commits preflight while preserving later output", async () => { + const ready: AdapterEvent = { type: "heartbeat", preflightReady: true }; + const values = [ready, text("resumed"), done]; + + const preflight = await preflightAdapterEvents(events(values)); + + expect(preflight.error).toBeUndefined(); + expect(preflight.replayUnsafe).toBe(false); + expect(await collect(preflight.stream)).toEqual(values); + }); + + test("buffered preflight continues past a cooldown heartbeat to the first refusal", async () => { + const ready: AdapterEvent = { type: "heartbeat", preflightReady: true }; + const error: AdapterEvent = { type: "error", status: 429, message: "rate limited" }; + + const preflight = await preflightAdapterEvents(events([ready, error]), undefined, { honorReady: false }); + + expect(preflight.error).toEqual(error); + expect(preflight.ready).toBeUndefined(); + expect(await collect(preflight.stream)).toEqual([ready, error]); + }); + + test("queued heartbeat coalescing retains the cooldown preflight signal", async () => { + const queue = createAdapterEventQueue(); + queue.push(heartbeat); + queue.push({ type: "heartbeat", preflightReady: true }); + queue.push(text("resumed")); + queue.close(); + + const preflight = await preflightAdapterEvents(queue.stream()); + + expect(await collect(preflight.stream)).toEqual([ + { type: "heartbeat", preflightReady: true }, + text("resumed"), + ]); + }); + test("10,000 leading heartbeats retain only the bounded tail and still complete", async () => { const values = [...Array.from({ length: 10_000 }, () => heartbeat), done]; const preflight = await preflightAdapterEvents(events(values)); diff --git a/tests/claude-integration/claude-estimate-projection.test.ts b/tests/claude-integration/claude-estimate-projection.test.ts new file mode 100644 index 00000000000..48c0108099b --- /dev/null +++ b/tests/claude-integration/claude-estimate-projection.test.ts @@ -0,0 +1,492 @@ +/** + * The Claude Messages request-token estimate, projected onto the route that will carry it. + * + * A Claude Code turn replays its own thinking blocks, and on a long session those blocks dominate + * the body: on a captured 260-message turn they were 78.8% of the messages JSON, 56.7% of that + * being base64 signatures. A routed OpenAI Chat wire serializes almost none of it — the signature + * never, the text only for preserve-listed models — so counting the caller's own blocks published + * a `message_start.usage.input_tokens` 3.28x the prompt the upstream actually received. Paseo's + * context meter reads that frame, so it showed 221% of a 180k window while compaction was healthy. + * + * These cases pin the measure itself and the wiring that feeds it. The estimator's + * attachment-pricing behavior lives with the endpoint suite. + */ +import { afterEach, expect, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../../src/config"; +import { startServer } from "../../src/server"; +import { estimateClaudeRequestTokens, handleClaudeCountTokens, handleClaudeMessages } from "../../src/server/claude-messages"; +import { estimateTokens } from "../../src/lib/token-estimate"; +import { CLAUDE_NATIVE_THINKING, projectClaudeRequest } from "../../src/lib/claude-request-projection"; +import { openAIChatSerializesThinking } from "../../src/adapters/openai-chat/messages"; +import { clearComboSelectionState, clearComboTargetCooldowns } from "../../src/combos"; +import type { OcxConfig, OcxProviderConfig } from "../../src/types"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { SERVER_BUDGET_MS } from "../helpers/test-budget"; + +let testDir = ""; +let previousHome: string | undefined; +let releaseSpendHome: (() => void) | undefined; +let isolatedHomeActive = false; + +/** Only the end-to-end case needs a home; the estimator cases below are pure. */ +function setUpIsolatedHome(): void { + previousHome = process.env.OPENCODEX_HOME; + testDir = mkdtempSync(join(tmpdir(), "ocx-claude-estimate-")); + process.env.OPENCODEX_HOME = testDir; + releaseSpendHome = acquireOwnedSpendHome(); + isolatedHomeActive = true; +} + +function restoreIsolatedHome(): void { + // The preload arms OPENCODEX_HOME for the whole process, so an unpaired restore must not + // touch it: deleting it here left sibling files running in the same process to write the + // real home, which the preload guard then refused. + if (!isolatedHomeActive) return; + isolatedHomeActive = false; + releaseSpendHome?.(); + releaseSpendHome = undefined; + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + if (testDir) removeTreeWithRetry(testDir); + testDir = ""; +} + +afterEach(restoreIsolatedHome); + +/** A Chat-completions upstream that records what the proxy actually sent it. */ +function mockChatUpstreamCapturing(): { server: ReturnType; captured: Array> } { + const captured: Array> = []; + const server = Bun.serve({ + port: 0, + async fetch(req) { + try { captured.push(await req.json() as Record); } catch { /* keep streaming */ } + const frames = [ + `data: ${JSON.stringify({ choices: [{ index: 0, delta: { role: "assistant", content: "Hello" } }] })}\n\n`, + `data: ${JSON.stringify({ choices: [{ index: 0, delta: {}, finish_reason: "stop" }], usage: { prompt_tokens: 12, completion_tokens: 3 } })}\n\n`, + "data: [DONE]\n\n", + ]; + return new Response(frames.join(""), { headers: { "Content-Type": "text/event-stream" } }); + }, + }); + return { server, captured }; +} + +function mockConfig(baseUrl: string): OcxConfig { + return { + port: 0, + defaultProvider: "mock", + providers: { + mock: { adapter: "openai-chat", baseUrl, apiKey: "k", allowPrivateNetwork: true }, + }, + } as OcxConfig; +} + +test("estimateClaudeRequestTokens drops replayed thinking the settled Chat wire does not send", () => { + // The defect this pins (#4857 family): a routed openai-chat turn serializes replayed + // thinking only as `reasoning_content`, and only for preserve-listed models — the + // signature is never sent at all. Counting the caller's own blocks made the published + // message_start floor 3.28x the upstream's reported prompt on a live 260-message turn. + const thinking = { + type: "thinking", + thinking: "T".repeat(60_000), + signature: "S".repeat(90_000), + }; + const raw = { + messages: [ + { role: "assistant", content: [thinking, { type: "text", text: "answer" }] }, + { role: "user", content: "next" }, + ], + }; + const partsWithoutThinking = [ + JSON.stringify([{ role: "assistant", content: [{ type: "text", text: "answer" }] }, { role: "user", content: "next" }]), + ]; + + // A route whose wire serializes no replayed thinking counts only what it would send. + const dropped = estimateClaudeRequestTokens(raw, "m", { text: false, signature: false }); + expect(dropped).toBe(Math.max(1, estimateTokens(partsWithoutThinking.join("\n"), "m"))); + // Signature-only projection still prices the model's own replayed text. + const textOnly = estimateClaudeRequestTokens(raw, "m", { text: true, signature: false }); + expect(textOnly).toBeGreaterThan(dropped); + // The native wire forwards the block verbatim, which is the default for an unknown route. + const native = estimateClaudeRequestTokens(raw, "m", { text: true, signature: true }); + expect(native).toBe(Math.max(1, estimateTokens(JSON.stringify(raw.messages), "m"))); + expect(native).toBe(estimateClaudeRequestTokens(raw, "m")); + // The dropped measure must not still be carrying the signature's bytes. + expect(dropped).toBeLessThan(native / 3); +}); + +test("a redacted_thinking blob is priced where the wire carries it and dropped where it cannot", () => { + // `redacted_thinking` rides in the same content array as `thinking` but is its own axis: the + // Anthropic-native lane replays the opaque blob verbatim, and the Chat wire has no + // representation for it at all. Folding it into the text/signature axes would either lose the + // blob's real bytes on the native lane or keep pricing it on a wire that never sends it, which + // is the same 40x over-count this file exists to prevent. + const data = "R".repeat(90_000); + const redactedOnly = { messages: [{ role: "assistant", content: [{ type: "redacted_thinking", data }] }] }; + const kept = estimateClaudeRequestTokens(redactedOnly, "m", { text: true, signature: true, redacted: true }); + const dropped = estimateClaudeRequestTokens(redactedOnly, "m", { text: true, signature: true, redacted: false }); + // The all-true projection is the identity fast path, so the native default must agree exactly. + expect(kept).toBe(estimateClaudeRequestTokens(redactedOnly, "m")); + expect(kept).toBeGreaterThan(10_000); + // What is left is the JSON envelope around an emptied block, not 90k characters of blob. + expect(dropped).toBeLessThan(kept / 20); + // A signature-less thinking block beside it still answers to the axes that own it. + const both = { + messages: [{ + role: "assistant", + content: [{ type: "thinking", thinking: "T".repeat(4_000), signature: "S".repeat(4_000) }, { type: "redacted_thinking", data }], + }], + }; + expect(estimateClaudeRequestTokens(both, "m", { text: true, signature: true, redacted: true })) + .toBe(estimateClaudeRequestTokens(both, "m")); + expect(estimateClaudeRequestTokens(both, "m", { text: false, signature: false, redacted: true })) + .toBeGreaterThan(estimateClaudeRequestTokens(both, "m", { text: false, signature: false, redacted: false }) * 20); +}); + +test("a preserve-listed Chat model still serializes no redacted_thinking", () => { + // The preserve list buys `reasoning_content`, nothing more: the assistant branch builds that + // string from `type: "thinking"` parts alone. The shared helper therefore reports the blob's + // axis false for every Chat provider, listed or not, and the estimate follows it. + const listed: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "http://127.0.0.1:1/v1", + apiKey: "k", + preserveReasoningContentModels: ["m"], + allowPrivateNetwork: true, + }; + expect(openAIChatSerializesThinking(listed, "m")).toEqual({ text: true, signature: false, redacted: false }); + const raw = { messages: [{ role: "assistant", content: [{ type: "redacted_thinking", data: "R".repeat(90_000) }] }] }; + const onChat = estimateClaudeRequestTokens(raw, "m", openAIChatSerializesThinking(listed, "m")); + expect(onChat).toBeLessThan(estimateClaudeRequestTokens(raw, "m") / 20); +}); + +test("projectClaudeRequest is pure, idempotent, and keeps emptied messages", () => { + const thinking = { type: "thinking", thinking: "replayed", signature: "sig" }; + const raw = { messages: [{ role: "assistant", content: [thinking] }] }; + + const projected = projectClaudeRequest(raw, { text: false, signature: false }); + expect(projected).not.toBe(raw); + expect(projected.messages).toEqual([{ role: "assistant", content: [] }]); + // The caller's body is shared with the outbound request builder, so it must be untouched. + expect(raw.messages).toEqual([{ role: "assistant", content: [thinking] }]); + // Re-running changes nothing: an emptied content array has no thinking left to drop. + expect(projectClaudeRequest(projected, { text: false, signature: false })).toEqual(projected); +}); + +test("estimateClaudeRequestTokens counts a body with no thinking identically under every projection", () => { + // Guards the default path the pre-existing estimator tests rely on: with nothing to + // project away, the projection cannot change the answer. + const raw = { + system: "be brief", + messages: [{ role: "user", content: [{ type: "text", text: "no thinking here" }] }], + tools: [{ name: "Read", input_schema: { type: "object" } }], + }; + const native = estimateClaudeRequestTokens(raw, "m"); + expect(estimateClaudeRequestTokens(raw, "m", { text: false, signature: false })).toBe(native); + expect(estimateClaudeRequestTokens(raw, "m", { text: true, signature: false })).toBe(native); +}); + +test("count_tokens prices the wire the modelAdapters override selects, in both directions", async () => { + // A count is a promise about the prompt a real turn would forward, so it has to settle the + // wire the same way that turn does. Routing fills in the provider's registry adapter, and a + // per-model override lands afterwards — pricing the provider-wide adapter instead gets the + // answer backwards whenever the two disagree, which is precisely the case overrides exist for. + const body = { + model: "mock/test-model", + messages: [ + { role: "user", content: "u" }, + { + role: "assistant", + content: [ + { type: "thinking", thinking: "T".repeat(4_000), signature: "S".repeat(4_000) }, + { type: "text", text: "a" }, + ], + }, + ], + }; + const chatWire = estimateClaudeRequestTokens(body, "mock/test-model", { text: false, signature: false, redacted: false }); + const nativeWire = estimateClaudeRequestTokens(body, "mock/test-model", CLAUDE_NATIVE_THINKING); + const countFor = async (providerAdapter: string, override: string): Promise => { + const config = { + port: 0, + defaultProvider: "mock", + providers: { + mock: { + adapter: providerAdapter, + baseUrl: "http://127.0.0.1:1/v1", + apiKey: "k", + allowPrivateNetwork: true, + modelAdapters: { "test-model": override }, + }, + }, + } as unknown as OcxConfig; + const response = await handleClaudeCountTokens(new Request("http://localhost/v1/messages/count_tokens", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(body), + }), config); + expect(response.status).toBe(200); + return ((await response.json()) as { input_tokens: number }).input_tokens; + }; + // Provider says Responses, the model says Chat: the Chat body is the one that gets sent. + expect(await countFor("openai-responses", "openai-chat")).toBe(chatWire); + // And the reverse: a Chat provider whose model speaks Responses forwards the thinking verbatim. + expect(await countFor("openai-chat", "openai-responses")).toBe(nativeWire); + // The two directions must stay distinguishable, or the assertions above are vacuous. + expect(chatWire).toBeLessThan(nativeWire / 20); +}); + +test("message_start floor describes the prompt the Chat wire actually sent, not the replayed thinking", async () => { + // End-to-end pin for the route-aware projection: the estimator must read the SETTLED route, + // not just accept a projection when handed one. A routed openai-chat turn with no + // preserveReasoningContentModels entry serializes no replayed thinking, so a floor that + // still counts it overstates the prompt — the live defect that published 3.28x. + setUpIsolatedHome(); + const upstream = mockChatUpstreamCapturing(); + saveConfig(mockConfig(`${upstream.server.url.toString().replace(/\/$/, "")}/v1`)); + const server = startServer(0); + try { + const thinking = { + type: "thinking", + thinking: "replayed reasoning ".repeat(400), + signature: "S".repeat(20_000), + }; + const response = await fetch(new URL("/v1/messages", server.url), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "mock/test-model", + max_tokens: 128, + stream: true, + messages: [ + { role: "user", content: "first" }, + { role: "assistant", content: [thinking, { type: "text", text: "answer" }] }, + { role: "user", content: "second" }, + ], + }), + }); + expect(response.status).toBe(200); + const text = await response.text(); + const startFrame = text.slice(text.indexOf("event: message_start")); + const published = (JSON.parse(startFrame.slice(startFrame.indexOf("data: ") + 6, startFrame.indexOf("\n\n"))) + .message.usage.input_tokens) as number; + + expect(upstream.captured).toHaveLength(1); + const sent = upstream.captured[0]!; + // What the wire actually carried: no signature field, and no reasoning_content because + // this model is not on a preserve list. + const serialized = JSON.stringify(sent.messages); + expect(serialized).not.toContain("S".repeat(64)); + expect(serialized).not.toContain("reasoning_content"); + + // The floor must therefore land near the serialized prompt, not near the caller's body. + const sentEstimate = estimateTokens(JSON.stringify(sent.messages), "mock/test-model"); + expect(published).toBeLessThan(sentEstimate * 1.5); + expect(published).toBeGreaterThan(sentEstimate * 0.5); + // And it must be far below what counting the caller's own thinking would produce. + expect(published).toBeLessThan(estimateClaudeRequestTokens({ messages: JSON.parse(JSON.stringify([thinking])) }, "mock/test-model") / 2); + } finally { + await server.stop(true); + upstream.server.stop(true); + restoreIsolatedHome(); + } +}, { timeout: SERVER_BUDGET_MS }); + +/** Frames an Anthropic-wire upstream answers with, including the usage frame a client reads. */ +const ANTHROPIC_SSE_FRAMES = [ + 'event: message_start\ndata: {"type":"message_start","message":{"id":"msg_b","type":"message","role":"assistant","model":"m2","content":[],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":7,"output_tokens":0}}}\n\n', + 'event: content_block_start\ndata: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}\n\n', + 'event: content_block_delta\ndata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"sunny"}}\n\n', + 'event: content_block_stop\ndata: {"type":"content_block_stop","index":0}\n\n', + 'event: message_delta\ndata: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":5}}\n\n', + 'event: message_stop\ndata: {"type":"message_stop"}\n\n', +].join(""); + +/** Chat-wire frames for the upstream that answers the combo's second target in the Chat case. */ +const CHAT_SSE_FRAMES = [ + `data: ${JSON.stringify({ choices: [{ index: 0, delta: { role: "assistant", content: "sunny" } }] })}\n\n`, + `data: ${JSON.stringify({ choices: [{ index: 0, delta: {}, finish_reason: "stop" }], usage: { prompt_tokens: 12, completion_tokens: 3 } })}\n\n`, + "data: [DONE]\n\n", +].join(""); + +const COMBO_THINKING = { + type: "thinking", + thinking: "replayed reasoning ".repeat(400), + signature: "S".repeat(20_000), +}; +const COMBO_MESSAGES = [ + { role: "user", content: "first" }, + { role: "assistant", content: [COMBO_THINKING, { type: "text", text: "answer" }] }, + { role: "user", content: "second" }, +]; + +/** + * One `combo/pair` turn whose first target refuses so the combo hops to `second`. + * + * The floor is read from the translated stream's own `message_start` frame, which is where a + * Claude client — and Paseo's context meter — reads it. + */ +async function comboFailoverFloor(second: { + provider: string; + model: string; + adapter: string; + frames: string; + /** Native ids the destination advertises, for a target the caller names by alias. */ + models?: string[]; + modelAliases?: Record; + preserveReasoningContentModels?: string[]; +}): Promise<{ published: number; secondBodies: Array> }> { + setUpIsolatedHome(); + clearComboSelectionState(); + clearComboTargetCooldowns(); + const firstBodies: Array> = []; + const first = Bun.serve({ + hostname: "127.0.0.1", + port: 0, + async fetch(req) { + firstBodies.push(await req.json() as Record); + return Response.json({ error: { message: "fixture outage" } }, { status: 503 }); + }, + }); + const secondBodies: Array> = []; + const secondServer = Bun.serve({ + hostname: "127.0.0.1", + port: 0, + async fetch(req) { + secondBodies.push(await req.json() as Record); + return new Response(second.frames, { headers: { "content-type": "text/event-stream" } }); + }, + }); + const loopback = (url: URL): string => url.toString().replace(/\/$/, ""); + const config = { + port: 0, + defaultProvider: "first", + providers: { + first: { adapter: "openai-chat", baseUrl: `${loopback(first.url)}/v1`, apiKey: "k", allowPrivateNetwork: true }, + [second.provider]: { + adapter: second.adapter, + baseUrl: second.adapter === "anthropic" ? loopback(secondServer.url) : `${loopback(secondServer.url)}/v1`, + apiKey: "k", + allowPrivateNetwork: true, + ...(second.models ? { models: second.models } : {}), + ...(second.modelAliases ? { modelAliases: second.modelAliases } : {}), + ...(second.preserveReasoningContentModels + ? { preserveReasoningContentModels: second.preserveReasoningContentModels } + : {}), + }, + }, + combos: { + pair: { + strategy: "failover", + targets: [{ provider: "first", model: "m1" }, { provider: second.provider, model: second.model }], + }, + }, + } as unknown as OcxConfig; + try { + const response = await handleClaudeMessages(new Request("http://localhost/v1/messages", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "combo/pair", max_tokens: 128, stream: true, messages: COMBO_MESSAGES }), + }), config, { model: "", provider: "" }, { requestId: `combo-floor-${crypto.randomUUID()}`, start: Date.now() }); + expect(response.status).toBe(200); + const text = await response.text(); + const startFrame = text.slice(text.indexOf("event: message_start")); + const published = (JSON.parse(startFrame.slice(startFrame.indexOf("data: ") + 6, startFrame.indexOf("\n\n"))) + .message.usage.input_tokens) as number; + // The hop really happened: the first target refused once, the second served once. + expect(firstBodies).toHaveLength(1); + expect(secondBodies).toHaveLength(1); + return { published, secondBodies }; + } finally { + first.stop(true); + secondServer.stop(true); + clearComboSelectionState(); + clearComboTargetCooldowns(); + restoreIsolatedHome(); + } +} + +test("a combo failover publishes the floor of the target that answered, not the ingress pick", async () => { + // The ingress route names the combo's first target; the physical send names whichever target + // answered. Those are different bodies when the targets sit on different wires, and a memo read + // from the ingress pick prices the wrong one: here target A is a Chat wire that would drop the + // replayed thinking entirely, while target B is an Anthropic wire that forwards it verbatim. + // A floor left at the ingress pick therefore understates a prompt B really received — the + // mirror image of the over-count this file pins, and just as wrong for a context meter. + const { published, secondBodies } = await comboFailoverFloor({ + provider: "b", model: "m2", adapter: "anthropic", frames: ANTHROPIC_SSE_FRAMES, + }); + // B's body is the caller's, replayed thinking and signature included. + const forwarded = JSON.stringify(secondBodies[0]!.messages); + expect(forwarded).toContain("replayed reasoning"); + expect(forwarded).toContain("S".repeat(64)); + + const nativeWire = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", CLAUDE_NATIVE_THINKING); + const chatWire = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", { text: false, signature: false, redacted: false }); + // The ablation: the ingress pick is the Chat target, whose projection prices this body at a + // rounding error next to what B received. Holding the floor above it is what a memo keyed on + // the settled wire buys. + expect(chatWire).toBeLessThan(nativeWire / 100); + expect(published).toBeGreaterThan(chatWire * 20); + expect(published).toBeLessThan(nativeWire * 1.5); + expect(published).toBeGreaterThan(nativeWire * 0.5); +}, { timeout: SERVER_BUDGET_MS }); + +test("a combo failover to a registry provider prices its merged preserve list", async () => { + // `preserveReasoningContentModels` is not a config-row field on most providers: it is merged + // in from the registry by `routedProviderConfig`. A dispatch that reads the raw row therefore + // prices a preserve-listed model as if its reasoning were dropped, which is the under-count + // this pair of cases exists to prevent. `moonshot` is a real registry entry whose endpoint a + // user may override, so a loopback row reaches the same merge path production does. + const { published, secondBodies } = await comboFailoverFloor({ + provider: "moonshot", model: "kimi-k3", adapter: "openai-chat", frames: CHAT_SSE_FRAMES, + }); + // The merged list is what made the adapter serialize the replayed text at all. + const forwarded = JSON.stringify(secondBodies[0]!.messages); + expect(forwarded).toContain("reasoning_content"); + expect(forwarded).toContain("replayed reasoning"); + // The signature has no Chat representation, so it is the one field the list does not buy. + expect(forwarded).not.toContain("S".repeat(64)); + + const textKept = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", { text: true, signature: false, redacted: false }); + const textDropped = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", { text: false, signature: false, redacted: false }); + expect(textKept).toBeGreaterThan(textDropped * 20); + expect(published).toBeGreaterThan(textDropped * 20); + expect(published).toBeLessThan(textKept * 1.5); + expect(published).toBeGreaterThan(textKept * 0.5); +}, { timeout: SERVER_BUDGET_MS }); + +test("a combo target named by alias prices the preserve list under its resolved id", async () => { + // A combo target may name a model by alias (`am`), and the Chat adapter resolves that to the + // provider's native id before it decides whether the wire serializes replayed thinking — its + // preserve list holds native ids, and matching is exact. An attempt that records the alias + // therefore reads the preserve list under a name that is not in it, prices the replayed text as + // dropped, and understates a prompt the upstream really received. The attempt row has to carry + // the id the adapter will actually send, which is what the routing result already holds. + const { published, secondBodies } = await comboFailoverFloor({ + provider: "aliased", + model: "am", + adapter: "openai-chat", + frames: CHAT_SSE_FRAMES, + models: ["aliased-model"], + modelAliases: { "aliased-model": "am" }, + preserveReasoningContentModels: ["aliased-model"], + }); + // The wire received the resolved id, and with it the replayed text the preserve list buys. + expect(secondBodies[0]!.model).toBe("aliased-model"); + const forwarded = JSON.stringify(secondBodies[0]!.messages); + expect(forwarded).toContain("reasoning_content"); + expect(forwarded).toContain("replayed reasoning"); + + const textKept = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", { text: true, signature: false, redacted: false }); + const textDropped = estimateClaudeRequestTokens({ messages: COMBO_MESSAGES }, "combo/pair", { text: false, signature: false, redacted: false }); + expect(textKept).toBeGreaterThan(textDropped * 20); + // The alias must not cost the projection the preserve list's verdict. + expect(published).toBeGreaterThan(textDropped * 20); + expect(published).toBeLessThan(textKept * 1.5); + expect(published).toBeGreaterThan(textKept * 0.5); +}, { timeout: SERVER_BUDGET_MS }); diff --git a/tests/config/client-config-export.test.ts b/tests/config/client-config-export.test.ts index 0b6fd2686a1..d1dd5ebca47 100644 --- a/tests/config/client-config-export.test.ts +++ b/tests/config/client-config-export.test.ts @@ -1138,7 +1138,9 @@ test("renamed CommandCode gathered effort tables reach DSH and ZCode exports", a const context = ctx({ models, config }); const dshConfig = dsh.buildDshClientConfig(context); const dshModels = Object.values(dshConfig["llm-pi-ai"].providers).flatMap(provider => provider.models); - expect(dshModels.find(model => model.id === `CommandCode/${known}`)?.reasoningEfforts).toEqual({ high: "high", max: "max" }); + expect(dshModels.find(model => model.id === `CommandCode/${known}`)?.reasoningEfforts).toEqual({ + low: "low", medium: "medium", high: "high", xhigh: "xhigh", max: "max", + }); expect(dshModels.find(model => model.id === `CommandCode/${overridden}`)?.reasoningEfforts).toEqual({ low: "low" }); expect(dshModels.find(model => model.id === "CommandCode/unknown-model")?.reasoningEfforts).toBeUndefined(); for (const id of [known, overridden, "unknown-model"]) { @@ -1146,7 +1148,7 @@ test("renamed CommandCode gathered effort tables reach DSH and ZCode exports", a } const zcodeModels = Object.assign({}, ...Object.values(zcode.buildZcodeClientConfig(context).provider).map(provider => provider.models)) as Record; expect(zcodeModels[`CommandCode/${known}`]).toBeDefined(); - expect(zcodeModels[`CommandCode/${known}`]!.reasoning?.variants).toEqual(["high", "max"]); + expect(zcodeModels[`CommandCode/${known}`]!.reasoning?.variants).toEqual(["low", "medium", "high", "xhigh", "max"]); expect(zcodeModels[`CommandCode/${overridden}`]).toBeDefined(); expect(zcodeModels[`CommandCode/${overridden}`]!.reasoning?.variants).toEqual(["low"]); expect(zcodeModels["CommandCode/unknown-model"]).toBeDefined(); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index a419094b1b4..ad60c306137 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1,4 +1,5 @@ { + "provider-antigravity-quota-retry.test.ts": "providers", "deepseek-artifact-tool-schema.test.ts": "providers", "client-config-export-output-limit.test.ts": "config", "openai-chat-serialized-tool-call-scaling.test.ts": "adapters/openai", @@ -292,6 +293,7 @@ "claude-picker-trust.test.ts": "claude-integration", "claude-desktop-remote-hub.test.ts": "claude-integration", "claude-dotenv-provenance-transport.test.ts": "claude-integration", + "claude-estimate-projection.test.ts": "claude-integration", "claude-first-party-union.test.ts": "claude-integration", "claude-gateway-cache.test.ts": "claude-integration", "claude-inbound-cache-stabilize.test.ts": "claude-integration", @@ -576,6 +578,7 @@ "combo-workspace-data.test.ts": "gui", "combos.test.ts": "codex-integration", "command-code-error-finish.test.ts": "providers", + "command-code-efforts.test.ts": "providers", "command-code-provider.test.ts": "providers", "command-code-quota.test.ts": "providers", "command-code-tool-text.test.ts": "providers", @@ -1461,6 +1464,7 @@ "server-auth.test.ts": "server", "server-background-lifecycle.test.ts": "server", "server-clickjacking-headers.test.ts": "server", + "server-combo-cooldown-fallback.test.ts": "server", "server-combo-failover-e2e.test.ts": "server", "server-combo-held-response.test.ts": "server", "server-combo-reasoning-replay-eligibility.test.ts": "server", diff --git a/tests/providers/command-code-efforts.test.ts b/tests/providers/command-code-efforts.test.ts new file mode 100644 index 00000000000..20f3eb845ee --- /dev/null +++ b/tests/providers/command-code-efforts.test.ts @@ -0,0 +1,96 @@ +import { afterEach, expect, test } from "bun:test"; +import { createCommandCodeAdapter } from "../../src/adapters/command-code"; +import { commandCodeReasoningEfforts, refreshCommandCodeReasoningEfforts, resetCommandCodeReasoningEffortsForTest } from "../../src/providers/command-code-efforts"; +import { PROVIDER_REGISTRY } from "../../src/providers/registry"; +import type { OcxParsedRequest } from "../../src/types"; + +// Issue #5096: measured accepted ladders, including the narrower exceptions. +const measured: Array<[string, string[]]> = [ + ["deepseek/deepseek-v4.1-flash", ["low", "medium", "high", "xhigh", "max"]], + ["deepseek/deepseek-v4-flash", ["low", "medium", "high", "xhigh", "max"]], + ["deepseek/deepseek-v4-flash-vision-exp", ["low", "medium", "high", "xhigh", "max"]], + ["z-ai/glm-5.3-flash", ["low", "medium", "high", "xhigh", "max"]], + ["zai-org/GLM-5.3", ["low", "medium", "high", "xhigh", "max"]], + ["Qwen/Qwen3.8-Flash", ["low", "medium", "high", "xhigh", "max"]], + ["google/gemini-3.7-flash", ["low", "medium", "high", "xhigh", "max"]], + ["moonshotai/Kimi-K3", ["low", "medium", "high", "xhigh", "max"]], + ["MiniMaxAI/MiniMax-M3", ["low", "medium", "high", "xhigh", "max"]], + ["xiaomi/mimo-v2.5", ["low", "medium", "high", "xhigh", "max"]], + ["xai/grok-4.5", ["low", "medium", "high", "xhigh", "max"]], + ["xai/grok-4.6", ["low", "medium", "high", "xhigh", "max"]], + ["tencent/hy3-paid", ["low", "medium", "high", "xhigh", "max"]], + ["tencent/hy4-preview", ["low", "medium", "high", "xhigh", "max"]], + ["stepfun/Step-3.7-Flash", ["low", "medium", "high", "xhigh", "max"]], + ["Qwen/Qwen3.8-Max", ["low", "medium", "high", "xhigh", "max"]], + ["Qwen/Qwen3.8-27B", ["low", "medium", "high", "xhigh", "max"]], + ["meta/muse-spark-1.2-contributor", ["low", "medium", "high", "xhigh", "max"]], + ["meta/muse-spark-1.3-contributor", ["low", "medium", "high", "xhigh", "max"]], + ["nvidia/nemotron-3-ultra-550b-a55b", ["low", "medium", "high", "xhigh", "max"]], + ["meituan/LongCat-2.0:free", ["low", "medium", "high", "xhigh", "max"]], + ["inclusionai/ling-3.0-flash-sante:free", ["low", "medium", "high", "xhigh", "max"]], + ["thinkingmachines/inkling-small", ["low", "medium", "high", "xhigh", "max"]], + ["moonshotai/Kimi-K2.7-Code", ["low", "medium", "high", "xhigh"]], + ["moonshotai/Kimi-K2.7-Code-Highspeed", ["low", "high", "xhigh", "max"]], + ["xiaomi/mimo-v2.5-pro", ["low", "medium", "high"]], + ["Qwen/Qwen3.7-32B", ["low", "medium", "high", "xhigh"]], + ["Qwen/Qwen3.7-72B", ["low", "medium", "high", "xhigh"]], + ["Qwen/Qwen3.6-35B-A22B", ["low", "medium", "high", "xhigh"]], + ["poolside/laguna-s-2.1-free", ["medium"]], + ["google/gemini-3.8-flash", ["low", "medium", "high"]], +]; + +const adapter = createCommandCodeAdapter({ adapter: "command-code", baseUrl: "https://api.commandcode.ai", apiKey: "synthetic-command-key" }); +function request(modelId: string, reasoning: string): OcxParsedRequest { + return { modelId, stream: true, context: { systemPrompt: [], messages: [], tools: [] }, + options: { reasoning, maxOutputTokens: 100 } }; +} +afterEach(() => resetCommandCodeReasoningEffortsForTest()); + +test.each(measured)("%s exposes and forwards every measured effort", async (modelId, ladder) => { + expect(commandCodeReasoningEfforts(modelId)).toEqual(ladder); + expect(commandCodeReasoningEfforts(modelId.toUpperCase())).toEqual(ladder); + for (const id of ["command-code", "commandcode"]) { + expect(PROVIDER_REGISTRY.find(entry => entry.id === id)?.modelReasoningEfforts?.[modelId]).toEqual(ladder); + } + for (const effort of ladder) { + const built = await adapter.buildRequest(request(modelId, effort)); + expect(JSON.parse(built.body).params.reasoning_effort).toBe(effort); + } + for (const effort of ["low", "medium", "high", "xhigh", "max"]) { + if (ladder.includes(effort)) continue; + const built = await adapter.buildRequest(request(modelId, effort)); + expect(JSON.parse(built.body).params).not.toHaveProperty("reasoning_effort"); + } +}); + +test("a measured row without a profile remembers rejections without guessing a URL", async () => { + const model = "moonshotai/Kimi-K3"; + let calls = 0; + const fetch = async () => { calls++; return new Response("", { status: 404 }); }; + expect(await refreshCommandCodeReasoningEfforts(model, fetch, "max")).toEqual(["low", "medium", "high", "xhigh"]); + expect(calls).toBe(0); + expect(commandCodeReasoningEfforts(model)).toEqual(["low", "medium", "high", "xhigh"]); + const built = await adapter.buildRequest(request(model, "max")); + expect(JSON.parse(built.body).params).not.toHaveProperty("reasoning_effort"); +}); + +test("the first rejected send for a measured row retries without its effort", async () => { + const sends: Array<{ url: string; body: string }> = []; + const fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + sends.push({ url, body: String(init?.body ?? "") }); + return sends.length === 1 + ? new Response(JSON.stringify({ error: "unsupported reasoning_effort" }), { status: 400 }) + : new Response("{}", { status: 200 }); + }) as typeof globalThis.fetch; + const sendingAdapter = createCommandCodeAdapter({ adapter: "command-code", baseUrl: "https://api.commandcode.ai", + apiKey: "synthetic-command-key", fetch } as Parameters[0] & { fetch: typeof globalThis.fetch }); + const built = await sendingAdapter.buildRequest(request("moonshotai/Kimi-K3", "max")); + expect(JSON.parse(built.body).params.reasoning_effort).toBe("max"); + const response = await sendingAdapter.fetchResponse(built); + expect(response.status).toBe(200); + expect(sends).toHaveLength(2); + expect(sends.every(send => send.url.endsWith("/alpha/generate"))).toBe(true); + expect(JSON.parse(sends[1]!.body).params).not.toHaveProperty("reasoning_effort"); + expect(commandCodeReasoningEfforts("moonshotai/Kimi-K3")).toEqual(["low", "medium", "high", "xhigh"]); +}); diff --git a/tests/providers/command-code-provider.test.ts b/tests/providers/command-code-provider.test.ts index 126f69649e0..577991a041a 100644 --- a/tests/providers/command-code-provider.test.ts +++ b/tests/providers/command-code-provider.test.ts @@ -110,7 +110,7 @@ describe("Command Code provider", () => { }); expect(registry?.models).toBeUndefined(); expect(registry?.modelReasoningEfforts).toMatchObject({ - "deepseek/deepseek-v4-flash": ["high", "max"], + "deepseek/deepseek-v4-flash": ["low", "medium", "high", "xhigh", "max"], "zai-org/GLM-5.2": ["high", "max"], }); expect(OAUTH_PROVIDERS["command-code"]?.providerConfig).toMatchObject({ @@ -138,7 +138,7 @@ describe("Command Code provider", () => { "zai-org/GLM-5": ["high", "max"], "zai-org/GLM-5.1": ["high", "max"], "zai-org/GLM-5.2-Fast": ["high", "max"], - "zai-org/GLM-5.3": ["low", "high", "max"], + "zai-org/GLM-5.3": ["low", "medium", "high", "xhigh", "max"], }); }); @@ -154,14 +154,14 @@ describe("Command Code provider", () => { const apiKey = PROVIDER_REGISTRY.find(row => row.id === "commandcode"); for (const [label, entry] of [["oauth", oauth], ["api-key", apiKey]] as const) { expect(entry?.modelReasoningEfforts?.["z-ai/glm-5.3-flash"], `${label} preset ladder`) - .toEqual(["low", "high", "max"]); + .toEqual(["low", "medium", "high", "xhigh", "max"]); } // Distinct rows for distinct upstream models: GLM-5.3 and GLM-5.3-Flash happen to // share a ladder today, but neither may be derived from the other. - expect(commandCodeReasoningEfforts("z-ai/glm-5.3-flash")).toEqual(["low", "high", "max"]); - expect(commandCodeReasoningEfforts("zai-org/GLM-5.3")).toEqual(["low", "high", "max"]); + expect(commandCodeReasoningEfforts("z-ai/glm-5.3-flash")).toEqual(["low", "medium", "high", "xhigh", "max"]); + expect(commandCodeReasoningEfforts("zai-org/GLM-5.3")).toEqual(["low", "medium", "high", "xhigh", "max"]); // The reported id arrives lowercase from live discovery; a caller may still fold case. - expect(commandCodeReasoningEfforts("Z-AI/GLM-5.3-Flash")).toEqual(["low", "high", "max"]); + expect(commandCodeReasoningEfforts("Z-AI/GLM-5.3-Flash")).toEqual(["low", "medium", "high", "xhigh", "max"]); // Nothing widened into a substring match: a sibling that upstream does not list // must stay unknown rather than inheriting the Flash ladder. expect(commandCodeReasoningEfforts("z-ai/glm-5.3-flash-vision")).toBeUndefined(); @@ -179,11 +179,11 @@ describe("Command Code provider", () => { const apiKey = PROVIDER_REGISTRY.find(row => row.id === "commandcode"); for (const [label, entry] of [["oauth", oauth], ["api-key", apiKey]] as const) { expect(entry?.modelReasoningEfforts?.["deepseek/deepseek-v4.1-flash"], `${label} preset ladder`) - .toEqual(["low", "high", "max"]); + .toEqual(["low", "medium", "high", "xhigh", "max"]); expect(entry?.modelReasoningEfforts?.["Qwen/Qwen3.8-Flash"], `${label} preset ladder`) .toEqual(["low", "medium", "high", "xhigh", "max"]); } - expect(commandCodeReasoningEfforts("deepseek/deepseek-v4.1-flash")).toEqual(["low", "high", "max"]); + expect(commandCodeReasoningEfforts("deepseek/deepseek-v4.1-flash")).toEqual(["low", "medium", "high", "xhigh", "max"]); expect(commandCodeReasoningEfforts("Qwen/Qwen3.8-Flash")).toEqual(["low", "medium", "high", "xhigh", "max"]); // The live-discovered id may arrive in any case; the lookup folds it. expect(commandCodeReasoningEfforts("qwen/qwen3.8-flash")).toEqual(["low", "medium", "high", "xhigh", "max"]); @@ -693,7 +693,7 @@ describe("Command Code provider", () => { }); test("does not advertise an unverified effort for models absent from the official table", async () => { - const built = await builtRequest(parsed("moonshotai/Kimi-K3")); + const built = await builtRequest(parsed("unknown/unmeasured-model")); expect(JSON.parse(built.body).params).not.toHaveProperty("reasoning_effort"); }); @@ -743,7 +743,7 @@ describe("Command Code provider", () => { options: { reasoning: "ultra", maxOutputTokens: 100 }, }); expect(JSON.parse(ultra.body).params).not.toHaveProperty("reasoning_effort"); - // Deepseek/glm still alias xhigh/ultra→max per their official profiles. + // DeepSeek retains the ultra alias but forwards its accepted xhigh rung unchanged. const deepseekUltra = await builtRequest({ ...parsed("deepseek/deepseek-v4-flash"), options: { reasoning: "ultra", maxOutputTokens: 100 }, @@ -753,14 +753,14 @@ describe("Command Code provider", () => { ...parsed("deepseek/deepseek-v4-flash"), options: { reasoning: "xhigh", maxOutputTokens: 100 }, }); - expect(JSON.parse(deepseekXhigh.body).params.reasoning_effort).toBe("max"); + expect(JSON.parse(deepseekXhigh.body).params.reasoning_effort).toBe("xhigh"); }); - test("maps ultra and xhigh to the max wire effort and honors legacy alias ids", async () => { + test("maps ultra to max, preserves xhigh, and honors legacy alias ids", async () => { const ultra = await builtRequest({ ...parsed(), options: { reasoning: "ultra", maxOutputTokens: 100 } }); expect(JSON.parse(ultra.body).params.reasoning_effort).toBe("max"); const xhigh = await builtRequest({ ...parsed(), options: { reasoning: "xhigh", maxOutputTokens: 100 } }); - expect(JSON.parse(xhigh.body).params.reasoning_effort).toBe("max"); + expect(JSON.parse(xhigh.body).params.reasoning_effort).toBe("xhigh"); // Legacy compatibility id resolves to the canonical effort table before the lookup. const legacy = await builtRequest({ ...parsed(), modelId: "deepseek-v4-flash" }); expect(JSON.parse(legacy.body).params.reasoning_effort).toBe("high"); @@ -843,7 +843,7 @@ describe("Command Code provider", () => { if (mode === "prepaid") expect(await response.text()).toContain("unsupported reasoning_effort"); else expect(JSON.parse(generated[1]!.body!).params).not.toHaveProperty("reasoning_effort"); } finally { dispose(); } - expect(commandCodeReasoningEfforts("deepseek/deepseek-v4-flash")).toEqual(["high"]); + expect(commandCodeReasoningEfforts("deepseek/deepseek-v4-flash")).toEqual(["low", "medium", "high", "xhigh"]); }); /* @@ -862,10 +862,10 @@ describe("Command Code provider", () => { * `modelReasoningEffortsAuthoritative` is never written by seeding, so its presence does. */ test("an authoritative operator ladder reaches the wire", async () => { - // Shipped: deepseek/deepseek-v4.1-flash is ["low", "high", "max"], so xhigh aliases to max. - expect(commandCodeReasoningEfforts("deepseek/deepseek-v4.1-flash")).toEqual(["low", "high", "max"]); + // Shipped: deepseek/deepseek-v4-flash-fast is ["low", "high", "max"], so xhigh aliases to max. + expect(commandCodeReasoningEfforts("deepseek/deepseek-v4-flash-fast")).toEqual(["low", "high", "max"]); const shipped = await builtRequest({ - ...parsed("deepseek/deepseek-v4.1-flash"), + ...parsed("deepseek/deepseek-v4-flash-fast"), options: { reasoning: "xhigh", maxOutputTokens: 100 }, }); expect(JSON.parse(shipped.body).params.reasoning_effort).toBe("max"); @@ -873,10 +873,10 @@ describe("Command Code provider", () => { const widened = createCommandCodeAdapter({ ...provider, modelReasoningEffortsAuthoritative: true, - modelReasoningEfforts: { "deepseek/deepseek-v4.1-flash": ["low", "medium", "high", "xhigh", "max"] }, + modelReasoningEfforts: { "deepseek/deepseek-v4-flash-fast": ["low", "medium", "high", "xhigh", "max"] }, } as OcxProviderConfig); const built = await widened.buildRequest({ - ...parsed("deepseek/deepseek-v4.1-flash"), + ...parsed("deepseek/deepseek-v4-flash-fast"), options: { reasoning: "xhigh", maxOutputTokens: 100 }, }); expect(JSON.parse(built.body).params.reasoning_effort).toBe("xhigh"); @@ -885,10 +885,10 @@ describe("Command Code provider", () => { const narrowed = createCommandCodeAdapter({ ...provider, modelReasoningEffortsAuthoritative: true, - modelReasoningEfforts: { "deepseek/deepseek-v4.1-flash": ["high"] }, + modelReasoningEfforts: { "deepseek/deepseek-v4-flash-fast": ["high"] }, } as OcxProviderConfig); const stripped = await narrowed.buildRequest({ - ...parsed("deepseek/deepseek-v4.1-flash"), + ...parsed("deepseek/deepseek-v4-flash-fast"), options: { reasoning: "max", maxOutputTokens: 100 }, }); expect(JSON.parse(stripped.body).params).not.toHaveProperty("reasoning_effort"); @@ -1054,18 +1054,18 @@ describe("Command Code provider", () => { const modelId = "deepseek/deepseek-v4-flash"; const alternate = "https://alternate.example/command-code"; const fetch = (async () => new Response("Reasoning efforts high, max are supported; no mapping.")) as typeof globalThis.fetch; - expect(await refreshCommandCodeReasoningEfforts(modelId, fetch, "max", provider.baseUrl)).toEqual(["high"]); - expect(commandCodeReasoningEfforts(modelId, alternate)).toEqual(["high", "max"]); + expect(await refreshCommandCodeReasoningEfforts(modelId, fetch, "max", provider.baseUrl)).toEqual(["low", "medium", "high", "xhigh"]); + expect(commandCodeReasoningEfforts(modelId, alternate)).toEqual(["low", "medium", "high", "xhigh", "max"]); const options = { reasoning: "max", maxOutputTokens: 100 }; const officialRequest = await createCommandCodeAdapter(provider).buildRequest({ ...parsed(modelId), options }); const alternateRequest = await createCommandCodeAdapter({ ...provider, baseUrl: alternate }).buildRequest({ ...parsed(modelId), options }); expect(JSON.parse(officialRequest.body).params).not.toHaveProperty("reasoning_effort"); expect(JSON.parse(alternateRequest.body).params.reasoning_effort).toBe("max"); - expect(await refreshCommandCodeReasoningEfforts(modelId, fetch, "high", alternate)).toEqual(["max"]); - expect(commandCodeReasoningEfforts(modelId, provider.baseUrl)).toEqual(["high"]); + expect(await refreshCommandCodeReasoningEfforts(modelId, fetch, "high", alternate)).toEqual(["low", "medium", "xhigh", "max"]); + expect(commandCodeReasoningEfforts(modelId, provider.baseUrl)).toEqual(["low", "medium", "high", "xhigh"]); }); - test("uses the 2026-09-23 profile ladders for newly cataloged models", async () => { + test("uses profile ladders plus measured corrections for newly cataloged models", async () => { const cases: Array<[string, string[]]> = [ ["claude-fable-5-1", ["low", "medium", "high", "xhigh", "max"]], ["claude-opus-5-5", ["low", "medium", "high", "xhigh", "max"]], @@ -1074,7 +1074,7 @@ describe("Command Code provider", () => { ["Qwen/Qwen3.8-Omni-Flash", ["low", "medium", "xhigh"]], ["Qwen/Qwen3.8-Max-0902", ["low", "medium", "xhigh"]], ["stepfun/Step-5-Preview", ["low", "medium", "high"]], - ["tencent/hy4-preview", ["low", "medium", "high"]], + ["tencent/hy4-preview", ["low", "medium", "high", "xhigh", "max"]], ["google/gemini-3.8-flash", ["low", "medium", "high"]], ["xai/grok-4.7", ["low", "medium", "high", "xhigh"]], ]; diff --git a/tests/providers/commandcode-provider.test.ts b/tests/providers/commandcode-provider.test.ts index b75324c973c..5061a08c64b 100644 --- a/tests/providers/commandcode-provider.test.ts +++ b/tests/providers/commandcode-provider.test.ts @@ -71,9 +71,9 @@ describe("Command Code provider", () => { apiKeyValidation: "unknown", reasoningEfforts: [], modelReasoningEfforts: { - "deepseek/deepseek-v4-flash-vision-exp": ["high", "max"], + "deepseek/deepseek-v4-flash-vision-exp": ["low", "medium", "high", "xhigh", "max"], "gpt-5.6-luna": ["low", "medium", "high", "xhigh", "max"], - "google/gemini-3.7-flash": ["low", "medium", "high"], + "google/gemini-3.7-flash": ["low", "medium", "high", "xhigh", "max"], }, modelDiscovery: { path: "models", @@ -216,7 +216,7 @@ describe("Command Code provider", () => { expect(deepseek.contextWindow).toBe(1_000_000); expect(deepseek.owned_by).toBe("command-code"); // #1800: discovered models now surface the curated effort table (command-code-efforts.ts). - expect(deepseek.reasoningEfforts).toEqual(["high", "max"]); + expect(deepseek.reasoningEfforts).toEqual(["low", "medium", "high", "xhigh", "max"]); const haiku = models.find(row => row.id === "claude-haiku-4-5-20251001")!; expect(haiku.contextWindow).toBe(200_000); @@ -228,7 +228,7 @@ describe("Command Code provider", () => { expect(models.find(row => row.id === "deepseek/deepseek-v4-flash-vision-exp")) .toMatchObject({ id: "deepseek/deepseek-v4-flash-vision-exp", - reasoningEfforts: ["high", "max"], + reasoningEfforts: ["low", "medium", "high", "xhigh", "max"], }); expect(models.find(row => row.id === "gpt-5.6-luna")).toMatchObject({ id: "gpt-5.6-luna", @@ -236,7 +236,7 @@ describe("Command Code provider", () => { }); expect(models.find(row => row.id === "google/gemini-3.7-flash")).toMatchObject({ id: "google/gemini-3.7-flash", - reasoningEfforts: ["low", "medium", "high"], + reasoningEfforts: ["low", "medium", "high", "xhigh", "max"], }); expect(models.find(row => row.id === "Qwen/Qwen3.8-Flash")?.reasoningEfforts) .toEqual(["low", "medium", "high", "xhigh", "max"]); diff --git a/tests/providers/devin-adapter-reset-wait.test.ts b/tests/providers/devin-adapter-reset-wait.test.ts index c4bf3a50815..c2a834afd80 100644 --- a/tests/providers/devin-adapter-reset-wait.test.ts +++ b/tests/providers/devin-adapter-reset-wait.test.ts @@ -1,15 +1,18 @@ /** - * The Devin adapter pins maxWaitMs to 0 when it calls the stated-reset retry - * helper, so an admitted HTTP turn never holds shared capacity while sleeping - * out a provider 429. The helper-level tests cannot see this: they pass their - * own maxWaitMs. This file drives the real adapter end to end and asserts the - * refusal surfaces without a replay — a mock.module spy would also work, but a + * The Devin adapter resolves its stated-reset wait allowance from + * OPENCODEX_DEVIN_STATED_RESET_WAIT_MS and defaults to 0, so an admitted HTTP + * turn never holds shared capacity while sleeping out a provider 429 unless + * the operator explicitly opts in. The helper-level tests cannot see this: + * they pass their own maxWaitMs. This file drives the real adapter end to end + * and asserts the default refusal surfaces without a replay while an opted-in + * allowance waits and replays — a mock.module spy would also work, but a * module mock registered at file scope leaks into every sibling test file Bun * loads into the same process. * - * Removing the adapter's maxWaitMs: 0 fails both tests fast: the helper would - * sleep out the stated window, the 5s abort signal cancels that sleep, and the - * turn ends in the 499 client-closed error instead of the provider's refusal. + * Dropping the adapter's wait resolution for the default case fails these + * tests fast: the helper would sleep out the stated window, the 5s abort + * signal cancels that sleep, and the turn ends in the 499 client-closed error + * instead of the provider's refusal. */ import { describe, expect, test, beforeEach, afterEach } from "bun:test"; import { mkdtempSync } from "node:fs"; @@ -29,6 +32,7 @@ const CHAT_URL = `${host}/exa.api_server_pb.ApiServerService/GetChatMessage`; let home = ""; const previousHome = process.env.OPENCODEX_HOME; const previousFetch = globalThis.fetch; +const previousWait = process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS; let chatPosts = 0; let seenUrls: string[] = []; @@ -70,7 +74,7 @@ function stubTransport(message: string): void { }) as typeof fetch; } -async function runOneTurn(): Promise { +async function runOneTurn(abortSignal: AbortSignal = AbortSignal.timeout(5_000), comboAttempt = false): Promise { const adapter = createDevinAdapter({ adapter: "devin", apiKey, baseUrl: host }); const events: AdapterEvent[] = []; await adapter.runTurn!({ @@ -81,7 +85,8 @@ async function runOneTurn(): Promise { }, { headers: new Headers(), translatorBudget: createTranslatorBudget(), - abortSignal: AbortSignal.timeout(5_000), + abortSignal, + comboAttempt, }, event => { events.push(event); }); return events; } @@ -90,6 +95,7 @@ describe("devin adapter stated-reset wait", () => { beforeEach(() => { home = mkdtempSync(join(tmpdir(), "ocx-devin-reset-wait-")); process.env.OPENCODEX_HOME = home; + delete process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS; seed(); }); afterEach(() => { @@ -97,6 +103,8 @@ describe("devin adapter stated-reset wait", () => { setCachedCatalogForTests(null); if (previousHome === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = previousHome; + if (previousWait === undefined) delete process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS; + else process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS = previousWait; removeTreeWithRetry(home); }); @@ -125,6 +133,43 @@ describe("devin adapter stated-reset wait", () => { expect(chatPosts).toBe(1); }); + test("an opted-in wait allowance sleeps out the stated reset and replays", async () => { + process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS = "3000"; + stubTransport("Your limit will reset in 1 second"); + + const events = await runOneTurn(AbortSignal.timeout(30_000)); + + expect(chatPosts).toBe(3); + expect(events.some(event => event.type === "heartbeat" && event.preflightReady === true)).toBe(true); + const error = events.find((event): event is Extract => event.type === "error"); + expect(error).toMatchObject({ status: 429, errorType: "rate_limit_error", code: "resource_exhausted" }); + expect(error?.message).toContain("retry after ~1s"); + expect(events.some(event => event.type === "done")).toBe(false); + }); + + test("a combo child returns the stated reset immediately despite a standalone wait allowance", async () => { + process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS = "3000"; + stubTransport("Your limit will reset in 1 second"); + + const events = await runOneTurn(AbortSignal.timeout(500), true); + + expect(events.find(event => event.type === "error")).toMatchObject({ + type: "error", status: 429, errorType: "rate_limit_error", code: "resource_exhausted", + }); + expect(events.some(event => event.type === "heartbeat" && event.preflightReady === true)).toBe(false); + expect(chatPosts).toBe(1); + }); + + test("an invalid wait allowance fails closed without replay", async () => { + process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS = "not-a-duration"; + stubTransport("Your limit will reset in 1 second"); + + const events = await runOneTurn(); + + expect(chatPosts).toBe(1); + expect(events.some(event => event.type === "error")).toBe(true); + }); + test("an alias-bound tenant refusal keeps one send and content-free reset timing", async () => { const tenantHost = "https://server.eu.windsurf.com"; await saveCredential("devin-cli", { diff --git a/tests/providers/devin-stated-reset-retry.test.ts b/tests/providers/devin-stated-reset-retry.test.ts index a94b7117862..75b4c37713f 100644 --- a/tests/providers/devin-stated-reset-retry.test.ts +++ b/tests/providers/devin-stated-reset-retry.test.ts @@ -4,7 +4,7 @@ * request replayed — but only while the stream produced zero events, and only * within the replay/wait bounds. */ -import { describe, expect, test } from "bun:test"; +import { describe, expect, jest, test } from "bun:test"; import { CloudChatError, type CloudChatEvent, type CloudChatRequest } from "../../src/adapters/devin/cloud-direct"; import { clearCachedCatalog } from "../../src/adapters/devin/cloud-direct/catalog"; import { streamChatEventsWithResetRetry } from "../../src/adapters/devin/cloud-direct/stated-reset-retry"; @@ -127,6 +127,39 @@ describe("streamChatEventsWithResetRetry", () => { expect(out.map(e => e.kind)).toEqual(["text", "finish"]); }); + test("wait heartbeats stay below the shortest stall budget and stop after sleep", async () => { + const waiting = Promise.withResolvers(); + const resume = Promise.withResolvers(); + let calls = 0; + let heartbeats = 0; + jest.useFakeTimers(); + try { + const pending = drain(streamChatEventsWithResetRetry(REQ, { + stream: () => ++calls === 1 + ? exhausting("Your limit will reset in 3 seconds")() + : events({ kind: "finish", reason: "stop" } as CloudChatEvent), + sleep: async ms => { waiting.resolve(ms); await resume.promise; }, + onWaitHeartbeat: () => { heartbeats += 1; }, + })); + + expect(await waiting.promise).toBe(3_000); + expect(heartbeats).toBe(1); + jest.advanceTimersByTime(500); + expect(heartbeats).toBe(2); + jest.advanceTimersByTime(500); + expect(heartbeats).toBe(3); + resume.resolve(); + expect((await pending).map(event => event.kind)).toEqual(["finish"]); + jest.advanceTimersByTime(1_000); + expect(heartbeats).toBe(3); + expect(calls).toBe(2); + } finally { + resume.resolve(); + jest.clearAllTimers(); + jest.useRealTimers(); + } + }); + test("waits the generated approximate retry delay and replays", async () => { const waits: number[] = []; let calls = 0; diff --git a/tests/providers/provider-account-quota.test.ts b/tests/providers/provider-account-quota.test.ts index 2ddae55d6dd..45f3e87e032 100644 --- a/tests/providers/provider-account-quota.test.ts +++ b/tests/providers/provider-account-quota.test.ts @@ -926,7 +926,7 @@ describe("google-antigravity per-account quota (#1082)", () => { }, }); expect(await fetchProviderAccountQuotas("google-antigravity")).toEqual([{ accountId: idFor("a@example.com"), quota: null, unavailable: true, quotaFailure: status < 400 ? "redirect_blocked" : "access_denied" }]); - expect(posted).toEqual(fallback ? [summaryUrl, modelsUrl] : [summaryUrl]); + expect(posted).toEqual(fallback ? [summaryUrl, modelsUrl] : status === 403 ? [summaryUrl, summaryUrl] : [summaryUrl]); expect(plainFetchCalls).toBe(0); }); } diff --git a/tests/providers/provider-antigravity-quota-retry.test.ts b/tests/providers/provider-antigravity-quota-retry.test.ts new file mode 100644 index 00000000000..6eea79bc3f9 --- /dev/null +++ b/tests/providers/provider-antigravity-quota-retry.test.ts @@ -0,0 +1,87 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { antigravityUserAgent } from "../../src/adapters/client-fingerprint"; +import { probeAntigravityUsageQuota, setAntigravityAccountQuotaTransportForTests } from "../../src/providers/quota/antigravity"; + +import { PROXY_ENV_KEYS } from "../../src/lib/proxy-env"; + +const proxyKeys = PROXY_ENV_KEYS.flatMap(key => [key, key.toLowerCase()]); +const originalProxyEnv = Object.fromEntries(proxyKeys.map(key => [key, process.env[key]])); +beforeEach(() => { for (const key of proxyKeys) delete process.env[key]; }); +afterEach(() => { + for (const key of proxyKeys) { + if (originalProxyEnv[key] === undefined) delete process.env[key]; + else process.env[key] = originalProxyEnv[key]; + } +}); + +const summaryUrl = "https://daily-cloudcode-pa.googleapis.com/v1internal:retrieveUserQuotaSummary"; +const modelsUrl = "https://daily-cloudcode-pa.googleapis.com/v1internal:fetchAvailableModels"; +const summary = { groups: [{ displayName: "Gemini", buckets: [{ window: "5h", remainingFraction: 0.6 }] }] }; +const models = { models: { gemini: { quotaInfo: { remainingFraction: 0.75 } } } }; + +function transport(responses: Array) { + const calls: Array<{ url: string; userAgent: string | null; authorization: string | null; body: string }> = []; + setAntigravityAccountQuotaTransportForTests({ + resolveAddresses: async () => ({ hostname: "daily-cloudcode-pa.googleapis.com", addresses: [{ address: "142.250.0.1", family: 4 }], privateNetwork: false }), + pinnedPost: async (url, _address, body, _signal, options) => { + const headers = new Headers(options?.headers); + calls.push({ url, userAgent: headers.get("user-agent"), authorization: headers.get("authorization"), body }); + const response = responses.shift(); + if (!response) throw new Error("Unexpected extra quota request"); + if (response instanceof Error) throw response; + return response; + }, + }); + return calls; +} + +function expectedCall(url: string, userAgent = antigravityUserAgent()) { + return { url, userAgent, authorization: "Bearer test-access", body: JSON.stringify({ project: "test-project" }) }; +} + +afterEach(() => setAntigravityAccountQuotaTransportForTests(null)); + +describe("Antigravity quota summary 403 compatibility retry (#5940)", () => { + for (const cancelFails of [false, true]) { + test(`retries once with identical bearer and project, even when cancellation ${cancelFails ? "fails" : "succeeds"}`, async () => { + let cancelled = false; + const denied = new Response(new ReadableStream({ cancel() { + cancelled = true; + if (cancelFails) throw new Error("cancel failed"); + } }), { status: 403 }); + const calls = transport([denied, Response.json(summary)]); + const result = await probeAntigravityUsageQuota("test-access", "test-project"); + expect(cancelled).toBe(true); + expect(calls).toEqual([expectedCall(summaryUrl), expectedCall(summaryUrl, "antigravity/1.0")]); + expect(result).toMatchObject({ kind: "available", source: "google-antigravity:retrieveUserQuotaSummary", quota: { customWindows: [{ label: "Gem", percent: 40 }] } }); + }); + } + + test("successful IDE summary needs no retry", async () => { + const calls = transport([Response.json(summary)]); + expect((await probeAntigravityUsageQuota("test-access", "test-project")).kind).toBe("available"); + expect(calls).toEqual([expectedCall(summaryUrl)]); + }); + + test("401 is not retried", async () => { + const calls = transport([new Response(null, { status: 401 })]); + expect(await probeAntigravityUsageQuota("test-access", "test-project")).toMatchObject({ kind: "unavailable", failure: "access_denied" }); + expect(calls).toEqual([expectedCall(summaryUrl)]); + }); + + for (const status of [401, 403, 302]) { + test(`retry status ${status} stops without further requests`, async () => { + const calls = transport([new Response(null, { status: 403 }), new Response(null, { status, headers: { location: "https://redirect.example/" } })]); + expect(await probeAntigravityUsageQuota("test-access", "test-project")).toMatchObject({ kind: "unavailable", failure: status === 302 ? "redirect_blocked" : "access_denied" }); + expect(calls).toEqual([expectedCall(summaryUrl), expectedCall(summaryUrl, "antigravity/1.0")]); + }); + } + + for (const failure of [new Error("transport failed"), new Response(null, { status: 500 }), Response.json({})]) { + test(`retry failure (${failure instanceof Error ? "transport" : failure.status}) recovers through IDE models probe`, async () => { + const calls = transport([new Response(null, { status: 403 }), failure, Response.json(models)]); + expect(await probeAntigravityUsageQuota("test-access", "test-project")).toMatchObject({ kind: "available", source: "google-antigravity:fetchAvailableModels", quota: { customWindows: [{ label: "Gem", percent: 25 }] } }); + expect(calls).toEqual([expectedCall(summaryUrl), expectedCall(summaryUrl, "antigravity/1.0"), expectedCall(modelsUrl)]); + }); + } +}); diff --git a/tests/providers/provider-quota.test.ts b/tests/providers/provider-quota.test.ts index 139daa50d04..2235e33f7b4 100644 --- a/tests/providers/provider-quota.test.ts +++ b/tests/providers/provider-quota.test.ts @@ -3557,7 +3557,7 @@ describe("fetchProviderQuotaReports", () => { pinnedPost: async url => { posted.push(url); return new Response(null, { status, headers: { location: modelsUrl } }); }, }); expect((await fetchProviderQuotaReports(config(), true)).reports).toEqual([]); - expect(posted).toEqual([summaryUrl]); + expect(posted).toEqual(status === 403 ? [summaryUrl, summaryUrl] : [summaryUrl]); expect(plainFetchCalls).toEqual([]); }); } diff --git a/tests/providers/provider-registry-parity.test.ts b/tests/providers/provider-registry-parity.test.ts index 36ab8d2b5c4..d6b75c5565f 100644 --- a/tests/providers/provider-registry-parity.test.ts +++ b/tests/providers/provider-registry-parity.test.ts @@ -1652,7 +1652,7 @@ describe("provider registry parity", () => { provider: "commandcode", }); expect(model.id).toBe("z-ai/glm-5.3-flash"); - expect(model.reasoningEfforts).toEqual(["low", "high", "max"]); + expect(model.reasoningEfforts).toEqual(["low", "medium", "high", "xhigh", "max"]); const entries = buildCatalogEntries(nativeTemplate() as never, [], [model]); const entry = entries.find(e => e.slug === "commandcode/z-ai-glm-5.3-flash"); @@ -1661,7 +1661,7 @@ describe("provider registry parity", () => { expect(entry?.supported_reasoning_levels).not.toEqual([]); // Routed catalogs append the synthetic top rung, as every other routed row above does. expect((entry?.supported_reasoning_levels as { effort: string }[]).map(l => l.effort)) - .toEqual(["low", "high", "max", "ultra"]); + .toEqual(["low", "medium", "high", "xhigh", "max", "ultra"]); }); /* * #1043. Zen publishes no modality metadata, so the classification below is an @@ -1856,8 +1856,8 @@ describe("renamed fixed-key destination reasoning metadata", () => { test("fills known model tables and unknown-model default for CommandCode", () => { const provider = make(); enrichProviderFromRegistry("CommandCode", provider); - expect(configuredReasoningEfforts(provider, known)).toEqual(["high", "max"]); - expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "high", "max"]); + expect(configuredReasoningEfforts(provider, known)).toEqual(["low", "medium", "high", "xhigh", "max"]); + expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "medium", "high", "xhigh", "max"]); expect(configuredReasoningEfforts(provider, "unknown-model")).toEqual([]); }); test("preserves explicit entries and clones arrays without losing other table rows", () => { @@ -1870,7 +1870,7 @@ describe("renamed fixed-key destination reasoning metadata", () => { enrichProviderFromRegistry("CommandCode", provider); expect(provider).toEqual(once); expect(configuredReasoningEfforts(provider, known)).toEqual(["low"]); - expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "high", "max"]); + expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "medium", "high", "xhigh", "max"]); expect(configuredReasoningEfforts(provider, "custom")).toEqual([]); expect(configuredReasoningEfforts(provider, "unknown-model")).toEqual(["medium"]); provider.modelReasoningEfforts![known]!.push("high"); @@ -1882,7 +1882,7 @@ describe("renamed fixed-key destination reasoning metadata", () => { const provider = make({ modelReasoningEfforts: { [known]: [] } }); enrichProviderFromRegistry("CommandCode", provider); expect(configuredReasoningEfforts(provider, known)).toEqual([]); - expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "high", "max"]); + expect(configuredReasoningEfforts(provider, newer)).toEqual(["low", "medium", "high", "xhigh", "max"]); }); test("does not infer metadata for a different adapter, OAuth, or unrelated endpoint", () => { for (const override of [{ adapter: "openai-responses" }, { authMode: "oauth" as const }, { baseUrl: "https://example.test/v1" }]) { diff --git a/tests/responses/responses-grok-devin-preflight.test.ts b/tests/responses/responses-grok-devin-preflight.test.ts index 9ba3704e42b..0647cfbd4c2 100644 --- a/tests/responses/responses-grok-devin-preflight.test.ts +++ b/tests/responses/responses-grok-devin-preflight.test.ts @@ -2,6 +2,7 @@ import { afterEach, beforeEach, expect, mock, test } from "bun:test"; import type { ProviderAdapter } from "../../src/adapters/base"; import type { AdapterEvent, OcxConfig, OcxProviderConfig } from "../../src/types"; import { saveCredential } from "../../src/oauth/store"; +import { clearGenericFailoverHealth } from "../../src/oauth/generic-account-failover"; import { SEND_BUDGET_EXHAUSTED_CODE } from "../../src/lib/errors"; import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; import { createTempHome } from "../helpers/temp-home"; @@ -36,6 +37,7 @@ let release: (() => void) | undefined; beforeEach(async () => { home = createTempHome("ocx-grok-devin-preflight-"); release = acquireOwnedSpendHome(); + clearGenericFailoverHealth(); calls = 0; events = [limit]; blockedRun = undefined; @@ -48,6 +50,7 @@ afterEach(() => { try { release?.(); } finally { + clearGenericFailoverHealth(); home.remove(); } }); @@ -103,6 +106,106 @@ test.each([ expect(calls).toBe(1); }); +test("buffered cooldown heartbeat keeps a final refusal as HTTP 429", async () => { + events = [{ type: "heartbeat", preflightReady: true }, limit]; + + const response = await run({ stream: false }); + + expect(response.status).toBe(429); + expect(response.headers.get("retry-after")).toBe("60"); + expect(calls).toBe(1); +}); + +test.each(["codex", "grok"] as const)("%s cooldown starts SSE before a later 429 rotates to a second account", async surface => { + await saveCredential("devin", { + access: "synthetic-devin-preflight-spare", refresh: "synthetic-refresh-spare", + expires: Date.now() + 3_600_000, accountId: "fixture-spare", + }); + const started = Promise.withResolvers(); + const continueTurn = Promise.withResolvers(); + blockedRun = async (_parsed, _incoming, emit) => { + if (calls === 1) { + emit({ type: "heartbeat", preflightReady: true }); + started.resolve(); + await continueTurn.promise; + emit(limit); + return; + } + emit({ type: "text_delta", text: "alternate answer" }); + emit({ type: "done" }); + }; + + const response = await waitForPreflightResponse( + run({ surface, oauthFailoverEnabled: true }), started.promise, () => continueTurn.resolve(), + ); + const body = await response.text(); + + expect(response.status).toBe(200); + expect(calls).toBe(2); + expect(body).toContain("alternate answer"); + expect(body).not.toContain("rate_limit_exceeded"); + expect(body).toContain("response.completed"); +}); + +test("a final cooldown refusal after SSE starts remains an in-stream error", async () => { + events = [{ type: "heartbeat", preflightReady: true }, limit]; + + const response = await run(); + const body = await response.text(); + + expect(response.status).toBe(200); + expect(response.headers.get("retry-after")).toBeNull(); + expect(body).toContain("response.failed"); + expect(calls).toBe(1); +}); + +test("a replay-unsafe heartbeat after cooldown readiness forbids account rotation", async () => { + await saveCredential("devin", { + access: "synthetic-devin-preflight-spare", refresh: "synthetic-refresh-spare", + expires: Date.now() + 3_600_000, accountId: "fixture-spare", + }); + events = [ + { type: "heartbeat", preflightReady: true }, + { type: "heartbeat", replayUnsafe: true }, + limit, + ]; + + const response = await run({ surface: "codex", oauthFailoverEnabled: true }); + + expect(response.status).toBe(200); + expect(await response.text()).toContain("response.failed"); + expect(calls).toBe(1); +}); + +test("a timed-out preflight can rotate a later pre-output 429", async () => { + await saveCredential("devin", { + access: "synthetic-devin-preflight-spare", refresh: "synthetic-refresh-spare", + expires: Date.now() + 3_600_000, accountId: "fixture-spare", + }); + const started = Promise.withResolvers(); + const continueTurn = Promise.withResolvers(); + blockedRun = async (_parsed, _incoming, emit) => { + if (calls === 1) { + started.resolve(); + await continueTurn.promise; + emit(limit); + return; + } + emit({ type: "text_delta", text: "after timeout" }); + emit({ type: "done" }); + }; + + const response = await waitForPreflightResponse( + run({ stallTimeoutSec: 1, oauthFailoverEnabled: true }), started.promise, () => continueTurn.resolve(), + ); + const body = await response.text(); + + expect(response.status).toBe(200); + expect(calls).toBe(2); + expect(body).toContain("after timeout"); + expect(body).toContain("response.completed"); +}); + test.each([false, true])("first text is replayed once and later errors stay SSE (failure=%s)", async failure => { events = [{ type: "heartbeat" }, { type: "text_delta", text: "answer" }, failure ? limit : { type: "done" }]; const response = await run(); diff --git a/tests/server/context-history-ownership.test.ts b/tests/server/context-history-ownership.test.ts index 04c10475847..86b320a30f2 100644 --- a/tests/server/context-history-ownership.test.ts +++ b/tests/server/context-history-ownership.test.ts @@ -4,8 +4,11 @@ import { join } from "node:path"; import { tmpdir } from "node:os"; import { handleResponses } from "../../src/server/responses"; import { handleContextHistory } from "../../src/server/context-history"; -import { tryAdmitTurn } from "../../src/server/lifecycle"; -import type { DataPlaneAdmission } from "../../src/server/auth-cors"; +import { getActiveTurnCount, tryAdmitTurn } from "../../src/server/lifecycle"; +import { requestPolicyView, resolveApiAuth, type DataPlaneAdmission } from "../../src/server/auth-cors"; +import { saveConfig } from "../../src/config"; +import { linkStorePath } from "../../src/link/paths"; +import { readLinkStore, writeLinkStore } from "../../src/link/store"; import { saveCodexAccountCredential } from "../../src/codex/account-store"; import { clearAccountNeedsReauth } from "../../src/codex/account-runtime-state"; import { clearAccountQuota, setAccountQuotaFromParsed } from "../../src/codex/quota"; @@ -23,6 +26,7 @@ const destination = "https://chatgpt.com/backend-api/codex"; const originalFetch = globalThis.fetch; let previousHome: string | undefined; let previousCodexHome: string | undefined; +let previousAdminToken: string | undefined; let home = ""; let sent: Array<{ url: string; headers: Headers }> = []; let failFirstAccount: string | undefined; @@ -75,6 +79,7 @@ function setContextFeature(enabled: boolean): void { beforeEach(() => { previousHome = process.env.OPENCODEX_HOME; previousCodexHome = process.env.CODEX_HOME; + previousAdminToken = process.env.OPENCODEX_ADMIN_AUTH_TOKEN; home = mkdtempSync(join(tmpdir(), "ocx-context-owner-")); process.env.OPENCODEX_HOME = home; process.env.CODEX_HOME = home; // Direct handler dispatches need the writer lease that startServer normally holds. @@ -110,6 +115,8 @@ afterEach(() => { removeTreeWithRetry(home); if (previousHome === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = previousHome; if (previousCodexHome === undefined) delete process.env.CODEX_HOME; else process.env.CODEX_HOME = previousCodexHome; + if (previousAdminToken === undefined) delete process.env.OPENCODEX_ADMIN_AUTH_TOKEN; + else process.env.OPENCODEX_ADMIN_AUTH_TOKEN = previousAdminToken; }); test("successful explicit account A owns context while active B remains selected", async () => { @@ -176,3 +183,122 @@ test("Direct proxy-bearer model and notes use stored main without leaking the pr expect(sent.map(row => row.headers.get("authorization"))).toEqual([`Bearer ${token}`, `Bearer ${token}`]); expect(sent.every(row => row.headers.get("chatgpt-account-id") === "physical-main")).toBe(true); }); + +test("post-body admission revalidation consults the live link policy, not the request-entry snapshot", async () => { + // The hub-link listener resolves its policy at request entry and again inside the context + // relay's post-body revalidation. This drives that gate with the same requestPolicyView/ + // resolveApiAuth pair the listener uses, so a key revoked mid-request must stop dispatch. + const LINK_KEY = "link-linked-revoke"; + const LINK_ID = "linked-key"; + const cfg = config(); + cfg.apiKeys = [{ id: LINK_ID, name: LINK_ID, key: LINK_KEY, createdAt: "2026-09-26T00:00:00.000Z" }]; + const linkIngress = { allowedKeyIds: new Set([LINK_ID]) }; + const linkPolicy = () => requestPolicyView(cfg, "opencodex-link.invalid", linkIngress); + + const linkRequest = (session: string) => new Request("http://opencodex-link.invalid/v1/alpha/notes/v2/read_file", { + method: "POST", + headers: { + "content-type": "application/json", + "x-opencodex-api-key": LINK_KEY, + authorization: "Bearer caller-native-token", + "chatgpt-account-id": "caller-account", + }, + body: JSON.stringify({ context: { session_id: session } }), + }); + const contextNotes = async (req: Request, admission: DataPlaneAdmission, revalidate: () => DataPlaneAdmission | null) => { + const lease = tryAdmitTurn(); expect(lease).not.toBeNull(); + try { + return await handleContextHistory(req, cfg, { model: "context_history", provider: "" }, + "alpha/notes/v2/read_file", lease!, admission, revalidate); + } finally { lease?.release(); } + }; + + const entryPolicy = linkPolicy(); + const entryAdmission = resolveApiAuth(linkRequest("root-link"), entryPolicy); + expect(entryAdmission?.contextPrincipalId).toBeDefined(); + // Record this principal's session owner the same way the ownership tests do: one model turn. + expect((await model(cfg, "root-link", "side/gpt-5.5", requestHeaders("root-link"), entryAdmission!)).status).toBe(200); + + // Revoke the key mid-request: the next policy rebuild no longer resolves this credential. + cfg.apiKeys = cfg.apiKeys?.filter(k => k.id !== LINK_ID); + + // Fixed wiring: the closure consults the live policy and the revoked key cannot dispatch. + { + const req = linkRequest("root-link"); + const denied = await contextNotes(req, entryAdmission!, () => resolveApiAuth(req, linkPolicy())); + expect(denied.status).toBe(401); + } + // Pre-fix wiring kept the request-entry snapshot and still dispatched upstream. + { + const req = linkRequest("root-link"); + const admitted = await contextNotes(req, entryAdmission!, () => resolveApiAuth(req, entryPolicy)); + expect(admitted.status).toBe(200); + } + expect(sent.filter(row => row.url.includes("/alpha/"))).toHaveLength(1); +}); + +test("a real link listener refuses a revoked key after reading a delayed context body", async () => { + const linkId = "linked-listener-key"; + const linkKey = "link-listener-revoke"; + const sessionId = "root-delayed-listener"; + const cfg = config(); + cfg.runtimeRole = "hub"; + cfg.apiKeys = [{ id: linkId, name: linkId, key: linkKey, createdAt: "2026-09-26T00:00:00.000Z" }]; + const admissionRequest = new Request("http://opencodex-link.invalid/v1/responses", { + headers: { "x-opencodex-api-key": linkKey }, + }); + const entryPolicy = requestPolicyView(cfg, "opencodex-link.invalid", { allowedKeyIds: new Set([linkId]) }); + const admission = resolveApiAuth(admissionRequest, entryPolicy); + expect(admission).not.toBeNull(); + expect((await model(cfg, sessionId, "side/gpt-5.5", requestHeaders(sessionId), admission!)).status).toBe(200); + const upstreamBefore = sent.length; + + process.env.OPENCODEX_ADMIN_AUTH_TOKEN = "test-listener-admin-token"; + saveConfig(cfg); + writeLinkStore(linkStorePath(), { + version: 1, listenerPort: null, + links: [{ id: "lnk_0123456789abcdef", alias: "delayed-test", direction: "client-initiated", + hostKeyFingerprint: "SHA256:abcdefghijklmnop", tunnelPort: 2222, apiKeyId: linkId, + createdAt: "2026-09-26T00:00:00.000Z" }], + }); + releaseSpendHome?.(); + releaseSpendHome = undefined; + const { startServer } = await import("../../src/server"); + const server = startServer(0); + let bodyController: ReadableStreamDefaultController | undefined; + try { + const port = readLinkStore(linkStorePath()).listenerPort; + expect(port).not.toBeNull(); + const body = new ReadableStream({ + start(controller) { + bodyController = controller; + controller.enqueue(new TextEncoder().encode('{"context":')); + }, + }); + const pending = originalFetch(`http://127.0.0.1:${port}/v1/alpha/notes/v2/read_file`, { + method: "POST", + headers: { "content-type": "application/json", "x-opencodex-api-key": linkKey, + authorization: "Bearer caller-native-token", "chatgpt-account-id": "caller-account" }, + body, + duplex: "half", + } as RequestInit & { duplex: "half" }); + const waitUntil = Date.now() + 5_000; + while (getActiveTurnCount() === 0 && Date.now() < waitUntil) await Bun.sleep(5); + expect(getActiveTurnCount()).toBe(1); + + const revoked = await originalFetch(new URL("/api/keys", server.url), { + method: "DELETE", + headers: { "content-type": "application/json", "x-opencodex-api-key": "test-listener-admin-token" }, + body: JSON.stringify({ id: linkId }), + }); + expect(revoked.status).toBe(200); + bodyController!.enqueue(new TextEncoder().encode(`{"session_id":"${sessionId}"}}`)); + bodyController!.close(); + bodyController = undefined; + expect((await pending).status).toBe(401); + expect(sent).toHaveLength(upstreamBefore); + } finally { + try { bodyController?.close(); } catch { /* already closed after an early response */ } + await server.stop(true); + } +}); diff --git a/tests/server/link-listener-admission.test.ts b/tests/server/link-listener-admission.test.ts index 004cee3f24a..0bda9ca8991 100644 --- a/tests/server/link-listener-admission.test.ts +++ b/tests/server/link-listener-admission.test.ts @@ -1,12 +1,14 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test"; -import { mkdtempSync, writeFileSync } from "node:fs"; +import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import type { OcxConfig } from "../../src/types"; import { linkStorePath } from "../../src/link/paths"; import { emptyLinkStore, writeLinkStore } from "../../src/link/store"; +import { requestPolicyView, resolveApiAuth } from "../../src/server/auth-cors"; import { installIsolatedCodexHome, type IsolatedCodexHome } from "../helpers/isolated-codex-home"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { repoPath } from "../helpers/repo-root"; const ENV_KEY = "link-env-admission"; const OTHER_KEY = "link-other-admission"; @@ -152,6 +154,19 @@ afterEach(async () => { }); describe("hub-link admission", () => { + test("context revalidation refreshes the link policy after asynchronous request work", () => { + const source = readFileSync(repoPath("src/server/index/serve-options.ts"), "utf8"); + expect(source).toContain('() => resolveApiAuth(req, ingress === "hub-link" ? linkPolicy() : policy)'); + + const liveConfig = config(); + const req = new Request("http://opencodex-link.invalid/v1/alpha/notes/v2/read_file", { headers: headers(LINKED_KEY) }); + const initialPolicy = requestPolicyView(liveConfig, "opencodex-link.invalid", { allowedKeyIds: new Set([LINKED_ID]) }); + expect(resolveApiAuth(req, initialPolicy)?.kind).toBe("configured"); + liveConfig.apiKeys = liveConfig.apiKeys?.filter(key => key.id !== LINKED_ID); + const refreshedPolicy = requestPolicyView(liveConfig, "opencodex-link.invalid", { allowedKeyIds: new Set([LINKED_ID]) }); + expect(resolveApiAuth(req, refreshedPolicy)).toBeNull(); + }); + test("applies the four-credential matrix on every allowlisted route", async () => { const linkPort = JSON.parse(await Bun.file(linkStorePath()).text()).listenerPort as number; const base = `http://127.0.0.1:${linkPort}`; diff --git a/tests/server/link-listener-lifecycle.test.ts b/tests/server/link-listener-lifecycle.test.ts index 1c6c5d4ee8c..87953ba05af 100644 --- a/tests/server/link-listener-lifecycle.test.ts +++ b/tests/server/link-listener-lifecycle.test.ts @@ -3,7 +3,9 @@ import { mkdtempSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import type { Server } from "bun"; -import { createLinkListenerLifecycle } from "../../src/server/index/link-listener"; +import type { OcxConfig } from "../../src/types"; +import { createLinkListenerLifecycle, linkListenerOwnsTarget } from "../../src/server/index/link-listener"; +import { createOptionalListenerSet } from "../../src/server/index/optional-listeners"; import { emptyLinkStore, type LinkStore } from "../../src/link/store"; import { removeTreeWithRetry } from "../helpers/remove-tree"; @@ -53,6 +55,44 @@ afterEach(async () => { }); describe("hub-link listener lifecycle", () => { + test("optional listener start only starts its supervisor after a successful bind", async () => { + tempHome = mkdtempSync(join(tmpdir(), "ocx-link-optional-bind-")); + const current = { value: store() }; + let bindFails = true; + let supervisorStarts = 0; + const listeners = createOptionalListenerSet({ + storePath: join(tempHome, "links.json"), + readStore: () => current.value, + writeStore: (_path, next) => { current.value = next; }, + warn: () => {}, + serve: options => { + if (bindFails) throw Object.assign(new Error("address already in use"), { code: "EADDRINUSE" }); + const bound = Bun.serve(options); + servers.push(bound); + return bound; + }, + }); + listeners.linkSupervisor().start = () => { supervisorStarts += 1; }; + const cfg = { port: 0, defaultProvider: "mock", providers: { + mock: { adapter: "openai-chat", baseUrl: "https://example.test/v1" }, + } } as OcxConfig; + const start = { config: cfg, publicPort: 0, requestedPort: 0, + maxRequestBodySize: 1024, dispatch: context().dispatch }; + try { + listeners.start(start); + expect(listeners.status()).toEqual({ state: "failed", port: null, reason: "bind" }); + expect(supervisorStarts).toBe(0); + + bindFails = false; + listeners.start(start); + expect(listeners.status().state).toBe("listening"); + expect(supervisorStarts).toBe(1); + expect(current.value.listenerPort).toBe(listeners.status().port); + } finally { + await listeners.stop(); + } + }); + test("degrades a bind collision while an independent public listener stays healthy", async () => { tempHome = mkdtempSync(join(tmpdir(), "ocx-link-bind-")); const publicServer = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("public-ok") }); @@ -77,6 +117,12 @@ describe("hub-link listener lifecycle", () => { expect(lifecycle.status()).toEqual({ state: "off", port: null, reason: null }); }); + test("only a successfully bound listener owns a reverse-forward target", () => { + expect(linkListenerOwnsTarget({ state: "failed", port: null, reason: "bind" })).toBe(false); + expect(linkListenerOwnsTarget({ state: "off", port: null, reason: null })).toBe(false); + expect(linkListenerOwnsTarget({ state: "listening", port: 45678, reason: null })).toBe(true); + }); + test("closes the real link socket when listenerPort persistence fails", async () => { tempHome = mkdtempSync(join(tmpdir(), "ocx-link-write-")); const publicServer = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: () => new Response("public-ok") }); diff --git a/tests/server/link-management-routes.test.ts b/tests/server/link-management-routes.test.ts index 2cabc5d852c..100c31e7797 100644 --- a/tests/server/link-management-routes.test.ts +++ b/tests/server/link-management-routes.test.ts @@ -215,6 +215,22 @@ describe("link management routes", () => { expect(h.store.links).toHaveLength(1); }); + test("issue starts the supervisor once a recovered listener binds", async () => { + temp = mkdtempSync(join(tmpdir(), "ocx-link-issue-recover-")); + const h = harness(); + h.setListenerState("failed"); + const deps = { + ...h.deps, + linkListener: () => ({ + ...h.listener, + ensureStarted: async () => { h.events.push("listener"); h.setListenerState("listening"); }, + }), + }; + const response = await call("/api/link/issue", "POST", { alias: "home", tunnelPort: 2200 }, deps, "admin-token", true, null, true, h.config); + expect(response?.status).toBe(200); + expect(h.events.indexOf("listener")).toBeLessThan(h.events.indexOf("supervisor")); + }); + test("rejects issue when ensureStarted leaves the listener failed and compensates", async () => { temp = mkdtempSync(join(tmpdir(), "ocx-link-listener-failed-")); const h = harness(); diff --git a/tests/server/server-combo-cooldown-fallback.test.ts b/tests/server/server-combo-cooldown-fallback.test.ts new file mode 100644 index 00000000000..76cabb070c0 --- /dev/null +++ b/tests/server/server-combo-cooldown-fallback.test.ts @@ -0,0 +1,258 @@ +import { afterAll, afterEach, beforeEach, describe, expect, mock, setDefaultTimeout, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ManagementRequest as Request } from "../helpers/management-auth"; +import { comboProviderFactory } from "../helpers/combo-provider"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "../helpers/isolated-codex-home"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { clearComboSelectionState, clearComboTargetCooldowns } from "../../src/combos"; +import { clearComboRecallForTests } from "../../src/server/responses/combo-session-recall"; +import { clearKeyCooldowns } from "../../src/providers/key-failover"; +import { clearCodexUpstreamHealth } from "../../src/codex/routing"; +import { clearRequestLogsForTests, type RequestLogContext } from "../../src/server/request-log"; +import { + clearResponseStateForTests, + flushResponseState, + responseStatePersistPendingForTests, +} from "../../src/responses/state"; +import type { ProviderAdapter } from "../../src/adapters/base"; +import type { AdapterEvent, OcxConfig, OcxProviderConfig } from "../../src/types"; + +// `mock.module` outlives this file: Bun keeps the override below for every file that runs after +// this one in the same process. This is a spread snapshot of the real module, taken before it. +const actualResolver = { ...(await import("../../src/server/adapter-resolve")) }; +const actualResolveAdapter = actualResolver.resolveAdapter; +let customRunTurn: NonNullable | undefined; + +mock.module("../../src/server/adapter-resolve", () => ({ + ...actualResolver, + resolveAdapter(provider: OcxProviderConfig, cacheRetention?: "none" | "short" | "long") { + if (provider.adapter !== "test-run-turn") { + return actualResolveAdapter(provider, cacheRetention); + } + return { + name: "test-run-turn", + buildRequest: () => ({ url: provider.baseUrl, method: "POST", headers: {}, body: "" }), + async *parseStream(): AsyncGenerator { + yield { type: "error", message: "test runTurn adapter does not use parseStream" }; + }, + async runTurn(parsed, incoming, emit) { + if (!customRunTurn) throw new Error("custom runTurn not installed"); + await customRunTurn(parsed, incoming, emit); + }, + } satisfies ProviderAdapter; + }, +})); + +afterAll(() => { // Put the real module back for every later file in the same process. + mock.module("../../src/server/adapter-resolve", () => actualResolver); +}); + +const { handleResponses } = await import("../../src/server/responses"); + +/** + * Cooldown-readiness combo failover: a first target that reports ready and then fails + * must still hand off to the next combo target. + * + * This case was written in `server-combo-failover-e2e.test.ts` and moved here. That + * file carries a file-size-ratchet cap that the new test pushed past its ceiling, so + * the case lives in this sibling file instead of raising the cap. + * + * The harness below is the subset of that file's fixture these cases actually use: real + * loopback upstreams, an isolated home, and the combo/request-log state that leaks + * between tests. The loopback cases drive real adapters; the runTurn case uses the same + * narrow resolver seam as the parent file to emit deterministic adapter events. + */ + +// The parent file raises this for the same reason: a real loopback server plus combo +// failover exceeds the 5s default under full-suite load on Windows. +setDefaultTimeout(30_000); + +let testDir = ""; +let previousHome: string | undefined; +let isolatedCodexHome: IsolatedCodexHome | null = null; +const servers: Array> = []; +const provider = comboProviderFactory(() => undefined); +let releaseSpendHome: (() => void) | undefined; + +beforeEach(() => { + previousHome = process.env.OPENCODEX_HOME; + isolatedCodexHome = installIsolatedCodexHome("ocx-combo-zero-output-codex-"); + testDir = mkdtempSync(join(tmpdir(), "ocx-combo-zero-output-")); + process.env.OPENCODEX_HOME = testDir; + // Direct handler dispatches need the writer lease that startServer normally holds. + releaseSpendHome = acquireOwnedSpendHome(); + clearComboSelectionState(); + clearComboRecallForTests(); + clearComboTargetCooldowns(); + clearKeyCooldowns(); + clearCodexUpstreamHealth(); + clearRequestLogsForTests(); + clearResponseStateForTests(); +}); + +afterEach(async () => { + customRunTurn = undefined; + // Release before home teardown to prevent Windows removal failures and a live unlinked database. + releaseSpendHome?.(); + releaseSpendHome = undefined; + let responseStatePending = true; + try { + for (const server of servers.splice(0)) await server.stop(true); + await flushResponseState(); + responseStatePending = responseStatePersistPendingForTests(); + } finally { + clearResponseStateForTests(); + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (testDir) removeTreeWithRetry(testDir); + clearComboSelectionState(); + clearComboRecallForTests(); + clearComboTargetCooldowns(); + clearKeyCooldowns(); + clearCodexUpstreamHealth(); + clearRequestLogsForTests(); + } + expect(responseStatePending).toBe(false); +}); + +/** Loopback upstream whose lifetime the afterEach owns. */ +function serve(handler: (request: Request) => Response | Promise) { + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, fetch: handler }); + servers.push(server); + return server; +} + +describe("combo cooldown-ready fallback", () => { + test("runTurn cooldown-ready heartbeat preserves combo fallback on a final 429", async () => { + let firstHits = 0; + let backupHits = 0; + customRunTurn = async (_parsed, _incoming, emit) => { + firstHits += 1; + emit({ type: "heartbeat", preflightReady: true }); + emit({ type: "error", status: 429, errorType: "rate_limit_error", message: "stated reset still active" }); + }; + const backup = serve(() => { + backupHits += 1; + return new Response([ + `data: ${JSON.stringify({ choices: [{ delta: { content: "backup after cooldown" } }] })}`, + "data: [DONE]", + "", + ].join("\n"), { headers: { "content-type": "text/event-stream" } }); + }); + const config = comboConfig({ + a: provider("test-run-turn", "test://run-turn", "key-a"), + b: provider("openai-chat", baseUrl(backup), "key-b"), + }); + + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "combo/free", input: "hello", stream: true }), + }), config, { model: "", provider: "" }); + const body = await response.text(); + + expect(response.status).toBe(200); + expect(firstHits).toBe(1); + expect(backupHits).toBe(1); + expect(body).toContain("backup after cooldown"); + expect(body).not.toContain("stated reset still active"); + }); + + test("a delayed stated reset does not hold a combo child before fallback", async () => { + const delayedReset = Promise.withResolvers(); + const firstStarted = Promise.withResolvers(); + let firstHits = 0; + let backupHits = 0; + customRunTurn = async (_parsed, incoming, emit) => { + firstHits += 1; + if (!incoming.comboAttempt) { + emit({ type: "heartbeat", preflightReady: true }); + firstStarted.resolve(); + await delayedReset.promise; + } else { + firstStarted.resolve(); + } + emit({ type: "error", status: 429, errorType: "rate_limit_error", message: "stated reset still active" }); + }; + const backup = serve(() => { + backupHits += 1; + return new Response([ + `data: ${JSON.stringify({ choices: [{ delta: { content: "backup after delayed reset" } }] })}`, + "data: [DONE]", + "", + ].join("\n"), { headers: { "content-type": "text/event-stream" } }); + }); + const config = comboConfig({ + a: provider("test-run-turn", "test://run-turn", "key-a"), + b: provider("openai-chat", baseUrl(backup), "key-b"), + }); + const pending = handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "combo/free", input: "hello", stream: true }), + }), config, { model: "", provider: "" }); + await firstStarted.promise; + let guard: ReturnType | undefined; + let response: Response | null; + try { + response = await Promise.race([ + pending, + new Promise(resolve => { guard = setTimeout(() => resolve(null), 2_500); }), + ]); + } finally { + if (guard !== undefined) clearTimeout(guard); + delayedReset.resolve(); + } + if (response === null) { + await (await pending).body?.cancel(); + throw new Error("combo held its response until the delayed reset arrived"); + } + const body = await response.text(); + expect(response.status).toBe(200); + expect(firstHits).toBe(1); + expect(backupHits).toBe(1); + expect(body).toContain("backup after delayed reset"); + expect(body).not.toContain("stated reset still active"); + }); +}); +/** Provider base URL for a fixture server, without the trailing slash. */ +function baseUrl(server: ReturnType): string { + return `${server.url.toString().replace(/\/$/, "")}/v1`; +} + +/** Minimal completed Responses payload the backup target answers with. */ +function responsesSuccess(text: string, model = "responses-model"): Record { + return { + id: `resp-${model}`, + object: "response", + status: "completed", + model, + output: [{ + id: "msg_backup", + type: "message", + role: "assistant", + status: "completed", + content: [{ type: "output_text", text, annotations: [] }], + }], + usage: { input_tokens: 2, output_tokens: 1, total_tokens: 3 }, + }; +} + +/** Failover combo over the supplied providers, one target per provider in order. */ +function comboConfig( + providers: OcxConfig["providers"], + targets = Object.keys(providers).map((name, index) => ({ provider: name, model: `m${index + 1}` })), + extra: Partial[string]> = {}, +): OcxConfig { + return { + port: 0, + defaultProvider: Object.keys(providers)[0]!, + providers, + combos: { free: { strategy: "failover", targets, ...extra } }, + }; +} diff --git a/tests/test-layout-tooling.test.ts b/tests/test-layout-tooling.test.ts index 792859a3683..ac8d023d6fa 100644 --- a/tests/test-layout-tooling.test.ts +++ b/tests/test-layout-tooling.test.ts @@ -273,6 +273,8 @@ describe("membership oracle", () => { "ci-structure-gate.test.ts": "ci-workflows", "responses-code-mode-patch-compile.test.ts": "responses", "gui-codex-usage-score-parity.test.ts": "gui", + "server-combo-cooldown-fallback.test.ts": "server", + "claude-estimate-projection.test.ts": "claude-integration", } as const; const classified = Object.fromEntries( Object.keys(owners).map(name => [name, layout.explicit[name] ?? null]),