Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions devlog/_plan/260926_claude_input_estimate/000_overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# A Claude input estimate the settled route actually sends

Unit opened 2026-09-26.

- [010_estimation.md](010_estimation.md) — the Messages ingress. **DONE.**

Origin: a Paseo agent on a DeepSeek V4.1 Flash conversation through this proxy drew its context
meter at 221%. The rate implied the agent had run far past the window without compacting, so the
report read as a broken compaction loop. Compaction was healthy; the number above it was not. Same
body, same upstream: the proxy published 432,068 input tokens on `message_start` for a prompt the
upstream billed at 131,907 — **3.28x**.

This is the `#4857` family (the floor `message_start` publishes when the upstream has sent no
confirmed usage before the first frame), but not a recurrence: `#4891` and `#5057` fixed *when* the
floor is used and *whose* count it reports. The floor faithfully reports
`estimateClaudeRequestTokens`, and that estimate was measuring the wrong body.
65 changes: 65 additions & 0 deletions devlog/_plan/260926_claude_input_estimate/010_estimation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# A Claude input estimate the settled route actually sends

Defect: `src/server/claude-messages.ts` `estimateClaudeRequestTokens`. The estimator measured the
body the **caller sent**, while `message_start` publishes it as the floor for the prompt this proxy
**actually forwarded**. On the Anthropic wire those are the same body. On every other wire they are
not, and the gap is whatever the target adapter drops.

Measured on the live path with a real Claude Code conversation replayed byte-identical (260
messages, 23 tools, 1,734,433 B) to `thehive/deepseek-ai/deepseek-v4.1-flash`
(`adapter: openai-chat`): `message_start` **432,068** against the upstream's `message_delta`
**131,907** — **3.28x**. The contract the estimator's own doc cites allows `>2x` drift
(`devlog/_fin/260711_claude_inbound/040_phase4_hardening.md` §3); this is outside it.

Why the two differ. Claude Code replays its own thinking blocks, and they dominate a long body:
80 blocks, 550,930 thinking chars and 750,284 `signature` chars. In the captured request the
thinking JSON is **78.8%** of all message JSON and the base64 `signature` is **56.7%** of the
thinking JSON. The `openai-chat` wire forwards that text only when the model is listed in
`preserveReasoningContentModels`, and it has no `signature` field at all — zero `signature`
references exist anywhere under `src/adapters/openai-chat*`. The `thehive` provider config declares
no reasoning policy keys, so it drops both, and upstreams do not bill replayed reasoning they never
receive. The estimator was counting 432,068 tokens of a prompt whose real size was 131,907.

The gap is a property of the route, not a constant, so no divisor can absorb it: a route that does
replay thinking must keep counting it. Char-per-token calibration and the `thehive/` alias prefix
were both ruled out as causes — CJK is 0.07% of the body, and moving 4 to 3.5 chars/token is ±14%,
while dropping the alias alone makes the number 14% *worse*.

Change:

- New leaf `src/lib/claude-request-projection.ts`: `ClaudeThinkingProjection`, the native
`{text:true, signature:true}`, `projectBlock`, and `projectClaudeRequest` — a pure, idempotent
projection that returns message content with the blocks a wire does not carry emptied. It never
mutates its input and survives a message whose content is not an array.
- `src/adapters/openai-chat/messages.ts`: new exported `openAIChatSerializesThinking(provider,
modelId)`, the single source of truth for whether that wire carries a replayed thinking block.
The `messagesToChatFormat` conversion reads the same answer once per request
(`wireSerializesThinking`) instead of re-deriving it per message, so the estimator and the
serializer cannot drift apart — they are one rule.
- `src/server/claude-messages.ts`: `estimateClaudeRequestTokens` takes an optional
`thinking: ClaudeThinkingProjection` defaulting to the native pair, and projects the messages
before measuring. `claudeRequestTokenFloor` passes the projection for the route the turn settled
on (`settledRoute`, recorded where routing resolves), and `handleClaudeCountTokens` resolves its
own route through the read-only `previewRouteModel`. A route that is not `openai-chat` keeps the
native projection, so the Anthropic-native lane stays byte-exact and pre-existing two-argument
callers keep their behavior.

The measurement only. The caller's body is never rewritten on its way to the adapter — the
projection exists to price the prompt, not to alter it, and every reader of the floor shares the one
estimate, so no number is double-counted.

Rejected: excluding replayed thinking unconditionally (wrong for the native lane and any adapter
that does replay it); a `billsReplayedReasoning?: boolean` flag on `ProviderAdapter` (states a
billing policy on an interface whose business is wire shape, and cannot express "text yes,
signature no", which is the shape that actually occurs).

Tests (new `tests/claude-integration/claude-estimate-projection.test.ts`): a replayed-thinking body
projects away exactly the unserialized fields; `projectClaudeRequest` is pure, idempotent, and keeps
a message it emptied; a body with no thinking is projection-invariant; and end-to-end, the
`message_start` floor for a real captured body lands within the confirmed usage instead of 3.28x
above it. Verified to fail when the projection is disabled — the end-to-end case reports 6946 where
it expects <42, so the assertion is load-bearing rather than decorative.

Live verification, same captured body against the real upstream (104,318 / 131,907 = **0.79x**, and
`message_delta` byte-identical at 131,907, so the upstream result is untouched and only the
published estimate moved). The Paseo meter that opened this unit reads 240% before and 58% after.
24 changes: 14 additions & 10 deletions docs-site/src/content/docs/reference/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -528,16 +528,20 @@ configuration that names the old id is rewritten at startup.
`CompletionConfiguration`, #2 is the output cap and #3 is the context window; swapping those two
makes every turn fail with an opaque `invalid_argument`. A temperature of exactly 0 is refused, so
it is clamped to the smallest accepted value.
- A pre-output 429 that states a recovery delay is retried in place only when the full stated
delay fits within the remaining cumulative wait allowance. The adapter waits that full delay
and replays the request up to twice; the default cumulative allowance is 30 minutes
(`OPENCODEX_DEVIN_STATED_RESET_WAIT_MS`, hard ceiling one hour). If the delay exceeds the
remaining allowance, the original 429 is surfaced without waiting or replaying. Retrying
earlier than the stated delay is deliberately not attempted — the hint is the provider's best
estimate of its own window, and each replay slot is finite. If the limit still refuses, the
final 429 surfaces to the client with the stated delay preserved as its cooldown hint. A `~`
in the surfaced message marks a delay recovered from a secondhand trailer sentence rather
than an exact header value; clients still receive the parsed number itself.
- A pre-output 429 with a stated recovery delay is surfaced immediately by default, releasing the
admitted turn's shared capacity. Set `OPENCODEX_DEVIN_STATED_RESET_WAIT_MS` to a positive cumulative
allowance in milliseconds to wait for the full stated delay and replay the same request up to twice.
The allowance has a one-hour ceiling; an absent, empty, invalid, or negative value disables waiting.
An opted-in standalone wait keeps the HTTP turn and its shared active-turn slot open throughout the delay.
Streaming turns start SSE on a safe cooldown heartbeat, then schedule heartbeats every 500 ms or less
during the wait so the stall watchdog stays fed. A later pre-output 429 may still rotate to another
eligible OAuth account; without one it is reported inside the already-open stream. Buffered Grok
turns retain an HTTP 429 and `Retry-After` on a final refusal.
Combo children surface the pre-output 429 immediately, even when waiting is enabled, so the combo
can try its next target without holding an uncommitted response. Delays exceeding the remaining
allowance on standalone turns surface the original 429 without an early retry. The
final 429 preserves the stated delay as a cooldown hint. A `~` in its message marks a delay recovered
from a secondhand trailer sentence rather than an exact header value.
- Experimental unofficial bridge; not shown in the dashboard preset by default. See the
[provider guide](/guides/providers/) for login instructions.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -286,12 +286,21 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity
| `directGeminiWireRenames?` | `boolean` | Google only. Applies only to direct AI Studio requests. Omitted or `true` keeps the `-tiered` wire rename for Gemini Flash ids (`gemini-3.7-flash` -> `gemini-3.7-flash-tiered`); `false` sends the requested bare ids to the wire unchanged. Vertex preserves the requested model ID, and Cloud Code Assist routing is unchanged. Set `false` when the configured upstream still serves the bare ids. |
| `project?` | `string` | Vertex or Antigravity Cloud Code Assist project id. |
| — | — | Antigravity account quota probes (`retrieveUserQuota` and `retrieveUserQuotaSummary`) always go to Google's own Cloud Code host through the pinned outbound transport, regardless of a configured `baseUrl`; the account bearer is never sent to an operator-configured endpoint and a redirect aborts the probe. Only the model-list fallback still honors `baseUrl`. |
| — | — | If Antigravity quota summary returns 403 for a valid OAuth account, OpenCodex retries that endpoint once with the legacy `antigravity/1.0` User-Agent and the same token and project. A 401 is not retried. Inference and model discovery retain the IDE User-Agent. |
| `location?` | `string` | Vertex location; environment fallback is `GOOGLE_CLOUD_LOCATION`. |
| `mcpServers?` | `Record<string, CursorMcpServerConfig>` | Cursor only: stdio or Streamable HTTP MCP servers. |
| `desktopExecutor?` | `DesktopExecutorConfig` | Cursor only: external computer-use and record-screen commands. |
| `unsafeAllowNativeLocalExec?` | `boolean` | Cursor legacy boolean, equivalent to `nativeLocalExec: "on"` only when the newer field is unset. |
| `nativeLocalExec?` | `"off" \| "codex-sandbox" \| "on"` | Cursor local-exec policy. `off` is default; `codex-sandbox` currently fails closed like `off`. |

Command Code's shipped per-model effort defaults include live API measurements. DeepSeek
v4/v4.1 Flash (including v4 Flash Vision), GLM-5.3 and GLM-5.3-Flash, Qwen3.8-Flash,
and Gemini-3.7-Flash support `low`, `medium`, `high`, `xhigh`, and `max`. Ladders remain
model-specific: `poolside/laguna-s-2.1-free` offers only `medium`, while Gemini-3.8-Flash
and MiMo-v2.5-Pro offer `low`, `medium`, and `high`. To override a pinned Command Code
row, set `modelReasoningEffortsAuthoritative: true` together with that model's
`modelReasoningEfforts` list.

Provider registration and replacement (`POST /api/providers`) validate `responsesPath` and `chatCompletionsPath` before changing live configuration or disk state. `PATCH /api/providers?name=<provider>` merges the request body with the stored provider; updates touching fields beyond `disabled` — except `requestPacing`-only updates — validate the merged provider's paths the same way before saving, and an invalid retained path returns `400` with the configuration unchanged. The same path rules apply when loading a configuration file.

### What a provider save keeps
Expand Down
4 changes: 4 additions & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@
}
},
"explicit": {
"provider-antigravity-quota-retry.test.ts": "providers",
"deepseek-artifact-tool-schema.test.ts": "providers",
"client-config-export-output-limit.test.ts": "config",
"openai-chat-serialized-tool-call-scaling.test.ts": "adapters/openai",
Expand Down Expand Up @@ -466,6 +467,7 @@
"claude-picker-trust.test.ts": "claude-integration",
"claude-desktop-remote-hub.test.ts": "claude-integration",
"claude-dotenv-provenance-transport.test.ts": "claude-integration",
"claude-estimate-projection.test.ts": "claude-integration",
"claude-first-party-union.test.ts": "claude-integration",
"claude-gateway-cache.test.ts": "claude-integration",
"claude-inbound-cache-stabilize.test.ts": "claude-integration",
Expand Down Expand Up @@ -750,6 +752,7 @@
"combo-workspace-data.test.ts": "gui",
"combos.test.ts": "codex-integration",
"command-code-error-finish.test.ts": "providers",
"command-code-efforts.test.ts": "providers",
"command-code-provider.test.ts": "providers",
"command-code-quota.test.ts": "providers",
"command-code-tool-text.test.ts": "providers",
Expand Down Expand Up @@ -1635,6 +1638,7 @@
"server-auth.test.ts": "server",
"server-background-lifecycle.test.ts": "server",
"server-clickjacking-headers.test.ts": "server",
"server-combo-cooldown-fallback.test.ts": "server",
"server-combo-failover-e2e.test.ts": "server",
"server-combo-held-response.test.ts": "server",
"server-combo-reasoning-replay-eligibility.test.ts": "server",
Expand Down
2 changes: 2 additions & 0 deletions src/adapters/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ export interface IncomingMeta {
headers: Headers;
translatorBudget: TranslatorBudget;
abortSignal?: AbortSignal;
/** Combo children must surface a pre-output refusal so the selector can try the next target. */
comboAttempt?: boolean;
/**
* Provider-scoped fetch prepared by the Responses router. Stateful transports that emit more
* than one physical HTTP request per logical turn must reuse it so every request participates in
Expand Down
15 changes: 9 additions & 6 deletions src/adapters/devin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import type { AdapterEvent, OcxAssistantMessage, OcxContentPart, OcxMessage, OcxParsedRequest, OcxProviderConfig, OcxTool, OcxToolCall, OcxToolResultMessage, OcxUsage } from "../types";
import { namespacedToolName } from "../types";
import type { IncomingMeta, ProviderAdapter } from "./base";
import { streamChatEventsWithResetRetry, allocateCascadeId, CloudChatError, type ChatHistoryItem, type ToolDef } from "./devin/cloud-direct";
import { streamChatEventsWithResetRetry, devinStatedResetWaitMs, allocateCascadeId, CloudChatError, type ChatHistoryItem, type ToolDef } from "./devin/cloud-direct";
import type { ContentPart } from "./devin/cloud-direct/chat";
import { getCachedCatalog, type CacheEntry } from "./devin/cloud-direct/catalog";
import { collapseDevinModelUid } from "./devin/live-models";
Expand Down Expand Up @@ -642,11 +642,12 @@ export function createDevinAdapter(
const maxOutputTokens = resolveDevinMaxOutputTokens(
provider, modelUid, parsed.options.maxOutputTokens,
);
// A combo child has not committed an outer response yet. Holding its preflight through
// a reset wait would also hold the next-target fallback with no client keepalive.
const resetWaitMs = incoming.comboAttempt ? 0 : devinStatedResetWaitMs();
// An admitted HTTP turn owns globally shared capacity until this call
// emits. Never retain that capacity while waiting out a provider 429:
// preserve the typed reset delay in generated diagnostic wording,
// never the raw trailer text that may reflect a credential. The
// refusal returns immediately so the caller can release its slot.
// emits. Without an explicit wait allowance, preserve the typed reset
// delay in generated diagnostic wording and return immediately.
for await (const event of streamChatEventsWithResetRetry({
apiKey,
apiServerUrl: host,
Expand All @@ -665,7 +666,9 @@ export function createDevinAdapter(
},
signal: incoming.abortSignal,
}, {
maxWaitMs: 0,
maxWaitMs: resetWaitMs,
onWaitHeartbeat: resetWaitMs > 0 && parsed.stream
? () => emit({ type: "heartbeat", preflightReady: true }) : undefined,
execution: {
executor: incoming.providerFetch,
sendBudget: incoming.sendBudget,
Expand Down
1 change: 1 addition & 0 deletions src/adapters/devin/cloud-direct/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ export {

export {
streamChatEventsWithResetRetry,
devinStatedResetWaitMs,
STATED_RESET_MAX_REPLAYS,
STATED_RESET_MAX_WAIT_MS,
type StatedResetRetryOptions,
Expand Down
22 changes: 18 additions & 4 deletions src/adapters/devin/cloud-direct/stated-reset-retry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,26 @@ export const STATED_RESET_MAX_WAIT_MS = 1_800_000;
/** Absolute maximum cumulative allowance, including explicit overrides. */
export const STATED_RESET_WAIT_CEILING_MS = 3_600_000;

function statedResetMaxWaitMs(): number {
function statedResetMaxWaitMs(defaultMs = STATED_RESET_MAX_WAIT_MS): number {
const raw = process.env.OPENCODEX_DEVIN_STATED_RESET_WAIT_MS?.trim();
if (!raw) return STATED_RESET_MAX_WAIT_MS;
if (!raw) return defaultMs;
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed < 0) return STATED_RESET_MAX_WAIT_MS;
if (!Number.isFinite(parsed) || parsed < 0) return defaultMs;
// Zero explicitly disables local waiting. Values above one hour are capped.
return Math.min(Math.floor(parsed), STATED_RESET_WAIT_CEILING_MS);
}
export const statedResetMaxWaitMsForTests = statedResetMaxWaitMs;

export function devinStatedResetWaitMs(): number {
return statedResetMaxWaitMs(0);
}

export interface StatedResetRetryOptions {
/** Test seam: defaults to the real cloud stream. */
stream?: (req: CloudChatRequest) => AsyncGenerator<CloudChatEvent>;
/** Test seam: must either honour the whole delay or reject on cancellation. */
sleep?: (ms: number, signal?: AbortSignal) => Promise<void>;
onWaitHeartbeat?: () => void;
maxReplays?: number;
/** CUMULATIVE wait allowance, not a fresh allowance on every failure. */
maxWaitMs?: number;
Expand Down Expand Up @@ -136,7 +141,16 @@ export async function* streamChatEventsWithResetRetry(
// scheduling: waking a few milliseconds late must not reject an already
// approved one-hour retry. No later wait can spend this allowance again.
waitedMs += waitMs;
await sleep(waitMs, req.signal);
const heartbeat = options?.onWaitHeartbeat;
if (waitMs > 0) heartbeat?.();
const beat = heartbeat && waitMs > 0
? setInterval(heartbeat, Math.min(500, Math.max(100, Math.floor(waitMs / 2))))
: undefined;
try {
await sleep(waitMs, req.signal);
} finally {
if (beat !== undefined) clearInterval(beat);
}
if (req.signal?.aborted) throw abortError(req.signal);
}
}
Expand Down
Loading
Loading