Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs-site/src/content/docs/guides/codex-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,14 @@ echo `default`, so request logs show the response tier as an observation with co
`assumed`. For latency-sensitive work, compare observed first-output times across the providers you
actually use rather than assuming any particular channel is faster.

For a gateway forwarding to a backend with the same metadata limitation, explicitly declare
[`responseTierAuthoritative: false`](/reference/configuration/providers/#response-service-tier-authority)
on that provider. The request still sends priority and records the raw echo, while actual Fast
scheduling remains unconfirmed. Undeclared gateways and the official API keep their existing
response-based interpretation. Updating OpenCodex alone does not add this declaration to existing
gateway entries: without it, an eligible priority request followed by a `default` echo still records
`response-declined`.

The proxy listens on port `10100` by default and serves `POST /v1/responses`,
`POST /v1/responses/compact`, `POST /v1/images/generations`, `POST /v1/images/edits`,
`GET /v1/models`, `GET /healthz`, and the `/api/*` management surface.
Expand Down
39 changes: 39 additions & 0 deletions docs-site/src/content/docs/reference/configuration/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,7 @@ Providers can expose a built-in shorthand, such as `agy` for `google-antigravity
| `allowEncryptedV2AgentTasks?` | `boolean` | Disabled by default. Trust a direct key-auth `openai-responses` provider to consume or relay opaque encrypted V2 sub-agent tasks unchanged. Eligible routes skip `agentTaskRecovery`; all other routes keep the existing recovery or fail-closed behavior. OpenCodex does not decrypt, translate, or recover tasks sent through this opt-in. |
| `upstreamWebsocket?` | `boolean` | Opt-in upstream Responses WebSocket transport for `openai-responses` requests (default false). Honored only for the first-party `https://api.openai.com/v1` upstream; custom-provider endpoints always use bounded HTTP/SSE because Bun cannot enforce an inbound WebSocket message limit before allocating the complete message. For the canonical ChatGPT `openai` provider, omitting it keeps the upstream WebSocket on eligible turns, an explicit `false` sends streaming turns over HTTP/SSE, and provider management rejects `true`; with `false`, native mid-turn steering and injection are unavailable. The field is independent of the client-facing `websockets` setting and changes neither the endpoint nor the credential. Plain HTTP remains on SSE; non-Responses paths and `openai-chat` requests stay on HTTP. |
| `supportsServiceTier?` | `boolean` | Tri-state canonical Fast capability fallback. `true` publishes Fast in the catalog, satisfies service-tier routing requirements, contributes a supported fingerprint, and lets fast mode inject the provider's canonical wire value on a compatible final adapter. `false` strips the field and never injects, and exact model declarations cannot reopen it. Absent leaves the provider unclassified: fast mode does not inject or normalize a canonical caller value, and caller values obey the final wire's forwarding permission (`chatServiceTier` on Chat; passthrough on Responses). The registry classifies canonical OpenAI (`true`), DeepSeek, and Volcengine Ark (`false`); set it explicitly only for custom gateways that genuinely support tiers. |
| `responseTierAuthoritative?` | `boolean` | Whether the response tier can confirm or deny Fast. Set `false` explicitly for routes with non-authoritative response metadata; omission preserves existing behavior. This does not enable Fast or change outbound parameters. See [Response service-tier authority](#response-service-tier-authority). |
| `fastEnabled?` | `boolean` | Operator switch for the provider's Fast lane. `false` turns Fast off (no Fast toggle, no `--fast` row, no fast wire field) and overrides `supportsServiceTier`. `true` enables a lane the registry marks opt-in. Absent keeps the registry default: off for `anthropic` and `anthropic-apikey`, whose fast mode spends usage credits at 2x price, and unchanged for every other provider. The dashboard Models page shows an Off/On row for opt-in providers. |
| `modelSupportsServiceTier?` | `Record<string, boolean>` | Exact upstream model capability overrides. Exact `true` enables canonical Fast for that model; exact `false` narrows provider defaults. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Exact `true` does not authorize foreign caller-tier forwarding on Chat. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. |
| `chatServiceTier?` | `boolean` | Provider-wide Chat-wire opt-in for forwarding caller `service_tier` values. On a classified route it governs foreign values such as `flex`, not proxy-owned canonical Fast after capability validation; on an unclassified route it governs every caller value because no Fast capability has been validated. Exact model capability does not authorize foreign forwarding. Responses routes retain their capability-based caller forwarding behavior. |
Expand Down Expand Up @@ -565,6 +566,44 @@ cleared or unresolved. The persisted catalog field is read by Codex for the curr
model, which is why a valid configured selector is copied to each applicable entry.
Provider-scoped selectors (above) are applied before this root fallback and win on routed rows.

### Response service-tier authority

Set `providers.<name>.responseTierAuthoritative` to `false` in `config.json` when that
provider's response `service_tier` cannot establish whether Fast was granted. This is an
operator declaration about the entire provider route, independent of `supportsServiceTier` and
`fastWire`. It neither enables Fast nor changes the outgoing `service_tier`.

For example, add this field to an existing gateway provider whose Codex backend has
non-authoritative response metadata:

```json
{ "responseTierAuthoritative": false }
```

**Gateway support is opt-in.** Updating OpenCodex alone does not add this declaration to
existing providers. Without it, an eligible priority request followed by `service_tier: "default"`
retains the legacy `response-declined` interpretation. Configure `false` only when the
route's response metadata is known to be non-authoritative.

For an eligible request serialized as `priority`, both a `default` and a `priority` echo remain
observations. Logs preserve `responseServiceTier` and record
`tierOutcome.responseTierAuthoritative: false`, `fastOutcome: "applied"`, and
`confirmation: "assumed"`, without `response-declined`. Here **applied means the request parameter
was sent**, and **assumed means the actual Fast effect is unconfirmed**. The model tooltip shows
the request and raw response separately with that confirmation. Neither this setting nor these
records prove an acceleration or a billed tier.

Cost estimates use the existing requested-tier fallback instead of treating the raw echo as a
confirmed price tier; pricing rules requiring response confirmation cannot use that echo. Historical records
without an authority flag retain their previous interpretation.

The field accepts only booleans. Omission and explicit `true` retain response-based confirmation
for other destinations, including the official API and undeclared gateways. Canonical
`https://chatgpt.com/backend-api/codex` with `authMode: "forward"` remains automatically
non-authoritative, even if `true` is configured. Gateway names and URLs are never inferred.
If one gateway mixes response contracts, use separate provider entries for those routes and
apply the declaration only to the relevant entry.

### FastWire B1 capability migration

Fast capability and arbitrary Chat caller-tier forwarding are independent after FastWire B1. The
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ selector,而不是分配一个新名称。
| `chatCompletionsPath?` | `string` | 用于 `openai-chat` 请求的相对资源路径,是 `responsesPath` 的对应项,适用相同的路径规则。当同一上游以不同前缀提供 Chat Completions 和 Responses 时需要此配置:按模型的 wire override 只更换适配器而不改动 `baseUrl`,否则已启用的 Chat 请求会被发送到 Responses base。随附示例为 Z.AI。 |
| `upstreamWebsocket?` | `boolean` | 为 `openai-responses` 请求选择性启用上游 Responses WebSocket 传输(默认 `false`)。仅对第一方 `https://api.openai.com/v1` 上游生效;自定义提供者端点始终使用有界 HTTP/SSE,因为 Bun 无法在分配完整消息之前对入站 WebSocket 消息实施大小限制。对于规范 ChatGPT `openai` 提供商,省略该字段会在符合条件的轮次使用上游 WebSocket,`false` 通过 HTTP/SSE 发送流式轮次,`true` 会被拒绝;设为 `false` 时,原生轮次中操控与注入不可用。该字段独立于客户端侧的 `websockets` 设置,且不改变端点或凭据。普通 HTTP 仍使用 SSE;非 Responses 路径和 `openai-chat` 请求仍使用 HTTP。 |
| `supportsServiceTier?` | `boolean` | `service_tier` 能力的三态。`true`:fast 模式可以注入,调用方提供的值也会被保留。`false`:剥离该字段且绝不注入(已明确不支持的上游不会收到它)。未设置:未分类——调用方提供的值原样保留,fast 模式绝不注入。注册表已对官方 OpenAI(`true`)、DeepSeek 和 Volcengine Ark(`false`)分类;仅对真正支持分层的自定义网关显式设置。 |
| `responseTierAuthoritative?` | `boolean` | 响应等级是否足以确认或否定 Fast。对响应元数据不具权威性的链路显式设为 `false`;未设置时保持原有行为。不会启用 Fast 或修改出站参数。详见[响应服务等级的可信度](#响应服务等级的可信度)。 |
| `preserveResponsesReasoningContent?` | `boolean` | 在重放的 Responses reasoning 项中保留明文 reasoning 内容,而不是清空(清空是 ChatGPT 后端的规则)。对接受 reasoning 重放的上游(如 DeepSeek)启用。代理生成的 `ocxr1` 信封始终会被剥离。 |
| `disabled?` | `boolean` | 将提供者保留在磁盘上,但从路由和模型/目录列表中排除。 |
| `apiKey?` | `string` | API key,或在请求时解析的 `${ENV_VAR}` / `$ENV_VAR` 引用。 |
Expand Down Expand Up @@ -172,6 +173,37 @@ API key 提供者可以持有字面量 key,或环境引用。OAuth 提供者

`PATCH /api/providers?name=<provider>` 只修改它指定的字段,无论目的地如何都保留其他所有已存储字段。它接受全部五项设置,`null` 表示清除。对于两个推理列表,空数组会作为显式退出选项保存,而不会被删除。

### 响应服务等级的可信度

当提供者响应中的 `service_tier` 无法确定 Fast 是否生效时,在 `config.json` 中将
`providers.<name>.responseTierAuthoritative` 设为 `false`。这是运营者对整个提供者链路的声明,
独立于 `supportsServiceTier` 和 `fastWire`;它不会启用 Fast,也不会改变出站 `service_tier`。

例如,对于已知 Codex 后端响应元数据不具权威性的网关,在现有 provider 对象中加入:

```json
{ "responseTierAuthoritative": false }
```

**网关需要显式配置。** 仅升级 OpenCodex 不会为现有 provider 自动添加该声明。未配置时,
符合 Fast 条件且已发送 priority 的请求收到 `service_tier: "default"`,仍会按原逻辑记录为 `response-declined`。
仅对已知响应元数据无法确认实际等级的链路设置 `false`。

对于符合 Fast 条件且已发送 `priority` 的请求,`default` 和 `priority` 响应都只作为观察值。
日志保留 `responseServiceTier`,并记录 `tierOutcome.responseTierAuthoritative: false`、
`fastOutcome: "applied"` 和 `confirmation: "assumed"`,不会仅凭响应回显判为 `response-declined`。
这里 **applied 表示请求参数已发出**,**assumed 表示 Fast 的实际效果尚未确认**。
模型提示会分别显示请求等级、原始响应等级和确认状态。这些记录不能证明实际加速或计费等级。

费用估算沿用请求等级的回退逻辑,不将原始回显当成已确认的价格等级;要求响应确认的定价规则
不能使用该回显作为证据。没有可信度标记的历史记录保留原解释。

该字段只接受布尔值。对于官方 API、未声明的网关等其他目的地,省略或显式设为 `true`
都保留基于响应的判定。标准 `https://chatgpt.com/backend-api/codex` 地址配合
`authMode: "forward"` 始终自动按非权威处理,即使配置为 `true` 也不例外。
OpenCodex 不会依据网关名称或 URL 猜测可信度。如果同一网关混合不同的响应契约,
应拆成独立 provider 条目,只为相关条目配置此声明。

## 提供者诊断出站安全性

仪表板连接测试和实时模型发现使用受限的、仅 GET 传输。没有出站代理时,opencodex 只会解析一次主机名,并仅连接到该已验证地址。HTTPS 仍会保留原始 Host、SNI 和证书验证;提供者配置不能关闭证书检查。
Expand Down
1 change: 1 addition & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,7 @@
"fastwire-characterization-routing.test.ts": "routing",
"fastwire-characterization-wire.test.ts": "routing",
"fastwire-observability.test.ts": "routing",
"fastwire-response-authority.test.ts": "routing",
"fastwire-policy.test.ts": "routing",
"featherless-provider.test.ts": "providers",
"fetch-header-timeout.test.ts": "server",
Expand Down
1 change: 1 addition & 0 deletions src/config/schema/leaf-validators.ts
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ export const providerConfigSchema = z.object({
annotateEmptyToolOutputs: z.boolean().optional(),
foldDeveloperRoleToSystem: z.boolean().optional(),
fastWire: fastWireSchema.nullable().optional(),
responseTierAuthoritative: z.boolean().optional(),
fastEnabled: z.boolean().optional(),
supportsServiceTier: z.boolean().optional(),
modelSupportsServiceTier: z.record(z.string().min(1), z.boolean()).optional(),
Expand Down
3 changes: 3 additions & 0 deletions src/providers/fastwire.ts
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,9 @@ export function createAdapterTierMetadata(
const loggedWireValue = wireValue === null ? null : sanitizeLogMetadataString(wireValue);
const outcome: AttemptTierOutcome = {
wireKind,
...(context.responseTierAuthoritative !== undefined
? { responseTierAuthoritative: context.responseTierAuthoritative }
: {}),
...(wireValue === null
? { wireValue: null }
: loggedWireValue ? { wireValue: loggedWireValue } : {}),
Expand Down
1 change: 1 addition & 0 deletions src/providers/model-rename-fields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export const PROVIDER_MODEL_RENAME_ROLES = {
mcpMaxResultBytes: "none",
modelAdapters: "record",
fastWire: "none",
responseTierAuthoritative: "none",
baseUrl: "none",
responsesPath: "none",
chatCompletionsPath: "none",
Expand Down
9 changes: 9 additions & 0 deletions src/providers/openai-tiers-destination.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@ export function isCanonicalOpenAiForwardProvider(provider: OcxProviderConfig): b
&& normalizedBaseUrl(provider.baseUrl) === CODEX_FORWARD_BASE_URL;
}

/**
* Response evidence is separate from Fast capability and request serialization. Preserve the
* known Codex exception (#2558); other destinations may declare their response contract without
* a gateway-name or URL heuristic. Absence retains the authoritative legacy default.
*/
export function responseTierAuthorityForProvider(provider: OcxProviderConfig): boolean | undefined {
return isCanonicalOpenAiForwardProvider(provider) ? false : provider.responseTierAuthoritative;
}

const OPENAI_API_ORIGIN = "https://api.openai.com";
const OPENAI_API_BASE_URL = `${OPENAI_API_ORIGIN}/v1`;
const OPENAI_API_RESPONSES_URL = `${OPENAI_API_BASE_URL}/responses`;
Expand Down
1 change: 1 addition & 0 deletions src/server/auth-cors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,7 @@ const PROVIDER_CONFIG_FIELD_POLICY = {
mcpMaxResultBytes: "editor",
modelAdapters: "editor",
fastWire: "editor",
responseTierAuthoritative: "editor",
fastEnabled: "editor",
baseUrl: "editor",
responsesPath: "editor",
Expand Down
7 changes: 3 additions & 4 deletions src/server/responses/core-normalize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import { resolveOpenCodeGoTransport } from "../../providers/opencode-go-transpor
import { getOrAllocateRequestSessionLane } from "../request-log-conversation";
import { shouldPreparePlaintextV2AgentMessages } from "../../responses/plaintext-v2-agent-messages";
import { hasValidatedActiveReasoningEffort } from "../../responses/parser";
import { responseTierAuthorityForProvider } from "../../providers/openai-tiers-destination";
import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers";
import { applyOpenAiVirtualModel } from "../../providers/openai-virtual-models";
import { renameRoutedIdentityInContext } from "../../adapters/identity";
Expand Down Expand Up @@ -220,14 +221,12 @@ export async function applyFinalRouteRequestNormalization(args: {
);
const modelServiceTierSupport = serviceTierSupportFromPolicy(fastPolicy);
const callerTier = parsed.options.serviceTier;
// The ChatGPT-internal Codex backend echoes `service_tier: "default"` even on turns it
// scheduled as priority, so its echo cannot confirm OR deny Fast. Believing it reported every
// Fast request as `response-declined` (#2558). The public API's echo stays authoritative.
// Capture destination evidence policy separately from the unchanged outbound Fast decision.
parsed.options.tierObservation = tierObservationContext(
fastPolicy,
config.fastMode,
callerTier,
isCanonicalOpenAiForwardProvider(route.provider) ? false : undefined,
responseTierAuthorityForProvider(route.provider),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply response authority to native Chat requests

When /v1/chat/completions targets an eligible openai-chat provider, handleChatCompletions exits through handleNativeChatCompletions before this normalizer runs. That path builds its tier decision directly in buildOpenAIChatPassthroughRequest and never calls responseTierAuthorityForProvider or creates tier metadata, so responseTierAuthoritative: false is ignored: the response echo is not retained as assumed evidence and persisted attempts omit the authority flag. Thread this policy through the native Chat request/response path and add focused streaming and non-streaming coverage.

AGENTS.md reference: AGENTS.md:L447-L451

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traced this and it holds in the narrow sense: the native Chat passthrough never consults the declaration. It also never observes the echo. buildOpenAIChatPassthroughRequest returns no tierLog (src/adapters/openai-chat/passthrough.ts), and neither the JSON path (src/server/chat-native.ts) nor the SSE path (src/server/chat-native-sse.ts) records service_tier into logCtx.responseServiceTier or an attempt outcome. So on that path an echo can't confirm or deny Fast, and pricing falls back to the requested tier (serviceTierContext in src/usage/cost.ts), which is what responseTierAuthoritative: false asks for anyway.

Nothing is mis-recorded there today. Adding response-tier observation to native Chat would be a new feature and is out of scope for this carry. I've noted in the PR description that the flag governs the Responses-path observation, and that native Chat records no response-tier evidence.

);
parsed.options.tierDecision = decideTier(fastPolicy, config.fastMode, callerTier);
parsed.options.serviceTier = tierValueAfterDecision(parsed.options.tierDecision, callerTier);
Expand Down
9 changes: 9 additions & 0 deletions src/types/provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ export interface AttemptTierOutcome {
callerFastSuppressedByConfig?: boolean;
confirmation: "confirmed" | "assumed" | "downgraded" | "unknown";
responseServiceTier?: string;
/** False retains the raw echo as evidence only, including during cost estimation. */
responseTierAuthoritative?: boolean;
}

/**
Expand Down Expand Up @@ -307,6 +309,13 @@ export interface OcxProviderConfig {
* absence derives from the final model adapter.
*/
fastWire?: FastWire | null;
/**
* Whether echoed service_tier can confirm or deny Fast. Set false for a relay whose
* response metadata cannot establish the granted tier. Absence keeps legacy authority;
* canonical ChatGPT Codex forwarding always treats the echo as non-authoritative.
* Observation only: this does not enable Fast or change request serialization.
*/
responseTierAuthoritative?: boolean;
baseUrl: string;
/**
* Optional relative resource path for key-auth openai-responses requests. Must start with `/`
Expand Down
9 changes: 6 additions & 3 deletions src/usage/cost.ts
Original file line number Diff line number Diff line change
Expand Up @@ -439,13 +439,16 @@ export function serviceTierContext(entry: ServiceTierContext): ServiceTierContex

/** Convert one adapter-observed attempt outcome into the existing pricing provenance shape. */
export function serviceTierContextFromOutcome(outcome: AttemptTierOutcome): ServiceTierContext {
if (outcome.canonical === "priority" && outcome.confirmation === "confirmed") {
const responseAuthoritative = outcome.responseTierAuthoritative !== false;
if (responseAuthoritative && outcome.canonical === "priority" && outcome.confirmation === "confirmed") {
return { responseServiceTier: "priority" };
}
if (outcome.responseServiceTier !== undefined) {
// Non-authoritative echoes remain in the log, but cannot become pricing confirmation either.
if (responseAuthoritative && outcome.responseServiceTier !== undefined) {
return { responseServiceTier: outcome.responseServiceTier };
}
if (outcome.canonical === "priority" && outcome.confirmation === "assumed") {
if (outcome.canonical === "priority"
&& (outcome.confirmation === "assumed" || (!responseAuthoritative && outcome.confirmation === "confirmed"))) {
return { requestedServiceTier: "priority" };
}
// An unclassified route makes no canonical Fast claim, but its adapter can still prove that
Expand Down
Loading
Loading