Skip to content
Merged
9 changes: 9 additions & 0 deletions docs-site/src/content/docs/guides/codex-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -911,6 +911,15 @@ it is not; `ocx doctor` reports restart safety (service/shim coverage).

## Routed models during Codex reserve mode

Codex Pool can optionally protect stored pool accounts at a selected 5-hour or weekly usage
threshold. The Desktop/main account keeps its separate 98% hard lock.
Set `codexPool.lowQuotaProtection` in configuration to pause accounts, record a log-and-API
alert, or both; see [routing configuration](/reference/configuration/routing/#codex-pool-low-quota-protection).
A pause takes effect for the next selection immediately, while saving it to disk is deferred.
Check this server’s authenticated `GET /api/codex-auth/low-quota-events` history for `logged`
alerts or save failures. Manual resume remains in force for the current quota episode. The
default alert reaches only the log and API; it does not produce a desktop or OS notification.

When the ChatGPT 5-hour quota is exhausted, Codex may offer a reserve fallback model
(`gpt-reserve` / Luna Reserve). While that state is active, the Codex model picker can make
**every other entry unselectable — including opencodex routed models**, even though those
Expand Down
33 changes: 33 additions & 0 deletions docs-site/src/content/docs/reference/configuration/routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,39 @@ Routing turns the model id sent by a client into one concrete provider and upstr
| `combos?` | `Record<string, OcxComboConfig>` | `{}` | Virtual `combo/<id>` models built from ordered provider/model targets. |
| `routingProfiles?` | `Record<string, OcxRoutingProfileConfig>` | `{}` | Virtual `policy/<id>` models that select among an explicit candidate allowlist using hard capability requirements and deterministic scoring. |

### Codex Pool low-quota protection

`codexPool.lowQuotaProtection` applies only to stored Codex Pool accounts. The Desktop/main
account keeps its separate 98% hard lock. The optional shape is:

```json
{"codexPool":{"lowQuotaProtection":{"enabled":true,"threshold":80,"actions":{"pause":true,"notify":true},"windows":{"short":true,"weekly":true}}}}
```

Absence or `enabled: false` disables it. `threshold` is a finite inclusive percentage from 1
through 100. An enabled policy needs at least one true action and one true window. A selected
5-hour or weekly window triggers at `usage >= threshold`; either selected window is enough.
Only fresh accepted observations qualify: there is no extra polling, and credits-only,
cached, expired, monthly, custom-window-only, and raw out-of-range usage observations are ignored
for this policy. Display bars may still clamp invalid upstream percentages. The policy is independent
of proactive account switching.

With `pause`, the account leaves Pool selection in memory before the next request. The server
coalesces a config save after the observation turn and retries failed saves for a bounded time.
Normal shutdown waits briefly for pending saves. A timed-out save that is already running remains
`pending` until it actually succeeds or fails; a queued save is cancelled. A failed save is visible
in the event history. A restart before a successful save cannot preserve the pause. In-flight requests keep their
captured account. Manual resume suppresses repause across all currently high windows for that
account until a below-threshold reading or a new reset boundary re-arms a window. A reset never resumes an account automatically.

With `notify`, the server writes a local log line with window and percentage but no account id
and records a bounded event. Authenticated `GET /api/codex-auth/low-quota-events` exposes only
that server’s events, including account id, window, usage percentage, known reset time, timestamp,
and status. The default status is `logged`: the alert reached the log and event history only.
There is no desktop or OS notification. Each account/window logs once per server-local episode.
If an injected notification sink fails, its failure is recorded and a later eligible observation
can retry it; only a successful sink is marked `delivered`.

## Model resolution order

opencodex resolves the requested model in this order:
Expand Down
1 change: 1 addition & 0 deletions docs-site/src/content/docs/reference/management-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -683,6 +683,7 @@ manager. Its routes are:
| `PUT, PATCH /api/codex-auth/pool-strategy` | Update Codex account-pool selection strategy | 400 invalid strategy/config |
| `PUT /api/codex-auth/failover` | Set the account failover threshold | 400 invalid threshold |
| `GET /api/codex-auth/quota` | Read cached quota state by account | — |
| `GET /api/codex-auth/low-quota-events?limit=20` | Read only this server’s last 0–100 low-quota log/notice and pause-save events (default 20); includes account id and status (`logged` for the default log-only alert; `delivered` for a successful injected notice sink; `succeeded` for a completed pause save) | 400 invalid limit; management authentication required |
| `GET /api/codex-auth/reset-credits` | Inspect reset-credit eligibility for an account | 400 missing account id; upstream status passthrough; 500 lookup failure |
| `POST /api/codex-auth/reset-credits/consume` | Consume an eligible reset credit. Optional `operationId` (UUIDv4) makes the redemption idempotent: the same id replays one durable outcome instead of spending a second credit. | 400 missing account id or invalid `operationId`; 409 `identity_mismatch` when the id belongs to another account; upstream status passthrough; 503 `server_busy`, `capacity`, or `unavailable`; 500 consume failure |
| `POST /api/codex-auth/login` | Start Codex login or reauthentication | 400 invalid request; conflict/busy login states |
Expand Down
1 change: 1 addition & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"root": "tests",
"explicit": {
"pnpm-command-isolation.test.ts": "update", "provider-antigravity-quota-retry.test.ts": "providers", "project-config-warning-snapshot.test.ts": "codex-integration", "codex-quota-auto-refresh-generation.test.ts": "codex-integration", "codex-account-clear-paused.test.ts": "codex-integration",
"low-quota-protection.test.ts": "codex-integration",
"responses-compaction-recovery.test.ts": "responses", "compaction-recovery-settings.test.ts": "config", "responses-compaction-recovery-policy.test.ts": "responses", "plugin-loader.test.ts": "lib", "plugin-upstream-hooks.test.ts": "lib",
"cli-kiro-auto-selection.test.ts": "cli", "codebuddy-live-models.test.ts": "providers", "kiro-auto-selection.test.ts": "providers/kiro", "kiro-quota-metrics.test.ts": "providers/kiro", "management-provider-request-pacing.test.ts": "server",
"desktop-supervised-restart.test.ts": "clients", "cli-restart-handoff.test.ts": "cli",
Expand Down
8 changes: 5 additions & 3 deletions src/codex/auth-api/login-flow.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { withCodexAccountLogLabel } from "../account-label";
import { getCodexAccountCredential, markCodexAccountValidated, readCodexAccountRecord, saveCodexAccountCredential, CodexCredentialRefreshLockTimeoutError, CodexCredentialRefreshBusyError, CodexCredentialRefreshStaleError } from "../account-store";
import { clearAccountQuota, isCodexQuotaExhausted, parseUsageQuota, setAccountQuotaFromParsed } from "../quota";
import { clearAccountQuota, isCodexQuotaExhausted, isValidWhamHistoryObservation, parseUsageQuota, setAccountQuotaFromParsed } from "../quota";
import type { StoredAccountQuota, WhamUsageResponse } from "../quota";
import { ConfigMutationLockError, withConfigMutationLockSync } from "../../config";
import { appendDefaultCodexAccountNamespace, codexAccountPickerEnabled } from "../account-namespaces";
Expand Down Expand Up @@ -265,6 +265,7 @@ export async function handleCodexAuthLoginStart(req: Request, config: OcxConfig,
let email = cred.email || accountId;
let plan: string | undefined;
let quota: Omit<StoredAccountQuota, "updatedAt"> | null = null;
let policyQuota: Omit<StoredAccountQuota, "updatedAt"> | null = null;
try {
const tokens = { access_token: cred.access, account_id: oauthAccountId };
const resp = await fetch("https://chatgpt.com/backend-api/wham/usage", {
Expand All @@ -276,6 +277,7 @@ export async function handleCodexAuthLoginStart(req: Request, config: OcxConfig,
email = data.email ?? email;
plan = nonEmptyPlan(data.plan_type) ?? undefined;
quota = parseUsageQuota(data);
policyQuota = isValidWhamHistoryObservation(data) ? quota : null;
}
} catch { /* wham fetch is non-blocking */ }
// Reauth must refresh the same ChatGPT identity already bound to this pool slot.
Expand Down Expand Up @@ -390,7 +392,7 @@ export async function handleCodexAuthLoginStart(req: Request, config: OcxConfig,
clearCodexPoolRefreshFailure(accountId);
if (warmup.validatedAt !== undefined) markCodexAccountValidated(accountId, warmup.validatedAt, generation);
clearAccountNeedsReauth(accountId);
if (quota) setAccountQuotaFromParsed(accountId, quota);
if (quota) setAccountQuotaFromParsed(accountId, quota, undefined, undefined, policyQuota);
// Keep the pool id stable; refresh display metadata after a successful login/reauth.
accounts[existingIdx] = withCodexAccountLogLabel({
...accounts[existingIdx],
Expand Down Expand Up @@ -420,7 +422,7 @@ export async function handleCodexAuthLoginStart(req: Request, config: OcxConfig,
// A new quota row is generation-gated by live account ownership. Reconcile the
// durable config owner first so a partial prior sweep cannot reject this write.
if (newAccountPersistence?.status === "committed" && quota) {
setAccountQuotaFromParsed(accountId, quota);
setAccountQuotaFromParsed(accountId, quota, undefined, undefined, policyQuota);
}
const { catalogRefreshPending } = await convergeAccountNamespaceCatalog(
latestConfig,
Expand Down
5 changes: 3 additions & 2 deletions src/codex/auth-api/pool-quota-probe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -338,8 +338,9 @@ export async function commitPoolQuotaResponse(
if (!isCodexAccountGenerationLive(accountId, generation)) {
return { quota: null, needsReauth: false, credentialGeneration: generation };
}
setAccountQuotaFromParsed(accountId, quota, writerGeneration, undefined, quota,
ctx.poolWriter && isValidWhamHistoryObservation(data) ? { writer: ctx.poolWriter, observedAt, source: "wham", raw: quota } : undefined);
const validPolicyObservation = isValidWhamHistoryObservation(data);
setAccountQuotaFromParsed(accountId, quota, writerGeneration, undefined, validPolicyObservation ? quota : null,
ctx.poolWriter && validPolicyObservation ? { writer: ctx.poolWriter, observedAt, source: "wham", raw: quota } : undefined);
return {
quota: getAccountQuota(accountId),
needsReauth: false,
Expand Down
28 changes: 28 additions & 0 deletions src/codex/low-quota-events.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/** Bounded event projection owned by one live low-quota registration. */
export type LowQuotaEvent = {
accountId: string;
window: "short" | "weekly";
percentUsed: number;
resetAt: number | null;
timestamp: number;
status: "pending" | "logged" | "delivered" | "succeeded" | "failed" | "cancelled";
delivery: "notice" | "pause-save";
};

const CAPACITY = 100;

export function createLowQuotaEventLedger(): {
publish(event: LowQuotaEvent): void;
list(limit?: number): LowQuotaEvent[];
} {
const events: LowQuotaEvent[] = [];
return {
publish(event) {
events.unshift({ ...event });
if (events.length > CAPACITY) events.length = CAPACITY;
},
list(limit = 20) {
return events.slice(0, Math.max(0, Math.min(CAPACITY, limit))).map(event => ({ ...event }));
},
};
}
23 changes: 23 additions & 0 deletions src/codex/low-quota-observer.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import type { StoredAccountQuota } from "./quota-types";

type Observer = (accountId: string, quota: Omit<StoredAccountQuota, "updatedAt">) => void;
const observers = new Map<symbol, Observer>();

/** The composition root owns the live config; quota writers know only this synchronous slot. */
export function registerLowQuotaObserver(observer: Observer): () => void {
const owner = Symbol();
observers.set(owner, observer);
return () => { observers.delete(owner); };
}

/** Only newly accepted evidence belongs here, never carried or disk-hydrated windows. */
export function observeCodexLowQuota(accountId: string, quota: Omit<StoredAccountQuota, "updatedAt">): void {
for (const observer of observers.values()) {
try {
observer(accountId, quota);
} catch {
// One server's optional policy cannot suppress another server's observation.
console.warn("[codex-low-quota] protection action failed");
}
}
}
Loading
Loading