Repository navigation
Make non-official main-generation retries idempotency-safe #17
Description
Problem
External audit reconciliation confirmed that non-official main-generation POST requests can still be retried without an explicit idempotency guarantee. During Codex App reconnect/retry storms this can duplicate upstream side effects or billing if a retry occurs after the upstream accepted the request.
Scope
- Review retry paths around main-generation requests in
src-python/codex_proxy.py. - Distinguish safe pre-open/connect retries from post-acceptance retries.
- Avoid changing the current Add official transport recurrence analyzer for post-open failures #12/Diagnose generic Gateway upstream transport instability across providers #16 transport diagnostics until recurrence evidence is reviewed.
Acceptance Criteria
- Non-official main-generation POST retry behavior is documented and covered by tests.
- Unsafe retry cases are disabled or require an explicit idempotency key/guarantee.
- Official passthrough behavior remains compatible with the current Add official transport recurrence analyzer for post-open failures #12 analyzer/probe work.
- Telemetry makes it clear whether a retry happened before or after upstream acceptance.
Required Verification
python -m pytest tests/test_routing.py tests/test_chat_completions_gateway.pyRelated
- Add official transport recurrence analyzer for post-open failures #12
- Diagnose generic Gateway upstream transport instability across providers #16
docs/reviews/v0.2-external-audit-reconciliation.md
Activity
NOirBRight commentedon Jul 9, 2026
Agent Brief
Category: bug
Summary: Make retry behavior safe for non-official main-generation POSTs.
Current behavior:
System.Collections.Hashtable.Current
Desired behavior:
System.Collections.Hashtable.Desired
Key interfaces:
Acceptance criteria:
Out of scope:
wzg0911 commentedon Jul 12, 2026
Great issue. External audit confirmation of an idempotency gap is exactly the kind of finding that's easy to rationalize away in review but hits hard in production.
Real-world data point: We tracked a production agent where retries of a POST without idempotency guarantee generated $3.82 in invalid API charges over 6 hours — not from infinite loops, but from the compound effect of 7 consecutive retries, each one landing after the upstream had already accepted and billed the previous request.
Two-layer approach worth considering:
The scoped approach you outlined (safe pre-open vs. post-acceptance distinction) is the right call. The trap is treating all retries uniformly — some are harmless transport retries, others are billing events waiting to happen.
wzg0911 commentedon Jul 13, 2026
test token verification
NOirBRight commentedon Jul 25, 2026
{ "contract": { "acceptance": [ "Deterministic tests prove non-Official main-generation POST retries remain possible for DNS, TCP-connect, and TLS failures that are explicitly classified before request write.", "Connection reset during request write, timeout or reset after request write, HTTP response errors, stream interruption before downstream output, unknown-phase errors, and every failure after downstream exposure are not retried without an explicit Model Access Path idempotency guarantee.", "When an explicit guarantee is present, all permitted attempts reuse one stable idempotency identity and tests prove the identity is never emitted in telemetry or error output.", "Capacity and Provider retry hints cannot override an unsafe phase by themselves; suppressed attempts return the existing sanitized protocol-correct failure with one completion and no fallback or duplicate terminal.", "Official Responses behavior, compaction retry policy, protocol conversion, transparent byte identity, tool continuity, cancellation, and the #223/#225 downstream and event-boundary contracts remain regression-green.", "Retry telemetry distinguishes safe pre-write, guaranteed idempotent, suppressed post-write, suppressed post-exposure, and unknown classifications with no secrets or request payloads.", "A static audit proves every non-Official main-generation retry path consults the central decision and no route-local bypass remains." ], "change_claims": { "paths": [ "src-python/codex_proxy.py", "tests/test_routing.py", "tests/test_chat_completions_gateway.py", "tests/test_proxy_event_logging.py" ], "resources": [ "gateway-main-generation-retry-safety", "gateway-upstream-attempt-identity", "gateway-retry-telemetry", "gateway-downstream-stream-commit" ] }, "dependencies": { "dispatch_after": [ 225 ], "merge_after": [ 225 ] }, "design": [ "Add one request-scoped retry-safety decision inside the existing Gateway retry seam. It consumes method, request kind, Official versus non-Official route, observed transport phase, downstream exposure, and an explicit Provider idempotency guarantee; it is not a new planner or Provider adapter.", "For non-Official main-generation POST requests, automatically retry only a failure proven to occur before request bytes could be accepted, such as DNS, TCP connect, or TLS setup. Treat request-write, response-header, stream-body, unknown-phase, and post-downstream-exposure failures as ambiguous or unsafe.", "An unsafe or ambiguous main-generation retry is allowed only when the exact Model Access Path has an explicit idempotency guarantee and every attempt reuses one stable idempotency identity. A caller-supplied header or Provider retry hint alone is not evidence of idempotent execution.", "Preserve Official transport behavior and separately configured compact or non-generation retry behavior. Existing downstream commit, cancellation, protocol conversion, stream parsing, fallback, and terminal-ledger contracts remain unchanged.", "Record bounded retry-safety telemetry for every considered retry: request kind, Provider, observed failure phase, safe-prewrite versus guaranteed-idempotent versus suppressed-unsafe disposition, attempt, and budget. Never record the idempotency value, authorization, request body, or raw upstream error payload.", "Keep the implementation in codex_proxy.py and extend existing retry tests. Do not add a universal policy module, change Provider configuration UX, or run real-client E2E on this intermediate candidate." ], "difficulty": "heavy", "done_when": [ "Focused phase-classification and main-generation retry regressions are observed failing before implementation and passing afterward.", "python -m pytest -q tests/test_routing.py tests/test_chat_completions_gateway.py tests/test_proxy_event_logging.py", "python -m pytest -q", "git diff --check", "python scripts/report_quality_gates.py (report-only)", "A static audit finds no non-Official main-generation retry bypass and no telemetry field that can expose an idempotency value or request payload.", "Exact-local-SHA Standards and Spec reviews report no blocker before the branch is pushed once.", "After local freeze, exactly one push/PR update and exact-SHA GitHub Actions run pass; no intermediate real-client qualification is run." ], "priority": "P1", "risk": "strict", "sha256": "43771a3253625d4c645c4bf5c40597b614ee50d1427a06f16a8d15d24c893590", "unresolved_decisions": [] }, "dispatch": { "attempt": 1, "base_sha": "9976e6358621e9e305988000ca8164382180ac93", "branch": "work/issue-17", "claimed_at": "2026-07-26T15:28:39.439095Z", "contract_sha256": "43771a3253625d4c645c4bf5c40597b614ee50d1427a06f16a8d15d24c893590", "creator_agent_id": "d81c5928-aff7-4d9d-b6a0-2b9f70ffad21", "generation": 3, "id": "dispatch-issue-17-a1", "status": "running", "worker_agent_id": "dc7769b0-dc0e-4055-8b51-e09b5e3bda2b", "workspace_id": "wks_5b3f142c72cf9b4b" } }Stabilize retries and qualify Ollama Cloud Responses for 0.1.7 (#241)
NOirBRight commentedon Jul 27, 2026
Completed by #241.
Evidence:
3ba17aa4a60fc9940bdebce77261322f5a296bfcdev:1333dcf2d124975fa8d6253a47c77f0646d288cd6f0062837281579acd840d7380d3f6f8b6c41050The merged change centralizes non-Official main-generation retry safety, retains stable private idempotency identity only on explicitly guaranteed paths, and keeps that identity out of telemetry and downstream errors.