fix(web-search): bound search replay ownership, reset replacement answers, and sidecar sends - #5575
Conversation
…ota evidence Combines two fork PRs on the web-search path: isolate the replay cache by request context (#563) and preserve quota evidence when Retry-After would outlive the sidecar deadline (#568), rebased onto current dev. The #568 test's unrecorded-destination case is adapted to the current foldDeveloperRoleToSystem fixture semantics. bun test: web-search-bridge-replay + web-search-sidecar-429 + server-key-failover-e2e + chat-inline-document-bytes: 65 pass
…tract The carried replay-isolation commit also rewrote this fixture. It is unrelated to replay or retry behavior and the dev version already covers the current role contract, so it stays as dev has it.
A loopback caller that presents no opencodex API key has no caller identity, but the replay scope gave every such caller the shared principal "loopback". Two local processes with the same conversation id and serving route then shared bridged-search replay cells, and a client-visible cell id was enough to recover another caller's retained search result. The scope now carries a principal only when resolveContextPrincipal finds one. Without it bridgeSearchReplayScope yields no scope, so nothing is recorded or restored for that caller. The key-switch regression now seeds under a real key-derived principal, with a positive control that the same seed restores and a keyless case that must leave the cell untouched. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
After an ambiguous reset spends the operator-authorized replacement, the first send may already have run the turn. The fence only covered the gateway statuses in isTransientUpstreamStatus, so a replacement that answered 429 or 529 reached the client and the proxy's quota rotation unmarked, and either could send the turn a third time. The fence now covers what actually resends: the client retry table (408, 409, 429, every 5xx) and proxy credential and quota recovery (401, 402). Those settle as the refusal with the replacement body released. Any other error keeps its real status for the caller but is marked non-replayable, so recovery loops such as the opaque-blob rebuild of a 400 cannot resend it. Successful answers are returned unchanged. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
Rewrite the principal field on every request so an absent principal also clears one a reused holder carried, and describe the field as absent for keyless callers instead of naming a shared loopback lane. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
A client that follows a 307 or 308 sends the same POST body again, so after a spent replacement those settle as the refusal as well. The regression tables now cover every 5xx class seen in the field and assert that a kept status is never mistaken for a proxy-synthesized refusal. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
…and 429 recovery Every sidecar 429 replay called fetchWithResetRetry with a fresh default allowance of three sends, so resets in front of each of the three quota legs could reach nine paid upstream requests. The sidecar now holds one three-send budget for the whole search: each helper call receives what is left and reports every send it makes, reset retries included. The budget is checked before the 429 body is released, so an exhausted budget ends with the observed 429 as the recorded outcome instead of a send-budget error recorded as a connection failure. The deadline-safe 429 handling from the carried change is unchanged. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
…d the replacement fence Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
…retry The answer to a spent operator replacement is marked non-replayable, and every recovery loop was meant to stop on that marker. The Codex pool's unsupported-model check did not read it, so a marked 400 from a gated model could still rebuild the turn and send it from another account. It now refuses a marked response, like the quota and transient ladders beside it. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
The answer to a spent ambiguous-reset replacement can keep its real status with only an in-memory non-replayable marker. A combo rebuilds a failed attempt as a new response, which dropped that marker, so a context overflow or a 413 read as target-local and the combo sent the same turn to its next target although the first send may already have run it. The consumed failure now records that the attempt was non-replayable, carries the marker onto the rebuilt response, and the combo loop stops on it. A 413 also joins the refusal set, because it is answered to the client as a context overflow the client compacts and resends. A two-target combo regression counts physical sends: the second target must receive none. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
The refusal wording became shorter and more generic, so a provider message containing the phrase could have been labelled as this proxy's refusal in the request log. Match the full sentence the proxy writes. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Independent security reviewAn independent reviewer, separate from the author of this change, reviewed the concrete diff by reading source only. Local checks were not run under this lane's policy; hosted CI is the verifier. Verdict: PASS, no blocking issues (lane commits through
A first review round found one blocking gap: a non-replayable answer could lose its marker inside a combo. It was fixed in Accepted residuals (also in the description): the sidecar deadline check does not reserve time for the replay send; a marked context-overflow answer outside a combo keeps its status and can still be hopped by routing-profile policy fallback. That last one is a follow-up. |
|
✅ Deterministic PR hygiene checks passed. |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: Repository: lidge-jun/opencodex/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (25)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
리뷰 · 우선순위 61 / 80이 PR은 웹 검색 다시 쓰기(replay)와 연결 끊김 뒤 한 번 더 보내기(reset replacement), 그리고 검색 사이드카의 재시도가 같은 요청을 너무 많이 보내거나 다른 사람 결과를 꺼내 쓰던 구멍을 막습니다. 검색 결과는 이제 같은 호출자·대화·제공자·어댑터·모델·목적지·실제 자격증명일 때만 되살립니다. 키가 없는 로컬(loopback) 호출자는 공통 라인 - src/server/responses/policy-fallback.ts — 교체 뒤 “다시 보내면 안 됨” 표시를 콤보·Codex 계정 재시도는 읽지만, 라우팅 프로필 policy fallback 길은 이 표시를 보지 않습니다. PR 본문도 후속으로 남겨 두었습니다. 표시만 있는 context overflow 400이 policy 쪽으로 넘어가면, 이미 돌았을 수 있는 턴이 다른 정책 경로로 한 번 더 나갈 수 있습니다. 라인 - 열린 PR #5562 · #5553 — 메인테이너의 판단이 필요한 지점 policy fallback에도 같은 “다시 보내면 안 됨” 표시를 이번 PR에 넣을지, 후속 이슈로 둘지. 키 없는 loopback이 턴을 넘기며 검색 결과를 되살리지 못하는 비용을 제품으로 받아들일지(문서·structure에는 이미 적혀 있음). #5562·#5553을 이 PR에 맞춰 닫을지, 남은 부분만 남길지. 너의 추천 보안·재전송 경계는 맞고 합쳐도 됩니다. 독립 보안 리뷰도 PASS입니다. 머지 전에 policy fallback에 표시 전파를 넣거나, 넣지 않을 거면 후속 이슈 번호만 고정하세요. #5562와 #5553은 이 변경에 흡수·무효화된 부분부터 닫는 쪽을 권합니다. 호스트 CI test 샤드가 아직 돌고 있으니, 그 통과를 머지 조건으로 두면 됩니다. 이 댓글은 grok-bot이 작성했습니다 |
…wers, and sidecar sends (lidge-jun#5575) * fix(web-search): combine replay-cache isolation with deadline-safe quota evidence Combines two fork PRs on the web-search path: isolate the replay cache by request context (lidge-jun#563) and preserve quota evidence when Retry-After would outlive the sidecar deadline (lidge-jun#568), rebased onto current dev. The lidge-jun#568 test's unrecorded-destination case is adapted to the current foldDeveloperRoleToSystem fixture semantics. bun test: web-search-bridge-replay + web-search-sidecar-429 + server-key-failover-e2e + chat-inline-document-bytes: 65 pass * fix(web-search): scope replay cells to key-resolved loopback principals * fix(retries): refuse transient 5xx after an operator-authorized reset replacement * fix(retries): word the replay refusal for the post-response path too * test(responses): keep the inline-document role fixture on its dev contract The carried replay-isolation commit also rewrote this fixture. It is unrelated to replay or retry behavior and the dev version already covers the current role contract, so it stays as dev has it. * fix(web-search): fail closed when a replay caller has no principal A loopback caller that presents no opencodex API key has no caller identity, but the replay scope gave every such caller the shared principal "loopback". Two local processes with the same conversation id and serving route then shared bridged-search replay cells, and a client-visible cell id was enough to recover another caller's retained search result. The scope now carries a principal only when resolveContextPrincipal finds one. Without it bridgeSearchReplayScope yields no scope, so nothing is recorded or restored for that caller. The key-switch regression now seeds under a real key-derived principal, with a positive control that the same seed restores and a keyless case that must leave the cell untouched. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(retries): keep every resend-inducing answer under replay suppression After an ambiguous reset spends the operator-authorized replacement, the first send may already have run the turn. The fence only covered the gateway statuses in isTransientUpstreamStatus, so a replacement that answered 429 or 529 reached the client and the proxy's quota rotation unmarked, and either could send the turn a third time. The fence now covers what actually resends: the client retry table (408, 409, 429, every 5xx) and proxy credential and quota recovery (401, 402). Those settle as the refusal with the replacement body released. Any other error keeps its real status for the caller but is marked non-replayable, so recovery loops such as the opaque-blob rebuild of a 400 cannot resend it. Successful answers are returned unchanged. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(web-search): clear a stale replay principal and document its absence Rewrite the principal field on every request so an absent principal also clears one a reused holder carried, and describe the field as absent for keyless callers instead of naming a shared loopback lane. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(retries): refuse a redirect that would resend a spent replacement A client that follows a 307 or 308 sends the same POST body again, so after a spent replacement those settle as the refusal as well. The regression tables now cover every 5xx class seen in the field and assert that a kept status is never mistaken for a proxy-synthesized refusal. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(web-search): share one physical-send budget across sidecar reset and 429 recovery Every sidecar 429 replay called fetchWithResetRetry with a fresh default allowance of three sends, so resets in front of each of the three quota legs could reach nine paid upstream requests. The sidecar now holds one three-send budget for the whole search: each helper call receives what is left and reports every send it makes, reset retries included. The budget is checked before the 429 body is released, so an exhausted budget ends with the observed 429 as the recorded outcome instead of a send-budget error recorded as a connection failure. The deadline-safe 429 handling from the carried change is unchanged. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * docs(structure): record replay ownership, the sidecar send budget, and the replacement fence Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(codex): keep a non-replayable 400 out of the gated-model account retry The answer to a spent operator replacement is marked non-replayable, and every recovery loop was meant to stop on that marker. The Codex pool's unsupported-model check did not read it, so a marked 400 from a gated model could still rebuild the turn and send it from another account. It now refuses a marked response, like the quota and transient ladders beside it. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(combos): stop on a spent replacement's answer instead of hopping The answer to a spent ambiguous-reset replacement can keep its real status with only an in-memory non-replayable marker. A combo rebuilds a failed attempt as a new response, which dropped that marker, so a context overflow or a 413 read as target-local and the combo sent the same turn to its next target although the first send may already have run it. The consumed failure now records that the attempt was non-replayable, carries the marker onto the rebuilt response, and the combo loop stops on it. A 413 also joins the refusal set, because it is answered to the client as a context overflow the client compacts and resends. A two-target combo regression counts physical sends: the second target must receive none. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * fix(errors): match the whole replay refusal sentence The refusal wording became shorter and more generic, so a provider message containing the phrase could have been labelled as this proxy's refusal in the request log. Match the full sentence the proxy writes. Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * docs: state that bridged-search replay needs a caller key Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> * docs(structure): add 413 and the combo stop to the replacement fence Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> --------- Co-authored-by: luvs01 <27862058+luvs01@users.noreply.github.com> (cherry picked from commit b7351dd)
Summary
Three replay and retry paths could send or expose more than they should. This change carries @luvs01's replay-isolation and reset-replacement work and closes the gaps review found in it.
"loopback"principal.Known costs, also recorded in
structure/: keyless loopback callers lose cross-turn restoration of bridged searches (the pre-#4587 behavior); a real 401, 402 or 429 on a replacement send is not recorded against its credential for that request; a reset after an observed 429 that exhausts the sidecar budget is still recorded as a connection failure, and a sidecar replay can still be sent close to the deadline. A marked context-overflow 400 outside a combo keeps its status, and a routing-profile policy fallback can still hop on it; the replacement carried the same body as the first send, so such an answer is evidence the first send was refused on size too. Carrying the marker into policy fallback is left as a follow-up.Carries the replay and retry commits from #5480 and #5423 (both closed by their author and folded into #5562 and #5553). The unrelated inline-document fixture change from the carried commit is left as dev has it.
Co-authored-by: luvs01 27862058+luvs01@users.noreply.github.com
Verification
server-key-failover-e2e,web-search-bridge-replay); a refusal for every resend-inducing status with body release, real status plus marker for other errors, unchanged 2xx, and no marking without a spent replacement (upstream-retry); a marked gated-model 400 opens no alternate-account retry (codex-model-denial-evidence); a two-target combo whose first target resets and then answers a context overflow or a 413 sends nothing to the second target and nothing is resent by the client (replay-refusal-parity); shared three-send budget across reset and 429, 429 preserved at exhaustion, caller abort recorded as neutral (web-search-sidecar-429).Checklist
Summary by CodeRabbit
Enhancements
Bug Fixes
Documentation