Skip to content

Retry with max_completion_tokens when the endpoint rejects max_tokens - #1231

Closed
Schro-code wants to merge 1 commit into
deeplethe:mainfrom
Schro-code:fix/compat-retries-unsupported-max-tokens
Closed

Schro-code wants to merge 1 commit into
deeplethe:mainfrom
Schro-code:fix/compat-retries-unsupported-max-tokens

Conversation

@Schro-code

Copy link
Copy Markdown

Summary

  • Problem: Newer OpenAI reasoning models (o1, o3, GPT-5 family) renamed max_tokens to max_completion_tokens and return a 400 error with error.code: "unsupported_parameter" when the old name is sent. They also reject temperature values other than 1 with error.code: "unsupported_value".
  • Fix: Adds a send_with_compat_retries helper that inspects the structured error.param / error.code fields in the 400 response (not free-text matching) and retries with the corrected parameter. Used by the non-streaming chat_at path.
  • Streaming path: The same compat-retry logic is woven into chat_at_streaming's existing retry loop, so both code paths handle the transition transparently.
  • Loop safety: Each adjustment applies at most once per request — the already-applied key is removed from the body before retrying, so the loop terminates even if the endpoint keeps rejecting.

Test plan

  • Verify existing tests still pass (cargo test -p utopia-llm)
  • Manual test against an o1/o3/GPT-5 endpoint: confirm the 400 → retry → success flow for max_tokens rename
  • Manual test: confirm temperature rejection is handled gracefully (parameter dropped, retry succeeds)
  • Confirm non-reasoning models (e.g. GPT-4o) are unaffected (no retry needed, passes through directly)

🤖 Generated with Claude Code

Newer OpenAI reasoning models (o1, o3, GPT-5 family) renamed
`max_tokens` to `max_completion_tokens` and return a 400 with
`error.code: "unsupported_parameter"` when the old name is sent.
Similarly, these models reject `temperature` values other than their
default (1) with `error.code: "unsupported_value"`.

This adds a `send_with_compat_retries` method that detects both cases
using the structured `error.param` / `error.code` fields (not
free-text matching) and retries once with the corrected parameter.
The same detection is added to the streaming path's existing retry
loop so both `chat_at` and `chat_at_streaming` handle the transition
transparently.

Each adjustment applies at most once per request — removing the
already-applied key from the body prevents an infinite loop if the
endpoint keeps rejecting.
@Schro-code

Copy link
Copy Markdown
Author

Closing to reopen with correct base branch (dev) and signed-off commit per CONTRIBUTING.md.

@Schro-code Schro-code closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant