fix: remove unsupported "thinking" param from evaluation judge request to avoid 400 error on non-thinking models - #134
Conversation
…to avoid 400 error on non-thinking models
|
PR #134 有没有问题:有问题,不建议按当前写法合 修复方向对:GPT-4.1 mini 这类严格 OpenAI 兼容上游收到 thinking 参数就 400(issue #131),删掉硬编码可以修好它。但 PR 问题 1 — 论证与代码不符。 PR body 声称"引擎层 sanitizeOpenAIRequestBody() 会对 Qwen3/QwQ/GLM 等按模型注入 • sanitizeOpenAIRequestBody(internal/inference/openai.go:531-586)对 thinking 只做一件事:disableThinking && 所以 PR 说的"引擎层已按模型处理"在评估这条路径上不成立。 问题 2 — 对 thinking 系 judge 模型有回归风险。 judge 全部走 • Qwen3/QwQ judge:安全,sanitize 仍会注入 enable_thinking=false。 问题 3 — 测试保护被删。 PR 把 provider_pool_evaluation_test.go:294-296 的 thinking 断言删了,等于撤掉了"judge 请求必须 |
Instead of dropping the thinking field for every judge, keep suppressing reasoning for model families that accept it (GLM, Kimi, Moonshot, DeepSeek-V4, Mimo) and omit it for strict OpenAI-compatible endpoints such as GPT-4.1 mini that reject unknown arguments with 400. Qwen3/QwQ judges are covered by the engine's enable_thinking=false injection in sanitizeOpenAIRequestBody, so the field stays off for them. Expose inference.OpenAIModelUsesThinkingTypeDisabled so the evaluation judge request can reuse the same per-model-family policy. Add a table test covering both groups.
|
用最新代码测试了不同的judge model(glm5.2和gpt-4.1 mini),均未发现问题。 |
fix: remove unsupported "thinking" param from evaluation judge request
The evaluationChatCompletion function hard-coded
thinking: {"type": "disabled"}in the judge request body, but not all upstream models support this parameter. Models like 'GPT-4.1 mini' reject it with "400: Unrecognized request argument supplied: thinking", causing the entire Listwise evaluation job to fail.The thinking-disable intent is already handled downstream by the engine layer: sanitizeOpenAIRequestBody() applies
enable_thinking: falseorthinking: {type: "disabled"}only for models that are known to support it (Qwen3, QwQ, GLM, etc.). Removing the unconditional hard-code lets the engine decide per-model, fixing the 400 error on non-thinking models while preserving the disable behavior where supported.Fix #131