fix(ai-proxy): use the OpenAI Responses API for tool calling#1749
Open
Scra3 wants to merge 5 commits into
Open
fix(ai-proxy): use the OpenAI Responses API for tool calling#1749Scra3 wants to merge 5 commits into
Scra3 wants to merge 5 commits into
Conversation
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| return rawResponse; | ||
| // Build the OpenAI wire shape from LangChain's normalized AIMessage (same path as Anthropic). | ||
| // The Responses API returns a different raw object, so we no longer read __raw_response. | ||
| return LangChainAdapter.convertResponse(response, this.modelName); |
There was a problem hiding this comment.
🟡 Medium src/provider-dispatcher.ts:113
dispatchOpenAI now rebuilds every response through LangChainAdapter.convertResponse, which hard-codes choices[0].finish_reason to 'stop' (or 'tool_calls' when tool calls exist). The real finish reason from response.response_metadata.finish_reason — such as 'length' or 'content_filter' — is silently dropped, so callers receive 'stop' even when generation was truncated or filtered. Consider reading finish_reason from response.response_metadata in convertResponse instead of always defaulting to 'stop'.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @packages/ai-proxy/src/provider-dispatcher.ts around line 113:
`dispatchOpenAI` now rebuilds every response through `LangChainAdapter.convertResponse`, which hard-codes `choices[0].finish_reason` to `'stop'` (or `'tool_calls'` when tool calls exist). The real finish reason from `response.response_metadata.finish_reason` — such as `'length'` or `'content_filter'` — is silently dropped, so callers receive `'stop'` even when generation was truncated or filtered. Consider reading `finish_reason` from `response.response_metadata` in `convertResponse` instead of always defaulting to `'stop'`.
|
Coverage Impact ⬆️ Merging this pull request will increase total coverage on Modified Files with Diff Coverage (2)
🛟 Help
|
OpenAI reasoning models (gpt-5.x) reject function tools combined with reasoning_effort on /v1/chat/completions (400). Set useResponsesApi: true on both ChatOpenAI construction sites (ProviderDispatcher + createBaseChatModel), so tools + reasoning work together. A user-supplied useResponsesApi still wins. The OpenAI proxy path no longer reads additional_kwargs.__raw_response (the Responses API returns a different raw shape); it now builds the chat.completion wire shape from the normalized AIMessage via LangChainAdapter.convertResponse, the same path already used for Anthropic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
LangChain's Responses API path silently drops string tool_choice values
('required'/'any') — its param builder forwards only object-form choices.
Under the global Responses switch this made ~20 OpenAI models return text
instead of a tool call (finish_reason !== 'tool_calls'), failing the LLM
integration test. Re-express "must call a tool" as forcing the single
available function, the only forcing shape the Responses path forwards.
Also deny-list gpt-3.5-turbo-16k (removed from the API, 404) and fix a
date-fragile assertion in the load-related-record executor test
(-1 matched dates like 2026-07-10; anchor on 'return -1').
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…oice none The Responses API rejects an empty input with its own wording, breaking the proxy's error contract — validate messages at the schema boundary instead. tool_choice 'none' is another string value LangChain's Responses path drops; not binding the tools is the only way to guarantee the model cannot call them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s API The deny-list from #1750 worked around the chat/completions 400; the Responses switch makes these models pass tool calling again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scra3
force-pushed
the
fix/ai-proxy-openai-responses-api
branch
from
ac553f1 to
7d9a0ef
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
OpenAI reasoning models (gpt-5.x) now reject function tools +
reasoning_efforton/v1/chat/completionswith a400("To use function tools, use /v1/responses or set reasoning_effort to 'none'"). This breaks theLLM Integration Tests (ai-proxy)job — globally, on main and every open PR (e.g. PRD-18 #1741).reasoning_effortis applied server-side by OpenAI (it appears nowhere in our code), so the fix is to switch endpoint, not to strip a param. Forcingreasoning_effort: 'none'was rejected — it would disable reasoning on every tool-calling step (i.e. all workflow AI steps). Instead we use the Responses API, which supports tools + reasoning together.Changes
useResponsesApi: trueon bothChatOpenAIconstruction sites —ProviderDispatcher(proxy path) andcreateBaseChatModel(workflow-executor path). A user-supplieduseResponsesApistill wins (spread after the default). No dependency bump (@langchain/openai@1.4.7+openai@6.42.0already support it).additional_kwargs.__raw_response(the Responses API returns a different raw object); it builds thechat.completionwire shape from the normalizedAIMessageviaLangChainAdapter.convertResponse— the exact path already used for Anthropic. Consumers/tests keep the same wire contract.Tests
useResponsesApi: true, the converted response shape, and tool-call conversion; a user-override test added).additional_kwargs.__raw_responseassertions dropped (path removed).The
LLM Integration Tests (ai-proxy)job hits the live OpenAI API and is the authoritative signal — please confirm it goes green here. In a local run the 400 disappeared for gpt-5.6-* (the fix took effect), but several models reportedsuccess: false(notool_callssurfaced) — unclear whether that's the local account (stale/preview models, e.g.gpt-3.5-turbo-16k404s) or aconvertResponse↔ Responses-API tool-call surfacing gap. If CI still shows gpt-5.x failing withsuccess: false, the follow-up is to verify how LangChain exposes tool calls underuseResponsesApiand adjust the converter.Follow-up (out of scope)
supported-models.tsexcludes-pro/-deep-researchas "responses-only"; after this switch they may become supportable — separate PR.🤖 Generated with Claude Code
Note
Fix tool calling in ai-proxy by switching OpenAI requests to the Responses API
useResponsesApi: trueby default when constructingChatOpenAIinstances increate-base-chat-model.tsandprovider-dispatcher.ts.__raw_responseextraction withLangChainAdapter.convertResponse()to convert LangChainAIMessageoutput to the OpenAIchat.completionwire format.forceToolChoiceForResponsesto coerce'required'/'any'tool_choiceto an explicit function choice when exactly one function tool is present; skips tool binding entirely whentool_choiceis'none'.gpt-3.5-turbo-16kis now rejected andgpt-5.6-*models are now allowed.messagesto be a non-empty array in theai-queryroute schema.useResponsesApi: falseto opt out.Macroscope summarized 7d9a0ef.