-
Notifications
You must be signed in to change notification settings - Fork 1.2k
fix(claude): state strict on a translated json_schema instead of letting it default #5885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -135,3 +135,35 @@ export function isAnthropicOutputSchema(schema: Record<string, unknown>): boolea | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| return false; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| /** | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * Does every object in this schema list ALL of its properties as required? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * OpenAI's structured-output strict mode demands exactly that, and rejects anything else with | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * `'required' is required to be supplied and to be an array including every key in properties`. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * Anthropic has no such rule, so a caller's legal optional field makes an otherwise identical | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * schema a 400 on one vendor and fine on the other. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * A caller that marks a field optional means it. Rewriting `required` to satisfy strict mode | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * would silently change the contract the caller asked for, so the only honest answer is to stop | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| * claiming strict for these schemas -- the schema is still sent and still honoured as guidance. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| */ | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| export function satisfiesOpenAiStrictSchema(value: unknown): boolean { | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if (Array.isArray(value)) return value.every(satisfiesOpenAiStrictSchema); | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if (!value || typeof value !== "object") return true; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| const node = value as Record<string, unknown>; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| // `allOf` is not supported under strict Structured Outputs at all, wherever it appears. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if ("allOf" in node) return false; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| const properties = node.properties; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if (isRecord(properties)) { | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| // An object node must list every property in `required` AND close itself to extras. The | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| // caller's schema is forwarded verbatim -- `isAnthropicOutputSchema` normalizes a CLONE for | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| // its own acceptance check -- so an object that never said `additionalProperties: false` | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| // reaches the wire without it and is refused, however complete its `required` is. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if (node.additionalProperties !== false) return false; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| const keys = Object.keys(properties); | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| const required = Array.isArray(node.required) ? node.required : []; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+160
to
+165
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- diff ---'
git diff --unified=80 08fd8a62844738c960e2da71681b9a064b2fede3 675f002af9ab07b2616fc4eca7f042ca34877014 -- src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- current file outline ---'
ast-grep outline src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- current file ---'
cat -n src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- symbol and strict usages ---'
rg -n --glob 'src/**' --glob 'tests/**' 'isAnthropicOutputSchema|satisfiesOpenAiStrictSchema|output_schema|outputSchema|strict[[:space:]]*:' .
printf '%s\n' '--- package declarations ---'
rg -n '(`@anthropic-ai`|anthropic|openai|zod|output)' package.json bun.lockb bun.lock 2>/dev/null || trueRepository: lidge-jun/opencodex Length of output: 28928 🤖 get_repo_knowledge executed:
Length of output: 14644 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- inbound model options ---'
cat -n src/claude/inbound-model-options.ts
printf '%s\n' '--- focused inbound tests ---'
sed -n '240,380p' tests/claude-integration/claude-inbound.test.ts
printf '%s\n' '--- schema-related tests and sources ---'
rg -n -C 5 'satisfiesOpenAiStrictSchema|isAnthropicOutputSchema|normalizeAnthropicOutputSchema|including every key|additionalProperties|required.*properties|json_schema|textFormat' tests/claude-integration tests/adapters src/claude src/adapters package.json
printf '%s\n' '--- relevant dependency source files ---'
find node_modules -type f \\( -path '*openai*' -o -path '*anthropic*' \\) 2>/dev/null | head -80 || trueRepository: lidge-jun/opencodex Length of output: 41908 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- Anthropic schema forwarding call sites ---'
rg -n -C 12 'normalizeAnthropicOutputSchema|textFormat|output_config' src/adapters/anthropic.ts tests/adapters/anthropic/anthropic-reasoning.test.ts
printf '%s\n' '--- exact adapter regions ---'
sed -n '180,255p' src/adapters/anthropic.ts
sed -n '870,1010p' src/adapters/anthropic.tsRepository: lidge-jun/opencodex Length of output: 41559 🌐 Web query:
💡 Result: 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- production callers ---'
rg -n -C 8 'formatFromOutputConfig' src
printf '%s\n' '--- OpenAI strict request construction ---'
sed -n '195,235p' src/adapters/openai-chat.tsRepository: lidge-jun/opencodex Length of output: 7236 Reject undeclared names in A Claude request with Suggested fix const keys = Object.keys(properties);
const required = Array.isArray(node.required) ? node.required : [];
- if (keys.some(key => !required.includes(key))) return false;
+ if (required.length !== keys.length || keys.some(key => !required.includes(key))) return false;📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if (keys.some(key => !required.includes(key))) return false; | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+152
to
+166
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: set -eu
printf '%s\n' '--- target outline ---'
ast-grep outline src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- target source ---'
cat -n src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- relevant symbols ---'
rg -n -S 'formatFromOutputConfig|satisfiesOpenAiStrictSchema|isAnthropicOutputSchema|outputSchema|output_config|strict' src tests structure 2>/dev/null | head -n 300
printf '%s\n' '--- changed hunk ---'
git diff --unified=80 08fd8a62844738c960e2da71681b9a064b2fede3 675f002af9ab07b2616fc4eca7f042ca34877014 -- src/adapters/anthropic-output-schema.tsRepository: lidge-jun/opencodex Length of output: 41175 🏁 Script executed: set -eu
printf '%s\n' '--- target source ---'
cat -n src/adapters/anthropic-output-schema.ts
printf '%s\n' '--- relevant symbols ---'
rg -n -S 'formatFromOutputConfig|satisfiesOpenAiStrictSchema|isAnthropicOutputSchema|outputSchema|output_config|strict' src tests structure 2>/dev/null | head -n 300
printf '%s\n' '--- changed hunk ---'
git diff --unified=80 08fd8a62844738c960e2da71681b9a064b2fede3 675f002af9ab07b2616fc4eca7f042ca34877014 -- src/adapters/anthropic-output-schema.tsRepository: lidge-jun/opencodex Length of output: 41360 🏁 Script executed: set -eu
printf '%s\n' '--- exact schema helper bindings ---'
rg -n -S 'anthropic-output-schema|formatFromOutputConfig|isAnthropicOutputSchema|satisfiesOpenAiStrictSchema|normalizeAnthropicOutputSchema' --glob '!node_modules/**' .
printf '%s\n' '--- Anthropic adapter outline ---'
ast-grep outline src/adapters/anthropic.ts
printf '%s\n' '--- Anthropic adapter candidate source ---'
rg -n -C 12 -S 'output_config|outputConfig|text.format|strict|formatFrom' src/adapters/anthropic.ts src/adapters 2>/dev/null | head -n 500
printf '%s\n' '--- provider documentation around output_config ---'
sed -n '420,455p' structure/providers/chat-compat.md
printf '%s\n' '--- related tests ---'
rg -n -C 10 -S 'output_config|oneOf|anyOf|strict.*schema|json_schema|output schema' tests src/adapters --glob '*anthropic*' --glob '*output*' 2>/dev/null | head -n 500Repository: lidge-jun/opencodex Length of output: 42664 🏁 Script executed: set -eu
printf '%s\n' '--- inbound model options ---'
cat -n src/claude/inbound-model-options.ts
printf '%s\n' '--- Claude inbound call path ---'
sed -n '420,485p' src/claude/inbound.ts
printf '%s\n' '--- text.format consumers ---'
rg -n -C 12 -S 'text\.format|format\.schema|json_schema|output_config\.format' src tests structure --glob '!devlog/**' | head -n 700
printf '%s\n' '--- oneOf and OpenAI strict references ---'
rg -n -C 8 -S 'oneOf|anyOf|allOf|Structured Outputs|strict.*format|strict.*schema' src tests structure --glob '!devlog/**' | head -n 700Repository: lidge-jun/opencodex Length of output: 43240 🏁 Script executed: set -eu
printf '%s\n' '--- adapter files ---'
fd -i 'openai|responses|chat' src/adapters -t f | sort
printf '%s\n' '--- textFormat adapter bindings ---'
rg -n -C 16 -S 'textFormat|options\.textFormat|response_format|json_schema' src/adapters src --glob '!src/claude/**' --glob '!devlog/**' | head -n 900
printf '%s\n' '--- chat compatibility contract ---'
sed -n '425,455p' structure/providers/chat-compat.md
printf '%s\n' '--- strict schema contract references ---'
rg -n -C 12 -S 'oneOf|anyOf|allOf|Structured Outputs|strict structured|strict mode|json_schema' structure/providers structure/transports src/adapters/openai* tests/providers tests/responses --glob '!devlog/**' | head -n 900Repository: lidge-jun/opencodex Length of output: 42812 🌐 Web query:
💡 Result: 🏁 Script executed: set -eu
printf '%s\n' '--- exact textFormat serializers ---'
rg -n -C 20 -S 'textFormat|responseFormatToText|response_format|json_schema' src/adapters/openai-chat.ts src/adapters/openai-responses.ts src/adapters/openai-responses-url.ts src/responses --glob '!devlog/**'
printf '%s\n' '--- adapter selection and Claude route binding ---'
rg -n -C 12 -S 'createOpenAIChatAdapter|createOpenAIResponsesAdapter|adapter.*openai|claude.*inbound|parseClaude|inbound.*Claude|buildRequest\(parsed' src/server src/claude src/routing src --glob '!devlog/**' | head -n 700Repository: lidge-jun/opencodex Length of output: 41982 Reject A Claude schema with Reject Suggested fix- if ("allOf" in node) return false;
+ if ("allOf" in node || "oneOf" in node) return false;📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
vadymhimself marked this conversation as resolved.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| return Object.values(node).every(satisfiesOpenAiStrictSchema); | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: lidge-jun/opencodex
Length of output: 7945
Check object schemas independently of
properties.When
format.schemais{ type: "object" },isAnthropicOutputSchemaaccepts a normalized clone that addsproperties: {}andadditionalProperties: false.formatFromOutputConfigstill forwards the original schema. BecausesatisfiesOpenAiStrictSchemachecksadditionalPropertiesonly whenpropertiesis a record, it returnstruefor the original schema and emitsstrict: truewithoutadditionalProperties: false. The destination can reject this strict object schema.Check
node.type === "object"independently of whetherpropertiesis present, and requireadditionalProperties === falseon that node. Add a regression test for{ type: "object" }.🤖 Prompt for AI Agents