Repository navigation
fix(openai-chat): read ogg, opus, flac, aac, aiff, webm and mpeg input_audio as their true media types - #16
Merged
MaximeRivest merged 1 commit intoSep 29, 2026
Conversation
…t_audio as their true media types request_from_openai_chat refused every input_audio format but wav and mp3, before any provider was chosen, so a DSPy ogg/opus recording bound for Gemini failed at ingest although Gemini takes audio/ogg natively. Each format now reads as its true media type (MAP-12 rule 4 as amended in lm15-contract changes/2026-09-29-input-audio-formats.md); an unknown format is still malformed, and the Chat Completions and Anthropic builders still refuse audio at send (MAP-10). Contract pin 307925a. Signed-off-by: Jonas Dreyøe Herfort <jdreyoe@gmail.com>
Member
|
Thank you, Jonas. Clear diagnosis (DSPy's |
This was referenced Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
request_from_openai_chataccepts aninput_audioblock only when itsformatiswavormp3. Any other format raisesValueError: messages[1].content[1].input_audio.format must be one of ['mp3', 'wav']at ingest, before a provider is chosen. DSPy rendersdspy.Audioasinput_audiowith the file's MIME subtype as the format, so an ogg/opus phone recording sent tovertex:gemini-3.8-flashfails even though Gemini takesaudio/oggnatively. An.mp3file fails too, because DSPy labels itmpeg.wav | mp3is the list for OpenAI's own server. Google's Chat Completions server accepts every audio MIME type, and MAP-12 rule 4 already says ingest is not where a wire gap is hidden.Each format now reads as its true media type:
wavasaudio/wav,mp3andmpegasaudio/mpeg, andogg,opus,flac,aac,aiffandwebmasaudio/<format>. No format is relabelled as another type. An unknown format is still malformed. Nothing changes at send time. Gemini sends the part asinlineDatawith its media type. The Chat Completions and Anthropic builders still refuse audio before the wire (MAP-10).m4a/mp4is left out because no receipt shows which spelling Gemini accepts.The contract change comes first: lm15-dev/lm15-contract#1 amends MAP-12 rule 4, updates the verdict row and adds the case
openai_chat.ingest_input_audio_ogg.CONTRACT_PINmoves to that commit. New tests intests/test_openai_chat_ingest.pycover the media type for each format, the ingest-to-GeminiinlineDatapath, the Chat Completions refusal of the same request and an unknown format. They failed before the fix (8 of 26 selected) and pass now. The full suite passes (3589 passed, 8 skipped), as doconformance/run_all.py --strict, the pinned harness (1829 of 1829, ingest 217 of 217; the new case fails on the old code) andtools/typecheck.py(0 new).