Skip to content

feat: let the audio codecs of a factory be limited and ordered - #340

Merged
devopvoid merged 1 commit into
mainfrom
feat/audio-codec-factories
Oct 14, 2026
Merged

devopvoid merged 1 commit into
mainfrom
feat/audio-codec-factories

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

The audio counterpart of the video codec factories (#303), as far as the Android SDK goes: Android's setAudioEncoderFactoryFactory/setAudioDecoderFactoryFactory only take native factories, in practice the built-in ones. This lets a PeerConnectionFactory limit and order WebRTC's built-in audio codecs; Java audio codecs are left out.

API

New package dev.onvoid.webrtc.media.audio.codec:

  • BuiltinAudioEncoderFactory(String... codecNames) and BuiltinAudioDecoderFactory(String... codecNames): without names, all of WebRTC's built-in codecs (Opus, G722, PCMU, PCMA), as before. With names, only those codecs, in that order of preference. Names are compared ignoring case and de-duplicated; an unknown name throws an IllegalArgumentException that lists the built-in codecs.
  • getSupportedCodecs() returns the resulting AudioCodecInfo list: name, clock rate, channels and format parameters.
  • AudioEncoderFactory / AudioDecoderFactory: abstract classes with package-private constructors, so the built-in factories are their only implementations for now, while Java audio codecs could be added later without changing the builder.

PeerConnectionFactory.Builder gains setAudioEncoderFactory() and setAudioDecoderFactory():

PeerConnectionFactory factory = PeerConnectionFactory.builder()
        .setAudioEncoderFactory(new BuiltinAudioEncoderFactory("opus"))
        .setAudioDecoderFactory(new BuiltinAudioDecoderFactory("opus", "PCMU"))
        .build();

WebRTC still adds the auxiliary codecs by itself, depending on the selected ones: telephone-event at the 8000 and 48000 Hz clock rates (so DTMF keeps working), CN next to 8000 Hz codecs, and red next to Opus. The javadoc and guide say so.

Native

  • FilteredAudioEncoderFactory / FilteredAudioDecoderFactory wrap the built-in factories: they list the selected codecs ordered by name (AudioCodecSelection, the same algorithm as the Java side), and refuse to query or create the others.
  • AudioCodecFactories builds the native factory from the Java object, reading its canonical codec names; no factory or no names means the built-in factory itself, unchanged.
  • PeerConnectionFactory.initialize takes the two factories; the existing constructors pass null.
  • New sources live in media/audio/codec/, with its own glob in CMakeLists.txt.

Docs

A new "Audio Codecs" guide (docs/guide/advanced/audio-codecs.md), linked from the sidebar and the guide index.

Tests

AudioCodecFactoryTests: the built-in list and Opus's clock rate and channels, ordering and case-insensitive de-duplication, unknown names, which codecs a factory's sender and receiver capabilities list (including telephone-event), the codec order of an actual offer, and a call that negotiates the only selected codec.

The full webrtc suite passes on Windows (267 run, 0 failures, 2 skipped); before the rebase onto the current main, also under -Pjni-check with no JNI errors.

PeerConnectionFactory.Builder gains setAudioEncoderFactory and
setAudioDecoderFactory, which take the new BuiltinAudioEncoderFactory
and BuiltinAudioDecoderFactory of dev.onvoid.webrtc.media.audio.codec.
Without codec names they are WebRTC's built-in factories, as before.
With names they offer only those codecs, in that order, for example
only Opus; an unknown name throws and lists the built-in codecs.
getSupportedCodecs() returns the resulting AudioCodecInfo list.

Natively, FilteredAudioEncoderFactory and FilteredAudioDecoderFactory
wrap the built-in factories, selecting and ordering by name the same way
the Java side does, and refusing to create codecs that were not
selected. WebRTC still adds telephone-event, CN and red by itself.

The factory classes are abstract with package-private constructors, so
that Java audio codecs can be added later without changing the builder.
@devopvoid
devopvoid force-pushed the feat/audio-codec-factories branch from 5302cac to 339b8ec Compare October 13, 2026 23:02
@devopvoid
devopvoid merged commit d4cbd61 into main Oct 14, 2026
15 of 19 checks passed
@devopvoid
devopvoid deleted the feat/audio-codec-factories branch October 14, 2026 05:20
SendableMetatype added a commit to EduGeyser/webrtc-java that referenced this pull request Oct 15, 2026
Brings in the work upstream added from October 11 to 15: hardware video
codecs on every platform, through Media Foundation, VideoToolbox, VA-API
and NVDEC, with WebRTC built with H.265 (devopvoid#346 to devopvoid#348, devopvoid#352, devopvoid#353, devopvoid#356,
devopvoid#359, devopvoid#371 to devopvoid#377); HEVC and AV1 decoding in the media player, with
dav1d and Vulkan Video (devopvoid#331, devopvoid#345); encoded frame injectors (devopvoid#365);
receiver source changes and audio packet infos (devopvoid#366); audio codec
factories (devopvoid#340); network thread observers called on a callback thread
(devopvoid#364); screen capture with ScreenCaptureKit (devopvoid#354); Windows audio
devices that mix more than two channels (devopvoid#358); autoninja (devopvoid#357); and
the release channels of the docs site (devopvoid#361).

Fork-side adaptations:
- The factory keeps the CreatePeerConnectionFactory overload without a
  worker thread, and still switches device capture on the network
  thread. devopvoid#355 copies WebRTC's factory setup and has the audio module
  learn a worker thread, which only matters for a worker thread distinct
  from the network thread, and WebRTC is removing those. The audio codec
  factories and hardware codec switches of devopvoid#340 and devopvoid#359 go to that
  overload.
- WebRTC stays on a commit on main, so the move to branch 8059 (devopvoid#360)
  and its fixed clang package are left out.
- The data channels variant builds WebRTC without H.265 and without the
  extra targets, leaves out the audio codec factory bindings and
  addTransceiverOfType, and skips the new tests that need audio or
  video.
- dav1d is built against the sysroot on native Linux builds as well, as
  FFmpeg is. CI installs Meson for the full variant only, checks out the
  dav1d submodule, and adds its commit to the FFmpeg cache key. The
  fork's Windows SDK step stays, without upstream's second one.
- The release workflow records the docs' version in the tagged commit,
  as upstream's now does. The docs site still builds from main, since
  this fork publishes none.
- The Maven files stay deleted: Jetty is bumped in the version catalog,
  and the docs lockfile takes upstream's bumps on top of the fork's
  newer VitePress.
- The media module's README builds with Gradle.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant