Repository navigation
feat: decode with Vulkan Video on Linux in the media player - #345
Merged
Merged
Conversation
On Linux the player decoded in hardware on NVIDIA GPUs only, through NVDEC. FFmpeg is now built with Vulkan Video as well, with the H.264, HEVC, VP9 and AV1 hwaccels, so the GPUs of Intel and AMD decode all four in hardware too. The player tries NVDEC first and Vulkan after it. FFmpeg loads libvulkan.so.1 with dlopen when a device is created, so the libraries load on a machine without Vulkan, as they do without the NVIDIA driver. VA-API, the alternative, would have linked libva into libavutil, and the Linux jar would not load where libva is missing. The build needs Vulkan headers newer than distributions ship (VP9 needs 1.4.317), so the C headers of Khronos Vulkan-Headers v1.4.317 are vendored, unchanged, with a pkg-config file for FFmpeg's configure, as the NVDEC headers are. Their licenses go into the Linux platform jars. The docs and the HEVC disclaimer in the NOTICE files name Vulkan Video; the disclaimer paragraph is rewrapped, as one of its lines ran long.
SendableMetatype
added a commit
to EduGeyser/webrtc-java
that referenced
this pull request
Oct 15, 2026
Brings in the work upstream added from October 11 to 15: hardware video codecs on every platform, through Media Foundation, VideoToolbox, VA-API and NVDEC, with WebRTC built with H.265 (devopvoid#346 to devopvoid#348, devopvoid#352, devopvoid#353, devopvoid#356, devopvoid#359, devopvoid#371 to devopvoid#377); HEVC and AV1 decoding in the media player, with dav1d and Vulkan Video (devopvoid#331, devopvoid#345); encoded frame injectors (devopvoid#365); receiver source changes and audio packet infos (devopvoid#366); audio codec factories (devopvoid#340); network thread observers called on a callback thread (devopvoid#364); screen capture with ScreenCaptureKit (devopvoid#354); Windows audio devices that mix more than two channels (devopvoid#358); autoninja (devopvoid#357); and the release channels of the docs site (devopvoid#361). Fork-side adaptations: - The factory keeps the CreatePeerConnectionFactory overload without a worker thread, and still switches device capture on the network thread. devopvoid#355 copies WebRTC's factory setup and has the audio module learn a worker thread, which only matters for a worker thread distinct from the network thread, and WebRTC is removing those. The audio codec factories and hardware codec switches of devopvoid#340 and devopvoid#359 go to that overload. - WebRTC stays on a commit on main, so the move to branch 8059 (devopvoid#360) and its fixed clang package are left out. - The data channels variant builds WebRTC without H.265 and without the extra targets, leaves out the audio codec factory bindings and addTransceiverOfType, and skips the new tests that need audio or video. - dav1d is built against the sysroot on native Linux builds as well, as FFmpeg is. CI installs Meson for the full variant only, checks out the dav1d submodule, and adds its commit to the FFmpeg cache key. The fork's Windows SDK step stays, without upstream's second one. - The release workflow records the docs' version in the tagged commit, as upstream's now does. The docs site still builds from main, since this fork publishes none. - The Maven files stay deleted: Jetty is bumped in the version catalog, and the docs lockfile takes upstream's bumps on top of the fork's newer VitePress. - The media module's README builds with Gradle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On Linux the media player decoded in hardware on NVIDIA GPUs only, through NVDEC. This adds Vulkan Video, so the GPUs of Intel and AMD decode H.264, HEVC, VP9 and AV1 in hardware too. It is the first step towards hardware H.264, HEVC, VP9 and AV1 on Windows, macOS and Linux; Windows and macOS already have all four in the media player.
Follows #331 (hardware HEVC/AV1, HEVC in hardware only).
--enable-vulkanand theh264_vulkan,hevc_vulkan,vp9_vulkanandav1_vulkanhwaccels on Linux x86-64 and ARM64, the targets that have NVDEC.libvulkan.so.1) withdlopenwhen a device is created, so the libraries keep loading on a machine without Vulkan, as they do without the NVIDIA driver. VA-API was the alternative, but FFmpeg links libva into libavutil, which would make the Linux jar fail to load wherever libva is not installed.Vulkan-Headersv1.4.317 are vendored independencies/vulkan, unchanged, with a pkg-config file for configure, the way the NVDEC headers are. Their licenses (Apache-2.0 or MIT) go into the Linux platform jars underMETA-INF/licenses/vulkan.VideoDecodertries CUDA (NVDEC) first and Vulkan second on Linux, so NVIDIA machines keep the path they had.MediaPlayerJavadoc and the HEVC disclaimer in the NOTICE files name Vulkan Video. The disclaimer paragraph is also rewrapped: one of its lines ran past the wrap in feat: decode HEVC and AV1 in hardware in the media player #331.Testing
--disable-x86asmand without dav1d, which this check did not need). All four Vulkan hwaccels and the four NVDEC ones are enabled (CONFIG_VULKAN_1_4is 1, so VP9 is in), andlibavutil.soandlibavcodec.soneed onlylibcandlibm, notlibvulkan.main, the media module builds and all 88 of its tests pass with hardware decoding required; the Windows FFmpeg build is unchanged.Not verified
Decoding through Vulkan Video has not run on an Intel or AMD GPU: there is no Linux machine with one here, and CI runners have no GPU, so
HardwareDecodingTestskips its hardware tests there. Which codecs work depends on the Mesa (ANV, RADV) or NVIDIA driver version; AV1 and VP9 Vulkan decoding need recent ones. The full Linux build of the module (with dav1d and x86 assembly) runs first in CI.