Repository navigation
fix: capture screens with ScreenCaptureKit on macOS - #354
Merged
Merged
Conversation
A VideoDesktopSource capturing a screen on macOS always used ScreenCapturerMac, since WebRTC tries ScreenCaptureKit only when allow_sck_capturer is set. That capturer takes its first frame from a display snapshot and afterwards only copies what its CGDisplayStream reports as changed, so when the stream stops calling back, it keeps sending the first frame, with a live cursor composited on top (#344). Screens are now captured with ScreenCaptureKit on macOS 14 and newer, and with ScreenCapturerMac as before on older releases. ScreenCaptureKit is told the source's frame rate, as it otherwise captures at the display refresh rate. Source lists stay with ScreenCapturerMac: without its system picker ScreenCaptureKit lists no screens, and both name a screen by its display ID. Window capture is unchanged. A new test switches a window of its own between black and white while the screen is captured, and checks the frames follow.
SendableMetatype
added a commit
to EduGeyser/webrtc-java
that referenced
this pull request
Oct 15, 2026
Brings in the work upstream added from October 11 to 15: hardware video codecs on every platform, through Media Foundation, VideoToolbox, VA-API and NVDEC, with WebRTC built with H.265 (devopvoid#346 to devopvoid#348, devopvoid#352, devopvoid#353, devopvoid#356, devopvoid#359, devopvoid#371 to devopvoid#377); HEVC and AV1 decoding in the media player, with dav1d and Vulkan Video (devopvoid#331, devopvoid#345); encoded frame injectors (devopvoid#365); receiver source changes and audio packet infos (devopvoid#366); audio codec factories (devopvoid#340); network thread observers called on a callback thread (devopvoid#364); screen capture with ScreenCaptureKit (devopvoid#354); Windows audio devices that mix more than two channels (devopvoid#358); autoninja (devopvoid#357); and the release channels of the docs site (devopvoid#361). Fork-side adaptations: - The factory keeps the CreatePeerConnectionFactory overload without a worker thread, and still switches device capture on the network thread. devopvoid#355 copies WebRTC's factory setup and has the audio module learn a worker thread, which only matters for a worker thread distinct from the network thread, and WebRTC is removing those. The audio codec factories and hardware codec switches of devopvoid#340 and devopvoid#359 go to that overload. - WebRTC stays on a commit on main, so the move to branch 8059 (devopvoid#360) and its fixed clang package are left out. - The data channels variant builds WebRTC without H.265 and without the extra targets, leaves out the audio codec factory bindings and addTransceiverOfType, and skips the new tests that need audio or video. - dav1d is built against the sysroot on native Linux builds as well, as FFmpeg is. CI installs Meson for the full variant only, checks out the dav1d submodule, and adds its commit to the FFmpeg cache key. The fork's Windows SDK step stays, without upstream's second one. - The release workflow records the docs' version in the tagged commit, as upstream's now does. The docs site still builds from main, since this fork publishes none. - The Maven files stay deleted: Jetty is bumped in the version catalog, and the docs lockfile takes upstream's bumps on top of the fork's newer VitePress. - The media module's README builds with Gradle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #344: screen capture on macOS freezing on its first frame while the cursor keeps moving.
Screens are captured with ScreenCaptureKit on macOS 14 and newer
VideoDesktopSourcealways gotScreenCapturerMacon macOS, because WebRTC tries ScreenCaptureKit only whenallow_sck_captureris set, and it never was.ScreenCapturerMactakes its first frame from a display snapshot and afterwards only copies in what its CGDisplayStream reports as changed. When that stream stops calling back, it keeps delivering the first frame asSUCCESS, andDesktopAndCursorComposerdraws a live cursor over it: the symptom in the issue.VideoTrackDesktopSource::capture()now setsallow_sck_captureron macOS. On releases before 14,CreateScreenCapturerSckreturns null and WebRTC falls back toScreenCapturerMacas before. The capturer is also given the source's frame rate, since ScreenCaptureKit otherwise captures at the display refresh rate.What stays as it was:
ScreenCapturer.getDesktopSources()): without its system picker, ScreenCaptureKit lists no screens, soDesktopCapturerkeepsScreenCapturerMac. Both name a screen by itsCGDirectDisplayID, so the listed IDs select the same screen.WindowCapturerMacis still used.One behavioural difference, from reading WebRTC rather than testing it: without screen recording permission, ScreenCaptureKit fails with
ERROR_PERMANENTand the capture stops, whereScreenCapturerMacdelivered the wallpaper and the app's own windows.Reproduction
Not reproduced. On macOS 14.5 (Apple silicon, screen recording permission granted), the unfixed build followed a window switching between black and white in another process every 400 ms, in three 12 s runs, and window capture followed it too. The reporter's macOS version isn't known yet.
Testing
macOS 14.5 arm64:
mvn -pl webrtc testpasses (284 tests, 0 failures, 15 skipped), with the desktop capture tests run, not skipped.With the fix, the WebRTC log shows
ScreenCapturerSckcapturing the display listed byScreenCapturerMac. Frames follow the same switching window as before, the first one arriving 120 to 230 ms afterstart(), at the right size and colours, with a single cursor.VideoDesktopSourceTests.screenFramesFollowScreen(macOS only) switches a window of its own between black and white while the screen is captured, and fails if the frames don't follow. It passes on this Mac with either capturer, so it guards against a frozen capture rather than reproducing #344.