Repository navigation
Cannot customize the push voice stream #213
Description
Describe the bug
I am unable to hear clear sound on the device using CustomAudioSource.pushAudio, and it often reports errors
To Reproduce
Steps to reproduce the behavior:
The audio stream returned by calling TTS
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
audio = ByteUtils.concat(audio, data);
// play tts audio
MediaManager bean = ApplicationContextUtil.getBean(MediaManager.class);
CustomAudioSource customAudioSource = bean.getCustomAudioSource();
customAudioSource.pushAudio(audio, 16, 16000, 1, 160);
Expected behavior
I expect the streaming voice to be played normally
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS] macos
- Version [e.g. 22] 15.6.1 (24G90)
Additional context
I hope to use WebRTC Java to implement a JavaScript request for Java, and then Java calls ASR, LLM synthesizes TTS speech stream, and then plays it, similar to a robot
Activity
vipzipp commentedon Sep 22, 2025
Frequently prompt this error
devopvoid commentedon Oct 19, 2025
Duplicate of #118.
You are not alone running into this issue. The problem here is that you are using the
CustomAudioSourcefrom multiple threads and then you are running into race conditions as the error states. To handle the CustomAudioSource properly, please wrap its usage into a single thread.ziyoushu commentedon Oct 24, 2025
Thanks! I redirected to the link to see the content. I had also added synchronized before and only came to seek advice after it was not resolved. I noticed that the issue # 118 has not been resolved, and several developers have encountered this problem @devopvoid
devopvoid commentedon Sep 13, 2026
Following up on this — I looked closer and my earlier "wrap it in a single thread" answer was incomplete. There's an actual native bug here, not just a caller-side threading issue.
CustomAudioSource(webrtc-jni/src/main/cpp/src/media/audio/CustomAudioSource.cpp) keeps its registered sinks in a plainstd::vector<webrtc::AudioTrackSinkInterface*> sinks_with zero synchronization: the locking that should protect it is commented out inAddSink,RemoveSink, andPushAudioData(//webrtc::CritScope lock(&crit_);), and thecrit_member itself doesn't even exist — it's commented out in the header too.AddSink/RemoveSinkare called by WebRTC's own internals whenever a track backed by this source gets attached to (or detached from) a peer connection's send stream — not by your code. So even if you perfectly single-thread your ownpushAudio()calls,PushAudioDataiteratingsinks_can still race against WebRTC's internal thread mutating that same vector viaAddSink/RemoveSinkthe moment a connection attaches. That matches both therace_checker.RaceDetected()crash in #118 and why addingsynchronizedaround your own calls here didn't fix it — you can't synchronize against a thread you don't control from the Java side.This needs an actual fix on the native side (restoring proper locking around
sinks_), not a usage workaround. Keeping this open and tracking it as a real bug rather than user error.devopvoid commentedon Sep 13, 2026
I dug into the actual native source and found something beyond the threading advice earlier in this thread:
CustomAudioSource'ssinks_vector has no synchronization at all — theCritScopelocks inAddSink/RemoveSink/PushAudioData, and the mutex member itself, are all commented out.That matters here because
AddSink/RemoveSinkare called by WebRTC's own thread as a track attaches to/detaches from a peer connection, not by your code — so no amount of synchronizing your own calls topushAudio()prevents the race. This is a genuine native bug.Opened #285 with a fix (restores the lock with
webrtc::Mutex) plus a regression test.fix: synchronize CustomAudioSource's sinks_ against AddSink/RemoveSink (
devopvoid commentedon Sep 16, 2026
The cause of this is now understood and fixed on
fix/adm-capture-gate. Full write-up in #217.The unclear audio you were hearing was the microphone being mixed into your TTS stream. The factory's
AudioDeviceModulefeeds captured audio into every audio sender of that factory, including one backed by aCustomAudioSource, so both arrive at the same send stream. The same double feed is what aborts the process when the timing lines up. My earlier answer, to wrap the usage into a single thread, was wrong: the second thread is WebRTC's own capture thread, which you cannot synchronise against from Java.On the branch, a factory that sends pushed audio never opens the recording device, so nothing is mixed into your stream.
Two things about
pushAudiothat matter for your code specifically. Each call carries one chunk, and 10 ms is the size WebRTC works with, so push 160 frames at a time rather than concatenating into a growing buffer and pushing that repeatedly. And only 16-bit samples are accepted, since WebRTC reads the data as 16-bit PCM whatever you declare. Both are validated now instead of failing obscurely or reading past the end of the array.@vipzipp @ziyoushu tagging you since you both ran into this.
devopvoid commentedon Sep 19, 2026
The unclear sound has a concrete cause, the same one behind #217.
When a
CustomAudioSourcetrack is sent, WebRTC also starts the factory'sAudioDeviceModulerecording and feeds the captured microphone frames into every send stream of the factory. The encoder then receives microphone frames interleaved with the pushed audio. When the two feeders overlap exactly, the process aborts instead.#289 fixes this: a factory that sends pushed audio never opens the recording device. Note also that
pushAudio()only ever accepted 16-bit PCM, despite the docs listing 8 and 32 bits; it now rejects other widths with anIllegalArgumentException.fix: keep device-captured audio out of sink-fed audio senders (#289)