Native Crash while using custom audio source for real time communication.

@devopvoid

I am facing this error while using custom audio source. I have create a seperate Executors.newSingleThreadScheduledExecutor(); and sending every 10 ms. How to resolve this error, I want real time audio communication , Is there any complete example which shows how to send or receive continous real time rtp pakets without any race error and where multoithreading handled properly?

Fatal error in: ../../audio/audio_send_stream.cc, line 368 last system error: 0 Check failed: !race_checker368.RaceDetected()

This is my testing code simple prerecorded .wav sent to webrtc client.

Can u help me on above error @devopvoid
`
package com.example.voip;

import com.google.gson.Gson;
import dev.onvoid.webrtc.CreateSessionDescriptionObserver;
import dev.onvoid.webrtc.PeerConnectionFactory;
import dev.onvoid.webrtc.PeerConnectionObserver;
import dev.onvoid.webrtc.RTCConfiguration;
import dev.onvoid.webrtc.RTCIceCandidate;
import dev.onvoid.webrtc.RTCOfferOptions;
import dev.onvoid.webrtc.RTCPeerConnection;
import dev.onvoid.webrtc.RTCPeerConnectionState;
import dev.onvoid.webrtc.RTCRtpReceiver;
import dev.onvoid.webrtc.RTCRtpTransceiver;
import dev.onvoid.webrtc.RTCSdpType;
import dev.onvoid.webrtc.RTCSessionDescription;
import dev.onvoid.webrtc.SetSessionDescriptionObserver;
import dev.onvoid.webrtc.media.audio.AudioTrack;
import dev.onvoid.webrtc.media.audio.AudioTrackSink;
import dev.onvoid.webrtc.media.audio.CustomAudioSource;
import org.apache.hc.client5.http.classic.methods.HttpPost;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.EntityUtils;
import org.apache.hc.core5.http.io.entity.StringEntity;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.UnsupportedAudioFileException;
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.security.MessageDigest;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Logger;

public class WebRtcBridge {
private static final Logger owner = Logger.getLogger(WebRtcBridge.class.getName());

private final AtomicBoolean disconnected = new AtomicBoolean(false);
private final Gson gson = new Gson();
private final CloseableHttpClient httpClient = HttpClients.createDefault();

private final ExecutorService signalingExecutor = Executors.newSingleThreadExecutor();

// Constants
private static final long AUDIO_FRAME_PERIOD_MS = 20L;
private static final int SAMPLE_RATE_IN = 16000;
private static final int SAMPLE_RATE_OUT = 48000;
private static final int CHANNELS = 1;
private static final int BPS = 2;
private static final int IN_SAMPLES_PER_FRAME = (int) (SAMPLE_RATE_IN * AUDIO_FRAME_PERIOD_MS / 1000.0);
private static final int IN_BYTES_PER_FRAME = IN_SAMPLES_PER_FRAME * BPS;
private static final int OUT_SAMPLES_PER_FRAME = (int) (SAMPLE_RATE_OUT * AUDIO_FRAME_PERIOD_MS / 1000.0);
private static final int OUT_BYTES_PER_FRAME = OUT_SAMPLES_PER_FRAME * BPS;



// Audio streaming components
private final ScheduledExecutorService audioStreamer;

private AudioFrameLogger audioFrameLogger;
private PeerConnectionFactory factory;
private RTCPeerConnection peerConnection;
private AudioTrack localAudioTrack;
private CustomAudioSource audioSource;

private final String serverUrl;

static class SdpPayload {
    String sdp, type;
}

public WebRtcBridge(String serverUrl) {
    this.serverUrl = serverUrl;
    this.audioStreamer = Executors.newSingleThreadScheduledExecutor(r -> {
        Thread t = new Thread(r, "audio-streamer");
        t.setDaemon(true);
        return t;
    });
}

public void connect() {
    factory = new PeerConnectionFactory();
    owner.info("Bridge: Initializing WebRTC...");

    RTCConfiguration rtcConfig = new RTCConfiguration();
    rtcConfig.iceServers = new ArrayList<>();

    peerConnection = factory.createPeerConnection(rtcConfig, new PeerConnectionObserverImpl());
    if (peerConnection == null) {
        owner.severe("Bridge ERROR: Failed to create RTCPeerConnection.");
        return;
    }

    // Create CustomAudioSource and track
    audioSource = new CustomAudioSource();

    audioFrameLogger = new AudioFrameLogger();
    localAudioTrack = factory.createAudioTrack("local-audio", audioSource);

    List<String> streamIds = new ArrayList<>();
    streamIds.add("stream1");
    peerConnection.addTrack(localAudioTrack, streamIds);

    startSignaling();
}

public static void main(String[] args) throws Exception {
    WebRtcBridge bridge = new WebRtcBridge("http://localhost:8811");
    bridge.connect();

    Thread.sleep(5000); // Wait for connection

    File wavFile = new File("wel16.wav");
    streamWavToBridge(wavFile, bridge);

    // Keep running for a bit after audio finishes
    Thread.sleep(5000);
    bridge.disconnect();
}

public static void streamWavToBridge(File wavFile, WebRtcBridge bridge) throws IOException, UnsupportedAudioFileException {
    AudioInputStream ais = AudioSystem.getAudioInputStream(wavFile);
    AudioFormat format = ais.getFormat();

    if (format.getSampleRate() != 16000.0f || format.getChannels() != 1 || format.getSampleSizeInBits() != 16) {
        throw new IllegalArgumentException("WAV must be 16kHz, mono, 16-bit PCM");
    }

    int frameSize = IN_BYTES_PER_FRAME;
    byte[] buffer = new byte[frameSize];
    int bytesRead;

    while ((bytesRead = ais.read(buffer, 0, frameSize)) != -1 && !bridge.disconnected.get()) {
        if (bytesRead < frameSize) {
            Arrays.fill(buffer, bytesRead, frameSize, (byte) 0);
        }
        bridge.pushSipAudio(Arrays.copyOf(buffer, frameSize));

        try {
            Thread.sleep(AUDIO_FRAME_PERIOD_MS);
        } catch (InterruptedException ignored) {
            Thread.currentThread().interrupt();
            break;
        }
    }

    ais.close();
    owner.info("WAV streaming completed");
}

public void pushSipAudio(byte[] sipAudio16kHz) {
    if (disconnected.get()) {
        return;
    }

    if (sipAudio16kHz == null || sipAudio16kHz.length == 0) {
        return;
    }

    owner.info(String.format("CLIENT SIP-IN | ts=%d | bytes=%d | sr=%d | ch=%d | sha=%s | preview=%s",
            System.currentTimeMillis(), sipAudio16kHz.length, SAMPLE_RATE_IN, 1,
            sha256Hex(sipAudio16kHz), samplePreview(sipAudio16kHz, 1, 8)));

    // Resample to 48kHz immediately on the calling thread
    byte[] resampledAudio = resample16kTo48k_Strict(sipAudio16kHz);
    audioSource.pushAudio(resampledAudio, 16, SAMPLE_RATE_OUT, CHANNELS, OUT_SAMPLES_PER_FRAME);
}



private void stopAudioStreaming() {
    audioStreamer.shutdown();
    try {
        if (!audioStreamer.awaitTermination(1000, TimeUnit.MILLISECONDS)) {
            audioStreamer.shutdownNow();
        }
    } catch (InterruptedException e) {
        audioStreamer.shutdownNow();
        Thread.currentThread().interrupt();
    }
}

// Signaling methods (same as before)
private void startSignaling() {
    signalingExecutor.execute(() -> {
        owner.info("Bridge: Creating offer...");
        RTCOfferOptions options = new RTCOfferOptions();
        peerConnection.createOffer(options, new CreateSessionDescriptionObserver() {
            @Override
            public void onSuccess(RTCSessionDescription description) {
                peerConnection.setLocalDescription(description, new SetSessionDescriptionObserver() {
                    @Override
                    public void onSuccess() {
                        sendSdpToServer(description);
                    }

                    @Override
                    public void onFailure(String err) {
                        owner.warning("setLocalDescription failed: " + err);
                    }
                });
            }

            @Override
            public void onFailure(String err) {
                owner.warning("createOffer failed: " + err);
            }
        });
    });
}

private void sendSdpToServer(RTCSessionDescription sdp) {
    if (sdp == null || sdp.sdp == null || peerConnection == null) return;
    SdpPayload payload = new SdpPayload();
    payload.sdp = sdp.sdp;
    payload.type = (sdp.sdpType != null) ? sdp.sdpType.toString().toLowerCase() : "offer";

    try {
        String json = gson.toJson(payload);
        HttpPost post = new HttpPost(serverUrl + "/offer");
        post.setEntity(new StringEntity(json, ContentType.APPLICATION_JSON));

        httpClient.execute(post, response -> {
            if (response.getCode() != 200) {
                owner.warning("Bridge ERROR: signaling returned HTTP " + response.getCode());
                return null;
            }
            String body = response.getEntity() != null ? EntityUtils.toString(response.getEntity()) : "";
            SdpPayload answer = gson.fromJson(body, SdpPayload.class);
            if (answer == null || answer.sdp == null) return null;

            RTCSdpType remoteType = switch ((answer.type != null ? answer.type.toLowerCase() : "answer")) {
                case "offer" -> RTCSdpType.OFFER;
                case "pranswer" -> RTCSdpType.PR_ANSWER;
                case "rollback" -> RTCSdpType.ROLLBACK;
                default -> RTCSdpType.ANSWER;
            };

            peerConnection.setRemoteDescription(new RTCSessionDescription(remoteType, answer.sdp), new SetSessionDescriptionObserver() {
                @Override
                public void onSuccess() {
                    owner.info("Remote description set");
                }

                @Override
                public void onFailure(String err) {
                    owner.warning("setRemoteDescription failed: " + err);
                }
            });

            return null;
        });
    } catch (IOException e) {
        owner.warning("Bridge ERROR: signaling HTTP request failed: " + e.getMessage());
    }
}

public void disconnect() {
    if (!disconnected.compareAndSet(false, true)) {
        return;
    }

    owner.info("Disconnecting WebRtcBridge...");

    // Stop audio streaming first
    stopAudioStreaming();

    // Shutdown signaling
    try {
        signalingExecutor.shutdown();
        if (!signalingExecutor.awaitTermination(1000, TimeUnit.MILLISECONDS)) {
            signalingExecutor.shutdownNow();
        }
    } catch (InterruptedException e) {
        signalingExecutor.shutdownNow();
        Thread.currentThread().interrupt();
    }

    // Close HTTP client
    try {
        httpClient.close();
    } catch (IOException ignored) {
    }

    // Cleanup WebRTC resources
    if (localAudioTrack != null) {
        localAudioTrack.setEnabled(false);
        if (audioFrameLogger != null) {
            localAudioTrack.removeSink(audioFrameLogger);
        }
    }

    if (peerConnection != null) {
        peerConnection.close();
    }

    // Dispose factory last
    if (factory != null) {
        factory.dispose();
        factory = null;
    }

    owner.info("WebRtcBridge disconnected");
}

private class PeerConnectionObserverImpl implements PeerConnectionObserver {
    @Override
    public void onConnectionChange(RTCPeerConnectionState state) {
        owner.info("State: " + state);
        if (state == RTCPeerConnectionState.CONNECTED) {
            owner.info("WebRTC connection established - audio can now flow");
        }
    }

    @Override
    public void onIceCandidate(RTCIceCandidate rtcIceCandidate) {
    }

    @Override
    public void onTrack(RTCRtpTransceiver transceiver) {
        RTCRtpReceiver receiver = transceiver.getReceiver();

        if (receiver.getTrack() instanceof AudioTrack) {
            AudioTrack remote = (AudioTrack) receiver.getTrack();
            remote.addSink((data, bitsPerSample, sampleRate, channels, frames) -> {
                // handle incoming audio from remote if needed
            });
        }
    }
}

public class AudioFrameLogger implements AudioTrackSink {
    private static final long LOG_INTERVAL_MS = 1000;
    private int frameCount = 0;
    private long lastLogTime = System.currentTimeMillis();

    @Override
    public void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, int frames) {
        frameCount++;
        long now = System.currentTimeMillis();
        if (now - lastLogTime >= LOG_INTERVAL_MS) {
            owner.info(String.format("Received %d audio frames in the last %.1f seconds",
                    frameCount, (now - lastLogTime) / 1000.0));
            frameCount = 0;
            lastLogTime = now;
        }
    }
}

// Resampling method (same as before)
private byte[] resample16kTo48k_Strict(byte[] input) {
    int bps = 2;
    short[] in = new short[IN_SAMPLES_PER_FRAME];
    int inSamples = Math.min(input.length / bps, IN_SAMPLES_PER_FRAME);

    for (int i = 0; i < inSamples; i++) {
        in[i] = getShortLE(input, i * bps);
    }
    short pad = inSamples > 0 ? in[inSamples - 1] : 0;
    for (int i = inSamples; i < IN_SAMPLES_PER_FRAME; i++) in[i] = pad;

    byte[] output = new byte[OUT_BYTES_PER_FRAME];
    for (int i = 0; i < IN_SAMPLES_PER_FRAME - 1; i++) {
        short s1 = in[i];
        short s2 = in[i + 1];
        int baseOutIndex = i * 3;
        setShortLE(output, (baseOutIndex + 0) * bps, s1);
        setShortLE(output, (baseOutIndex + 1) * bps, (short) ((2 * s1 + s2) / 3));
        setShortLE(output, (baseOutIndex + 2) * bps, (short) ((s1 + 2 * s2) / 3));
    }
    short last = in[IN_SAMPLES_PER_FRAME - 1];
    setShortLE(output, (OUT_SAMPLES_PER_FRAME - 1) * bps, last);
    return output;
}

private short readLe16(byte[] buf, int offset) {
    return (short) (((buf[offset] & 0xFF)) | ((buf[offset + 1] & 0xFF) << 8));
}

private void writeLe16(byte[] buf, int offset, short value) {
    buf[offset] = (byte) (value & 0xff);
    buf[offset + 1] = (byte) ((value >> 8) & 0xff);
}

private short getShortLE(byte[] buf, int offset) {
    return readLe16(buf, offset);
}

private void setShortLE(byte[] buf, int offset, short value) {
    writeLe16(buf, offset, value);
}

static String sha256Hex(byte[] data) {
    try {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        byte[] digest = md.digest(data);
        StringBuilder sb = new StringBuilder();
        for (byte b : digest) sb.append(String.format("%02x", b));
        return sb.toString();
    } catch (Exception e) {
        return "sha-error";
    }
}

static String samplePreview(byte[] data, int channels, int n) {
    try {
        int totalSamples = data.length / 2 / channels;
        int show = Math.min(n, totalSamples);
        int[] vals = new int[show];
        ByteBuffer bb = ByteBuffer.wrap(data).order(ByteOrder.LITTLE_ENDIAN);
        for (int i = 0; i < show; i++) {
            int byteIndex = (i * channels) * 2;
            if (byteIndex + 1 >= data.length) {
                vals[i] = 0;
                continue;
            }
            short s = bb.getShort(byteIndex);
            vals[i] = s;
        }
        return Arrays.toString(vals);
    } catch (Exception e) {
        return "preview-error";
    }
}

}`

Related to #213
Duplicate of #118.

You are not the only one to have run into this issue. The problem is that you are using the CustomAudioSource from multiple threads, which causes race conditions, as the error message states. To handle the CustomAudioSource properly, please ensure that it is only used by a single thread.

You could try to avoid the signalingExecutor.

Hint: There is a utility class called AudioConverter for resampling.

@devopvoid

Hi @devopvoid , I have implement all the webrtc stack in a single thread and also used the syncronised lock , still that error persists.

`package com.example.voip;

import com.google.gson.Gson;
import dev.onvoid.webrtc.*;
import dev.onvoid.webrtc.media.audio.AudioTrack;
import dev.onvoid.webrtc.media.audio.AudioTrackSink;
import dev.onvoid.webrtc.media.audio.CustomAudioSource;
import org.apache.hc.client5.http.classic.methods.HttpPost;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.core5.http.ContentType;
import org.apache.hc.core5.http.io.entity.EntityUtils;
import org.apache.hc.core5.http.io.entity.StringEntity;

import javax.sound.sampled.*;
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.security.MessageDigest;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Logger;

public class WebRtcBridge {
private static final Logger owner = Logger.getLogger(WebRtcBridge.class.getName());

private final AtomicBoolean disconnected = new AtomicBoolean(false);
private final Gson gson = new Gson();
private final CloseableHttpClient httpClient = HttpClients.createDefault();

// Single dedicated thread for all WebRTC native interactions
private final ExecutorService webrtcExecutor = Executors.newSingleThreadExecutor(r -> {
    Thread t = new Thread(r, "webrtc-single-thread");
    t.setDaemon(true);
    return t;
});

// Single lock guarding **all** native-accessing operations to avoid races
private final Object nativeLock = new Object();

// Constants
private static final long AUDIO_FRAME_PERIOD_MS = 20L;
private static final int SAMPLE_RATE_IN = 16000;
private static final int SAMPLE_RATE_OUT = 48000;
private static final int CHANNELS = 1;
private static final int BPS = 2;
private static final int IN_SAMPLES_PER_FRAME = (int) (SAMPLE_RATE_IN * AUDIO_FRAME_PERIOD_MS / 1000.0);
private static final int IN_BYTES_PER_FRAME = IN_SAMPLES_PER_FRAME * BPS;
private static final int OUT_SAMPLES_PER_FRAME = (int) (SAMPLE_RATE_OUT * AUDIO_FRAME_PERIOD_MS / 1000.0);
private static final int OUT_BYTES_PER_FRAME = OUT_SAMPLES_PER_FRAME * BPS;

private AudioFrameLogger audioFrameLogger;
private PeerConnectionFactory factory;
private RTCPeerConnection peerConnection;
private AudioTrack localAudioTrack;
private CustomAudioSource audioSource;

private final String serverUrl;

static class SdpPayload {
    String sdp, type;
}

public WebRtcBridge(String serverUrl) {
    this.serverUrl = serverUrl;
}

// Create PeerConnection, audioSource, track — all on webrtc thread and under nativeLock
public void connect() {
    webrtcExecutor.execute(() -> {
        synchronized (nativeLock) {
            factory = new PeerConnectionFactory();
            owner.info("webrtc-thread: Initializing WebRTC...");

            RTCConfiguration rtcConfig = new RTCConfiguration();
            rtcConfig.iceServers = new ArrayList<>();

            peerConnection = factory.createPeerConnection(rtcConfig, new PeerConnectionObserverImpl());
            if (peerConnection == null) {
                owner.severe("Bridge ERROR: Failed to create RTCPeerConnection.");
                return;
            }

            audioSource = new CustomAudioSource();

            audioFrameLogger = new AudioFrameLogger();
            localAudioTrack = factory.createAudioTrack("local-audio", audioSource);

            List<String> streamIds = new ArrayList<>();
            streamIds.add("stream1");
            peerConnection.addTrack(localAudioTrack, streamIds);
        }

        // Start signaling (still on webrtc thread)
        startSignalingOnWebrtcThread();
    });
}

// Small helper used by main() example to stream a wav by calling enqueueSipAudio
public static void streamWavToBridge(File wavFile, WebRtcBridge bridge) throws IOException, UnsupportedAudioFileException {
    AudioInputStream ais = AudioSystem.getAudioInputStream(wavFile);
    AudioFormat format = ais.getFormat();

    if (format.getSampleRate() != 16000.0f || format.getChannels() != 1 || format.getSampleSizeInBits() != 16) {
        throw new IllegalArgumentException("WAV must be 16kHz, mono, 16-bit PCM");
    }

    int frameSize = IN_BYTES_PER_FRAME;
    byte[] buffer = new byte[frameSize];
    int bytesRead;

    try {
        while (!bridge.disconnected.get() && (bytesRead = ais.read(buffer, 0, frameSize)) != -1) {
            if (bytesRead < frameSize) Arrays.fill(buffer, bytesRead, frameSize, (byte) 0);
            bridge.enqueueSipAudio(Arrays.copyOf(buffer, frameSize));
            try {
                Thread.sleep(AUDIO_FRAME_PERIOD_MS);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                break;
            }
        }
    } finally {
        ais.close();
        owner.info("WAV streaming completed");
    }
}

// Public API used by file-loop: posts resample+push to webrtcExecutor
public void enqueueSipAudio(byte[] sipAudio16kHz) {
    if (sipAudio16kHz == null || sipAudio16kHz.length == 0) return;
    if (disconnected.get()) return;

    owner.info(String.format("CLIENT SIP-IN | ts=%d | bytes=%d | sr=%d | ch=%d | sha=%s | preview=%s",
            System.currentTimeMillis(), sipAudio16kHz.length, SAMPLE_RATE_IN, CHANNELS,
            sha256Hex(sipAudio16kHz), samplePreview(sipAudio16kHz, CHANNELS, 8)));

    // Post resample + push work to the dedicated thread
    webrtcExecutor.execute(() -> {
        try {
            // Resample (CPU work performed on webrtc thread for simplicity)
            byte[] out = resample16kTo48k_Strict(sipAudio16kHz);

            owner.info(String.format("CLIENT PUSH | ts=%d | bytes=%d | sr=%d | ch=%d | sha=%s | preview=%s",
                    System.currentTimeMillis(), out.length, SAMPLE_RATE_OUT, CHANNELS,
                    sha256Hex(out), samplePreview(out, CHANNELS, 8)));

            // pushAudio must be synchronized with other native ops
            synchronized (nativeLock) {
                if (audioSource != null) {
                    // ensure channels param matches actual data (1 = mono)
                    audioSource.pushAudio(out, 16, SAMPLE_RATE_OUT, CHANNELS, OUT_SAMPLES_PER_FRAME);
                }
            }
        } catch (Throwable t) {
            owner.warning("webrtc-thread: enqueueSipAudio failed: " + t.getMessage());
        }
    });
}

// Signaling: create offer & set local (executed on webrtc thread)
private void startSignalingOnWebrtcThread() {
    owner.info("webrtc-thread: Creating offer...");
    RTCOfferOptions options = new RTCOfferOptions();
    peerConnection.createOffer(options, new CreateSessionDescriptionObserver() {
        @Override
        public void onSuccess(RTCSessionDescription description) {
            // setLocalDescription touches native state — do it under lock
            webrtcExecutor.execute(() -> {
                synchronized (nativeLock) {
                    peerConnection.setLocalDescription(description, new SetSessionDescriptionObserver() {
                        @Override
                        public void onSuccess() {
                            // send SDP to server (HTTP done on this thread). setRemoteDescription will be applied on webrtc thread.
                            sendSdpToServer(description);
                        }

                        @Override
                        public void onFailure(String err) {
                            owner.warning("webrtc-thread: setLocalDescription failed: " + err);
                        }
                    });
                }
            });
        }

        @Override
        public void onFailure(String err) {
            owner.warning("webrtc-thread: createOffer failed: " + err);
        }
    });
}

// send SDP to server (HTTP call) and then set remote description on webrtc thread under lock
private void sendSdpToServer(RTCSessionDescription sdp) {
    if (sdp == null || sdp.sdp == null || peerConnection == null) return;
    SdpPayload payload = new SdpPayload();
    payload.sdp = sdp.sdp;
    payload.type = (sdp.sdpType != null) ? sdp.sdpType.toString().toLowerCase() : "offer";

    try {
        String json = gson.toJson(payload);
        HttpPost post = new HttpPost(serverUrl + "/offer");
        post.setEntity(new StringEntity(json, ContentType.APPLICATION_JSON));

        httpClient.execute(post, response -> {
            if (response.getCode() != 200) {
                owner.warning("Bridge ERROR: signaling returned HTTP " + response.getCode());
                return null;
            }
            String body = response.getEntity() != null ? EntityUtils.toString(response.getEntity()) : "";
            SdpPayload answer = gson.fromJson(body, SdpPayload.class);
            if (answer == null || answer.sdp == null) return null;

            RTCSdpType remoteType = switch ((answer.type != null ? answer.type.toLowerCase() : "answer")) {
                case "offer" -> RTCSdpType.OFFER;
                case "pranswer" -> RTCSdpType.PR_ANSWER;
                case "rollback" -> RTCSdpType.ROLLBACK;
                default -> RTCSdpType.ANSWER;
            };

            // setRemoteDescription must run on webrtc thread and under lock
            webrtcExecutor.execute(() -> {
                synchronized (nativeLock) {
                    peerConnection.setRemoteDescription(new RTCSessionDescription(remoteType, answer.sdp),
                            new SetSessionDescriptionObserver() {
                                @Override
                                public void onSuccess() {
                                    owner.info("webrtc-thread: Remote description set");
                                }

                                @Override
                                public void onFailure(String err) {
                                    owner.warning("webrtc-thread: setRemoteDescription failed: " + err);
                                }
                            });
                }
            });

            return null;
        });
    } catch (IOException e) {
        owner.warning("webrtc-thread: Bridge ERROR: signaling HTTP request failed: " + e.getMessage());
    }
}

// Disconnect and cleanup — perform native cleanup on webrtc thread under lock, then shutdown executor
public void disconnect() {
    if (!disconnected.compareAndSet(false, true)) return;

    owner.info("Disconnecting WebRtcBridge...");

    try { httpClient.close(); } catch (IOException ignored) {}

    // perform native cleanup on webrtc thread
    webrtcExecutor.execute(() -> {
        synchronized (nativeLock) {
            try {
                if (localAudioTrack != null) {
                    localAudioTrack.setEnabled(false);
                    if (audioFrameLogger != null) localAudioTrack.removeSink(audioFrameLogger);
                }
                if (peerConnection != null) {
                    peerConnection.close();
                    peerConnection = null;
                }
                if (audioSource != null) {
                    try { audioSource.dispose(); } catch (Throwable ignored) {}
                    audioSource = null;
                }
                if (factory != null) {
                    factory.dispose();
                    factory = null;
                }
            } catch (Throwable t) {
                owner.warning("webrtc-thread: error during cleanup: " + t.getMessage());
            }
        }
    });

    // shutdown executor
    webrtcExecutor.shutdown();
    try {
        if (!webrtcExecutor.awaitTermination(2000, TimeUnit.MILLISECONDS)) {
            webrtcExecutor.shutdownNow();
        }
    } catch (InterruptedException e) {
        webrtcExecutor.shutdownNow();
        Thread.currentThread().interrupt();
    }

    owner.info("WebRtcBridge disconnected");
}

// Observer — forward any native callbacks onto the webrtc thread to avoid concurrent usage
private class PeerConnectionObserverImpl implements PeerConnectionObserver {
    @Override
    public void onConnectionChange(RTCPeerConnectionState state) {
        // just log; avoid calling native objects directly here
        owner.info("State: " + state);
        if (state == RTCPeerConnectionState.CONNECTED) {
            owner.info("WebRTC connection established - audio can now flow");
        }
    }

    @Override
    public void onIceCandidate(RTCIceCandidate rtcIceCandidate) {
        // no-op for this example
    }

    @Override
    public void onTrack(RTCRtpTransceiver transceiver) {
        // This callback can be invoked on a native thread — post to our webrtcExecutor
        webrtcExecutor.execute(() -> {
            synchronized (nativeLock) {
                RTCRtpReceiver receiver = transceiver.getReceiver();
                if (receiver.getTrack() instanceof AudioTrack) {
                    AudioTrack remote = (AudioTrack) receiver.getTrack();
                    // Add a simple sink (safe to call on webrtc thread under lock)
                    remote.addSink((data, bitsPerSample, sampleRate, channels, frames) -> {
                        // log or handle incoming remote audio (avoid heavy processing here)
                        // this callback itself may be invoked on native thread; keep it light
                    });
                }
            }
        });
    }
}

public class AudioFrameLogger implements AudioTrackSink {
    private static final long LOG_INTERVAL_MS = 1000;
    private int frameCount = 0;
    private long lastLogTime = System.currentTimeMillis();

    @Override
    public void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, int frames) {
        frameCount++;
        long now = System.currentTimeMillis();
        if (now - lastLogTime >= LOG_INTERVAL_MS) {
            owner.info(String.format("Received %d audio frames in the last %.1f seconds",
                    frameCount, (now - lastLogTime) / 1000.0));
            frameCount = 0;
            lastLogTime = now;
        }
    }
}

// Resampling method (same as before)
private byte[] resample16kTo48k_Strict(byte[] input) {
    int bps = 2;
    short[] in = new short[IN_SAMPLES_PER_FRAME];
    int inSamples = Math.min(input.length / bps, IN_SAMPLES_PER_FRAME);

    for (int i = 0; i < inSamples; i++) {
        in[i] = getShortLE(input, i * bps);
    }
    short pad = inSamples > 0 ? in[inSamples - 1] : 0;
    for (int i = inSamples; i < IN_SAMPLES_PER_FRAME; i++) in[i] = pad;

    byte[] output = new byte[OUT_BYTES_PER_FRAME];
    for (int i = 0; i < IN_SAMPLES_PER_FRAME - 1; i++) {
        short s1 = in[i];
        short s2 = in[i + 1];
        int baseOutIndex = i * 3;
        setShortLE(output, (baseOutIndex + 0) * bps, s1);
        setShortLE(output, (baseOutIndex + 1) * bps, (short) ((2 * s1 + s2) / 3));
        setShortLE(output, (baseOutIndex + 2) * bps, (short) ((s1 + 2 * s2) / 3));
    }
    short last = in[IN_SAMPLES_PER_FRAME - 1];
    setShortLE(output, (OUT_SAMPLES_PER_FRAME - 1) * bps, last);
    return output;
}

private short readLe16(byte[] buf, int offset) {
    return (short) (((buf[offset] & 0xFF)) | ((buf[offset + 1] & 0xFF) << 8));
}

private void writeLe16(byte[] buf, int offset, short value) {
    buf[offset] = (byte) (value & 0xff);
    buf[offset + 1] = (byte) ((value >> 8) & 0xff);
}

private short getShortLE(byte[] buf, int offset) {
    return readLe16(buf, offset);
}

private void setShortLE(byte[] buf, int offset, short value) {
    writeLe16(buf, offset, value);
}

static String sha256Hex(byte[] data) {
    try {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        byte[] digest = md.digest(data);
        StringBuilder sb = new StringBuilder();
        for (byte b : digest) sb.append(String.format("%02x", b));
        return sb.toString();
    } catch (Exception e) {
        return "sha-error";
    }
}

static String samplePreview(byte[] data, int channels, int n) {
    try {
        int totalSamples = data.length / 2 / channels;
        int show = Math.min(n, totalSamples);
        int[] vals = new int[show];
        ByteBuffer bb = ByteBuffer.wrap(data).order(ByteOrder.LITTLE_ENDIAN);
        for (int i = 0; i < show; i++) {
            int byteIndex = (i * channels) * 2;
            if (byteIndex + 1 >= data.length) {
                vals[i] = 0;
                continue;
            }
            short s = bb.getShort(byteIndex);
            vals[i] = s;
        }
        return Arrays.toString(vals);
    } catch (Exception e) {
        return "preview-error";
    }
}

// Simple example main: connect, wait, stream wav, wait, disconnect.
public static void main(String[] args) throws Exception {
    WebRtcBridge bridge = new WebRtcBridge("http://localhost:8811");
    bridge.connect();
    Thread.sleep(5000);
    File wavFile = new File("wel16.wav");
    streamWavToBridge(wavFile, bridge);
    Thread.sleep(5000);
    bridge.disconnect();
}

}`

Did you manage to solve this?

Yes

@sc-sityad

@sc-sityad how did you fix it?

@devopvoid

@devopvoid thank you for the great library! However, I'm hitting the same issue as this. I'm calling pushAudio in a single thread. So I'm not sure why I get this crash.

Also, I tried running the Web Client example. After running the server, I ran the WebClientExample class. It did something for a few seconds and then it also crashed:

[SEVERE] WebSocket error
[SEVERE] Signaling error


#
# Fatal error in: ../../audio/audio_send_stream.cc, line 368
# last system error: 11
# Check failed: !race_checker368.RaceDetected()
# zsh: IOT instruction (core dumped)  mvn exec:java

Are the examples up-to-date? Are they supposed to work correctly?

A few more strange things with the demo web app - I managed to get audio and video to my browser... but only once. I ran the java-client 20 times and 19 times it either crashes as above or it freezes after [INFO] Added ICE candidate: candidate:662523881 1 udp 2113937151 ac209e0f-edff-4327-9300-51ff6ac9d13c.local 47908 typ host generation 0 ufrag jZZX network-cost 999. That is, only heartbeats are received. Also, when I press some key to kill the client, I get:

[WARNING]
java.lang.Error: Native object was not deleted. A reference is still around somewhere.
    at dev.onvoid.webrtc.media.MediaStreamTrack.dispose (Native Method)
    at dev.onvoid.webrtc.media.audio.AudioTrack.dispose (AudioTrack.java:44)
    at dev.onvoid.webrtc.examples.web.media.MediaManager.dispose (MediaManager.java:261)
    at dev.onvoid.webrtc.examples.web.WebClientExample.disconnect (WebClientExample.java:136)
    at dev.onvoid.webrtc.examples.web.WebClientExample.main (WebClientExample.java:226)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:104)
    at java.lang.reflect.Method.invoke (Method.java:565)
    at org.codehaus.mojo.exec.AbstractExecJavaBase.executeMainMethod (AbstractExecJavaBase.java:402)
    at org.codehaus.mojo.exec.ExecJavaMojo.executeMainMethod (ExecJavaMojo.java:142)
    at org.codehaus.mojo.exec.AbstractExecJavaBase.doExecClassLoader (AbstractExecJavaBase.java:377)
    at org.codehaus.mojo.exec.AbstractExecJavaBase.lambda$execute$0 (AbstractExecJavaBase.java:287)
    at java.lang.Thread.run (Thread.java:1474)

@devopvoid

@devopvoid sorry to ping you again but can you help with these issues? I'm not really sure what I'm doing wrong.

@devopvoid

@devopvoid ping again, sorry about that. I still have the same issue with version 0.17.0. Do you have any ideas how to debug this/how to figure out what I'm doing wrong?

Fixed on main in da75ca2 (#285): CustomAudioSource's internal sink list wasn't synchronized, so AddSink/RemoveSink (invoked as tracks attach/detach) could race with PushAudioData from your own capture thread — that's the race the checker in audio_send_stream.cc was catching. It's now guarded by a mutex. Will be included in the next release.

Thanks a lot! I’ll try it out when you release the new version and let you know if it works correctly. Message ID: ***@***.***>

@devopvoid

@devopvoid thanks for taking the time for fixing this! However, even using 0.18.0, I get the same error:

# Fatal error in: ../../audio/audio_send_stream.cc, line 397
# last system error: 1
# Check failed: !race_checker397.RaceDetected()

I don't think I'm doing anything strange - I just add a single audioSource.pushAudio call (called from a single thread) which leads to this failure. Is there anything I should know about this? Is there anything specific I should be doing/not doing for that not to happen? Are there any prerequisites?

@boris-petrov

The mutex in #285 was not the cause of this, and my earlier advice in this thread about wrapping everything in one thread was wrong. Sorry for sending people down that path.

The abort comes from two threads feeding the same audio send stream:

Your push thread. A CustomAudioSource hands its audio to the track's sinks on whatever thread calls pushAudio. One of those sinks is WebRTC's own LocalAudioSinkAdapter, which calls AudioSendStream::SendAudioData synchronously on that thread.
The factory's AudioDeviceModule. As soon as any audio send stream starts, AudioState::AddSendingStream starts recording on the module, and AudioTransportImpl::SendProcessedData fans every captured 10 ms frame out to every send stream of that factory. There is no per-stream opt-out.

So the microphone and your pushed audio reach the same send stream from two threads, and the RTC_CHECK_RUNS_SERIALIZED in audio_send_stream.cc aborts the process. This is why single-threading your own calls never helped: the second thread is WebRTC's capture thread, which you do not control from Java. It also explains the reports of distorted audio when it does not crash, since microphone frames are interleaved with the pushed ones.

Hosts without a working microphone never see it, because InitRecording fails there. That is why it looked so inconsistent between users.

The fix is on fix/adm-capture-gate:

The factory's audio device module is wrapped and its capture path stays closed until createAudioSource(AudioOptions) is called. A factory that only sends pushed audio never opens the recording device.
A factory commits to one kind of audio input. Mixing device-captured and pushed audio in one factory now raises an IllegalStateException pointing at the conflict, instead of aborting later.
Two further bugs in HeadlessAudioDeviceModule came out of this. Its recording path pulled the render mix back through the module and handed it to WebRTC as captured audio, so a connection sent the audio it had just received straight back to the remote peer. And its playout never registered its audio transport if playout was started before the first peer connection, which is the order the guide gives, so received audio never reached an AudioTrackSink at all.
pushAudio now validates its arguments instead of reading past the end of a short array or aborting on a chunk too large for WebRTC's audio frame. It requires 16-bit samples; the documentation previously suggested 8 and 32 also work, which they do not.

@boris-petrov if you can build from that branch, I would appreciate confirmation before I cut a release. Reproducing the race needs a host with a real microphone, so the test suite cannot prove it on its own.

Hi, I tried building the branch but I'm stuck in the beginning - building webrtc-java-jni:

[INFO] --- cmake:3.31.5-b1:generate (cmake-generate) @ webrtc-java-jni ---
[INFO] Executing cmake on PATH
[INFO] CMake Warning:
[INFO]   Ignoring extra path from command line:
[INFO]
[INFO]    "/home/boris/code/webrtc-java/webrtc-jni/target/linux-x86_64/"${cmake.options}""
[INFO]
[INFO]
[INFO] CMake Warning:
[INFO]   Ignoring extra path from command line:
[INFO]
[INFO]    """"
[INFO]
[INFO]
[INFO] -- Toolchain file: toolchain/x86_64-linux-clang.cmake
[INFO] CMake Error at toolchain/x86_64-linux-clang.cmake:17 (message):
[INFO]   No debian amd64 sysroot found in /opt/sysroot/
[INFO] Call Stack (most recent call first):
[INFO]   /usr/share/cmake/Modules/CMakeDetermineSystem.cmake:158 (include)
[INFO]   CMakeLists.txt:8 (project)
[INFO]
[INFO]
[INFO] CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
[INFO] -- Configuring incomplete, errors occurred!

I'm on Arch Linux. I installed all prerequisites mentioned in the Chromium page. I tried building with CMAKE_MAKE_PROGRAM=make mvn install but the same happened. What am I missing?

The cause is not the push thread, and the sinks_ mutex in 0.18.0 could not fix it.

AudioSendStream::SendAudioData has two feeders. One is the CustomAudioSource, on the thread that calls pushAudio(). The other is WebRTC's own capture thread: as soon as any audio send stream starts, AudioState starts the factory's AudioDeviceModule recording and fans every captured frame out to every send stream of that factory. Both enter the same function, and its RTC_CHECK_RUNS_SERIALIZED aborts the process. That is why pushing from a single thread never helped, and why hosts without a microphone never crash (recording fails to start there).

#289 fixes this in webrtc-java by gating the module's capture path: a factory that sends CustomAudioSource audio never opens the recording device, and a factory now sends only one kind of audio input, device-captured or pushed. pushAudio() also validates its arguments now; only 16-bit PCM was ever accepted by WebRTC.

fix: keep device-captured audio out of sink-fed audio senders (#289)

Navigation Menu

Repository navigation

Native Crash while using custom audio source for real time communication. #217

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Native Crash while using custom audio source for real time communication. #217

Description

Activity

devopvoid commented on Oct 19, 2025

sc-sityad commented on Oct 19, 2025

devopvoid commented on Oct 22, 2025

sc-sityad commented on Oct 22, 2025

boris-petrov commented on Nov 27, 2025

boris-petrov commented on Nov 27, 2025

boris-petrov commented on Nov 27, 2025

boris-petrov commented on Jan 9, 2026

boris-petrov commented on Sep 14, 2026

devopvoid commented on Sep 14, 2026

boris-petrov commented on Sep 15, 2026

boris-petrov commented on Sep 15, 2026

devopvoid commented on Sep 16, 2026

boris-petrov commented on Sep 17, 2026

devopvoid commented on Sep 19, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions