ten-vad-ggml

Dependency-free C++17 port of danielbodart/ten-vad-ggml — a lightweight Voice Activity Detection (VAD) model, itself a GGML port of TEN-framework/ten-vad.

This project is a complete C++ reimplementation of the reference project: it parses the GGML model file directly and runs the whole pipeline (feature extraction, separable convolutions, LSTM, dense) in plain C++17 with zero third-party dependencies (no onnxruntime, no ggml, no FFTW — its own 1024-point FFT).

Output probabilities match the reference implementation (max delta < 0.001, validated by 4745 unit-test assertions).

Highlights

• ~222× faster than the Python ONNX pipeline (RTF 0.011 vs 2.47), ~5.9× faster inference alone
• 5.5 MB peak working set vs 57 MB for onnxruntime
• Runs ~90× faster than realtime on a single CPU core (1 s of audio in ~11 ms)
• Pure C++17, zero external dependencies
• Builds a static library, a shared library (DLL + import lib), a CLI, and a benchmark

Input: 256 samples (16ms @ 16kHz)
  ↓
Feature Extraction:
  Pre-emphasis (0.97) → STFT (Hann 768, hop 256, FFT 1024)
  → Power spectrum → 40-band Mel filterbank (0-8kHz, Slaney)
  → Log → Z-normalize (precomputed means/stds)
  → + Pitch (LPC-residual autocorrelation, Viterbi tracking)
  → Context stack [3 frames × 41 features]
  ↓
Separable Convolutions:
  SepConv2D(1→16, 3×3, VALID) + ReLU
  → MaxPool(1×3, stride 1×2)
  → SepConv1D(16→16, k=3, stride 2) + ReLU
  → SepConv1D(16→16, k=3, stride 2) + ReLU
  → Flatten → [80]
  ↓
LSTM + Dense:
  LSTM Layer 0 (input=80, hidden=64)
  → LSTM Layer 1 (input=64, hidden=64)
  → Concat(h0, h1) → [128]
  → Dense(128→32) + ReLU
  → Dense(32→1) + Sigmoid
  → Speech probability [0, 1]

LSTM hidden/cell states reset every 1875 frames (30 seconds).

Repository layout

cpp/                 C++17 port (the implementation)
  include/ten_vad/   public C API header (ten_vad.h)
  src/               FFT, DSP, pitch estimation, convolutions, model + network
  tools/             CLI, debug dump, benchmark
  tests/             unit tests (4745 assertions)
  examples/          DLL consumer example
scripts/             GGML model conversion, numpy reference, validation, benchmarks
test/                sample WAVs (silence / tone / vowel / noise / mixed)
docs/                interface documentation (C API, build, linking, benchmarks)

Compilation

Platform support: this project is developed and tested on Windows x64 only. It has not been tested on Linux or macOS. The code is written in portable C++17, but no build or runtime validation has been performed on those platforms.

Requirements

• CMake ≥ 3.15
• Microsoft Visual Studio 2019/2022 (MSVC, C++17)
• No third-party libraries required

Windows (MSVC)

# from the repository root
cmake -S cpp -B cpp/build -G "Visual Studio 17 2022" -A x64
cmake --build cpp/build --config Release

CMake options

Option	Default	Description
TEN_VAD_BUILD_TESTS	ON	build the unit tests
TEN_VAD_BUILD_CLI	ON	build CLI / benchmark / dump / DLL example

Disable what you do not need, e.g. -DTEN_VAD_BUILD_TESTS=OFF.

Build artifacts (Windows x64 Release)

File	Description
ten_vad_ggml.lib	static library
ten_vad_ggml_shared.dll + .lib	shared library + import library
ten_vad_cli.exe	WAV → per-frame speech probabilities
ten_vad_tests.exe	unit tests (ctest --test-dir cpp/build -C Release)
ten_vad_bench.exe	benchmark: peak memory, per-frame latency, RTF
ten_vad_dllexample.exe	DLL consumer example

GGML model conversion

The model used by this project is the TEN-VAD model in GGML binary format (ten-vad-ggml.bin, 296 KB, FP32). You can either download a pre-built model or convert it yourself from the original ONNX model.

Option A — Download the pre-built model

hf download danielbodart/ten-vad-ggml ten-vad-ggml.bin --local-dir .

Option B — Convert from the ONNX model

The converter is scripts/convert-ten-vad-to-ggml.py (pattern follows whisper.cpp's silero-vad converter). It loads the official TEN-VAD ONNX model, reorders the LSTM gate blocks from ONNX order i/o/f/c to PyTorch order i/f/g/o, splits input/recurrent biases, and writes the 21 tensors in GGML layout.

# 1. Get the official ONNX model from the TEN-VAD upstream repo
git clone https://github.com/TEN-framework/ten-vad.git

# 2. Install Python dependencies
pip install onnx numpy          # or: uv run --with "onnx,numpy" ...

# 3. Convert
python3 scripts/convert-ten-vad-to-ggml.py \
    --onnx ten-vad/src/onnx_model/ten-vad.onnx \
    --output ten-vad-ggml.bin

The converter prints one line per tensor (name, shape, type, bytes). On success you get a ten-vad-ggml.bin with this structure:

Header:
  magic:      u32 = 0x67676d6c ("ggml")
  model_type: len(u32) + "ten-vad" (7 bytes)
  version:    u32 × 3 = (1, 0, 0)
  hparams:    u32 × 8 = (n_sep_conv=3, n_lstm=2, hidden=64,
              lstm1_in=80, lstm2_in=64, dense1_in=128, dense1_out=32, dense2_out=1)

Tensors (21 total):
  Per tensor: n_dims(u32), name_len(u32), ftype(u32=0 for F32),
              dims(u32 × n_dims, reversed), name(bytes), data(f32[])

  Sep Conv × 3:  sep_conv_{0,1,2}_{dw,pw,bias}    (9 tensors)
  LSTM × 2:     lstm_{0,1}_{ih_weight,hh_weight,ih_bias,hh_bias}  (8 tensors)
  Dense × 2:    dense_{0,1}_{weight,bias}           (4 tensors)

Note on naming: the model uses the classic GGML format (magic 0x67676d6c), not the newer GGUF format.

Verify the model works

cpp/build/ten_vad_cli ten-vad-ggml.bin test/vowel.wav
# frame N <probability 0..1>  (one line per 16 ms frame)

Using the library

C API

The exported C API is stable and callable from C, C++ and any FFI language (ctypes / P/Invoke):

#include "ten_vad/ten_vad.h"

ten_vad_ctx *vad = ten_vad_create("ten-vad-ggml.bin");   // NULL on failure
float prob = ten_vad_process(vad, samples_i16, 256);     // 16 ms frame → [0,1]
ten_vad_reset(vad);                                      // optional
ten_vad_destroy(vad);

• TEN_VAD_SAMPLE_RATE = 16000, TEN_VAD_HOP_SIZE = 256 (16 ms), TEN_VAD_FRAME_BYTES = 512
• Link against ten_vad_ggml_shared.lib (shared) or ten_vad_ggml.lib (static)
• Full API reference, build/link instructions, and a runnable consumer example: docs/cpp_api.md

CLI

cpp/build/ten_vad_cli ten-vad-ggml.bin test/vowel.wav          # print probabilities
cpp/build/ten_vad_cli ten-vad-ggml.bin test/vowel.wav --out probs.txt
cpp/build/ten_vad_cli ten-vad-ggml.bin test/mixed.wav --threshold 0.5
cpp/build/ten_vad_cli ten-vad-ggml.bin --mic                    # live VAD from microphone (Windows)
cpp/build/ten_vad_cli ten-vad-ggml.bin --mic --device 1         # pick capture device

--mic captures live audio from a microphone (WaveIn API, no extra dependencies) and prints a real-time per-frame speech/silence verdict with a probability bar. Press Ctrl+C to stop.

Performance (Windows x64, MSVC Release, 30 s audio / 1875 frames)

Metric	This C++ port	Python ONNX	Speed-up
End-to-end RTF	0.011	2.47	222×
Per-frame (full pipeline)	178 µs	39 513 µs	222×
Inference-only RTF	0.0049	0.029	5.9×
Peak working set	5.5 MB	57 MB	10.4×

RTF = processing time / audio duration (< 1 means faster than realtime). Reproduce with ten_vad_bench.exe and python scripts/bench_onnx.py ten-vad.onnx 30.

Python validation scripts

The Python scripts are kept for cross-validation and benchmarking against the ONNX model:

Script	Purpose
scripts/convert-ten-vad-to-ggml.py	ONNX → GGML model conversion
scripts/ten_vad_reference.py	numpy reference implementation (probability reference stream)
scripts/compare_probs.py	compare C++ CLI output vs reference (compare_probs.py ref.txt cpp.txt)
scripts/debug-ten-vad-ggml.py	layer-by-layer validation against the ONNX model
scripts/bench_onnx.py	Python ONNX benchmark (RTF + peak memory)

Cross-check the C++ port against the numpy reference:

python scripts/ten_vad_reference.py test/vowel.wav --model ten-vad-ggml.bin --out-probs ref.txt
cpp/build/ten_vad_cli ten-vad-ggml.bin test/vowel.wav > cpp.txt
python scripts/compare_probs.py ref.txt cpp.txt
# expected: max_delta < 0.001 -> OK

Parameters

Audio

Parameter	Value
Sample rate	16,000 Hz
Sample format	S16_LE (16-bit signed)
Frame size	256 samples (16ms)
Bytes per frame	512

Feature Extraction

Parameter	Value
Pre-emphasis	0.97
Window	Hann, 768 samples
FFT size	1024
Hop size	256 samples
Mel bands	40 (0–8 kHz, Slaney norm)
Pitch	LPC-residual autocorrelation with Viterbi tracking
Feature vector	41 (40 mel + 1 pitch)
Context frames	3
Normalization	Z-score with precomputed means/stds

Model

Parameter	Value
Sep conv layers	3
LSTM layers	2
LSTM hidden dim	64
LSTM layer 0 input	80 (from flattened conv output)
LSTM layer 1 input	64
Dense layer 0	128 → 32 (ReLU)
Dense layer 1	32 → 1 (Sigmoid)
LSTM reset interval	1875 frames (30 seconds)
Weight tensors	21
Model size	296 KB (FP32, no quantization)

Recommended VAD Thresholds

Parameter	Value
Onset (speech starts)	0.6
Offset (speech continues)	0.5
Min silence duration	1000 ms

Testing

• C++: 4745 assertions in cpp/tests/test_ten_vad.cpp (FFT round-trip, DSP primitives, pitch estimator, model loading, determinism, reference parity). Run with ctest --test-dir cpp/build -C Release.
• Cross-implementation: scripts/compare_probs.py compares probability streams between the C++ port and the numpy reference (max delta < 0.001).

Origin

This is a C++ port of danielbodart/ten-vad-ggml, which was originally developed as part of the capsper push-to-talk voice dictation project. The underlying model is from TEN-framework/ten-vad.

License

MIT. The original TEN-VAD model and native library are from TEN-framework/ten-vad.