stt-web

Browser-native speech-to-text running 100% client-side via Rust/WASM + WebGPU.

Try the demo →

Disclaimer: This is an experimental port. The model weights are Q4-quantized from kyutai/stt-1b-en_fr (CC-BY 4.0). Transcription quality may differ from the original PyTorch implementation. This project is not affiliated with or endorsed by Kyutai Labs.

Status

Work in progress. The pipeline runs end-to-end in Chrome/Edge with WebGPU:

Microphone → AudioWorklet (24kHz mono) → Mimi codec (WASM/CPU) → STT transformer (WASM/WebGPU) → text

Requirements

• Chrome 113+ or Edge 113+ (WebGPU required)
• localhost is a secure context already, so no cert is needed for local dev

Quick Start

# 1. Quantize model weights (requires Python + PyTorch)
python scripts/quantize.py

# 2. Build the wasm package, assemble the site and start a local server
scripts/build.sh
python3 scripts/serve.py

Development

CI (.github/workflows/pages.yml) runs, on the toolchain pinned in rust-toolchain.toml:

cargo test -p stt-wasm --release     # non-GPU tests only; parity gates need model files
cargo clippy --workspace --all-targets -- -D warnings
ENGINE_BUILD=<tag> scripts/build.sh  # builds the wasm package and assembles _site/

scripts/build.sh needs wasm-pack and a wasm-bindgen-cli whose version matches the one in Cargo.lock. It rewrites the ?v= build tag on the wasm loading URL in web/worker.js to ENGINE_BUILD (CI passes the commit sha, local builds default to dev). Pass a new tag on every real deploy, otherwise browsers keep running the cached module.

python3 scripts/serve.py serves _site/ on localhost, which counts as a secure context for WebGPU.

Architecture

• Mimi codec (mimi-rs, candle, shared with tts-web), compiled into the stt-wasm package: SEANet encoder + transformer + RVQ quantizer. Runs on CPU. Converts 24kHz mono PCM → 32 codebook tokens per frame at 12.5Hz.
• STT transformer (crates/stt-wasm/): 16-layer decoder-only transformer with Q4-quantized weights. Runs on GPU via Burn + wgpu. Predicts text tokens from audio tokens.
• Web UI (web/): AudioWorklet captures mic at 24kHz, Web Worker orchestrates pipeline, SentencePiece decoder produces text.

Testing

# Generate reference data from PyTorch model
source .venv/bin/activate
python scripts/test_reference.py

# Verify Q4 quantization quality
python scripts/test_quantization.py

# Test JS tokenizer
node scripts/test_tokenizer.mjs

# Verify Q4 roundtrip encoding
python scripts/test_q4_roundtrip.py