Skip to content

ONNX demo: 10 min audio, single resample, Streamlit NMS 0.7 - #25

Merged
El1as42 merged 3 commits into
mainfrom
feat/onnx-demo-long-audio
Oct 21, 2026
Merged

El1as42 merged 3 commits into
mainfrom
feat/onnx-demo-long-audio

Conversation

@El1as42

@El1as42 El1as42 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Streamlit NMS IoU 0.5 → 0.7. The CLI default and the ONNX graph both use 0.7, and the window fusion constants were tuned with it. On 13 test recordings, 0.5 alone changed about 3% of the Just-Bird boxes.
  • ONNX demo accepts up to 10 minutes of audio (was 60 s), the same limit as the Streamlit demo. The ONNX graph has no length limit of its own: it walks the file in 60 s PCEN segments inside an ONNX Loop.
  • Audio is resampled once. It is decoded straight into a 32 kHz OfflineAudioContext. Before, the default AudioContext resampled to the hardware rate (often 48 kHz) first.
  • The demo sends unpadded audio to the graph. The graph pads to the clip grid itself and trims detections before window fusion, as detect_birds.py does.
  • The spectrogram is drawn as 60 s canvas tiles. A single canvas broke at about 5.5 min (browser limit of about 32k px).
  • Removed a duplicate write_metadata in onnx_export.py.

Parity check

detect_birds.py with models/Just-Bird.pt (conf 0.18, phrase gap 0.05) against the all-in-one graph, run in Python ONNX Runtime. Audio: tests/test.wav plus the 12 parity soundscapes, 1231 reference boxes. A box counts as matched at IoU ≥ 0.5.

Compared with detect_birds.py (NMS 0.7) Matched Mean IoU
ONNX fp32 99.7% 0.999
ONNX fp16 (docs/models) 99.6% 0.995
Streamlit before this PR (NMS 0.5) 97.3% 0.965

Browser demo in headless Firefox, compared with Python ONNX on librosa-resampled audio (60 s excerpts):

Input Old page (double resample) This PR
32 kHz 16 boxes vs 15, mean IoU 0.993 15 vs 15, mean IoU 0.998
48 kHz 40/42 matched, 0.933 41/42, 0.970
44.1 kHz 16/18 matched, 0.935 16/18, 0.969

For input that isn't 32 kHz, the remaining gap comes from the browser's resampler. The page now says so.

Test plan

  • mkdocs build, then the demo page in headless Firefox on 300 s (32 kHz) and 200 s (48 kHz) files: 104 and 129 boxes, 5 and 4 spectrogram tiles, 25 s and 15 s of inference
  • The 300 s run matches Python ONNX: 104/104 boxes, mean IoU 0.996
  • Screenshot across a tile edge: no visible seam, and boxes and labels continue across it
  • onnx_export.py compiles
  • Full 10-minute files (11 min input, cut to 600 s), 32 kHz mono and 48 kHz stereo:
    • Firefox 136 (headless): 344 and 354 boxes in 48 s and 53 s; 10 spectrogram tiles; largest process peaked at 2.2 GB, about 1.1 GB during inference
    • Chromium 156 (Playwright): 344 and 357 boxes in about 47 s each; browser process about 1.1 GB during inference
    • All three engines give 344/344 boxes against Python ONNX on the 32 kHz file (mean IoU 0.998)
  • WebKit 18.2 (Playwright's frozen Ubuntu 20.04 build, the closest to Safari on Linux): results are correct (344/344 on 600 s), but its web process used 4.9 GB on a 60 s file and up to 12 GB on 600 s, and one 600 s run crashed the page. The page before this PR uses the same 4.9 GB on the same 60 s file, so this predates the PR and depends on ONNX Runtime Web in this WebKit build, not on the audio length
  • Real Safari on macOS / iOS (not available on Linux); WebKit's memory use makes this the one to check

🤖 Generated with Claude Code

El1as42 and others added 3 commits October 21, 2026 13:42
Streamlit hard-coded 0.5 while the CLI default and the ONNX export use
0.7, the value the window fusion constants were tuned with. On 13 test
recordings this alone changed about 3% of the Just-Bird boxes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Limit raised from 60 s to 600 s, as in the Streamlit demo. The graph
  itself has no length limit.
- Decode straight into a 32 kHz OfflineAudioContext. The default
  AudioContext resampled to the hardware rate first, so 32 kHz files
  were resampled twice.
- Feed the unpadded audio: the graph pads to the clip grid and trims
  detections before window fusion, as detect_birds.py does.
- Spectrogram drawn as 60 s canvas tiles; one canvas broke past the
  browser limit of about 32k px (5.5 min).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@El1as42
El1as42 merged commit 9aebee7 into main Oct 21, 2026
1 check passed
@El1as42
El1as42 deleted the feat/onnx-demo-long-audio branch October 21, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant