Skip to content

Fix audio/video decode for newer H.265 3-lens devices (IMA-ADPCM, RTP timestamps, codec detection) - #5

Open
Vasang123 wants to merge 2 commits into
felipemarques:mainfrom
Vasang123:main
Open

Fix audio/video decode for newer H.265 3-lens devices (IMA-ADPCM, RTP timestamps, codec detection)#5
Vasang123 wants to merge 2 commits into
felipemarques:mainfrom
Vasang123:main

Conversation

@Vasang123

Copy link
Copy Markdown

Summary

Fixes several real bugs affecting newer H.265 3-lens V380 devices, found and fixed
while debugging a dual-camera Frigate NVR integration. Audio was completely garbled and
video intermittently corrupted/dropped on this device family; both are now working
reliably.

Audio

  • The real audio codec on these devices is IMA-ADPCM, not G.711 A-law as the SDP
    claims and the code assumed. Confirmed against
    jericjan/v380-audio-player's
    pyima.py, itself based on acida/pyima. Added a
    proper IMA-ADPCM decoder, then re-encode to real A-law so nothing downstream (SDP,
    RTSP consumers) needs to change.
  • Fixed the packet header offset (16 bytes, not 20) that was also misaligning the
    AES-ECB decryption.
  • Fixed an RTP timestamp bug where the audio clock was reset to 0 on every frame
    instead of increasing, causing playback to glitch once then go silent.

Video

  • Fixed H264/H265 codec misdetection: the check for "is this H264?" (low 5 bits of the
    NAL header) ran before the H265 check on the same byte, and real H265 headers
    frequently satisfy the H264 range by coincidence - silently routing real H265 frames
    around the transcoder and sending raw HEVC labeled as H264.
  • Fixed the video packet header offset (same root cause as audio).
  • Fixed the RTP timestamp using the camera's raw device clock directly (* 90),
    which produced multi-billion-tick jumps and caused clients to drop the connection
    every ~10-15s.
  • Fixed the RTP marker bit being set on every NAL (AUD/SPS/PPS included) instead of
    only the last NAL of an access unit, per RFC 6184.

New flag

  • --no-video: skips starting the HEVC->H264 transcoder entirely, for deployments
    that only need this instance's audio track (video coming from the camera's own
    native RTSP instead). Avoids burning CPU on an unused encode.

Backward compatibility

The two changes with the widest blast radius are gated rather than unconditional:

  • The header offset / ADPCM decode only activates when audioBits==16 (read from the
    login handshake). Devices reporting audioBits==8 (older G.711 cameras) keep the
    original 20-byte offset and pass-through behavior untouched.
  • Codec detection checks the unambiguous H264 VCL slice types (1/5) first - these are
    what real video frames actually use, and real H265 headers essentially never produce
    them under the H264 5-bit reading - before falling back to the (now reordered)
    broader H264/H265 range checks.

I don't have access to the older camera hardware (referenced as Camera A/B/C in the
README) to test directly, so this gating is reasoned from the protocol fields rather
than verified against real older-device hardware. Flagging that explicitly in case
anyone hitting this PR does have one of those units to test against.

Testing

Validated against a dual-cam (2-lens PTZ) unit — see Camera D entry added to the
README, running both units simultaneously through Frigate NVR: clean video (no more
corruption/dropped frames), clean continuous audio, stable RTSP connections over
extended runs (zero reconnects vs. one dropping every ~10-15s before the timestamp fix).

Note

These fixes were developed with AI assistance during a live debugging session against real hardware; noted in the README as well. Some results may vary across devices depending on internet connection and stability.

- Audio: correct IMA-ADPCM decode (was assumed G.711 A-law), fix RTP
  timestamp reset bug, fix packet header offset - gated on audioBits==16
  so older 8-bit G.711 devices keep their original behavior untouched
- Video: fix H264/H265 codec misdetection (staged detection, unambiguous
  VCL types checked first), fix header offset, fix RTP timestamp
  (wall-clock based instead of raw device clock), fix marker bit per
  RFC 6184 (only on last NAL of access unit)
- Add --no-video flag to skip the HEVC->H264 transcoder for
  audio-only deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant