Repository navigation
Declare one spatial layer per stream for single-track video publishes - #1045
pabloFuente wants to merge 2 commits into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if kind == TrackKindVideo { | ||
| layerMode = livekit.VideoLayer_ONE_SPATIAL_LAYER_PER_STREAM | ||
| } |
There was a problem hiding this comment.
🔴 Custom SVC tracks lose spatial layers
Publishing a custom SVC VP9 or AV1 track now declares layerMode as single-layer. The server previously inferred multilayer mode from those codecs. Subscribers lose access to the other spatial layers.
Learn more
PublishTrack accepts any webrtc.TrackLocal, including implementations of TrackLocalWithCodec that emit SVC VP9 or AV1 RTP. A single RTP stream can contain multiple SVC spatial layers; only the SDK's LocalTrack sample implementation is known to emit a single spatial layer. Previously the unset mode let the server infer multilayer mode for VP9 and AV1. The unconditional mode now instructs the server to treat those custom streams as one spatial layer, so subscribers cannot request their higher SVC layers.
Example: A caller supplies a custom VP9 track with three SVC spatial layers to PublishTrack. Its codec now signals one layer even though its RTP contains three; the server cannot expose the two extra layers as selectable qualities.
Recommended fix: Set ONE_SPATIAL_LAYER_PER_STREAM for tracks known to be single-layer, such as LocalTrack, and provide a way for other TrackLocalWithCodec publishers to declare their actual spatial-layer mode. Apply the chosen mode to the backup codec only after checking that backup track's own encoding.
Was this helpful? React with 👍 or 👎 to provide feedback.
The previous commit set ONE_SPATIAL_LAYER_PER_STREAM on the codec entries of every single-track video publish. But publishTrack accepts any webrtc.TrackLocal, and a caller-owned track may relay SVC VP9 or AV1. Declaring one layer for it makes the server stop treating it as SVC, which is a change from v2.19.0, where the server inferred the mode from the codec.
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if _, ok := track.(*LocalTrack); ok && kind == TrackKindVideo { | ||
| return livekit.VideoLayer_ONE_SPATIAL_LAYER_PER_STREAM | ||
| } | ||
| return livekit.VideoLayer_MODE_UNUSED |
There was a problem hiding this comment.
🔴 Single-layer external tracks declared as SVC
For a single-layer VP9 or AV1 TrackLocalStaticSample, sampleTrackLayerMode returns MODE_UNUSED because it is not *LocalTrack. The server infers multiple spatial layers, mislabeling the primary or backup video stream as SVC.
Learn more
PublishTrack accepts any webrtc.TrackLocal, and prepareTrackPublication declares the codec for external tracks implementing TrackLocalWithCodec. The server infers multiple spatial layers for VP9 and AV1 when VideoLayerMode is unset. The concrete type test only recognizes SDK LocalTrack, so a single-layer Pion sample track gets an unset mode even though its samples contain only one spatial layer. The backup codec entry uses the same helper.
Example: Publish a VP9 webrtc.TrackLocalStaticSample with ordinary single-layer samples. Its codec is declared, but its video layer mode is unset; the server reports MULTIPLE_SPATIAL_LAYERS_PER_STREAM rather than ONE_SPATIAL_LAYER_PER_STREAM.
Recommended fix: Set ONE_SPATIAL_LAYER_PER_STREAM for known single-layer sample tracks beyond *LocalTrack, including Pion's static sample tracks. Preserve an explicit route for external tracks that genuinely carry SVC if that behavior is required.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
What's wrong
Since v2.19.0 (#989),
PublishTrackreports a plain VP9 or AV1 track to the server as SVC. It is a single RTP encoding. A sample track can't carry SVC.You can see it in the
TrackInfothe server returns:codecs[0].video_layer_modeONE_SPATIAL_LAYER_PER_STREAMMULTIPLE_SPATIAL_LAYERS_PER_STREAMONE_SPATIAL_LAYER_PER_STREAMWhy it happens
SimulcastCodecs. The server made up a codec entry withONE_SPATIAL_LAYER_PER_STREAM.SimulcastCodecsfor video, so the track's codec is known. The new entries leaveVideoLayerModeunset.MULTIPLE_SPATIAL_LAYERS_PER_STREAM.The simulcast path already sets
ONE_SPATIAL_LAYER_PER_STREAMon its entries, and the single-track path didn't. VP8 and H264 are unaffected, because the server only infers SVC for VP9 and AV1.The fix
In
prepareTrackPublication, setVideoLayerMode: ONE_SPATIAL_LAYER_PER_STREAMon the primary codec entry and on the backup codec entry. This is what the simulcast path already does. Audio is unchanged.How I found it
Publishing plain VP9 and AV1 with server-sdk-go and asserting the server reports one spatial layer per stream. This passes with v2.18.1 and fails with v2.19.0.