Repository navigation
Add proportional jitter to the default reconnect delays - #1045
Open
xianshijing-lk wants to merge 1 commit into
Open
xianshijing-lk wants to merge 1 commit into
xianshijing-lk wants to merge 1 commit into
Conversation
DefaultReconnectPolicy returned a fixed delay per retry count, so every client disconnected by the same event waited exactly 100ms, then 300ms, then 300ms, in lockstep. They retried as one herd, and because the schedule is identical they could stay synchronised across attempts rather than drifting apart. Scale each delay by a random factor in [0.5, 1.5) so clients spread further apart with every retry. A first retry configured as zero draws from a fixed 0-500ms window instead, since there is no base delay to scale; a zero at any later index still means retry immediately. Port of livekit/client-sdk-js#2124. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
xianshijing-lk
requested review from
MaxHeimbrock and
davidliu
as code owners
October 23, 2026 22:20
🦋 Changeset detectedLatest commit: 1a36ff7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
Diffuse output: AARJAR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port of livekit/client-sdk-js#2124.
The bug
DefaultReconnectPolicy.getNextRetryDelayreturnedretryDelays[context.retryCount]verbatim — no jitter at any attempt.Every client disconnected by the same event (a server restart, a region failover, a dropped uplink) therefore waits exactly 100ms, then 300ms, then 300ms, in lockstep. They retry as one herd against the server that just came back. Worse, because the schedule is fully deterministic, clients do not drift apart over successive attempts the way they would with any jitter at all — they stay synchronised for the whole backoff.
This is the same defect the JS SDK fixed, and a bit more pronounced here: the pre-fix JS code at least jittered attempts after the second.
Repro
Added
DefaultReconnectPolicyTest. Against the unfixed code:100 callers asking for their first retry delay all received the identical value.
The fix
Scale each delay by a random factor in
[0.5, 1.5), so the spread widens as the base delay grows. A first retry whose configured delay is zero has nothing to scale, so it draws from a fixed 0–500ms window instead; a zero at any later index is an explicit request to retry immediately and still passes through as zero.The remaining five tests in the new file cover the behaviour that must not change: exhausting the delay list, exceeding
maxReconnectionTimeout, custom list lengths, and the later-zero passthrough.Note on behaviour
DefaultReconnectPolicyis public API and callers can supply their ownretryDelays. Those custom delays are now jittered too, which is the intent — the herd problem is independent of who picked the numbers — but it does mean a caller who previously relied on an exact delay will see a range. Flagging in case you'd rather gate it.Verification
:livekit-android-test:testReleaseUnitTest— the two assertions above fail before the change and pass after; full module suite green.Scope
The upstream PR also changed
RTCEngineto route a server-requested reconnect through the policy instead of hardcodingdelay = 0. This SDK has no such shortcut —RTCEngine.ktalready takes its delay solely from the policy — so only the policy half applies here.🤖 Generated with Claude Code