fix(sfu): only advertise the primary public IP in ICE candidates#19
Merged
Conversation
LiveKit was offering host candidates for every local mapping: the real
public IP, the DO reserved IP (NAT'd via the anchor IP 10.48.x), the VPC
IP, and the docker bridge. ICE flapped between the real-IP and
reserved-IP pairs (10 switches in 30 min in prod logs), and the reserved
IP's NAT drops UDP mid-session — publishers died with dtls timeouts and
clients surfaced "could not establish pc connection" a minute or two
into streaming. Pin candidate gathering to ${EXTERNAL_IP} via
rtc.ips.includes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
livekit-server rejects bare IPs in rtc.ips.includes with 'invalid CIDR address' and crash-loops. Verified on sfu.pairux.com: with /32 the server starts and advertises only the primary IP. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the recurring "Streaming error: could not establish pc connection" that hits desktop clients a minute or two into a stream (while YouTube/Twitch egress keeps working).
Root cause (from prod logs on sfu.pairux.com)
LiveKit advertises host candidates for every local IP mapping on the droplet: the real public IP (146.190.163.128), the DO reserved IP (134.199.140.185, NAT'd through the anchor IP 10.48.x — not bound to any interface), the VPC IP, and the docker bridge. Prod logs showed ICE flapping between the real-IP and reserved-IP candidate pairs (10
ice reconnected or switched pairevents in 30 minutes), followed by publisherdtls timeouterrors when the reserved-IP NAT dropped the UDP flow mid-session. The client then surfacescould not establish pc connectionon its rejoin.Nothing resolves to the reserved IP (sfu.pairux.com → 146.190.163.128), so those candidates are pure liability.
Fix
Pin candidate gathering to the primary IP via
rtc.ips.includes: [${EXTERNAL_IP}]in the generatedlivekit.yaml. The setup script is idempotent — re-run it on the droplet (or apply the same block by hand +docker compose restart livekit) to take effect.Test plan
bash -nsyntax check; pre-commit lint/build greenice reconnected or switched pairflapping to 134.199.140.185 and no client pc-connection errors🤖 Generated with Claude Code