Repository navigation
Re-apply subscribed codecs after a publisher answer #1238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| patch type="fixed" "The last subscribed codecs are now re-applied to the local video senders after each publisher renegotiation, so simulcast layers that dynacast paused are no longer re-enabled by the SFU answer" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -50,6 +50,8 @@ class SimulcastTrackInfo { | |
| }); | ||
| } | ||
|
|
||
| bool _hasPausedLayer(lk_rtc.SubscribedCodec codec) => codec.qualities.any((q) => !q.enabled); | ||
|
|
||
| /// A video track from the local device. Use static methods in this class to create | ||
| /// video tracks. | ||
| class LocalVideoTrack extends LocalTrack with VideoTrack { | ||
|
|
@@ -323,15 +325,48 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| }); | ||
| } | ||
|
|
||
| /// Re-applies the last subscribed codecs to the sender. A publisher answer | ||
| /// re-enables every layer, and dynacast only signals changes, so without this | ||
| /// the layers it paused keep uploading. | ||
| /// | ||
| /// Re-applying a pause needs a forced write: the answer re-enabled the layers | ||
| /// without going through `setParameters`, so where `sender.parameters` is a | ||
| /// local cache it still reports the layers as paused, and a conditional write | ||
| /// would find no change and send nothing. Only the codecs that actually have | ||
| /// a paused layer are forced; see [setPublishingCodecs]. | ||
| @internal | ||
| Future<void> refreshSubscribedCodecs() async { | ||
| if (subscribedCodecs.isEmpty) { | ||
| return; | ||
| } | ||
| try { | ||
| await setPublishingCodecs(subscribedCodecs, this, force: true); | ||
|
Comment on lines
+338
to
+343
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Old pauses affect republished video When a video track is republished, Learn moreA Example: A camera's last subscription disables LOW and MEDIUM; the app unpublishes and republishes the same camera track. Its next publisher answer triggers a forced write of the previous publication's LOW/MEDIUM pause to the new sender. Recommended fix: Reset subscription state when a track starts a new publish session, including unpublish and full reconnect, rather than carrying the old session's codec list into the new sender. Ensure a current-session SFU update can be applied before or after the answer without being overwritten. Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| } catch (e) { | ||
| logger.warning('Failed to refresh subscribed codecs: $e'); | ||
| } | ||
| } | ||
|
|
||
| /// Re-applies [codecs] to the senders publishing them. | ||
| /// | ||
| /// [force] asks for a write even where the sender already reports the state | ||
| /// being applied, for a caller re-asserting a pause the sender lost without | ||
| /// being told. It is narrowed per codec: one with every layer subscribed has | ||
| /// nothing to re-assert, so its sender is left alone. | ||
| Future<List<String>> setPublishingCodecs( | ||
| List<lk_rtc.SubscribedCodec> codecs, | ||
| LocalTrack track, | ||
| ) async { | ||
| LocalTrack track, { | ||
| bool force = false, | ||
| }) async { | ||
| logger.fine('setPublishingCodecs $codecs'); | ||
|
|
||
| // only enable simulcast codec for preference codec setted | ||
| if (codec == null && codecs.isNotEmpty) { | ||
| await setPublishingLayers(track, codecs[0].qualities, isSVC: isSVCCodec(codecs[0].codec)); | ||
| await setPublishingLayers( | ||
| track, | ||
| codecs[0].qualities, | ||
| isSVC: isSVCCodec(codecs[0].codec), | ||
| force: force && _hasPausedLayer(codecs[0]), | ||
| ); | ||
| return []; | ||
| } | ||
|
|
||
|
|
@@ -340,8 +375,9 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| final List<String> newCodecs = []; | ||
|
|
||
| for (var codec in codecs) { | ||
| final forceCodec = force && _hasPausedLayer(codec); | ||
| if (this.codec?.toLowerCase() == codec.codec.toLowerCase()) { | ||
| await setPublishingLayers(track, codec.qualities, isSVC: isSVCCodec(codec.codec)); | ||
| await setPublishingLayers(track, codec.qualities, isSVC: isSVCCodec(codec.codec), force: forceCodec); | ||
| } else { | ||
| final simulcastCodecInfo = simulcastCodecs[codec.codec]; | ||
| logger.fine('setPublishingCodecs $codecs'); | ||
|
|
@@ -359,6 +395,7 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| simulcastCodecInfo.encodings!, | ||
| codec.qualities, | ||
| isSVC: isSVCCodec(codec.codec), | ||
| force: forceCodec, | ||
| ); | ||
| } | ||
| } | ||
|
|
@@ -371,6 +408,7 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| LocalTrack? track, | ||
| List<lk_rtc.SubscribedQuality> layers, { | ||
| bool isSVC = false, | ||
| bool force = false, | ||
| }) async { | ||
| logger.fine('Update publishing layers: $layers'); | ||
|
|
||
|
|
@@ -391,7 +429,7 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| return; | ||
| } | ||
|
|
||
| return setPublishingLayersForSender(track!.sender!, encodings, layers, isSVC: isSVC); | ||
| return setPublishingLayersForSender(track!.sender!, encodings, layers, isSVC: isSVC, force: force); | ||
| } | ||
|
|
||
| lk_models.VideoQuality _videoQualityForRid(String rid) { | ||
|
|
@@ -412,12 +450,13 @@ extension LocalVideoTrackExt on LocalVideoTrack { | |
| List<rtc.RTCRtpEncoding> encodings, | ||
| List<lk_rtc.SubscribedQuality> layers, { | ||
| bool isSVC = false, | ||
| bool force = false, | ||
| }) async { | ||
| logger.fine('Update publishing layers: $layers'); | ||
|
|
||
| final params = sender.parameters; | ||
|
|
||
| var hasChanged = false; | ||
| var hasChanged = force; | ||
|
|
||
| // NOTE: closable spatial layer is disabled due to video blur / frozen issues | ||
| // with Chrome 113+ and LiveKit SFU PLI handling. See JS SDK LocalVideoTrack.ts:529-568. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| // Copyright 2026 LiveKit, Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| import 'dart:typed_data'; | ||
|
|
||
| import 'package:flutter_webrtc/flutter_webrtc.dart' as rtc; | ||
|
|
||
| class FakeMediaStream extends rtc.MediaStream { | ||
| final List<rtc.MediaStreamTrack> _tracks = []; | ||
|
|
||
| FakeMediaStream(String id) : super(id, 'fake-owner'); | ||
|
|
||
| @override | ||
| bool? get active => true; | ||
|
|
||
| @override | ||
| Future<void> addTrack(rtc.MediaStreamTrack track, {bool addToNative = true}) async { | ||
| _tracks.add(track); | ||
| } | ||
|
|
||
| @override | ||
| Future<rtc.MediaStream> clone() async => FakeMediaStream('${id}_clone'); | ||
|
|
||
| @override | ||
| List<rtc.MediaStreamTrack> getAudioTracks() => _tracks.where((t) => t.kind == 'audio').toList(); | ||
|
|
||
| @override | ||
| Future<void> getMediaTracks() async {} | ||
|
|
||
| @override | ||
| List<rtc.MediaStreamTrack> getTracks() => List<rtc.MediaStreamTrack>.from(_tracks); | ||
|
|
||
| @override | ||
| List<rtc.MediaStreamTrack> getVideoTracks() => _tracks.where((t) => t.kind == 'video').toList(); | ||
|
|
||
| @override | ||
| Future<void> removeTrack(rtc.MediaStreamTrack track, {bool removeFromNative = true}) async { | ||
| _tracks.remove(track); | ||
| } | ||
| } | ||
|
|
||
| class FakeMediaStreamTrack implements rtc.MediaStreamTrack { | ||
| @override | ||
| rtc.StreamTrackCallback? onEnded; | ||
|
|
||
| @override | ||
| rtc.StreamTrackCallback? onMute; | ||
|
|
||
| @override | ||
| rtc.StreamTrackCallback? onUnMute; | ||
|
|
||
| @override | ||
| bool enabled; | ||
|
|
||
| @override | ||
| final String id; | ||
|
|
||
| @override | ||
| final String kind; | ||
|
|
||
| @override | ||
| String? get label => '$kind-track'; | ||
|
|
||
| @override | ||
| bool? get muted => false; | ||
|
|
||
| FakeMediaStreamTrack({ | ||
| required this.id, | ||
| required this.kind, | ||
| this.enabled = true, | ||
| }); | ||
|
|
||
| @override | ||
| Future<void> adaptRes(int width, int height) async {} | ||
|
|
||
| @override | ||
| Future<void> applyConstraints([Map<String, dynamic>? constraints]) async {} | ||
|
|
||
| @override | ||
| Future<ByteBuffer> captureFrame() { | ||
| throw UnimplementedError(); | ||
| } | ||
|
|
||
| @override | ||
| Future<rtc.MediaStreamTrack> clone() async => FakeMediaStreamTrack(id: id, kind: kind, enabled: enabled); | ||
|
|
||
| @override | ||
| Future<void> dispose() async {} | ||
|
|
||
| @override | ||
| Map<String, dynamic> getConstraints() => const {}; | ||
|
|
||
| @override | ||
| Map<String, dynamic> getSettings() => const {}; | ||
|
|
||
| @override | ||
| Future<bool> hasTorch() async => false; | ||
|
|
||
| @override | ||
| void enableSpeakerphone(bool enable) {} | ||
|
|
||
| @override | ||
| Future<void> setTorch(bool torch) async {} | ||
|
|
||
| @override | ||
| Future<void> stop() async {} | ||
|
|
||
| @override | ||
| Future<bool> switchCamera() async => false; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Legacy dynacast pauses stay re-enabled
After a legacy
subscribedQualitiesupdate,refreshSubscribedCodecshas no codecs to replay. The legacy update handler never records them, so a publisher answer leaves paused layers uploading.Learn more
The room supports two shapes of dynacast update: a list of subscribed codecs and the older list of subscribed qualities. The latter goes straight to
setPublishingLayersin the legacy update handler, without storing replayable state. After an answer re-enables paused layers,refreshSubscribedCodecsreturns because its codec list is empty. Those layers keep uploading until the SFU changes the subscription again.Example: A server sends only
subscribedQualitiesasking a camera to disable LOW. An audio-track publication triggers a publisher renegotiation; the resulting answer re-enables LOW, and the refresh cannot restore the pause.Recommended fix: Save the last legacy quality update per local video track and replay it through
setPublishingLayerswith a forced write after publisher answers. Clear this state when the corresponding publish session ends.Was this helpful? React with 👍 or 👎 to provide feedback.