feat(livekit): add a stereo option to TrackPublishOptions - #1330
Conversation
The server only adds stereo=1;maxaveragebitrate=510000 to the publisher's SDP answer (and sprop-stereo=1 for subscribers) when the published track carries the TF_STEREO audio feature. The Rust SDK never sets it, so a stereo AudioSource is downmixed to mono by the Opus encoder and encoded in speech mode regardless of channel count or max_bitrate. Add TrackPublishOptions::stereo (default false, unchanged behaviour) and forward it as both AddTrackRequest::stereo and the TF_STEREO audio feature for audio tracks. Expose the same flag through the FFI TrackPublishOptions message so the Node and Python SDKs can opt in as well.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| /// Publish the audio track as stereo. The server only signals `stereo=1;maxaveragebitrate=510000` | ||
| /// to the publisher (and `sprop-stereo=1` to subscribers) when the track carries the | ||
| /// `TF_STEREO` audio feature, so stereo sources are otherwise downmixed to mono by the encoder. | ||
| pub stereo: bool, |
There was a problem hiding this comment.
🟡 Release notes entry missing for the new publishing option
The repository requires every pull request to include a changeset file describing the change and the crates to bump, but this change adds a new public publishing option (stereo at livekit/src/room/options.rs:127) without adding one, so the release notes and version bumps will be missing this feature.
Impact: The new stereo option ships without documentation in release notes and the affected crates may not be version-bumped.
Missing /.changeset entry required by AGENTS.md
AGENTS.md ("Documenting changes") states: "Every PR needs a changeset" and "Changeset must list any crates which need to be bumped stemming from the change". The current /.changeset directory only contains add_caching_to_livekit_token_source_crate.md and automatically_retry_webrtc_build_downloads.md from previous PRs; no file was added for this change, which touches livekit (new public field on TrackPublishOptions) and livekit-ffi (new proto field and conversion).
Prompt for agents
AGENTS.md requires a changeset for every PR, listing the crates that need to be bumped. Add a markdown changeset file under /.changeset (following the format of the existing files there, e.g. add_caching_to_livekit_token_source_crate.md) describing the new TrackPublishOptions::stereo option and the corresponding FFI proto field, and listing both the livekit and livekit-ffi crates for version bumps.
Was this helpful? React with 👍 or 👎 to provide feedback.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 33a5f23a89
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| optional DegradationPreference degradation_preference = 13; | ||
| // Publish an audio track as stereo. Required for the server to signal | ||
| // stereo=1 to the publisher and sprop-stereo=1 to subscribers. | ||
| optional bool stereo = 14; |
There was a problem hiding this comment.
Adding this FFI proto field is not enough for the checked-in Node package: livekit-ffi-node-bindings/proto/room_pb.js still defines TrackPublishOptions only through field 13, and the .d.ts type likewise has no stereo property. In contexts using the committed @livekit/rtc-ffi-bindings package, a Node caller cannot serialize tag 14 from new TrackPublishOptions({ stereo: true }), so the FFI conversion here never sees opts.stereo and Node still cannot opt into stereo until those generated bindings are updated.
Useful? React with 👍 / 👎.
1 participant
Problem
An audio track published from this SDK is always encoded as mono, in speech mode, regardless of how many channels the
AudioSourcewas created with or how highaudio_encoding.max_bitrateis set.The server decides stereo on the publisher side in
configurePublisherAnswer(livekit/livekit,pkg/rtc/participant_sdp.go): it appendsstereo=1;maxaveragebitrate=510000to the Opusfmtpline of the answer only when the track was published with theTF_STEREOaudio feature, and stripsstereo=1otherwise. It also gatessprop-stereo=1for subscribers on the same flag (pkg/rtc/transport.go,configureSenderAudio).This SDK never sets that flag.
publish_trackfillsAddTrackRequestwithdisable_dtxand the encodings, but nothing ever pushesAudioTrackFeature::TfStereoor setsAddTrackRequest::stereo— so the answer comes back withoutstereo=1, libwebrtc's Opus encoder downmixes to mono and selects speech mode, and subscribers negotiate a mono decoder.The practical effect: a server-side music bot built on
@livekit/rtc-node(which is this crate through the FFI) cannot publish stereo music at all. Raisingmax_bitratedoes not help, because the encoder never learns it is sending stereo.Change
TrackPublishOptions::stereo, defaulting tofalseso existing behaviour is unchanged.stereo: true, setAddTrackRequest::stereoand pushAudioTrackFeature::TfStereo.optional bool stereofield to the FFITrackPublishOptionsmessage and map it in the conversion, so the Node and Python SDKs can opt in without another protocol change.14 added lines, no behaviour change for callers that do not set the flag.
Verification
cargo check -p livekitis clean on this branch.I could not build
livekit-ffilocally:yuv-sys's build script needs thelibyuvsubmodule contents, which my environment did not have. The FFI part of this diff is one proto field plus one line inFrom<proto::TrackPublishOptions> for TrackPublishOptionsthat mirrors the existingpreconnect_bufferline, so I would appreciate CI confirming that half.Notes
Happy to adjust naming (
stereovs. mirroring the JS SDK's terminology), or to gate it behind the audio encoding options instead of a top-level flag, if you prefer a different shape.