feat(audio): AudioManager audio session management#1108
Conversation
Process-wide audio session control on AudioManager: session options and management modes (automatic/manual), Android audio session configuration (mode/focus/routing) backed by an AudioSwitch-based manager in the native plugin, Apple speakerphone routing, and speaker output preferences. Platform audio sessions are global to the app process, so this lives on AudioManager rather than Room.
Picks up the aligned audioswitch revision (flutter-webrtc#2084), which resolves the Android build conflict on the shared classpath.
hiroshihorie
requested review from
cloudwebrtc and
xianshijing-lk
as code owners
|
Caution Breaking change detected without major changeset
If this is intentional, please add a changeset with |
There was a problem hiding this comment.
Pull request overview
Introduces first-class, process-wide audio session/routing management via AudioManager, consolidating previously scattered session behavior (track-counting, Hardware, and implicit native defaults) into a single, typed API with native backends on Apple and Android.
Changes:
- Adds typed audio-session configuration (
AudioSessionOptions, per-platform overrides, andAudioSessionManagementMode) plus supporting copy/serialization helpers. - Updates native plugins to let LiveKit own platform audio sessions: iOS/macOS engine-lifecycle observer drives session activation; Android adds an AudioSwitch-based manager for focus/mode/routing.
- Deprecates/rewires legacy entry points (
Hardware, track audio management mixins) to forward throughAudioManager, and updates examples + adds unit tests.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/audio/audio_session_test.dart | Adds unit tests covering session options, copyWith semantics, and serialization. |
| shared_swift/LiveKitPlugin.swift | Disables flutter_webrtc audio management; adds engine-lifecycle observer + new method-channel handlers for Apple session management and speaker routing. |
| pubspec.yaml | Bumps flutter_webrtc dependency to 1.5.1. |
| pubspec.lock | Locks flutter_webrtc to 1.5.1 with updated hash. |
| lib/src/track/remote/audio.dart | Removes legacy remote audio management mixin usage. |
| lib/src/track/local/audio.dart | Removes legacy local audio management mixin usage. |
| lib/src/track/audio_management.dart | Replaces track-counting session management with AudioManager connect/apply and Android stop handling. |
| lib/src/support/value_or_absent.dart | Adds ValueOrAbsent utility to support nullable-field copyWith APIs. |
| lib/src/support/native.dart | Extends Apple configureNativeAudio payload + adds Android/Apple session/routing method-channel APIs and engine-state callback handling. |
| lib/src/support/native_audio.dart | Updates NativeAudioConfiguration.copyWith to use ValueOrAbsent; removes old static presets. |
| lib/src/livekit.dart | Initializes AudioManager defaults and passes Android init configuration when needed. |
| lib/src/hardware/hardware.dart | Deprecates audio-related members and forwards to AudioManager. |
| lib/src/core/room.dart | Ensures audio session start/stop is cleaned up on connect failure; routes speaker toggles via AudioManager. |
| lib/src/audio/audio_session.dart | New public API types for session intent + per-platform configuration. |
| lib/src/audio/audio_manager.dart | Implements the new process-wide audio session manager, including engine-state stream and platform apply logic. |
| lib/src/audio/android_audio_session_adapter.dart | Serializes Android session configuration to method-channel wire format and applies it via native. |
| lib/livekit_client.dart | Exports the new audio session API. |
| example/lib/widgets/controls.dart | Updates example UI to read speaker state from AudioManager. |
| example/lib/pages/room.dart | Updates example to toggle speaker via AudioManager. |
| android/src/main/kotlin/io/livekit/plugin/LKAudioSwitchManager.kt | Adds AudioSwitch-based Android session/focus/mode/routing manager. |
| android/src/main/kotlin/io/livekit/plugin/LiveKitPlugin.kt | Disables flutter_webrtc audio management; wires method-channel handlers to LKAudioSwitchManager. |
| android/build.gradle | Adds JitPack repository and AudioSwitch dependency. |
| .changes/audio-manager-api | Adds a changeset entry documenting the new API. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| allprojects { | ||
| repositories { | ||
| google() | ||
| mavenCentral() | ||
| maven { url 'https://jitpack.io' } |
There was a problem hiding this comment.
Thanks for flagging this. A quick bit of context here. flutter_webrtc, which is a direct dependency of this plugin, already declares the JitPack repository at rootProject.allprojects scope, so every consuming app already gets it applied app wide. This block uses plain allprojects with no rootProject. prefix, which is actually a narrower scope, so it does not add anything the host app was not already getting transitively through flutter_webrtc.
We also pin the same audioswitch commit that flutter_webrtc and the LiveKit Android SDK use, so dependency resolution stays consistent across the whole stack. Keeping our own declaration here just makes the plugin self sufficient rather than relying on the transitive one.
| // Audio device/focus/mode routing. Pinned to the same revision used by | ||
| // the LiveKit Android SDK (AudioSwitchHandler). | ||
| implementation 'com.github.davidliu:audioswitch:039a35aefab7747c557242fa216c9ea11743b604' |
There was a problem hiding this comment.
Thanks. This one is a deliberate tradeoff, and it matches the rest of the LiveKit stack. The same audioswitch commit is pinned by both flutter_webrtc and the LiveKit Android SDK, so dependency resolution stays consistent end to end. A JitPack artifact is immutable per commit, so the pin is reproducible in practice, and the main residual risk is JitPack availability rather than the artifact changing under us.
The fork is needed for its CommDeviceAudioSwitch, which provides the API 31 setCommunicationDevice routing path, and that class has no tagged release or MavenCentral coordinate to point at yet. If upstream publishes a release that carries it, we would happily switch over.
| "configureAndroidAudioSession" -> { | ||
| @Suppress("UNCHECKED_CAST") | ||
| val configuration = call.arguments as? Map<String, Any?> ?: emptyMap() | ||
| audioSwitchManager?.configure(configuration) | ||
| audioSwitchManager?.start() | ||
| result.success(null) | ||
| } |
There was a problem hiding this comment.
Good observation. The coupling here is intentional on Android. Unlike iOS, the Android side has no audio engine lifecycle observer to defer activation to, so the session is activated when LiveKit applies its options (at connect, or on an explicit apply) and released on stop or disconnect. That keeps focus and mode ownership tied to a clear start and stop, which is the standard call session pattern.
Separating configure from activate, or adding an activate flag, is a reasonable future enhancement for finer grained manual mode control, and we can revisit it if a concrete need shows up. For this change the current behavior is the intended one, so we will keep it as is for now.
| // Leave sessionActive untrue so cached state still reflects the | ||
| // live session. Flipping it to false here would make a later | ||
| // configureNativeAudio(automatic:) cache-only while the session | ||
| // is in fact still active. |
Relocate AppleAudioCategory / AppleAudioCategoryOption / AppleAudioMode from the internal wire file (support/native_audio.dart) into audio_session.dart, where the public AppleAudioSessionConfiguration already lives. native_audio.dart keeps only the internal NativeAudioConfiguration wire type and its toStringValue serialization, importing the enums from the audio layer. Removes the import-plus-export of native_audio from audio_session.dart. Public API and behavior are unchanged.
The automatic-mode session expressed the speaker preference twice: through the audio mode (videoChat for speaker, voiceChat for receiver) and again through overrideOutputAudioPort. The override hard-routes to the speaker even over a connected headset, so a plain setSpeakerphoneOn(true) ignored the documented headset priority and behaved like a forced speaker. Gate the speaker override on the forced case only (carried as the defaultToSpeaker category option) and otherwise clear it, letting the audio mode and connected devices decide. This matches the Swift SDK, which selects playAndRecordSpeaker/playAndRecordReceiver by mode and never overrides the output port. Removes the now-unused preferSpeakerOutput plumbing from the native observer and apply path. Manual mode keeps its direct route override.
Rename the new (unreleased) AudioManager speaker surface for clarity and to
match the Swift SDK vocabulary:
setSpeakerphoneOn(enable, {forceSpeakerOutput}) -> setSpeakerOutputPreferred(preferred, {force})
get preferSpeakerOutput -> get isSpeakerOutputPreferred
get forceSpeakerOutput -> get isSpeakerOutputForced
Drop the duplicate speakerphoneOn getter (was identical to preferSpeakerOutput).
Update the deprecated Hardware forwards, Room.setSpeakerOn, the example app,
the audio guide, and the tests. AudioSessionOptions.preferSpeakerOutput and the
per-platform config fields keep their names. No behavior change.
The headset-priority fix made the native side ignore the preferSpeakerOutput wire field (the speaker preference now lives entirely in the audio mode). Remove the now-dead field from NativeAudioConfiguration (field, toMap, copyWith) and stop sending it from the resolved Apple policy. The wire-format test now pins its absence.
| await setAndroidAudioSessionConfiguration(config); | ||
| await Native.setAndroidSpeakerphoneOn(policy.preferSpeakerOutput); |
| val device = if (enable) { | ||
| switch.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone } | ||
| } else { | ||
| switch.availableAudioDevices.firstOrNull { | ||
| it is AudioDevice.BluetoothHeadset || it is AudioDevice.WiredHeadset || it is AudioDevice.Earpiece | ||
| } | ||
| } | ||
| switch.selectDevice(device) |
setSpeakerphoneOn(true) explicitly selected the speaker device even when a wired/Bluetooth headset was connected, forcing the speaker and ignoring the documented headset priority. Select a connected headset first and fall back to the speaker only when none is present (earpiece when the speaker is not preferred). This mirrors the iOS fix and removes the forced-speaker routing the media preset would otherwise apply on every options change.
| } else { | ||
| // Manual mode: route without re-applying category/mode the app owns. | ||
| await Native.setAppleSpeakerphoneOn(preferred, force: _forceSpeakerOutput); | ||
| } |
| // AudioSwitch is not threadsafe, so confine all access to a single long-lived | ||
| // thread. Do not recreate it on stop/start. Queued lifecycle work must stay | ||
| // serialized. |
| @@ -0,0 +1,174 @@ | |||
| # Audio session management | |||
|
|
|||
| LiveKit owns the platform audio session on iOS, macOS, and Android through a single process-wide entry point, `AudioManager`. You configure session intent once with typed options and LiveKit applies the right native category, mode, focus, and routing for you. This replaces the older approach that relied on flutter_webrtc defaults, `Hardware` speaker calls, and Dart track counting. | |||
| /// This intentionally does not expose [preferSpeakerOutput] because speaker | ||
| /// preference only affects communication/playAndRecord policies. Use [apple] | ||
| /// or [android] for exact platform behavior, or switch at runtime with | ||
| /// `AudioManager.setSpeakerOutputPreferred`. |
| /// Selects whether LiveKit manages the platform audio session automatically. | ||
| /// | ||
| /// In [AudioSessionManagementMode.manual], LiveKit does not update the audio | ||
| /// session from room, connect, or track lifecycle. The app can still apply a | ||
| /// configuration explicitly with [setAudioSessionOptions]. | ||
| /// | ||
| /// Prefer setting this before connecting to a room. flutter_webrtc's own | ||
| /// native audio management is always disabled (LiveKit owns the session). | ||
| /// Changing the mode at runtime only affects LiveKit's own automatic | ||
| /// configuration. | ||
| Future<void> setAudioSessionManagementMode(AudioSessionManagementMode mode) async { | ||
| _managementMode = mode; | ||
| await _syncAppleAudioSessionManagementMode(); | ||
| } |
|
@copilot Are your findings addressed with latest commits ? |
Yes, all findings are addressed in the latest commits. Here's what changed per thread:
|
|
|
||
| ## Per platform overrides | ||
|
|
||
| When the presets are not enough you can pin exact platform values. Anything you do not set falls back to the preset behavior. |
What
First-class, process-wide audio session control on
AudioManager— replacing session/routing behavior that was scattered acrossHardware,track/audio_management.dart, and implicit native defaults.API
Typed session configuration —
AudioSessionOptionswithcommunication()/media()presets and per-platform escape hatches:AppleAudioSessionConfiguration: category / mode / category optionsAndroidAudioSessionConfiguration: audio mode, focus mode, stream type, audio attributes usage/content typesAutomatic vs manual management —
AudioSessionManagementMode. Defaultautomaticpreserves today's behavior (the SDK tracks local/remote audio presence and configures the session).manuallets apps that own their audio lifecycle drive it themselves.Android session backend — a new AudioSwitch-based
LKAudioSwitchManagerin the native plugin handles device routing, audio focus acquisition/release, and communication-device management, driven byconfigureAndroidAudioSession/stopAndroidAudioSessionchannel methods.Unified speakerphone/routing —
setSpeakerphoneOn,preferSpeakerOutput,forceSpeakerOutputmove toAudioManagerwith consistent Apple (overrideOutputAudioPort) and Android (AudioSwitch) implementations.Compatibility
Hardware's audio members become deprecated forwarders toAudioManager— migration, not removal.RoomusesAudioManagerinternally; no behavior change for apps that touch nothing.Why AudioManager
Platform audio sessions are global to the app process — they cannot be per-
Roomor per-track. This is the second half of makingAudioManagerthe home for process-wide audio: #1107 added the engine-wide processing state read-back, this adds session/routing control — mirroring the Swift SDK'sAudioManager.Notes
AudioManagersurfaces are unified in one class.minor type="added").