Recreate audio source and republish track on sample-rate change#312
Open
MaxHeimbrock wants to merge 5 commits into
Open
Recreate audio source and republish track on sample-rate change#312MaxHeimbrock wants to merge 5 commits into
MaxHeimbrock wants to merge 5 commits into
Conversation
The native (Rust) audio source was created with a hardcoded sample rate (48000) and channel count (2). Microphone frames flow through Unity's audio graph (AudioProbe) at the actual DSP output configuration, which often differs — e.g. with a Bluetooth headset. The Rust source does not resample; it rejects frames whose rate/channels don't match, causing the metadata-mismatch warning and capture failures. Read the source's sample rate and channel count from Unity's output configuration (AudioSettings.GetConfiguration) instead of hardcoded defaults, falling back to the defaults only when Unity can't report one. The base constructor now exposes a device-mode overload (type only) and an explicit overload (type, sampleRate, channels) for sources that generate a fixed format. MicrophoneSource and BasicAudioSource use device mode; BasicAudioSource drops its unused channels parameter. SineWaveAudioSource declares its exact format. If a frame's format still doesn't match (inconsistent Unity report or a runtime output change), drop it with a throttled warning instead of sending a mismatch the native side would error on. Also removes the redundant Microphone.Start in the Meet sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Remove logging changes
When a capture device disappears mid-call (e.g. unplugging a Bluetooth headset), the local microphone went silent and never recovered: MicrophoneSource stayed bound to the gone device name and never re-registered the AudioProbe tap that Unity detaches when it rebuilds its audio graph. Subscribe to AudioSettings.OnAudioConfigurationChanged (mirroring the playback-side AudioStream handler) and restart capture on a device change, resolving to the OS default device when the preferred device is no longer present. Track the active device separately from the preferred one so Microphone.IsRecording/GetPosition/End target the right device, and guard against overlapping restarts. The native source's rate is fixed at construction, so if the device change moves Unity's DSP output rate, frames are still dropped; warn clearly in that case. Full rate-change recovery follows separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MaxHeimbrock
force-pushed
the
max/mic-restart-on-device-change
branch
from
37725a6 to
d19145d
Compare
Commit 1 restarts capture on a device change but the native source's rate stays fixed at construction, so a device whose rate differs from the original silently has every frame dropped. Add RtcAudioSource.Reconfigure(sampleRate, channels): it disposes the old native handle, rebuilds the source at the new format, and raises a new FormatChanged event. The native source stays alive via the track's reference until the track is dropped, so disposing the handle before the old track is unpublished is safe. The FFI exposes no in-place source reconfigure, so a fresh source (and re-bound track) is required. MicrophoneSource detects the format change via ResolveDeviceFormat in its config-changed handler and calls Reconfigure inside the restart while capture is paused (no AudioRead callbacks in flight). MeetManager subscribes to FormatChanged and republishes the local audio track against the source's new handle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MaxHeimbrock
force-pushed
the
max/mic-rate-change-republish
branch
from
9de6396 to
817b1a6
Compare
MaxHeimbrock
commented
Jun 15, 2026
| if (sampleRate == 0 || channels == 0) return false; | ||
| if (sampleRate == _expectedSampleRate && channels == _expectedChannels) return false; | ||
|
|
||
| Utils.Debug($"{DebugTag} reconfigure {_expectedSampleRate}/{_expectedChannels} -> {sampleRate}/{channels}"); |
Contributor
Author
There was a problem hiding this comment.
Without this change, we can have:
LiveKit: MicrophoneSource: audio device change moved the DSP output rate to 44100Hz, but the native source is fixed at 48000Hz. Captured frames will be dropped until the track is recreated at the new rate.
UnityEngine.Logger:LogWarning (string,object)
LiveKit.Internal.Utils:Warning (object) (at /Users/maxheimbrock/dev/unity/client-sdk-unity/Runtime/Scripts/Internal/Utils.cs:41)
LiveKit.MicrophoneSource:OnAudioConfigurationChanged (bool) (at /Users/maxheimbrock/dev/unity/client-sdk-unity/Runtime/Scripts/MicrophoneSource.cs:248)
UnityEngine.AudioSettings:InvokeOnAudioConfigurationChanged (bool) (at /Users/bokken/build/output/unity/unity/Modules/Audio/Public/ScriptBindings/Audio.bindings.cs:413)
d19145d to
442e2b3
Compare
MaxHeimbrock
force-pushed
the
max/mic-restart-on-device-change
branch
from
442e2b3 to
ace30fa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#311 restarts capture on a device change, but the native audio source's rate is fixed at construction. If the new device has a different sample rate,
RtcAudioSourcesilently drops every captured frame (it rejects frames whose rate/channels don't match the source) — so audio stays dead.The FFI exposes no in-place source reconfigure (only
NewAudioSource), so recovery requires a fresh native source and a re-bound track.Change
RtcAudioSource.Reconfigure(sampleRate, channels): disposes the old native handle, rebuilds the source at the new format, and raises a newFormatChangedevent. The native source stays alive via the track's reference until the track is dropped, so disposing the handle before the old track is unpublished is safe.MicrophoneSourcedetects the format change viaResolveDeviceFormatin its config-changed handler and callsReconfigureinside the restart while capture is paused (noAudioReadcallbacks in flight), replacing the warning from Restart microphone capture on audio device change #311.MeetManagersubscribes toFormatChangedand republishes the local audio track against the source's new handle (unsubscribing on unpublish/cleanup).Note
Republishing yields a new track SID, so remote peers resubscribe on a rate change — acceptable but observable.
Testing
Scripts~/run_unity.sh build macos→ Build SUCCEEDED.🤖 Generated with Claude Code