[DRAFT] Runtime controls for microphone audio processing and bitrate - #1787
Draft
aleksandar-apostolov wants to merge 5 commits into
Draft
[DRAFT] Runtime controls for microphone audio processing and bitrate#1787aleksandar-apostolov wants to merge 5 commits into
aleksandar-apostolov wants to merge 5 commits into
Conversation
The platform noise suppressor is decided when the audio device module is built, from the audio bitrate profile alone, and MUSIC_HIGH_QUALITY is the only thing that turns it off — a profile that cannot be selected once the call is joined. A broadcaster who starts playing music mid-broadcast is suppressed to near silence with no way out. The effect is attached to the live recording session, so unlike the builder flag it can be changed while capture runs. Expose that on MicrophoneManager as its own control, independent of the profile: the software audio processing, the noise-cancellation processor and the bitrate are untouched. A recording session drops the platform effects and rebuilds them from the builder flags, so the request is remembered and re-applied whenever capture starts. Without that it would silently revert on the next reconnect.
Echo cancellation, noise suppression, automatic gain control and the high-pass filter are all tuned for speech — automatic gain control audibly pumps sustained music. They are fixed when the audio source is created, from the audio bitrate profile, so today the only way to change them is to pick MUSIC_HIGH_QUALITY before joining. Expose them on MicrophoneManager as their own control. Applying a change builds a fresh audio source and track and moves the live sender onto it, which costs a brief gap in captured audio but needs no renegotiation. The swap is the delicate part. RtpSender.setTrack disposes the track it currently holds only when it owns it, and MediaManagerImpl already owns and disposes the audio track, so the sender is handed the new track with ownership left behind — disposal stays in exactly one place. The rebuild runs under the media lock and rolls the new pair back if no sender accepted it, so a failed swap never tears down the source that is still live.
Adds the two runtime controls to the in-call debug submenu so they can be exercised on a device: each item shows the current state, highlights when the stage is on, and flips it. Both setters report whether the platform actually applied the change, and the menu surfaces a refusal as a toast. Without it a toggle that did nothing — unsupported device, or no capture running — looks identical to one that worked, which is the failure the controls are most likely to hit in the field.
The audio bitrate was fixed when the transceiver was created, from the audio bitrate profile, so it could only be chosen before joining. It rides on the sender's encoding rather than the SDP, so it can be changed on a running call through RtpParameters — the same way the video layers and the degradation preference already are. No renegotiation, no track swap, no gap. Verifying that needed stats that did not exist. Every group CallStats consumed was video; audio was dropped on the floor even though the report carries it. Adds audio send and receive bitrate measured from the RTP byte counters, the encoder's target bitrate, and the negotiated audio codec with its fmtp line. The two existing bitrate rows are renamed to say what they are. They report availableOutgoingBitrate from the candidate pair — a bandwidth estimate, not a transmitted rate — and they sit at zero whenever the selected pair omits it, which reads as a broken stat rather than an absent one. Measured on a Pixel 7: outbound-rtp targetBitrate moves 64000 -> 128000 with the negotiated fmtp untouched, so nothing clamps the request.
Contributor
PR checklist ❌The following issues were detected:
What we check
|
Contributor
SDK Size Comparison 📏
|
…al. Mode Normal is used for OEM devices
aleksandar-apostolov
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Expose each stage as its own runtime control.
Implementation
Three independent controls on
MicrophoneManagersetHardwareNoiseSuppressorEnabled/hardwareNoiseSuppressorEnabledsetSoftwareAudioProcessingEnabled/softwareAudioProcessingEnabled— thegoog*constraintssetAudioMaxBitrate/audioMaxBitrateBps— applied through the sender'sRtpParametersUsing it
Switching a live broadcast to music and back:
Each has a
StateFlowfor rendering a toggle —hardwareNoiseSuppressorEnabled,softwareAudioProcessingEnabled,audioMaxBitrateBps. They report the requested state; the setter's return value tells you whether the platform took it.Testing
Check and verify that each toggle affects the audio quality especially when music is playing.