[bug] iOS: No remote audio output when using Speech-to-Text with LiveKit (AVAudioSession conflict with playAndRecord + measurement mode)

[shivanshu877]

🚨 iOS: No Remote Audio Output When Using Speech-to-Text with LiveKit

(AVAudioSession conflict with playAndRecord + measurement mode)

---

📋 Description

I am building a Flutter chat application using:

• 🎥 LiveKit for real-time audio/video
• 🎙️ A WebSocket-based Speech-to-Text (STT) service

The goal is to run both simultaneously:

• Bi-directional audio via LiveKit
• Microphone capture for transcription
• Send transcription result to lk.chat topic
• Hear remote participants while STT runs

---

🧩 Use Case Flow

1. User joins a LiveKit room (audio/video enabled).
2. User speaks in their native language.
3. Microphone audio is captured.
4. Audio is sent to a managed STT service via WebSocket.
5. Transcription result is sent to lk.chat.
6. User should simultaneously hear remote participant audio.

---

❗ Problem

When I initialize Speech-to-Text during an active LiveKit session:

• ✅ Transcription works correctly
• ❌ Remote audio stops playing
• ❌ No sound from speaker or connected output device
• ❌ Microphone continues to work

If I remove STT initialization:

• ✅ LiveKit audio works perfectly

This strongly suggests an AVAudioSession configuration conflict.

---

🎛 Current Audio Session Configuration

I am manually configuring the audio session as follows:

Future configureAudioSession() async {
  final session = await AudioSession.instance;

  await session.configure(
    AudioSessionConfiguration(
      avAudioSessionCategory: AVAudioSessionCategory.playAndRecord,
      avAudioSessionCategoryOptions:
          AVAudioSessionCategoryOptions.allowBluetooth |
          AVAudioSessionCategoryOptions.defaultToSpeaker,
      avAudioSessionMode: AVAudioSessionMode.measurement,
      avAudioSessionRouteSharingPolicy:
          AVAudioSessionRouteSharingPolicy.defaultPolicy,
      androidAudioAttributes: const AndroidAudioAttributes(
        contentType: AndroidAudioContentType.speech,
        usage: AndroidAudioUsage.voiceCommunication,
      ),
      androidAudioFocusGainType: AndroidAudioFocusGainType.gain,
      androidWillPauseWhenDucked: false,
    ),
  );
}

---

🧪 What I Tried

• Calling configureAudioSession() before initializing STT
• Calling it after initializing STT
• Changing AVAudioSessionMode
• Re-configuring session after STT starts
• Multiple ordering combinations

None of these resolved the issue.

---

🔍 Observations

• STT works fine independently
• LiveKit works fine independently
• Issue only occurs when both are active
• STT likely reconfigures AVAudioSession internally
• Possible conflict between:
  • AVAudioSessionMode.measurement
  • voiceCommunication
  • LiveKit’s internal audio handling

---

❓ Questions

1. Does LiveKit internally override AVAudioSession after room connection?
2. Is there a recommended approach to use:
   • LiveKit (bi-directional audio)
   • Simultaneous mic capture for STT
3. Should AVAudioSessionMode.measurement be avoided in this case?
4. Is there a known working configuration for:
   • playAndRecord
   • Speaker output
   • Bluetooth support
   • LiveKit
   • Parallel microphone capture for STT?

---

🧾 Environment

• Flutter version: [add version]
• livekit_client version: [add version]
• iOS version: [add version]
• Device: [add device model]

---

🙏 Additional Context

If there is any recommended pattern, sample implementation, or best practice for running LiveKit audio + Speech-to-Text simultaneously on iOS, that would be greatly appreciated.