Skip to content

fix(openai): warn when semantic_vad is configured with Azure Realtime #6871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CrimsonSithria wants to merge 1 commit into
base: main
Choose a base branch
from
+12 −2
Open

fix(openai): warn when semantic_vad is configured with Azure Realtime #6871

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/realtime_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -585,7 +585,7 @@ def with_azure(
modalities (list[Literal["text", "audio"]] | NotGiven): Modalities to enable. Defaults to ["text", "audio"] if not provided.
input_audio_transcription (AudioTranscription | InputAudioTranscription | None | NotGiven): Transcription options; defaults to Azure-optimized values when not provided.
input_audio_noise_reduction (NoiseReductionType | InputAudioNoiseReduction | None): Input noise reduction settings. Defaults to None.
turn_detection (RealtimeAudioInputTurnDetection | TurnDetection | None | NotGiven): Server-side VAD; defaults to Azure-optimized values when not provided.
turn_detection (RealtimeAudioInputTurnDetection | TurnDetection | None | NotGiven): Server-side VAD; defaults to Azure-optimized values (server_vad) when not provided. Note: Azure has been observed to accept semantic_vad but never emit input_audio_buffer.speech_started, breaking server-side interruption/barge-in.
speed (float | NotGiven): Audio playback speed multiplier.
tracing (Tracing | None | NotGiven): Tracing configuration for OpenAI Realtime.
reasoning (RealtimeReasoning | None | NotGiven): Reasoning config for reasoning-capable models, e.g. ``RealtimeReasoning(effort="low")``.
Expand Down Expand Up @@ -690,6 +690,13 @@ def with_azure(
can_disable_turn_detection = not is_given(turn_detection)
if not is_given(turn_detection):
turn_detection = AZURE_DEFAULT_TURN_DETECTION
elif turn_detection is not None and getattr(turn_detection, "type", None) == "semantic_vad":

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Warn on every supported Azure configuration path

When callers use the supported Azure constructor directly (RealtimeModel(azure_deployment=..., turn_detection=SemanticVad(...))) or later select semantic VAD through update_options, this branch never executes because it exists only in with_azure(). Those configurations have the same broken Azure barge-in behavior but remain silent; move or share the Azure-aware check with the constructor and option-update paths.

Useful? React with 👍 / 👎.

logger.warning(
"semantic_vad on Azure OpenAI Realtime has been observed to accept the "
"session config but never emit input_audio_buffer.speech_started, so "
"server-side interruption/barge-in will not fire. Consider server_vad "
"(the Azure default here) until this is resolved on the Azure side."
)

model = RealtimeModel(
voice=voice,
Expand Down
5 changes: 4 additions & 1 deletion livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@
model="gpt-4o-mini-transcribe",
)

# use beta version TurnDetection and InputAudioTranscription for compatibility
# use beta version TurnDetection and InputAudioTranscription for compatibility.
# server_vad (not semantic_vad) is intentional: Azure OpenAI Realtime has been
# observed to accept a semantic_vad session config but never emit
# input_audio_buffer.speech_started, which breaks server-side interruption/barge-in.
AZURE_DEFAULT_TURN_DETECTION = TurnDetection(
type="server_vad",
threshold=0.5,
Expand Down