fix(chat_ctx): preserve conversation order in merge() for realtime mo…#4927
Draft
RishbhaJain wants to merge 1 commit intolivekit:mainfrom
Draft
fix(chat_ctx): preserve conversation order in merge() for realtime mo…#4927RishbhaJain wants to merge 1 commit intolivekit:mainfrom
RishbhaJain wants to merge 1 commit intolivekit:mainfrom
Conversation
…dels ChatContext.merge() was using find_insertion_index() to re-sort items by created_at when merging a completed AgentTask's context back into the TaskGroup. This broke conversation ordering for realtime models (e.g. Gemini native audio) where the assistant message's created_at is set to started_speaking_at -- which can be earlier than the user message's created_at (transcript finalisation time) even though the assistant is responding to that user message. Fix: replace find_insertion_index() + insert() with append() in merge(), preserving the natural insertion order from other_chat_ctx. This is safe because merge() is only called in one place (AgentTask.__await_impl__), where other_chat_ctx is always a superset of self and new items are already in correct conversation order from prior append() calls. Also add two regression tests: - test_merge_preserves_conversation_order_over_timestamps: directly replicates the Gemini timing scenario. - test_merge_deduplicates_existing_items: confirms deduplication still works correctly after the change.
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR fixes a conversation ordering bug in ChatContext.merge() that affected realtime models like Gemini native audio. The issue occurred because merge() was sorting items by created_at timestamp using find_insertion_index(), but for realtime models, assistant messages use started_speaking_at as their timestamp, which can be earlier than the user message's transcript finalization time.
Changes:
- Replace sorted insertion with append() in
merge()to preserve natural conversation order - Update docstring to document the new ordering behavior
- Add two regression tests to verify ordering preservation and deduplication
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| livekit-agents/livekit/agents/llm/chat_context.py | Modified merge() to append items instead of inserting by timestamp, updated docstring |
| tests/test_chat_ctx.py | Added two new tests: one for conversation order preservation and one for deduplication validation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Author
|
Fixes #4792 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…dels
ChatContext.merge() was using find_insertion_index() to re-sort items by created_at when merging a completed AgentTask's context back into the TaskGroup. This broke conversation ordering for realtime models (e.g. Gemini native audio) where the assistant message's created_at is set to started_speaking_at -- which can be earlier than the user message's created_at (transcript finalisation time) even though the assistant is responding to that user message.
Fix: replace find_insertion_index() + insert() with append() in merge(), preserving the natural insertion order from other_chat_ctx. This is safe because merge() is only called in one place (AgentTask.await_impl), where other_chat_ctx is always a superset of self and new items are already in correct conversation order from prior append() calls.
Also add two regression tests: