Streaming tool yields in live mode cause model to re-invoke the tool in a loop

[kazunori279]

Description

When using an async generator streaming tool in live (bidi) mode, each yield from the tool causes the model to be interrupted and re-invoke the same tool, creating an infinite loop of duplicate tool calls.

Root Cause

In _process_function_live_helper (functions.py), streaming tool results are sent back to the model via live_request_queue.send_content():

async for result in agen:
    updated_content = types.Content(
        role='user',
        parts=[
            types.Part.from_text(
                text=f'Function {tool.name} returned: {result}'
            )
        ],
    )
    invocation_context.live_request_queue.send_content(updated_content)

In GeminiLlmConnection.send_content(), this is forwarded as:

await self._gemini_session.send_client_content(turns=[content], turn_complete=True)

The combination of role='user' and turn_complete=True makes the model treat each streaming update as a new complete user turn. The model:

1. Gets interrupted (stops current audio/text generation)
2. Receives what looks like a new user message: "Function run_my_tool returned: {'status': 'running', ...}"
3. Responds to it — often by calling the same tool again
4. The tool is already running, so it either starts a duplicate or returns "busy"
5. Repeat

This produces a rapid interrupted → turn_complete → tool_call → interrupted → ... loop visible in the logs, with the model never completing a coherent response.

Steps to Reproduce

1. Define an async generator tool that yields progress updates:

async def long_running_task(query: str):
    """A tool that takes time and yields progress updates."""
    yield {"status": "running", "message": "Step 1: fetching data..."}
    await asyncio.sleep(5)
    yield {"status": "running", "message": "Step 2: processing..."}
    await asyncio.sleep(5)
    yield {"status": "completed", "result": "Done!"}

2. Create an agent with this tool and run in live mode with a native-audio model
3. Send a message that triggers the tool
4. Observe: the model gets interrupted by each yield, re-calls the tool repeatedly, and never delivers a coherent spoken response

Expected Behavior

Streaming tool updates should be delivered to the model in a way that does not interrupt the current generation or trigger re-invocation. The model should be able to narrate progress updates naturally between yields.

Possible Fixes

• Send streaming updates with turn_complete=False so the model treats them as partial/background context rather than a new turn requiring a response
• Use a different role (e.g., 'tool' or function_response) instead of 'user' so the model doesn't interpret yields as user messages
• Queue streaming updates and let the model consume them after its current generation completes, rather than interrupting it
• Provide a mechanism to send out-of-band status updates that bypass the model entirely (e.g., via a callback to the application layer)

Environment

• google-adk: 2.1.0
• Model: gemini-live-2.5-flash-native-audio (Vertex AI)
• Python: 3.12

[surajksharma07]

@kazunori279 A workaround worth trying: add a turn_complete: bool = True field to LiveRequest and LiveRequestQueue.send_content(), thread it through _send_to_model in base_llm_flow.py, update GeminiLlmConnection.send_content() to accept and use it, then in _process_function_live_helper call send_content(updated_content, turn_complete=False) for intermediate yields.

Could you test this against your setup and confirm it stops the re-invocation loop?

[kazunori279]

@surajksharma07 Tested the turn_complete workaround — confirmed it stops the re-invocation loop.

Threading turn_complete=False through LiveRequest → _send_to_model → GeminiLlmConnection.send_content() and using it for intermediate yields in _process_function_live_helper prevents the interrupted → turn_complete → tool_call → interrupted cycle described in this issue.

Patch required (4 files):

• live_request_queue.py — add turn_complete: bool = True field to LiveRequest, thread through send_content()
• base_llm_connection.py / gemini_llm_connection.py — add turn_complete param to send_content(), use it instead of hardcoded True
• base_llm_flow.py — pass live_request.turn_complete through to llm_connection.send_content()
• functions.py — intermediate yields use turn_complete=False

This is a bug in ADK — GeminiLlmConnection.send_content() hardcodes turn_complete=True, which makes every streaming tool yield look like a new complete user turn to the model. Could the ADK team incorporate this fix? It's a small, backwards-compatible change (default remains True).

[boyangsvl]

/adk-issue-analyze

[adk-bot]

GitHub Issue #5947 Analysis: Streaming tool yields in live mode cause model to re-invoke the tool in a loop

Detailed Analysis

1. Root Cause Analysis ("What is broken?")

When using async generator streaming tools in live (bidirectional streaming) mode, each yield from the tool's generator triggers an interruption in the current generation of the model and forces a re-invocation of the same streaming tool in an infinite loop.

The failure flow operates as follows:

• In _process_function_live_helper in functions.py, streaming tool updates are yielded and placed in live_request_queue as a types.Content object text string message. These have role 'user' without any custom parameters.
• In _send_to_model in base_llm_flow.py, elements from this queue are processed and sent to the LLM connection via await llm_connection.send_content(live_request.content).
• In GeminiLlmConnection.send_content() in gemini_llm_connection.py, they are sent via self._gemini_session.send() nested in types.LiveClientContent(...) with turn_complete=True.
• Because turn_complete is hardcoded to True for text inputs and the message has 'user' role, Gemini treats each progress update as a finished user turn, triggering an automatic interruption and prompting the model to evaluate the whole conversation history and generate a response (which usually triggers the tool again).

---

2. Legitimacy Analysis

• Status: Legitimate Bug
• Evidence:
  • The send_content method in gemini_llm_connection.py hardcodes turn_complete=True for any text inputs (other than Gemini 3.1 streaming text).
  • The streaming tool generator loop in functions.py sends intermediate generator updates via live_request_queue.send_content(...) as standard user-role content, which defaults to turn_complete=True. This results in client-side interruptions of model generations and rapid duplication loops.

---

3. Fix Recommendation

• Recommendation: Should Fix (High Priority)
• Rationale:
  • This is a critical blocker for real-time applications using async generator tools with progress/streaming updates in live channels.
  • Adding support for flowing a turn_complete flag from LiveRequestQueue down to types.LiveClientContent is a minor, low-risk, and elegant architectural improvement.
• Proposed Fix Steps:
  1. Add turn_complete: bool = True to the LiveRequest class in live_request_queue.py.
  2. Implement turn_complete parameter in LiveRequestQueue.send_content in live_request_queue.py.
  3. Change the streaming tool update generator loop in functions.py to execute live_request_queue.send_content(updated_content, turn_complete=False).
  4. Extend BaseLlmConnection.send_content in base_llm_connection.py and GeminiLlmConnection.send_content in gemini_llm_connection.py to accept turn_complete: bool = True.
  5. In gemini_llm_connection.py, use the passed turn_complete flag, and also skip the Gemini 3.1 real-time input optimization check if turn_complete is False.
  6. In _send_to_model in base_llm_flow.py, invoke llm_connection.send_content(live_request.content, turn_complete=live_request.turn_complete).
  7. Add/update tests in test_live_request_queue.py and test_gemini_llm_connection.py.

---

4. Existing Pull Requests

• Linked PR: None
• PR URL: None
• Analysis: We ran a query searching for pull requests mentioning 5947 and found no existing pull requests.

---

Executive Summary

1. What is broken? Standard text inputs sent to the live session connection default to turn_complete=True. When streaming tools yield updates, they are processed as text inputs. This makes the model treat intermediate streaming progress updates as finished user turns, interrupting active generations and re-invoking the tool in a loop.
2. Is the issue legitimate? Yes. Verified via code inspection in gemini_llm_connection.py and functions.py.
3. Should we fix it? Yes, High Priority. This completely breaks async generator streaming tools in real-time setups.
4. Is there a linked PR that fixes this issue? None.

---

Call to Action / Approval

Would you like me to create and implement a fix for this issue in the workspace? (Note: The changes and tests will be created in a new branch but NOT committed, so you can review and iterate on them.)

[surajksharma07]

@kazunori279 the turn_complete=False propagation cleanly breaks the interrupted → tool_call loop, and the 4-file surface is exactly the right scope.
The True default keeps it fully backwards-compatible.

One additional issue worth folding into the same : in _send_to_model (base_llm_flow.py lines 774–786), the session-persistence guard only skips function_response parts:

is_function_response = content.parts and any(
    part.function_response for part in content.parts
)
if not is_function_response:
    await invocation_context.session_service.append_event(...)

Streaming tool yields are role='user' text content — they pass that check, so every intermediate "Function X returned: {status: running, ...}" gets appended to session history as a user event.
On a long-running streaming tool this fills session history with noise and can interfere with session replay.
The cleanest fix is to also gate append_event on live_request.turn_complete — skip persistence for intermediate yields and only persist when turn_complete=True (the final yield or a genuine user message).
This threads naturally from the turn_complete field you're already adding to LiveRequest.

On backend coverage: the non-Gemini-3.1 send_client_content(turn_complete=True) path applies to both Vertex AI and Gemini API backends, so your patch covers both.
The Gemini 3.1 branch (send_realtime_input) is a separate code path and is unaffected.

For test coverage: test_live_request_queue.py (verify turn_complete threads through LiveRequest.send_content()), test_gemini_llm_connection.py (verify turn_complete=False reaches LiveClientContent and turn_complete=True stays unchanged), and a streaming-yield path test in test_functions.py confirming run_tool_and_update_queue sends intermediate yields with turn_complete=False and the final yield with turn_complete=True.

Team is looking more into it.

[kazunori279]

@surajksharma07 Thanks for the detailed analysis. A few thoughts:

The turn_complete=False approach looks right — minimal surface (4 files), backwards-compatible with the True default, and cleanly breaks the interrupt loop. Good scope.

The session history point is an excellent catch. Intermediate yields getting persisted as user events would accumulate fast for any long-running streaming tool, and could degrade session replay and inflate context on subsequent turns. Gating append_event on turn_complete is the natural fix given the field is already being added.

One thing worth considering beyond turn_complete=False: for pure progress/status updates, an out-of-band mechanism (e.g., a callback to the application layer) might be a better fit than sending them through the model at all. turn_complete=False stops the re-invocation loop, but the model still receives intermediate updates in its context — updates it doesn't need to reason about. A callback path would let tools report progress directly to the UI without adding noise to the model's context window. This could coexist with the turn_complete fix — use turn_complete=False for updates the model should eventually see, and a callback for purely presentational status.