Parallel tool child cancellation leaves sibling tasks running past invocation completion

[jaywang172]

Problem

When one tool in a parallel tool batch undergoes child-originated cancellation, the batch can return without cancelling and awaiting its sibling tool calls. Here, child-originated cancellation means either the tool raises asyncio.CancelledError, or it cancels its own task with asyncio.current_task().cancel() and reaches an await. The caller does not cancel the batch.

With the public Runner, the invocation iterator finishes while a sibling tool is still executing. That sibling can subsequently perform a side effect. Callback ordering is an observable symptom; the core issue is that the parallel batch returns while a child task it spawned remains unsettled.

Minimal reproduction

1. Use a development checkout of google/adk-python at f33d4923388a963d0c5cbf8f7855a88d255b0dea (also the latest main when checked on 2026-09-18), with its Python dependencies installed.
2. Save the following as /tmp/adk-child-cancel.py.
3. From the repository root, run PYTHONPATH=src:. python /tmp/adk-child-cancel.py using that environment.

This uses the repository's MockModel to supply one parallel function-call response. It does not use LiteLLM, a model API, or a network service.

import asyncio

from google.adk.agents import LlmAgent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from tests.unittests.testing_utils import MockModel


async def main():
    trace = []
    started, release = asyncio.Event(), asyncio.Event()
    sibling_task = None

    async def stopper() -> dict:
        """Cancel this tool after its sibling starts."""
        await started.wait()
        trace.append("stopper:self_cancel")
        asyncio.current_task().cancel()
        await asyncio.sleep(0)  # Deliver this task's own cancellation.
        return {}

    async def sibling() -> dict:
        """Represent a delayed side effect."""
        nonlocal sibling_task
        sibling_task = asyncio.current_task()
        trace.append("sibling:start")
        started.set()
        try:
            await release.wait()
            trace.append("sibling:side_effect")  # Local append only.
            return {"ok": True}
        finally:
            trace.append("sibling:finally")

    async def after_agent(callback_context):
        trace.append("after_agent")

    async def after_tool(tool, args, tool_context, tool_response):
        trace.append("after_tool:" + tool.name)

    model = MockModel.create(responses=[[
        types.Part(function_call=types.FunctionCall(name=n, id=n, args={}))
        for n in ("stopper", "sibling")
    ]])
    agent = LlmAgent(
        name="probe", model=model, tools=[stopper, sibling],
        after_agent_callback=after_agent, after_tool_callback=after_tool,
    )
    service = InMemorySessionService()
    runner = Runner(app_name="scout", agent=agent, session_service=service)
    session = await service.create_session(app_name="scout", user_id="u")
    try:
        async for _ in runner.run_async(
            user_id="u", session_id=session.id,
            new_message=types.Content(
                role="user", parts=[types.Part(text="Run both tools.")]
            ),
        ):
            pass
        trace.append("runner_returned:complete")
        pending = sibling_task is not None and not sibling_task.done()
    finally:
        # Release/drain any surviving sibling so the repro itself leaks no task.
        release.set()
        if sibling_task is not None:
            await asyncio.gather(sibling_task, return_exceptions=True)
        await runner.close()

    print("\n".join(trace))
    print(f"sibling_pending_at_runner_return={pending}")


asyncio.run(main())

Observed output

sibling:start
stopper:self_cancel
after_agent
runner_returned:complete
sibling:side_effect
sibling:finally
after_tool:sibling
sibling_pending_at_runner_return=True

The sibling is gated with asyncio.Event, so this does not depend on timing sleeps. The only sleep(0) delivers the tool task's own cancellation. The local sibling:side_effect trace append stands in for an external write; the reproducer performs no external side effect and drains the surviving task before exiting.

Expected behavior

A parallel tool batch should not return while sibling tasks spawned by that batch are still running. If one child terminates the batch through cancellation, unfinished siblings should be cancelled and awaited before the original cancellation is propagated out of the batch.

The invariant is that sibling cleanup completes before the batch exits. This does not require converting cancellation into a normal tool error, changing callbacks to run from finally blocks, or changing how Runner exposes root-node cancellation.

Reproduction matrix

Condition	Async: sibling pending at batch return	Live: sibling pending at batch return
Normal completion	no	no
Tool raises ValueError	no	no
Caller cancels the entire batch	no	no
Tool raises CancelledError	yes	yes
Tool cancels its own task	yes	yes

Separate full Runner reproductions cover run_async with StreamingMode.NONE, run_async with SSE, and run_live with BIDI. All three exhibit the same child-cancellation behavior. Live uses the repository's MockModel, not a live model service.

Root cause and candidate repair

_gather_or_cancel catches Exception, cancels unfinished child tasks, awaits their cleanup, and re-raises. asyncio.CancelledError derives from BaseException, so child-originated cancellation bypasses that path. asyncio.gather() propagating a child's cancellation does not itself cancel its siblings.

A minimal candidate repair is to include asyncio.CancelledError in the existing cleanup path, for example with except (Exception, asyncio.CancelledError), then reuse the existing cancel/await/re-raise logic. I tested this as a process-local monkeypatch only; production source was not modified. Cancellation remains cancellation, rather than becoming a tool error response.

Verification and environment

Baseline: f33d4923 (2026-09-17), macOS arm64, Python 3.11.9.

• Existing BaseAgent, async tool callback, and live tool callback suites: 111 passed.
• Local scout matrix on unmodified main: 21 passed, 10 failed; all failures concern surviving siblings after child-originated cancellation.
• Same matrix with the process-local candidate repair: 31 passed.
• Stability: 40/40 deterministic reproductions (10 rounds × 2 tool execution modes × 2 child-cancellation forms).
• Dependencies: google-genai 2.22.0, pydantic 2.12.5, pytest 9.1.1, pytest-asyncio 1.4.0.

The existing local environment was reused. Full supported-Python-version tests and real-backend live E2E have not been run. I have not established which release first introduced this behavior. I can provide the broader Runner probe and regression matrix if useful, and am happy to send a focused PR if this direction fits.

Duplicate check

I searched current issue/PR listings and cancellation-related terms and did not find the same case. Nearby reports differ:

• Parallel calls to the same streaming tool leave an orphan task after live teardown #7058 / fix: track concurrent streaming tool calls #7059: concurrent streaming tools with the same name overwrite task tracking. This repro uses two distinct ordinary async tools.
• "Failed to detach context" (Token created in a different Context) logged as ERROR when caller stops runner.run_async() early #6559 / fix: avoid OTel "Failed to detach context" error on early run_async close #6560: OpenTelemetry context errors during early iterator close. This repro fully consumes the Runner iterator.
• "Stop generating" — ability to stop run_async() from outside the agent #4796: public stop-generating support. This report concerns cleanup of sibling tasks already spawned by a parallel tool batch.