Skip to content

fix(tools): cancel sibling tools on child-originated cancellation - #7180

Open
Anusha0501 wants to merge 2 commits into
google:mainfrom
Anusha0501:fix/parallel-tool-cancel-siblings
Open

Anusha0501 wants to merge 2 commits into
google:mainfrom
Anusha0501:fix/parallel-tool-cancel-siblings

Conversation

@Anusha0501

@Anusha0501 Anusha0501 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:
When one tool in a parallel batch is cancelled (it raises asyncio.CancelledError, or it cancels its own task), _gather_or_cancel does not tear down the rest of the batch. asyncio.CancelledError is a BaseException, so the existing except Exception path never runs. asyncio.gather() also does not cancel siblings when a child is cancelled. The public Runner iterator can therefore finish while a sibling tool is still executing and can still perform a side effect.

Solution:
Catch (Exception, asyncio.CancelledError) in _gather_or_cancel and reuse the existing cancel-unfinished-tasks / await-cleanup / re-raise path. Cancellation stays cancellation; it is not converted into a tool error.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
PYTHONPATH=src pytest tests/unittests/flows/llm_flows/tools/test_batch_executor.py tests/unittests/flows/llm_flows/tools/test_functions_parallel_error.py

19 passed

Added:

  • test_gather_or_cancel_cancels_siblings_on_cancelled_error
  • test_gather_or_cancel_cancels_siblings_when_child_cancels_itself
  • test_parallel_function_call_cancels_siblings_on_cancelled_error (parametrized: run_async, SSE, run_live)
  • test_parallel_function_call_cancels_siblings_when_tool_cancels_itself (parametrized: run_async, SSE, run_live)

The Runner-level tests use the reporter's Event handshake from #7172 and assert the sibling is cancelled and not still pending when the iterator returns. They now cover non-streaming run_async, run_async with SSE, and run_live BIDI, matching the #7172 matrix.

Manual End-to-End (E2E) Tests:

Covered by the Runner-level unit tests above (MockModel, no live model API). Those reproduce the #7172 matrix for CancelledError and self-cancel across run_async, SSE, and run_live.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Reporter's diagnosis and candidate repair from #7172 (@jaywang172): _gather_or_cancel catches Exception only; CancelledError bypasses sibling cleanup. Caller-cancel of the whole batch already works because gather cancels children when the gather wait itself is cancelled. This PR only covers child-originated cancellation.

asyncio.CancelledError is a BaseException, so _gather_or_cancel only
tore down siblings on Exception. A parallel tool that raised
CancelledError or cancelled itself left sibling tasks running after
the batch returned.

Fixes google#7172
@jaywang172

Copy link
Copy Markdown
Contributor

Thanks for putting together the fix and for crediting the reporter's Event handshake in the testing section.

One attribution correction: the “Maintainer diagnosis on #7172” paragraph refers to my investigation as the issue reporter, rather than a maintainer response. The root-cause analysis, deterministic reproducer, and candidate (Exception, asyncio.CancelledError) repair were included in my original report. Could you change that wording to “Reporter's diagnosis and candidate repair from #7172 (@jaywang172)”?

My local probes also cover run_async with SSE and run_live with BIDI using MockModel, in addition to non-streaming run_async. Both child-cancellation forms reproduce across those paths. I'd be happy to contribute focused SSE/live regression coverage here so we can preserve the batch-ownership invariant across execution modes without opening a duplicate fix PR. Let me know if you'd like to coordinate that addition.

google#7172 reproduced child-originated cancellation on run_async SSE and
run_live BIDI as well as non-streaming run_async. Parametrize the
Runner tests across those three modes so the batch-ownership invariant
holds on each path.
@Anusha0501

Copy link
Copy Markdown
Contributor Author

@jaywang172 Thanks for the correction, and for the original diagnosis, reproducer, and candidate repair on #7172.

The additional-context wording is now Reporter's diagnosis and candidate repair from #7172 (@jaywang172).

I also added the SSE and run_live BIDI coverage you described, using the same Event handshake. The two Runner tests are parametrized across non-streaming run_async, run_async with SSE, and run_live, for both child-cancellation forms. Locally: 19 passed on test_batch_executor.py and test_functions_parallel_error.py.

That should keep the batch-ownership invariant on those paths in this PR, so a follow-up fix PR is not needed. Further cases you still want to add are welcome as review comments or a follow-up on this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel tool child cancellation leaves sibling tasks running past invocation completion

2 participants