`asyncio.shield()` leaks cancelled waiters

[deadlovelll]

Bug report

Bug description:

asyncio.shield() never releases a waiter that was cancelled, the cancelled task stays referenced by the shielded task forever. As a result, printing call graph raises an exception

Repro:

import asyncio

async def main():
    inner = asyncio.create_task(asyncio.sleep(3600), name="INNER")

    async def waiter():
        await asyncio.shield(inner)

    alive= asyncio.create_task(waiter(), name="ALIVE")
    for i in range(2):
        t = asyncio.create_task(waiter(), name=f"CANCELLED-{i}")
        await asyncio.sleep(0)
        t.cancel()
    await asyncio.sleep(0)

    asyncio.print_call_graph(inner)
    inner.cancel()

asyncio.run(main())

Expected:

* Task(name='INNER', id=0x35f290b0410)
  + Call stack:
  |   File '/Users/timofeiivankov/cpython/Lib/asyncio/tasks.py', line 705, in async sleep()
  + Awaited by:
    * Task(name='ALIVE', id=0x35f290b0610)
      + Call stack:
      |   File '/Users/timofeiivankov/cpython/repro.py', line 7, in async main.<locals>.waiter()

Actually:

Traceback (most recent call last):
  File "/Users/timofeiivankov/cpython/repro.py", line 19, in <module>
    asyncio.run(main())
    ~~~~~~~~~~~^^^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/runners.py", line 205, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/runners.py", line 128, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/timofeiivankov/cpython/repro.py", line 16, in main
    asyncio.print_call_graph(inner)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/graph.py", line 276, in print_call_graph
    print(format_call_graph(future, depth=depth, limit=limit), file=file)
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/graph.py", line 260, in format_call_graph
    render_level(graph, buf, 0)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/graph.py", line 252, in render_level
    render_level(fut, buf, level + 1)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/Users/timofeiivankov/cpython/Lib/asyncio/graph.py", line 217, in render_level
    if f.f_generator is None:
       ^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'f_generator'

Proposed fix: add cleanup in _outer_done_callback of asyncio.shield()

    def _outer_done_callback(outer):
        if not inner.done():
            inner.remove_done_callback(_inner_done_callback)
            if cur_task is not None:
                inner.remove_done_callback(_clear_awaited_by_callback)
                futures.future_discard_from_awaited_by(inner, cur_task)
            # Keep only one callback to log on cancel
            inner.remove_done_callback(_log_on_exception)
            inner.add_done_callback(_log_on_exception)

I have a fix ready

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-155854: Fix asyncio.shield() leaking cancelled waiters #155855

[deadlovelll]

So sorry, mine was a duplicate of #155144. I didn't notice this when I was working on the bug.