Repository navigation
ResourceWarning in test_write_timeout due to Unclosed Async Generator #3597
Description
Hi!
Problem:
The test_write_timeout is failing with the following ResourceWarning:
ResourceWarning: Async generator 'httpx._content.ByteStream.__aiter__' was garbage collected before it had been exhausted. Surround its use in 'async with aclosing(...):' to ensure that it gets cleaned up as soon as you're done using it.
Analysis:
This warning occurs within httpcore/_async/http11.py -> AsyncHTTP11Connection.handle_async_request. The current implementation uses a try...except WriteError block around the request sending logic. If a WriteError is raised inside _send_request_body, the exception is caught and suppressed. However, the function proceeds without properly closing the request body's async iterator, leading to it being garbage collected later and triggering the ResourceWarning.
Proposed Solution:
I recommend addressing this directly in httpcore by ensuring proper cleanup of the request body's async iterator. This can be achieved by wrapping it in an async with aclosing(...) block. This guarantees that the stream's aclose() method is called upon exiting the block, whether due to successful completion or an exception. This will directly resolve the ResourceWarning and allow the correct WriteTimeout exception to surface.
I have working code that implements this fix and can share it. (Please note that I was unable to push a PR directly to httpcore)
Temporary Workaround (if immediate fix in httpcore is not feasible):
To suppress the warning in your local environment, you can add the following in pyproject.toml under filterwarnings:
"ignore::pytest.PytestUnraisableExceptionWarning"
Activity
Close request body async iterator on write errors
jacksonsr451 commentedon Oct 11, 2026
Hi! I'd like to work on this issue.
I'm interested in investigating the unclosed async generator warning in
test_write_timeoutand identifying a fix that ensures proper resource cleanup without affecting the existing timeout behavior.Would it be okay for me to take this on and submit a PR with the appropriate tests?
Thanks!
NurielWainstein commentedon Oct 11, 2026
Hi, yes no problem!
Thanks!
jacksonsr451 commentedon Oct 11, 2026
Hi! Thanks for allowing me to investigate this issue.
I've been able to deterministically reproduce the underlying async generator cleanup problem under both asyncio and Trio.
The root cause is in HTTPCore's HTTP/1.1 request-body handling, where an interrupted write can leave the request-body iterator unclosed.
I also found that the main fix is already being addressed in [encode/httpcore#1119](encode/httpcore#1119).
During my investigation, I identified an additional cancellation-safety issue: under Trio, cancellation can interrupt
aclose()before the generator finishes its cleanup. I've implemented and validated a complementary fix usingAsyncShieldCancellation, with regression tests passing under both asyncio and Trio.I'll coordinate this improvement with the existing HTTPCore PR to avoid duplicating work.
Thanks!
jacksonsr451 commentedon Oct 11, 2026
I've investigated this issue alongside the ongoing work in encode/httpcore#1119.
I identified a cancellation-safety gap during async request-body iterator cleanup and prepared a complementary fix using
AsyncShieldCancellation, with regression coverage for both asyncio and Trio.The implementation is available here:
jacksonsr451/httpcore@816058e
I've shared the contribution with the author of encode/httpcore#1119 and am currently waiting for feedback on how best to integrate it.
The regression test fails against the unshielded PR head and passes with the proposed fix.
For now, the next step is to wait for review and integration of the upstream changes.