shutdown CPU busy-loop in HTTP/SSE transports#2147
Open
akshan-main wants to merge 1 commit intomodelcontextprotocol:mainfrom
Open
shutdown CPU busy-loop in HTTP/SSE transports#2147akshan-main wants to merge 1 commit intomodelcontextprotocol:mainfrom
akshan-main wants to merge 1 commit intomodelcontextprotocol:mainfrom
Conversation
19fece9 to
1dbe910
Compare
Exiting the streamable_http_client or sse_client context with pending requests causes AnyIO to spin in _deliver_cancellation because transport tasks are stuck on zero-buffer stream send() calls that can't be cancelled cooperatively. Close streams before cancelling the task group so blocked operations raise ClosedResourceError and exit on their own. Guard all read_stream_writer.send() calls against ClosedResourceError and BrokenResourceError during teardown. Bump stream buffer from 0 to 1 to reduce blocking probability. Same fix applied to BaseSession.__aexit__ for consistency. Github-Issue: modelcontextprotocol#1805
1dbe910 to
29937a2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exiting the streamable_http_client or sse_client context with pending requests causes AnyIO to spin in _deliver_cancellation because transport tasks are stuck on zero-buffer stream send() calls that can't be cancelled cooperatively.
Same fix applied to BaseSession.aexit for consistency.
Fixes #1805
Motivation and Context
When a client exits with in-flight requests (e.g. tool calls to a slow server-side handler), the transport tasks are blocked inside zero-buffer memory stream
send()calls. AnyIO tries to cancel these tasks butsend()on a zero-buffer stream isn't cooperatively cancellable while blocked - this causes_deliver_cancellationto busy-loop, spiking CPU to ~90% after shutdown.The thread count growth also mentioned in #1805 is asyncio's
getaddrinfoexecutor behavior - threads are pooled and reused by the runtime. This PR only addresses the CPU busy-loop.How Has This Been Tested?
test_streamable_http_client_exit_with_pending_requests- fires tool calls to a handler that blocks indefinitely, exits with pending responses, verifies shutdown completes within timeouttest_streamable_http_client_rapid_connect_disconnect- 5 rapid connect/init/disconnect cycles, verifies no resource leaktest_session_exit_closes_streams_before_cancel- sends a ping to an unresponsive mock server, verifies BaseSession.aexit completes promptly without busy-loopingBreaking Changes
None. Stream buffer change from 0 to 1 is internal and does not affect public API.
Types of changes
Checklist
Additional context
Files changed:
src/mcp/client/streamable_http.py— buffer 0→1, close streams before cancel, guard all send() callssrc/mcp/client/sse.py— same pattern applied consistentlysrc/mcp/shared/session.py— BaseSession.aexit closes streams before cancelling task grouptests/shared/test_streamable_http.py— two regression teststests/shared/test_session.py— BaseSession shutdown test