InvokeModelWithBidirectionalStreamAsync() can block the calling thread forever when the peer never sends any response - no timeout at any layer

[oleg-boulanov]

Summary

BedrockRuntimeClient::InvokeModelWithBidirectionalStreamAsync() is documented and named as an asynchronous method, but part of the initial event-stream write work runs synchronously on the calling thread, inside the call itself, with no timeout anywhere in the chain down to raw HTTP/2 flow control. If the server accepts the connection/stream but never sends any response (and therefore never sends a WINDOW_UPDATE frame either), the calling thread can hang forever inside the "async" call, not merely leave a callback unfired.

Environment

• aws-sdk-cpp version: 1.11.890 (built with the CRT HTTP client, AWS_SDK_USE_CRT_HTTP / push-based WriteData path)
• Service: Bedrock Runtime, InvokeModelWithBidirectionalStreamAsync (used for Nova Sonic real-time voice streaming)
• Platform: Windows, MSVC, x86/Win32
• Underlying CRT: aws-crt-cpp (commit pinned by aws-sdk-cpp 1.11.890) / aws-c-http (commit pinned by that aws-crt-cpp)

Repro

1. Stand up a local HTTP/2 (TLS, self-signed cert is fine) server that:
   • Accepts the TCP/TLS/HTTP2 connection and the incoming stream normally.
   • Never sends any WINDOW_UPDATE frame beyond the connection/stream defaults, and never sends any response (no headers, no data).
2. Call BedrockRuntimeClient::InvokeModelWithBidirectionalStreamAsync() against it (dummy credentials are fine - the server never needs to validate auth), with a streamReadyHandler that sends a normal small sequence of bidirectional-streaming events (e.g. a Nova Sonic sessionStart/promptStart/contentStart/textInput/contentEnd/promptEnd/sessionEnd sequence) and then calls Close() on the input stream.
3. Place a diagnostic statement immediately after the call to InvokeModelWithBidirectionalStreamAsync() in your own code.

Expected: the call returns promptly (it's documented/named as async), and only the response-received/outcome callback is left waiting - ideally itself bounded by some timeout, or at least clearly the caller's own responsibility to bound since it's callback-based.

Actual: the diagnostic statement never executes. The calling thread is parked indefinitely (confirmed via process inspection: near-zero CPU, threads alive, no progress even after 9+ minutes in one run) inside the call itself.

Root cause (traced through the actual 1.11.890 source)

1. generated/src/aws-cpp-sdk-bedrock-runtime/source/BedrockRuntimeClient.cpp (~L401-427), the CRT/push-based branch of InvokeModelWithBidirectionalStreamAsync():

   auto asyncTask = smithy::client::CreateSmithyBidirectionalWriteDataTask<InvokeModelWithBidirectionalStreamOutcome>(
       this, requestCopy, handler, handlerContext, eventEncoderStream, writeDataStreamBuf, std::move(endpointCallback),
       std::move(authCallback));
   auto sem = asyncTask.GetSemaphore();
   m_clientConfiguration.executor->Submit(std::move(asyncTask));
   sem->WaitOne();
   streamReadyHandler(*eventEncoderStream);

   sem->WaitOne() is unconditional (no timeout). Once it returns (i.e. once the background task's HttpWriteDataStreamBuf::Initialize() succeeds), streamReadyHandler - the caller-supplied write callback - is invoked directly on the calling thread, as the last statement of this method. There is nothing else in the function body after this call; the method returns only once streamReadyHandler returns.

2. Inside streamReadyHandler, calling Close() on the event-stream input (Model::InvokeModelWithBidirectionalStreamInput → inherited Aws::Utils::Event::EventEncoderStream::Close()) is a pure passthrough down to HttpWriteDataStreamBuf::Close() → SendBuffer(endStream=true). The same SendBuffer() is used for every write (overflow(), xsputn(), sync(), and Close()).

3. src/aws-cpp-sdk-core/source/utils/stream/HttpWriteDataStreamBuf.cpp (~L148-181), SendBuffer():

   m_stream->WriteData(data, [this](int errorCode) -> void {
       std::unique_lock<std::mutex> const lock{m_writeMutex};
       m_writeInProgress = false;
       m_writeError = (errorCode != AWS_ERROR_SUCCESS);
       m_writeComplete.notify_one();
   }, endStream);
   
   std::unique_lock<std::mutex> lock{m_writeMutex};
   m_writeComplete.wait(lock, [this]() -> bool { return !m_writeInProgress; });

   This wait has no timeout parameter and no timed overload anywhere in the class.

4. The completion callback only fires once aws-c-http's aws_h2_stream_encode_data_frame() (source/h2_stream.c, ~L829-846) actually encodes the write into an outgoing DATA frame:

   if (stream->thread_data.window_size_peer <= AWS_H2_MIN_WINDOW_SIZE) {
       /* The stream is stalled now */
       *data_encode_status = AWS_H2_DATA_ENCODE_ONGOING_WINDOW_STALLED;
       return AWS_OP_SUCCESS;
   }

   If the stream is window-stalled, this returns having done nothing - no error, no retry scheduling, no deadline. The queued write just sits there.

5. window_size_peer is this side's outbound HTTP/2 flow-control credit as granted by the peer - seeded from the RFC 7540 §6.5.2 default (65535 bytes) at connection/stream setup (source/h2_connection.c ~L387-388, source/h2_stream.c ~L741-745), and only ever replenished by a WINDOW_UPDATE frame from the peer. A peer that never sends anything never sends one either. Once cumulative outstanding bytes across the whole stream exceed that window, every subsequent write - including, in this repro, the one inside Close() - stalls forever.

6. The one native mechanism that exists specifically for "peer never responds" is response_first_byte_timeout_ms (aws-c-http's aws_http_connection_manager_options / aws_http_make_request_options), but per its own doc comment it is explicitly HTTP/1.1-only:

   /**
    * ...
    * TODO: Only supported in HTTP/1.1 now, support it in HTTP/2
    */
   uint64_t response_first_byte_timeout_ms;

   confirmed by grepping the whole aws-c-http tree: it's referenced only in h1_connection.c/h1_stream.c, never in h2_connection.c/h2_stream.c. It is also not exposed anywhere through aws-crt-cpp's or aws-sdk-cpp's C++ wrapper types, even for HTTP/1.1.

7. Separately, ClientConfiguration::requestTimeoutMs exists and is checked in the SDK's synchronous MakeRequest() response wait (src/aws-cpp-sdk-core/source/http/crt/CRTHttpClient.cpp ~L537-559), but is never referenced anywhere in the connection-acquisition (AcquireConnection, ~L721-745) or bidirectional-streaming write path - so it provides no protection here even when configured.

Impact

For a caller of InvokeModelWithBidirectionalStreamAsync(), there is currently no supported way, at any layer, to bound how long the call itself (not just the eventual response callback) can block against a peer that accepts the stream but never communicates further. This is worse than "the outcome callback never fires" - the calling application's own thread is the one that hangs, which for an application built expecting an async, callback-driven API (e.g. a call-handling engine dispatching this from a request-processing thread) can freeze application logic entirely, not just leave a background task dangling.

Suggested areas to address (not prescribing the fix, just where the gaps are)

• Expose response_first_byte_timeout_ms (or an HTTP/2-appropriate equivalent) through aws-crt-cpp and aws-sdk-cpp's ClientConfiguration, and implement it for HTTP/2 in aws-c-http (currently HTTP/1.1-only by an existing TODO).
• At minimum, give HttpWriteDataStreamBuf::SendBuffer()'s write-completion wait a timeout path, so a stalled write surfaces as an error to the caller instead of blocking forever.
• Consider whether streamReadyHandler needs to run synchronously on the calling thread inside InvokeModelWithBidirectionalStreamAsync() at all - if the intent is a genuinely async method, dispatching this initial write burst through the executor (like the rest of the task) rather than back on the caller's thread would preserve the documented async contract even without a new timeout feature.

Related, but distinct issues (searched, not duplicates)

• Possible regression introduced in 1.11.801 (2nd attempt) #3917 / Possible regression introduced in 1.11.801 #3911 - Aws::ShutdownAPI() hang after a stream has already completed (self-reference cycle pinning a connection) - already fixed in PR fix/bidirectional stream shutdown #3919 (1.11.890). This report is a different bug: the hang happens during the initial call, before any response, triggered by HTTP/2 flow control against a non-responsive peer, not by a resource cycle at shutdown.
• Bedrock Runtime Nova Sonic bidirectional streaming on EC2 instance - no output event stream #3649, Stream error is not returned #3650 - connectivity/error-visibility issues, not hangs of this kind.