fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880)#10884
Open
localai-bot wants to merge 1 commit into
Open
fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880)#10884localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…ng the widget (#10880) Establishing an MCP session held the session-cache mutex across client.Connect with no per-connect timeout. An unreachable remote server (bounded only by the 360s httpClient timeout) or a stdio server whose initialize handshake never completes therefore blocked the caller and, because the mutex was held, every other MCP request for that model too. In the UI this shows up as the MCP "Servers" widget spinning forever. It is most visible for cloud-proxy models: their chat path bails out before the MCP tool block, so it never warms the session cache in the background. The widget's /v1/mcp/servers/<model> call is then the first and only code that connects synchronously, in the request foreground. The session, once established, stays bound to the shared context (it is cancelled later via the cached cancel func on eviction/shutdown), so we can't pass a WithTimeout context to Connect: firing the timeout would tear a healthy session down, and cancelling the shared context would also kill sibling servers that already connected. Instead connectMCP runs Connect on the shared context in a goroutine and stops waiting after the discovery timeout, returning an error for that one server without disturbing the others. A stalled goroutine is reaped when the model's sessions are cancelled. Applied to both SessionsFromMCPConfig and NamedSessionsFromMCPConfig. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:opus-4.8 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
For a model that declares
mcp:servers, the chat UI's MCP Servers widget "spins forever" and the servers never become selectable. Reported in #10880 for acloud-proxymodel.Root cause
Building an MCP session (
SessionsFromMCPConfig/NamedSessionsFromMCPConfigincore/http/endpoints/mcp/tools.go) holds the session-cache mutex acrossclient.Connectwith no per-connect timeout:initializehandshake never completes has no bound at all.Either one blocks the request that the widget's
GET /v1/mcp/servers/<model>makes, and because the mutex is held, it also wedges every other MCP request for that model. The frontend keeps its spinner up until the request returns, so it appears to hang forever.Why cloud-proxy makes it visible: a cloud-proxy passthrough model returns early in the chat path (
chat.goIsCloudProxyBackendPassthrough()) before the MCP tool block, so its session cache is never warmed in the background. The widget'slistServerscall is then the first code that connects — synchronously, in the request foreground — so any slow/unreachable server shows up as a permanent spinner. (Normal backends warm the cache during chat, so by the time the widget opens,listServershits the fast alive-session path.)Fix
Add
connectMCP, which bounds eachclient.Connectwithconfig.DefaultMCPDiscoveryTimeout(60s — long enough for a first-run stdio image pull, finite instead of "forever").The subtlety: the
ctxpassed toConnectis the session's lifetime context (bound intojsonrpc2.NewConnection; cancelled later via the cached cancel func), and it is shared across all servers for a model. So we cannot pass aWithTimeoutcontext — firing it would tear a healthy session down — nor cancel the shared context on timeout, which would kill sibling servers that already connected. InsteadconnectMCPrunsConnecton the shared context in a goroutine and stops waiting after the timeout, returning an error for just that one server. A genuinely stalled goroutine holds only the shared context and is reaped when the model's sessions are evicted/cancelled. Applied to both session builders.Verification
go build,go vet(incl. thelostcancelanalyzer — clean), andgolangci-lint(new-from-merge-base): 0 issues.connect_timeout_test.go: a stdio server that starts but never speaks MCP (sleep) makesconnectMCPreturn a timeout error in ~the timeout instead of hanging. Fullcore/http/endpoints/mcpsuite passes.Scope
This fixes the reported symptom (widget hang). Making MCP tool-calling actually function through a cloud-proxy passthrough model (it's bypassed at
chat.go) is a separate, larger change and is intentionally not included here.Part of #10880
Assisted-by: Claude:opus-4.8 [Claude Code]