Skip to content

fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880)#10884

Open
localai-bot wants to merge 1 commit into
masterfrom
fix/mcp-connect-timeout-10880
Open

fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880)#10884
localai-bot wants to merge 1 commit into
masterfrom
fix/mcp-connect-timeout-10880

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Problem

For a model that declares mcp: servers, the chat UI's MCP Servers widget "spins forever" and the servers never become selectable. Reported in #10880 for a cloud-proxy model.

Root cause

Building an MCP session (SessionsFromMCPConfig / NamedSessionsFromMCPConfig in core/http/endpoints/mcp/tools.go) holds the session-cache mutex across client.Connect with no per-connect timeout:

  • a remote server that is unreachable is bounded only by the 360s httpClient timeout;
  • a stdio server whose initialize handshake never completes has no bound at all.

Either one blocks the request that the widget's GET /v1/mcp/servers/<model> makes, and because the mutex is held, it also wedges every other MCP request for that model. The frontend keeps its spinner up until the request returns, so it appears to hang forever.

Why cloud-proxy makes it visible: a cloud-proxy passthrough model returns early in the chat path (chat.go IsCloudProxyBackendPassthrough()) before the MCP tool block, so its session cache is never warmed in the background. The widget's listServers call is then the first code that connects — synchronously, in the request foreground — so any slow/unreachable server shows up as a permanent spinner. (Normal backends warm the cache during chat, so by the time the widget opens, listServers hits the fast alive-session path.)

Fix

Add connectMCP, which bounds each client.Connect with config.DefaultMCPDiscoveryTimeout (60s — long enough for a first-run stdio image pull, finite instead of "forever").

The subtlety: the ctx passed to Connect is the session's lifetime context (bound into jsonrpc2.NewConnection; cancelled later via the cached cancel func), and it is shared across all servers for a model. So we cannot pass a WithTimeout context — firing it would tear a healthy session down — nor cancel the shared context on timeout, which would kill sibling servers that already connected. Instead connectMCP runs Connect on the shared context in a goroutine and stops waiting after the timeout, returning an error for just that one server. A genuinely stalled goroutine holds only the shared context and is reaped when the model's sessions are evicted/cancelled. Applied to both session builders.

Verification

  • go build, go vet (incl. the lostcancel analyzer — clean), and golangci-lint (new-from-merge-base): 0 issues.
  • Added connect_timeout_test.go: a stdio server that starts but never speaks MCP (sleep) makes connectMCP return a timeout error in ~the timeout instead of hanging. Full core/http/endpoints/mcp suite passes.

Note: the local pre-commit coverage gate (make test-coverage-check) folds in DB-backed suites that launch a postgres:16 testcontainer; it couldn't run in my Docker-less environment (25 unrelated BeforeEach failures at core/services/testutil/testdb.go:33). CI runs it with Docker.

Scope

This fixes the reported symptom (widget hang). Making MCP tool-calling actually function through a cloud-proxy passthrough model (it's bypassed at chat.go) is a separate, larger change and is intentionally not included here.

Part of #10880

Assisted-by: Claude:opus-4.8 [Claude Code]

…ng the widget (#10880)

Establishing an MCP session held the session-cache mutex across
client.Connect with no per-connect timeout. An unreachable remote server
(bounded only by the 360s httpClient timeout) or a stdio server whose
initialize handshake never completes therefore blocked the caller and,
because the mutex was held, every other MCP request for that model too.
In the UI this shows up as the MCP "Servers" widget spinning forever.

It is most visible for cloud-proxy models: their chat path bails out
before the MCP tool block, so it never warms the session cache in the
background. The widget's /v1/mcp/servers/<model> call is then the first
and only code that connects synchronously, in the request foreground.

The session, once established, stays bound to the shared context (it is
cancelled later via the cached cancel func on eviction/shutdown), so we
can't pass a WithTimeout context to Connect: firing the timeout would tear
a healthy session down, and cancelling the shared context would also kill
sibling servers that already connected. Instead connectMCP runs Connect on
the shared context in a goroutine and stops waiting after the discovery
timeout, returning an error for that one server without disturbing the
others. A stalled goroutine is reaped when the model's sessions are
cancelled. Applied to both SessionsFromMCPConfig and
NamedSessionsFromMCPConfig.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:opus-4.8 [Claude Code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants