Skip to content

feat: Make model loading non-blocking in MCP serve#136

Merged
Pringled merged 6 commits into
mainfrom
fix-async
May 21, 2026
Merged

feat: Make model loading non-blocking in MCP serve#136
Pringled merged 6 commits into
mainfrom
fix-async

Conversation

@Pringled
Copy link
Copy Markdown
Member

@Pringled Pringled commented May 21, 2026

This PR makes the MCP's serve non-blocking for model loading (as well as direct indexing from a given path). This should hopefully resolve #133, or at least help since the model download no longer blocks the initial startup for Semble. force_download is also set to False for the model so we don't check HF everytime if the model is already cached. Tested locally by deleting my own cached model and it makes the server respond in ~0.3 seconds instead of ~5 seconds (on very very fast internet).

@Pringled Pringled requested a review from stephantul May 21, 2026 07:19
@codecov
Copy link
Copy Markdown

codecov Bot commented May 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/semble/index/dense.py 100.00% <100.00%> (ø)
src/semble/mcp.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@stephantul stephantul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval contingent on discussion, maybe i misunderstood

Comment thread src/semble/mcp.py
Comment thread src/semble/mcp.py
@Pringled Pringled merged commit d362683 into main May 21, 2026
16 checks passed
@Pringled Pringled deleted the fix-async branch May 21, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP server semble_search tool times out on first queries (CLI works fine)

2 participants