Skip to content

maxylev/searchfetch

Repository files navigation

SearchFetch (MCP Server)

A fault-tolerant, stealth-enabled Model Context Protocol (MCP) server for web searching and content fetching. Built for AI Agents (Cursor, Claude Code, OpenCode), it uses a stealth browser engine to fetch pages, dynamically handles SPAs/React, and converts bloat into token-optimized Markdown.

Features

  • Stealth Engine: Powered by CloakBrowser C++ patches + humanize logic. The browser renders and moves like a real user, reducing bot-detection scores.
  • Fault Tolerance: Auto-healing browser sessions, grace-period timeouts for SPAs, and network-level blocking of tracking scripts and media.
  • Token-Optimized Output: Strips base64 images, SVGs, scripts, and inline styles from the DOM before Markdown conversion.
  • Dual Runtime: Natively supports zero-install execution via both Python (uvx) and Node.js (npx).
  • Template-Driven Extraction: Structured extraction via shared JSON templates (GitHub, npm, PyPI, crates.io, docs pages, Docker Hub, and more). Supports custom inline templates.

Usage & Installation

You do not need to install this repository manually. Configure your agent to use the zero-install commands npx or uvx.

Claude Desktop Configuration

Option A: Python (uvx - Recommended)

{
  "mcpServers": {
    "searchfetch": {
      "command": "uvx",
      "args": ["searchfetch"]
    }
  }
}

Option B: Node.js (npx)

{
  "mcpServers": {
    "searchfetch": {
      "command": "npx",
      "args": ["-y", "searchfetch"]
    }
  }
}

Cursor / IDE Configuration

Add via the MCP panel in Cursor settings:

  • Type: command
  • Command: uvx searchfetch (or npx -y searchfetch)

Available Tools

1. websearch

Search the web through the template pipeline. DuckDuckGo and Google are built-in; custom search templates can be selected by name.

Parameter Type Default Description
query string required The search query string.
engine string "duckduckgo" "duckduckgo", "google", or a custom search template name.
max_results number 10 Maximum number of results to return.
region string/null null Region/language code (e.g. "us-en", "de-de"). DDG maps directly; Google maps to gl/hl.
safe_search boolean/null null Enable safe search. null uses the template default.
block_media boolean true Block images, media, and fonts at the network layer.

2. webfetch

Fetch a page with the stealth browser and extract structured Markdown using a template. Falls back to generic Markdown extraction for unknown pages.

Parameter Type Default Description
url string required Full URL (must start with http/https).
template string "auto" "auto", a built-in name, or inline JSON template.
start_index number 0 Character offset for pagination.
max_length number 10000 Maximum characters per request.
block_media boolean true Block images, videos, and fonts at the network layer.

Template extraction supports text, markdown, attribute, and html formats; nested children; repeated sections; URL-decoding transforms; per-template cookies; and per-template resource blocking.

Built-in templates live in templates/*.json and are shared by the Node.js and Python implementations.


Local Development

# Install dependencies
npm install
pip install -e ".[dev]"

# Run tests
npm test                # runs all tests (JS + Python)
npm run test:js         # Node.js unit tests (built-in test runner)
npm run test:py         # Python unit tests (pytest)

# Lint
npm run lint            # runs all linters
npm run lint:js         # ESLint
npm run lint:py         # Ruff

# Format
npm run format          # auto-format all source files
npm run format:check    # check formatting without changes

# MCP inspector (for manual testing)
npm run inspector-js    # test with MCP Inspector (Node.js)
npm run inspector-py    # test with MCP Inspector (Python)

Architecture

This repository uses a flat dual-manifest structure (package.json and pyproject.toml in the root). Both runtimes (index.js for Node, server.py for Python) share the same templates/*.json files and maintain feature parity.

About

Fault-tolerant MCP Server for Stealth Web Search and Fetching

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors