diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a62ee5..4225e18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.7.0] - 2026-07-06 + +### Added +- `--raw` (CLI) / `raw` (MCP): return the unprocessed body verbatim — no Readability, no content-type gate; `MARKFETCH_MAX_BYTES` still applies. + +### Fixed +- Cloudflare-class CDNs 403 valid Chrome headers over HTTP/2 but pass them over HTTP/1.1 — markfetch now defaults to HTTP/1.1. +- Empty-heading pruning truncated headings containing an inline `#` (`## Step 1 # download` became `# download`). + ## [0.6.0] - 2026-05-14 ### Added diff --git a/README.md b/README.md index 4fb3ae3..845ecce 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ [![node](https://img.shields.io/node/v/markfetch.svg?color=10b981)](https://nodejs.org/) [![license](https://img.shields.io/npm/l/markfetch.svg?color=10b981)](https://github.com/vasylenko/markfetch/blob/main/LICENSE) -The built-in fetch tools that ship with AI coding agents return raw HTML, broken markdown, or `403` from Cloudflare more often than you'd like. `markfetch` sends **HTTP/2 with a coherent Chrome header set** so bot-detection systems see a real browser, then runs the response through the **same Reader View pipeline your browser uses** (Mozilla's Readability → turndown). The output is markdown indistinguishable from a human running "Save as Markdown" — on sites that would block a naive curl. +The built-in fetch tools that ship with AI coding agents return raw HTML, broken markdown, or `403` from Cloudflare more often than you'd like. `markfetch` sends **a coherent Chrome header set** so bot-detection systems see a real browser, then runs the response through the **same Reader View pipeline your browser uses** (Mozilla's Readability → turndown). The output is markdown indistinguishable from a human running "Save as Markdown" — on sites that would block a naive curl. One command, two surfaces: @@ -61,13 +61,13 @@ gemini mcp add -s user markfetch npx -y markfetch | CloudFlare `/markdown` | ✓ | ✓ | – | paid | | **`markfetch`** | **✓** | **✓** | **✓ (8 codes)** | **✓** | -- **Real-browser HTTP/2 + Chrome fingerprint.** ALPN-negotiated h2, `User-Agent`, `Sec-CH-UA-*`, `Sec-Fetch-*`, `Accept-*`. A Chrome UA with no client hints is a *stronger* automation signal than curl — `markfetch` sends the full coherent set, derived from the UA at startup so an override stays internally consistent. +- **Real-browser request fingerprint.** `User-Agent`, `Sec-CH-UA-*`, `Sec-Fetch-*`, `Accept-*` — a coherent Chrome header set. A Chrome UA with no client hints is a *stronger* automation signal than curl, so `markfetch` sends the full set, derived from the UA at startup so an override stays internally consistent. HTTP/1.1 over TLS: some CDNs fingerprint undici's HTTP/2 connection and 403 it. - **Reader-View-quality extraction.** [linkedom](https://github.com/WebReflection/linkedom) → [@mozilla/readability](https://github.com/mozilla/readability) → [turndown](https://github.com/mixmark-io/turndown) with GFM tables, strikethrough, and task lists. Code fences preserve `language-X` hints. Sphinx-style bare `
` blocks render as code, not escaped prose. Intraword underscores stay un-escaped — no more `list\_tools`.
 
-- **One tool, one shape (MCP).** `fetch_markdown(url, savePath?)` returns markdown in `content[0].text`. No `structuredContent`, no frontmatter, no metadata fields. Several major MCP clients (Claude Code CLI, VS Code/Copilot) forward only `structuredContent` to the model and drop `content[]` when both are present — `markfetch` deliberately stays on the channel your LLM can actually read.
+- **One tool, one shape (MCP).** `fetch_markdown(url, savePath?, raw?)` returns markdown in `content[0].text`. No `structuredContent`, no frontmatter, no metadata fields. Several major MCP clients (Claude Code CLI, VS Code/Copilot) forward only `structuredContent` to the model and drop `content[]` when both are present — `markfetch` deliberately stays on the channel your LLM can actually read.
 
-- **`savePath` / `-o` escape valve.** Pass an absolute path (MCP `savePath`) or `-o ` (CLI) and the markdown lands on disk instead of the response channel. Use it when your client's inline tool-result cap would truncate large responses, or to redirect output from a shell pipeline. The file is only ever the markdown of the URL — fetch errors return a `[code]` string and never touch the disk.
+- **`savePath` / `-o` escape valve.** Pass an absolute path (MCP `savePath`) or `-o ` (CLI) and the output lands on disk instead of the response channel. Use it when your client's inline tool-result cap would truncate large responses, or to redirect output from a shell pipeline. The file is only ever the fetched output (extracted markdown, or the raw body with `--raw`) — fetch errors return a `[code]` string and never touch the disk.
 
 - **Whole document or honest failure.** No pagination, no truncation. If the document doesn't fit in `MARKFETCH_MAX_BYTES`, you get `too_large` — never a half-truth.
 
@@ -88,6 +88,9 @@ npx -y markfetch https://example.com/article -o article.md
 
 # Pipe into another tool
 npx -y markfetch https://example.com/article | pandoc -o article.pdf
+
+# Fetch JSON / APIs / page source verbatim
+npx -y markfetch --raw https://api.github.com/repos/vasylenko/markfetch
 ```
 
 For repeat use, install once:
@@ -102,7 +105,8 @@ Flags:
 
 | Flag | Purpose |
 |---|---|
-| `-o, --output ` | Save markdown to a file (absolute or relative path). Default is stdout. |
+| `-o, --output ` | Save the output to a file (absolute or relative path). Default is stdout. |
+| `--raw` | Return the unprocessed response body as UTF-8 text — skips Readability and the content-type gate. For JSON, XML, plain text, or page source (binary is not byte-preserved). |
 | `-V, --version` | Print version and exit. |
 | `-h, --help` | Print usage and exit. |
 
@@ -115,8 +119,8 @@ Errors carry one of eight deterministic codes:
 | `network_error` | DNS / TCP / TLS failure, or an unexpected internal error from the fetcher. |
 | `http_error` | Upstream returned a non-2xx status. |
 | `timeout` | Per-request budget `MARKFETCH_TIMEOUT_MS` exceeded. |
-| `unsupported_content_type` | Response was not `text/html` or `application/xhtml+xml`. |
-| `extraction_failed` | Readability returned no article content (typical for pure client-rendered SPAs). |
+| `unsupported_content_type` | Response was not `text/html` or `application/xhtml+xml` (not raised with `--raw` / `raw`). |
+| `extraction_failed` | Readability returned no article content (typical for pure client-rendered SPAs). Not raised with `--raw` / `raw`. |
 | `too_large` | Response body or extracted markdown exceeded `MARKFETCH_MAX_BYTES`. |
 | `save_failed` | `savePath` was given but `writeFile` failed (parent directory missing, permission denied, etc.). |
 | `save_forbidden` | `savePath` resolves outside the allowed write roots — see [Write sandbox](#write-sandbox). MCP-only; the CLI has no sandbox. |
diff --git a/docs/SPEC.md b/docs/SPEC.md
index 1e53c92..4999a79 100644
--- a/docs/SPEC.md
+++ b/docs/SPEC.md
@@ -6,13 +6,15 @@ Text processing pipeline:
 
 ```
 URL
-  → undici.fetch       HTTP/2 via ALPN; full Chrome header set; Sec-CH-UA-* derived from UA
+  → undici.fetch       HTTP/1.1; full Chrome header set; Sec-CH-UA-* derived from UA
   → linkedom.parseHTML
   → @mozilla/readability
   → turndown + GFM     HTML → markdown
   → caller             markdown body, or "Saved N bytes to /path" confirmation
 ```
 
+`raw` mode (`--raw` / MCP `raw`) returns the body straight from `undici.fetch`, skipping the parse → extract → convert steps.
+
 Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `network_error`, `http_error`, `timeout`, `unsupported_content_type`, `extraction_failed`, `too_large`, `save_failed`; plus `save_forbidden`, emitted by the MCP adapter only (before `fetchMarkdown` runs — see "Asymmetric write sandbox" under Core Decisions). CLI emits `[code] message` to stderr and exits 1; MCP emits `{ isError: true, content: [{ text: "[code] message" }] }`.
 
 ## Core Decisions
@@ -23,7 +25,9 @@ Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `
 
 - **Core throws, adapters translate.** Seven of the eight error codes surface from `core.ts` — five are thrown explicitly as `MarkfetchError`; `network_error`, `timeout`, and (sometimes) `http_error` are translated by `classifyError` from underlying-API errors (undici TypeErrors, AbortSignal timeouts). The eighth code, `save_forbidden`, is the exception — it's emitted by the MCP adapter before `fetchMarkdown` is invoked (see "Asymmetric write sandbox" below). New core codes need an `ErrorCode` union member + a throw site; adapters don't change.
 
-- **HTTP/2 + coherent Chrome fingerprint.** Wire protocol, headers, and UA must agree — a Chrome UA over HTTP/1.1 or without `Sec-CH-UA-*` is *more* suspicious than curl. `Sec-CH-UA-*` is derived from `MARKFETCH_USER_AGENT` at startup so override-coherence is mechanical.
+- **HTTP/1.1 + coherent Chrome header fingerprint.** Headers and UA must agree — a Chrome UA without `Sec-CH-UA-*` is a stronger bot signal than curl, so `Sec-CH-UA-*` is derived from `MARKFETCH_USER_AGENT` at startup and override-coherence is mechanical. HTTP/1.1 is deliberate: undici's h2 path hands `node:http2` a pre-connected socket whose first-flight frames some CDNs (Cloudflare, seen on `openai.com`) score as a bot and 403; the identical request over h1.1 passes. h2 buys nothing for single-shot GETs.
+
+- **`raw` passthrough.** Returns the fetched body verbatim — no Readability, no content-type gate; the fetch layer and `MARKFETCH_MAX_BYTES` cap stay. Same `fetchMarkdown` code path via a `raw` flag — no second entry point. Body is UTF-8 text; binary is not byte-preserved.
 
 - **Single-channel MCP response.** `content[0].text` only. Several major MCP clients (Claude Code CLI, VS Code/Copilot) forward only `structuredContent` to the model and drop `content[]` when both are present — a single-channel response keeps the markdown reachable from those clients.
 
@@ -40,6 +44,7 @@ Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `
 - **Authentication.** `MARKFETCH_AUTH_HEADER` env var (simple), or Chrome-cookie import for sites where the user is already logged in (frictionless, platform-specific, security-sensitive). Trigger: first useful internal / paywalled doc.
 - **JS rendering fallback for SPAs.** Playwright / headless Chrome as a companion package (`markfetch-heavy`) so the lean package stays lean. Trigger: enough useful sites returning `extraction_failed`.
 - **CloudFlare `/markdown` fallback.** Gated by `CF_AUTH_TOKEN`; fall back when Readability fails. Trigger: extraction failure rate stays high after Readability tuning.
+- **Browser-grade TLS + HTTP/2 impersonation.** Stricter CDN tiers fingerprint Node's TLS (JA3/JA4) and 403 every protocol. Apify's `impit` (BoringSSL-based) ships a Chrome-matching TLS + h2 fingerprint, near-drop-in for `undici.fetch`. Trigger: target sites 403 the current h1.1 + Chrome-header approach.
 - **Cookie reuse across redirects within a single fetch.** Currently none. Trigger: a target serves content only after a session-cookie redirect.
 - **Proxy support** (`MARKFETCH_PROXY_URL`) and **`Accept-Language` control** (`MARKFETCH_ACCEPT_LANGUAGE`). Trigger: corporate proxy / locale-specific content.
 - **Single-binary distribution.** Bun's `build --compile`, Node SEA, or similar. Trigger: `npx` first-run latency feedback, or an offline / airgapped need.
diff --git a/package-lock.json b/package-lock.json
index 3a1e30b..7d4aa69 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,12 +1,12 @@
 {
   "name": "markfetch",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "markfetch",
-      "version": "0.6.0",
+      "version": "0.7.0",
       "license": "MIT",
       "dependencies": {
         "@modelcontextprotocol/sdk": "^1.29.0",
diff --git a/package.json b/package.json
index 9522e90..fb91208 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "markfetch",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "Fetch a URL, return clean markdown. MCP server and CLI for AI agents.",
   "license": "MIT",
   "author": {
diff --git a/src/cli.ts b/src/cli.ts
index 20af7f2..6c9fbb6 100644
--- a/src/cli.ts
+++ b/src/cli.ts
@@ -27,9 +27,13 @@ program
   .argument("", "absolute http(s) URL to fetch")
   .option(
     "-o, --output ",
-    "save markdown to file (absolute or relative path); default is stdout",
+    "save output to a file (absolute or relative path); default is stdout",
   )
-  .action(async (url: string, options: { output?: string }) => {
+  .option(
+    "--raw",
+    "return the unprocessed response body (skip Readability and the content-type gate)",
+  )
+  .action(async (url: string, options: { output?: string; raw?: boolean }) => {
     // CLI resolves relative output paths against cwd before calling core;
     // core requires an absolute path so the contract is unambiguous regardless
     // of which adapter invokes it. Tilde expansion is intentionally NOT done
@@ -43,9 +47,10 @@ program
       const { markdown, bytes, savedTo } = await fetchMarkdown({
         url,
         savePath,
+        raw: options.raw,
       });
       if (savedTo === undefined) {
-        // Raw markdown body — no added newline, matches MCP content[0].text.
+        // Verbatim output — no added newline, matches MCP content[0].text.
         process.stdout.write(markdown);
       } else {
         // Confirmation message — the only stdout newline the CLI ever adds.
diff --git a/src/core.ts b/src/core.ts
index f2a6d4b..d8a5d89 100644
--- a/src/core.ts
+++ b/src/core.ts
@@ -86,13 +86,8 @@ function deriveClientHints(ua: string): {
 
 const clientHints = deriveClientHints(config.userAgent);
 
-// Enable HTTP/2 via TLS ALPN. Modern bot-detection systems and CDNs consider
-// wire protocol alongside header fingerprint; HTTP/2 over TLS pairs cleanly
-// with a Chrome header set. Servers that don't advertise h2 in ALPN fall back
-// to HTTP/1.1 transparently during the TLS handshake — no manual retry needed.
-// Plain-HTTP connections (port 80) skip ALPN entirely and use HTTP/1.1,
-// accepting the protocol/fingerprint mismatch in that case.
-setGlobalDispatcher(new Agent({ allowH2: true }));
+// HTTP/1.1 because undici's h2 handshake trips CDN bot-scoring (Cloudflare 403s Chrome headers over h2 but not h1.1).
+setGlobalDispatcher(new Agent({ allowH2: false }));
 
 const TURNDOWN = new TurndownService({
   headingStyle: "atx",
@@ -140,8 +135,11 @@ TURNDOWN.escape = (s: string): string =>
 TURNDOWN.addRule("barePre", {
   filter: (node) => node.nodeName === "PRE" && !node.querySelector("code"),
   replacement: (_content, node) => {
-    const text = (node.textContent ?? "").replace(/\n+$/, "");
-    return "\n\n```\n" + text + "\n```\n\n";
+    // index-based trailing-newline trim — /\n+$/ backtracks super-linearly
+    const text = node.textContent ?? "";
+    let end = text.length;
+    while (end > 0 && text[end - 1] === "\n") end--;
+    return "\n\n```\n" + text.slice(0, end) + "\n```\n\n";
   },
 });
 
@@ -228,7 +226,8 @@ function chromeHeaders(): Record {
   };
 }
 
-async function fetchHtml(url: string): Promise {
+// raw = skip the HTML content-type gate and pass any body through verbatim.
+async function fetchBody(url: string, raw: boolean): Promise {
   const response = await fetch(url, {
     signal: AbortSignal.timeout(config.timeoutMs),
     headers: chromeHeaders(),
@@ -241,15 +240,17 @@ async function fetchHtml(url: string): Promise {
     );
   }
 
-  const contentType = (response.headers.get("content-type") ?? "").toLowerCase();
-  if (
-    !contentType.startsWith("text/html") &&
-    !contentType.startsWith("application/xhtml+xml")
-  ) {
-    throw new MarkfetchError(
-      "unsupported_content_type",
-      `Content-Type: ${contentType || "(missing)"}`,
-    );
+  if (!raw) {
+    const contentType = (response.headers.get("content-type") ?? "").toLowerCase();
+    if (
+      !contentType.startsWith("text/html") &&
+      !contentType.startsWith("application/xhtml+xml")
+    ) {
+      throw new MarkfetchError(
+        "unsupported_content_type",
+        `Content-Type: ${contentType || "(missing)"}`,
+      );
+    }
   }
 
   const declaredLength = Number(response.headers.get("content-length") ?? 0);
@@ -257,12 +258,13 @@ async function fetchHtml(url: string): Promise {
     throw enforceTooLarge("Content-Length", declaredLength);
   }
 
-  const html = await response.text();
-  if (Buffer.byteLength(html, "utf8") > config.maxBytes) {
-    throw enforceTooLarge("Body", Buffer.byteLength(html, "utf8"));
+  const body = await response.text();
+  const size = Buffer.byteLength(body, "utf8");
+  if (size > config.maxBytes) {
+    throw enforceTooLarge("Body", size);
   }
 
-  return html;
+  return body;
 }
 
 function enforceTooLarge(stage: string, actual: number): MarkfetchError {
@@ -379,24 +381,39 @@ function convertToMarkdown(article: {
   // an interactive widget (MDN browser-compat tables, MCP spec diagrams,
   // etc.) but leaves the orphan heading. Iterate until stable so a parent
   // section that becomes empty after its last child heading is pruned also
-  // gets removed.
+  // gets removed. Blank spans match line-by-line — \s* backtracks super-linearly
+  // and lets a mid-line "#" pass as the next heading.
   let prev: string;
   do {
     prev = result;
-    result = result.replaceAll(/^(#{1,6}) [^\n]+\s*(?=#{1,6} )/gm, "");
+    result = result.replaceAll(/^#{1,6} [^\n]+\n(?:[ \t]*\n)*[ \t]*(?=#{1,6} )/gm, "");
   } while (result !== prev);
   // The lookahead-based iteration above can't catch a trailing empty
   // heading at EOF (no following heading to anchor on). One-shot pass.
-  result = result.replace(/(?:^|\n)#{1,6} [^\n]+\s*$/, "");
+  result = result.replace(/(?:^|\n)#{1,6} [^\n]+(?:\n[ \t]*)*$/, "");
   return result;
 }
 
+// Throws extraction_failed when Readability finds nothing (pure client-rendered SPAs).
+function extractMarkdown(html: string, url: string): string {
+  const article = extractArticle(html, url);
+  if (!article) {
+    throw new MarkfetchError(
+      "extraction_failed",
+      "Readability returned no article content.",
+    );
+  }
+  return convertToMarkdown(article);
+}
+
 // --- Unified entry point ---
 
 // Adapters call this with already-validated inputs (URL syntax checked by the
 // adapter's schema; savePath, if present, is an absolute path — adapters
 // resolve any relative-vs-absolute concerns before calling).
 //
+// With raw, unsupported_content_type and extraction_failed cannot arise.
+//
 // Errors are thrown uniformly as MarkfetchError. Adapters catch and translate:
 //   - mcp.ts catches → errorResult(code, message) → MCP {isError, content}
 //   - cli.ts catches → console.error("[code] message") → sets process.exitCode = 1
@@ -412,31 +429,18 @@ function convertToMarkdown(article: {
 export async function fetchMarkdown(input: {
   url: string;
   savePath?: string;
+  raw?: boolean;
 }): Promise<{ markdown: string; bytes: number; savedTo?: string }> {
-  const { url, savePath } = input;
-  const html = await fetchHtml(url);
-  const article = extractArticle(html, url);
-  if (!article) {
-    throw new MarkfetchError(
-      "extraction_failed",
-      "Readability returned no article content.",
-    );
-  }
-  const markdown = convertToMarkdown(article);
-  const bytes = Buffer.byteLength(markdown, "utf8");
-  if (bytes > config.maxBytes) {
-    throw new MarkfetchError(
-      "too_large",
-      `Markdown ${bytes} bytes > MARKFETCH_MAX_BYTES (${config.maxBytes})`,
-    );
-  }
-  // The file at savePath is only ever the markdown of the URL. Fetch /
-  // extraction / size-cap failures all throw above and never reach this
-  // branch, so the file is never written for them. save_failed is its own
-  // phase, handled here.
+  const { url, savePath, raw = false } = input;
+  const body = await fetchBody(url, raw);
+  const content = raw ? body : extractMarkdown(body, url);
+  const bytes = Buffer.byteLength(content, "utf8");
+  if (bytes > config.maxBytes) throw enforceTooLarge("Output", bytes);
+  // The file only ever holds the fetched output: failures throw above and
+  // never reach this branch. save_failed is its own phase, handled here.
   if (savePath !== undefined) {
     try {
-      await writeFile(savePath, markdown, "utf8");
+      await writeFile(savePath, content, "utf8");
     } catch (err) {
       // Node's fs errors prefix the message with the errno already
       // (e.g. "ENOENT: no such file or directory, open '/path'"), so
@@ -448,7 +452,7 @@ export async function fetchMarkdown(input: {
         e.message || (e.code ?? "unknown error"),
       );
     }
-    return { markdown, bytes, savedTo: savePath };
+    return { markdown: content, bytes, savedTo: savePath };
   }
-  return { markdown, bytes };
+  return { markdown: content, bytes };
 }
diff --git a/src/mcp.ts b/src/mcp.ts
index 94648f4..049a226 100644
--- a/src/mcp.ts
+++ b/src/mcp.ts
@@ -33,7 +33,7 @@ server.registerTool(
   "fetch_markdown",
   {
     description:
-      "Fetch a single public HTTP/S URL and return its main article content as clean markdown. Best for articles, documentation, blog posts, news, and reference pages. Non-HTML responses return `unsupported_content_type`. Pure client-rendered SPAs with no extractable static HTML return `extraction_failed`; SPAs that ship server-rendered or SEO-prerendered HTML will extract whatever static content they expose. Also supports saving the markdown to a file, e.g., to bypass client tool-result size limits or to reuse later. Saved files must land inside the allowed write roots (defaults: system temp dir and the server's working directory; configurable via `MARKFETCH_ALLOWED_WRITE_ROOTS`); paths outside return `save_forbidden`.",
+      "Fetch a public HTTP/S URL and return its main article content as clean markdown. Best for articles, documentation, blog posts, and reference pages. Non-HTML responses return `unsupported_content_type` unless `raw` is set; pure client-rendered SPAs return `extraction_failed`. Set `savePath` to write the output to a file instead of returning it inline.",
     inputSchema: {
       url: z
         .string()
@@ -46,11 +46,17 @@ server.registerTool(
         .refine(isAbsolute, "savePath must be an absolute filesystem path")
         .optional()
         .describe(
-          "Optional. When provided, the fetched markdown is written to this absolute filesystem path and the response becomes a small confirmation. Use this when the markdown might exceed your client's tool-result inline cap. Must be an absolute path on the host platform (e.g., `/foo/bar.md` on POSIX; `C:\\foo\\bar.md` or `\\\\server\\share\\bar.md` on Windows); relative paths and tilde paths (`~/...`) are rejected by the schema. Writes are confined to an allow-listed sandbox — defaults are the system temp dir (`os.tmpdir()`) and the server's working directory; operators can override with `MARKFETCH_ALLOWED_WRITE_ROOTS` (path-delimiter-separated). A `savePath` outside the allowed roots returns `save_forbidden` and no file is created. Existing files are overwritten; the parent directory must exist (caller's responsibility). The file is written only on fetch success — fetch / extraction / size-cap errors return a `[code]` string and never touch the file.",
+          "Absolute path to write the output to instead of returning it inline; the response becomes a short confirmation. Use when the output might exceed your client's tool-result cap. Relative and `~` paths are rejected. Writes are sandboxed to allowed roots (defaults: system temp dir and the server's working directory; override with `MARKFETCH_ALLOWED_WRITE_ROOTS`) — paths outside return `save_forbidden`. Existing files are overwritten; the parent directory must exist. Fetch errors never touch the file.",
+        ),
+      raw: z
+        .boolean()
+        .optional()
+        .describe(
+          "Return the response body verbatim as UTF-8 text (binary is not byte-preserved), skipping Readability and the HTML content-type gate — for JSON, APIs, or raw page source. `MARKFETCH_MAX_BYTES` still applies.",
         ),
     },
   },
-  async ({ url, savePath }) => {
+  async ({ url, savePath, raw }) => {
     // Sandbox gate (MCP-only; CLI is intentionally unbounded). Runs before
     // fetchMarkdown so a forbidden path short-circuits the fetch. The
     // canonicalized check.resolved — not the caller's savePath — is what
@@ -68,6 +74,7 @@ server.registerTool(
       const { markdown, bytes, savedTo } = await fetchMarkdown({
         url,
         savePath: resolvedSavePath,
+        raw,
       });
       if (savedTo !== undefined) {
         // Echo the caller's original savePath in the confirmation. The bytes
diff --git a/tests/cli.test.ts b/tests/cli.test.ts
index ebdcdd5..83c9bb1 100644
--- a/tests/cli.test.ts
+++ b/tests/cli.test.ts
@@ -76,6 +76,23 @@ test("CLI: happy path → markdown to stdout, stderr empty, exit 0", async () =>
   }
 });
 
+test("CLI: --raw returns the unprocessed body verbatim to stdout", async () => {
+  const RAW = '{"hello":"world","n":42}';
+  const mock = await startMock((_req, res) => {
+    res.writeHead(200, { "Content-Type": "application/json" });
+    res.end(RAW);
+  });
+  try {
+    const { code, stdout, stderr } = await runCli([mock.url, "--raw"]);
+    assert.equal(code, 0, `stderr was: ${stderr}`);
+    assert.equal(stderr, "");
+    // Byte-for-byte: content-type gate and Readability skipped, no added newline.
+    assert.equal(stdout, RAW);
+  } finally {
+    await mock.close();
+  }
+});
+
 test("CLI: -o  writes file, prints confirmation to stdout", async () => {
   const mock = await startMock((_req, res) => {
     res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
diff --git a/tests/fixtures/10-inline-hash-heading.expected.md b/tests/fixtures/10-inline-hash-heading.expected.md
new file mode 100644
index 0000000..b8c1d64
--- /dev/null
+++ b/tests/fixtures/10-inline-hash-heading.expected.md
@@ -0,0 +1,11 @@
+# Runner setup guide
+
+This guide walks through installing and registering a self-hosted runner, from download to the first green build. Each step is safe to re-run if something goes wrong halfway.
+
+## Step 1 # download the agent
+
+Grab the latest release tarball for your platform and unpack it into an empty directory owned by the service user. Keep the directory outside the repository checkout.
+
+## Step 2 # register with the server
+
+Run the config script with the registration token from the project settings page. The token is single-use and expires after one hour, so generate it right before registering.
\ No newline at end of file
diff --git a/tests/fixtures/10-inline-hash-heading.html b/tests/fixtures/10-inline-hash-heading.html
new file mode 100644
index 0000000..a9ecdb5
--- /dev/null
+++ b/tests/fixtures/10-inline-hash-heading.html
@@ -0,0 +1,14 @@
+
+
+Runner setup guide
+
+  

+    
Runner setup guide

+    
This guide walks through installing and registering a self-hosted runner, from download to the first green build. Each step is safe to re-run if something goes wrong halfway.

+    
Step 1 # download the agent

+    
Grab the latest release tarball for your platform and unpack it into an empty directory owned by the service user. Keep the directory outside the repository checkout.

+    
Step 2 # register with the server

+    
Run the config script with the registration token from the project settings page. The token is single-use and expires after one hour, so generate it right before registering.

+  

+
+
diff --git a/tests/server.test.ts b/tests/server.test.ts
index 77f05a0..cacdae8 100644
--- a/tests/server.test.ts
+++ b/tests/server.test.ts
@@ -219,6 +219,73 @@ test("error: extraction_failed when Readability finds nothing", async () => {
   }
 });
 
+test("raw: non-HTML body is returned verbatim (content-type gate and Readability bypassed)", async () => {
+  const RAW = '{"hello":"world","items":[1,2,3]}';
+  const mock = await startMock((_req, res) => {
+    res.writeHead(200, { "Content-Type": "application/json" });
+    res.end(RAW);
+  });
+  const client = await spawnClient();
+  try {
+    const result = await client.callTool({
+      name: "fetch_markdown",
+      arguments: { url: mock.url, raw: true },
+    });
+    // A JSON body would be unsupported_content_type without raw; here it comes
+    // back byte-for-byte, proving both the gate and extraction were skipped.
+    assert.equal(result.isError, false);
+    assert.equal(textOf(result), RAW);
+  } finally {
+    await client.close();
+    await mock.close();
+  }
+});
+
+test("raw: HTML is returned unprocessed, not run through Readability", async () => {
+  const HTML =
+    "
Title
Body text.
";
+  const mock = await startMock((_req, res) => {
+    res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
+    res.end(HTML);
+  });
+  const client = await spawnClient();
+  try {
+    const result = await client.callTool({
+      name: "fetch_markdown",
+      arguments: { url: mock.url, raw: true },
+    });
+    assert.equal(result.isError, false);
+    // Verbatim HTML — the