pdf-server: annotations, interact tool, page extraction & prompt engineering

examples/pdf-server/README.md

@@ -156,11 +156,92 @@ bun examples/pdf-server/main.ts --stdio ./papers/
 
 ## Tools
 
-| Tool             | Visibility | Purpose                                |
-| ---------------- | ---------- | -------------------------------------- |
-| `list_pdfs`      | Model      | List available local files and origins |
-| `display_pdf`    | Model + UI | Display interactive viewer             |
-| `read_pdf_bytes` | App only   | Stream PDF data in chunks              |
+| Tool             | Visibility | Purpose                                               |
+| ---------------- | ---------- | ----------------------------------------------------- |
+| `list_pdfs`      | Model      | List available local files and origins                |
+| `display_pdf`    | Model + UI | Display interactive viewer                            |
+| `interact`       | Model      | Navigate, annotate, search, extract pages, fill forms |
+| `read_pdf_bytes` | App only   | Stream PDF data in chunks                             |
+
+## Example Prompts
+
+After the model calls `display_pdf`, it receives the `viewUUID` and a description of all capabilities. Here are example prompts and follow-ups that exercise annotation features:
+
+### Annotating
+
+> **User:** Show me the Attention Is All You Need paper
+>
+> _Model calls `display_pdf` → viewer opens_
+>
+> **User:** Highlight the title and add an APPROVED stamp on the first page.
+>
+> _Model calls `interact` with `highlight_text` for the title and `add_annotations` with a stamp_
+
+> **User:** Can you annotate this PDF? Mark important sections for me.
+>
+> _Model calls `interact` with `get_pages` to read content first, then `add_annotations` with highlights/notes_
+
+> **User:** Add a note on page 1 saying "Key contribution" at position (200, 500), and highlight the abstract.
+>
+> _Model calls `interact` with `add_annotations` containing a `note` and either `highlight_text` or a `highlight` annotation_
+
+### Navigation & Search
+
+> **User:** Search for "self-attention" in the paper.
+>
+> _Model calls `interact` with action `search`, query `"self-attention"`_
+
+> **User:** Go to page 5.
+>
+> _Model calls `interact` with action `navigate`, page `5`_
+
+### Page Extraction
+
+> **User:** Give me the text of pages 1–3.
+>
+> _Model calls `interact` with action `get_pages`, intervals `[{start:1, end:3}]`, getText `true`_
+
+> **User:** Take a screenshot of the first page.
+>
+> _Model calls `interact` with action `get_pages`, intervals `[{start:1, end:1}]`, getScreenshots `true`_
+
+### Stamps & Form Filling
+
+> **User:** Stamp this document as CONFIDENTIAL on every page.
+>
+> _Model calls `interact` with `add_annotations` containing `stamp` annotations on each page_
+
+> **User:** Fill in the "Name" field with "Alice" and "Date" with "2026-02-26".
+>
+> _Model calls `interact` with action `fill_form`, fields `[{name:"Name", value:"Alice"}, {name:"Date", value:"2026-02-26"}]`_
+
+## Testing
+
+### E2E Tests (Playwright)
+
+```bash
+# Run annotation E2E tests (renders annotations in a real browser)
+npx playwright test tests/e2e/pdf-annotations.spec.ts
+
+# Run all PDF server tests
+npx playwright test -g "PDF Server"
+```
+
+### API Prompt Discovery Tests
+
+These tests verify that Claude can discover and use annotation capabilities by calling the Anthropic Messages API with the tool schemas. They are **disabled by default** — skipped unless `ANTHROPIC_API_KEY` is set:
+
+```bash
+ANTHROPIC_API_KEY=sk-ant-... npx playwright test tests/e2e/pdf-annotations-api.spec.ts
+```
+
+The API tests simulate a conversation where `display_pdf` has already been called, then send a follow-up user message and verify the model uses annotation actions (or at least the `interact` tool). Three scenarios are tested:
+
+| Scenario             | User prompt                                                       | Expected model behavior                    |
+| -------------------- | ----------------------------------------------------------------- | ------------------------------------------ |
+| Direct annotation    | "Highlight the title and add an APPROVED stamp"                   | Uses `highlight_text` or `add_annotations` |
+| Capability discovery | "Can you annotate this PDF?"                                      | Uses interact or mentions annotations      |
+| Specific notes       | "Add a note saying 'Key contribution' and highlight the abstract" | Uses `interact` tool                       |
 
 ## Architecture
 
@@ -182,8 +263,12 @@ src/
 | External links    | `app.openLink()`                            |
 | View persistence  | `viewUUID` + localStorage                   |
 | Theming           | `applyDocumentTheme()` + CSS `light-dark()` |
+| Annotations       | DOM overlays + pdf-lib embed on download    |
+| Command queue     | Server enqueues → client polls + processes  |
+| File download     | `app.downloadFile()` for annotated PDF      |
 
 ## Dependencies
 
 - `pdfjs-dist`: PDF rendering (frontend only)
+- `pdf-lib`: Client-side PDF modification for annotated download
 - `@modelcontextprotocol/ext-apps`: MCP Apps SDK

examples/pdf-server/mcp-app.html

@@ -60,6 +60,14 @@
             <button id="zoom-in-btn" class="zoom-btn" title="Zoom in (+)">
               +
             </button>
+            <button
+              id="download-btn"
+              class="download-btn"
+              title="Download PDF"
+              style="display: none"
+            >
+              <svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M8 2v8m0 0l-3-3m3 3l3-3"/><path d="M2 12v2h12v-2"/></svg>
+            </button>
             <button
               id="search-btn"
               class="search-btn"
@@ -111,6 +119,7 @@
         <div class="canvas-container">
           <div class="page-wrapper">
             <canvas id="pdf-canvas"></canvas>
+            <div id="annotation-layer" class="annotation-layer"></div>
             <div id="highlight-layer" class="highlight-layer"></div>
             <div id="text-layer" class="text-layer"></div>
           </div>

examples/pdf-server/package.json

@@ -28,6 +28,7 @@
     "@modelcontextprotocol/sdk": "^1.24.0",
     "cors": "^2.8.5",
     "express": "^5.1.0",
+    "pdf-lib": "^1.17.1",
     "pdfjs-dist": "^5.0.0",
     "zod": "^4.1.13"
   },