Skip to content

map-server: interactive annotations (markers, routes, areas, circles)#505

Open
ochafik wants to merge 17 commits intomainfrom
ochafik/map-interact
Open

map-server: interactive annotations (markers, routes, areas, circles)#505
ochafik wants to merge 17 commits intomainfrom
ochafik/map-interact

Conversation

@ochafik
Copy link
Contributor

@ochafik ochafik commented Feb 26, 2026

Summary

Adds full interactivity and rich annotation support to the CesiumJS map server:

  • Unified annotation system — discriminated union (oneOf in JSON Schema) supporting marker, route (polyline), area (polygon), and circle (ellipse) annotation types
  • Interact toolnavigate, add, update, remove actions via command queue with 200ms batching
  • Streaming supportontoolinputpartial progressively renders annotations as the model streams them (skipping last potentially-truncated item)
  • Geocode tool — multi-query OpenStreetMap Nominatim search with rate limiting
  • show-map — accepts initial annotations, supports bounding box or center+radius viewport modes
  • Persistence — annotations and camera state saved to localStorage, restored on revisit
  • Copy/export — toolbar button copies all annotations as Markdown table + GeoJSON (multi-mime clipboard)
  • Canvas label billboards — DPR-aware rendered labels with rounded-rect backgrounds, anchored per annotation type
  • Fullscreen — keyboard shortcuts (Escape, Ctrl/Cmd+Enter) and toolbar button
  • Model context updates — debounced camera position + screenshot sent to model on navigation

Annotation types

Type Cesium Entity Key Fields
marker Point + label billboard lat, lon, label, color
route Polyline (clamped to ground) points[], width, dashed, color
area Polygon (fill + outline) points[], color, fillColor
circle Ellipse lat, lon, radiusKm, color, fillColor

JSON Schema

Uses z.discriminatedUnion("type", [...]) which produces proper oneOf with const discriminator values. Separate full schemas (for add) and update schemas (partial fields, for update).

Test plan

  • show-map with no annotations → globe loads at default/bbox location
  • show-map with mixed annotations → markers, routes, areas, circles render correctly
  • interact add/update/remove → annotations appear/change/disappear
  • Navigate action → camera flies to new bounding box
  • Geocode with multiple queries → results returned with coordinates and bounding boxes
  • Annotations persist across page reloads (localStorage)
  • Copy button → clipboard contains Markdown + GeoJSON
  • Streaming (ontoolinputpartial) → annotations render progressively
  • Fullscreen toggle works via button and keyboard shortcuts

🤖 Generated with Claude Code

@pkg-pr-new
Copy link

pkg-pr-new bot commented Feb 26, 2026

Open in StackBlitz

@modelcontextprotocol/ext-apps

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/ext-apps@505

@modelcontextprotocol/server-basic-preact

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-preact@505

@modelcontextprotocol/server-basic-react

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-react@505

@modelcontextprotocol/server-basic-solid

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-solid@505

@modelcontextprotocol/server-basic-svelte

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-svelte@505

@modelcontextprotocol/server-basic-vanillajs

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-vanillajs@505

@modelcontextprotocol/server-basic-vue

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-basic-vue@505

@modelcontextprotocol/server-budget-allocator

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-budget-allocator@505

@modelcontextprotocol/server-cohort-heatmap

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-cohort-heatmap@505

@modelcontextprotocol/server-customer-segmentation

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-customer-segmentation@505

@modelcontextprotocol/server-debug

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-debug@505

@modelcontextprotocol/server-map

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-map@505

@modelcontextprotocol/server-pdf

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-pdf@505

@modelcontextprotocol/server-scenario-modeler

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-scenario-modeler@505

@modelcontextprotocol/server-shadertoy

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-shadertoy@505

@modelcontextprotocol/server-sheet-music

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-sheet-music@505

@modelcontextprotocol/server-system-monitor

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-system-monitor@505

@modelcontextprotocol/server-threejs

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-threejs@505

@modelcontextprotocol/server-transcript

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-transcript@505

@modelcontextprotocol/server-video-resource

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-video-resource@505

@modelcontextprotocol/server-wiki-explorer

npm i https://pkg.pr.new/modelcontextprotocol/ext-apps/@modelcontextprotocol/server-wiki-explorer@505

commit: ca25f6c

ochafik and others added 6 commits February 26, 2026 03:21
Add semi-transparent dark background behind labels, use bold font,
thicker outline, and disable depth test so labels are always visible.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change add_marker to add_markers accepting an array of markers
- Assign UUIDs to each marker and return them in the tool result
- Add update_markers and remove_markers actions to the interact tool
- Track markers by ID with Cesium entity references for edit/removal
- Add copy button (clipboard icon) that appears when markers exist
- Multi-mime clipboard: text/plain gets Markdown table + GeoJSON code
  block, text/html gets an HTML table with GeoJSON in a details block

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Geocode tool now accepts `queries` (string array) and performs
  batched lookups (respecting Nominatim rate limit between each)
- Show-map accepts initial `markers` array — IDs are assigned
  server-side and returned in structuredContent
- Show-map supports center+radius mode: `latitude`/`longitude` with
  optional `radiusKm` (default 50) as alternative to bounding box
- App handles initial markers from both ontoolinput and ontoolresult

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
structuredContent is only seen by the model when content is empty
(per MCP spec). Move initial marker data to _meta for the app, and
include marker IDs in the content text for the model. Also remove
structuredContent from add_markers (IDs already in text).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Marker id is now required and chosen by the caller on show-map
(initial markers), add_markers, and update_markers. Removes
sequential counter and server-side ID allocation — simpler and
produces shorter, more meaningful IDs in the conversation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Cesium's built-in label with canvas-rendered billboard images:
- Proper vertical text alignment (textBaseline: middle)
- Rounded rectangle background (quadraticCurveTo corners)
- System font stack, DPR-aware sizing
- Dark semi-transparent background (rgba 30,30,30,0.78)

Persist markers to localStorage keyed by `${viewUUID}:markers`.
Restored on reconnect before adding any new initial markers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ochafik and others added 7 commits February 26, 2026 04:01
The model already knows the IDs it picked — no need to repeat them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Split marker into two Cesium entities (point + billboard label)
  to avoid point/billboard conflict that suppressed labels entirely
- Remove heightReference: CLAMP_TO_GROUND (no terrain loaded, caused
  3D offset on tilted views)
- Use canvas roundRect() for cleaner rounded corners
- Use actualBoundingBoxAscent/Descent for precise text centering
- Add hint in show-map description: skip markers for single location
- Update/remove now correctly handles both entities

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Markers from ontoolinput are added before viewUUID is available,
so persistMarkers() was a no-op. Now explicitly persist after
ontoolresult finishes setting up viewUUID and all markers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move persistMarkers() out of addMarker/updateMarker/removeMarker into
processCommands (once after the entire batch). The ontoolresult handler
already has its own explicit persistMarkers() call.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The app runs in a sandboxed iframe — Clipboard API requires the host
to set allow="clipboard-write". Declare it via permissions metadata.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace marker-specific types with a discriminated union (oneOf in JSON
Schema) supporting four annotation types: marker, route, area, circle.

- z.discriminatedUnion for proper oneOf in tool schemas
- Separate full schemas (show-map/add) and update schemas (partial fields)
- Unified add/update/remove actions replacing add_markers/update_markers/remove_markers
- Cesium rendering: point (marker), polyline (route), polygon (area), ellipse (circle)
- Canvas label billboards for all types at appropriate anchor points
- ontoolinputpartial streaming: render all-but-last annotation progressively
- Persistence and clipboard export updated for all annotation types
- GeoJSON export: Point/LineString/Polygon with type-specific properties

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ochafik ochafik changed the title map-server: add interact tool with command queue map-server: interactive annotations (markers, routes, areas, circles) Feb 26, 2026
ochafik and others added 3 commits February 26, 2026 05:30
Annotations were added after `await waitForTilesLoaded()` which blocks
up to 10 seconds. If the host doesn't forward `_meta.initialAnnotations`
to ontoolresult, annotations wouldn't appear until tiles finish loading.

Now annotations are added immediately after camera positioning, before
the tile-loading await. Also adds field validation in ontoolinputpartial
to prevent creating broken entities from truncated streaming data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The camera only moved after the full tool input was received. Now
ontoolinputpartial positions the camera as soon as bbox/center fields
are available (once), so markers appear in the right location during
progressive streaming.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
If the camera was positioned during streaming (ontoolinputpartial),
skip re-positioning in ontoolinput. This preserves the user's view
if they panned or zoomed while annotations were streaming in.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant