Capture why agents call each tool

[masnwilliams]

Summary

Turns on intent capture, which was deliberately left off when analytics landed. Every advertised tool now carries a required context argument the agent fills with why it is making the call, recorded as $mcp_intent.

This is the one signal the existing data can't produce. Today we know manage_browsers fails 10% of the time; we don't know what agents were trying to do when it did. Same for the two toolsets that are advertised and never called — no way to tell whether agents don't want them or don't understand them.

What changes on the tool surface

tools/list gains one string property per tool, in required. Verified against the running route: all 17 tools get it, no tool is skipped, and manage_browsers comes back with required: ["action", "context"].

A client that ignores it is not broken. The argument is injected into the advertised schema only, so a call with no context validates and runs exactly as before — it just records no intent. Confirmed with the same tool called both ways.

The description is ours, not the SDK's

The default is ~600 characters and repeats per tool, which is ~9 KB added to every tools/list response on a server that answers thousands a day. The replacement is roughly half that, and names the things agents must not copy in:

Why this tool is being called and how it fits the user's overall goal, in 15-25 words, third person. Used for product analytics. Never restate argument values, and never include credentials, tokens, URLs, file contents, or personal data. Example: "Inspecting a running browser session to diagnose a checkout automation that stopped responding partway through the flow."

Privacy

Intent is the only free-form text this integration captures, and a model writes it rather than it being user data copied verbatim. Two guards:

• $mcp_intent and $mcp_intent_source are added to the send allow-list explicitly. Everything else stays excluded — arguments, results, and error text are still dropped.
• Intent is truncated at 300 characters in beforeSend. 15-25 words fits comfortably; a client that ignores the instruction can't stream a payload or a prompt into an event property.

Verification

Ran the route locally against the staging project and checked the captured events:

call	$mcp_intent_source	intent length	params / response / error message captured
manage_profiles with context	context_parameter	109	none
manage_profiles with no context	none	—	none
manage_browsers with a 2,000-char context	context_parameter	300 (truncated)	none

No $exception events, and no property outside the allow-list. tsc --noEmit and prettier --check pass.

Note for whoever is watching the rollout

The rollout checks currently treat any $mcp_intent as a leak and page RED on it, since capture was off. That threshold needs to flip to expecting intent on tool calls when this deploys.

---

Note

Medium Risk
Adds agent-written free-form text to analytics and changes every tool schema in tools/list; rollout alerts must expect $mcp_intent instead of treating it as a leak.

Overview
Enables MCP tool-call intent analytics so product can see why agents invoke tools, not only which tools fail or go unused.

PostHog instrumentation turns on SDK context capture with a custom, shorter context argument description (replaces the default that bloats every tools/list). Each advertised tool gains a required context field in the schema; values flow to $mcp_intent and $mcp_intent_source after those properties are added to the send allow-list. Calls without context still run; they simply omit intent.

Privacy guardrails stay tight: call payloads and error text remain excluded. beforeSend now truncates intent strings to 300 characters so oversized agent text cannot flood event properties.

^{Reviewed by Cursor Bugbot for commit 866f1b4. Bugbot is set up for automated code reviews on this repo. Configure here.}

One discrepancy with PostHog's docs

The docs say the SDK "strips that argument before your handler runs, so your tool implementation never sees it." In 0.10.1 it doesn't — context is excluded from the captured parameters, but it's still on the request handed to the tool. Our handlers are unaffected because every tool parses arguments through a plain z.object, which drops unknown keys, and no tool forwards raw arguments to the Kernel API. Worth knowing before anyone adds a .passthrough() schema or a tool that spreads its arguments into a request.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
mcp	Ready	Preview	Jul 31, 2026 8:51pm

[dcruzeneil2]

Approving.

Traced the full path — schema injection on tools/list, context extraction on tools/call, the beforeSend allow-list — and it does exactly what it says:

• context is required only in the advertised schema, so clients that omit it run unchanged.
• Privacy boundary stays fail-closed: only $mcp_intent and $mcp_intent_source added to the allow-list, payloads and error text still excluded, 300-char truncation bounds non-compliant clients.
• context can't reach the Kernel API — the SDK strips it before the handler, and the strict z.object schemas drop unknown keys as a second layer.

One thing before deploy: update the rollout-monitoring jobs so $mcp_intent on tool calls reads as expected instead of paging as a leak — ideally flipping the check so missing intent becomes the anomaly.

[masnwilliams]

Thanks — monitoring is already updated, and one correction on the second point.

Monitoring: done before this review landed. The t+72h rollout check no longer treats $mcp_intent as leakage. It now resolves this PR's merge state first, adds $mcp_intent/$mcp_intent_source to the property-hygiene allow-list, and flips intent into a coverage metric — yellow under 25% (agents ignoring the argument would mean this bought little), red on any intent over the 300-char cap. It also samples intent text on failed calls, which is the specific thing it was turned on to answer.

Correction: the SDK does not strip context before the handler, despite the docs saying so. In the pinned 0.10.1, context is only excluded from the captured parameters (buildCapturedMcpArguments); the downstream request still carries it — the only clone the instrumentation makes is cloneRequestWithoutConversationId, which is a different argument.

The conclusion holds, but on one layer rather than two: the tool schemas are plain z.object, not .strict(), so zod drops the unknown key at parse time and the handler never sees it. Verified — z.object({action: z.string()}).parse({action, context}) returns {action}, and no tool forwards raw arguments to the Kernel API.

Practical consequence: if anyone later adds a .passthrough() schema or a tool that spreads its arguments into a request, context would ride along. Noted in the PR body.