Run a Bot's tools on the server, not in somebody's browser - #31
Merged
Conversation
The loop ran in the browser: every granted MCP tool was registered with `useFrontendTool` and its handler posted back to `/api/plugins/call`. That made a browser a hard requirement for a Bot to do anything, which rules out an embedded widget, an unattended run, and any surface that is not our own app. `BuiltInAgent` takes `tools` directly, so the agent is handed what the Bot may call and executes it itself. Governance does not move with it. The tools are NOT raw `mcpServers`, which would let the agent reach a vendor directly and walk around everything: each definition executes through `pluginStore.callTool`, which checks the grant, evaluates the policy and writes the audit row before anything leaves the process. A refusal comes back as the tool's result rather than as a thrown error, so the run continues and the person is told what was blocked. Tested against a real MCP server (`@copilotkit/aimock`) rather than a stubbed fetch: a stub passes whether or not we understood the protocol.
They execute in the runtime now, so a second registration in the surface would offer the model every tool twice. The result rendering survives: `forDisplay` moves to a shared module and the transcript's fallback draws it, which is no longer the exception but the ordinary path for every MCP call.
Two things found by driving it in a browser, neither of which any gate would have caught. A run stopped after one step. The AI SDK sets no `stopWhen` unless `maxSteps` is given, so a Bot called its tool, the result arrived, and the run ended before the model could say what it found: the person saw their own question and nothing else, while the audit trail recorded a successful call. Only set when there are tools, and capped at eight. The tool line read `mcp__notes__search_notes`. That prefix exists so a tool name is unique across every server a Bot holds, which is not the reader's problem. It now reads "Search notes", with the server beside it, and the server dropped when the action already names it. The server's own words stay behind the disclosure. Also here: a mock knowledge MCP server standing in for a customer's Notion, so slice 1 can answer with nothing real connected. It is `@copilotkit/aimock`, so a Bot talks to the same protocol implementation the tests do.
Three defects on the path the server-side tool loop now takes, found by driving it. A refused call was described from the browser fields. Those are present on an MCP context and deliberately empty, so that a rule written about a page evaluates to false against a tool call rather than being unevaluable, and `describeRefusal` branched on the file field being there rather than on it having a path. Every refused tool call therefore read "the file is blocked by the rule", naming a workspace it never went near and a path that was not there. It now names the tool and the server, and the file branch asks for a path. A tool returns a string and the runtime carries it in a message as JSON, so what reached the screen was that string encoded: the whole thing in quotes, every quote inside it escaped. A refusal read `"... blocked by the rule \"mcp.server == \\"notes\\"\"."`. That also broke more than the look, because the refusal marker is matched against the start of the text and an encoded string starts with a quote. And there was no marker to match. Refusals came back as the policy message alone, so the transcript had only the wording to go on, and drew a refusal as an ordinary result: the Bot said it had been blocked while the line above it looked like a successful call. `REFUSAL_MARKER` now leads a refused answer. The model is told it too, which is right for the model. The reader is not, because the line already says "Blocked" and three sayings of the same thing in one sentence is two too many. The marker is written out at both ends rather than shared, like a status code, because it crosses a network. The integration test asserts it rather than only that the reason mentions policy, which is what let it be missing.
davidmckayv
requested review from
MikeRyanDev,
guidovizoso and
tylerslaton
as code owners
August 20, 2026 03:39
This was referenced Aug 20, 2026
guidovizoso
approved these changes
Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Every MCP tool a Bot could call was registered in the browser with
useFrontendTooland executed by posting back to/api/plugins/call. A Bot could therefore do nothing at all unless a person had a tab open on it. That rules out an embedded widget, any surface that is not our own app, and every run nobody is watching.The loop now runs in the runtime.
server/src/plugins/tools.tshands each Bot the tools it holds a grant for, and the 209-line frontend registration is deleted.Governance does not move with it.
callToolis still the only path to a vendor: it checks the grant, evaluates the policy, writes the audit row, and only then calls out. PassingmcpServersto the agent instead would have let it reach a vendor directly and walk around all three, which is why the tools are handed over as executables rather than as server addresses.Lifted from
CopilotKit/openkai, which did this first, as four cherry-picks rather than a hand-port so the authorship and the reasoning survive.Where it runs
grantedToolsis built per run from the store and holds nothing between calls.Boundary and audit
mcp.call_rejected; successes writemcp.call_succeeded.Known regression, closing next
Only built-in Bots get tools in this PR. A remote AG-UI Bot runs its own loop and has to call back in, and there is no authenticated way for it to do that yet:
/api/agent-tools/calldoes not exist. So a remote Bot has no MCP tools between this PR and the next one, where the per-agent callback token lands. Named here rather than discovered.Three defects found by driving it
The last commit fixes these, each with a test:
This deployment's policy does not allow that: the file is blocked by the rule .... A tool call carries the browser fields present and empty, deliberately, so a rule written about a page evaluates to false rather than being unevaluable;describeRefusalbranched on the file field existing rather than on it having a path. Pre-existing onmain, not introduced here.\"mcp.server == \\\"notes\\\"\".Proof
Driven in Chrome against a real MCP server (
bun run mock:knowledge, added here), registered through/admin/plugins, granted to the Knowledge Bot.search_notes, answered with the policy and a working link, and the tool line read "Search notes" rather than the identifier the model was offered.mcp.server == "notes"on the deny list: the Bot was stopped, the line read Blocked, the reason named the rule, and the Bot told the person it had been restricted rather than inventing an answer.mcp.call_succeeded,notes/search_notes, botknowledge, actordev-local-user, effectwrite,{"allowed": true, "rule": "true", "source": "allow", "carriedOut": true}.format:check,lint,typecheckclean. 665 pass, 5 skip, 0 fail across 77 files.