diff --git a/ts/docs/plans/copilot-typed-actions/DESIGN.md b/ts/docs/plans/copilot-typed-actions/DESIGN.md new file mode 100644 index 0000000000..81484e2b66 --- /dev/null +++ b/ts/docs/plans/copilot-typed-actions/DESIGN.md @@ -0,0 +1,252 @@ + + +# Letting Copilot run TypeAgent actions directly + +**Status:** first version built and working. The +[open questions](#open-questions-for-discussion) are genuinely open — that is +what this document is for. +**Area:** `ts/packages/copilot-plugin/` (MCP mode), with a shared helper in +`ts/packages/dispatcher/types/` also used by `ts/packages/commandExecutor/` + +## In one sentence + +If Copilot already knows exactly which TypeAgent action it wants to run, it can +now run it — instead of writing an English sentence and asking TypeAgent to +figure out what it meant. + +## Background: the two things in play + +**TypeAgent's job** is to turn human language into a typed action. You say +"play some jazz," and TypeAgent produces a structured action: +`player.playMusic({ target: { kind: "genre", genre: "jazz" } })`. It then runs +it. Translation is the valuable part, and TypeAgent is good at it. + +**Copilot's plugin** lets Copilot CLI reach TypeAgent. Until now it could only +hand over English. Copilot passed along the user's words, and TypeAgent +translated them. + +That works well when a human wrote the words. The problem shows up when nobody +did. + +## The problem, as a story + +Copilot is working through a multi-step task. It has just assembled a list of 20 +songs — pulled from a file, or from the results of earlier steps. Now it wants to +save them as a playlist. It already knows precisely what it wants: + +``` +player.createPlaylist({ + name: "Deep Focus", + songs: [ + { trackName: "Kind of Blue", artist: "Miles Davis" }, + { trackName: "Naima", artist: "John Coltrane" }, + ... 18 more + ] +}) +``` + +But the only door available took English. So Copilot had to write a sentence +describing all of it: + +> "create a playlist called Deep Focus with Kind of Blue by Miles Davis, Naima by +> John Coltrane, ..." + +TypeAgent then translated that sentence back into the action Copilot started +with. Two things go wrong: + +1. **It costs a model call** to do a translation that was not needed. Copilot + already had the answer. +2. **It can corrupt the data.** Twenty exact track and artist names have to + survive being flattened into a sentence and parsed back out. Long lists, + identifiers, and file paths are exactly the things that get dropped, + truncated, or subtly misspelled on that round trip. + +The deeper point: translation exists to interpret human intent. When there is no +human sentence — when a machine composed the step — there is nothing to +interpret. The translation step is doing no useful work, but it can still do +damage. + +## What we built + +Two new MCP tools, alongside the existing one. + +| Tool | What it does | +| --------------------------- | ----------------------------------------------------------------------- | +| `typeagent-processCommand` | **Unchanged, still the default.** Send English, TypeAgent translates it | +| `typeagent-executeAction` | Run one action directly by name and parameters | +| `typeagent-discoverActions` | Look up which actions exist and what parameters they take | + +`executeAction` hands the action straight to the dispatcher, which validates it +against the schema and runs it. No translation step in the middle. + +## The value + +Three things, and it is worth being precise about them: + +- **Fidelity.** A track list, file path, or record ID that Copilot already has + reaches the action intact. It never gets flattened into prose and parsed back. +- **Determinism.** The same request runs the same action every time. There is no + interpretation step that could land somewhere different. +- **Composability.** Copilot can use TypeAgent actions as steps inside a larger + plan it is executing, which is the direction agent work is going. + +### What it is _not_ + +**It is not faster for normal user requests, and we should not present it that +way.** This matters, because it is the obvious thing to assume. + +TypeAgent remembers translations. Once it has seen "play some jazz," it +recognizes that phrase again and produces the action with **no model call at +all**. Nothing here beats that. An early version of this work claimed it avoided +"paying for translation on every request" — that was simply wrong, and the +correction is why the framing is narrower now. + +So: for anything a person typed, the existing path is as fast or faster. The new +path is for the case where no person typed anything. + +## When each path is used + +The question is not _what the request does_. It is **where the request came +from**. + +> Words a human typed → translation. +> Structure Copilot already has → direct. + +| Situation | Path | +| --------------------------------------------------------------- | ------------------------ | +| User types "play some jazz music" | `processCommand` | +| User types anything starting with `learn:`, `dev:` or `record:` | `processCommand`, always | +| Request is conversational, vague, or multi-step | `processCommand` | +| User typed it, and Copilot cannot name the action | `processCommand` | +| Copilot planned this step itself | `executeAction` | +| Copilot already knows the action and parameters | `executeAction` | + +### Two scenarios, side by side + +**Scenario A — a person asks for something.** +The user types "tidy up my desktop and put on some focus music." This goes to +`processCommand`. A human wrote it, it covers several steps, and TypeAgent is +better at breaking it apart than Copilot guessing. If the user has said something +like this before, it resolves with no model call. + +**Scenario B — Copilot composed the step.** +Copilot is six steps into a task it planned. Step four gathers a set of tracks +from earlier results and saves them as a playlist. Nobody ever said that step out +loud, and the track list came from data rather than from a sentence. This goes to +`executeAction`, carrying all twenty entries exactly as gathered. + +### The mistake to avoid + +The tempting wrong move is: user asks for something → Copilot looks up the action +with `discoverActions` → Copilot runs it with `executeAction`. + +That is the **most expensive** option available. It spends extra Copilot turns to +avoid a translation that would likely have been free. The instructions we ship to +the model say this explicitly. Discovery is for looking up something Copilot will +use repeatedly — not for answering a sentence the user already typed. + +## Cost, concretely + +Copilot takes its turn either way, so _choosing_ the direct tool costs nothing +extra. Looking things up first does. + +| What happens | Copilot turns | TypeAgent model calls | +| -------------------------------------- | ------------- | --------------------- | +| `processCommand`, phrase already known | 1 | 0 | +| `processCommand`, new phrase | 1 | 1 | +| `executeAction`, action already known | 1 | 0 | +| `discoverActions` then `executeAction` | 2-3 | 0 | + +Note the last row is the worst one. That is why the guidance pushes against it. + +Also worth knowing: when Copilot runs several actions in a row, it looks up the +contract **once** and reuses it. The lookup cost is paid per task, not per +action. + +## Does this undermine TypeAgent's purpose? + +A fair question, and worth addressing head-on, because "skip the translation" +sounds like "skip TypeAgent." + +It is not. Two reasons. + +**First, it skips translation, not execution.** The action still goes through the +dispatcher. Permission checks, multi-step action chaining, linking results to +known entities, recording to memory, cancellation, and confirmation prompts all +work exactly as they do for a normal request. + +**Second, it feeds the translation cache rather than starving it.** When the +user's words do match the action one-for-one, Copilot passes those words along +too. TypeAgent learns the phrasing, so next time that sentence resolves with no +model call. The direct path makes the normal path smarter over time. + +## Will this scale? + +MCP has a known scaling trap: if you expose 400 capabilities as 400 tools, all +400 descriptions sit in the model's context on **every single turn**, whether +relevant or not. TypeAgent has roughly 418 actions today and is designed for far +more, so that approach would not survive. + +We avoided it. **We added two tools, not one per action.** The actions live +_behind_ a tool rather than _as_ tools. Nothing about the action list is in +context until something asks for it, and that stays true at any number of +actions. + +Lookup is also layered — agents, then one agent's actions, then one action's +parameters — and there is deliberately no "list everything" call. + +## What is missing today + +Honest list of gaps. None are blocking, all are documented in the code. + +| Gap | What it means | +| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| No memory of earlier turns | "Play it again" cannot work on the direct path — pass real values | +| No automatic retry via reasoning | If an action fails, the error comes back rather than being retried | +| Cannot answer follow-up questions | If an agent asks the user to pick something, the tool reports the question instead of pretending it finished | +| Lookup results are not paginated | Fine at today's size; the agent list will outgrow it before any single agent's action list does | +| A new connection per call | Each call opens and closes its own connection — already true of the existing path, but it is the main slowdown when running several actions | + +We also fixed three real bugs along the way, independent of the new tools: + +- Lookup could advertise actions that would then refuse to run, because it + checked the wrong "is this enabled" flag. +- The other MCP server filtered enabled actions at the wrong level. +- The other MCP server corrupted any phrase containing an apostrophe. + +## Open questions for discussion + +These are the ones worth arguing about, and the reason this is a discussion +document rather than a finished decision. + +1. **Is the agent-composed case common enough to justify this?** + The whole value rests on Copilot running TypeAgent actions as steps in plans + it made itself. If that stays rare, this is two tools of surface area for + little return. This is the central question. + +2. **Will the model follow the guidance?** + "Prefer the English path for user requests" is written instruction, not an + enforced rule. If the model over-uses the direct path, TypeAgent sees fewer + phrases, learns less, and the fast path slowly gets worse. We should measure + the ratio rather than assume. + +3. **Should we cache lookups on the server?** + It would cut latency without putting anything extra in the model's context. + Worth doing if chained actions become common. + +4. **Should the direct path get memory of earlier turns?** + It would close the most surprising gap. But it needs an answer to "what does + the previous turn mean when the caller is a machine, not a person?" + +5. **Do we need a connection pool?** + Only matters if chains are common — but it is the dominant cost if they are. + +## Related + +- `ts/packages/copilot-plugin/README.md` — the operational detail: exact routing + rules, cost table, scaling notes. +- [Dispatcher](../../architecture/core/dispatcher.md) — the translation and + execution pipeline this sits in front of. +- [Action grammar](../../architecture/core/actionGrammar.md) — the cache that + makes the normal path fast. diff --git a/ts/packages/commandExecutor/src/commandServer.ts b/ts/packages/commandExecutor/src/commandServer.ts index 4ac6245354..ff588a4dc3 100644 --- a/ts/packages/commandExecutor/src/commandServer.ts +++ b/ts/packages/commandExecutor/src/commandServer.ts @@ -20,6 +20,12 @@ import type { } from "@typeagent/dispatcher-types"; import type { Dispatcher } from "@typeagent/dispatcher-types"; import { awaitCommand } from "@typeagent/dispatcher-types"; +import { + buildActionCommand, + filterActiveAgentSchemas, + findActionSubSchema, + getAgentActionNames, +} from "@typeagent/dispatcher-types/helpers/actionDispatch"; import { DisplayAppendMode } from "@typeagent/agent-sdk"; import { getStructuredFallback, @@ -874,20 +880,12 @@ export class CommandServer { ); } - // Filter to active agents when connected + // Filter to the sub-schemas this session actually has enabled. let visible = agents; if (this.dispatcher) { try { const status = await this.dispatcher.getStatus(); - const activeNames = new Set( - status.agents - .filter((a) => a.active) - .map((a) => a.name.toLowerCase()), - ); - const filtered = agents.filter((a) => - activeNames.has(a.name.toLowerCase()), - ); - if (filtered.length > 0) visible = filtered; + visible = filterActiveAgentSchemas(agents, status); } catch { // Use unfiltered list } @@ -913,18 +911,13 @@ export class CommandServer { if (request.actionName) { // Level 3 — full TypeScript source for one specific action - const needle = request.actionName.toLowerCase(); - const subSchema = agent.subSchemas.find((s) => - s.actions.some((a) => a.name.toLowerCase() === needle), - ); - if (!subSchema) { - const allActions = agent.subSchemas - .flatMap((s) => s.actions.map((a) => a.name)) - .join(", "); + const found = findActionSubSchema(agent, request.actionName); + if (!found) { return toolResult( - `Action '${request.actionName}' not found in agent '${agent.name}'.\n\nAvailable actions: ${allActions}`, + `Action '${request.actionName}' not found in agent '${agent.name}'.\n\nAvailable actions: ${getAgentActionNames(agent).join(", ")}`, ); } + const { subSchema } = found; if (!subSchema.schemaText) { return toolResult( `TypeScript schema not available for action '${request.actionName}'.`, @@ -1041,18 +1034,19 @@ export class CommandServer { parameters = flowParams || {}; } - const paramStr = - parameters && Object.keys(parameters).length > 0 - ? `--parameters '${JSON.stringify(parameters).replaceAll("'", "\\u0027")}'` - : ""; - - const nlStr = request.naturalLanguage - ? `--naturalLanguage '${request.naturalLanguage.replaceAll("'", "\\u0027")}'` - : ""; - - const actionCommand = - `@action ${schemaName} ${actionName} ${paramStr} ${nlStr}`.trim(); - + let actionCommand: string; + try { + actionCommand = buildActionCommand({ + schemaName, + actionName, + parameters, + naturalLanguage: request.naturalLanguage, + }); + } catch (error) { + return toolResult( + error instanceof Error ? error.message : String(error), + ); + } this.logger.log(`Dispatching: ${actionCommand}`); this.responseCollector.messages = []; this.responseCollector.rawData = undefined; diff --git a/ts/packages/copilot-plugin/README.md b/ts/packages/copilot-plugin/README.md index 268b699cac..82e1c3ea8d 100644 --- a/ts/packages/copilot-plugin/README.md +++ b/ts/packages/copilot-plugin/README.md @@ -294,16 +294,180 @@ different MCP tool catalog. ### Direct Mode (default) -The hook connects directly to TypeAgent over WebSocket. When TypeAgent recognizes and handles the request, the hook returns `{ handled: true, responseContent: "..." }` — Copilot skips the LLM entirely. +The hook connects directly to TypeAgent over WebSocket. When TypeAgent recognizes and handles the request, the hook returns `{ handled: true, responseContent: "..." }` — Copilot skips the LLM entirely. TypeAgent still translates the prompt; what is skipped is Copilot's model. - **Pros:** Fast (~1-3s), no LLM tokens consumed - **Cons:** No streaming output, response is returned all at once ### MCP Mode -The hook injects a directive into the prompt context, instructing the LLM to call the `typeagent-processCommand` MCP tool. TypeAgent's MCP server streams progress notifications to the CLI timeline. - -- **Pros:** Streaming output visible during processing, LLM-formatted responses +The hook injects a directive into the prompt context, instructing the LLM to +call TypeAgent's MCP tools. TypeAgent's MCP server streams progress +notifications to the CLI timeline. + +There are two ways in. + +**`typeagent-processCommand` is the default.** It sends the user's own words and +lets TypeAgent translate them. This is the right choice for anything a person +phrased, not just conversational or multi-step requests: TypeAgent caches +translations, so a phrase it has seen before resolves with no model call at all. +It is also the only path that honors `learn:` / `dev:` / `record:` directives. + +**The typed-action shortcut exists for actions Copilot already holds.** In one +line: _a caller that already knows the action it wants can run it directly, +instead of writing a sentence for TypeAgent to translate back into the action it +started with._ + +The case that motivates it is agentic. The MCP servers are registered +independently of the prompt hook, so their tools stay in Copilot's catalog on +every turn of a multi-step loop — including steps Copilot planned itself, where +no user ever said anything. Previously the only way to run such a step was to +describe it in prose. That round trip costs a model call, and it can lose or +distort a parameter that was never ambiguous to begin with. Two tools cover it: + +- `typeagent-executeAction` runs a single typed action by `schemaName`, + `actionName` and `parameters`, skipping translation and TypeAgent reasoning. +- `typeagent-discoverActions` supplies the contract when it is not already + known. It only reports agents and schemas whose actions the session has + enabled, so anything it lists is runnable. + +So what it buys is **determinism and fidelity for machine-composed steps** — a +structure the caller already holds reaches the dispatcher intact, schema +validated, with no interpretation step in between. What it does **not** buy is +speed on ordinary user requests, and it should not be sold as though it does; a +warm translation cache already answers those with no model call. + +#### Which path a request takes + +The deciding question is not what the request does, it is **where the request +came from**. Words the user typed go to translation. Structure Copilot is +already holding goes to the shortcut. + +| The situation | Path | Why | +| ----------------------------------------------------------------- | ----------------------------------- | --------------------------------------------------------- | +| User types `play some jazz music` | `processCommand` | The user supplied the phrasing; the cache likely knows it | +| User types something with a `learn:`, `dev:` or `record:` prefix | `processCommand`, always | Directives only work through translation | +| User asks something conversational, ambiguous, or multi-step | `processCommand` | The dispatcher is better at this than Copilot guessing | +| User phrased it and Copilot cannot name the action | `processCommand`, **not** discovery | Translating is cheaper than a discovery round-trip | +| Copilot composed the action itself as a step of a task it planned | `executeAction` | There is no user sentence to translate | +| Copilot already holds the schema, action and parameters | `executeAction` | The lookup is already paid for | + +A worked example. The user says "tidy up my desktop and put on some focus +music." That sentence goes to `processCommand` — the user wrote it, it is +multi-step, and TypeAgent decomposes it. Now contrast: Copilot is midway through +a longer task it planned itself, has already fetched the `player` contract for +an earlier step, and now needs to start a specific playlist as step four of six. +Nobody said that step out loud. Copilot calls `executeAction` with the schema +and parameters it is already holding, rather than composing an English sentence +for TypeAgent to parse back into the structure it just had. + +The failure mode worth naming: seeing a user request, calling +`discoverActions` to find the matching action, then calling `executeAction`. +That is the most expensive route available and the guidance tells the model not +to do it. Discovery is for contracts that get reused, not for answering a +sentence the user already phrased. + +#### When the shortcut is actually cheaper + +Copilot's model runs either way — the hook has already given it the turn — so +choosing the shortcut costs no extra inference. A discovery round-trip does. + +| Situation | Copilot turns | TypeAgent model calls | +| -------------------------------------- | ------------- | --------------------- | +| `processCommand`, phrase in cache | 1 | 0 | +| `processCommand`, phrase not in cache | 1 | 1 | +| `executeAction`, contract known | 1 | 0 | +| `discoverActions` then `executeAction` | 2-3 | 0 | + +A warm cache is unbeatable, so the rule is: when the user supplied the phrasing +and the contract is unknown, translating is cheaper than discovering — and the +injected guidance says exactly that. The shortcut earns its keep when the +contract is already in hand, or when there was no user phrasing to begin with. + +Passing `naturalLanguage` populates the translation cache through the same +explanation pipeline a normal request uses, so a request served by the shortcut +still teaches TypeAgent the phrasing for next time. + +#### How discovery scales + +MCP's usual scaling failure is catalog bloat: expose N capabilities as N tools +and every tool definition sits in the model's context on every turn, whether or +not it is relevant. TypeAgent's action space is far too large for that. So the +action space lives _behind_ a tool rather than _as_ tools — this integration +adds exactly two, and that stays true whether TypeAgent exposes hundreds of +actions or many thousands. Discovery output is a tool result, so it enters +context only when something asks for it. + +Discovery is tiered for the same reason, and there is deliberately no "list +every action" call — `agentName` is required before any actions come back: + +| Call | Returns | +| -------------------------- | ------------------------------------------------------- | +| no arguments | one line per enabled agent | +| `agentName` | that agent's sub-schemas and actions, with descriptions | +| `agentName` + `actionName` | one action's TypeScript parameters | + +Results are filtered to enabled schemas, so a session sees its active subset +rather than everything installed. + +Neither tier is paginated, which is fine at present scale — the largest agent in +this repo exposes on the order of 80 actions — but it is worth knowing which +tier gives first. The per-agent listing grows with one agent's action count, +while the agent list grows with the number of enabled agents; the latter is the +one to watch, since a deployment is more likely to accumulate many agents than +to put many hundreds of actions on a single one. Either would need paging before +it reached that point. + +#### Why discovery is live rather than prefetched + +Discovery deliberately re-reads dispatcher status on every call instead of being +snapshotted once at startup. Which agents are enabled changes during a session, +so a cached catalog would eventually offer actions that `@action` then refuses — +the same class of mismatch the `actionActive` flag exists to prevent. + +Prefetching would also have to live somewhere. Anything handed over at handshake +for the model to keep in mind ends up in the tool catalog or system context, +which is per-turn cost — the bloat this shape is built to avoid, just relocated. + +That cost is better avoided than amortized, and for chained work it already is: +discovery is paid per chain, not per call. One lookup for an agent, then any +number of `executeAction` calls reusing that contract from context, which is +what the guidance means by reusing a contract and never re-requesting one +already held. + +If discovery ever does become a bottleneck, the useful lever is latency rather +than context: every tool call currently opens and closes its own dispatcher +connection, which costs more across a chain than re-reading the catalog does. + +#### Not the same as Direct Mode + +The two are independent and pull in opposite directions. Direct Mode skips +**Copilot's** LLM and lets TypeAgent translate; the typed-action shortcut skips +**TypeAgent's** translation and lets Copilot's LLM choose the action. The +shortcut lives entirely inside MCP mode. + +#### What the shortcut does and does not skip + +It skips translation, not execution. It runs the dispatcher's `@action` +command, which uses the same `executeActions` engine as an ordinary request, so +enabled-action gating, chained multi-step actions, result-entity resolution, +action results recorded to memory, cancellation, and per-agent confirmation all +behave identically. Two differences are worth knowing: + +- No prior-turn entity context is supplied, so references like "play it again" + cannot be resolved — pass concrete parameters instead. +- Schemas that opt into `errorReasoning` are not retried through TypeAgent + reasoning on failure; the error is returned to the caller, which is the + reasoner in this arrangement. + +Neither tool can answer an agent's follow-up question: this MCP client has no +return path for a choice or form. When an agent asks one, the tool reports the +pending question instead of reporting success, so the user can answer it in the +TypeAgent shell. + +- **Pros:** Streaming output visible during processing, LLM-formatted responses; + the typed-action shortcut avoids a translation round-trip for actions Copilot + already holds or composed itself - **Cons:** Slower (~3-5s), consumes LLM tokens ### Dev Mode @@ -387,18 +551,20 @@ The plugin stores config at `%USERPROFILE%\.typeagent-copilot\config.json` (Wind The plugin starts three logical MCP servers from the same bundled entry point and single-file release executable: -| Server | Tool | Description | -| --------------------- | -------------------------- | --------------------------------------------------------------------------------------- | -| `typeagent` | `typeagent-processCommand` | Send a command to the TypeAgent agent-server | -| `typeagent` | `typeagent-listAgents` | List available TypeAgent agents | -| `typeagent` | `typeagent-getStatus` | Get TypeAgent server status | -| `typeagent-workspace` | `read` | Read bounded text under approved workspace roots | -| `typeagent-workspace` | `glob` | Find bounded, deterministically ordered workspace files | -| `typeagent-workspace` | `grep` | Search bounded workspace text | -| `typeagent-workspace` | `fetch` | Fetch bounded public HTTP(S) text without ambient credentials or private-network access | -| `typeagent-macros` | `list_macros` | List and search reusable captured procedures | -| `typeagent-macros` | `run_macro` | Replay an approved macro or return an agent-runner handoff | -| `typeagent-macros` | lifecycle tools | Capture-derived draft validation, approval, disablement, and candidate submission | +| Server | Tool | Description | +| --------------------- | --------------------------- | --------------------------------------------------------------------------------------- | +| `typeagent` | `typeagent-processCommand` | Default path: send the user's words for TypeAgent to translate | +| `typeagent` | `typeagent-discoverActions` | List the enabled agents, their actions, and one action's TypeScript contract | +| `typeagent` | `typeagent-executeAction` | Run a typed action Copilot already holds, skipping translation | +| `typeagent` | `typeagent-listAgents` | List available TypeAgent agents | +| `typeagent` | `typeagent-getStatus` | Get TypeAgent server status | +| `typeagent-workspace` | `read` | Read bounded text under approved workspace roots | +| `typeagent-workspace` | `glob` | Find bounded, deterministically ordered workspace files | +| `typeagent-workspace` | `grep` | Search bounded workspace text | +| `typeagent-workspace` | `fetch` | Fetch bounded public HTTP(S) text without ambient credentials or private-network access | +| `typeagent-macros` | `list_macros` | List and search reusable captured procedures | +| `typeagent-macros` | `run_macro` | Replay an approved macro or return an agent-runner handoff | +| `typeagent-macros` | lifecycle tools | Capture-derived draft validation, approval, disablement, and candidate submission | Workspace tools are available in direct, MCP, and dev modes. In bypass mode they remain discoverable because Copilot fixes the MCP catalog when the session diff --git a/ts/packages/copilot-plugin/agents/typeagent.agent.md b/ts/packages/copilot-plugin/agents/typeagent.agent.md index 54c958482d..3831abd3ba 100644 --- a/ts/packages/copilot-plugin/agents/typeagent.agent.md +++ b/ts/packages/copilot-plugin/agents/typeagent.agent.md @@ -3,6 +3,8 @@ name: TypeAgent description: Delegates requests to TypeAgent for calendar, email, music, browser, and other domain-specific actions tools: - typeagent-processCommand + - typeagent-discoverActions + - typeagent-executeAction - typeagent-listAgents - typeagent-getStatus infer: true @@ -11,10 +13,30 @@ userInvocable: true You are a bridge to TypeAgent. When the user asks you to perform an action (schedule meetings, send emails, play music, control browser, manage lists, etc.), -use the typeagent-processCommand tool to delegate the request. +delegate the request to TypeAgent. Do not attempt to handle action requests +yourself. If TypeAgent returns an error or unknown action, inform the user clearly. -Do not attempt to handle action requests yourself. Always delegate to TypeAgent. -If TypeAgent returns an error or unknown action, inform the user clearly. +There are two ways to delegate: -For multi-step tasks, use typeagent-listAgents first to discover available agents -and their capabilities, then use typeagent-processCommand for each step. +- `typeagent-processCommand` sends the user's own words and lets TypeAgent + translate them. This is the default. TypeAgent caches translations, so a + phrase it has seen costs no model call — cheaper than looking the action up + yourself. Use it for conversational, ambiguous or multi-step requests, and + always for prompts carrying a `learn:`, `dev:` or `record:` prefix (keep the + prefix exactly as written). +- `typeagent-executeAction` runs one typed action by `schemaName`, `actionName` + and `parameters`, skipping translation. Prefer it when you already know the + contract, or when you composed the action yourself as a step of a larger task + and there is no user phrasing to translate. + +Use `typeagent-discoverActions` to learn a contract: call it with no arguments +to list enabled agents, with `agentName` to list an agent's schemas and actions, +and with `agentName` + `actionName` to get the action's TypeScript parameters. +Do not discover a contract just to satisfy a request the user phrased — sending +their words to `typeagent-processCommand` is cheaper than the round-trip. Never +re-request a contract you already have in this conversation. + +When the user's request maps exactly to the action you ran, pass their words +verbatim as `naturalLanguage` so TypeAgent learns the phrasing and can handle it +next time with no model call. Omit it if you paraphrased, inferred the action, +or ran it as one step of a larger task. diff --git a/ts/packages/copilot-plugin/src/hooks/hook-mcp-redirect.ts b/ts/packages/copilot-plugin/src/hooks/hook-mcp-redirect.ts index e98aa0084a..ebf2ec197b 100644 --- a/ts/packages/copilot-plugin/src/hooks/hook-mcp-redirect.ts +++ b/ts/packages/copilot-plugin/src/hooks/hook-mcp-redirect.ts @@ -52,13 +52,51 @@ function getSpecialPrefixGuidance(prompt: string): string | undefined { "Example: If user says 'learn: create a playlist from top songs',", " pass 'learn: create a playlist from top songs' — NOT just 'create a playlist from top songs'.", "Stripping the prefix will cause the recording to fail.", + "Do NOT use typeagent-executeAction for this request — the directive only works through typeagent-processCommand.", "", ].join("\n"); } +/** + * Tell the model when running a typed action itself beats paying for + * translation. Unrelated to the plugin's `direct` routing mode, which sends + * the raw prompt to TypeAgent and skips Copilot's LLM instead. + * + * The cost model matters. Copilot's turn happens either way, so choosing the + * shortcut is not an extra inference - but a discovery round-trip is. When the + * user supplied the phrasing and the contract is not already known, letting + * TypeAgent translate is cheaper: a cache hit costs no model call at all, + * while discover-then-execute costs two or three. The shortcut earns its keep + * when the contract is already in hand, or when there is no user phrasing to + * translate because the model composed the action itself. + */ +function getTypedActionGuidance(hasRecordingDirective: boolean): string { + if (hasRecordingDirective) { + return ""; + } + return [ + "", + "[TYPED ACTION SHORTCUT]", + "Default to typeagent-processCommand with the user's exact words. TypeAgent caches translations,", + "so a phrase it has seen costs no model call — cheaper than any tool round-trip you could make.", + "Prefer typeagent-executeAction only when one of these holds:", + "- you already know the schemaName, actionName and parameters, so no lookup is needed; or", + "- you composed this action yourself as a step of a larger task, so there is no user phrasing to translate.", + "Do NOT call typeagent-discoverActions just to satisfy a request the user phrased — translating it is cheaper", + "than discovering it. Use discovery when you will reuse the contract, and never re-request one you already have.", + "When the user's request maps exactly to the action you ran, pass their words verbatim as naturalLanguage —", + "TypeAgent learns the phrasing and handles it next time without any model call. Omit it if you paraphrased,", + "inferred the action, or ran it as one step of a larger task.", + "When the request is conversational, multi-step, ambiguous, or you cannot name the action, use typeagent-processCommand.", + ].join("\n"); +} + export function handleMcpRedirect(input: HookInput): HookOutput { const psGuidance = getPowerShellSessionGuidance() ?? ""; - const prefixGuidance = getSpecialPrefixGuidance(input.prompt) ?? ""; + const prefixGuidance = getSpecialPrefixGuidance(input.prompt); + const typedActionGuidance = getTypedActionGuidance( + prefixGuidance !== undefined, + ); return { modifiedPrompt: input.prompt, @@ -66,14 +104,16 @@ export function handleMcpRedirect(input: HookInput): HookOutput { "[SYSTEM HOOK DIRECTIVE — MANDATORY]", "A pre-processing hook has classified this request as a TypeAgent action.", "TypeAgent is the ONLY system that can fulfill this request.", - "You MUST call the typeagent-processCommand tool with the user's exact request as the 'command' parameter.", + "You MUST fulfill it through TypeAgent's MCP tools: typeagent-processCommand with the user's exact request as the 'command' parameter,", + "or typeagent-executeAction when you already know the exact typed action to run (see below).", "Do NOT use bash, file tools, web search, or any other tool — they cannot handle this type of request.", "Do NOT attempt to answer or fulfill the request yourself.", "Do NOT add any reasoning or commentary before calling the tool.", - "Simply call typeagent-processCommand immediately, then present the COMPLETE result to the user.", + "Simply call the tool immediately, then present the COMPLETE result to the user.", "CRITICAL: Display the tool result in FULL — do NOT summarize, truncate, or paraphrase it.", "The tool result is the authoritative response. Show it exactly as returned.", - prefixGuidance, + prefixGuidance ?? "", + typedActionGuidance, psGuidance, ].join("\n"), }; diff --git a/ts/packages/copilot-plugin/src/mcp/agentServer.ts b/ts/packages/copilot-plugin/src/mcp/agentServer.ts new file mode 100644 index 0000000000..745ac46fb7 --- /dev/null +++ b/ts/packages/copilot-plugin/src/mcp/agentServer.ts @@ -0,0 +1,755 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +/** + * TypeAgent agent-server MCP tools for Copilot CLI. + * + * Two ways in: + * - `typeagent-processCommand` sends the user's own words and lets TypeAgent + * translate them. Best for conversational or multi-step requests, and the + * only path that honors recording directives ("learn:", "dev:", "record:"). + * - `typeagent-discoverActions` + `typeagent-executeAction` run one typed + * action directly through the dispatcher's `@action` command. No + * translation and no TypeAgent reasoning, so the client decides exactly + * what runs. + * + * Every call connects to the agent server, runs, and disconnects, so a + * restarted agent server is picked up on the next tool call. + */ + +import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; +import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; +import { z } from "zod"; +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; +import type { + ClientIO, + Dispatcher, + IAgentMessage, +} from "@typeagent/agent-server-client"; +import type { + AgentSchemaInfo, + CommandResult, +} from "@typeagent/dispatcher-types"; +import { + buildActionCommand, + filterActiveAgentSchemas, + findActionSubSchema, + getAgentActionNames, + type DirectActionRequest, +} from "@typeagent/dispatcher-types/helpers/actionDispatch"; +import type { DisplayAppendMode } from "@typeagent/agent-sdk"; +import { + createClientIO, + connectToTypeAgent, + submitCancellableCommand, + TYPEAGENT_URL, +} from "../shared/typeagent-client.js"; +import { + extractMessageText, + extractRawData, +} from "../shared/message-formatter.js"; +import { getMode, type Mode } from "../shared/plugin-config.js"; + +// ── Helpers ────────────────────────────────────────────────────────────────── + +function stripAnsi(text: string): string { + return text.replace(/\x1b\[[0-9;]*m/g, ""); +} + +export function toolResult(text: string, rawData?: unknown): CallToolResult { + const result: CallToolResult = { content: [{ type: "text", text }] }; + if (rawData !== undefined) { + // MCP structuredContent must be a JSON object; wrap anything else. + result.structuredContent = + typeof rawData === "object" && + rawData !== null && + !Array.isArray(rawData) + ? (rawData as Record) + : ({ data: rawData } as Record); + } + return result; +} + +function toolError(text: string): CallToolResult { + return { isError: true, content: [{ type: "text", text }] }; +} + +/** + * Format a large result for display. Strips markdown formatting and wraps + * in a code fence so the CLI preserves newlines and structured layout. + */ +function formatLargeResult(response: string): CallToolResult { + const lines = response.split("\n").length; + if (lines > 5) { + // Strip markdown bold (**text**) — doesn't render inside code fences + const plain = response.replace(/\*\*([^*]+)\*\*/g, "$1"); + return toolResult("```\n" + plain + "\n```"); + } + return toolResult(response); +} + +export function log(message: string): void { + process.stderr.write( + `[${new Date().toISOString()}] [typeagent-mcp] ${message}\n`, + ); +} + +function errorMessage(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +// Type for the extra parameter passed to tool callbacks +export interface ToolExtra { + _meta?: { + progressToken?: string | number; + }; + sendNotification: (notification: { + method: string; + params: Record; + }) => Promise; + signal?: AbortSignal; +} + +export interface AgentToolDependencies { + connect: (clientIO: ClientIO) => Promise; + getMode: () => Mode; + log: (message: string) => void; +} + +const defaultDependencies: AgentToolDependencies = { + connect: connectToTypeAgent, + getMode, + log, +}; + +// ── Adapter ────────────────────────────────────────────────────────────────── + +/** + * Implements the agent-server tools. Split from the MCP registration below so + * the behavior can be tested without a transport. + */ +export class TypeAgentToolAdapter { + constructor( + private readonly dependencies: AgentToolDependencies = defaultDependencies, + ) {} + + async processCommand( + command: string, + extra?: ToolExtra, + ): Promise { + const disabled = this.getDisabledReason(); + if (disabled) { + return toolError(disabled); + } + this.dependencies.log(`processCommand: ${command}`); + + try { + const { result, collected } = await this.runCommand(command, extra); + if (result?.lastError) { + return toolResult(`Error: ${result.lastError}`); + } + if (result?.cancelled) { + // Before the output check: partial output from a cancelled + // request must not read as a completed one. + return toolResult( + cancelledText( + "TypeAgent request was cancelled.", + collected.messages, + ), + ); + } + if (collected.pendingPrompts.length > 0) { + return toolResult(pendingPromptText(collected.pendingPrompts)); + } + if (collected.messages.length > 0) { + return formatLargeResult(collected.messages.join("\n\n")); + } + return toolResult(`Successfully executed: ${command}`); + } catch (error) { + const message = errorMessage(error); + this.dependencies.log(`processCommand error: ${message}`); + return toolResult(`Error executing command: ${message}`); + } + } + + /** + * Run one typed action directly. The dispatcher validates it against the + * action schema and executes it; no translation and no reasoning run. + * + * `@action` runs the same `executeActions` engine as an ordinary request, so + * enabled-action gating, chained actions, entity resolution, memory recording + * and confirmation are unchanged. What it skips is translation - and the + * request path's reasoning retry for schemas with `errorReasoning`, whose + * error is returned to the caller instead. + */ + async executeAction( + request: DirectActionRequest, + extra?: ToolExtra, + ): Promise { + const disabled = this.getDisabledReason(); + if (disabled) { + return toolError(disabled); + } + + let command: string; + try { + command = buildActionCommand(request); + } catch (error) { + return toolError(errorMessage(error)); + } + this.dependencies.log(`executeAction: ${command}`); + + try { + const { result, collected } = await this.runCommand(command, extra); + if (result?.lastError) { + return toolError(`Action error: ${result.lastError}`); + } + if (result?.cancelled) { + return toolError( + cancelledText( + `Action ${request.schemaName}.${request.actionName} was cancelled.`, + collected.messages, + ), + ); + } + if (collected.pendingPrompts.length > 0) { + return toolError(pendingPromptText(collected.pendingPrompts)); + } + if (collected.messages.length > 0) { + return toolResult( + collected.messages.join("\n\n"), + collected.rawData, + ); + } + return toolResult( + `Action ${request.schemaName}.${request.actionName} executed successfully.`, + collected.rawData, + ); + } catch (error) { + const message = errorMessage(error); + this.dependencies.log(`executeAction error: ${message}`); + return toolError(`Action execution failed: ${message}`); + } + } + + /** + * Report what the session can currently run: agents, then an agent's + * sub-schemas and actions, then one action's TypeScript contract. + */ + async discoverActions(request: { + agentName?: string | undefined; + actionName?: string | undefined; + }): Promise { + const disabled = this.getDisabledReason(); + if (disabled) { + return toolError(disabled); + } + + try { + const agents = await this.getActiveAgentSchemas(request.agentName); + if (request.agentName === undefined) { + return toolResult(formatAgentList(agents)); + } + + const agent = agents[0]; + if (agent === undefined) { + return toolError( + `Agent '${request.agentName}' is not available or has no enabled actions.`, + ); + } + return request.actionName === undefined + ? toolResult(formatAgentActions(agent)) + : formatActionContract(agent, request.actionName); + } catch (error) { + return toolError( + `Error discovering actions: ${errorMessage(error)}`, + ); + } + } + + async listAgents(): Promise { + const disabled = this.getDisabledReason(); + if (disabled) { + return toolError(disabled); + } + try { + // Unfiltered on purpose: this is the pre-existing "what is + // installed" view. typeagent-discoverActions is the filtered + // "what can I run right now" view. + const schemas = await this.withDispatcher( + createClientIO({}), + (dispatcher) => dispatcher.getAgentSchemas(), + ); + return toolResult( + JSON.stringify( + schemas.map((schema) => ({ + name: schema.name, + emoji: schema.emoji, + description: schema.description, + })), + null, + 2, + ), + ); + } catch (error) { + return toolResult(`Error listing agents: ${errorMessage(error)}`); + } + } + + async getStatus(): Promise { + const disabled = this.getDisabledReason(); + if (disabled) { + return toolError(disabled); + } + try { + const status = await this.withDispatcher( + createClientIO({}), + (dispatcher) => dispatcher.getStatus(), + ); + return toolResult(JSON.stringify(status, null, 2)); + } catch (error) { + return toolResult(`Error getting status: ${errorMessage(error)}`); + } + } + + /** + * Agents with the sub-schemas this session has enabled. `getAgentSchemas` + * reports every installed schema, so the status is what makes the result + * match what `@action` will actually accept. + */ + private async getActiveAgentSchemas( + agentName?: string, + ): Promise { + return this.withDispatcher( + createClientIO({}), + async (dispatcher): Promise => { + const [schemas, status] = await Promise.all([ + dispatcher.getAgentSchemas(agentName), + dispatcher.getStatus(), + ]); + return filterActiveAgentSchemas(schemas, status); + }, + ); + } + + private async withDispatcher( + clientIO: ClientIO, + operation: (dispatcher: Dispatcher) => Promise, + ): Promise { + const dispatcher = await this.dependencies.connect(clientIO); + try { + return await operation(dispatcher); + } finally { + await dispatcher.close(); + } + } + + /** + * Submit a command, stream its progress messages, and collect its display + * output. Cancels the request when the MCP client aborts the tool call. + */ + private async runCommand( + command: string, + extra?: ToolExtra, + ): Promise<{ result: CommandResult | undefined; collected: Collected }> { + const collected = new Collected(); + let messageCount = 0; + const sendProgress = (text: string) => { + if (!extra) return; + messageCount++; + void sendProgressNotification(extra, text, messageCount); + }; + + const clientIO = createClientIO({ + onPendingPrompt: (prompt: string) => { + collected.pendingPrompts.push(prompt); + sendProgress(prompt); + }, + onSetDisplay: (message: IAgentMessage) => { + collected.add(message); + }, + onAppendDisplay: ( + message: IAgentMessage, + mode: DisplayAppendMode, + ) => { + const text = extractMessageText(message); + if (!text) return; + + if (mode === "temporary") { + // Status updates that a live UI would replace — stream + // them instead of returning them as content. + sendProgress(stripAnsi(text)); + return; + } + + // Status/info/warning/error messages (reasoning "thinking", + // tool calls, and their results) are progress, not final + // content, so they stream too and keep every tool call paired + // with its result. + if (isProgressKind(message)) { + sendProgress(stripAnsi(text)); + return; + } + + collected.add(message); + }, + }); + + const result = await this.withDispatcher(clientIO, (dispatcher) => + submitCancellableCommand( + dispatcher, + command, + extra?.signal, + (error) => + this.dependencies.log( + `cancel error: ${errorMessage(error)}`, + ), + ), + ); + return { result, collected }; + } + + private getDisabledReason(): string | undefined { + const mode = this.dependencies.getMode(); + if (mode === "dev" || mode === "bypass") { + return `TypeAgent agent-server MCP tools are disabled in ${mode} mode.`; + } + return undefined; + } +} + +/** Collected display text plus the last structured payload an agent returned. */ +class Collected { + public readonly messages: string[] = []; + public readonly pendingPrompts: string[] = []; + public rawData?: unknown; + + add(message: IAgentMessage): void { + const text = extractMessageText(message); + if (text) { + this.messages.push(stripAnsi(text)); + } + const rawData = extractRawData(message); + if (rawData !== undefined) { + this.rawData = rawData; + } + } +} + +/** + * TypeAgent asked the user something this client cannot answer, so the work + * is parked rather than done. Say so instead of reporting success. + */ +function pendingPromptText(prompts: string[]): string { + return ( + "TypeAgent is waiting for a decision this tool cannot make, so the request did not complete: " + + prompts.join("; ") + + ". Ask the user to answer it in the TypeAgent shell, or re-run with parameters that avoid the prompt." + ); +} + +/** + * Keep whatever the request produced before it stopped, but label it, so the + * caller does not read partial output as a finished request. + */ +function cancelledText(summary: string, messages: string[]): string { + return messages.length > 0 + ? `${summary} Partial output before cancellation:\n\n${messages.join("\n\n")}` + : summary; +} + +function isProgressKind(message: IAgentMessage): boolean { + const msg = message?.message; + if (typeof msg !== "object" || !msg || !("kind" in msg)) { + return false; + } + const kind = (msg as { kind: unknown }).kind; + return ( + kind === "info" || + kind === "status" || + kind === "warning" || + kind === "error" + ); +} + +/** + * Send an MCP progress notification if the client provided a progressToken. + */ +async function sendProgressNotification( + extra: ToolExtra, + message: string, + progress: number, +): Promise { + if (extra._meta?.progressToken === undefined) return; + try { + await extra.sendNotification({ + method: "notifications/progress", + params: { + progressToken: extra._meta.progressToken, + progress, + total: 0, + message, + }, + }); + } catch { + // Progress notifications are best-effort + } +} + +/** + * Submit a command and await its completion, cancelling the accepted request + * when `signal` aborts so an interrupted tool call does not leave the + * dispatcher running work nobody is waiting for. + */ + +// ── Discovery formatting ───────────────────────────────────────────────────── + +function formatAgentList(agents: AgentSchemaInfo[]): string { + if (agents.length === 0) { + return "No TypeAgent agents are enabled. Make sure the TypeAgent agent server is running."; + } + const lines = agents.map( + (agent) => `${agent.emoji} ${agent.name} — ${agent.description}`, + ); + return ( + `Enabled TypeAgent agents (${agents.length}):\n\n` + + lines.join("\n") + + "\n\nCall typeagent-discoverActions with agentName to see that agent's actions." + ); +} + +function formatAgentActions(agent: AgentSchemaInfo): string { + const sections = agent.subSchemas.map((subSchema) => { + const actions = subSchema.actions + .map((action) => ` - ${action.name} — ${action.description}`) + .join("\n"); + return ` ${subSchema.schemaName} — ${subSchema.description}\n${actions}`; + }); + const actionCount = getAgentActionNames(agent).length; + + return ( + `${agent.emoji} ${agent.name} — ${agent.description}\n\n` + + sections.join("\n\n") + + `\n\n(${actionCount} actions across ${agent.subSchemas.length} schema(s))\n` + + "Use the schemaName shown above as executeAction's schemaName. " + + "Add actionName to this tool to see an action's TypeScript parameters." + ); +} + +function formatActionContract( + agent: AgentSchemaInfo, + actionName: string, +): CallToolResult { + const found = findActionSubSchema(agent, actionName); + if (found === undefined) { + return toolError( + `Action '${actionName}' is not available in agent '${agent.name}'.\n\n` + + `Enabled actions: ${getAgentActionNames(agent).join(", ")}`, + ); + } + const { subSchema, action } = found; + if (subSchema.schemaText === undefined) { + return toolError( + `No TypeScript schema is available for action '${action.name}'.`, + ); + } + return toolResult( + `${action.name} — ${action.description}\n` + + `Run it with typeagent-executeAction({ schemaName: "${subSchema.schemaName}", actionName: "${action.name}", parameters: { ... } }).\n\n` + + "```typescript\n" + + subSchema.schemaText + + "\n```", + ); +} + +// ── Server ─────────────────────────────────────────────────────────────────── + +export class TypeAgentMcpServer { + private readonly server: McpServer; + + constructor(private readonly adapter = new TypeAgentToolAdapter()) { + this.server = new McpServer({ + name: "typeagent", + version: "0.1.0", + }); + this.registerTools(); + } + + async start(): Promise { + const transport = new StdioServerTransport(); + await this.server.connect(transport); + log( + `TypeAgent MCP server started (target: ${TYPEAGENT_URL}, mode: ${getMode()})`, + ); + } + + private registerTools(): void { + this.server.registerTool( + "typeagent-processCommand", + { + title: "TypeAgent Command Processor", + description: + "Send a natural language command to TypeAgent for processing. " + + "Use this for conversational or multi-step action requests, and whenever you do not know which typed action to run. " + + "For a single action whose schemaName, actionName and parameters you already know, prefer typeagent-executeAction — it skips translation. " + + "Do NOT use this for general knowledge questions. " + + "CRITICAL: Preserve special prefixes EXACTLY as written - do NOT strip them: " + + "'learn:', 'dev:', 'record:', 'dev: learn:'. " + + "These are TypeAgent directives that trigger special behavior (e.g., flow recording) and must always go through this tool, never typeagent-executeAction. " + + "If user says 'learn: create a playlist', pass 'learn: create a playlist' - NOT just 'create a playlist'. " + + "IMPORTANT: Always display the FULL output to the user exactly as returned. " + + "Do NOT summarize, truncate, or paraphrase the tool result. " + + "Present it in a code block if it contains a list or structured data.", + inputSchema: z.object({ + command: z + .string() + .describe( + "The natural language command to execute, including any special prefixes like 'learn:', 'dev:', 'record:'", + ), + }), + annotations: { + displayVerbatim: true, + } as Record, + _meta: { + "com.github/displayVerbatim": true, + }, + }, + async (params, extra) => + this.adapter.processCommand( + params.command, + extra as unknown as ToolExtra, + ), + ); + + this.server.registerTool( + "typeagent-discoverActions", + { + title: "TypeAgent Action Discovery", + description: + "Discover the TypeAgent actions this session can run right now, so you can call typeagent-executeAction with an exact typed action.\n" + + "- No arguments: lists the enabled agents.\n" + + "- agentName: lists that agent's schemas and their actions with descriptions. The schemaName shown is what typeagent-executeAction needs.\n" + + "- agentName + actionName: returns the TypeScript definition of that action's parameters.\n" + + "Only agents and schemas whose actions are enabled are reported, so anything listed here is runnable. " + + "Skip this tool when you already know the schemaName, actionName and parameters of the action you need. " + + "Do not call it just to satisfy a request the user phrased — sending their words to typeagent-processCommand is cheaper than a discovery round-trip, because TypeAgent caches translations. " + + "It pays off when you will reuse the contract across several calls.", + inputSchema: z.object({ + agentName: z + .string() + .optional() + .describe( + "Top-level agent name, e.g. 'player'. Omit to list all enabled agents.", + ), + actionName: z + .string() + .optional() + .describe( + "Action name to inspect. Requires agentName. Returns the action's TypeScript parameter schema.", + ), + }), + annotations: { readOnlyHint: true }, + }, + async (params) => this.adapter.discoverActions(params), + ); + + this.server.registerTool( + "typeagent-executeAction", + { + title: "TypeAgent Action Executor", + description: + "Run one TypeAgent action you already know, by schema, action name and parameters. " + + "The dispatcher validates it against its schema and executes it, skipping natural language translation and TypeAgent reasoning.\n" + + "Prefer typeagent-processCommand when the user phrased the request and you do not already know the action: TypeAgent caches translations, so a familiar phrase costs no model call, which beats discovering the contract first. " + + "This tool is the better choice when you already hold the contract, or when you composed the action yourself as a step of a larger task and there is no user phrasing to translate.\n" + + "Use typeagent-processCommand for conversational requests, multi-step requests, or any prompt carrying a 'learn:', 'dev:' or 'record:' prefix.", + inputSchema: z.object({ + schemaName: z + .string() + .describe( + "Exact schemaName from typeagent-discoverActions, e.g. 'player' or 'desktop.desktop-taskbar'.", + ), + actionName: z + .string() + .describe("Action name, e.g. 'createPlaylist'."), + parameters: z + .record(z.unknown()) + .optional() + .describe( + "Action parameters matching the action's TypeScript schema.", + ), + naturalLanguage: z + .string() + .optional() + .describe( + "The user's request, VERBATIM, when this action is exactly what they asked for. " + + "TypeAgent stores it as a translation for this action, so a later request phrased the same way runs this action without an LLM call. " + + "Omit it if you paraphrased, inferred the action, or are running it as one step of a larger task — a wrong pairing here mis-translates future requests.", + ), + }), + }, + async (params, extra) => + this.adapter.executeAction( + { + schemaName: params.schemaName, + actionName: params.actionName, + parameters: params.parameters, + naturalLanguage: params.naturalLanguage, + }, + extra as unknown as ToolExtra, + ), + ); + + this.server.registerTool( + "typeagent-listAgents", + { + title: "TypeAgent Agent List", + description: + "List available TypeAgent agents and their capabilities.", + inputSchema: z.object({}), + annotations: { readOnlyHint: true }, + }, + async () => this.adapter.listAgents(), + ); + + this.server.registerTool( + "typeagent-getStatus", + { + title: "TypeAgent Status", + description: "Get the current TypeAgent dispatcher status.", + inputSchema: z.object({}), + annotations: { readOnlyHint: true }, + }, + async () => this.adapter.getStatus(), + ); + + // TypeAgent PowerShell tools + this.server.tool( + "typeagent-powershell-list", + "List registered TypeAgent PowerShell flows. " + + "These are reusable automation scripts managed by TypeAgent's PowerShell agent " + + "that can be invoked by natural language.", + {}, + async () => this.adapter.processCommand("@powershell list"), + ); + + this.server.tool( + "typeagent-powershell-import", + "Import an existing PowerShell (.ps1) script file as a reusable TypeAgent PowerShell flow. " + + "The script is analyzed by TypeAgent's PowerShell agent and registered for future natural language invocation. " + + "Only .ps1 files are supported. The path can be absolute or relative to the working directory.", + { + filePath: z + .string() + .describe( + "Absolute or relative path to the .ps1 file to import", + ), + }, + async (params, extra) => + this.adapter.processCommand( + `@powershell import ${params.filePath}`, + extra as unknown as ToolExtra, + ), + ); + } +} diff --git a/ts/packages/copilot-plugin/src/mcp/server.ts b/ts/packages/copilot-plugin/src/mcp/server.ts index c619ec7dcf..744d6eacef 100644 --- a/ts/packages/copilot-plugin/src/mcp/server.ts +++ b/ts/packages/copilot-plugin/src/mcp/server.ts @@ -2,367 +2,18 @@ // Licensed under the MIT License. /** - * TypeAgent MCP Server for Copilot CLI. + * MCP server entry point for the TypeAgent Copilot CLI plugin. * - * Exposes TypeAgent dispatcher operations as MCP tools, allowing the - * Copilot LLM to delegate action requests to TypeAgent. - * - * Uses MCP progress notifications to stream display messages to the - * Copilot CLI timeline in real-time as TypeAgent processes the command. - * - * Connection to TypeAgent is lazy — established on first tool call, - * not during MCP server startup. + * One bundled entry point serves three logical servers, selected by argv: + * the agent-server tools (default), the read-only workspace tools + * (`--workspace`), and the macro catalog tools (`--macros`). */ -import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; -import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; -import { z } from "zod"; -import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; -import type { Dispatcher, IAgentMessage } from "@typeagent/agent-server-client"; -import { awaitCommand } from "@typeagent/dispatcher-types"; -import type { DisplayAppendMode } from "@typeagent/agent-sdk"; -import { - createClientIO, - connectToTypeAgent, - TYPEAGENT_URL, -} from "../shared/typeagent-client.js"; -import { extractMessageText } from "../shared/message-formatter.js"; -import { getMode } from "../shared/plugin-config.js"; +import { TypeAgentMcpServer, log } from "./agentServer.js"; import { TypeAgentMacroMcpServer } from "./macroServer.js"; import { selectMcpServer } from "./serverSelector.js"; import { TypeAgentWorkspaceMcpServer } from "./workspaceServer.js"; -// ── Helpers ────────────────────────────────────────────────────────────────── - -function stripAnsi(text: string): string { - return text.replace(/\x1b\[[0-9;]*m/g, ""); -} - -function toolResult(text: string): CallToolResult { - return { content: [{ type: "text", text }] }; -} - -function toolError(text: string): CallToolResult { - return { isError: true, content: [{ type: "text", text }] }; -} - -/** - * Format a large result for display. Strips markdown formatting and wraps - * in a code fence so the CLI preserves newlines and structured layout. - */ -function formatLargeResult(response: string): CallToolResult { - const lines = response.split("\n").length; - if (lines > 5) { - // Strip markdown bold (**text**) — doesn't render inside code fences - const plain = response.replace(/\*\*([^*]+)\*\*/g, "$1"); - return toolResult("```\n" + plain + "\n```"); - } - return toolResult(response); -} - -function log(message: string): void { - process.stderr.write( - `[${new Date().toISOString()}] [typeagent-mcp] ${message}\n`, - ); -} - -// Type for the extra parameter passed to tool callbacks -interface ToolExtra { - _meta?: { - progressToken?: string | number; - }; - sendNotification: (notification: { - method: string; - params: Record; - }) => Promise; - signal: AbortSignal; -} - -// ── Server ─────────────────────────────────────────────────────────────────── - -class TypeAgentMcpServer { - private server: McpServer; - - constructor() { - this.server = new McpServer({ - name: "typeagent", - version: "0.1.0", - }); - this.registerTools(); - } - - async start(): Promise { - const transport = new StdioServerTransport(); - await this.server.connect(transport); - const mode = getMode(); - log( - `TypeAgent MCP server started (target: ${TYPEAGENT_URL}, mode: ${mode})`, - ); - } - - private registerTools(): void { - this.server.registerTool( - "typeagent-processCommand", - { - title: "TypeAgent Command Processor", - description: - "Send a natural language command to TypeAgent for processing. " + - "Use this for action requests like scheduling meetings, sending emails, " + - "playing music, controlling the browser, managing lists, etc. " + - "Do NOT use this for general knowledge questions. " + - "CRITICAL: Preserve special prefixes EXACTLY as written - do NOT strip them: " + - "'learn:', 'dev:', 'record:', 'dev: learn:'. " + - "These are TypeAgent directives that trigger special behavior (e.g., flow recording). " + - "If user says 'learn: create a playlist', pass 'learn: create a playlist' - NOT just 'create a playlist'. " + - "IMPORTANT: Always display the FULL output to the user exactly as returned. " + - "Do NOT summarize, truncate, or paraphrase the tool result. " + - "Present it in a code block if it contains a list or structured data.", - inputSchema: z.object({ - command: z - .string() - .describe( - "The natural language command to execute, including any special prefixes like 'learn:', 'dev:', 'record:'", - ), - }), - annotations: { - displayVerbatim: true, - } as Record, - _meta: { - "com.github/displayVerbatim": true, - }, - }, - async (params, extra) => - this.processCommand(params.command, extra as ToolExtra), - ); - - this.server.tool( - "typeagent-listAgents", - "List available TypeAgent agents and their capabilities.", - {}, - async () => this.listAgents(), - ); - - this.server.tool( - "typeagent-getStatus", - "Get the current TypeAgent dispatcher status.", - {}, - async () => this.getStatus(), - ); - - // TypeAgent PowerShell tools - this.server.tool( - "typeagent-powershell-list", - "List registered TypeAgent PowerShell flows. " + - "These are reusable automation scripts managed by TypeAgent's PowerShell agent " + - "that can be invoked by natural language.", - {}, - async () => this.processCommand("@powershell list"), - ); - - this.server.tool( - "typeagent-powershell-import", - "Import an existing PowerShell (.ps1) script file as a reusable TypeAgent PowerShell flow. " + - "The script is analyzed by TypeAgent's PowerShell agent and registered for future natural language invocation. " + - "Only .ps1 files are supported. The path can be absolute or relative to the working directory.", - { - filePath: z - .string() - .describe( - "Absolute or relative path to the .ps1 file to import", - ), - }, - async (params, extra) => { - const command = `@powershell import ${params.filePath}`; - return this.processCommand(command, extra as ToolExtra); - }, - ); - } - - /** - * Send an MCP progress notification if the client provided a progressToken. - */ - private async sendProgress( - extra: ToolExtra, - message: string, - progress: number, - total: number, - ): Promise { - if (extra._meta?.progressToken === undefined) return; - try { - await extra.sendNotification({ - method: "notifications/progress", - params: { - progressToken: extra._meta.progressToken, - progress, - total, - message, - }, - }); - } catch { - // Progress notifications are best-effort - } - } - - private async processCommand( - command: string, - extra?: ToolExtra, - ): Promise { - const disabled = this.getDisabledReason(); - if (disabled) { - return toolError(disabled); - } - log(`processCommand: ${command}`); - - const responseCollector = { messages: [] as string[] }; - let messageCount = 0; - let dispatcher: Dispatcher | null = null; - - try { - const clientIO = createClientIO({ - onSetDisplay: (message: IAgentMessage) => { - const text = extractMessageText(message); - if (text) { - const cleaned = stripAnsi(text); - responseCollector.messages.push(cleaned); - } - }, - onAppendDisplay: ( - message: IAgentMessage, - mode: DisplayAppendMode, - ) => { - const text = extractMessageText(message); - if (!text) return; - const cleaned = stripAnsi(text); - - if (mode === "temporary") { - // Temporary messages are status updates — stream as progress only - messageCount++; - if (extra) { - void this.sendProgress( - extra, - cleaned, - messageCount, - 0, - ); - } - return; - } - - // Emit progress for status/info/warning/error messages - // (reasoning "thinking", tool calls, and their results - // including error results). These are progress, not final - // content, so we stream them and skip responseCollector — - // keeping every tool call paired with its result. - const msg = message?.message; - if (typeof msg === "object" && msg && "kind" in msg) { - const kind = (msg as { kind: unknown }).kind; - if ( - kind === "info" || - kind === "status" || - kind === "warning" || - kind === "error" - ) { - messageCount++; - if (extra) { - void this.sendProgress( - extra, - cleaned, - messageCount, - 0, - ); - } - return; - } - } - - responseCollector.messages.push(cleaned); - }, - }); - - dispatcher = await connectToTypeAgent(clientIO); - const result = await awaitCommand(dispatcher, command); - - if (result?.lastError) { - return toolResult(`Error: ${result.lastError}`); - } - - if (responseCollector.messages.length > 0) { - const response = responseCollector.messages.join("\n\n"); - return formatLargeResult(response); - } - - return toolResult(`Successfully executed: ${command}`); - } catch (error) { - const msg = error instanceof Error ? error.message : String(error); - log(`processCommand error: ${msg}`); - return toolResult(`Error executing command: ${msg}`); - } finally { - if (dispatcher) { - await dispatcher.close(); - } - } - } - - private async listAgents(): Promise { - const disabled = this.getDisabledReason(); - if (disabled) { - return toolError(disabled); - } - let dispatcher: Dispatcher | null = null; - try { - const clientIO = createClientIO({}); - dispatcher = await connectToTypeAgent(clientIO); - const schemas = await dispatcher.getAgentSchemas(); - const agents = schemas.map((s) => ({ - name: s.name, - emoji: s.emoji, - description: s.description, - })); - return toolResult(JSON.stringify(agents, null, 2)); - } catch (error) { - return toolResult( - `Error listing agents: ${error instanceof Error ? error.message : String(error)}`, - ); - } finally { - if (dispatcher) { - await dispatcher.close(); - } - } - } - - private async getStatus(): Promise { - const disabled = this.getDisabledReason(); - if (disabled) { - return toolError(disabled); - } - let dispatcher: Dispatcher | null = null; - try { - const clientIO = createClientIO({}); - dispatcher = await connectToTypeAgent(clientIO); - const status = await dispatcher.getStatus(); - return toolResult(JSON.stringify(status, null, 2)); - } catch (error) { - return toolResult( - `Error getting status: ${error instanceof Error ? error.message : String(error)}`, - ); - } finally { - if (dispatcher) { - await dispatcher.close(); - } - } - } - - private getDisabledReason(): string | undefined { - const mode = getMode(); - if (mode === "dev" || mode === "bypass") { - return `TypeAgent agent-server MCP tools are disabled in ${mode} mode.`; - } - return undefined; - } -} - -// ── Main ───────────────────────────────────────────────────────────────────── - const serverKind = selectMcpServer(process.argv.slice(2)); const server = serverKind === "workspace" diff --git a/ts/packages/copilot-plugin/src/shared/message-formatter.ts b/ts/packages/copilot-plugin/src/shared/message-formatter.ts index 553596a920..588778ae5a 100644 --- a/ts/packages/copilot-plugin/src/shared/message-formatter.ts +++ b/ts/packages/copilot-plugin/src/shared/message-formatter.ts @@ -110,6 +110,22 @@ export function extractMessageText(message: IAgentMessage): string | undefined { return extractText(message.message); } +/** + * Extracts an agent's machine-readable payload from an IAgentMessage, when + * the agent produced StructuredContent. Returned to MCP clients as + * `structuredContent` so they can act on the data instead of re-parsing the + * display text. + */ +export function extractRawData(message: IAgentMessage): unknown { + if (typeof message !== "object" || !("message" in message)) { + return undefined; + } + const msg = message.message; + return isStructuredContent(msg as DisplayContent) + ? (msg as StructuredContent).rawData + : undefined; +} + /** * Convert HTML to plain text using html-to-text, matching the * pattern used by the TypeAgent commandExecutor MCP server. diff --git a/ts/packages/copilot-plugin/src/shared/tool-identities.ts b/ts/packages/copilot-plugin/src/shared/tool-identities.ts index 66e77c9786..e353c506ad 100644 --- a/ts/packages/copilot-plugin/src/shared/tool-identities.ts +++ b/ts/packages/copilot-plugin/src/shared/tool-identities.ts @@ -3,6 +3,8 @@ const TYPEAGENT_AGENT_SERVER_TOOLS = [ "typeagent-processcommand", + "typeagent-discoveractions", + "typeagent-executeaction", "typeagent-listagents", "typeagent-getstatus", "typeagent-powershell-list", diff --git a/ts/packages/copilot-plugin/src/shared/typeagent-client.ts b/ts/packages/copilot-plugin/src/shared/typeagent-client.ts index 7c93bbd4f7..c4f1ff3e96 100644 --- a/ts/packages/copilot-plugin/src/shared/typeagent-client.ts +++ b/ts/packages/copilot-plugin/src/shared/typeagent-client.ts @@ -5,6 +5,7 @@ * Shared TypeAgent agent-server connection management. */ +import { randomUUID } from "node:crypto"; import { connectAgentServer, connectDispatcher, @@ -14,9 +15,12 @@ import { type IAgentMessage, } from "@typeagent/agent-server-client"; import type { DisplayAppendMode } from "@typeagent/agent-sdk"; -import type { - RequestId, - TemplateEditConfig, +import { + QueueFullError, + ServerStoppingError, + type CommandResult, + type RequestId, + type TemplateEditConfig, } from "@typeagent/dispatcher-types"; export const TYPEAGENT_HOST = process.env.TYPEAGENT_HOST || "localhost"; @@ -26,6 +30,12 @@ export const TYPEAGENT_URL = `ws://${TYPEAGENT_HOST}:${TYPEAGENT_PORT}`; export interface DisplayCallbacks { onSetDisplay?: (message: IAgentMessage) => void; onAppendDisplay?: (message: IAgentMessage, mode: DisplayAppendMode) => void; + /** + * A prompt the agent needs answered before it can finish. This client + * cannot answer it - there is no return path for `respondToChoice` - so + * callers report it instead of implying the work completed. + */ + onPendingPrompt?: (message: string) => void; } /** @@ -55,8 +65,26 @@ export function createClientIO(callbacks: DisplayCallbacks): ClientIO { notify(): void {}, async openLocalView(): Promise {}, async closeLocalView(): Promise {}, - requestChoice(): void {}, - requestForm(): void {}, + requestChoice( + _requestId: RequestId, + _choiceId: string, + _type: "yesNo" | "multiChoice" | "pickRemember", + message: string, + choices: string[], + ): void { + callbacks.onPendingPrompt?.( + `${message} (options: ${choices.join(", ")})`, + ); + }, + requestForm( + _requestId: RequestId, + _choiceId: string, + form: { title?: string }, + ): void { + callbacks.onPendingPrompt?.( + form.title ?? "A form must be filled in.", + ); + }, takeAction(): void {}, shutdown(): void {}, async question( @@ -89,3 +117,61 @@ export async function connectToTypeAgent( export function connectToAgentServer(): Promise { return connectAgentServer(TYPEAGENT_URL); } + +/** + * Submit a command and await its completion, cancelling the request if + * `signal` aborts. Without this an interrupted MCP tool call would leave + * TypeAgent running work nobody is waiting for. + * + * Same submit-time errors as `awaitCommand`: `QueueFullError` when the + * request queue is full, `ServerStoppingError` during shutdown. + */ +export async function submitCancellableCommand( + dispatcher: Dispatcher, + command: string, + signal?: AbortSignal, + onCancelError?: (error: unknown) => void, +): Promise { + if (signal?.aborted) { + // Never start side-effecting work for a call the client already gave + // up on - connecting to TypeAgent takes long enough for this to happen. + return { cancelled: true }; + } + const clientRequestId = `copilot-plugin-${randomUUID()}`; + let requestId: string | undefined; + const cancel = () => { + try { + if (requestId !== undefined) { + void dispatcher.cancelCommand(requestId).catch(onCancelError); + } else { + // Early-cancel path: the server-assigned id has not + // round-tripped back to us yet. + dispatcher.cancelCommandByClientId(clientRequestId); + } + } catch (error) { + onCancelError?.(error); + } + }; + signal?.addEventListener("abort", cancel, { once: true }); + try { + const submitted = await dispatcher.submitCommand( + command, + undefined, + undefined, + clientRequestId, + ); + if (!submitted.ok) { + throw submitted.error === "queue_full" + ? new QueueFullError(submitted.maxDepth) + : new ServerStoppingError(); + } + requestId = submitted.entry.requestId; + if (signal?.aborted) { + // The abort landed before the entry had a server-side id. + await dispatcher.cancelCommand(requestId); + } + return await submitted.entry.completion; + } finally { + signal?.removeEventListener("abort", cancel); + } +} diff --git a/ts/packages/copilot-plugin/test/actionDispatch.spec.ts b/ts/packages/copilot-plugin/test/actionDispatch.spec.ts new file mode 100644 index 0000000000..5de6da2b15 --- /dev/null +++ b/ts/packages/copilot-plugin/test/actionDispatch.spec.ts @@ -0,0 +1,257 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { describe, expect, it } from "@jest/globals"; +import type { + AgentSchemaInfo, + DispatcherStatus, +} from "@typeagent/dispatcher-types"; +import { + buildActionCommand, + filterActiveAgentSchemas, + findActionSubSchema, + getAgentActionNames, +} from "@typeagent/dispatcher-types/helpers/actionDispatch"; + +function agent( + name: string, + subSchemas: { schemaName: string; actions: string[] }[], +): AgentSchemaInfo { + return { + name, + emoji: "🎵", + description: `${name} agent`, + subSchemas: subSchemas.map((subSchema) => ({ + schemaName: subSchema.schemaName, + description: `${subSchema.schemaName} schema`, + schemaText: `type ${subSchema.schemaName} = {};`, + actions: subSchema.actions.map((actionName) => ({ + name: actionName, + description: `${actionName} description`, + })), + })), + }; +} + +function status(active: Record): DispatcherStatus { + return { + agents: Object.entries(active).map(([name, isActive]) => ({ + emoji: "🎵", + name, + lastUsed: false, + priority: false, + request: false, + active: isActive, + actionActive: isActive, + })), + details: "", + }; +} + +describe("buildActionCommand", () => { + it("dispatches through @action instead of natural language", () => { + expect( + buildActionCommand({ + schemaName: "player", + actionName: "createPlaylist", + parameters: { name: "Top Jazz" }, + }), + ).toBe( + `@action player createPlaylist --parameters '{"name":"Top Jazz"}'`, + ); + }); + + it("omits an empty parameter object", () => { + expect( + buildActionCommand({ + schemaName: "player", + actionName: "pause", + parameters: {}, + }), + ).toBe("@action player pause"); + }); + + it("keeps apostrophes in parameters parseable by the command tokenizer", () => { + const command = buildActionCommand({ + schemaName: "list", + actionName: "addItems", + parameters: { items: ["Bob's milk"] }, + }); + + const json = command.slice( + command.indexOf("'") + 1, + command.lastIndexOf("'"), + ); + expect(json).not.toContain("'"); + expect(JSON.parse(json)).toEqual({ items: ["Bob's milk"] }); + }); + + it("quotes a natural language phrase with a quote character it does not use", () => { + expect( + buildActionCommand({ + schemaName: "player", + actionName: "play", + naturalLanguage: "play Bob's song", + }), + ).toBe(`@action player play --naturalLanguage "play Bob's song"`); + }); + + it("drops a natural language phrase that cannot round-trip", () => { + expect( + buildActionCommand({ + schemaName: "player", + actionName: "play", + naturalLanguage: `play Bob's "best" song`, + }), + ).toBe("@action player play"); + }); + + it("ignores a blank natural language phrase", () => { + expect( + buildActionCommand({ + schemaName: "player", + actionName: "play", + naturalLanguage: " ", + }), + ).toBe("@action player play"); + }); + + it("rejects names that would add command tokens", () => { + expect(() => + buildActionCommand({ + schemaName: "player --parameters '{}'", + actionName: "play", + }), + ).toThrow("Invalid schema name"); + expect(() => + buildActionCommand({ + schemaName: "player", + actionName: "play; @shutdown", + }), + ).toThrow("Invalid action name"); + }); + + it("accepts dotted sub-schema names", () => { + expect( + buildActionCommand({ + schemaName: "desktop.desktop-taskbar", + actionName: "alignTaskbar", + }), + ).toBe("@action desktop.desktop-taskbar alignTaskbar"); + }); +}); + +describe("filterActiveAgentSchemas", () => { + const schemas = [ + agent("player", [{ schemaName: "player", actions: ["play"] }]), + agent("desktop", [ + { schemaName: "desktop", actions: ["launchApp"] }, + { schemaName: "desktop.desktop-taskbar", actions: ["align"] }, + ]), + ]; + + it("drops disabled sub-schemas but keeps the rest of the agent", () => { + const filtered = filterActiveAgentSchemas( + schemas, + status({ + player: true, + desktop: true, + "desktop.desktop-taskbar": false, + }), + ); + + expect(filtered.map((a) => a.name)).toEqual(["player", "desktop"]); + expect( + filtered[1].subSchemas.map((subSchema) => subSchema.schemaName), + ).toEqual(["desktop"]); + }); + + it("drops an agent whose sub-schemas are all disabled", () => { + const filtered = filterActiveAgentSchemas( + schemas, + status({ + player: false, + desktop: true, + "desktop.desktop-taskbar": true, + }), + ); + + expect(filtered.map((a) => a.name)).toEqual(["desktop"]); + }); + + it("keeps sub-schemas the status does not mention", () => { + const filtered = filterActiveAgentSchemas(schemas, status({})); + + expect(filtered).toEqual(schemas); + }); + + it("does not mutate the input schemas", () => { + filterActiveAgentSchemas(schemas, status({ desktop: false })); + + expect(schemas[1].subSchemas).toHaveLength(2); + }); + + it("hides a schema whose actions are off even when its commands are on", () => { + // `active` is true when either actions or commands are enabled, but + // @action only runs when the actions themselves are enabled. + const filtered = filterActiveAgentSchemas(schemas, { + agents: [ + { + emoji: "🎵", + name: "player", + lastUsed: false, + priority: false, + request: false, + active: true, + actionActive: false, + }, + ], + details: "", + }); + + expect(filtered.map((a) => a.name)).toEqual(["desktop"]); + }); + + it("falls back to active for servers that do not report actionActive", () => { + const filtered = filterActiveAgentSchemas(schemas, { + agents: [ + { + emoji: "🎵", + name: "player", + lastUsed: false, + priority: false, + request: false, + active: false, + }, + ], + details: "", + }); + + expect(filtered.map((a) => a.name)).toEqual(["desktop"]); + }); +}); + +describe("agent action lookup", () => { + const desktop = agent("desktop", [ + { schemaName: "desktop", actions: ["launchApp"] }, + { schemaName: "desktop.desktop-taskbar", actions: ["alignTaskbar"] }, + ]); + + it("finds the sub-schema that declares an action, ignoring case", () => { + expect(findActionSubSchema(desktop, "aligntaskbar")).toMatchObject({ + subSchema: { schemaName: "desktop.desktop-taskbar" }, + action: { name: "alignTaskbar" }, + }); + }); + + it("returns undefined for an unknown action", () => { + expect(findActionSubSchema(desktop, "nope")).toBeUndefined(); + }); + + it("lists every action name across sub-schemas", () => { + expect(getAgentActionNames(desktop)).toEqual([ + "launchApp", + "alignTaskbar", + ]); + }); +}); diff --git a/ts/packages/copilot-plugin/test/agentServer.spec.ts b/ts/packages/copilot-plugin/test/agentServer.spec.ts new file mode 100644 index 0000000000..7e397d8c88 --- /dev/null +++ b/ts/packages/copilot-plugin/test/agentServer.spec.ts @@ -0,0 +1,457 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { jest } from "@jest/globals"; +import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js"; +import type { ClientIO, Dispatcher } from "@typeagent/agent-server-client"; +import type { + AgentSchemaInfo, + CommandResult, + DispatcherStatus, +} from "@typeagent/dispatcher-types"; +import { TypeAgentToolAdapter } from "../src/mcp/agentServer.js"; +import type { AgentToolDependencies } from "../src/mcp/agentServer.js"; + +const playerSchemas: AgentSchemaInfo[] = [ + { + name: "player", + emoji: "🎵", + description: "Play music", + subSchemas: [ + { + schemaName: "player", + description: "Playback control", + schemaText: "export type PlayAction = { actionName: 'play' };", + actions: [{ name: "play", description: "Play a track" }], + }, + { + schemaName: "player.playlist", + description: "Playlist management", + schemaText: undefined, + actions: [ + { name: "createPlaylist", description: "Make a playlist" }, + ], + }, + ], + }, +]; + +function agentStatus(name: string, active: boolean) { + return { + emoji: "🎵", + name, + lastUsed: false, + priority: false, + request: true, + active, + actionActive: active, + }; +} + +const status: DispatcherStatus = { + agents: [ + agentStatus("player", true), + agentStatus("player.playlist", false), + ], + details: "", +}; + +type FakeDispatcher = { + dispatcher: Dispatcher; + commands: string[]; + close: jest.Mock; + cancelCommand: jest.Mock; +}; + +function fakeDispatcher(options?: { + schemas?: AgentSchemaInfo[]; + status?: DispatcherStatus; + result?: CommandResult; + onSubmit?: (clientIO: ClientIO) => void; + clientIO?: ClientIO; +}): FakeDispatcher { + const commands: string[] = []; + const close = jest.fn(async () => {}); + const cancelCommand = jest.fn(async () => ({ + kind: "cancelled_running" as const, + requestId: "request-1", + })); + const dispatcher = { + getAgentSchemas: jest.fn(async () => options?.schemas ?? playerSchemas), + getStatus: jest.fn(async () => options?.status ?? status), + submitCommand: jest.fn(async (command: string) => { + commands.push(command); + if (options?.clientIO) { + options.onSubmit?.(options.clientIO); + } + return { + ok: true as const, + entry: { + requestId: "request-1", + completion: Promise.resolve(options?.result), + }, + }; + }), + cancelCommand, + cancelCommandByClientId: jest.fn(), + close, + } as unknown as Dispatcher; + return { dispatcher, commands, close, cancelCommand }; +} + +function makeAdapter( + fake: FakeDispatcher, + overrides: Partial = {}, +): { adapter: TypeAgentToolAdapter; capture: { clientIO?: ClientIO } } { + const capture: { clientIO?: ClientIO } = {}; + const adapter = new TypeAgentToolAdapter({ + connect: async (clientIO: ClientIO) => { + capture.clientIO = clientIO; + return fake.dispatcher; + }, + getMode: () => "mcp", + log: () => {}, + ...overrides, + }); + return { adapter, capture }; +} + +function textOf(result: CallToolResult): string { + return result.content + .map((entry) => ("text" in entry ? String(entry.text) : "")) + .join("\n"); +} + +describe("TypeAgent action discovery", () => { + it("lists the agents that have enabled actions", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.discoverActions({}); + + expect(result.isError).toBeUndefined(); + expect(textOf(result)).toContain("player"); + expect(fake.close).toHaveBeenCalledTimes(1); + }); + + it("hides actions from sub-schemas the session disabled", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.discoverActions({ agentName: "player" }); + + const text = textOf(result); + expect(text).toContain("play"); + expect(text).not.toContain("createPlaylist"); + }); + + it("returns the TypeScript contract for one action", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.discoverActions({ + agentName: "player", + actionName: "play", + }); + + expect(result.isError).toBeUndefined(); + expect(textOf(result)).toContain("export type PlayAction"); + }); + + it("reports enabled alternatives for an unknown action", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.discoverActions({ + agentName: "player", + actionName: "createPlaylist", + }); + + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("play"); + }); + + it("is disabled in dev and bypass mode without connecting", async () => { + const connect = jest.fn(async () => fakeDispatcher().dispatcher); + const adapter = new TypeAgentToolAdapter({ + connect: connect as unknown as AgentToolDependencies["connect"], + getMode: () => "bypass", + log: () => {}, + }); + + const result = await adapter.discoverActions({}); + + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("bypass"); + expect(connect).not.toHaveBeenCalled(); + }); +}); + +describe("TypeAgent direct action execution", () => { + it("dispatches @action instead of natural language", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.executeAction({ + schemaName: "player", + actionName: "play", + parameters: { track: "Yesterday" }, + }); + + expect(result.isError).toBeUndefined(); + expect(fake.commands).toEqual([ + `@action player play --parameters '{"track":"Yesterday"}'`, + ]); + expect(fake.close).toHaveBeenCalledTimes(1); + }); + + it("passes the original phrasing through for cache seeding only", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + await adapter.executeAction({ + schemaName: "player", + actionName: "play", + naturalLanguage: "play yesterday", + }); + + expect(fake.commands[0]).toBe( + `@action player play --naturalLanguage 'play yesterday'`, + ); + }); + + it("rejects a schema name that could inject extra command tokens", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.executeAction({ + schemaName: "player --flag", + actionName: "play", + }); + + expect(result.isError).toBe(true); + expect(fake.commands).toEqual([]); + }); + + it("reports a dispatcher action error as a tool error", async () => { + const fake = fakeDispatcher({ result: { lastError: "no such track" } }); + const { adapter } = makeAdapter(fake); + + const result = await adapter.executeAction({ + schemaName: "player", + actionName: "play", + }); + + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("no such track"); + }); + + it("forwards an agent's structured payload to the MCP client", async () => { + const fake = fakeDispatcher(); + const captured: { clientIO?: ClientIO } = {}; + const adapter = new TypeAgentToolAdapter({ + connect: async (clientIO: ClientIO) => { + captured.clientIO = clientIO; + return fake.dispatcher; + }, + getMode: () => "mcp", + log: () => {}, + }); + (fake.dispatcher.submitCommand as jest.Mock).mockImplementation( + async () => { + captured.clientIO?.setDisplay({ + message: { + type: "structured", + blocks: [], + rawData: { tracks: ["Yesterday"] }, + alternates: [{ type: "text", content: "1 track" }], + }, + } as never); + return { + ok: true as const, + entry: { + requestId: "request-1", + completion: Promise.resolve(undefined), + }, + }; + }, + ); + + const result = await adapter.executeAction({ + schemaName: "player", + actionName: "play", + }); + + expect(result.structuredContent).toEqual({ tracks: ["Yesterday"] }); + }); + + it("cancels the dispatcher request when the tool call is aborted", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + const controller = new AbortController(); + let completed: (value: CommandResult | undefined) => void = () => {}; + const completion = new Promise((resolve) => { + completed = resolve; + }); + let submitted: () => void = () => {}; + const submittedSignal = new Promise((resolve) => { + submitted = resolve; + }); + (fake.dispatcher.submitCommand as jest.Mock).mockImplementation( + async () => { + submitted(); + return { + ok: true as const, + entry: { requestId: "request-1", completion }, + }; + }, + ); + + const pending = adapter.executeAction( + { schemaName: "player", actionName: "play" }, + { + sendNotification: async () => {}, + signal: controller.signal, + }, + ); + await submittedSignal; + controller.abort(); + completed({ cancelled: true }); + + const result = await pending; + expect(fake.cancelCommand).toHaveBeenCalledWith("request-1"); + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("cancelled"); + }); + + it("never submits an action for a tool call that was already aborted", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + const controller = new AbortController(); + controller.abort(); + + const result = await adapter.executeAction( + { schemaName: "player", actionName: "play" }, + { sendNotification: async () => {}, signal: controller.signal }, + ); + + expect(fake.dispatcher.submitCommand).not.toHaveBeenCalled(); + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("cancelled"); + }); + + it("labels output produced before a cancellation", async () => { + const fake = fakeDispatcher({ result: { cancelled: true } }); + const { adapter, capture } = makeAdapter(fake); + (fake.dispatcher.submitCommand as jest.Mock).mockImplementation( + async () => { + capture.clientIO?.setDisplay({ + message: "started queuing tracks", + } as never); + return { + ok: true as const, + entry: { + requestId: "request-1", + completion: Promise.resolve({ cancelled: true }), + }, + }; + }, + ); + + const result = await adapter.executeAction({ + schemaName: "player", + actionName: "play", + }); + + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("was cancelled"); + expect(textOf(result)).toContain("started queuing tracks"); + }); + + it("reports a prompt it cannot answer instead of claiming success", async () => { + const fake = fakeDispatcher(); + const { adapter, capture } = makeAdapter(fake); + (fake.dispatcher.submitCommand as jest.Mock).mockImplementation( + async () => { + capture.clientIO?.requestChoice( + { requestId: "request-1" }, + "choice-1", + "yesNo", + "Delete the shopping list?", + ["Yes", "No"], + "list", + ); + return { + ok: true as const, + entry: { + requestId: "request-1", + completion: Promise.resolve(undefined), + }, + }; + }, + ); + + const result = await adapter.executeAction({ + schemaName: "list", + actionName: "deleteList", + }); + + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("Delete the shopping list?"); + expect(textOf(result)).toContain("did not complete"); + }); + + it("is disabled in dev mode without connecting", async () => { + const connect = jest.fn(async () => fakeDispatcher().dispatcher); + const adapter = new TypeAgentToolAdapter({ + connect: connect as unknown as AgentToolDependencies["connect"], + getMode: () => "dev", + log: () => {}, + }); + + const result = await adapter.executeAction({ + schemaName: "player", + actionName: "play", + }); + + expect(result.isError).toBe(true); + expect(connect).not.toHaveBeenCalled(); + }); +}); + +describe("TypeAgent processCommand", () => { + it("still sends the request unchanged for translation", async () => { + const fake = fakeDispatcher(); + const { adapter } = makeAdapter(fake); + + const result = await adapter.processCommand("learn: play yesterday"); + + expect(result.isError).toBeUndefined(); + expect(fake.commands).toEqual(["learn: play yesterday"]); + expect(textOf(result)).toContain("learn: play yesterday"); + }); + + it("returns dispatcher errors as text, not tool errors", async () => { + const fake = fakeDispatcher({ result: { lastError: "boom" } }); + const { adapter } = makeAdapter(fake); + + const result = await adapter.processCommand("play yesterday"); + + expect(result.isError).toBeUndefined(); + expect(textOf(result)).toContain("boom"); + }); + + it("stays disabled in bypass mode", async () => { + const connect = jest.fn(async () => fakeDispatcher().dispatcher); + const adapter = new TypeAgentToolAdapter({ + connect: connect as unknown as AgentToolDependencies["connect"], + getMode: () => "bypass", + log: () => {}, + }); + + const result = await adapter.processCommand("play yesterday"); + + expect(result.isError).toBe(true); + expect(connect).not.toHaveBeenCalled(); + }); +}); diff --git a/ts/packages/copilot-plugin/test/pluginArtifact.spec.ts b/ts/packages/copilot-plugin/test/pluginArtifact.spec.ts index 03ec0fa4d0..979ede9d29 100644 --- a/ts/packages/copilot-plugin/test/pluginArtifact.spec.ts +++ b/ts/packages/copilot-plugin/test/pluginArtifact.spec.ts @@ -55,4 +55,40 @@ describe("staged plugin artifact", () => { await client.close(); } }); + + it("exposes the direct action tools on the bundled agent server", async () => { + const testDir = path.dirname(fileURLToPath(import.meta.url)); + const pluginRoot = path.resolve(testDir, "..", ".."); + const manifest = JSON.parse( + await readFile(path.join(pluginRoot, ".mcp.json"), "utf8"), + ) as PluginMcpManifest; + const registration = manifest.mcpServers["typeagent"]; + + const transport = new StdioClientTransport({ + command: process.execPath, + args: registration.args.map((argument) => + argument.replace("${PLUGIN_ROOT}", pluginRoot), + ), + stderr: "pipe", + }); + const client = new Client({ + name: "typeagent-plugin-artifact-test", + version: "1.0.0", + }); + try { + await client.connect(transport); + const catalog = await client.listTools(); + expect(catalog.tools.map((tool) => tool.name)).toEqual( + expect.arrayContaining([ + "typeagent-processCommand", + "typeagent-discoverActions", + "typeagent-executeAction", + "typeagent-listAgents", + "typeagent-getStatus", + ]), + ); + } finally { + await client.close(); + } + }); }); diff --git a/ts/packages/dispatcher/dispatcher/src/command/command.ts b/ts/packages/dispatcher/dispatcher/src/command/command.ts index 4841ec8041..861770b995 100644 --- a/ts/packages/dispatcher/dispatcher/src/command/command.ts +++ b/ts/packages/dispatcher/dispatcher/src/command/command.ts @@ -709,6 +709,7 @@ export function getDispatcherStatus( active: context.agents.isActionActive(config.schemaName) || context.agents.isCommandEnabled(appAgentName), + actionActive: context.agents.isActionActive(config.schemaName), }; }); @@ -722,6 +723,7 @@ export function getDispatcherStatus( priority: false, request: false, active: context.agents.isCommandEnabled(agentName), + actionActive: false, }); } } diff --git a/ts/packages/dispatcher/types/package.json b/ts/packages/dispatcher/types/package.json index fa0f670ba7..813c8c5017 100644 --- a/ts/packages/dispatcher/types/package.json +++ b/ts/packages/dispatcher/types/package.json @@ -13,6 +13,7 @@ "type": "module", "exports": { ".": "./dist/index.js", + "./helpers/actionDispatch": "./dist/helpers/actionDispatch.js", "./helpers/status": "./dist/helpers/status.js" }, "files": [ diff --git a/ts/packages/dispatcher/types/src/dispatcher.ts b/ts/packages/dispatcher/types/src/dispatcher.ts index 7bdbda27cb..4de21ef4d1 100644 --- a/ts/packages/dispatcher/types/src/dispatcher.ts +++ b/ts/packages/dispatcher/types/src/dispatcher.ts @@ -203,6 +203,14 @@ export type AppAgentStatus = { priority: boolean; request: boolean; active: boolean; + + /** + * Whether this schema's actions can actually be executed. `active` is + * true when either the actions or the agent's commands are enabled, so + * it cannot be used to decide whether `@action` will run. Optional + * because older agent servers do not report it. + */ + actionActive?: boolean; }; export type DispatcherStatus = { diff --git a/ts/packages/dispatcher/types/src/helpers/actionDispatch.ts b/ts/packages/dispatcher/types/src/helpers/actionDispatch.ts new file mode 100644 index 0000000000..1e1d216e15 --- /dev/null +++ b/ts/packages/dispatcher/types/src/helpers/actionDispatch.ts @@ -0,0 +1,160 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +// Helpers for discovering the actions a dispatcher session currently exposes +// and for running one of them directly through the `@action` command, which +// validates and executes a typed action without natural language translation. +// +// Shared by the MCP servers that front the dispatcher (command-executor and +// the Copilot CLI plugin) so both build the same command and read the same +// discovery data the same way. + +import type { + ActionInfo, + AgentSchemaInfo, + AgentSubSchemaInfo, + DispatcherStatus, +} from "../dispatcher.js"; + +/** A typed action to run without translation. */ +export type DirectActionRequest = { + /** Exact sub-schema name from discovery, e.g. "desktop.desktop-taskbar". */ + schemaName: string; + actionName: string; + parameters?: Record | undefined; + /** + * The user's phrasing of this exact request, when there is one. The + * dispatcher stores it as a translation for this action, so a later + * request phrased the same way resolves without an LLM call. It does not + * change what this call runs, but a phrase that does not match the action + * mis-translates future requests - omit it rather than guess. + */ + naturalLanguage?: string | undefined; +}; + +// Schema and action names are identifiers; sub-schema names add a dot (and +// occasionally a dash). Anything else could add tokens to the command line. +const namePattern = /^[A-Za-z0-9_.-]+$/; + +/** + * The dispatcher's command tokenizer reads a quoted token up to the next + * matching quote and strips only the outer quotes - it does not unescape + * anything inside. So a value is safe to quote with a quote character the + * value itself does not contain, and cannot be quoted at all when it contains + * both. + */ +function quoteCommandValue(value: string): string | undefined { + if (!value.includes("'")) { + return `'${value}'`; + } + if (!value.includes('"')) { + return `"${value}"`; + } + return undefined; +} + +function checkName(kind: string, name: string): void { + if (!namePattern.test(name)) { + throw new Error(`Invalid ${kind} '${name}'`); + } +} + +/** + * Build the `@action ` command that dispatches a + * typed action directly. Throws when a name could not appear in a real + * schema, so a caller-supplied name can never inject extra command tokens. + */ +export function buildActionCommand(request: DirectActionRequest): string { + checkName("schema name", request.schemaName); + checkName("action name", request.actionName); + + const parts = ["@action", request.schemaName, request.actionName]; + const parameters = request.parameters; + if (parameters !== undefined && Object.keys(parameters).length > 0) { + // \u0027 is a legal JSON escape for an apostrophe, so replacing it + // keeps the value parseable and leaves no quote that would end the + // token early. + const json = JSON.stringify(parameters).replaceAll("'", "\\u0027"); + parts.push("--parameters", `'${json}'`); + } + + const naturalLanguage = request.naturalLanguage?.trim(); + if (naturalLanguage) { + const quoted = quoteCommandValue(naturalLanguage); + // A phrase containing both quote characters cannot survive the + // tokenizer intact. Seeding the cache with a mangled phrase is worse + // than not seeding it, so drop the flag instead. + if (quoted !== undefined) { + parts.push("--naturalLanguage", quoted); + } + } + return parts.join(" "); +} + +/** + * Drop the sub-schemas whose actions this session cannot execute, plus any + * agent left with nothing callable. `getAgentSchemas` reports every installed + * schema; only `getStatus` says which ones are enabled, keyed by the same + * sub-schema name. A sub-schema missing from the status is kept - unknown is + * not disabled. + * + * Uses `actionActive`, which mirrors the check `@action` itself makes. + * `active` is not usable here: it is also true when only the agent's commands + * are enabled, which would list actions that then refuse to run. Older agent + * servers do not report `actionActive`, so fall back to `active` there. + */ +export function filterActiveAgentSchemas( + schemas: AgentSchemaInfo[], + status: DispatcherStatus, +): AgentSchemaInfo[] { + const activeBySchemaName = new Map(); + for (const agent of status.agents) { + activeBySchemaName.set( + agent.name.toLowerCase(), + agent.actionActive ?? agent.active, + ); + } + + const result: AgentSchemaInfo[] = []; + for (const agent of schemas) { + const subSchemas = agent.subSchemas.filter( + (subSchema) => + activeBySchemaName.get(subSchema.schemaName.toLowerCase()) !== + false, + ); + if (subSchemas.length === 0) { + continue; + } + result.push({ ...agent, subSchemas }); + } + return result; +} + +export type ActionLookup = { + subSchema: AgentSubSchemaInfo; + action: ActionInfo; +}; + +/** Find the sub-schema that declares `actionName`, ignoring case. */ +export function findActionSubSchema( + agent: AgentSchemaInfo, + actionName: string, +): ActionLookup | undefined { + const needle = actionName.toLowerCase(); + for (const subSchema of agent.subSchemas) { + const action = subSchema.actions.find( + (candidate) => candidate.name.toLowerCase() === needle, + ); + if (action !== undefined) { + return { subSchema, action }; + } + } + return undefined; +} + +/** Every action name the agent exposes, for "did you mean" style messages. */ +export function getAgentActionNames(agent: AgentSchemaInfo): string[] { + return agent.subSchemas.flatMap((subSchema) => + subSchema.actions.map((action) => action.name), + ); +}