Skip to content

Make LangChain the default harness, and let a framework Bot call tools back - #33

Open
davidmckayv wants to merge 1 commit into
feat/server-side-toolsfrom
feat/langchain-default
Open

Make LangChain the default harness, and let a framework Bot call tools back#33
davidmckayv wants to merge 1 commit into
feat/server-side-toolsfrom
feat/langchain-default

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

Stacked on #31. Review that first; this targets its branch.

What this changes

agent-bot hand-writes the AG-UI protocol over the raw OpenAI SDK, 235 lines, with its own tool loop. It is a fine reference and a bad thing to build a deployment on, and it was the default: MANAGED_AGENT_AG_UI_URL pointed at it on 4200, so the Bot we ship as the example of "bring your own" was the hand-rolled one while the real framework sat unused on 4201.

The default is now agent-langgraph: real @langchain/langgraph, running a real loop. agent-bot stays in the tree as the reference it always was.

This also closes the regression #31 disclosed. A framework Bot runs its own loop in its own process, which is the honest shape, but it must not have a route to a vendor that goes around this deployment. So it calls back to /api/agent-tools/call, which goes through the same plugin store as everything else: same grant check, same policy, same audit row. Remote Bots have MCP tools again.

The runtime sends a remote Bot its granted tools on every run rather than at configuration time, so a grant added or revoked applies to the next run instead of after a restart.

Taken from CopilotKit/openkai (bc246d4), minus its visitor demo, with OPENKAI_/x-openkai- renamed and the refusal marker from #31 applied to this path too.

Two defects it also fixes

  • BOT_MODEL arrived from compose as an empty string, which is a value, so the agent asked its provider for a model named "" and every run died with "you must provide a model parameter". That reads as a broken Bot rather than as missing configuration. Compose now defaults it.
  • The run emitted its answer before the tool calls that informed it, so the transcript drew a Bot that spoke first and worked afterwards. Tool events are sent as they happen, each stretch of prose owning its own message id.

Where it runs

  • New state that outlives a request? None. The token is configuration, the tool list is read per run.
  • What happens on the second replica? Nothing. The Bot calls the load balancer, any replica can serve it, and the grant and policy are read from Postgres on that call.
  • Anything serialised? No.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? No new listener. One new route on the existing API, mounted only when AGENT_TOOL_TOKEN is set, so a deployment that has not configured it refuses rather than accepting anybody who can reach the port.

Boundary and audit

  • The callback goes through pluginStore.callTool: grant, then policy, then audit, then the vendor.
  • Refusals return the marker and are recorded, not thrown.
  • 401 without the token, verified.

Known weakness, and the next PR

The actor on this path is not trustworthy yet. /api/agent-tools/call reads actorId from the request body and falls back to "agent". Two consequences, both visible in the audit row I captured below:

  1. A remote Bot's tool call cannot be attributed to the person who asked. The trail says agent.
  2. Anything holding the one deployment-wide token can name any actor and any Bot, so the trail is forgeable by any service that can reach the route.

The trail is the product, so a forgeable trail is worse than a thin one. The next PR replaces the shared secret with a per-agent token plus a signed, expiring run assertion that names the Bot and the person, and refuses a call whose assertion does not match the token presenting it. Landing immediately after this one.

Proof

Driven in Chrome. agent-langgraph rebuilt and started on 4201, AGENT_TOOL_TOKEN set, search_notes granted to Risk Analyst through /admin/plugins.

  • Asked Risk Analyst for the receipt threshold. It called the tool and answered: "meals under $75 need no receipt. Anything above $75 requires a receipt." The tool line drew before the answer, which is the ordering fix.
  • Audit row: mcp.call_succeeded, notes/search_notes, bot risk-analyst, allowed true. Actor agent, which is the weakness above.
  • Route auth: 401 with no token, 400 with the token and no body.

format:check, lint, typecheck clean. 665 pass, 5 skip, 0 fail.

`agent-bot` hand-writes the AG-UI protocol and leaves the tool loop to
whatever is watching. That is a fine reference and a bad thing to build
a deployment on, so the default teammate endpoint is now
`agent-langgraph` on 4201: a real framework, running a real loop.

The loop moves into the agent, and stays governed. A framework Bot that
called an MCP server directly would be a Bot that walked around the
grant, the policy and the audit row, which are the product. So it calls
back to `/api/agent-tools/call`, authenticated by a shared secret
because the caller is a service with no person behind it, and that
route goes through the same plugin store as everything else. No secret
configured means the route does not exist and the Bot is told so, rather
than a deployment quietly accepting anybody who can reach the port.

The runtime now sends a remote agent the tools its Bot was granted, on
every run rather than configured once, so a grant added or revoked
applies to the next run and not after a restart.

Two defects fixed on the way. `BOT_MODEL` arrived from compose as an
empty string, which is a value, so the agent asked its provider for a
model named "" and every run died with "you must provide a model
parameter" — read as a broken Bot rather than missing configuration.
And the run emitted its answer before the tool calls that informed it,
so the surface drew a Bot that spoke first and worked afterwards; tool
events are now sent as they happen, with each stretch of prose owning
its own message id.

Driven in a browser: the LangGraph Bot answered a question about the
expense policy, the transcript drew the tool line and then the answer
with its source link, and the audit row names risk-analyst calling
through the deployment rather than the vendor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant