Repository navigation
Engines keep each organization apart: Claude plan queries, and every harness on its run's own gateway token - #1898
Merged
Merged
Conversation
davidmckayv
requested review from
MikeRyanDev,
dustywusty,
guidovizoso,
mxmzb,
samjulien and
tylerslaton
as code owners
November 19, 2026 21:26
davidmckayv
force-pushed
the
feat/one-org-engines
branch
from
November 20, 2026 13:40
851ad3e to
c40b773
Compare
…ization too One Claude engine serves every organization of an OpenBot. The engine named a Claude plan's query by (Bot, thread) alone, so the same thread id for a Bot of the same id in two organizations went to one query: one transcript and one set of turns for both. The server now names the run's organization in forwardedProps.openbotOrganization, and the engine joins it to the query's name. A run with no organization keeps the name it always had, so a thread that already has a query keeps it.
…en the server sends one One harness serves every organization of an OpenBot, and the key in its environment is the deployment's gateway token, which means one organization. Where more than one organization is allowed the server refuses that token, so every model call has to say which organization it is for. The server now sends the harness it runs the token it mints for each run, in the run's organization (forwardedProps.openbotModelRoute), as it already does for the LangGraph TypeScript engine. The harness calls an OpenAI or Anthropic model with that token as its key; a plan's route carries none, and a run without a route keeps the key in the environment. The route and openbotOrganization are server props, kept out of graph state and never passed to the model. fake_provider can now write down the key each request presented.
…s may not carry one The compose probe stands in for the run's settings with an object that has only a model and an effort. Read the token as _retries reads send_once, so a settings object without it means no token.
…s organization One harness serves every organization of an OpenBot, and the key in its environment is the deployment's gateway token, which names one organization. Where more than one organization is allowed the gateway refuses that token, so the server sends each run a token of its own (forwardedProps.openbotModelRoute), and every harness now calls the gateway with it for that run. - Python harnesses (CrewAI, LlamaIndex, Agno, LangChain, Google ADK, Pydantic AI, Microsoft Agent Framework, Strands, AG2, Langroid): shared/run_model_key.py. RunModelKey reads the token from the run's body, takes the server's props out so no framework holds the token, and keeps it for the run; every httpx and httpx2 request to the gateway's path then carries it in place of the environment's key. The frameworks build their model clients once, at start, so the HTTP request is the one place they all meet. - Claude Agent SDK: Claude Code makes the calls from a process that lives across runs, so a query on the gateway is pointed back at the harness on a secret loopback path, which passes each call on with the newest token a run of that thread brought (src/run_model_proxy.py). - Mastra: the token arrives as the OpenBot model route context entry, and a run with one answers on a model that calls with it. - Antigravity reaches Gemini with Google's own key, never the gateway; its server props are still taken out. Only to the gateway: a request to any other address keeps its own key, and a run with no token keeps the environment's. This replaces the LangGraph harness's own reading of the token in the previous commit. Each harness has a test that calls a local stand-in for the gateway, at the gateway's paths, with a run that brings a token and one that does not, on both wires; each fails on main.
…alls on to the gateway The image had no httpx: the harness's tests install it for themselves, the image does not. Found by the live run of the built image through the gateway.
… in step httpx in the Claude Agent SDK harness is not a model client, so it is not pinned exactly: the model lock reads every exact pin in a harness's requirements as one. The compose probe's stand-in FastAPI takes add_middleware, which the LangGraph harness now calls.
davidmckayv
force-pushed
the
feat/one-org-engines
branch
from
November 20, 2026 14:03
c40b773 to
9e3bc65
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One set of engines serves every organization of an OpenBot. Two things the engines here named or spent for the whole deployment are now per organization.
A Claude plan's query, per organization (LangGraph TypeScript engine)
The engine named a Claude plan's query by (Bot, thread) alone, so the same thread id for a Bot of the same id in two organizations went to one query: one transcript and one set of turns for both.
forwardedProps.openbotOrganizationwhen the server sends one.claudeThreadId(botId, threadId, organization?)joins the organization to the query's name.Every harness calls the model gateway with its run's own token
The key in a harness's environment is the deployment's gateway token, which names one organization. Where more than one organization is allowed the server refuses that token, and sends each run a token of its own, minted in its organization (
forwardedProps.openbotModelRoute; on Mastra theOpenBot model routecontext entry). Every harness now calls the gateway with it for that run.shared/run_model_key.py.RunModelKeyreads the token from the run's body, takes the server's props out so no framework ever holds the token, and keeps it for the run in a context variable; everyhttpxandhttpx2request to the gateway then carries it in place of the environment's key. The frameworks build their model clients once, at start, so the HTTP request is where they all meet.agent-claude-sdk/src/run_model_proxy.py).Only to the gateway (
/api/model-provider/): a request to any other address keeps its own key, and a run with no token keeps the environment's. A server that sends no token, OpenBot Desktop among them, sees no change.Tests
Each fails on main and passes here.
test_a_run_calls_the_gateway_with_its_own_key(OpenAI and Anthropic wires): the harness's real app, a run with a token and a run without, against a local stand-in for the gateway at its own paths (fake_providernow serves them and records the key each request presented). The run with a token calls with it; the run without calls with the environment's key; neither the token nor the organization reaches the model.agent-claude-sdk/tests/test_run_model_key.py: the real Claude Code this harness ships, through the harness on a real port, against the gateway stand-in. The run's token, then a later run's newer token on the same thread, then the environment's key for a run with none. A secret it never handed out is answered 404 and spends nothing.agent-mastra: a run with the context entry calls with the token, one without calls with the environment's key, on both wires.agent-langgraph-ts/test/claude-plan.test.ts: the same thread in two organizations is two Claude queries; with no organization the name is unchanged.Run locally: every Python harness's whole suite passes (agent-langgraph-agui: all but the two Docker-only token-store tests); agent-mastra 37 pass; agent-langgraph-ts 63 pass, 1 skipped.