Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ the [workspaces & quota operations guide](./packages/agent/README.md#workspaces-
and the [durable-workflow how-to](./packages/agent/README.md#durable-workflows-persist-resume-recover)
(persisting `chatId` across job/queue boundaries, drop & timeout recovery)
in the agent README;
[surfaces and authentication modes](./packages/provider/README.md#the-two-surfaces)
[named providers and authentication modes](./packages/provider/README.md#named-providers-and-the-two-wire-protocols)
and the [enterprise governance & security reference](./packages/provider/README.md#enterprise-governance--security)
(data flow, credential isolation, audit capture, required permissions)
in the provider README.
Expand Down
80 changes: 62 additions & 18 deletions packages/provider/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,29 +55,73 @@ Requires Node ≥ 22, `ai` v7, and a Coder deployment with **AI Gateway enabled*
(stable since Coder **v2.29**, GA in v2.30, on by default in v2.34; requires the
AI Governance Add-On).

## The two surfaces
## Named providers and the two wire protocols

AI Gateway exposes **two provider-namespaced surfaces** on your deployment, and
routing is **by URL path, not by model id** — so each surface reaches a fixed set
of upstreams:
Your Coder admins define AI Gateway **providers** — named routes on the
deployment (`/api/v2/aibridge/<name>/v1/…`), each speaking one of **two wire
protocols** determined by its admin-configured type:

| Surface | Reaches | Accessor |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| **OpenAI-compatible** (`/api/v2/aibridge/openai/v1`) | OpenAI, Azure, Google, OpenRouter, Vercel, openai-compat — and **Copilot** (incl. Claude via Copilot) | `coder.openai(id)` / `coder.chat(id)` |
| **Anthropic-compatible** (`/api/v2/aibridge/anthropic`) | **native Claude** + **Bedrock-hosted Claude** | `coder.anthropic(id)` / `coder.messages(id)` |
| Wire protocol | Provider types behind it | Default name |
| ------------------------ | ------------------------------------------------------------------------------- | ------------ |
| **OpenAI-compatible** | `openai`, `azure`, `google`, `copilot`, `openai-compat`, `openrouter`, `vercel` | `openai` |
| **Anthropic-compatible** | `anthropic` (native Claude), `bedrock` (Bedrock-hosted Claude) | `anthropic` |

The bare call `coder(modelId)` picks a surface by heuristic — model ids starting
with `claude`/`anthropic` go to the Anthropic surface, everything else to the
OpenAI surface. Use the explicit accessors to override (e.g. to reach Claude
through a Copilot-typed provider on the OpenAI surface):
Routing is **by URL path, not by model id** — the provider name in the URL
decides which upstream handles the request. `createCoder` fronts the default
`openai` / `anthropic` pair: the bare call `coder(modelId)` picks between them
by heuristic (model ids starting with `claude`/`anthropic` go to the
Anthropic-protocol provider, everything else to the OpenAI-protocol one), and
the explicit accessors override it (e.g. to reach Claude through a
Copilot-typed provider on the OpenAI protocol):

```ts
coder("gpt-4o"); // → OpenAI surface
coder("claude-sonnet-4-6"); // → Anthropic surface (heuristic)
coder.openai("claude-sonnet-4"); // → OpenAI surface (e.g. Copilot)
coder.anthropic("claude-opus-4-5"); // → Anthropic surface (explicit)
coder("gpt-4o"); // → `openai` provider
coder("claude-sonnet-4-6"); // → `anthropic` provider (heuristic)
coder.openai("claude-sonnet-4"); // → `openai` provider (e.g. Copilot)
coder.anthropic("claude-opus-4-5"); // → `anthropic` provider (explicit)
```

### Custom-named providers

Provider names are admin-chosen (matching `^[a-z0-9]+(-[a-z0-9]+)*$`), so a
deployment may expose e.g. an Azure-backed `azure-openai` next to a
Bedrock-backed `anthropic-bedrock`. Reach them in two ways:

**Sub-provider accessors** — `openaiProvider(name)` / `anthropicProvider(name)`
return a full sub-provider bound to that gateway provider, so one
`createCoder` instance can target any number of providers. Pick the accessor
matching the provider's wire protocol:

```ts
const azure = coder.openaiProvider("azure-openai"); // OpenAI-compatible type
const bedrock = coder.anthropicProvider("anthropic-bedrock"); // Anthropic-compatible type

await generateText({ model: azure("gpt-4o"), prompt: "Hi" });
await generateText({ model: bedrock("claude-sonnet-4-6"), prompt: "Hi" });
```

A name outside the gateway's grammar throws the AI SDK's
`InvalidArgumentError` at accessor time (such a name can never be registered);
a well-formed name that is not configured on your deployment fails at request
time with the gateway's 404.

**Re-pointing the defaults** — when your deployment simply names its one
OpenAI/Anthropic pair differently, override the names once and keep using the
bare call and the `openai` / `anthropic` accessors:

```ts
const renamed = createCoder({
baseURL: "https://coder.example.com",
apiKey: process.env.CODER_API_TOKEN!,
providers: { openai: "azure-openai", anthropic: "anthropic-bedrock" },
});
```

**Provider names come from your platform team.** Discovery is admin-only
server-side: `GET /api/v2/ai/providers` returns `403` for regular users, and
the models endpoint does not attribute models to providers. Ask your Coder
admins which provider names your deployment defines.

Model ids are passed through **unchanged** to the upstream provider (no
`vendor/model` namespacing) — use whatever ids your deployment's providers accept.

Expand Down Expand Up @@ -119,15 +163,15 @@ createCoder({
| `coderToken` | `string` | — | Enables BYOK mode; sent in `X-Coder-AI-Governance-Token`. |
| `headers` | `Record<string,string>` | — | Extra headers merged into every request. |
| `aiGatewayPath` | `string` | `/api/v2/aibridge` | Override if your deployment uses a different mount path. |
| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Override the admin-configured provider path segments. |
| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Re-point the default pair at differently-named providers. |
| `fetch` | `typeof fetch` | global `fetch` | Custom fetch (testing / middleware). |

## Enterprise governance & security

Reference for security reviewers evaluating this package. The boundary between
the two kinds of claims below matters: **client behavior** (what this package
puts on the wire) is verifiable in [`src/provider.ts`](./src/provider.ts) —
~175 lines with no dependencies beyond the official AI SDK provider packages —
~250 lines with no dependencies beyond the official AI SDK provider packages —
while **gateway behavior** (key custody, audit capture, retention) is a
property of your Coder deployment, enforced server-side regardless of what any
client does, and documented in the
Expand Down
120 changes: 88 additions & 32 deletions packages/provider/src/provider.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,23 @@
import { type AnthropicProvider, createAnthropic } from "@ai-sdk/anthropic";
import { createOpenAICompatible, type OpenAICompatibleProvider } from "@ai-sdk/openai-compatible";
import { type EmbeddingModelV4, type LanguageModelV4, NoSuchModelError } from "@ai-sdk/provider";
import {
type EmbeddingModelV4,
InvalidArgumentError,
type LanguageModelV4,
NoSuchModelError,
} from "@ai-sdk/provider";

/** Default mount path of AI Gateway on a Coder deployment. */
const DEFAULT_AI_GATEWAY_PATH = "/api/v2/aibridge";
/** Default provider path segments (the admin-configured provider names). */
const DEFAULT_OPENAI_PROVIDER = "openai";
const DEFAULT_ANTHROPIC_PROVIDER = "anthropic";
/**
* AI Gateway's provider-name grammar: lowercase alphanumeric segments
* separated by single hyphens. Names outside this grammar can never be
* registered on a deployment, so they are rejected client-side.
*/
const PROVIDER_NAME_PATTERN = /^[a-z0-9]+(-[a-z0-9]+)*$/;
/** Header that carries the Coder token to AI Gateway in bring-your-own-key mode. */
const CODER_TOKEN_HEADER = "X-Coder-AI-Governance-Token";

Expand Down Expand Up @@ -72,6 +83,23 @@ export interface CoderProvider {
chat(modelId: string): LanguageModelV4;
/** Shorthand for an {@link CoderProvider.anthropic} messages model. */
messages(modelId: string): LanguageModelV4;
/**
* An OpenAI-compatible sub-provider bound to the *named* gateway provider
* (`<aiGatewayPath>/<name>/v1`). Use it to reach admin-defined providers
* beyond the default pair, e.g.
* `coder.openaiProvider("azure-openai")("gpt-4o")`. Throws the AI SDK's
* `InvalidArgumentError` when `name` does not match the gateway's
* provider-name grammar (`^[a-z0-9]+(-[a-z0-9]+)*$`); a well-formed name
* that is not configured on the deployment fails at request time with the
* gateway's 404.
*/
openaiProvider(name: string): OpenAICompatibleProvider;
/**
* An Anthropic-compatible sub-provider bound to the *named* gateway
* provider, e.g. `coder.anthropicProvider("anthropic-bedrock")("claude-sonnet-4-6")`.
* Same name validation as {@link CoderProvider.openaiProvider}.
*/
anthropicProvider(name: string): AnthropicProvider;
/**
* Text embeddings are not supported: AI Gateway does not yet intercept
* `/v1/embeddings`, so this always throws {@link NoSuchModelError} instead of
Expand Down Expand Up @@ -112,6 +140,25 @@ function unsupportedEmbeddingModel(modelId: string): EmbeddingModelV4 {
});
}

/**
* Fail fast at accessor time: a name outside the gateway's provider-name
* grammar can never be registered on a deployment, so a request to it would
* always die with a confusing 404. Same philosophy as the embeddings guard
* (https://github.com/coder/ai-sdk/issues/69).
*/
function assertValidProviderName(name: string): void {
if (!PROVIDER_NAME_PATTERN.test(name)) {
throw new InvalidArgumentError({
argument: "name",
message:
`Invalid AI Gateway provider name "${name}": provider names are ` +
`lowercase alphanumeric segments separated by single hyphens ` +
`(${PROVIDER_NAME_PATTERN}). Ask your Coder admins for the provider ` +
`names configured on your deployment.`,
});
}
}

/**
* Create a {@link CoderProvider} that routes Vercel AI SDK calls through a Coder
* deployment's AI Gateway (formerly "AI Bridge"). AI Gateway exposes two
Expand Down Expand Up @@ -141,15 +188,6 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider {

const deployment = trimTrailingSlash(settings.baseURL);
const gatewayPath = settings.aiGatewayPath ?? DEFAULT_AI_GATEWAY_PATH;
const openaiName = settings.providers?.openai ?? DEFAULT_OPENAI_PROVIDER;
const anthropicName = settings.providers?.anthropic ?? DEFAULT_ANTHROPIC_PROVIDER;

// Both sub-providers append their route to a baseURL that INCLUDES `/v1`:
// openai-compatible POSTs `${baseURL}/chat/completions`, and @ai-sdk/anthropic
// POSTs `${baseURL}/messages`. AI Gateway's intercepted routes are
// `/aibridge/<name>/v1/chat/completions` and `/aibridge/<name>/v1/messages`.
const openaiBaseURL = `${deployment}${gatewayPath}/${openaiName}/v1`;
const anthropicBaseURL = `${deployment}${gatewayPath}/${anthropicName}/v1`;

// BYOK mode: the Coder token authenticates via a dedicated header and `apiKey`
// carries the upstream key. Centralized mode (default): `apiKey` is the Coder
Expand All @@ -160,29 +198,45 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider {
...settings.headers,
};

const openai = createOpenAICompatible({
name: "coder.openai",
baseURL: openaiBaseURL,
apiKey: settings.apiKey, // → `Authorization: Bearer <apiKey>`
headers,
fetch: settings.fetch,
includeUsage: true,
});
// Both sub-provider kinds append their route to a baseURL that INCLUDES `/v1`:
// openai-compatible POSTs `${baseURL}/chat/completions`, and @ai-sdk/anthropic
// POSTs `${baseURL}/messages`. AI Gateway's intercepted routes are
// `/aibridge/<name>/v1/chat/completions` and `/aibridge/<name>/v1/messages`.
const providerBaseURL = (name: string): string => `${deployment}${gatewayPath}/${name}/v1`;

// `coder.openai` is public, so its embedding accessors must fail fast too —
// otherwise they bypass the top-level guard and hit the gateway 404.
openai.embeddingModel = unsupportedEmbeddingModel;
openai.textEmbeddingModel = unsupportedEmbeddingModel;

const anthropic = createAnthropic({
name: "coder.anthropic",
baseURL: anthropicBaseURL,
// Centralized: send the Coder token via `Authorization: Bearer` (the
// documented path). BYOK: send the upstream key via `x-api-key`.
...(byok ? { apiKey: settings.apiKey } : { authToken: settings.apiKey }),
headers,
fetch: settings.fetch,
});
const openaiProvider = (name: string): OpenAICompatibleProvider => {
assertValidProviderName(name);
const provider = createOpenAICompatible({
name: `coder.${name}`,
baseURL: providerBaseURL(name),
apiKey: settings.apiKey, // → `Authorization: Bearer <apiKey>`
headers,
fetch: settings.fetch,
includeUsage: true,
});
// Sub-providers are public, so their embedding accessors must fail fast
// too — otherwise they bypass the top-level guard and hit the gateway 404.
provider.embeddingModel = unsupportedEmbeddingModel;
provider.textEmbeddingModel = unsupportedEmbeddingModel;
return provider;
};

const anthropicProvider = (name: string): AnthropicProvider => {
assertValidProviderName(name);
return createAnthropic({
name: `coder.${name}`,
baseURL: providerBaseURL(name),
// Centralized: send the Coder token via `Authorization: Bearer` (the
// documented path). BYOK: send the upstream key via `x-api-key`.
...(byok ? { apiKey: settings.apiKey } : { authToken: settings.apiKey }),
headers,
fetch: settings.fetch,
});
};

// The default surfaces are ordinary named sub-providers — one code path.
const openai = openaiProvider(settings.providers?.openai ?? DEFAULT_OPENAI_PROVIDER);
const anthropic = anthropicProvider(settings.providers?.anthropic ?? DEFAULT_ANTHROPIC_PROVIDER);

const languageModel = (modelId: string): LanguageModelV4 =>
isAnthropicModelId(modelId) ? anthropic(modelId) : openai(modelId);
Expand All @@ -193,6 +247,8 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider {
anthropic,
chat: (modelId: string): LanguageModelV4 => openai(modelId),
messages: (modelId: string): LanguageModelV4 => anthropic(modelId),
openaiProvider,
anthropicProvider,
textEmbeddingModel: unsupportedEmbeddingModel,
});
}
Expand Down
90 changes: 89 additions & 1 deletion packages/provider/test/provider.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { NoSuchModelError } from "@ai-sdk/provider";
import { InvalidArgumentError, NoSuchModelError } from "@ai-sdk/provider";
import { generateText } from "ai";
import { describe, expect, it } from "vitest";
import { createCoder, isAnthropicModelId } from "../src/index.js";
Expand Down Expand Up @@ -174,6 +174,94 @@ describe("createCoder URL construction", () => {
});
});

describe("createCoder sub-provider accessors", () => {
it("openaiProvider(name) targets the named gateway provider on the OpenAI protocol", async () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch });
await trigger(coder.openaiProvider("azure-openai")("gpt-4o"));
expect(calls).toHaveLength(1);
expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/azure-openai/v1/chat/completions`);
expect(calls[0]!.headers.get("authorization")).toBe(`Bearer ${TOKEN}`);
expect(calls[0]!.body?.model).toBe("gpt-4o");
});

it("anthropicProvider(name) targets the named gateway provider on the Anthropic protocol", async () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch });
await trigger(coder.anthropicProvider("anthropic-bedrock")("claude-sonnet-4-6"));
expect(calls).toHaveLength(1);
expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/anthropic-bedrock/v1/messages`);
expect(calls[0]!.headers.get("authorization")).toBe(`Bearer ${TOKEN}`);
expect(calls[0]!.body?.model).toBe("claude-sonnet-4-6");
});

it("honors a custom aiGatewayPath", async () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({
baseURL: BASE,
apiKey: TOKEN,
aiGatewayPath: "/api/v2/ai-gateway",
fetch,
});
await trigger(coder.openaiProvider("azure-openai")("gpt-4o"));
expect(calls[0]!.url).toBe(`${BASE}/api/v2/ai-gateway/azure-openai/v1/chat/completions`);
});

it("openaiProvider with the default name behaves exactly like the default surface", async () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch });
await trigger(coder.openaiProvider("openai")("gpt-4o"));
await trigger(coder.openai("gpt-4o"));
expect(calls).toHaveLength(2);
expect(calls[0]!.url).toBe(calls[1]!.url);
expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/openai/v1/chat/completions`);
});

it("rejects names outside the gateway grammar with InvalidArgumentError, before any request", () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch });

for (const bad of ["Azure-OpenAI", "azure_openai", "-azure", "azure-", "a--b", "a/b", ""]) {
for (const accessor of [
() => coder.openaiProvider(bad),
() => coder.anthropicProvider(bad),
]) {
let error: unknown;
try {
accessor();
} catch (e) {
error = e;
}
expect(InvalidArgumentError.isInstance(error)).toBe(true);
expect((error as InvalidArgumentError).argument).toBe("name");
expect((error as InvalidArgumentError).message).toContain(`"${bad}"`);
}
}

expect(calls).toHaveLength(0);
});

it("named OpenAI-protocol sub-providers fail fast on embeddings too", () => {
const { fetch, calls } = capturingFetch();
const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch });

let error: unknown;
try {
coder.openaiProvider("azure-openai").textEmbeddingModel("text-embedding-3-small");
} catch (e) {
error = e;
}
expect(NoSuchModelError.isInstance(error)).toBe(true);
expect(calls).toHaveLength(0);
});

it("validates renamed default providers against the same grammar at createCoder time", () => {
expect(() =>
createCoder({ baseURL: BASE, apiKey: TOKEN, providers: { openai: "Bad_Name" } }),
).toThrow(/Invalid AI Gateway provider name/);
});
});

describe("createCoder validation", () => {
it("throws when baseURL is missing", () => {
expect(() => createCoder({ baseURL: "", apiKey: TOKEN })).toThrow(/baseURL/);
Expand Down