diff --git a/README.md b/README.md index c632b5b..6bd2d11 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ the [workspaces & quota operations guide](./packages/agent/README.md#workspaces- and the [durable-workflow how-to](./packages/agent/README.md#durable-workflows-persist-resume-recover) (persisting `chatId` across job/queue boundaries, drop & timeout recovery) in the agent README; -[surfaces and authentication modes](./packages/provider/README.md#the-two-surfaces) +[named providers and authentication modes](./packages/provider/README.md#named-providers-and-the-two-wire-protocols) and the [enterprise governance & security reference](./packages/provider/README.md#enterprise-governance--security) (data flow, credential isolation, audit capture, required permissions) in the provider README. diff --git a/packages/provider/README.md b/packages/provider/README.md index 21d0b42..98442cc 100644 --- a/packages/provider/README.md +++ b/packages/provider/README.md @@ -55,29 +55,73 @@ Requires Node ≥ 22, `ai` v7, and a Coder deployment with **AI Gateway enabled* (stable since Coder **v2.29**, GA in v2.30, on by default in v2.34; requires the AI Governance Add-On). -## The two surfaces +## Named providers and the two wire protocols -AI Gateway exposes **two provider-namespaced surfaces** on your deployment, and -routing is **by URL path, not by model id** — so each surface reaches a fixed set -of upstreams: +Your Coder admins define AI Gateway **providers** — named routes on the +deployment (`/api/v2/aibridge//v1/…`), each speaking one of **two wire +protocols** determined by its admin-configured type: -| Surface | Reaches | Accessor | -| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------------------------- | -| **OpenAI-compatible** (`/api/v2/aibridge/openai/v1`) | OpenAI, Azure, Google, OpenRouter, Vercel, openai-compat — and **Copilot** (incl. Claude via Copilot) | `coder.openai(id)` / `coder.chat(id)` | -| **Anthropic-compatible** (`/api/v2/aibridge/anthropic`) | **native Claude** + **Bedrock-hosted Claude** | `coder.anthropic(id)` / `coder.messages(id)` | +| Wire protocol | Provider types behind it | Default name | +| ------------------------ | ------------------------------------------------------------------------------- | ------------ | +| **OpenAI-compatible** | `openai`, `azure`, `google`, `copilot`, `openai-compat`, `openrouter`, `vercel` | `openai` | +| **Anthropic-compatible** | `anthropic` (native Claude), `bedrock` (Bedrock-hosted Claude) | `anthropic` | -The bare call `coder(modelId)` picks a surface by heuristic — model ids starting -with `claude`/`anthropic` go to the Anthropic surface, everything else to the -OpenAI surface. Use the explicit accessors to override (e.g. to reach Claude -through a Copilot-typed provider on the OpenAI surface): +Routing is **by URL path, not by model id** — the provider name in the URL +decides which upstream handles the request. `createCoder` fronts the default +`openai` / `anthropic` pair: the bare call `coder(modelId)` picks between them +by heuristic (model ids starting with `claude`/`anthropic` go to the +Anthropic-protocol provider, everything else to the OpenAI-protocol one), and +the explicit accessors override it (e.g. to reach Claude through a +Copilot-typed provider on the OpenAI protocol): ```ts -coder("gpt-4o"); // → OpenAI surface -coder("claude-sonnet-4-6"); // → Anthropic surface (heuristic) -coder.openai("claude-sonnet-4"); // → OpenAI surface (e.g. Copilot) -coder.anthropic("claude-opus-4-5"); // → Anthropic surface (explicit) +coder("gpt-4o"); // → `openai` provider +coder("claude-sonnet-4-6"); // → `anthropic` provider (heuristic) +coder.openai("claude-sonnet-4"); // → `openai` provider (e.g. Copilot) +coder.anthropic("claude-opus-4-5"); // → `anthropic` provider (explicit) ``` +### Custom-named providers + +Provider names are admin-chosen (matching `^[a-z0-9]+(-[a-z0-9]+)*$`), so a +deployment may expose e.g. an Azure-backed `azure-openai` next to a +Bedrock-backed `anthropic-bedrock`. Reach them in two ways: + +**Sub-provider accessors** — `openaiProvider(name)` / `anthropicProvider(name)` +return a full sub-provider bound to that gateway provider, so one +`createCoder` instance can target any number of providers. Pick the accessor +matching the provider's wire protocol: + +```ts +const azure = coder.openaiProvider("azure-openai"); // OpenAI-compatible type +const bedrock = coder.anthropicProvider("anthropic-bedrock"); // Anthropic-compatible type + +await generateText({ model: azure("gpt-4o"), prompt: "Hi" }); +await generateText({ model: bedrock("claude-sonnet-4-6"), prompt: "Hi" }); +``` + +A name outside the gateway's grammar throws the AI SDK's +`InvalidArgumentError` at accessor time (such a name can never be registered); +a well-formed name that is not configured on your deployment fails at request +time with the gateway's 404. + +**Re-pointing the defaults** — when your deployment simply names its one +OpenAI/Anthropic pair differently, override the names once and keep using the +bare call and the `openai` / `anthropic` accessors: + +```ts +const renamed = createCoder({ + baseURL: "https://coder.example.com", + apiKey: process.env.CODER_API_TOKEN!, + providers: { openai: "azure-openai", anthropic: "anthropic-bedrock" }, +}); +``` + +**Provider names come from your platform team.** Discovery is admin-only +server-side: `GET /api/v2/ai/providers` returns `403` for regular users, and +the models endpoint does not attribute models to providers. Ask your Coder +admins which provider names your deployment defines. + Model ids are passed through **unchanged** to the upstream provider (no `vendor/model` namespacing) — use whatever ids your deployment's providers accept. @@ -119,7 +163,7 @@ createCoder({ | `coderToken` | `string` | — | Enables BYOK mode; sent in `X-Coder-AI-Governance-Token`. | | `headers` | `Record` | — | Extra headers merged into every request. | | `aiGatewayPath` | `string` | `/api/v2/aibridge` | Override if your deployment uses a different mount path. | -| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Override the admin-configured provider path segments. | +| `providers` | `{ openai?, anthropic? }` | `openai` / `anthropic` | Re-point the default pair at differently-named providers. | | `fetch` | `typeof fetch` | global `fetch` | Custom fetch (testing / middleware). | ## Enterprise governance & security @@ -127,7 +171,7 @@ createCoder({ Reference for security reviewers evaluating this package. The boundary between the two kinds of claims below matters: **client behavior** (what this package puts on the wire) is verifiable in [`src/provider.ts`](./src/provider.ts) — -~175 lines with no dependencies beyond the official AI SDK provider packages — +~250 lines with no dependencies beyond the official AI SDK provider packages — while **gateway behavior** (key custody, audit capture, retention) is a property of your Coder deployment, enforced server-side regardless of what any client does, and documented in the diff --git a/packages/provider/src/provider.ts b/packages/provider/src/provider.ts index 86f8d93..50c7010 100644 --- a/packages/provider/src/provider.ts +++ b/packages/provider/src/provider.ts @@ -1,12 +1,23 @@ import { type AnthropicProvider, createAnthropic } from "@ai-sdk/anthropic"; import { createOpenAICompatible, type OpenAICompatibleProvider } from "@ai-sdk/openai-compatible"; -import { type EmbeddingModelV4, type LanguageModelV4, NoSuchModelError } from "@ai-sdk/provider"; +import { + type EmbeddingModelV4, + InvalidArgumentError, + type LanguageModelV4, + NoSuchModelError, +} from "@ai-sdk/provider"; /** Default mount path of AI Gateway on a Coder deployment. */ const DEFAULT_AI_GATEWAY_PATH = "/api/v2/aibridge"; /** Default provider path segments (the admin-configured provider names). */ const DEFAULT_OPENAI_PROVIDER = "openai"; const DEFAULT_ANTHROPIC_PROVIDER = "anthropic"; +/** + * AI Gateway's provider-name grammar: lowercase alphanumeric segments + * separated by single hyphens. Names outside this grammar can never be + * registered on a deployment, so they are rejected client-side. + */ +const PROVIDER_NAME_PATTERN = /^[a-z0-9]+(-[a-z0-9]+)*$/; /** Header that carries the Coder token to AI Gateway in bring-your-own-key mode. */ const CODER_TOKEN_HEADER = "X-Coder-AI-Governance-Token"; @@ -72,6 +83,23 @@ export interface CoderProvider { chat(modelId: string): LanguageModelV4; /** Shorthand for an {@link CoderProvider.anthropic} messages model. */ messages(modelId: string): LanguageModelV4; + /** + * An OpenAI-compatible sub-provider bound to the *named* gateway provider + * (`//v1`). Use it to reach admin-defined providers + * beyond the default pair, e.g. + * `coder.openaiProvider("azure-openai")("gpt-4o")`. Throws the AI SDK's + * `InvalidArgumentError` when `name` does not match the gateway's + * provider-name grammar (`^[a-z0-9]+(-[a-z0-9]+)*$`); a well-formed name + * that is not configured on the deployment fails at request time with the + * gateway's 404. + */ + openaiProvider(name: string): OpenAICompatibleProvider; + /** + * An Anthropic-compatible sub-provider bound to the *named* gateway + * provider, e.g. `coder.anthropicProvider("anthropic-bedrock")("claude-sonnet-4-6")`. + * Same name validation as {@link CoderProvider.openaiProvider}. + */ + anthropicProvider(name: string): AnthropicProvider; /** * Text embeddings are not supported: AI Gateway does not yet intercept * `/v1/embeddings`, so this always throws {@link NoSuchModelError} instead of @@ -112,6 +140,25 @@ function unsupportedEmbeddingModel(modelId: string): EmbeddingModelV4 { }); } +/** + * Fail fast at accessor time: a name outside the gateway's provider-name + * grammar can never be registered on a deployment, so a request to it would + * always die with a confusing 404. Same philosophy as the embeddings guard + * (https://github.com/coder/ai-sdk/issues/69). + */ +function assertValidProviderName(name: string): void { + if (!PROVIDER_NAME_PATTERN.test(name)) { + throw new InvalidArgumentError({ + argument: "name", + message: + `Invalid AI Gateway provider name "${name}": provider names are ` + + `lowercase alphanumeric segments separated by single hyphens ` + + `(${PROVIDER_NAME_PATTERN}). Ask your Coder admins for the provider ` + + `names configured on your deployment.`, + }); + } +} + /** * Create a {@link CoderProvider} that routes Vercel AI SDK calls through a Coder * deployment's AI Gateway (formerly "AI Bridge"). AI Gateway exposes two @@ -141,15 +188,6 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider { const deployment = trimTrailingSlash(settings.baseURL); const gatewayPath = settings.aiGatewayPath ?? DEFAULT_AI_GATEWAY_PATH; - const openaiName = settings.providers?.openai ?? DEFAULT_OPENAI_PROVIDER; - const anthropicName = settings.providers?.anthropic ?? DEFAULT_ANTHROPIC_PROVIDER; - - // Both sub-providers append their route to a baseURL that INCLUDES `/v1`: - // openai-compatible POSTs `${baseURL}/chat/completions`, and @ai-sdk/anthropic - // POSTs `${baseURL}/messages`. AI Gateway's intercepted routes are - // `/aibridge//v1/chat/completions` and `/aibridge//v1/messages`. - const openaiBaseURL = `${deployment}${gatewayPath}/${openaiName}/v1`; - const anthropicBaseURL = `${deployment}${gatewayPath}/${anthropicName}/v1`; // BYOK mode: the Coder token authenticates via a dedicated header and `apiKey` // carries the upstream key. Centralized mode (default): `apiKey` is the Coder @@ -160,29 +198,45 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider { ...settings.headers, }; - const openai = createOpenAICompatible({ - name: "coder.openai", - baseURL: openaiBaseURL, - apiKey: settings.apiKey, // → `Authorization: Bearer ` - headers, - fetch: settings.fetch, - includeUsage: true, - }); + // Both sub-provider kinds append their route to a baseURL that INCLUDES `/v1`: + // openai-compatible POSTs `${baseURL}/chat/completions`, and @ai-sdk/anthropic + // POSTs `${baseURL}/messages`. AI Gateway's intercepted routes are + // `/aibridge//v1/chat/completions` and `/aibridge//v1/messages`. + const providerBaseURL = (name: string): string => `${deployment}${gatewayPath}/${name}/v1`; - // `coder.openai` is public, so its embedding accessors must fail fast too — - // otherwise they bypass the top-level guard and hit the gateway 404. - openai.embeddingModel = unsupportedEmbeddingModel; - openai.textEmbeddingModel = unsupportedEmbeddingModel; - - const anthropic = createAnthropic({ - name: "coder.anthropic", - baseURL: anthropicBaseURL, - // Centralized: send the Coder token via `Authorization: Bearer` (the - // documented path). BYOK: send the upstream key via `x-api-key`. - ...(byok ? { apiKey: settings.apiKey } : { authToken: settings.apiKey }), - headers, - fetch: settings.fetch, - }); + const openaiProvider = (name: string): OpenAICompatibleProvider => { + assertValidProviderName(name); + const provider = createOpenAICompatible({ + name: `coder.${name}`, + baseURL: providerBaseURL(name), + apiKey: settings.apiKey, // → `Authorization: Bearer ` + headers, + fetch: settings.fetch, + includeUsage: true, + }); + // Sub-providers are public, so their embedding accessors must fail fast + // too — otherwise they bypass the top-level guard and hit the gateway 404. + provider.embeddingModel = unsupportedEmbeddingModel; + provider.textEmbeddingModel = unsupportedEmbeddingModel; + return provider; + }; + + const anthropicProvider = (name: string): AnthropicProvider => { + assertValidProviderName(name); + return createAnthropic({ + name: `coder.${name}`, + baseURL: providerBaseURL(name), + // Centralized: send the Coder token via `Authorization: Bearer` (the + // documented path). BYOK: send the upstream key via `x-api-key`. + ...(byok ? { apiKey: settings.apiKey } : { authToken: settings.apiKey }), + headers, + fetch: settings.fetch, + }); + }; + + // The default surfaces are ordinary named sub-providers — one code path. + const openai = openaiProvider(settings.providers?.openai ?? DEFAULT_OPENAI_PROVIDER); + const anthropic = anthropicProvider(settings.providers?.anthropic ?? DEFAULT_ANTHROPIC_PROVIDER); const languageModel = (modelId: string): LanguageModelV4 => isAnthropicModelId(modelId) ? anthropic(modelId) : openai(modelId); @@ -193,6 +247,8 @@ export function createCoder(settings: CoderProviderSettings): CoderProvider { anthropic, chat: (modelId: string): LanguageModelV4 => openai(modelId), messages: (modelId: string): LanguageModelV4 => anthropic(modelId), + openaiProvider, + anthropicProvider, textEmbeddingModel: unsupportedEmbeddingModel, }); } diff --git a/packages/provider/test/provider.test.ts b/packages/provider/test/provider.test.ts index 67fb903..d797972 100644 --- a/packages/provider/test/provider.test.ts +++ b/packages/provider/test/provider.test.ts @@ -1,4 +1,4 @@ -import { NoSuchModelError } from "@ai-sdk/provider"; +import { InvalidArgumentError, NoSuchModelError } from "@ai-sdk/provider"; import { generateText } from "ai"; import { describe, expect, it } from "vitest"; import { createCoder, isAnthropicModelId } from "../src/index.js"; @@ -174,6 +174,94 @@ describe("createCoder URL construction", () => { }); }); +describe("createCoder sub-provider accessors", () => { + it("openaiProvider(name) targets the named gateway provider on the OpenAI protocol", async () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch }); + await trigger(coder.openaiProvider("azure-openai")("gpt-4o")); + expect(calls).toHaveLength(1); + expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/azure-openai/v1/chat/completions`); + expect(calls[0]!.headers.get("authorization")).toBe(`Bearer ${TOKEN}`); + expect(calls[0]!.body?.model).toBe("gpt-4o"); + }); + + it("anthropicProvider(name) targets the named gateway provider on the Anthropic protocol", async () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch }); + await trigger(coder.anthropicProvider("anthropic-bedrock")("claude-sonnet-4-6")); + expect(calls).toHaveLength(1); + expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/anthropic-bedrock/v1/messages`); + expect(calls[0]!.headers.get("authorization")).toBe(`Bearer ${TOKEN}`); + expect(calls[0]!.body?.model).toBe("claude-sonnet-4-6"); + }); + + it("honors a custom aiGatewayPath", async () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ + baseURL: BASE, + apiKey: TOKEN, + aiGatewayPath: "/api/v2/ai-gateway", + fetch, + }); + await trigger(coder.openaiProvider("azure-openai")("gpt-4o")); + expect(calls[0]!.url).toBe(`${BASE}/api/v2/ai-gateway/azure-openai/v1/chat/completions`); + }); + + it("openaiProvider with the default name behaves exactly like the default surface", async () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch }); + await trigger(coder.openaiProvider("openai")("gpt-4o")); + await trigger(coder.openai("gpt-4o")); + expect(calls).toHaveLength(2); + expect(calls[0]!.url).toBe(calls[1]!.url); + expect(calls[0]!.url).toBe(`${BASE}/api/v2/aibridge/openai/v1/chat/completions`); + }); + + it("rejects names outside the gateway grammar with InvalidArgumentError, before any request", () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch }); + + for (const bad of ["Azure-OpenAI", "azure_openai", "-azure", "azure-", "a--b", "a/b", ""]) { + for (const accessor of [ + () => coder.openaiProvider(bad), + () => coder.anthropicProvider(bad), + ]) { + let error: unknown; + try { + accessor(); + } catch (e) { + error = e; + } + expect(InvalidArgumentError.isInstance(error)).toBe(true); + expect((error as InvalidArgumentError).argument).toBe("name"); + expect((error as InvalidArgumentError).message).toContain(`"${bad}"`); + } + } + + expect(calls).toHaveLength(0); + }); + + it("named OpenAI-protocol sub-providers fail fast on embeddings too", () => { + const { fetch, calls } = capturingFetch(); + const coder = createCoder({ baseURL: BASE, apiKey: TOKEN, fetch }); + + let error: unknown; + try { + coder.openaiProvider("azure-openai").textEmbeddingModel("text-embedding-3-small"); + } catch (e) { + error = e; + } + expect(NoSuchModelError.isInstance(error)).toBe(true); + expect(calls).toHaveLength(0); + }); + + it("validates renamed default providers against the same grammar at createCoder time", () => { + expect(() => + createCoder({ baseURL: BASE, apiKey: TOKEN, providers: { openai: "Bad_Name" } }), + ).toThrow(/Invalid AI Gateway provider name/); + }); +}); + describe("createCoder validation", () => { it("throws when baseURL is missing", () => { expect(() => createCoder({ baseURL: "", apiKey: TOKEN })).toThrow(/baseURL/);