Correct OAuth protected-resource identity and discovery challenges

docs/oauth-cutover.md

@@ -1,21 +1,29 @@
 # Canonical OAuth discovery release
 
-This change only updates production protected-resource discovery. It does not
-move MCP traffic, proxy OAuth requests, modify registrations, or remove
-compatibility code. Review approval is not deployment approval.
+This change corrects protected-resource discovery and its unauthenticated
+challenge. It does not move MCP traffic, proxy OAuth requests, modify
+registrations, or remove compatibility code. Review approval is not deployment
+approval.
 
 ## Contract
 
-- Resource remains `https://mcp.onkernel.com`; MCP remains hosted at
-  `https://mcp.onkernel.com/mcp`.
+- Resource is `https://mcp.onkernel.com/mcp`, exactly matching the MCP endpoint.
+  The previous origin-only value was incorrect: RFC 9728 section 3.3 requires
+  the resource to match the identifier used to derive the metadata URL.
 - `/.well-known/oauth-protected-resource/mcp` advertises
   `authorization_servers: ["https://auth.onkernel.com"]` and canonical
   `/authorize`, `/token`, and `/register` endpoints. Responses use `no-store`.
+  Unauthenticated MCP requests reach the route's bearer-token validation rather
+  than Clerk's page protection and include this path-specific URL in the
+  `WWW-Authenticate` challenge's `resource_metadata` parameter. The root
+  `/.well-known/oauth-protected-resource` endpoint is not provided; it is not
+  the discovery URL for the `/mcp` resource.
 - Legacy authorization-server metadata and all existing TypeScript OAuth routes,
   picker/consent pages, token verification, and Redis behavior remain unchanged.
   There is no legacy Go relay or MCP DNS change.
-- Local, staging, and preview discovery retain their own origin. There is no
-  client-specific opt-in discovery header.
+- Local, staging, and preview discovery retain their own origin and use `/mcp`
+  as the resource path. Their authorization server remains their own origin.
+  There is no client-specific opt-in discovery header.
 
 ## Compatibility dependency
 
@@ -37,6 +45,22 @@ Existing credentials must remain available: preserve Clerk applications, durable
 registrations, static clients, shared token context, and both issuer endpoints.
 Token validation and refresh behavior are unchanged by this metadata update.
 
+Deploy client issuer/registration/resource pinning and explicit reconnect
+handling **before** the metadata correction. Clients that rediscover during an
+OAuth callback or refresh must not silently move existing credentials to the
+newly advertised issuer or replace their original resource binding. A client
+that previously rejected the mismatched metadata may start accepting canonical
+discovery after this fix; that does not migrate its retained legacy registration.
+
+The Go broker accepts HTTPS resources including `/mcp`, but token exchange
+compares the submitted resource exactly with the resource stored in the
+authorization transaction. Preserve the origin-only resource for old in-flight
+transactions; use `/mcp` for new authorizations based on corrected discovery.
+The retained TypeScript token route forwards the caller's resource unchanged,
+including when omitted. Do not rewrite stored transactions or credentials.
+Local regression tests are not live registration, login, token, or refresh
+compatibility tests; those remain separate, explicitly authorized rollout checks.
+
 ## Forward deployment order
 
 1. Obtain explicit release approval after the applicable acceptance checks and
@@ -45,16 +69,20 @@ Token validation and refresh behavior are unchanged by this metadata update.
    Go service, registry, shared token context, static CLI overlay, dashboard
    picker/consent, and canonical Clerk callback are usable. Keep the existing
    MCP deployment and all legacy routes in service.
-2. Merge/apply the separately reviewed **auth DNS-only** change to the existing
-   production API load balancer. Leave MCP DNS on Vercel. Verify public auth
-   DNS/TLS convergence, canonical issuer/endpoints, and callback reachability.
-   With the current 300-second TTL, allow two observed TTLs (10 minutes); recheck
-   the actual TTL and public resolvers at execution time. During this interval,
-   MCP discovery still advertises the working legacy TypeScript path.
-3. Only after step 2 is verified, merge/deploy this MCP PR. Treat merging as a
-   possible production deployment. Verify canonical protected-resource metadata,
-   unchanged resource identity, unchanged legacy authorization-server metadata
-   and routes, and the agreed cached-client outcome.
+2. Verify the separately approved **auth DNS-only** cutover to the existing
+   production API load balancer; do not reapply it if already complete. Leave
+   MCP DNS on Vercel. Verify public auth DNS/TLS convergence, canonical
+   issuer/endpoints, and callback reachability. If that cutover is still pending,
+   follow its separately reviewed deployment procedure. With a 300-second TTL,
+   allow two observed TTLs (10 minutes); recheck the actual TTL and public
+   resolvers at execution time. The incorrect origin-only resource is not a
+   safe mechanism for keeping clients on legacy discovery.
+3. Only after step 2 and the client pinning/reconnect prerequisite are verified,
+   merge/deploy this MCP PR. Treat merging as a possible production deployment.
+   Verify the exact `/mcp` resource identity at path-specific discovery, its
+   unauthenticated challenge, canonical authorization server, unchanged legacy
+   authorization-server metadata and routes, and the agreed cached-client
+   outcome.
 
 Discovery-first is not safe merely because the auth hostname already resolves:
 that hostname must serve the intended canonical service and callbacks, not the
@@ -68,8 +96,10 @@ clients from an auth DNS change.
    production discovery selection in `src/lib/oauth-discovery.ts` back to
    `MCP_ORIGIN`, and updates its test. Deploy that change first; verify the served
    protected-resource JSON advertises only the legacy origin again. Retain
-   `no-store`, resource identity, and all existing routes. Do not reset main,
-   revert unrelated commits, or promote an old whole MCP deployment.
+   `no-store`, the corrected `/mcp` resource identity, and all existing routes.
+   Change only the authorization-server selection, not the resource or challenge.
+   Do not reset main, revert unrelated commits, or promote an old whole MCP
+   deployment.
 2. Keep auth DNS on Go while accounting for cached canonical registrations and
    in-flight Go authorizations/codes. Metadata rollback affects future discovery;
    it cannot erase cached issuers or registrations. The old TypeScript token
@@ -84,6 +114,6 @@ clients from an auth DNS change.
    observed TTLs, and verify public routing plus the retained auth deployment.
    Preserve registration and credential data; rollback requires no data cleanup.
 
-If the failure occurs before step 3 of the forward sequence, MCP discovery needs
-no rollback: it still points at the legacy service. Auth DNS rollback still has
-the cached-canonical/in-flight constraints above.
+If the resource correction has not been deployed, it needs no rollback. Any
+separate issuer-selection or auth DNS rollback still has the cached-canonical
+and in-flight constraints above.

src/app/[transport]/route.test.ts

@@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";
 import { Kernel } from "@onkernel/sdk";
 import type { McpConnectionScopeFailureAnalytics } from "@/lib/mcp/analytics";
 import { defaultMcpDependencies } from "@/lib/mcp/dependencies";
+import { oauthResourceMetadata } from "@/lib/oauth-discovery";
 
 process.env.CLERK_SECRET_KEY ??= "test-clerk-secret";
 
@@ -28,7 +29,7 @@ mock.module("@/lib/mcp/analytics", () => ({
 }));
 
 const originalCreateKernelClient = defaultMcpDependencies.createKernelClient;
-const { POST, connectionScopeFailureResponse } = await import("./route");
+const { GET, POST, connectionScopeFailureResponse } = await import("./route");
 
 function initializeRequest(token = "sk_opaque_key") {
   return new Request("https://mcp.example.test/sse", {
@@ -72,6 +73,46 @@ afterEach(() => {
   defaultMcpDependencies.createKernelClient = originalCreateKernelClient;
 });
 
+describe("unauthenticated discovery", () => {
+  for (const method of ["GET", "POST"]) {
+    test.each([
+      [
+        "https://mcp.onkernel.com",
+        "mcp.onkernel.com",
+        "https://mcp.onkernel.com",
+      ],
+      ["http://localhost:3002", "mcp.onkernel.com", "https://mcp.onkernel.com"],
+      ["http://localhost:3002", "localhost:3002", "http://localhost:3002"],
+      [
+        "https://localhost:3000",
+        "mcp-staging.onkernel.com",
+        "https://mcp-staging.onkernel.com",
+      ],
+    ])(
+      `${method} challenges point to path-specific discovery at %s (Host: %s)`,
+      async (origin, host, publicOrigin) => {
+        const req = new nextServer.NextRequest(`${origin}/mcp`, {
+          method,
+          headers: { Host: host, "X-Forwarded-Host": "ignored.example" },
+        });
+        const response = await (method === "GET" ? GET : POST)(req);
+        expect(response.status).toBe(401);
+        const challenge = response.headers.get("WWW-Authenticate");
+        expect(challenge).toContain('Bearer realm="OAuth"');
+        expect(challenge).toContain('error="invalid_token"');
+        const metadataUrl = challenge?.match(
+          /resource_metadata="([^"]+)"/,
+        )?.[1];
+        expect(metadataUrl).toBe(
+          `${publicOrigin}/.well-known/oauth-protected-resource/mcp`,
+        );
+        const metadata = oauthResourceMetadata(new Request(metadataUrl!), {});
+        expect(metadata.resource).toBe(`${publicOrigin}/mcp`);
+      },
+    );
+  }
+});
+
 describe("connection scope failures through the handler", () => {
   test("answers a refused credential with 401 rather than a server error", async () => {
     failingAuthContext(Object.assign(new Error("revoked"), { status: 401 }));
@@ -243,10 +284,13 @@ describe("vault entitlement routing", () => {
 
 describe("connectionScopeFailureResponse", () => {
   test("names an inactive project instead of blaming the credential", async () => {
-    const response = connectionScopeFailureResponse({
-      status: "rejected",
-      statusCode: 404,
-    });
+    const response = connectionScopeFailureResponse(
+      new Request("https://mcp.example.test/mcp"),
+      {
+        status: "rejected",
+        statusCode: 404,
+      },
+    );
 
     expect(response.status).toBe(404);
     expect(response.headers.get("WWW-Authenticate")).toBeNull();
@@ -258,10 +302,16 @@ describe("connectionScopeFailureResponse", () => {
   });
 
   test("challenges a refused credential so clients re-authenticate", () => {
-    const response = connectionScopeFailureResponse({
-      status: "rejected",
-      statusCode: 401,
-    });
+    const response = connectionScopeFailureResponse(
+      new Request("https://mcp.example.test/mcp"),
+      {
+        status: "rejected",
+        statusCode: 401,
+      },
+    );
+    expect(response.headers.get("WWW-Authenticate")).toContain(
+      'resource_metadata="https://mcp.example.test/.well-known/oauth-protected-resource/mcp"',
+    );
 
     expect(response.headers.get("WWW-Authenticate")).toContain(
       'error="invalid_token"',

src/app/[transport]/route.ts

@@ -6,6 +6,10 @@ import {
 import { verifyToken } from "@clerk/nextjs/server";
 import { after, NextRequest } from "next/server";
 import { isValidJwtFormat } from "@/lib/auth-utils";
+import {
+  OAUTH_RESOURCE_METADATA_PATH,
+  oauthResourceMetadataUrl,
+} from "@/lib/oauth-discovery";
 import {
   captureMcpConnectionScopeFailure,
   flushMcpAnalytics,
@@ -60,15 +64,17 @@ function errorResponse(
 
 // Helper function to create authentication error response
 function createAuthErrorResponse(
+  req: Request,
   error: string = "invalid_token",
   description: string = "Missing or invalid access token",
 ): Response {
   return errorResponse(401, error, description, {
-    "WWW-Authenticate": `Bearer realm="OAuth", error="${error}", error_description="${description}"`,
+    "WWW-Authenticate": `Bearer realm="OAuth", error="${error}", error_description="${description}", resource_metadata="${oauthResourceMetadataUrl(req)}"`,
   });
 }
 
 export function connectionScopeFailureResponse(
+  req: Request,
   failure: Exclude<McpConnectionContextFailure, { status: "invalid" }>,
 ): Response {
   if (failure.status === "rejected") {
@@ -90,6 +96,7 @@ export function connectionScopeFailureResponse(
         );
       case 401:
         return createAuthErrorResponse(
+          req,
           "invalid_token",
           "The Kernel API rejected this credential",
         );
@@ -169,7 +176,7 @@ async function handleMcpRequestWithIdentity({
     if (connection.status === "invalid") {
       throw new Error("Unable to resolve Kernel connection scope");
     }
-    return connectionScopeFailureResponse(connection);
+    return connectionScopeFailureResponse(req, connection);
   }
   // Recheck with the current credential on every request, including tools/call.
   const vaults = await resolveMcpVaultAccess({ token, signal: req.signal });
@@ -194,7 +201,7 @@ async function handleMcpRequestWithIdentity({
     }),
     {
       required: true,
-      resourceMetadataPath: "/.well-known/oauth-protected-resource/mcp",
+      resourceMetadataPath: OAUTH_RESOURCE_METADATA_PATH,
     },
   );
   return await authHandler(req);
@@ -211,6 +218,7 @@ async function handleAuthenticatedRequest(
     : null;
   if (!token) {
     return createAuthErrorResponse(
+      req,
       "invalid_token",
       "Missing or invalid access token",
     );
@@ -239,13 +247,15 @@ async function handleAuthenticatedRequest(
     });
     if (!payload.sub) {
       return createAuthErrorResponse(
+        req,
         "invalid_token",
         "Invalid token: No user ID found in token payload",
       );
     }
     userId = payload.sub;
   } catch (authError) {
     return createAuthErrorResponse(
+      req,
       "invalid_token",
       `Invalid token: ${authError instanceof Error ? authError.message : "Authentication failed"}`,
     );

src/app/token/route.test.ts

@@ -115,6 +115,33 @@ function dependencies({
 }
 
 describe("POST /token", () => {
+  for (const grantType of ["authorization_code", "refresh_token"]) {
+    test.each([
+      "https://mcp.example.test",
+      "https://mcp.example.test/mcp",
+      undefined,
+    ])(
+      `preserves the existing resource %s during ${grantType}`,
+      async (resource) => {
+        const deps = dependencies();
+        const response = await tokenRequest(
+          request({
+            grant_type: grantType,
+            client_id: "client_1",
+            ...(grantType === "authorization_code"
+              ? { code: "code_1", code_verifier: "verifier_1" }
+              : { refresh_token: "refresh-old" }),
+            ...(resource ? { resource } : {}),
+          }),
+          deps.value,
+        );
+        expect(response.status).toBe(200);
+        expect(deps.calls.exchanges).toHaveLength(1);
+        expect(deps.calls.exchanges[0].get("resource")).toBe(resource ?? null);
+      },
+    );
+  }
+
   test("issues an organization-wide token and persists both contexts", async () => {
     const deps = dependencies();
     const response = await tokenRequest(

src/lib/oauth-discovery-route.test.ts

@@ -0,0 +1,53 @@
+import { describe, expect, mock, test } from "bun:test";
+import { NextRequest } from "next/server";
+
+const clerkMetadata = {
+  resource: "https://clerk.example.test",
+  authorization_servers: ["https://clerk.example.test"],
+  jwks_uri: "https://clerk.example.test/.well-known/jwks.json",
+};
+mock.module("@clerk/mcp-tools/next", () => ({
+  protectedResourceHandlerClerk: () => async () => Response.json(clerkMetadata),
+}));
+
+const { GET, OPTIONS } = await import(
+  "@/app/.well-known/oauth-protected-resource/mcp/route"
+);
+
+describe("/.well-known/oauth-protected-resource/mcp", () => {
+  test.each([
+    ["https://mcp.onkernel.com", "https://auth.onkernel.com"],
+    ["http://localhost:3002", "http://localhost:3002"],
+    ["https://mcp-staging.onkernel.com", "https://mcp-staging.onkernel.com"],
+  ])("serves uncached metadata bound to %s/mcp", async (origin, issuer) => {
+    const resource = new URL(`${origin}/mcp`);
+    const discoveryUrl = new URL(resource);
+    discoveryUrl.pathname = `/.well-known/oauth-protected-resource${resource.pathname}`;
+    const response = await GET(new NextRequest(discoveryUrl));
+
+    expect(response.status).toBe(200);
+    expect(response.headers.get("Cache-Control")).toBe("no-store");
+    expect(response.headers.get("Access-Control-Allow-Origin")).toBe("*");
+    expect(await response.json()).toEqual({
+      resource: resource.href,
+      authorization_servers: [issuer],
+      authorization_endpoint: `${issuer}/authorize`,
+      token_endpoint: `${issuer}/token`,
+      registration_endpoint: `${issuer}/register`,
+      scopes_supported: ["openid"],
+      jwks_uri: clerkMetadata.jwks_uri,
+    });
+  });
+
+  test("keeps discovery available to cross-origin clients", async () => {
+    const response = await OPTIONS();
+    expect(response.status).toBe(204);
+    expect(response.headers.get("Access-Control-Allow-Origin")).toBe("*");
+    expect(response.headers.get("Access-Control-Allow-Methods")).toBe(
+      "GET, OPTIONS",
+    );
+    expect(response.headers.get("Access-Control-Allow-Headers")).toBe(
+      "Content-Type, Authorization",
+    );
+  });
+});