Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,45 @@
command: 'java -jar conformance-tests/client-jdk-http-client/target/client-jdk-http-client-1.0.0-SNAPSHOT.jar'
scenario: ${{ matrix.scenario }}
expected-failures: ./conformance-tests/conformance-baseline.yml

auth:
name: Auth Conformance
runs-on: ubuntu-latest
strategy:
matrix:
scenario:
- auth/metadata-default
- auth/metadata-var1
- auth/metadata-var2
- auth/metadata-var3
- auth/basic-cimd
- auth/scope-from-www-authenticate
- auth/scope-from-scopes-supported
- auth/scope-omitted-when-undefined
- auth/scope-step-up
- auth/scope-retry-limit
- auth/token-endpoint-auth-basic
- auth/token-endpoint-auth-post
- auth/token-endpoint-auth-none
- auth/pre-registration
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'

- name: Build client
run: mvn clean install -DskipTests

- name: Run conformance test
uses: modelcontextprotocol/conformance@v0.1.15
with:
node-version: '22' # see https://github.com/modelcontextprotocol/conformance/pull/162
mode: client
command: 'java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-1.0.0-SNAPSHOT.jar'
scenario: ${{ matrix.scenario }}
expected-failures: ./conformance-tests/conformance-baseline.yml
56 changes: 50 additions & 6 deletions conformance-tests/VALIDATION_RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

## Summary

**Server Tests:** 37/40 passed (92.5%)
**Server Tests:** 37/40 passed (92.5%)
**Client Tests:** 3/4 scenarios passed (9/10 checks passed)
**Auth Tests:** 12/14 scenarios fully passing (178 passed, 1 failed, 1 warning, 85.7% scenarios, 98.9% checks)

## Server Test Results

Expand All @@ -20,7 +21,7 @@
### Failing (3/40)

1. **resources-subscribe** - Not implemented in SDK
2. **resources-unsubscribe** - Not implemented in SDK
2. **resources-unsubscribe** - Not implemented in SDK

## Client Test Results

Expand All @@ -32,17 +33,45 @@

### Partially Passing (1/4 scenarios, 1/2 checks)

- **sse-retry (1/2 + 1 warning):**
- **sse-retry (1/2 + 1 warning):**
- ✅ Reconnects after stream closure
- ❌ Does not respect retry timing
- ⚠️ Does not send Last-Event-ID header (SHOULD requirement)

**Issue:** Client treats `retry:` SSE field as invalid instead of parsing it for reconnection timing.

## Auth Test Results (Spring HTTP Client)

**Status: 178 passed, 1 failed, 1 warning across 14 scenarios**

Uses the `client-spring-http-client` module with Spring Security OAuth2 and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library.

### Fully Passing (12/14 scenarios)

- **auth/metadata-default (12/12):** Default metadata discovery
- **auth/metadata-var1 (12/12):** Metadata discovery variant 1
- **auth/metadata-var2 (12/12):** Metadata discovery variant 2
- **auth/metadata-var3 (12/12):** Metadata discovery variant 3
- **auth/scope-from-www-authenticate (13/13):** Scope extraction from WWW-Authenticate header
- **auth/scope-from-scopes-supported (13/13):** Scope extraction from scopes_supported
- **auth/scope-omitted-when-undefined (13/13):** Scope omitted when not defined
- **auth/scope-retry-limit (11/11):** Scope retry limit handling
- **auth/token-endpoint-auth-basic (17/17):** Token endpoint with HTTP Basic auth
- **auth/token-endpoint-auth-post (17/17):** Token endpoint with POST body auth
- **auth/token-endpoint-auth-none (17/17):** Token endpoint with no client auth
- **auth/pre-registration (6/6):** Pre-registered client credentials flow

### Partially Passing (2/14 scenarios)

- **auth/basic-cimd (12/12 + 1 warning):** Basic Client-Initiated Metadata Discovery — all checks pass, minor warning
- **auth/scope-step-up (11/12):** Scope step-up challenge — 1 failure, client does not fully handle scope escalation after initial authorization

## Known Limitations

1. **Resource Subscriptions:** SDK doesn't implement `resources/subscribe` and `resources/unsubscribe` handlers
2. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header
3. **Auth Scope Step-Up:** Client does not fully handle scope step-up challenges where the server requests additional scopes after initial authorization
4. **Auth Basic CIMD:** Minor conformance warning in the basic Client-Initiated Metadata Discovery flow

## Running Tests

Expand Down Expand Up @@ -70,11 +99,26 @@ for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-re
done
```

### Auth (Spring HTTP Client)

Ensure you run with the conformance testing suite `0.1.15` or higher.

```bash
# Build
cd conformance-tests/client-spring-http-client
../../mvnw clean package -DskipTests

# Run auth suite
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar target/client-spring-http-client-0.18.0-SNAPSHOT.jar" \
--suite auth
```

## Recommendations

### High Priority
1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport`
2. Implement resource subscription handlers in `McpStatelessAsyncServer`

### Medium Priority
3. Add Host/Origin validation in `HttpServletStreamableServerTransportProvider` for DNS rebinding protection
3. Implement CIMD
4. Implement scope step up
124 changes: 124 additions & 0 deletions conformance-tests/client-spring-http-client/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# MCP Conformance Tests - Spring HTTP Client (Auth Suite)

This module provides a conformance test client implementation for the Java MCP SDK's **auth** suite.

OAuth2 support is not implemented in the SDK itself, but we provide hooks to implement the Authorization section of the specification. One such implementation is done in Spring, with Sprign AI and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library.

This is a Spring web application, we interact with it through a normal HTTP-client that follows redirects and performs OAuth2 authorization flows.

## Overview

The conformance test client is designed to work with the [MCP Conformance Test Framework](https://github.com/modelcontextprotocol/conformance). It validates that the Java MCP SDK client, combined with Spring Security's OAuth2 support, properly implements the MCP authorization specification.

Test with @modelcontextprotocol/conformance@0.1.15.

## Conformance Test Results

**Status: 178 passed, 1 failed, 1 warning across 14 scenarios**

| Scenario | Result | Details |
|---|---|---|
| auth/metadata-default | ✅ Pass | 12/12 |
| auth/metadata-var1 | ✅ Pass | 12/12 |
| auth/metadata-var2 | ✅ Pass | 12/12 |
| auth/metadata-var3 | ✅ Pass | 12/12 |
| auth/basic-cimd | ⚠️ Warning | 12/12 passed, 1 warning |
| auth/scope-from-www-authenticate | ✅ Pass | 13/13 |
| auth/scope-from-scopes-supported | ✅ Pass | 13/13 |
| auth/scope-omitted-when-undefined | ✅ Pass | 13/13 |
| auth/scope-step-up | ❌ Fail | 11/12 (1 failed) |
| auth/scope-retry-limit | ✅ Pass | 11/11 |
| auth/token-endpoint-auth-basic | ✅ Pass | 17/17 |
| auth/token-endpoint-auth-post | ✅ Pass | 17/17 |
| auth/token-endpoint-auth-none | ✅ Pass | 17/17 |
| auth/pre-registration | ✅ Pass | 6/6 |

See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the full project validation results.

## Architecture

The client is a Spring Boot application that reads test scenarios from environment variables and accepts the server URL as a command-line argument, following the conformance framework's conventions:

- **MCP_CONFORMANCE_SCENARIO**: Environment variable specifying which test scenario to run
- **MCP_CONFORMANCE_CONTEXT**: Environment variable with JSON context (used by `auth/pre-registration`)
- **Server URL**: Passed as the last command-line argument

### Scenario Routing

The application uses Spring's conditional configuration to select the appropriate scenario at startup:

- **`DefaultConfiguration`** — Activated for all scenarios except `auth/pre-registration`. Uses the OAuth2 Authorization Code flow with dynamic client registration via `McpClientOAuth2Configurer`.
- **`PreRegistrationConfiguration`** — Activated only for `auth/pre-registration`. Uses the Client Credentials flow with pre-registered client credentials read from `MCP_CONFORMANCE_CONTEXT`.

### Key Dependencies

- **Spring Boot 4.0** with Spring Security OAuth2 Client
- **Spring AI MCP Client** (`spring-ai-starter-mcp-client`)
- **mcp-client-security** — Community library providing MCP-specific OAuth2 integration (metadata discovery, dynamic client registration, transport context)

## Building

Build the executable JAR:

```bash
cd conformance-tests/client-spring-http-client
../../mvnw clean package -DskipTests
```

This creates an executable JAR at:
```
target/client-spring-http-client-0.18.0-SNAPSHOT.jar
```

## Running Tests

### Using the Conformance Framework

Run the full auth suite:

```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-0.18.0-SNAPSHOT.jar" \
--suite auth
```

Run a single scenario:

```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-0.18.0-SNAPSHOT.jar" \
--scenario auth/metadata-default
```

Run with verbose output:

```bash
npx @modelcontextprotocol/conformance@0.1.15 client \
--spec-version 2025-11-25 \
--command "java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-0.18.0-SNAPSHOT.jar" \
--scenario auth/metadata-default \
--verbose
```

### Manual Testing

You can also run the client manually if you have a test server:

```bash
export MCP_CONFORMANCE_SCENARIO=auth/metadata-default
java -jar conformance-tests/client-spring-http-client/target/client-spring-http-client-0.18.0-SNAPSHOT.jar http://localhost:3000/mcp
```

## Known Issues

1. **auth/scope-step-up** (1 failure) — The client does not fully handle scope step-up challenges where the server requests additional scopes after initial authorization.
2. **auth/basic-cimd** (1 warning) — Minor conformance warning in the basic Client-Initiated Metadata Discovery flow.

## References

- [MCP Specification — Authorization](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization)
- [MCP Conformance Tests](https://github.com/modelcontextprotocol/conformance)
- [mcp-client-security Library](https://github.com/springaicommunity/mcp-client-security)
- [SDK Integration Guide](https://github.com/modelcontextprotocol/conformance/blob/main/SDK_INTEGRATION.md)
90 changes: 90 additions & 0 deletions conformance-tests/client-spring-http-client/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.0.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>client-spring-http-client</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>MCP Conformance Tests - Spring HTTP Client</name>
<description>Spring HTTP Client conformance tests for the Java MCP SDK</description>
<url>https://github.com/modelcontextprotocol/java-sdk</url>

<scm>
<url>https://github.com/modelcontextprotocol/java-sdk</url>
<connection>git://github.com/modelcontextprotocol/java-sdk.git</connection>
<developerConnection>git@github.com/modelcontextprotocol/java-sdk.git</developerConnection>
</scm>

<properties>
<java.version>17</java.version>
<spring-ai.version>2.0.0-M2</spring-ai.version>
</properties>

<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-restclient</artifactId>
</dependency>

<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-mcp-client</artifactId>
<version>${spring-ai.version}</version>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>

<dependency>
<groupId>org.springaicommunity</groupId>
<artifactId>mcp-client-security</artifactId>
<version>0.1.2</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

<repositories>
<repository>
<id>maven-central</id>
<url>https://repo.maven.apache.org/maven2/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>

</project>
Loading