Skip to content

Commit 255650d

Browse files
authored
ci(conformance): run server --suite draft and baseline the 2026-07-28 scenarios (#2878)
1 parent a3689ab commit 255650d

2 files changed

Lines changed: 43 additions & 4 deletions

File tree

.github/actions/conformance/expected-failures.yml

Lines changed: 38 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,41 @@ client:
4646
# client support for the token-exchange + JWT bearer flow.
4747
- auth/enterprise-managed-authorization
4848

49-
# The `active` suite (30 scenarios / 42 assertions) is fully green against
50-
# mcp-everything-server. Draft-suite entries are added when the workflow
51-
# gains a `--suite draft` step.
52-
server: []
49+
server:
50+
# --- Draft-spec scenarios (in `--suite draft`; the `active` suite is green) ---
51+
# SEP-2575 (stateless HTTP / _meta envelope): server has no stateless mode,
52+
# _meta-derived capabilities, error-code mappings, or server/discover yet.
53+
- server-stateless
54+
# SEP-2322 (multi-round-trip requests / IncompleteResult): not implemented;
55+
# most scenarios currently fail early with "Missing session ID" because
56+
# mcp-everything-server only runs in stateful mode.
57+
- input-required-result-basic-elicitation
58+
- input-required-result-basic-sampling
59+
- input-required-result-basic-list-roots
60+
- input-required-result-request-state
61+
- input-required-result-multiple-input-requests
62+
- input-required-result-multi-round
63+
- input-required-result-non-tool-request
64+
- input-required-result-result-type
65+
- input-required-result-tampered-state
66+
- input-required-result-capability-check
67+
# SEP-2549 (caching): no ttlMs/cacheScope support; scenario also hits the
68+
# stateful-mode "Missing session ID" error.
69+
- caching
70+
# SEP-2243 (HTTP header standardization): -32001 HeaderMismatch handling and
71+
# case-insensitive/whitespace-trimmed header validation not implemented.
72+
- http-header-validation
73+
- http-custom-header-server-validation
74+
# WARNING-only entries: these scenarios emit no FAILURE checks, only SHOULD-level
75+
# WARNINGs, but the expected-failures evaluator counts WARNINGs as failures.
76+
# SEP-2164: server returns -32600 (not -32602) and omits error.data.uri.
77+
- sep-2164-resource-not-found
78+
# SEP-2322 SHOULD-level behaviours (re-request missing inputResponses, ignore
79+
# unrecognized inputResponses keys).
80+
- input-required-result-missing-input-response
81+
- input-required-result-ignore-extra-params
82+
# Intentionally NOT baselined (2 of 19 draft scenarios): the SEP-2322
83+
# negative-case scenarios input-required-result-unsupported-methods and
84+
# input-required-result-validate-input pass today only because the stateful
85+
# server's -32600 "Missing session ID" satisfies their assertions. They will
86+
# start failing for real once stateless mode lands; add them then.

.github/workflows/conformance.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@ jobs:
3838
./.github/actions/conformance/run-server.sh
3939
--suite active
4040
--expected-failures ./.github/actions/conformance/expected-failures.yml
41+
- name: Run server conformance (draft suite)
42+
run: >-
43+
./.github/actions/conformance/run-server.sh
44+
--suite draft
45+
--expected-failures ./.github/actions/conformance/expected-failures.yml
4146
4247
client-conformance:
4348
runs-on: ubuntu-latest

0 commit comments

Comments
 (0)