the comment in mcp-diff.yml about keeping the go-sdk v1.6.1 to v1.7.0-pre.1 bump "an honest, signal-only diff" is the right instinct, and i think there's a natural second half to it.
a schema diff answers "did the declared tool surface change". #1877 and #2417 are the case where that isn't enough: inputs got silently stripped from actions_run_trigger under OpenAI strict mode, so the declared schema and the shape a strict client actually received had diverged. the tool list looked stable, what models could do with it wasn't.
the complementary check is behavioural. record one client to server session as an ordered list of tool calls and results, replay it against the bumped SDK, diff. that catches the layer a schema comparison structurally can't see: strict-mode coercions, dropped optional fields, changed error shapes, transport-level omissions.
disclosure: i maintain an open source tool that does the replay half (reelier, MIT, github.com/seldonframe/reelier). to be clear about ordering, the static diff is the cheaper gate and should stay first, this is a complement to mcp-server-diff rather than a replacement. happy to open a PR adding a replay step so you can see whether it earns its runtime, and to close it if it doesn't.
the comment in mcp-diff.yml about keeping the go-sdk v1.6.1 to v1.7.0-pre.1 bump "an honest, signal-only diff" is the right instinct, and i think there's a natural second half to it.
a schema diff answers "did the declared tool surface change". #1877 and #2417 are the case where that isn't enough: inputs got silently stripped from actions_run_trigger under OpenAI strict mode, so the declared schema and the shape a strict client actually received had diverged. the tool list looked stable, what models could do with it wasn't.
the complementary check is behavioural. record one client to server session as an ordered list of tool calls and results, replay it against the bumped SDK, diff. that catches the layer a schema comparison structurally can't see: strict-mode coercions, dropped optional fields, changed error shapes, transport-level omissions.
disclosure: i maintain an open source tool that does the replay half (reelier, MIT, github.com/seldonframe/reelier). to be clear about ordering, the static diff is the cheaper gate and should stay first, this is a complement to mcp-server-diff rather than a replacement. happy to open a PR adding a replay step so you can see whether it earns its runtime, and to close it if it doesn't.