Add MCP test automation: wire-protocol tests and setup script smoke test by Copilot · Pull Request #58 · cloudengine-labs/devops_os

Copilot · 2026-04-01T05:32:34Z

Create tests/test_mcp_protocol.py — 15 MCP JSON-RPC wire-protocol tests
Create .github/workflows/mcp-setup-smoke.yml — setup script smoke test
Update sanity.yml to include the new protocol test scenario
Fix CI failure: mcp has no __version__ attribute — use importlib.metadata.version('mcp') + import FastMCP directly to verify the package is usable

Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/bcbbfe9c-443e-4778-b9fa-0d9a5d00fc75 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com>

Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/67dfed60-b36b-48f1-83f5-84cca466abb9 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com>

Copilot

Pull request overview

This PR adds automated validation around the DevOps-OS MCP server by introducing true wire-protocol (stdio JSON-RPC) tests and a GitHub Actions workflow that smoke-tests the end-to-end MCP setup script on a clean runner.

Changes:

Add tests/test_mcp_protocol.py to spawn python -m mcp_server.server and verify MCP handshake, tool discovery, tool invocation, error handling, and sequential calls over stdio.
Add .github/workflows/mcp-setup-smoke.yml to run setup_devops_os_mcp.sh --local and verify venv creation, dependency importability, CLI registration (via stub), and basic JSON-RPC interactions.
Update .github/workflows/sanity.yml to run the new wire-protocol tests and include them in the HTML report generation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File	Description
`tests/test_mcp_protocol.py`	New subprocess-based MCP stdio wire-protocol test suite (handshake/tools/list/tools/call/error/sequential).
`.github/workflows/sanity.yml`	Runs the new protocol tests in CI and includes them in the combined pytest-html report.
`.github/workflows/mcp-setup-smoke.yml`	New workflow to smoke-test the setup script and validate basic MCP server operability in a fresh venv.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-01T05:57:02Z

tests/test_mcp_protocol.py

+        self._proc.stdin.write(json.dumps(msg) + "\n")
+        self._proc.stdin.flush()
+        raw = self._proc.stdout.readline()
+        if not raw:
+            stderr_out = self._proc.stderr.read()


_send_request() uses blocking stdout.readline() with no timeout. If the server deadlocks or fails to respond, the test suite can hang indefinitely in CI. Add a per-request timeout (e.g., via selectors/polling + deadline) and fail fast while terminating the subprocess and surfacing stderr for debugging.

Copilot · 2026-04-01T05:57:02Z

tests/test_mcp_protocol.py

+        finally:
+            s._proc.terminate()
+            s._proc.wait(timeout=5)
+


The direct subprocess cleanup in this test calls terminate() then wait(timeout=5) but doesn’t handle TimeoutExpired (unlike _MCPSession.exit). If the server doesn’t exit quickly, the test will error/flap instead of ensuring cleanup; reuse the context manager or mirror the kill-on-timeout logic here.

tests/test_mcp_protocol.py

.github/workflows/mcp-setup-smoke.yml

Copilot · 2026-04-01T05:57:04Z

.github/workflows/mcp-setup-smoke.yml

+              proc.stdin.write(json.dumps(msg) + "\n")
+              proc.stdin.flush()
+              if req_id is not None:
+                  return json.loads(proc.stdout.readline())
+


The embedded Python reads responses with proc.stdout.readline() without any timeout. If the server never replies (or writes a partial line), this step can hang until the workflow timeout. Add a bounded read (selectors/alarm) or use communicate() with timeout and surface stderr on failure.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

chefgs · 2026-04-01T06:02:02Z

@copilot apply changes based on this feedback

* docs(mcp): improve MCP setup for beginners across all AI tools - Rewrote mcp_server/README.md with step-by-step guides for Claude Code, Claude Desktop, Cursor, VS Code Copilot, Windsurf, and Zed - Added tools table entry for generate_unittest_config (was missing) - Added troubleshooting section for common setup failures - Fixed setup_devops_os_mcp.sh: replaced hardcoded ~/gsdev path with configurable INSTALL_DIR (default ~/devops_os), added --local flag for users already inside a clone, added --help and Python version check - Added .mcp.json (Claude Code project-level config) - Added .cursor/mcp.json (Cursor IDE project-level config) - Added .vscode/mcp.json (VS Code Copilot project-level config) - Updated .gitignore to track IDE MCP configs while ignoring personal settings - Removed "WIP" label from MCP section in main README.md, added curl one-liner Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add MCP test automation: wire-protocol tests and setup script smoke test (#58) * Add MCP test automation: wire-protocol tests and setup script smoke test Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/bcbbfe9c-443e-4778-b9fa-0d9a5d00fc75 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com> * fix: use importlib.metadata to check mcp version in smoke test Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/67dfed60-b36b-48f1-83f5-84cca466abb9 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com> * Update tests/test_mcp_protocol.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/test_mcp_protocol.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/mcp-setup-smoke.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/mcp-setup-smoke.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com> Co-authored-by: Saravanan Gnanaguru <g.gsaravanan@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .mcp.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update mcp_server/setup_devops_os_mcp.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update mcp_server/setup_devops_os_mcp.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update mcp_server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update mcp_server/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * docs: replace curl|bash with download-then-inspect-then-run in README.md Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/a2bcc76b-f5a7-421e-9984-542b39e55436 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Add MCP test automation: wire-protocol tests and setup script smoke test

7902198

Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/bcbbfe9c-443e-4778-b9fa-0d9a5d00fc75 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com>

Copilot AI assigned Copilot and chefgs Apr 1, 2026

Copilot created this pull request from a session on behalf of chefgs April 1, 2026 05:32 View session

Copilot started work on behalf of chefgs April 1, 2026 05:34 View session

fix: use importlib.metadata to check mcp version in smoke test

dc401d9

Agent-Logs-Url: https://github.com/cloudengine-labs/devops_os/sessions/67dfed60-b36b-48f1-83f5-84cca466abb9 Co-authored-by: chefgs <7605658+chefgs@users.noreply.github.com>

Copilot finished work on behalf of chefgs April 1, 2026 05:37

Copilot AI requested a review from chefgs April 1, 2026 05:37

chefgs approved these changes Apr 1, 2026

View reviewed changes

chefgs marked this pull request as ready for review April 1, 2026 05:53

Copilot AI review requested due to automatic review settings April 1, 2026 05:53

Copilot started reviewing on behalf of chefgs April 1, 2026 05:54 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

chefgs and others added 4 commits April 1, 2026 11:29

Update tests/test_mcp_protocol.py

9cb7781

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/test_mcp_protocol.py

7eaafd3

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update .github/workflows/mcp-setup-smoke.yml

5dc4b95

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update .github/workflows/mcp-setup-smoke.yml

596bb37

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of chefgs April 1, 2026 06:02 View session

chefgs merged commit 23a29dc into improve/mcp-setup-docs Apr 1, 2026
2 of 3 checks passed

Copilot stopped work on behalf of chefgs due to an error April 1, 2026 06:07
The session was cancelled by the user.

Copilot AI requested a review from chefgs April 1, 2026 06:07

chefgs deleted the copilot/add-test-automation-claude-mcp branch April 1, 2026 07:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MCP test automation: wire-protocol tests and setup script smoke test#58

Add MCP test automation: wire-protocol tests and setup script smoke test#58
chefgs merged 6 commits intoimprove/mcp-setup-docsfrom
copilot/add-test-automation-claude-mcp

Copilot AI commented Apr 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

chefgs commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

chefgs commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 1, 2026 •

edited

Loading