Skip to content

Copilot reviewer exits successfully after tool discovery without a terminal safe output #72702

Open

Description

@jonathanpeppers

Summary

Our Android PR Reviewer stopped before it read the pull request or submitted a review. Copilot CLI exited with code 0 after a response that contained only a server-side tool search. gh-aw then correctly reported missing_terminal_safe_output.

We have identified where execution stopped, but have not established whether the defect is in Copilot CLI, the upstream service, or response handling between them.

Issue and logs

The artifacts may require GitHub sign-in and are subject to Actions retention. Relevant file names are listed below; this report does not copy credentials or full raw logs.

Versions and trigger

Component Version
gh-aw compiler v0.92.0
Copilot CLI 1.0.102
Model gpt-6.1-sol
AWF v0.28.57
MCP gateway v0.4.31

Repository: dotnet/android. Workflow: Android PR Reviewer. Trigger: issue_comment (/review). Run commit: 39d9c36f28e4b883527f702fb8dcc60878e96aac.

Observed sequence

All times below are UTC on 28 October 2026.

  1. At 13:56:39 and 13:56:41, the Copilot API returned HTTP 401 with authentication_failed and the message You do not have access to the organization tied to the API key. The CLI retried.
  2. At 13:56:47.983, the API proxy recorded an HTTP 200 response on /responses.
  3. At 13:56:47.998, skill(code-review) completed successfully. The skill content was delivered. This was the only locally executed tool.
  4. At 13:56:59.601, the CLI's decoded response contained no text or local tool request. Its relevant fields were:
{
  "message": {
    "content": null,
    "refusal": null,
    "role": "assistant",
    "serverTools": {
      "provider": "openai-responses",
      "items": [
        {
          "arguments": {
            "paths": ["github"],
            "query": "pull_request_read"
          },
          "call_id": null,
          "execution": "server",
          "status": "completed",
          "type": "tool_search_call"
        }
      ]
    }
  },
  "finish_reason": "stop"
}

The response item's opaque ID is omitted from this excerpt.

  1. At 13:56:59.609–13:56:59.631, the CLI finalized the turn, reported hasError=false, and exited with code 0. No subsequent model request or review action was recorded.
  2. The harness reported success on attempt 1 after about 22 seconds. The configured timeout was 60 minutes.
  3. gh-aw produced a collector-generated report_incomplete item with reason missing_terminal_safe_output. The agent did not call report_incomplete itself.

Tool and safe-output evidence

The MCP gateway connected to both github and safeoutputs. It recorded two tools/list requests and one prompts/list request, but no tools/call request.

The safe-output server advertised six tools, including submit_pull_request_review, noop, and report_incomplete. No safe-output call was attempted. safe-output-items.jsonl was empty. There was no shell invocation or shell permission rejection.

The workflow already explicitly instructs the agent to call submit_pull_request_review once, including when there are no inline findings. This was not a completed review whose output was rejected during publication; the review never started.

Separate authentication failure

The later threat-detection job received three HTTP 401 responses with the same organization-access message. It failed without producing detection_result.json.

That authentication failure is confirmed, but it does not by itself explain the earlier missing review output: the agent recovered from its initial 401 responses, loaded the skill, and then stopped at tool discovery. Whether authentication also affected the final tool-search-only response is unknown.

Expected behavior

Tool discovery should allow the review to continue. If the response cannot be continued, the agent execution should report that failure rather than treating an empty, tool-search-only response as successful task completion. The existing gh-aw missing-output check should remain in place.

Evidence files and remaining uncertainty

In the agent artifact:

  • agent-session.jsonl: the skill call, final empty assistant message with serverTools, and session shutdown.
  • agent-stdio.log: two initial retry messages, skill invocation, and harness success with exit code 0.
  • sandbox/agent/logs/process-1793195798063-389.log: decoded final response around lines 884–945; successful headless exit around lines 992–994.
  • mcp-logs/gateway.md and mcp-logs/rpc-messages.jsonl: server registration and absence of tools/call requests.
  • sandbox/firewall/logs/api-proxy-logs/upstream-errors.jsonl: initial HTTP 401 responses.

The detection artifact contains its separate API authentication errors. The run's detection job logs show Authentication failed with provider, followed by the missing detection result.

The final response is available in the CLI's decoded log, not as a complete captured upstream response stream. We cannot yet determine whether the upstream service omitted a search result, the CLI failed to continue, or another response-handling component lost the continuation.

Service request ID for the final model call: 1fef84da-c878-463d-a8c6-f37e74889abc, approximately 13:56:48–13:56:59 UTC.

This report is based on the existing run and its downloaded artifacts. No reproduction run was dispatched. CLI release notes through 1.0.105 did not identify a verified fix for this specific failure.

Please help identify which component owns continuation after this server-side tool_search_call, and whether a known fix applies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions