Skip to content

bug: diagnose whitespace-only LLM responses from third-party relays #815

Description

@ZhulongNT

Summary

GenericAgent can receive a syntactically valid but semantically empty completion: for example, the response payload contains one text block whose value is a single whitespace character (" "). The TUI then shows:

· LLM returned an empty response. Retrying...

The retry succeeds in the observed case, so this is recoverable, but it is confusing to diagnose.

Observation

  • Session trace: model_responses_858881.txt, 2026-09-21 16:27:46.
  • The response was represented as a Python object equivalent to:
    [{"type": "text", "text": " "}]
  • There was no meaningful content or thinking; GA correctly treated it as blank and regenerated.
  • The relevant guard is ga.py / do_no_tool() (currently around lines 459–467). It emits [Warn] LLM returned an empty response. Retrying... and routes through _retry_or_exit().
  • The TUI replaces [Warn] with ·, which makes the displayed line look less like an actionable warning.

Important context: third-party relays / gateways

This has been observed when using a non-official API relay/gateway rather than the model provider's official endpoint. The relay may return an HTTP-successful response while dropping, truncating, or normalizing streamed content to whitespace. This is an observation, not a claim that every relay is faulty; the same symptom could also originate upstream. However, intermediaries make the failure mode substantially harder to attribute because the request technically succeeds.

There is a related lower-level path in llmcore.py where a stream produces no output and is retried as ConnectionError("empty response"), but the whitespace-only case reaches the agent-level blank-response guard instead.

Why this deserves a separate issue

#201 added the useful retry limit (retry up to three times, then stop). It does not expose enough context to distinguish:

  1. a provider-side empty completion,
  2. a relay/gateway that returned a malformed or whitespace-only successful response, and
  3. a stream that ended before producing any content.

Suggested improvements

  1. Keep the current safe retry behavior.
  2. Log a compact, redacted diagnostic for blank responses: session/model name, configured base URL host (no credentials), response block types, text lengths / whitespace-only flag, whether any thinking was present, and retry count.
  3. Make the TUI warning retain a visible severity label, e.g. · [Warn] Empty LLM response; retrying (1/3).
  4. Optionally classify whitespace_only, no_stream_output, and empty_payload separately in the trace/logs.
  5. Documentation: recommend reproducing against the official endpoint before filing provider-specific issues when a third-party relay is in the path.

This should make relay-related incidents diagnosable without weakening compatibility with OpenAI-compatible gateways.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions