Skip to content

fix(ga): match truncation markers only at tail, not anywhere in last N chars - #872

Closed
leonsong09 wants to merge 1 commit into
lsdefine:mainfrom
leonsong09:fix/incomplete-response-tail
Closed

leonsong09 wants to merge 1 commit into
lsdefine:mainfrom
leonsong09:fix/incomplete-response-tail

Conversation

@leonsong09

@leonsong09 leonsong09 commented Nov 12, 2026 •

Copy link
Copy Markdown

Problem

GenericAgentHandler.do_no_tool() reports an incomplete response whenever a truncation marker appears anywhere in the trailing window of the reply:

if '[!!! 流异常中断' in content[-100:] or '!!!Error:' in content[50:][-100:] or ...

If the model quotes such a marker in its own text — which happens naturally when it is debugging this very check, writing documentation about truncation handling, or writing a regression test — the whole reply is discarded and replaced with:

[ERROR] Incomplete response. Regenerate and tooluse.

The user sees the answer stop mid-way and restart, and after 3 attempts _retry_or_exit aborts the task. This is a self-triggering false positive: the more the model explains the mechanism, the more likely it is to trip it.

Fix

llmcore always appends these markers at the very end of the response, so anchoring the match to the end is both sufficient and precise.

_tail = content.rstrip()
_MARK_TAIL = re.compile(r'\[!!!\s*(?:流异常中断|Response truncated|output truncated).*?!!!\]\s*$')
if (_MARK_TAIL.search(_tail)
        or _tail.endswith('!!!Error:') or _tail.startswith('!!!Error:')
        or (_tail.endswith('</summary>') and len(_tail) < 100)):
    return self._retry_or_exit("[ERROR] Incomplete response. Regenerate and tooluse.")

While touching this, two markers that llmcore actually emits were not checked at all and are now covered:

  • [!!! Response truncated: max_tokens !!!] (llmcore.py:245)
  • [!!! output truncated: {reason}] (llmcore.py:331, 444)

!!!Error: is matched at start/end instead of "anywhere in window", and max_tokens !!!] is matched at the end.

Verification

End-to-end against do_no_tool() with 9 cases (fake _retry_or_exit, generator drained, StopIteration.value.next_prompt inspected):

case expected result
body mentions a marker accept PASS
body quotes a full marker incl. [DONE] accept PASS
plain reply accept PASS
real streaming abort at tail retry PASS
Response truncated: max_tokens at tail retry PASS
output truncated at tail retry PASS
only <summary>, < 100 chars retry PASS
<summary> + real body accept PASS
content starts with !!!Error: retry PASS

Note the marker often contains a bracket of its own, e.g. [!!! 流异常中断,未收到完整响应(无 [DONE])!!!]. A naive [^\]]* pattern stops at the inner ] and silently fails to detect genuine truncation — .*? up to !!!] is required. This was caught by the test, not by inspection.

…N chars

do_no_tool() decided "incomplete response" whenever a truncation marker
appeared anywhere in the trailing window of the reply. If the model quotes
such a marker in its own text (debugging this very check, writing docs, or
writing a test), the whole reply is discarded and replaced with
"[ERROR] Incomplete response. Regenerate and tooluse." -- the user sees the
answer stop mid-way and restart, and after 3 attempts the task aborts.

llmcore appends these markers at the very end of the response, so an
end-anchored match is both sufficient and precise.

Changes:
- match markers of form "[!!! 流异常中断 ... !!!]",
  "[!!! Response truncated ... !!!]" and "[!!! output truncated ... !!!]"
  only when they terminate the content. Previously only the first kind was
  checked at all, so genuinely truncated responses could slip through.
- "!!!Error:" matched at start/end instead of "anywhere in window"
- "max_tokens !!!]" matched at end

Verified end-to-end against do_no_tool with 9 cases: body references are
now accepted; real tail markers (including markers that themselves contain
a bracket, e.g. "(no [DONE])") are still rejected.
@leonsong09

Copy link
Copy Markdown
Author

superseded by #880 — see #880 / #881.

Folded into #880 as commit 2 of 6, unchanged. Its ga.py hunk and the tail-anchored regex are byte-identical there — the merge was clean and the diff carries the same +12/-2 in that file.

Closing to keep one open diff per change.

@leonsong09 leonsong09 closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant