Skip to content

fix(llmcore): don't replay partial output on retry; treat empty/thinking-only reply as failure - #869

Closed
leonsong09 wants to merge 1 commit into
lsdefine:mainfrom
leonsong09:fix/stream-partial-retry-and-empty-response
Closed

leonsong09 wants to merge 1 commit into
lsdefine:mainfrom
leonsong09:fix/stream-partial-retry-and-empty-response

Conversation

@leonsong09

@leonsong09 leonsong09 commented Nov 12, 2026 •

Copy link
Copy Markdown

问题

llmcore 的流式重试路径上有两个独立缺陷,叠加起来正好造成"界面卡很久,然后突然蹦出一段,或者干脆什么都没有"。

1. 半截正文被重放

_stream_with_retry 在 Timeout / ConnectionError / ChunkedEncodingError 分支里,只要 attempt < max_retries 就 continue 重试,不检查是否已经 yield 过内容。调用方(UI / 上层)已经收到的前半段,会在下一次尝试里从头再来一遍。

实测(本地假 SSE 服务器,第 1 次尝试发 AAA BBB 后断流,max_retries=3):

流式正文 = 'AAABBBAAABBBAAABBB!!!Error: ChunkedEncodingError: Response ended prematurely'
请求次数 = 3

MixinSession 在同一层次上也有同样问题:拿到 !!!Error: 就换渠道重跑整个请求,同样会造成重复输出。

2. 空回答 / 只有 thinking 的回答被当成"成功"

_stream_with_retry 里判空用的是:

if not e.value and not streamed: raise requests.ConnectionError("empty response")

只要求 blocks 非空。但"只有 thinking 块、没有正文、没有 tool_call"的响应,blocks 是非空的 —— 于是判为成功,既不重试也不换渠,用户端拿到一个空字符串。MixinSession.test_error 也只认 !!!Error: / [Error: 前缀,同样兜不住。

结果就是:模型把 token 全烧在思考上、或者上游吞了正文时,界面一片空白,且系统认为一切正常。

改动

llmcore.py,三处,共 +46/-3:

  1. 新增 _has_usable_output(blocks):text 非空白 或 存在 tool_use 才算有效输出(纯 tool_call 的响应仍然算正常,不会被误判)。
  2. _stream_with_retry:
    • StopIteration 处改用 _has_usable_output() 判空;
    • 网络异常分支加 if streamed 守卫:已经吐出内容就不再重试,只追加 [!!! 流异常中断 ...] 标记(沿用文件里已有的通用 except 分支的同一约定)。
  3. MixinSession.raw_ask:is_err and yielded 时不再换渠重跑,保留本轮已输出的部分,只把下一次调用指到另一个渠道。

测试

新增 tests/test_stream_retry.py(6 个用例 / 16 条断言,全部走本地 ThreadingHTTPServer 假 SSE,不联网、确定性):

用例 断言
1 半截后断流 前半段只出现一次;有中断标记;不再发第二次请求
2 完全空响应 重试一次后返回错误(不再静默成功)
3 仅 thinking 产出显式标记文本块,标记能到达调用方
4 仅 tool_call 工具块保留、不判为空、只请求一次(防误判)
5 正常响应 原样返回,无标记,只请求一次(回归)
6 Mixin 双渠 渠道 1 的半截保留一次、渠道 2 不被重放、下一次调用路由到渠道 2

反向验证:同一份测试在未打补丁的树上失败 6 条断言,打补丁后 16/16 全过。

python tests/test_stream_retry.py    # ALL PASS (rc=0)

兼容性

  • 只影响"流已经产出内容之后出错"和"响应没有任何可用内容"这两条异常路径,正常路径行为不变。
  • 纯 tool_call 响应、finish_reason=length 等既有路径未改动。
  • 不依赖任何本地私有特性(stream_hide_reasoning 等)。

Co-Authored-By: GenericAgent bot@gaagent.ai

…ing-only reply as failure

Two independent defects in the streaming retry path, both reproduced with a
local fake SSE server (no network):

1. Partial output was replayed on retry.
   `_stream_with_retry` retried Timeout/ConnectionError even when chunks had
   already been yielded to the caller, so the already-delivered half was
   emitted again on every attempt. A single interrupted stream could reach the
   user as "AAABBBAAABBBAAABBB". MixinSession had the same problem one level up:
   on a retryable error it switched channel and re-ran the whole request.

   Now: once anything has been yielded the stream is never retried; a
   `[!!! 流异常中断 ...]` marker is appended instead (same convention as the
   generic except-branch). MixinSession keeps the partial output, only routes
   the *next* call to another channel.

2. An empty answer (or one made only of `thinking` blocks) was reported as a
   successful empty reply, so no retry and no failover ever happened and the
   user just saw nothing.

   Now `_has_usable_output()` requires non-blank text or a tool_use block;
   otherwise a ConnectionError is raised so the normal retry / failover path
   applies. Tool-only responses are still valid answers.

Tests: tests/test_stream_retry.py (6 cases, 16 assertions). Fails 6 assertions
on the unpatched tree, all pass after the fix.

Co-Authored-By: GenericAgent <bot@gaagent.ai>
@leonsong09

Copy link
Copy Markdown
Author

superseded by #880 — see #880 / #881.

Folded into #880 as commit 3 of 6, unchanged (_has_usable_output, the if streamed guard, the MixinSession change, tests/test_stream_retry.py).

Closing to keep one open diff per change.

@leonsong09 leonsong09 closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant