Repository navigation
fix(llmcore): don't replay partial output on retry; treat empty/thinking-only reply as failure - #869
Closed
leonsong09 wants to merge 1 commit into
Closed
leonsong09 wants to merge 1 commit into
leonsong09 wants to merge 1 commit into
Conversation
…ing-only reply as failure Two independent defects in the streaming retry path, both reproduced with a local fake SSE server (no network): 1. Partial output was replayed on retry. `_stream_with_retry` retried Timeout/ConnectionError even when chunks had already been yielded to the caller, so the already-delivered half was emitted again on every attempt. A single interrupted stream could reach the user as "AAABBBAAABBBAAABBB". MixinSession had the same problem one level up: on a retryable error it switched channel and re-ran the whole request. Now: once anything has been yielded the stream is never retried; a `[!!! 流异常中断 ...]` marker is appended instead (same convention as the generic except-branch). MixinSession keeps the partial output, only routes the *next* call to another channel. 2. An empty answer (or one made only of `thinking` blocks) was reported as a successful empty reply, so no retry and no failover ever happened and the user just saw nothing. Now `_has_usable_output()` requires non-blank text or a tool_use block; otherwise a ConnectionError is raised so the normal retry / failover path applies. Tool-only responses are still valid answers. Tests: tests/test_stream_retry.py (6 cases, 16 assertions). Fails 6 assertions on the unpatched tree, all pass after the fix. Co-Authored-By: GenericAgent <bot@gaagent.ai>
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
llmcore的流式重试路径上有两个独立缺陷,叠加起来正好造成"界面卡很久,然后突然蹦出一段,或者干脆什么都没有"。1. 半截正文被重放
_stream_with_retry在Timeout / ConnectionError / ChunkedEncodingError分支里,只要attempt < max_retries就continue重试,不检查是否已经 yield 过内容。调用方(UI / 上层)已经收到的前半段,会在下一次尝试里从头再来一遍。实测(本地假 SSE 服务器,第 1 次尝试发
AAA BBB后断流,max_retries=3):MixinSession在同一层次上也有同样问题:拿到!!!Error:就换渠道重跑整个请求,同样会造成重复输出。2. 空回答 / 只有 thinking 的回答被当成"成功"
_stream_with_retry里判空用的是:只要求
blocks非空。但"只有 thinking 块、没有正文、没有 tool_call"的响应,blocks是非空的 —— 于是判为成功,既不重试也不换渠,用户端拿到一个空字符串。MixinSession.test_error也只认!!!Error:/[Error:前缀,同样兜不住。结果就是:模型把 token 全烧在思考上、或者上游吞了正文时,界面一片空白,且系统认为一切正常。
改动
llmcore.py,三处,共 +46/-3:_has_usable_output(blocks):text非空白 或 存在tool_use才算有效输出(纯 tool_call 的响应仍然算正常,不会被误判)。_stream_with_retry:StopIteration处改用_has_usable_output()判空;if streamed守卫:已经吐出内容就不再重试,只追加[!!! 流异常中断 ...]标记(沿用文件里已有的通用 except 分支的同一约定)。MixinSession.raw_ask:is_err and yielded时不再换渠重跑,保留本轮已输出的部分,只把下一次调用指到另一个渠道。测试
新增
tests/test_stream_retry.py(6 个用例 / 16 条断言,全部走本地ThreadingHTTPServer假 SSE,不联网、确定性):反向验证:同一份测试在未打补丁的树上失败 6 条断言,打补丁后 16/16 全过。
兼容性
finish_reason=length等既有路径未改动。stream_hide_reasoning等)。Co-Authored-By: GenericAgent bot@gaagent.ai