Skip to content

Pseudo-XML tool calls emitted as plain text are treated as a final answer, silently ending the task (<code_run>, <parameter name=>, <function=>, DSML) #863

Open

Description

@leonsong09

Symptom

A task stops mid-way with no error right after the assistant prints bare XML-looking tags instead of calling a tool. Copying the last lines out of the TUI gives literally:

</code_run>
<code_run>
<

</code_run>

User's words: "对话中出现这些然后中断了" — the turn ends, nothing executes, no exception, no [!!! …] marker.

Root cause

Some upstreams emit the tool call as plain content text instead of a real tool_calls array. Measured over temp/model_responses/ (session logs), the affected shape is always the same: a Response with type: text containing <code_run>…, and zero tool_use blocks.

23 such responses across 6 sessions, 2026-11-10 16:41 → 2026-11-11 10:32, every one with zero tool_use blocks, all on one DeepSeek-family model behind a local OpenAI-compatible relay (api_mode="chat_completions", stream=True). Observed spellings:

# Shape found in content
A <code_run>\n<script>…code…</script>\n</code_run>
B <code_run>\n<parameter name="script">…</parameter><parameter name="timeout">600</parameter>\n</code_run>
C <file_read">\n<parameter name="path">…</parameter>\n</file_read> (stray quote right after the tool name)
D <type>\n<parameter name="x">…</parameter>
E DSML: <||DSML||tool_calls><||DSML||invoke name="code_run"><||DSML||parameter name="script" string="true">…
F <function=code_run><parameter=script>…</parameter></function>
G empty shell: </code_run>\n<code_run>\n\n\n</code_run> ← what the user pasted
H unclosed, stream cut mid-call: <code_run>\n<script>import os\nprint(os.getcwd() (no </script>)

Two fallbacks exist. Both miss all eight:

  • llmcore._parse_text_tool_calls (main: llmcore.py:917) recognises only (1) a trailing JSON array [{"type":"tool_use",…}] and (2) <tool_use>/<tool_call> wrapping JSON. No pseudo-XML tag family, no <script>, no <parameter name=…>, no <function=…>, no DSML.
  • ga.do_no_tool (main: ga.py:471) has exactly one DSML guard: content.rstrip()[-50:] contains DSML and | and ends with >. Shapes A–H never satisfy it.

So they fall through to return StepOutcome(response, next_prompt=None) (ga.py:511), agent_loop treats that as the final answer, and the loop breaks. That is the silent stop. Worse, the pseudo-XML text is then fed back to the model as an assistant turn, so later turns get polluted with </code_run> fragments — which is how a user ends up with a screen containing nothing but bare closing/opening tags.

Note this is distinct from the leaks in #804/#837/#846: those scrub stray DSML markers out of content / tool_call.arguments. Here the marker text is the only place the intended call exists — scrubbing it throws the call away. The fix is to reconstruct the call, and to treat an unreconstructable shell as a retryable error, not a final answer.

Proposed fix (two small edits)

1) llmcore.py — reconstruct pseudo-XML calls. Add _parse_pseudo_xml_tool_calls(content), invoked from _parse_text_tool_calls when the two existing formats find nothing:

  • mask fenced ``` blocks first, so examples that merely quote the syntax are never executed;
  • per invocation collect a body, then extract args: <script>…</script> → script; <parameter name="k">v</parameter> / <parameter=k>v</parameter> (also with a ||DSML|| prefix) → k: v;
  • accept name="code_run" / name='…' / bare, and <function=name>; validate the name against assets/tools_schema.json so prose that merely contains a tag is not turned into a call;
  • reconstruct closed blocks only; strip consumed text plus leftover DSML shell tags from content.

Also export textual_tool_residue(content) → 'unclosed' / 'empty' / None, for the guard below.

2) ga.py — a residue must not pass as a final answer. In do_no_tool, after the existing guards:

_residue = llmcore.textual_tool_residue(content)
if _residue in ('unclosed', 'empty'):
    return (yield from self._retry_or_exit(
        "[ERROR] Your tool call was emitted as plain XML text …; no tool was executed. "
        "Regenerate using the native tool-call interface (not <code_run>/<parameter>/DSML text)."))

_retry_or_exit already caps at 3 consecutive failures, so this cannot loop forever. The prompt explicitly names the bogus formats, otherwise the same model just re-emits the same pseudo-XML.

Verification

Added tests/test_text_tool_call.py (3 tests, stdlib-only, no network):

  • positives: all eight shapes → expected MockToolCall + args;
  • negatives: fenced block, ordinary prose, <thinking>-only → zero calls;
  • residue: G → 'empty', H → 'unclosed', rest → None.

End-to-end with a fake raw_ask that streams such text and returns [{"type":"text",…}] (no native tool_use — the real failing condition):

[A code_run+script ]  tools=[('code_run', '{"script": "import os\\nprint(...)"}')]   stop=tool_use
[B parameter name  ]  tools=[('code_run', '{"script": "print(...)", "timeout": 600}')]  stop=tool_use
[C DSML            ]  tools=[('code_run', '{"script": "print(2)"}')]                stop=tool_use
[D function=       ]  tools=[('code_run', '{"script": "print(3)"}')]                stop=tool_use
[E normal prose    ]  tools=[]   content unchanged                                  stop=end_turn

and driving do_no_tool with MockResponse:

content before after
user's </code_run><code_run></code_run> next_prompt=None → task ends retry + [Info] Detected a textual tool block with no arguments. Regenerating.
unclosed <code_run>\n<script>… next_prompt=None → task ends retry + [Info] Detected an unterminated textual tool block (stream cut mid-call).
normal prose next_prompt=None unchanged

pytest tests/ -q → 17 passed (3 new), no regressions.

Activity

  1. aron-intframe commented on Nov 11, 2026

    @aron-intframe

    Your eight shapes reproduce on current main, and the open PR #853 (which targets this same failure) closes most of them but not all. Before you open a separate PR, these are the gaps I measured on it. Two of them overlap your proposed fix directly.

    What I ran. I fed each of your shapes A–H as plain content with zero native tool calls. Each one went first through llmcore._parse_text_tool_calls, then (if nothing came back) through GenericAgentHandler.do_no_tool. dispatch was stubbed so nothing actually ran. The trees were main 68d996a and PR #853 head cae8519:

    == GenericAgent (68d996a)
    [A code_run+script ] FINAL ANSWER -> loop stops silently
    [B parameter name= ] FINAL ANSWER -> loop stops silently
    [C stray quote     ] FINAL ANSWER -> loop stops silently
    [D <type> wrapper  ] FINAL ANSWER -> loop stops silently
    [E DSML            ] retry: [ERROR] Tool call format invalid; no tool was called. Regene
    [F function=       ] FINAL ANSWER -> loop stops silently
    [G empty shell     ] FINAL ANSWER -> loop stops silently
    [H unclosed        ] FINAL ANSWER -> loop stops silently
    == GA-pr853 (cae8519)
    [A code_run+script ] parsed as tool call [["code_run", "{\"script\": \"print(1)\"}"]]
    [B parameter name= ] parsed as tool call [["code_run", "{\"script\": \"print(1)\", \"timeout\": 600}"]]
    [C stray quote     ] parsed as tool call [["file_read", "{\"dsml_parameter\": \"/tmp/a.txt\"}"]]
    [D <type> wrapper  ] retry: [ERROR] 上一轮没有产生有效的工具调用:正文里混入了残缺的工具调用协议标记(DSML / <code_run> 等
    [E DSML            ] parsed as tool call [["code_run", "{\"script\": \"print(1)\"}"]]
    [F function=       ] FINAL ANSWER -> loop stops silently
    [G empty shell     ] retry: [ERROR] 上一轮没有产生有效的工具调用:正文里混入了残缺的工具调用协议标记(DSML / <code_run> 等
    [H unclosed        ] parsed as tool call [["code_run", "{\"script\": \"import os\\nprint(os.getcwd()\"}"]]
    == GA-pr853 guards on H
    H via _parse_text_tool_calls -> [('code_run', '{"script": "import os\\nprint(os.getcwd()"}')]
    H leaked_markup_incomplete    -> True
    H salvaged_call_problem       -> python 语法错误(第 2 行: '(' was never closed)
    

    What #853 still gets wrong

    1. F (<function=code_run><parameter=script>) still ends the task silently on fix: 伪工具调用救回机制全面加固(误执行 0 / 漏检 0) #853, the same as on main.
    2. H (a call cut off mid-stream) gets executed. fix: 伪工具调用救回机制全面加固(误执行 0 / 漏检 0) #853 has the right guards: leaked_markup_incomplete and salvaged_call_problem both flag H (output above). But they only run inside do_no_tool. fix: 伪工具调用救回机制全面加固(误执行 0 / 漏检 0) #853 also hooks salvage_leaked_tool_calls into _parse_text_tool_calls, and that function feeds the normal response path (llmcore.py:924). There, H turns into a real code_run before do_no_tool is ever reached. Here the truncated script happens to be a syntax error, so it only wastes a turn. A cut that happens to leave valid Python would run half a script.
    3. C (stray quote) is rebuilt with the wrong argument name: {"dsml_parameter": "/tmp/a.txt"} instead of {"path": ...}, so file_read gets called without a path.

    How to fix it. Run the same two checks inside _parse_text_tool_calls before it returns salvaged calls. If either one fires, return no calls, so do_no_tool takes over and retries. Add the <function=name><parameter=k> form, which your proposal already covers. Map <parameter name="k"> to k no matter what quoting the tool tag uses. All eight of your shapes plus the H case above would make a good regression table for whichever PR lands.

    We hit the same pattern (the agent loop reads "nothing happened" as a finished answer). The rule we use for it is item 1 in our repo (ours): https://github.com/aron-intframe/agent-guards#1-the-guard-that-prints-nothing

  2. leonsong09 commented on Nov 11, 2026

    @leonsong09
    Author

    Thanks — all three gaps you measured are now closed in #864, and the two that overlapped are addressed by construction rather than by adding a second pass.

    C (stray quote). The argument name is parsed independently of the tool tag:

    r'<\s*parameter\s*(?:name\s*)?=\s*["\']?([\w.-]+)["\']?[^<>]*>(.*?)</\s*parameter\s*>'

    so <file_read"> no longer leaks a quote into the key and there is no dsml_parameter fallback path to get it wrong. <parameter name="k">, <parameter=k>, and the ||DSML|| prefix all go through the same expression. Covered by a shape-C test asserting {"path": "/tmp/a.txt"}.

    F (<function=name><parameter=k>). Separate form for the block opener:

    r'<function\s*=\s*["\']?([A-Za-z_]\w*)["\']?\s*>(.*?)</function\s*>'

    Body goes through the same <script> / <parameter> extraction, so print(1) lands as {"script": "print(1)"}.

    H (cut mid-stream) — the one you flagged as running half a script. You were right that this has to be decided before _parse_text_tool_calls returns. The gate is now inside it:

    call = MockToolCall(b['name'], args)
    if salvaged_call_problem(call): return [], content   # half a script is never a call

    and a closed block whose script fails ast.parse is not silently dropped either — textual_tool_residue returns 'invalid' so do_no_tool retries. Two test cases pin both halves of your point: the cut that leaves a syntax error, and the cut that happens to leave valid Python (import os\nprint(1)), which would otherwise have executed.

    textual_tool_residue now returns unclosed / empty / orphan / invalid; do_no_tool retries on any of them. Shape D (<type>-wrapped parameters) lands in orphan and retries instead of ending the task — that was still broken in the version I had.

    One more silent-stop I found while wiring this up: _retry_or_exit returned StepOutcome({}, should_exit=True) with nothing printed after three unusable replies, so the give-up path was itself invisible. It now yields a [任务中止] note first.

    pytest tests/test_text_tool_call.py -q → 6 passed, covering A–H plus the negatives (prose, inline code, fenced block at offset 0, JSON example). A fenced block that starts at index 0 was in fact being executed before — the fence scanner used 0 as its "not open" sentinel.

    #864: #864

  3. leonsong09 commented on Nov 14, 2026

    @leonsong09
    Author

    This has converged, so here is the state for anyone landing on this issue later.

    The root cause, in one sentence

    A turn is allowed to end without a tool call whenever nothing looked like a tool call — and every failure shape below produces something that doesn't look like one.

    report what it looked like what actually happened
    #863 (this) plain-text tool call parsed as prose → do_no_tool → "final answer"
    #860 reply quoting a truncation marker do_no_tool matched the marker mid-text, discarded the reply
    #875 nothing code_run slept a flat second; not a bug, just 1s of every call
    #866 a finished report 600s reply-wait after the work was done, killed at rc=-999
    #877 results in the right order empty tool_use_id path prepends → order reversed

    Where it is now

    Two open PRs carry everything, both rebased onto 68d996a with linear history and no merge commits:

    The single invariant #880 enforces:

    A turn may only end without a tool call if the model produced usable output and nothing is left dangling.

    _has_usable_output() and textual_tool_residue() (unclosed / empty / orphan / invalid) are the two halves of that sentence.

    On the three gaps @aron-intframe measured in #853

    All three are closed in #880, structurally rather than by a second pass:

    • F (<function=name><parameter=k>) — separate block-opener regex, so it no longer falls through to a final answer.
    • H (cut mid-stream) — the guard is inside _parse_text_tool_calls, before a salvaged call can become real. Two cases are pinned: the cut that leaves a syntax error, and the cut that happens to leave valid Python (which would otherwise have executed half a script).
    • C (stray quote) — the argument name is parsed independently of the tool tag, so <file_read"> yields {"path": ...} and there is no dsml_parameter fallback to get it wrong.

    Shape D (<type>-wrapped) returns orphan and retries; E (||DSML||) executes; G retries.

    If you're hitting this from a different upstream

    The list of spellings in #863 is the useful artifact here, not the patch. Any agent loop that reads "no tool call seen" as "the model is done" will reproduce the same silent stop — #853 hit it independently on this repo, and the shapes above are how we found out which spellings actually occur in the wild rather than which ones are theoretically possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions