fix: preserve tool_calls in multi-turn history and skip malformed tool-arg JSON for OpenAI-compatible backends#1373
Conversation
…l-arg JSON (OpenAI-compatible backends)
Two bugs broke multi-turn agent loops (start_session + repeated
instruct(..., tool_calls=True)) on OpenAI-compatible backends:
1. History serialization dropped tool calls. A tool-only assistant turn was
stored as str(msg['tool_calls']) in Message.content, and
message_to_openai_message emitted only {role, content} with that repr and no
tool_calls key. Spec-strict providers (Cohere via OpenRouter, etc.) rejected
the replayed history with '400 ... must have non-empty content or tool
calls'. Message now carries an optional tool_calls list; _parse passes the
raw OpenAI-shape list through (and builds it for the ollama branch), and the
serializer emits tool_calls with content: null for tool-only turns.
2. Truncated streamed tool arguments raised an uncaught json.JSONDecodeError in
extract_model_tool_requests that killed the agent. The parse is now wrapped
to warn and skip the bad call, mirroring the existing unknown-tool skip.
Scoped to the two reported bugs; the role:'tool'/tool_call_id feedback half is
left as a follow-up.
Closes generative-computing#1349
|
Thanks for the PR @mvanhorn ! As an FYI, you'll need to sign-off so this can pass this DCO check in order to merge. |
There was a problem hiding this comment.
A few nits but overall looks good, thanks for the PR @mvanhorn
Appears worth opening an issue to track this portion you mentioned as well:
The role: "tool" / tool_call_id feedback half (the reporter currently feeds tool results back as user text) is left as a follow-up if you'd like it.
Normalize absent/empty tool_calls to None when reconstructing Messages, and document the tool-only assistant turn shape in the helper docstring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
|
Nits addressed in b977987 (signed off for DCO): tool_calls now normalizes to None instead of an empty list when reconstructing Messages, and the helper docstring documents the tool-only assistant turn shape. Verified locally with 117 tests passing across the chat component and helper suites. |
AngeloDanducci
left a comment
There was a problem hiding this comment.
Thanks for the update, my comments are covered. A small mypy fix is needed for CI otherwise LGTM.
| result = { | ||
| "role": msg.role, | ||
| "content": [{"type": "text", "text": content}, *img_list], | ||
| } |
There was a problem hiding this comment.
| result = { | |
| "role": msg.role, | |
| "content": [{"type": "text", "text": content}, *img_list], | |
| } | |
| result: dict[str, Any] = { | |
| "role": msg.role, | |
| "content": [{"type": "text", "text": content}, *img_list], | |
| } |
Summary
Multi-turn agent loops (
start_session+ repeatedinstruct(..., tool_calls=True)) now replay correctly on OpenAI-compatible backends. Two bugs from #1349 broke them: tool-only assistant turns lost their tool calls when serialized back into history, and a truncated streamed tool-argument string crashed the run.Why this matters
The reporter hit both on OpenRouter with a free Cohere model and gave full tracebacks and a repro.
Message._parsestored a tool-only assistant turn asstr(msg["tool_calls"])incontent, andmessage_to_openai_messagethen emitted only{"role", "content"}with that Python repr and notool_callskey. Spec-strict providers reject the replayed turn with400 ... must have non-empty content or tool calls, so the secondinstructin a loop fails.extract_model_tool_requestscalledjson.loads(tool_args)bare; a truncated argument string during streaming raisedjson.JSONDecodeErrorthat propagated up throughbackends/openai.pyand killed the run.Changes
mellea/stdlib/components/chat.py:Messagetakes an optional keyword-onlytool_callslist (with atool_callsproperty)._parsepasses the raw OpenAI-shape list through for the openai/watsonx/litellm branch and builds spec-correct dicts (via the existingbuild_tool_calls) for the ollama branch, instead of stringifying.contentstays astr(empty when the provider sent null) so nothing that readsmsg.contentbreaks.mellea/helpers/openai_compatible_helpers.py:message_to_openai_messageemitstool_callswhen the message carries them, withcontent: Nonefor tool-only turns (OpenAI spec).extract_model_tool_requestswraps thejson.loadsin a try/except that warns and skips the bad call, mirroring the existing unknown-tool skip four lines up.Scoped to the two reported bugs. The
role: "tool"/tool_call_idfeedback half (the reporter currently feeds tool results back as user text) is left as a follow-up if you'd like it.Testing
pytest test/helpers/test_openai_compatible_helpers.py test/stdlib/components/test_chat.py: 73 pass, covering truncated tool args (single-bad returns None + warns; good+bad returns only the good),message_to_openai_messageemittingcontent: None+tool_callsfor a tool-only Message (and unchanged for a plain one),_parsepreserving the raw list for the openai and ollama branches, and an end-to-end history round-trip matching the OpenAI shape from the issue.pytest test/backends/test_openai_unit.py: 27 pass (no regressions).ruff checkon the changed modules: clean.Closes #1349