Skip to content

fix: preserve tool_calls in multi-turn history and skip malformed tool-arg JSON for OpenAI-compatible backends#1373

Open
mvanhorn wants to merge 2 commits into
generative-computing:mainfrom
mvanhorn:fix/1349-multiturn-tool-calls-openai
Open

fix: preserve tool_calls in multi-turn history and skip malformed tool-arg JSON for OpenAI-compatible backends#1373
mvanhorn wants to merge 2 commits into
generative-computing:mainfrom
mvanhorn:fix/1349-multiturn-tool-calls-openai

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Multi-turn agent loops (start_session + repeated instruct(..., tool_calls=True)) now replay correctly on OpenAI-compatible backends. Two bugs from #1349 broke them: tool-only assistant turns lost their tool calls when serialized back into history, and a truncated streamed tool-argument string crashed the run.

Why this matters

The reporter hit both on OpenRouter with a free Cohere model and gave full tracebacks and a repro.

  1. History serialization dropped tool calls. Message._parse stored a tool-only assistant turn as str(msg["tool_calls"]) in content, and message_to_openai_message then emitted only {"role", "content"} with that Python repr and no tool_calls key. Spec-strict providers reject the replayed turn with 400 ... must have non-empty content or tool calls, so the second instruct in a loop fails.
  2. Malformed tool args crashed the agent. extract_model_tool_requests called json.loads(tool_args) bare; a truncated argument string during streaming raised json.JSONDecodeError that propagated up through backends/openai.py and killed the run.

Changes

  • mellea/stdlib/components/chat.py: Message takes an optional keyword-only tool_calls list (with a tool_calls property). _parse passes the raw OpenAI-shape list through for the openai/watsonx/litellm branch and builds spec-correct dicts (via the existing build_tool_calls) for the ollama branch, instead of stringifying. content stays a str (empty when the provider sent null) so nothing that reads msg.content breaks.
  • mellea/helpers/openai_compatible_helpers.py: message_to_openai_message emits tool_calls when the message carries them, with content: None for tool-only turns (OpenAI spec). extract_model_tool_requests wraps the json.loads in a try/except that warns and skips the bad call, mirroring the existing unknown-tool skip four lines up.

Scoped to the two reported bugs. The role: "tool" / tool_call_id feedback half (the reporter currently feeds tool results back as user text) is left as a follow-up if you'd like it.

Testing

  • pytest test/helpers/test_openai_compatible_helpers.py test/stdlib/components/test_chat.py: 73 pass, covering truncated tool args (single-bad returns None + warns; good+bad returns only the good), message_to_openai_message emitting content: None + tool_calls for a tool-only Message (and unchanged for a plain one), _parse preserving the raw list for the openai and ollama branches, and an end-to-end history round-trip matching the OpenAI shape from the issue.
  • pytest test/backends/test_openai_unit.py: 27 pass (no regressions).
  • ruff check on the changed modules: clean.

Closes #1349

…l-arg JSON (OpenAI-compatible backends)

Two bugs broke multi-turn agent loops (start_session + repeated
instruct(..., tool_calls=True)) on OpenAI-compatible backends:

1. History serialization dropped tool calls. A tool-only assistant turn was
   stored as str(msg['tool_calls']) in Message.content, and
   message_to_openai_message emitted only {role, content} with that repr and no
   tool_calls key. Spec-strict providers (Cohere via OpenRouter, etc.) rejected
   the replayed history with '400 ... must have non-empty content or tool
   calls'. Message now carries an optional tool_calls list; _parse passes the
   raw OpenAI-shape list through (and builds it for the ollama branch), and the
   serializer emits tool_calls with content: null for tool-only turns.

2. Truncated streamed tool arguments raised an uncaught json.JSONDecodeError in
   extract_model_tool_requests that killed the agent. The parse is now wrapped
   to warn and skip the bad call, mirroring the existing unknown-tool skip.

Scoped to the two reported bugs; the role:'tool'/tool_call_id feedback half is
left as a follow-up.

Closes generative-computing#1349
@mvanhorn mvanhorn requested a review from a team as a code owner July 8, 2026 19:11
@github-actions github-actions Bot added the bug Something isn't working label Jul 8, 2026
@psschwei

psschwei commented Jul 8, 2026

Copy link
Copy Markdown
Member

Thanks for the PR @mvanhorn ! As an FYI, you'll need to sign-off so this can pass this DCO check in order to merge.

@AngeloDanducci AngeloDanducci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits but overall looks good, thanks for the PR @mvanhorn

Appears worth opening an issue to track this portion you mentioned as well:
The role: "tool" / tool_call_id feedback half (the reporter currently feeds tool results back as user text) is left as a follow-up if you'd like it.

Comment thread mellea/helpers/openai_compatible_helpers.py Outdated
Comment thread mellea/stdlib/components/chat.py Outdated
Normalize absent/empty tool_calls to None when reconstructing Messages,
and document the tool-only assistant turn shape in the helper docstring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
@mvanhorn

Copy link
Copy Markdown
Author

Nits addressed in b977987 (signed off for DCO): tool_calls now normalizes to None instead of an empty list when reconstructing Messages, and the helper docstring documents the tool-only assistant turn shape. Verified locally with 117 tests passing across the chat component and helper suites.

@AngeloDanducci AngeloDanducci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, my comments are covered. A small mypy fix is needed for CI otherwise LGTM.

Comment on lines +217 to 220
result = {
"role": msg.role,
"content": [{"type": "text", "text": content}, *img_list],
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
result = {
"role": msg.role,
"content": [{"type": "text", "text": content}, *img_list],
}
result: dict[str, Any] = {
"role": msg.role,
"content": [{"type": "text", "text": content}, *img_list],
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-turn tool calling is broken for OpenAI-compatible backends

3 participants