Replay OpenAI tool calls as plain, stable JSON

[gtokman]

Fixes #324, and a prompt-caching miss it uncovered.

1. Replayed arguments were GeneratedContent's Codable form

OpenAILanguageModel encoded each earlier tool call's arguments with JSONEncoder().encode(call.arguments), which writes {"kind":{"type":"structure","properties":…}} rather than the arguments' JSON. Models that see that shape in the history copy it into their next tool calls, which then fail to decode and end the response. The .responses variant builds its function_call items from the same message, so it was affected too.

2. A tool round was sent differently in flight and in history

During a turn, the assistant message carrying tool calls was the model's raw message (its own argument text, "content": ""). On the next turn, the same round was rebuilt from the transcript (re-encoded arguments, "content": null). The request stopped matching the previous one at the first tool call, so a provider's prompt cache (Anthropic's is an exact-prefix match) missed everything after it on every later turn. We measured Claude reading only the system prompt from cache on turn 2 of a conversation, instead of all of turn 1.

Both are now built by one helper from the transcript's calls (Chat Completions; the Responses variant replays output items and is unchanged). Arguments are serialized with sorted keys, because jsonString goes through an unordered [String: Any], so the same arguments could otherwise come out in a different key order from one request to the next. That is also why the second test failed intermittently before the sort, depending on which suites ran alongside it.

Tests

OpenAIToolCallHistoryTests (own URLProtocol, like OllamaRequestHistoryTests):

• replayedToolCallArgumentsArePlainJSON: both API variants replay {"city":"Paris"}. Fails on main.
• toolCallMessageIsTheSameInFlightAndInHistory: streaming and non-streaming, a tool whose arguments the model writes in the opposite order from its schema. The tool-call message in the in-flight request equals the one in the next turn's request. Fails without the change.

swift test --filter "OpenAI|Transcript|ProviderUsage|ProviderSchema" passes (114 tests), and swift format lint --strict is clean on the changed files.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B4WBsc1Jp8fGufe89MhQFB

[Copilot]

🟡 Changes recommended

Non-empty tool-call text still differs between in-flight and replayed messages, defeating prompt-cache stability.

1 open finding

• Preserve assistant content during tool-call transcript replay · New

What changed in this PR

Fixes OpenAI tool-call replay by using stable plain JSON and aligning in-flight/history message construction.

Changes:

• Canonicalizes replayed tool arguments with sorted JSON keys.
• Adds history and prompt-cache regression tests.

File	Description
OpenAILanguageModel.swift	Unifies tool-call message serialization.
OpenAIToolCallHistoryTests.swift	Tests plain JSON replay and message consistency.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[mattt]

Thanks, @gtokman. CI is green. Copilot caught one more case (here): when the model writes text alongside its tool calls, the in-flight message sends that text as content, but the replay passes nil, so the two still differ. Could you either keep that text for replay or send null in both paths, and add a test with non-empty content?