Skip to content

AG2 and Langroid Bots: OpenAI's newest models take tools through the Responses API - #1777

Merged
davidmckayv merged 2 commits into
mainfrom
fix/openai-responses-ag2-langroid
Nov 17, 2026
Merged

davidmckayv merged 2 commits into
mainfrom
fix/openai-responses-ag2-langroid

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

Why

The model compatibility probe in #1775 found the AG2 and Langroid Bots refusing OpenAI's newest models, in OpenAI's own words:

  • gpt-5.6-*, gpt-6*, gpt-6.1-*, gpt-6.2-*: "Function tools with reasoning_effort are not supported for this model in /v1/chat/completions. To use function tools, use /v1/responses".
  • gpt-5-pro, gpt-5.2-pro, gpt-5.4-pro, gpt-5.5-pro, o3-pro: "This is not a chat model and thus not supported in the v1/chat/completions endpoint."
  • Langroid only, gpt-5.5: "Unsupported value: 'temperature' does not support 0.2 with this model."
  • Langroid only, o3: "Unsupported parameter: 'parallel_tool_calls' is not supported with this model."

Both Bots sent every OpenAI model to Chat Completions, and Langroid sent temperature and parallel_tool_calls to models that refuse them.

What

One rule for which API a model takes. shared/model_providers.py gains requires_responses_api and openai_api, the Python copy of requiresResponsesApi in shared/model-providers.ts. The model decides, not the address, so a gateway in front of OpenAI gets the Responses API for these models too. COPILOTKIT_OPENAI_API=responses or chat still wins, as documented.

AG2. AG2 1.3.2 ships OpenAIResponsesConfig, so the Bot uses it for those models and OpenAICompletionsConfig for the rest. Both are named explicitly: AG2 deprecates the bare OpenAIConfig and turns it into the Responses config in 1.4.0, which would move every OpenAI-compatible endpoint (most serve only Chat Completions) onto an API they lack. ag2>=1.3.2. No adapter code.

Langroid. Langroid has no Responses API support up to its latest release (0.69.24, today), and its own model table stops at gpt-5.2, so it sends temperature to gpt-5.5. Langroid already routes models through litellm, so an OpenAI model litellm knows now goes that way: litellm/openai/<model>, or litellm/openai/responses/<model> for the Responses-only ones. litellm's model map decides which fields each model takes (Langroid turns on its drop_params), and its responses/ route sends the chat to /v1/responses and hands back a chat completion, so Langroid's tool calls and the AG-UI bridge work unchanged. A model litellm does not know (a local model behind an OpenAI-compatible endpoint, possibly with no key) stays on Langroid's own client, as before. litellm>=1.105.7,<1.106.0. No adapter code.

AG-UI behaviour is the same: tool calls stream as before, the browser's tool result continues the run, and the text answer streams. The real round trip below checks each step on every model.

Sources

Tests

Each harness gains tests/test_responses_api.py. The real framework, provider SDK and AG-UI stack run; only the provider at OPENAI_BASE_URL is local, and it refuses what OpenAI refuses, in OpenAI's words. Each test streams a tool call over AG-UI, sends the tool's result back as the browser does, and checks the answer uses it. They check the request shape: Responses-only models reach /v1/responses with the tool in Responses form and the result as a function_call_output; gpt-5.5, o3 and gpt-4.1-mini stay on Chat Completions with no temperature on the reasoning models and no parallel_tool_calls on o3.

Harness New tests on main's code (red) New tests on this branch (green) Full suite before Full suite after
agent-ag2 7 failed, 4 passed 11 passed 11 passed 22 passed
agent-langroid 8 failed, 2 passed 10 passed 14 passed 24 passed

AG2's four tests that pass on main are the Chat Completions ones: AG2 never sent temperature or parallel_tool_calls there, and the tests now hold it to that.

Functional check

Run on a throwaway Linux box with the dev OpenAI key, against every OpenAI model the picker lists (29). "Before" is the published latest image, "after" is the image built from this branch.

#1775's probe (scripts/model-compat.ts --harness agent-ag2,agent-langroid --provider openai): one AG-UI run per model, carrying one tool.

Harness Before After
agent-ag2 14 passed, 15 failed 29 passed, 0 failed
agent-langroid 12 passed, 17 failed 29 passed, 0 failed
Model AG2 before AG2 after Langroid before Langroid after
gpt-4.1 pass pass pass pass
gpt-4.1-mini pass pass pass pass
gpt-4o pass pass pass pass
gpt-4o-mini pass pass pass pass
gpt-5 pass pass pass pass
gpt-5-mini pass pass pass pass
gpt-5-nano pass pass pass pass
gpt-5-pro fail: responses only pass fail: responses only pass
gpt-5.1 pass pass pass pass
gpt-5.2 pass pass pass pass
gpt-5.2-pro fail: not a chat model pass fail: not a chat model pass
gpt-5.4 pass pass pass pass
gpt-5.4-mini pass pass pass pass
gpt-5.4-nano pass pass pass pass
gpt-5.4-pro fail: not a chat model pass fail: not a chat model pass
gpt-5.5 pass pass fail: temperature refused pass
gpt-5.5-pro fail: not a chat model pass fail: not a chat model pass
gpt-5.6-luna fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-5.6-sol fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-5.6-terra fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6-astra fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6-luna fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6-sol fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6.1-sol fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6.1-terra fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6.2-astra fail: tools need /v1/responses pass fail: tools need /v1/responses pass
gpt-6.2-sol fail: tools need /v1/responses pass fail: tools need /v1/responses pass
o3 pass pass fail: parallel_tool_calls refused pass
o3-pro fail: not a chat model pass fail: not a chat model pass

A real tool round trip, since the probe's prompt does not call its tool: run 1 asks for look_up_order and must stream the call with its arguments; run 2 sends the thread back with the tool's result and must stream an answer that uses it. Same 29 models.

Harness Before After
agent-ag2 14 passed, 15 failed 29 passed, 0 failed
agent-langroid 12 passed, 17 failed 29 passed, 0 failed

Every failure before was run 1, with the same refusals as the probe. After, every model called the tool with {"order": "A-1001"} and answered from its result, with the same AG-UI events per run as before.

…Responses API

gpt-5.6 and later and the -pro models take function tools only on /v1/responses, so both Bots now send those models there, decided by the model as every Bot in the box decides it (requires_responses_api, now in shared/model_providers.py). AG2 names OpenAIResponsesConfig and OpenAICompletionsConfig explicitly. Langroid sends OpenAI models litellm knows through litellm, whose model map leaves out the fields a model refuses (temperature on gpt-5.5, parallel_tool_calls on o3) and whose responses/ route carries a chat with tools to /v1/responses.
@davidmckayv
davidmckayv merged commit 05c9c75 into main Nov 17, 2026
16 checks passed
@davidmckayv
davidmckayv deleted the fix/openai-responses-ag2-langroid branch November 17, 2026 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant