Repository navigation
AG2 and Langroid Bots: OpenAI's newest models take tools through the Responses API - #1777
Merged
Merged
Conversation
…Responses API gpt-5.6 and later and the -pro models take function tools only on /v1/responses, so both Bots now send those models there, decided by the model as every Bot in the box decides it (requires_responses_api, now in shared/model_providers.py). AG2 names OpenAIResponsesConfig and OpenAICompletionsConfig explicitly. Langroid sends OpenAI models litellm knows through litellm, whose model map leaves out the fields a model refuses (temperature on gpt-5.5, parallel_tool_calls on o3) and whose responses/ route carries a chat with tools to /v1/responses.
davidmckayv
requested review from
MikeRyanDev,
dustywusty,
guidovizoso,
mxmzb,
samjulien and
tylerslaton
as code owners
November 17, 2026 21:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The model compatibility probe in #1775 found the AG2 and Langroid Bots refusing OpenAI's newest models, in OpenAI's own words:
gpt-5.6-*,gpt-6*,gpt-6.1-*,gpt-6.2-*: "Function tools with reasoning_effort are not supported for this model in /v1/chat/completions. To use function tools, use /v1/responses".gpt-5-pro,gpt-5.2-pro,gpt-5.4-pro,gpt-5.5-pro,o3-pro: "This is not a chat model and thus not supported in the v1/chat/completions endpoint."gpt-5.5: "Unsupported value: 'temperature' does not support 0.2 with this model."o3: "Unsupported parameter: 'parallel_tool_calls' is not supported with this model."Both Bots sent every OpenAI model to Chat Completions, and Langroid sent
temperatureandparallel_tool_callsto models that refuse them.What
One rule for which API a model takes.
shared/model_providers.pygainsrequires_responses_apiandopenai_api, the Python copy ofrequiresResponsesApiinshared/model-providers.ts. The model decides, not the address, so a gateway in front of OpenAI gets the Responses API for these models too.COPILOTKIT_OPENAI_API=responsesorchatstill wins, as documented.AG2. AG2 1.3.2 ships
OpenAIResponsesConfig, so the Bot uses it for those models andOpenAICompletionsConfigfor the rest. Both are named explicitly: AG2 deprecates the bareOpenAIConfigand turns it into the Responses config in 1.4.0, which would move every OpenAI-compatible endpoint (most serve only Chat Completions) onto an API they lack.ag2>=1.3.2. No adapter code.Langroid. Langroid has no Responses API support up to its latest release (0.69.24, today), and its own model table stops at
gpt-5.2, so it sendstemperaturetogpt-5.5. Langroid already routes models through litellm, so an OpenAI model litellm knows now goes that way:litellm/openai/<model>, orlitellm/openai/responses/<model>for the Responses-only ones. litellm's model map decides which fields each model takes (Langroid turns on itsdrop_params), and itsresponses/route sends the chat to/v1/responsesand hands back a chat completion, so Langroid's tool calls and the AG-UI bridge work unchanged. A model litellm does not know (a local model behind an OpenAI-compatible endpoint, possibly with no key) stays on Langroid's own client, as before.litellm>=1.105.7,<1.106.0. No adapter code.AG-UI behaviour is the same: tool calls stream as before, the browser's tool result continues the run, and the text answer streams. The real round trip below checks each step on every model.
Sources
reasoning_effortother thannone, and the newer models require the Responses API for tool calling. https://developers.openai.com/api/docs/guides/migrate-to-responsesOpenAIResponsesConfiginag2.config): https://docs.ag2.ai/latest/docs/beta/model_configuration/ . AG2 1.3.2 (2026-11-15) on PyPI: https://pypi.org/project/ag2/1.3.2/ . ItsOpenAIConfigdeprecation names the 1.4.0 alias switch./v1/responses,o3,o3-pro,gpt-5-pro,gpt-5.5,gpt-5.5-pro,gpt-5.6-lunaandgpt-6-solall refusetemperature=0.2("Unsupported parameter: 'temperature' is not supported with this model") and all acceptparallel_tool_callstrue and false. On Chat Completions,o3refusesparallel_tool_callsandgpt-5.5refusestemperature=0.2.Tests
Each harness gains
tests/test_responses_api.py. The real framework, provider SDK and AG-UI stack run; only the provider atOPENAI_BASE_URLis local, and it refuses what OpenAI refuses, in OpenAI's words. Each test streams a tool call over AG-UI, sends the tool's result back as the browser does, and checks the answer uses it. They check the request shape: Responses-only models reach/v1/responseswith the tool in Responses form and the result as afunction_call_output;gpt-5.5,o3andgpt-4.1-ministay on Chat Completions with notemperatureon the reasoning models and noparallel_tool_callsono3.AG2's four tests that pass on main are the Chat Completions ones: AG2 never sent
temperatureorparallel_tool_callsthere, and the tests now hold it to that.Functional check
Run on a throwaway Linux box with the dev OpenAI key, against every OpenAI model the picker lists (29). "Before" is the published
latestimage, "after" is the image built from this branch.#1775's probe (
scripts/model-compat.ts --harness agent-ag2,agent-langroid --provider openai): one AG-UI run per model, carrying one tool.A real tool round trip, since the probe's prompt does not call its tool: run 1 asks for
look_up_orderand must stream the call with its arguments; run 2 sends the thread back with the tool's result and must stream an answer that uses it. Same 29 models.Every failure before was run 1, with the same refusals as the probe. After, every model called the tool with
{"order": "A-1001"}and answered from its result, with the same AG-UI events per run as before.