Skip to content

Pin the LangGraph harness on release/1.1 to the set it was proven with - #1842

Open
jerelvelarde wants to merge 2 commits into
CopilotKit:release/1.1from
jerelvelarde:jerel/harness-pin-1.1
Open

jerelvelarde wants to merge 2 commits into
CopilotKit:release/1.1from
jerelvelarde:jerel/harness-pin-1.1

Conversation

@jerelvelarde

Copy link
Copy Markdown
Contributor

The problem

Every pull request into release/1.1 fails "python harness regressions" today, whatever it touches, and the aggregate "verify" check fails with it. The LangGraph harness on this branch installs its model clients unpinned, so each CI run resolves whatever LangChain released that morning. langchain-anthropic 1.7.11 came out on 2026-11-18 and sends anthropic-version: 2023-06-01, where the Anthropic SDK defaults to 2026-10-21. The provider boundary test asserts the SDK default, so five of its cases now fail on a branch nobody changed.

main fixed this in #1826. That fix never reached release/1.1, and 1.1 is the line that ships, so a PR that has to land on OpenBot 1.1 (for example #1837) cannot go green until the release branch has it too.

The approach

This backports #1826 as a cherry-pick. It applied cleanly, and the commit keeps its original author and message. It does three things.

Pins the model clients in agent-langgraph-agui/requirements.txt to the set main last passed on: langgraph 1.2.22, langchain-core 1.8.2, langchain-openai 1.8.1, langchain-anthropic 1.7.10, anthropic 2.10.0 and langchain-google-genai 4.4.3. The harness Dockerfile installs the same file, so CI and the image stay on the same versions.

Drops the langchain-openai==1.6.0 override from requirements-test.txt, so the tests run against the client the image ships, not an older one.

Loosens the provider boundary assertion from "the SDK's default version" to "a version Anthropic serves" (2023-06-01 or 2026-10-21), and adds the check that matters for 2026-10-21: the request body carries no temperature, top_p or top_k. The pin alone would fix today's failure. The looser assertion stops the next langchain-anthropic release that picks a different, still-served version from breaking the branch again.

A second commit changes one comment. The pin's comment on main cites shared/model-lock.json and the "Model lock" section of docs/releasing.md. Both came to main with the model check (#1779), which release/1.1 does not have, so on this branch the comment pointed at files that do not exist. It now says where the set was proven and that it moves as one set. Nothing else differs from #1826.

The cost: the harness on 1.1 no longer picks up client fixes on its own. Moving these versions is now a deliberate change, the same as on main.

What is not covered

  • No recording. This change is to CI dependencies and one test assertion, and there is nothing on screen to show. The "python harness regressions" job is the evidence.
  • The model check (Offer a model only when the model check has proven it on the Bot's harness #1779) is not backported. On release/1.1 the pinned set has not been tested against live models. It is the set main proved. The tests here check what the harness sends, not live replies.
  • The other harnesses in the same job are unchanged. On Let a person pick Kite Bot: their Bots first, each as a kite #1837 the CrewAI and Claude SDK steps passed, but every step after the LangGraph one was skipped once it failed. Any of those harnesses that resolve unpinned clients could break the same way.
  • This PR carries no server change.

Verification

  • Locally, in a fresh Python 3.12 venv installed from requirements.txt and requirements-test.txt exactly as the CI step does: python -m pytest agent-langgraph-agui/tests -q gave 171 passed, with langchain-anthropic 1.7.10, anthropic 2.10.0, langchain-core 1.8.2 and langgraph 1.2.22 installed. The LangGraph step in CI runs this same command.
  • Baseline: on Let a person pick Kite Bot: their Bots first, each as a kite #1837, which runs on the current release/1.1, the same step reports 5 failed, 166 passed, all five in test_anthropic_selection_reaches_anthropic_boundary_without_openai_key on assert '2023-06-01' == '2026-10-21'.
  • CI's "python harness regressions" job on this PR is the full check. It also runs the nine harness steps and the compose checks that Let a person pick Kite Bot: their Bots first, each as a kite #1837's run skipped.

Merge notes

Once this merges, #1837 and any other open PR into release/1.1 needs to merge or rebase on the new branch tip to pick up the fix. If #1779 is later backported to 1.1, put the comment in requirements.txt back to main's wording so it cites the model lock again.

davidmckayv and others added 2 commits November 18, 2026 13:51
agent-langgraph-agui's requirements were unpinned, and today's resolve took langchain-anthropic 1.7.11, which sends anthropic-version 2023-06-01 instead of the SDK default 2026-10-21. "python harness regressions" went red on a PR that never touched the harness.

The model clients are now pinned to the set the model lock verified and main last passed on: langchain-anthropic 1.7.10, anthropic 2.10.0, langchain-core 1.8.2, langchain-openai 1.8.1, langgraph 1.2.22 and langchain-google-genai 4.4.3. The Dockerfile installs the same file, so CI and the image agree. requirements-test.txt no longer overrides langchain-openai to 1.6.0, so CI tests what the image ships.

The provider boundary test no longer requires the SDK's default version. Anthropic serves both 2023-06-01 and 2026-10-21, and 2026-10-21 only adds refusals for temperature, top_p and top_k (plus two content-block cases). The test now checks that the request carries a version Anthropic serves and that the body has no sampling parameters. A live request on each version answered from Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.2.

Tested: the unpinned resolve fails 5 tests on the old assertion; the pinned set passes all 188, and the new assertion passes on both resolves. The model lock workflow rebuilt the harness image and passed.
(cherry picked from commit 0f69809)
The comment carried over from CopilotKit#1826 cites shared/model-lock.json and the "Model lock" section of
docs/releasing.md. Both arrived on main with the model check, which release/1.1 does not have, so on
this branch the comment pointed at nothing. It now names the pinned set by where it was proven.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants