Repository navigation
Pin the LangGraph harness to the set it was proven with - #1826
Merged
Merged
Conversation
The harness's requirements were unpinned, so every fresh resolve took that morning's LangChain releases. langchain-anthropic 1.7.11 (2026-11-18) pins `anthropic-version: 2023-06-01` on its client, and the provider-boundary test expected the anthropic SDK's own default (2026-10-21), so "python harness regressions" went red on a pull request that changed nothing here. The pins are the set shared/model-lock.json verified against real models and CI last passed on: langchain-anthropic 1.7.10, anthropic 2.10.0, langchain-core 1.8.2, langchain-openai 1.8.1, langgraph 1.2.22, langchain 1.5.0, langchain-google-genai 4.4.3, fastapi 0.164.0, python-multipart 0.0.34, uvicorn 0.54.0. The Dockerfile installs requirements.txt, so the image gets the same set. requirements-test.txt no longer pins langchain-openai 1.6.0 over it: CI tested 1.6.0 while the image shipped 1.8.1, and the suite passes on 1.8.1. The test asserted the SDK's default version rather than what Anthropic needs. Both 2023-06-01 and 2026-10-21 are served, and the current models answer on either. What 2026-10-21 adds is a 400 for temperature, top_p and top_k, so the test now asserts a served version and a body without those.
davidmckayv
requested review from
MikeRyanDev,
dustywusty,
guidovizoso,
mxmzb,
samjulien and
tylerslaton
as code owners
November 18, 2026 15:47
scripts/model-lock.test.ts allows an exact pin in a harness's requirements only for a client the model check records, so the lock can say which client versions each model was proven on. langchain, fastapi, python-multipart and uvicorn go back to unpinned, as in every other harness. The model clients stay pinned to the versions in shared/model-lock.json.
davidmckayv
added a commit
that referenced
this pull request
Nov 18, 2026
…date OpenBot when it's too old (#1811) ## What this does Every OpenBot app now says who it is on every request, every OpenBot server says which API versions and which app releases it serves, and when an app is older than its OpenBot serves, the person reads one sentence instead of a confusing failure: **Update OpenBot**. When the minimum was raised for a security problem, the sentence is **This version of OpenBot has a security problem. Update to keep using it.** This is the first piece of OpenBot's API versioning, for the two places where OpenBot runs both sides or the server: OpenBot Desktop with its own OpenBot, and OpenBot Cloud, where people's apps lag the server. Desktop connected to a company's own OpenBot gets nothing new: the company rolls out both, and the existing "ask whoever runs it to update" sentence stays as it was. ## The contract (spec first, v2, additive) - `OpenBot-Client: <kind>/<version>` on every request, kind `web`, `desktop`, `mobile` or `recorder`. A header may name several, separated by spaces: OpenBot Desktop adds its own after its window page's (`web/1.1.0 desktop/1.1.0`). A program of your own sends nothing and is never refused for a version. - `getDeployment` gains `api` (the API versions served, `["v2"]`), `minimumClients` (the oldest release of each app served), `securityMinimumClients` (the part of those raised for a security problem, only when there is one), `launcherApi` and `extensionApi` (the numbers `release-policy.json` keeps). It is never refused for the app's version, so an app reads it first. - Every other `/api/v2` operation refuses an app below its minimum with 400 `application/problem+json`, code `CLIENT_UPDATE_REQUIRED`, with `client`, `minimumVersion`, `security: true` when the minimum is there for security, and a detail sentence. Not 426, which RFC 9110 keeps for protocol upgrades. - `shared/client-version.ts` holds the rules; `shared/fixtures/client-version/cases.json` holds deployment answers and what every app must decide from them. The web app, the Recorder, Desktop (a Rust port) and the phone app (a self-contained copy) each run every case. **The support window is 12 months.** A server serves every app released in the last 12 months, so a release builds in a minimum for an app only at a release published more than 12 months before it. Every OpenBot release is younger than that, so the built-in minimums are empty today, and apps from before this change send no header: nothing released is refused by it. A server's minimums are raised in one place, `OPENBOT_MINIMUM_CLIENTS` on the server (how OpenBot Cloud raises its own), and a minimum is never lowered. ## Each app - **Web app**: the header on the one fetch in `lib/client.ts` (so the generated client too), the CopilotKit provider, sign-in and the consent verifier. A gate at the root, before sign-in as well as after, reads `deployment` and any `CLIENT_UPDATE_REQUIRED` refusal, and replaces the app with the sign-in screen's own frame: "Update OpenBot" with Update (loads the new page), or inside the Desktop window, when the refusal names Desktop, its own Update and restart. A tab left open across an upgrade gets the sentence on its next request. - **OpenBot Desktop**: the header on every request the shell makes, and the relay adds it to the page's. With its own OpenBot, the shell reads that OpenBot's deployment before opening the window and shows "Update OpenBot" with Update and restart when the app is too old; a local OpenBot from an older release that still serves v2 opens exactly as before. Connected to OpenBot Cloud, the same check runs when the person connects and on every launch: "The OpenBot at {host} needs a newer version of this app. Update OpenBot, then connect again.", with Update and restart. - **Phone app**: the header (the OpenBot release it was built from, not Expo's own version) on every request and on sign-in, and a root gate with the update sentence. - **Recorder**: the header on every request, and a refusal now reaches the person as the update sentence on the popup, the Connect page and Connect with OpenBot Desktop, where before it read "OpenBot did not answer" or "This connect request has expired". ## Tests Each new test was seen failing before its change and passing after. - `shared/tests/client-version.test.ts`: release order, the header, every fixture case, the empty built-in minimum. - `app/tests/client-name-header.test.ts`, `app/tests/client-version-gate.test.tsx`: the header reaches a real local server; the app's real root, sign-in gate and screens against a local server answering the fixture's deployments and the spec's own refusals, the security one included. - Desktop: `client_version.rs` runs every fixture case; `connected/address.rs`, `relay.rs`, `sign_in.rs` and `main.rs` through the real HTTP path with the fake OpenBot; the window's React tests and layout test. - `mobile/tests/*`, `extension/tests/openbot-client.test.ts`, `connection.test.ts`, `recording.test.ts`. - Every existing suite still passes: app, extension, Desktop crate and window, root copy and boundary tests. ## End to end On the all-in-one cluster (two server replicas, the backend in the cluster, Keycloak sign-in), from images built from this branch: - **Nothing built in, nothing refused.** `deployment` answers `api: ["v2"]`, `minimumClients: {}`, `launcherApi: 1`, `extensionApi: 1`; `/api/capabilities` answers exactly its previous fields. Every app version (`desktop/0.0.1` to `desktop/1.1.0`, `web/1.0.0`, `mobile/0.1.0`, `recorder/1.0.0`), no header, `curl/8.7.1` and `desktop/main-ci` go on to sign-in as before. 17 of 17 checks. - **Matching versions unaffected.** The sign-in screen and home, light and dark, at 1440px and 390px, as before; every request the page makes to `/api` carries `OpenBot-Client: web/1.1.0` (27 of 27; the identity provider's redirect back is a navigation and carries no page header). - **Minimums raised, as Cloud raises its own** (`OPENBOT_MINIMUM_CLIENTS=web=1.1.0,desktop=1.2.0,mobile=1.2.0,recorder=1.2.0`): `deployment` publishes them; `desktop/1.1.0`, `mobile/1.1.0`, `recorder/1.1.0`, `web/1.0.0` and `web/1.1.0 desktop/1.1.0` are refused with 400 `application/problem+json`, `CLIENT_UPDATE_REQUIRED`, the app, its minimum and the sentence ("This OpenBot needs OpenBot Desktop 1.2.0 or later. Update OpenBot, then try again."); `desktop/1.2.0`, no header and `curl` go on. 24 of 24. - **A tab left open across an upgrade.** Four tabs (light and dark, desktop and phone width) signed in on release 1.1.0; the server rolled to a 1.2.0 build whose admin raised every app to 1.2.0. Moving around the app without reloading, each tab's next request was refused and the page showed "Update OpenBot" / "This page is from an older version of OpenBot." with Update, no sideways scroll; Update loaded the 1.2.0 page, which works and names itself `web/1.2.0`. 28 of 28. ## OpenBot Desktop on a Mac and on Windows The pass of record ran on the tested head 0fe5a14. The installers came from that head, with the server built from the same change, and every check passed on both: - An update over v1.1.0 opens OpenBot, and so does Quit and reopen. - With the minimum raised, Start shows Update OpenBot. A relaunch that finds OpenBot running shows the same screen, never "stopped". With the minimum removed, OpenBot opens again. - Connected to a real OpenBot for Business server: the sentence with no Connect when the minimum is raised, and the API-version refusal. Sign-in works when the minimum isn't raised. A relaunch after the minimum is raised says so, and opens again once it's removed. On the Mac, the last of these showed OpenBot's relay page instead of OpenBot. That's a separate, earlier fault in connected Desktop on macOS, with its own fix; the version check's part of the step passed. The merge head differs from the tested head only by #1826 (pinning the LangGraph harness's Python dependencies), which landed on main in between: ``` agent-langgraph-agui/requirements-test.txt | 1 - agent-langgraph-agui/requirements.txt | 16 ++++++++++++---- agent-langgraph-agui/tests/test_provider_boundaries.py | 10 +++++++--- 3 files changed, 19 insertions(+), 8 deletions(-) ```
davidmckayv
added a commit
that referenced
this pull request
Nov 18, 2026
…stead (#1813) ## What this does OpenBot Desktop now reads a release's launcher contract (`compatibility.launcherApi` in its `release-manifest.json`) before it lays out or starts that release, so an app never starts an OpenBot it can't launch. - **First install.** When the latest release needs a newer launcher contract than this app speaks, the app lays out its own release instead and the existing update bar offers the newer app. Same or lower: the latest, as before. A manifest that can't be read: the app's own release. - **Start.** When the installation's release needs a newer launcher contract than this app speaks, the app doesn't start it and doesn't move the installation back. The window shows "Update OpenBot" with "The OpenBot on this computer needs a newer version of this app. Update OpenBot to keep going." and Update and restart (the same screen the version check shows). Same or lower: starts as before. Offline, when the manifest can't be read: starts as before, and a too-new OpenBot then says so itself. - `launcherApi` is not raised here: raising it is a release decision, since the upgrade planner compares it exactly. Builds on OpenBot PR 1811 (the version check), now merged. ## Tests Each seen failing first. - `deployment_release.rs`: three new tests (a latest release beyond this app's contract lays out the app's own; one at or below lays out the latest; an unreadable manifest lays out the app's own), 19 of 19 pass, with the real v1.1.0 manifest as a fixture (`test-fixtures/release-manifest-v1.1.0.json`). - `App.test.tsx`: the stopped screen shows only the update card, no failed row, Retry or Repair; the browser layout test's update states, light and dark. - cargo fmt, clippy (`--all-targets -D warnings`), the crate's lib and main tests, the window's tests and typecheck, format and lint: clean. ## Mac and Windows pass The pass of record ran on the tested head 6209b9a, on both a Mac and Windows. The installers came from that head, with the server built from the same change. - A fresh install starts normally, with no "Update OpenBot", and so does Quit and reopen. - With the minimum raised, Start shows Update OpenBot. With it removed, OpenBot opens again. The Mac also shows the same screen in dark mode. - The released 1.0.3 app, against a server of this release: - The page reads "This OpenBot needs a newer version of the OpenBot app", and the server stays up with the missing input logged. - Every other request is refused, while /health and /api/capabilities answer. - 1.0.3's host-access poll stays quiet, and the page is dark when the system is. Not runnable live, because they need a published release whose manifest says launcherApi 2 (none exists, and none was published for a test): - A first install against such a release, and an update offered to one. Covered by `a_first_install_lays_out_the_apps_own_release_when_the_latest_needs_a_newer_app`, `a_release_that_needs_a_newer_app_is_not_started_and_says_update_openbot`, and the window test above. - Its offline leg, covered by that test's start-launcher-unknown case. The merge head differs from the tested head only by #1826 (pinning the LangGraph harness's Python dependencies), which landed on main in between: ``` agent-langgraph-agui/requirements-test.txt | 1 - agent-langgraph-agui/requirements.txt | 16 ++++++++++++---- agent-langgraph-agui/tests/test_provider_boundaries.py | 10 +++++++--- 3 files changed, 19 insertions(+), 8 deletions(-) ```
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
"python harness regressions" (and so "verify") went red on a pull request that touched nothing in the LangGraph harness.
agent-langgraph-agui/requirements.txtwas unpinned, and today's fresh resolve took langchain-anthropic 1.7.11, published 2026-11-18. Everything else in the resolve was identical to main's last green run.1.7.11 pins
anthropic-version: 2023-06-01on its Anthropic client (langchain-ai/langchain#41770, "to avoid silently changing older-model sampling and message behavior"). The anthropic SDK 2.10.0 default is2026-10-21.test_anthropic_selection_reaches_anthropic_boundary_without_openai_keyasserted the request carried the SDK's default, so all 5 of its cases failed:assert '2023-06-01' == '2026-10-21'.What changed
Pinned the model clients to the set they were proven with. These are the versions
shared/model-lock.jsonverified against real models, and the ones main last passed CI on:ag-ui-langgraphwas already pinned.google-genaiandhttpxkeep the ranges they already had, with their existing reasons. The repo has no lock or constraints file for the Python harnesses. They pin inrequirements.txtwith a comment, andscripts/model-lock.test.tsallows an exact pin only for a model client the lock records. Solangchain,fastapi,python-multipartanduvicornstay unpinned, as in every other harness.anthropicandlangchain-coreare not imported by the harness, but they decide the request langchain-anthropic builds, so they are pinned with the rest.The Dockerfile installs
requirements.txt, so the image and CI get the same set.requirements-test.txtused to pinlangchain-openai==1.6.0on top of it, so CI tested 1.6.0 while the image shipped 1.8.1 (the version in the model lock). That override is gone. The suite passes on 1.8.1.Fixed what the test asserts. It checked that the request used the SDK's default version. That is not what Anthropic needs:
2023-06-01and2026-10-21(API versions). The only Messages API changes in2026-10-21are 400 errors fortemperature,top_pandtop_k, for adocumentblock in anassistantmessage, and forcitationsonusertext. The migration guide says a request that needs one of those can stay on2023-06-01. Neither page ties any model or feature to2026-10-21.answerpath, on both versions, got HTTP 200 and an answer fromclaude-opus-5-5,claude-sonnet-5-5,claude-haiku-5-5andclaude-fable-5-2.output_config.effort, which both versions accept. It never sends sampling parameters.The test now asserts that the request carries a version Anthropic serves, and that the body has no
temperature,top_portop_k, which2026-10-21would refuse. Under that contract the header langchain-anthropic picks doesn't matter. The pin keeps the set fixed until someone moves it on purpose.Evidence
Run locally in fresh venvs inside the worktree (Python 3.12, same as CI).
pytest agent-langgraph-agui/tests -q:The unpinned venv's
pip freezematches the install line of the red CI run exactly. The only difference from main's last green run islangchain-anthropic1.7.10 to 1.7.11.Live check, one request per model through
src.main.answer:The Model lock workflow runs on this PR, because
requirements.txtchanged. It rebuilds the harness image from this branch and sends every model in the lock one request.