Skip to content

fix(llm): add provider-specific verified model lists for gemini, deepseek, moonshot, minimax#2386

Merged
juanmichelini merged 1 commit intomainfrom
fix/verified-models-provider-lists
Mar 11, 2026
Merged

fix(llm): add provider-specific verified model lists for gemini, deepseek, moonshot, minimax#2386
juanmichelini merged 1 commit intomainfrom
fix/verified-models-provider-lists

Conversation

@juanmichelini
Copy link
Copy Markdown
Collaborator

@juanmichelini juanmichelini commented Mar 11, 2026

Summary

Fixes #2385

Models in VERIFIED_OPENHANDS_MODELS were appearing in the OpenHands verified models UI but not under their actual provider, because no provider-specific list existed for gemini, deepseek, moonshot (Kimi), and minimax.

Diagnosis

The following models were in VERIFIED_OPENHANDS_MODELS but had no corresponding provider list:

  • gemini-3-pro-preview → no gemini list
  • deepseek-chat → no deepseek list
  • kimi-k2-thinking, kimi-k2.5 → no moonshot list
  • minimax-m2.5 → no minimax list

(The original reporter's example claude-sonnet-4-6 was already fixed in #2104.)

Changes

openhands-sdk/openhands/sdk/llm/utils/verified_models.py

  • Added VERIFIED_GEMINI_MODELS = ["gemini-3-pro-preview"]
  • Added VERIFIED_DEEPSEEK_MODELS = ["deepseek-chat"]
  • Added VERIFIED_MOONSHOT_MODELS = ["kimi-k2-thinking", "kimi-k2.5"]
  • Added VERIFIED_MINIMAX_MODELS = ["minimax-m2.5"]
  • Registered all new lists in VERIFIED_MODELS dict

tests/sdk/llm/test_model_list.py

  • Added test_openhands_models_all_have_provider_list — ensures every model in VERIFIED_OPENHANDS_MODELS also appears in at least one provider-specific list, preventing this class of regression.

.github/run-eval/ADDINGMODEL.md

  • Added a clear warning in the "Files to Modify" section explaining that when a model is added to VERIFIED_OPENHANDS_MODELS, it must also be added to its provider-specific list (creating a new one if needed).

Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:6294ebb-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-6294ebb-python \
  ghcr.io/openhands/agent-server:6294ebb-python

All tags pushed for this build

ghcr.io/openhands/agent-server:6294ebb-golang-amd64
ghcr.io/openhands/agent-server:6294ebb-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:6294ebb-golang-arm64
ghcr.io/openhands/agent-server:6294ebb-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:6294ebb-java-amd64
ghcr.io/openhands/agent-server:6294ebb-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:6294ebb-java-arm64
ghcr.io/openhands/agent-server:6294ebb-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:6294ebb-python-amd64
ghcr.io/openhands/agent-server:6294ebb-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:6294ebb-python-arm64
ghcr.io/openhands/agent-server:6294ebb-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:6294ebb-golang
ghcr.io/openhands/agent-server:6294ebb-java
ghcr.io/openhands/agent-server:6294ebb-python

About Multi-Architecture Support

  • Each variant tag (e.g., 6294ebb-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 6294ebb-python-amd64) are also available if needed

…seek, moonshot, minimax

Models in VERIFIED_OPENHANDS_MODELS were missing from their provider-specific
lists (gemini, deepseek, moonshot, minimax), causing them to appear in the
OpenHands verified models UI but not under their actual provider.

Changes:
- Add VERIFIED_GEMINI_MODELS with gemini-3-pro-preview
- Add VERIFIED_DEEPSEEK_MODELS with deepseek-chat
- Add VERIFIED_MOONSHOT_MODELS with kimi-k2-thinking, kimi-k2.5
- Add VERIFIED_MINIMAX_MODELS with minimax-m2.5
- Register new lists in VERIFIED_MODELS dict
- Add test that every model in VERIFIED_OPENHANDS_MODELS also appears in a
  provider-specific list
- Update ADDINGMODEL.md to document the requirement

Fixes #2385

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Copy Markdown
Contributor

API breakage checks (Griffe)

Result: Passed

Action log

@github-actions
Copy link
Copy Markdown
Contributor

Agent server REST API breakage checks (OpenAPI)

Result: Failed

Log excerpt (first 1000 characters)
{"asctime": "2026-03-11 13:45:58,374", "levelname": "WARNING", "name": "openhands.agent_server.config", "filename": "config.py", "lineno": 173, "message": "\u26a0\ufe0f OH_SECRET_KEY was not defined. Secrets will not be persisted between restarts."}
::error title=openhands-agent-server REST API::Breaking REST API change detected without MINOR version bump (1.13.0 -> 1.13.0).

Breaking REST API changes detected compared to baseline release:
- the 'file' request property type/format changed from 'string'/'' to 'string'/'binary'
/home/runner/work/software-agent-sdk/software-agent-sdk/.venv/lib/python3.13/site-packages/litellm/llms/custom_httpx/async_client_cleanup.py:66: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()

Action log

@github-actions
Copy link
Copy Markdown
Contributor

Coverage

Coverage Report •
FileStmtsMissCoverMissing
TOTAL19917579970% 
report-only-changed-files is enabled. No files were changed during this commit :)

@juanmichelini juanmichelini marked this pull request as ready for review March 11, 2026 15:52
Copy link
Copy Markdown
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good taste - Clean fix following existing patterns exactly. Regression test prevents this from happening again. lgtm! 👍

@juanmichelini juanmichelini merged commit aa9df69 into main Mar 11, 2026
31 checks passed
@juanmichelini juanmichelini deleted the fix/verified-models-provider-lists branch March 11, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LLM models appears in OpenHand's Verified Models but not in its Provider's Verified models

3 participants