feat(models): fusion models — give a text-only model vision

[ericleepi314]

The problem

Some strong reasoning models cannot see images at all. deepseek-v4-pro rejects any non-text content block outright — probed against api.deepseek.com:

400  Failed to deserialize the JSON body into the target type:
     messages[0]: unknown variant `image_url`, expected `text`

So pasting a screenshot, @-mentioning an image, or letting Read return one ended the turn. Reproduced through the real CLI before this change:

$ clawcodex -p --model deepseek-v4-pro "Read probe.png and tell me the colour of the LEFT half."
Query error: Error code: 400 - ... unknown variant `image_url`, expected `text`

What this adds

A fusion model pairs that base model with a second, vision-capable one. Every image is sent to the vision model first, and its description is what the base model reads.

/fusion create deepseek-v4-pro-V deepseek:deepseek-v4-pro openrouter:google/gemini-2.5-flash
/model deepseek-v4-pro-V

Same prompt, same image, after:

$ clawcodex -p --model deepseek-v4-pro-V "Read probe.png and tell me the colour of the LEFT half."
The left half is red, and the shape in the middle is a white horizontal bar.

Ported from claude-code-router's Fusion Models, keeping its new model = base + capability shape and its create / configure / save / manage lifecycle.

• /fusion list | create | delete | enable | disable — registered as a real builtin, so it works in the TUI, -p headless, and the SDK.
• A saved fusion model behaves like a normal model: /model <name>, --model <name>, it appears first in the /model picker, and it survives a restart. (CCR: "a Fusion model appears in routing and Agent Profiles like a normal model.")
• Stored in ~/.clawcodex/config.json → fusionModels.

Mechanism — and one deliberate divergence from CCR

CCR is a proxy, so its only lever is exposing vision as an MCP tool the base model may choose to call. That cannot fix a pasted image: it is already on the wire, and DeepSeek 400s before the model gets a turn.

clawcodex owns the agent loop, so it does what CCR's docs actually describe — "connects a vision model in front of the base model … passes that visual result to the base model" — directly, by substituting image blocks in place. One consequence worth the trade: every entry point works at once (paste, @file.png, Read on an image, Bash image output), with no reliance on the model electing to call a tool.

FusionProvider is a delegating wrapper (the established _EffortProvider pattern) with three invariants:

1. No image block survives — even on vision failure, timeout, or the per-request cap. A leftover one is the 400 this feature prevents, so failures degrade to a text note naming the cause.
2. The caller's message list is never mutated — the session owns the conversation, and the original images must survive a switch back to a vision-capable model. Copy-on-write, so a turn with no images returns the original list object and costs nothing.
3. Each distinct image is described once — history is replayed every turn, so a process-wide bounded LRU keyed by content hash plus the vision configuration keeps an N-turn conversation from paying N times for one screenshot. Failures are cached too, with a 90s TTL so a transient outage neither repeats nor poisons.

Other design points:

• .model reports the BASE model id, not the fusion name — context window, cost, capability probes, and the wire's own model field all key off it. The fusion name rides fusion_name plus a fusion field on the init envelope and set_model reply, which is what the TUI displays.
• Anthropic-wire base providers are rejected at validation: six live isinstance(provider, AnthropicProvider) sites (prompt caching, deferred tools, the goal judge, the memory selector) are blind to a wrapper, and every Anthropic-wire model already accepts images.
• Global config tier only. A fusion model names the provider that receives image bytes — exactly what _UNTRUSTED_TIER_BLOCKED_KEYS withholds from committable tiers — so a checked-out repo can never contribute one.
• _install_provider is extracted from _do_set_provider so selecting a fusion model (which can be a cross-provider switch) performs an identical rebuild instead of a second, drifting copy.

Model-table corrections (probed 2026-07-30)

supports_vision was dead outside tests. It is now load-bearing — the validator rejects a known text-only vision half — and correspondingly corrected:

Model	Endpoint result
deepseek-v4-pro, deepseek-v4-flash	400 unknown variant image_url → supports_vision=False
glm-5.2, glm-5.1	400 code 1210 content.type is invalid, allowed values: ['text'] on both Z.ai endpoints → supports_vision=False
glm-4.5v, glm-4.6v, glm-5v-turbo	accept images (429 insufficient-balance on a Coding-Plan-only key = entitlement, not shape) → registered explicitly

glm-5.2 is not multimodal, despite the family having vision siblings. Z.ai's vision models are the separate *v family — which is what CCR's own GLM-5.2 + GLM-5V-Turbo = GLM-5.2V example refers to. That is the feature's most inviting mistake, so the validator rejects it and names the *v family in the error.

supports_vision also stops trusting a prefix-inherited False: get_model_config falls back to key.rsplit("-", 1)[0], and the glm-5.2 row's base is bare glm — so every glm* id, including the vision models the error message recommends, would have inherited False. A False reached by prefix match is not evidence.

Verification

• Live, through the real CLI with real keys, via the Read tool (the nested tool_result path): base model 400s, fusion model describes the image correctly. Also verified starting fused with an unconfigured default provider, which the credential-ordering fix enables.
• All /fusion and set_model controls driven over the real agent-server --stdio transport — create/duplicate/validate, picker ordering, switch on, switch away, disable → refuse → re-enable.
• Python suite: 9225 passed, 3 skipped, 0 failed.
• vitest: baseline + 11. Its 8 failures are pre-existing, established independently by reverting this branch's TS change (createGatewayEventHandler ×2, cursorDriftRegression, statusRule ×2, useConfigSync ×2, virtualHeights — none touches fusion).
• Mutation-tested: every fix in the review pass was verified to be gated by a named test — reverting the fix fails that test and no other.

Second commit: model restore, unified (cdfa11ef)

Applies the TS reference's approach (main.tsx:1984 — userSpecifiedModel ?? getUserSpecifiedModelSetting() ?? null). clawcodex had two partial read sides and one dead one: _build_runtime applied the persisted model after constructing the provider, reading settings.model raw; headless ignored it entirely (so a /model switch had to be re-stated with --model every -p run); and settings.apply_persisted_model was a third implementation with no callers.

Now one helper — settings.get_persisted_model — used by both entrypoints before construction, which is what lets a persisted fusion model decide which provider to build. A fusion model persists its name (it was persisting the base id, so a restart silently dropped vision) and is exempt from the provider-match guard, since its record names its own provider — but not from an explicit --provider, which still wins.

Three defects found while building it, each with a regression test:

Found by	Defect
review	A disabled fusion name still reached the wire — get_persisted_model declined it, the raw-seed backstop assigned it anyway. Backstop deleted, not guarded; a mutation-verified test asserts it stays gone.
self-probe	A persisted fusion model overrode an explicit --provider.
full suite	get_persisted_model broke its own "never raises" contract — only the load was wrapped, not the attribute reads, so a partial settings stub raised AttributeError inside headless startup.

Deleting a fusion model clears a persisted selection naming it; disabling needs no clearing (the record survives and the read side checks enabled).

Verified live: /model deepseek-v4-pro-V → kill → fresh session with no flags restores it and reads an image; disable → provider default, no dangling name; re-enable → restored.

Known limitations (stated, not overlooked)

• The vision leg's tokens are not in /cost. The cost tracker follows a single model — correctly the base, since that is what pricing and context-window lookups key off. The usage is logged so the spend is discoverable; threading a second model into single-model accounting is a cost-subsystem change.
• The name-shadowing guard is create-time only. A hand-edited config can still register a fusion model named after another vendor's id. Self-inflicted rather than an attack surface — global tier, so no repo can contribute one — and documented on load_fusion_models.
• Vision only. CCR also fuses web search, image/video generation, and arbitrary MCP tools; those are separate capabilities with their own config surfaces.

🤖 Generated with Claude Code

[github-actions]

Test Results

    1 files      1 suites   8m 1s ⏱️
9 233 tests 9 227 ✅ 6 💤 0 ❌
9 479 runs  9 473 ✅ 6 💤 0 ❌

Results for commit aaa9f8b.

♻️ This comment has been updated with latest results.