Skip to content

fix: text-only Qwen3.5/Qwen3.6 MoE checkpoints falsely auto-detected as VLM - #157

Merged
solderzzc merged 1 commit into
mainfrom
fix/qwen35-moe-vlm-false-positive
Aug 25, 2026
Merged

fix: text-only Qwen3.5/Qwen3.6 MoE checkpoints falsely auto-detected as VLM#157
solderzzc merged 1 commit into
mainfrom
fix/qwen35-moe-vlm-false-positive

Conversation

@solderzzc

Copy link
Copy Markdown
Member

Summary

Fixes #156, reported and fully root-caused by @traderjoe1968.

qwen3_5/qwen3_5_moe are registered as that exact model_type string in both LLMModelFactory (text_config only, no vision field at all) and VLMModelFactory (vision_config required) — the identical ambiguity already fixed for gemma4 in #152. Matching on model_type alone routed every text-only checkpoint sharing that string to the VLM factory, whose config decoder requires a field the checkpoint never has:

[SwiftLM] Auto-detected VLM config (qwen3_5_moe); enabling vision mode.
[SwiftLM] Loading VLM (vision-language model)...
Error: Failed to parse config.json ... Missing field 'vision_config'

Repro from the issue: destynova002/Qwen3.6-35B-A3B-mixed-3-4bit-3.85bpw-mlx, a genuinely text-only checkpoint with no vision_config.

Fix

Removes qwen3_5/qwen3.5/qwen3_5_moe from knownVisionModelTypes. The existing vision_config-presence check already distinguishes the two correctly, since only the VLM struct requires that field.

Verified unaffected

mlx-community/Qwen3.5-2B-4bit — used as a text-only speculative-decoding model in this repo's own CI — genuinely carries a vision_config in its checkpoint even in that role, confirmed by inspecting its cached config.json directly. It's still correctly detected as VLM-capable via the presence check, independent of this model-type-list removal, so this fix doesn't reopen or interact with the earlier --draft-model/--dflash/--mtp auto-detect guard from #152.

Test plan

  • swift build --target SwiftLM succeeds.
  • CI fixture suite green (local run hit an unrelated missing-metallib environment gap in this worktree, not exercised here).

…as VLM

Fixes #156. qwen3_5 and qwen3_5_moe are registered as that exact
model_type string in both LLMModelFactory (text_config only, no vision
field at all) and VLMModelFactory (vision_config required) — the same
ambiguity already fixed for gemma4 in #152. Matching on model_type
alone routed every text-only checkpoint sharing that string (e.g. the
reported destynova002/Qwen3.6-35B-A3B-mixed-3-4bit-3.85bpw-mlx, which
has no vision_config) to a factory whose config decoder requires a
field the checkpoint never has — a hard startup failure, not a
degraded one.

Removes qwen3_5/qwen3.5/qwen3_5_moe from knownVisionModelTypes; the
existing vision_config-presence check already distinguishes the two
correctly, and continues to catch genuine Qwen3.5-VL checkpoints
(e.g. mlx-community/Qwen3.5-2B-4bit, which does carry a real
vision_config even when used as a text-only speculative-decoding
model in this repo's own CI — unaffected by this change).

Reported with full root-cause diagnosis by @traderjoe1968.
@solderzzc
solderzzc merged commit 75f3b56 into main Aug 25, 2026
14 checks passed
@solderzzc
solderzzc deleted the fix/qwen35-moe-vlm-false-positive branch August 25, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: text-only Qwen3.5/Qwen3.6 MoE models are incorrectly auto-detected as VLM

1 participant