From e3bfb16f2140877691c088b7f6b286cf482f5c2a Mon Sep 17 00:00:00 2001
From: Simba Zhang <solderzzc@gmail.com>
Date: Tue, 25 Aug 2026 06:29:11 -0700
Subject: [PATCH] fix: text-only Qwen3.5/Qwen3.6 MoE checkpoints falsely
 auto-detected as VLM
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Fixes #156. qwen3_5 and qwen3_5_moe are registered as that exact
model_type string in both LLMModelFactory (text_config only, no vision
field at all) and VLMModelFactory (vision_config required) — the same
ambiguity already fixed for gemma4 in #152. Matching on model_type
alone routed every text-only checkpoint sharing that string (e.g. the
reported destynova002/Qwen3.6-35B-A3B-mixed-3-4bit-3.85bpw-mlx, which
has no vision_config) to a factory whose config decoder requires a
field the checkpoint never has — a hard startup failure, not a
degraded one.

Removes qwen3_5/qwen3.5/qwen3_5_moe from knownVisionModelTypes; the
existing vision_config-presence check already distinguishes the two
correctly, and continues to catch genuine Qwen3.5-VL checkpoints
(e.g. mlx-community/Qwen3.5-2B-4bit, which does carry a real
vision_config even when used as a text-only speculative-decoding
model in this repo's own CI — unaffected by this change).

Reported with full root-cause diagnosis by @traderjoe1968.
---
 .../ModelArchitectureProbe.swift              | 27 ++++++++++---------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/Sources/MLXInferenceCore/ModelArchitectureProbe.swift b/Sources/MLXInferenceCore/ModelArchitectureProbe.swift
index 19b2d47..1a46799 100644
--- a/Sources/MLXInferenceCore/ModelArchitectureProbe.swift
+++ b/Sources/MLXInferenceCore/ModelArchitectureProbe.swift
@@ -22,21 +22,22 @@ public enum ModelArchitectureProbe {
         "qwen2.5-vl",
         "qwen3_vl",
         "qwen3-vl",
-        "qwen3_5",
-        "qwen3.5",
-        "qwen3_5_moe",
+        // "gemma4" and "qwen3_5"/"qwen3_5_moe" deliberately absent: this codebase
+        // registers each of these exact model_type strings in BOTH LLMModelFactory
+        // (text_config only, no vision field at all) and VLMModelFactory
+        // (vision_config required) — see LLMModelFactory.swift/VLMModelFactory.swift.
+        // The string alone cannot tell the two apart — matching on it treated every
+        // real text-only checkpoint sharing that model_type as a VLM and routed it to
+        // a factory whose config decoder requires a field the checkpoint never has,
+        // which is a hard failure, not a degraded one (reported for gemma4 in #152,
+        // and again for qwen3_5_moe in #156, against a genuinely text-only
+        // Qwen3.6-35B-A3B checkpoint with no vision_config). The vision_config-presence
+        // check below already distinguishes the two correctly, since only the VLM
+        // struct requires that field — these entries were strictly redundant on the
+        // VLM side and wrong on the LLM side. ("qwen3.5" is likewise omitted: it would
+        // only ever normalize to "qwen3_5" anyway, so it was already a dead duplicate.)
         "idefics3",
         "gemma3",
-        // "gemma4" deliberately absent: this codebase registers that exact model_type
-        // in both LLMModelFactory (MLXLLM/Gemma4.swift, text_config only, no vision
-        // field at all) and VLMModelFactory (MLXVLM/Gemma4.swift, vision_config
-        // required). The string alone cannot tell them apart — matching on it treated
-        // every real text-only Gemma4 checkpoint as a VLM and routed it to a factory
-        // whose config decoder requires a field the checkpoint never has, which is a
-        // hard failure, not a degraded one. The vision_config-presence check below
-        // already distinguishes the two correctly, since only the VLM struct requires
-        // that field — this entry was strictly redundant on the VLM side and wrong on
-        // the LLM side.
         "smolvlm",
         "fastvlm",
         "llava_qwen2",