Repository navigation
fix(mlx-vlm): bump mlx-vlm pin to v0.6.12 - #12818
Merged
Merged
Conversation
genx7up
force-pushed
the
fix/mlx-vlm-pin-v0.6.12
branch
from
October 21, 2026 23:42
750f426 to
2f6d995
Compare
Collaborator
|
@mudler Code-review sign-off at All six MLX-VLM variants use the same v0.6.12 pin. Upstream retains the generation and tool-parser APIs used by this backend, and its image-placement logic excludes tool messages, addressing the reported image/tool-call failure. Integration with master I could not independently reproduce dependency resolution or run backend tests because this environment lacks a usable Python runtime. Metal, CUDA and L4T execution remain unverified; the current check snapshot contains only passing DCO. This is a source-review sign-off, not confirmation of runtime readiness. |
mlx-vlm v0.4.4 attaches the image placeholder to the last message that is not system or assistant. After a tool call that is the tool result, which the chat template renders as text, so the next turn fails with "Image features and image tokens do not match: tokens: 0". v0.6.12 keeps the placeholder on the user message that carried the image. The v0.4.4 pin (mudler#9568) worked around the mlx-cuda wheels stopping at 0.31.1. They are at 0.32.4 now, and v0.6.12 (mlx>=0.32.0) resolves with mlx[cuda12] and mlx[cuda13]. v0.7.x would need backend changes: it drops mlx_vlm.tool_parsers, which backend.py imports. Signed-off-by: gman <genx7up@users.noreply.github.com>
genx7up
force-pushed
the
fix/mlx-vlm-pin-v0.6.12
branch
from
October 22, 2026 00:29
2f6d995 to
3313c0c
Compare
mudler
approved these changes
Oct 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
With the v0.4.4 pin, a chat that sends an image and then goes through a tool call fails on the next turn with
Image features and image tokens do not match: tokens: 0, features 646. v0.4.4 puts the image placeholder on the tool result; v0.6.12 is the first release that keeps it on the user message, so this bumps the pin in all six requirements files.Notes for Reviewers
The pin came from #9568, when the mlx-cuda wheels stopped at 0.31.1. They are at 0.32.4 now and
pip install --dry-runresolves v0.6.12 withmlx[cuda12]andmlx[cuda13]. I stopped at v0.6.12 because v0.7.x dropsmlx_vlm.tool_parsers, which backend.py imports.Tested on Linux CPU (python:3.12, requirements-cpu.txt): backend.py imports and serves,
TestSharedHelperspasses, and a tool-call round rendered with a Qwen2.5-VL processor gets 0 image placeholders at v0.4.4 and 1 at v0.6.12.test_load_modelfails at both pins because facebook/opt-125m has no safetensors on the Hub. Not run on Metal, CUDA or L4T hardware.Signed commits