fix: 兼容 vLLM 0.23 的权重加载与 spark25 工具解析 - #10
Open
MittaPei wants to merge 1 commit into
Open
Conversation
Signed-off-by: MittaPei <315415437+MittaPei@users.noreply.github.com>
MittaPei
force-pushed
the
fix/vllm-0.23-compat
branch
from
September 4, 2026 06:09
1edcc07 to
ddef50f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Fixes #8
Description:
Issue #8 的首个异常来自 API 版本差异:vLLM 0.23 的
WeightsMapper不接受orig_to_new_stacked。但只移除这个参数并不足以修复原始启动命令:gate_proj/up_proj,旧版 vLLM 仍需将它们分别装入 packedgate_up_proj的 shard 0/1;--tool-call-parser spark25,而 vLLM 0.23 的vllm.tool_parsers.utils还没有插件当前导入的find_tool_name。本 PR 做了以下局部修改:
q_k_v_proj的直接加载、PP missing 参数处理和 tiedlm_head跳过;AutoWeightsLoader + WeightsMapper;find_tool_name的 vLLM 提供最小兼容 helper,并兼容两类工具对象形态;验证结果:
pytest tests/ -qvllm/vllm-openai:v0.23.081efe788…源码快照合肥tool_choice=auto+spark25get_weather({"city":"合肥"})真实运行固定了模型 revision
1e4c2477f330d1f11304d0dc092a9dbf4075f470和 vLLM 镜像 digestsha256:6d8429e38e3747723ca07ee1b17972e09bb9c51c4032b266f24fb1cc3b22ed8f。为控制共享 GPU 资源,max_model_len设为 131072。补充复现材料与完整原始记录:MittaPei/spark-x2-5-vllm-review-case
Checklist:
ruff check .)pytest tests/ -q)src/vllm_spark2_5_plugin/are noted if changedSpecial notes for reviewers:
本 PR 修改了 vendored 文件
spark2_5.py和spark2_5_tool_parser.py;来源仍为 README 记录的 vLLM commit81efe7883f30582696b69f9b9ea93c4819a8c608,原有 SPDX 标识保持不变,仅在其上叠加局部兼容层。两处文件依赖 vLLM 内部 API,因此使用能力检测同时覆盖 vLLM 0.23 和插件记录的较新 API 快照。尚未验证量化 checkpoint、多机、Pipeline Parallel 实机运行或 vLLM 0.28;最终冷启动也没有复刻 Issue 报告者的加速器插件环境。
AI 辅助说明:我使用 Spark-X2.5-4B 对冻结候选提交做受控代码审查,并使用通用编码助手辅助实现、测试和文档整理;我逐项复核补丁与验证结果后提交,并对本次贡献负责。正确性以本 PR 的确定性测试和真实运行结果为准。