Enabling Minimax, Deepseek and GLM with weightfree support by quic-amitraj · Pull Request #1500 · qualcomm/efficient-transformers

Amit Raj (quic-amitraj) · 2026-11-13T10:53:27Z

Summary

Adds QEfficient weight-free export, compilation, and QAIC execution support for MiniMax M3, DeepSeek V4, and GLM-5.3.

MiniMax M3

Adds paged and contiguous KV-cache support for decode and prefill-to-decode workflows.
Supports DP/CP attention, expert-parallel prefill, FP16/BF16, and weight-free export.

DeepSeek V4

Adds decode support for sliding, heavily compressed, and compressed sparse attention.
Supports folded DP/CP caches, FFN blocking, expert-parallel prefill, and Dynamo weight-free export.

GLM-5.3

Adds dense MLA and DeepSeek Sparse Attention paths with configurable parallel and blocked execution.
Supports folded caches, FP8 weight-free export, and separate prefill/decode workflows.

Validation

Adds focused PyTorch, ONNX Runtime, export, cache, blocking, and QAIC parity coverage. QAIC parity tests pending for GLM and Deepseek, but 4 layer runs verified to match huggingface with example scripts.

Signed-off-by: ochougul <ochougul@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: ochougul <ochougul@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

The raw sliding-window attention path in all four cache layers (sliding, HCA, CSA) gathered context_length (the full ctx_len-wide retained-state buffer) on every decode step instead of just the sliding_window-wide recent window, and the model-level causal mask was built at the same unnecessarily wide ctx_len. Both are now narrowed to sliding_window width via a shared _sliding_window_context_indices helper, removing O(ctx_len) wasted attention/ReduceMax compute from every decode step without changing model output. Verified via the existing CPU parity tests (test_deepseek_v4_*, full test_model_quickcheck.py suite unaffected) and by matching generated token IDs on real QAIC hardware, before and after the fix, for both the current ping-pong CSA cache and (previously) the legacy overlap layout. Signed-off-by: divytrip <divytrip@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: divytrip3005 <divytrip@qti.qualcomm.com> (cherry picked from commit 9b9e932e0fa81856e0d52f49138907fa4e94946a) Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

This reverts commit 06e7407. Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>

Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com> Signed-off-by: Kushal Dulla <kdulla@qti.qualcomm.com>