Embedding::init_params falls back to F32 whenever support_get_rows() rejects the stored type, and the allowlist only contains F16/Q8_0/Q5_1/Q5_0/Q4_1/Q4_0 (src/core/ggml_extend.hpp). ggml's CPU get_rows has dispatched all the K-quants for a while now (ggml-cpu/ops.cpp, ggml_compute_forward_get_rows_q).
Practical impact: city96's umt5-xxl Q3_K_S stores token_embd.weight as q6_K (862MB). It loads as F32 = 4.2GB, which is the difference between running and OOM on an 8GB Android device. Nothing in the logs points at the embedding; the file size suggests it should fit.
Locally we extended the allowlist with Q2_K..Q6_K and verified q6_K end-to-end on-device (wan2.1 video pipeline, CPU backend). Two things we did NOT verify, which is why this is an issue rather than a PR: coverage of the other K-quants beyond q6_K, and whether every non-CPU backend's get_rows handles K-quants (if not, a ggml_backend_supports_op-style check may be better than a static list).
Embedding::init_paramsfalls back to F32 wheneversupport_get_rows()rejects the stored type, and the allowlist only contains F16/Q8_0/Q5_1/Q5_0/Q4_1/Q4_0 (src/core/ggml_extend.hpp). ggml's CPUget_rowshas dispatched all the K-quants for a while now (ggml-cpu/ops.cpp,ggml_compute_forward_get_rows_q).Practical impact: city96's umt5-xxl Q3_K_S stores
token_embd.weightas q6_K (862MB). It loads as F32 = 4.2GB, which is the difference between running and OOM on an 8GB Android device. Nothing in the logs points at the embedding; the file size suggests it should fit.Locally we extended the allowlist with Q2_K..Q6_K and verified q6_K end-to-end on-device (wan2.1 video pipeline, CPU backend). Two things we did NOT verify, which is why this is an issue rather than a PR: coverage of the other K-quants beyond q6_K, and whether every non-CPU backend's get_rows handles K-quants (if not, a
ggml_backend_supports_op-style check may be better than a static list).