feat(gguf): serve GLM-4.5, GLM-4.6 and GLM-4.7 GGUF checkpoints

[jason-fxz]

Summary

Serves llama.cpp glm4moe GGUF files (GLM-4.5, GLM-4.5-Air, GLM-4.6, GLM-4.7 and their finetunes) on the existing glm4_moe model classes. Text only.

Changes

• models/glm4_moe/gguf.py (new):
  • Builds the config from the GGUF metadata, with llama.cpp's defaults for the router keys. The trailing MTP block is not read.
  • The converter writes no flag for q/k norms or the qkv bias, so they come from the tensor table (GLM-4.5-Air has no q/k norms, the 355B models do).
  • llama.cpp stores the tensors unchanged (NeoX rope on both sides), so nothing is undone.
  • Refuses non-sigmoid gating, rope scaling, fused attn_qkv, prism files, per-tensor scales, MTP-only draft files and metadata-only GGUFs.
• attention.py, model.py: with a GGUF file, q/k/v/o and the embedding take the quant config instead of the DF11 layers of the NVFP4 recipe. NVFP4 checkpoints build the same layers as before.
• gguf/tokenizer.py: glm4 gets the Llama 3 split and ignore_merges (fix(gguf): set ignore_merges for llama 3 gguf tokenizers #728), as in HF tokenizer.json and llama.cpp. transformers maps no split for it.
• gguf/config.py, register.py, kernel/aot_models.py: register the glm4moe arch and claim it in the AOT model table.

Tested models

RTX PRO 6000 Blackwell (sm_120), ft serve --model-path <file>. AIME25 problems 1–8 (temperature 1.0, top_p 0.95, from the GLM-4.7 model card). Also 48 staggered requests at concurrency 16.

Model	Quant	bs=1 tok/s	AIME FreeToken / llama.cpp f2cc728
unsloth/GLM-4.5-Air-GGUF	Q2_K	118	31/32 / 30/32
unsloth/GLM-4.7-GGUF	UD-TQ1_0, offload	35	7/8 (1 sample per problem) / –

Staggered requests: no corrupted output on either file.

Results

• Weights: after ggml.dequantize, norms, biases, the router and e_score_correction_bias are exact against zai-org/GLM-4.5-Air and nvidia/GLM-4.7-NVFP4, and Q8_0 tensors reach cos 0.99998. Dry load against the meta-built model matches 645/645 params (Air) and 1469/1469 (GLM-4.7).
• Tokenizer: every vocabulary token plus 20k random concatenations encode as in HF tokenizer.json and llama.cpp (main: 22,840 vocabulary tokens differed).
• vs llama.cpp f2cc728: same file, greedy, 6 prompts × 128 tokens. GLM-4.5-Air: 1/6 identical; every divergence is a near-tie in llama.cpp's own probabilities (e.g. 0.506 vs 0.484).
• Coverage: header check (dry load, kernel coverage, tokenizer, chat template) of 1249 files from 152 HF repos: 1133 pass. The rest are IQ1_M, ik_llama.cpp types, GLM-4.7-Flash (deepseek2 arch), MTP-only drafts and a few broken uploads.
• Unit tests: tests/models, tests/moe and tests/kernels/test_ggml.py pass, except failures that also fail on main.

Not covered

• IQ1_M weights (no MMQ kernel) and files with fused attn_qkv.
• GLM-4.5V / 4.6V image input, GLM-4.7-Flash, MTP.
• TP > 1.