Admit and benchmark Qwen3.5 27B Q4_K_M on native Psionic

[AtlantisPleb]

Goal: download a real Hugging Face Qwen3.5 27B Q4_K_M GGUF, make it run end to end on native Psionic, and publish honest benchmark numbers in this issue.

Bounded target for this issue:

• pick one real HF artifact for the 27B Q4_K_M row
• make the current loader/runtime admit it instead of refusing or crashing
• keep support honest about backend boundaries
• run a real benchmark receipt on the host(s) we can reach now
• post TTFT, total latency, and end-to-end tok/s as comments

Likely references to inspect while implementing:

• current qwen35 loader and runtime in psionic-models and psionic-serve
• MLX Qwen support if a matching 27B row exists
• llama.cpp GGUF/Qwen handling for metadata and layer-shape assumptions

This issue is done when one explicit 27B Q4_K_M Hugging Face row runs on Psionic with posted receipts.

[AtlantisPleb]

Pinned the target row and references before touching code.

Current working target:

• unsloth/Qwen3.5-27B-GGUF -> Qwen3.5-27B-Q4_K_M.gguf

Reference rows that exist now:

• mlx-community/Qwen3.5-27B-4bit
• bartowski/Qwen_Qwen3.5-27B-GGUF -> Qwen_Qwen3.5-27B-Q4_K_M.gguf

Current repo truth before changes:

• native qwen35 is a CUDA runtime
• qwen35 also has a CPU llama.cpp proxy runtime in psionic-serve/src/gguf.rs
• the current proven rows are still the smaller 0.8B/2B/4B/9B lanes

Next step is to run the 27B GGUF through the existing loader/runtime and post the first real refusal or receipt instead of guessing where the gap is.

[AtlantisPleb]

Progress update:

I checked the native service seam. The current non-CUDA blocker is not the math core. qwen35 is still routed into a llama.cpp proxy wrapper at the CPU service boundary, while the dense GGUF engine already carries generic attention and FFN logic that should be usable for text-only native execution once the family gate is removed.

Current concrete blockers on the real 27B GGUF:

• new default chat-template digest: e60df41481b6ad20571cda7e2e1290cfee25670f1b50973f74576c9354c66336
• pad token moved from the older 0.8B fixture value to 248055
• the older qwen35.vision.* family facts do not appear in this artifact, so text admission must not assume the earlier multimodal projection metadata is present

Next step is patching native CPU admission and adding a native benchmark path, then rerunning on the real 27B artifact.

[AtlantisPleb]

Native Qwen 3.5 27B support is now past the public-artifact admission blockers on this worktree.

What changed:

• native Psionic CPU routing stays on the real qwen35 runtime and does not proxy to llama.cpp
• qwen35.attention.head_count_kv now accepts the scalar form used by the Hugging Face 27B GGUF
• the loader now accepts the public blk.N.ssm_dt.bias naming variant instead of requiring only blk.N.ssm_dt
• missing qwen35.ssm.v_head_reordered now defaults to false instead of rejecting the artifact
• regression coverage now includes the scalar-KV-plus-ssm_dt.bias shape

Real artifact used:

• /Users/christopherdavid/models/qwen3.5/Qwen3.5-27B-Q4_K_M.gguf

Benchmark command:

target/debug/examples/qwen35_bench \
  --model-path /Users/christopherdavid/models/qwen3.5/Qwen3.5-27B-Q4_K_M.gguf \
  --backend cpu \
  --prompt 'Write one short sentence about open source AI.' \
  --max-output-tokens 16 \
  --repeats 1 \
  --json

Receipt:

• load: 30.646 s
• prompt pass: 303.782 s
• decode: 220.120 s
• total: 523.902 s
• decode throughput: 0.0727 tok/s
• end-to-end throughput: 0.0305 tok/s for 16 output tokens

One honest note: this benchmark is the low-level raw prompt harness, not the higher-level chat-completions surface with prompt rendering, so I am treating the timing receipt as the important output here, not the sampled text quality from that raw prompt path.

[AtlantisPleb]

Landed on main in b7f96f52.

This closes the native-admission gap for the public Hugging Face Qwen3.5-27B-Q4_K_M.gguf artifact on Psionic CPU.

Shipped support:

• native qwen35 CPU execution path, no llama.cpp proxy on the runtime path
• GGML_TYPE_Q5_K admission and CPU decode/dot support
• scalar qwen35.attention.head_count_kv
• blk.N.ssm_dt.bias admission
• missing qwen35.ssm.v_head_reordered defaults to false
• committed benchmark harness at crates/psionic-serve/examples/qwen35_bench.rs
• committed regression coverage for the scalar-KV plus ssm_dt.bias shape

Real artifact benchmark receipt:

• model: /Users/christopherdavid/models/qwen3.5/Qwen3.5-27B-Q4_K_M.gguf
• backend: cpu
• load: 30.646 s
• prompt pass: 303.782 s
• decode: 220.120 s
• total: 523.902 s
• decode throughput: 0.0727 tok/s
• end-to-end throughput: 0.0305 tok/s across 16 output tokens

The raw bench harness still measures a low-level prompt path rather than the higher-level chat-completions surface, so the timing receipt is the main claim here.

[AtlantisPleb]

Closed as already shipped on main in b7f96f5.

[AtlantisPleb]

Closing this as shipped. The issue definition of done was one explicit Hugging Face Qwen3.5-27B-Q4_K_M.gguf row running on native Psionic with posted receipts, and that work is already on main in b7f96f52 (qwen35: admit and benchmark hf 27b gguf).

Shipped scope on main:

• native qwen35 CPU execution path for the public 27B GGUF row
• GGML_TYPE_Q5_K admission and CPU decode/dot support
• scalar qwen35.attention.head_count_kv
• blk.N.ssm_dt.bias admission
• missing qwen35.ssm.v_head_reordered defaults
• committed qwen35_bench harness and regression coverage

Receipt already posted in this issue:

• artifact: /Users/christopherdavid/models/qwen3.5/Qwen3.5-27B-Q4_K_M.gguf
• backend: cpu
• load: 30.646 s
• prompt: 303.782 s
• decode: 220.120 s
• total: 523.902 s
• decode throughput: 0.0727 tok/s
• end-to-end throughput: 0.0305 tok/s across 16 output tokens

No further repo change is needed for this specific issue; the follow-on performance work lives in #899 and #900.