Koboldcpp-nocuda 1.116.1 crashing trying to run inference with gemma-4-31B-it-UD-IQ3_XXS.gguf

[8u6man]

Kobold version: 1.116.1
RX7900GRE + Windows 11
model: gemma-4-31B-it-UD-IQ3_XXS.gguf

Model loads but fails during first batch of tokens being processed.

Link to Unsloth repo where the model can be found: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF

[LostRuins]

@henk717 another AMD thing again. I dunno what we can do. On the upside, not a single nvidia related issue.

[LostRuins]

@8u6man can you try setting the env var GGML_VK_MAX_NODES_PER_SUBMIT=1

[henk717]

Same thing as the original plan, people having the issue need to try the isolated builds so why know why it broke and can report it upstream.
Its not happening on my AMD Laptop and neither of us have other AMD capable devices so all we can do is have people reporting it testing it.
If they don't report that kinda info back it will be eventually maybe hopefully fixed one day if upstream fixes the issue.

[8u6man]

can you try setting the env var GGML_VK_MAX_NODES_PER_SUBMIT=1

This got it to complete inference without crashing. Processing speed is as slow as if it was overflowing to RAM though, about 1/5 the speed of the same Gemma model at IQ3_XS. :/

(Edit: I am not doing that again. After changing the environment variable kobold started crashing trying to run the previously working model and there was leftover VRAM (0.1GB) allocated after the terminal closed.)

[wbruna]

This got it to complete inference without crashing. Processing speed is as slow as if it was overflowing to RAM though, about 1/5 the speed of the same Gemma model at IQ3_XS. :/

The value can be tuned for your hardware. 1 is the minimum, just to be sure it is failing for this specific reason (a Device Lost error could happen for other reasons). You can try increasing it a bit; the default value is 100.

(Edit: I am not doing that again. After changing the environment variable kobold started crashing trying to run the previously working model and there was leftover VRAM (0.1GB) allocated after the terminal closed.)

I believe this is a separate issue, because the only thing that variable changes is the number of operations submitted to the hardware at once.