Git commit
b290693
Operating System & Version
archlinux 7.1.3-arch2-1
GGML backends
Vulkan
Command-line arguments used
docker run --rm -v ~/gguf/sd:/models --name sd-server --gpus all --entrypoint "/sd-server" ghcr.io/leejet/stable-diffusion.cpp:master-vulkan --listen-ip 0.0.0.0 --listen-port 8081--fa --diffusion-fa --vae-tiling --diffusion-model /models/flux-2-klein-9b-Q8_0.gguf --llm /models/Qwen3-8B-Q8_0.gguf --vae /models/flux2-klein_diffusion_pytorch_model.safetensors --steps 4 --cfg-scale 1.0 --img-cfg-scale 1.0 --guidance 4.0
Steps to reproduce
- install
nvidia-container-toolkit
- run
nvidia-ctk runtime configure --runtime=docker
- restart docker service
- run docker container with
--gpus all
What you expected to happen
This steps I used to make llama.cpp ghcr.io/ggml-org/llama.cpp:server-vulkan to work with Nvidia GPU and expected it to also work for stable-diffusion.cpp ghcr.io/leejet/stable-diffusion.cpp:master-vulkan.
What actually happened
Doesn't see GPU and runs on CPU.
Logs / error messages / stack trace
ggml_vulkan: No devices found. load_backend: loaded Vulkan backend from /sd.cpp/bin/libggml-vulkan.so load_backend: loaded CPU backend from /sd.cpp/bin/libggml-cpu-haswell.so [INFO ] stable-diffusion.cpp:710 - loading diffusion model from '/models/flux-2-klein-9b-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/flux-2-klein-9b-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:772 - loading llm from '/models/Qwen3-8B-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/Qwen3-8B-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:786 - loading vae from '/models/flux2-klein_diffusion_pytorch_model.safetensors' [INFO ] model_loader.cpp:242 - load /models/flux2-klein_diffusion_pytorch_model.safetensors using safetensors format [INFO ] stable-diffusion.cpp:847 - Version: Flux.2 klein [INFO ] stable-diffusion.cpp:902 - Weight type stat: f32: 225 | q8_0: 365 | i32: 1 | bf16: 259 [INFO ] stable-diffusion.cpp:903 - Conditioner weight type stat: f32: 145 | q8_0: 253 [INFO ] stable-diffusion.cpp:904 - Diffusion model weight type stat: f32: 80 | q8_0: 112 | bf16: 9 [INFO ] stable-diffusion.cpp:905 - VAE weight type stat: i32: 1 | bf16: 250 [INFO ] stable-diffusion.cpp:1359 - using VAE for encoding / decoding [INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128 [INFO ] stable-diffusion.cpp:1463 - Using flash attention [INFO ] stable-diffusion.cpp:1477 - Using flash attention in the diffusion model [INFO ] stable-diffusion.cpp:1583 - total params memory size = 17346.24MB (VRAM 0.00MB, RAM 17346.24MB): text_encoders 7669.77MB(RAM), diffusion_model 9516.04MB(RAM), vae 160.43MB(RAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A) [INFO ] stable-diffusion.cpp:1699 - running in Flux FLOW mode [INFO ] main.cpp:148 - listening on: http://0.0.0.0:8081
Additional context / environment details
Compiling vulkan version locally works as expected, issue is only present with docker image.
Git commit
b290693
Operating System & Version
archlinux 7.1.3-arch2-1
GGML backends
Vulkan
Command-line arguments used
docker run --rm -v ~/gguf/sd:/models --name sd-server --gpus all --entrypoint "/sd-server" ghcr.io/leejet/stable-diffusion.cpp:master-vulkan --listen-ip 0.0.0.0 --listen-port 8081--fa --diffusion-fa --vae-tiling --diffusion-model /models/flux-2-klein-9b-Q8_0.gguf --llm /models/Qwen3-8B-Q8_0.gguf --vae /models/flux2-klein_diffusion_pytorch_model.safetensors --steps 4 --cfg-scale 1.0 --img-cfg-scale 1.0 --guidance 4.0
Steps to reproduce
nvidia-container-toolkitnvidia-ctk runtime configure --runtime=docker--gpus allWhat you expected to happen
This steps I used to make llama.cpp
ghcr.io/ggml-org/llama.cpp:server-vulkanto work with Nvidia GPU and expected it to also work for stable-diffusion.cppghcr.io/leejet/stable-diffusion.cpp:master-vulkan.What actually happened
Doesn't see GPU and runs on CPU.
Logs / error messages / stack trace
ggml_vulkan: No devices found. load_backend: loaded Vulkan backend from /sd.cpp/bin/libggml-vulkan.so load_backend: loaded CPU backend from /sd.cpp/bin/libggml-cpu-haswell.so [INFO ] stable-diffusion.cpp:710 - loading diffusion model from '/models/flux-2-klein-9b-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/flux-2-klein-9b-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:772 - loading llm from '/models/Qwen3-8B-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/Qwen3-8B-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:786 - loading vae from '/models/flux2-klein_diffusion_pytorch_model.safetensors' [INFO ] model_loader.cpp:242 - load /models/flux2-klein_diffusion_pytorch_model.safetensors using safetensors format [INFO ] stable-diffusion.cpp:847 - Version: Flux.2 klein [INFO ] stable-diffusion.cpp:902 - Weight type stat: f32: 225 | q8_0: 365 | i32: 1 | bf16: 259 [INFO ] stable-diffusion.cpp:903 - Conditioner weight type stat: f32: 145 | q8_0: 253 [INFO ] stable-diffusion.cpp:904 - Diffusion model weight type stat: f32: 80 | q8_0: 112 | bf16: 9 [INFO ] stable-diffusion.cpp:905 - VAE weight type stat: i32: 1 | bf16: 250 [INFO ] stable-diffusion.cpp:1359 - using VAE for encoding / decoding [INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128 [INFO ] stable-diffusion.cpp:1463 - Using flash attention [INFO ] stable-diffusion.cpp:1477 - Using flash attention in the diffusion model [INFO ] stable-diffusion.cpp:1583 - total params memory size = 17346.24MB (VRAM 0.00MB, RAM 17346.24MB): text_encoders 7669.77MB(RAM), diffusion_model 9516.04MB(RAM), vae 160.43MB(RAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A) [INFO ] stable-diffusion.cpp:1699 - running in Flux FLOW mode [INFO ] main.cpp:148 - listening on: http://0.0.0.0:8081
Additional context / environment details
Compiling vulkan version locally works as expected, issue is only present with docker image.