Skip to content

[Bug] master-vulkan docker image doesn't work with Nvidia GPU. #1792

Description

@somewhatfrog

Git commit

b290693

Operating System & Version

archlinux 7.1.3-arch2-1

GGML backends

Vulkan

Command-line arguments used

docker run --rm -v ~/gguf/sd:/models --name sd-server --gpus all --entrypoint "/sd-server" ghcr.io/leejet/stable-diffusion.cpp:master-vulkan --listen-ip 0.0.0.0 --listen-port 8081--fa --diffusion-fa --vae-tiling --diffusion-model /models/flux-2-klein-9b-Q8_0.gguf --llm /models/Qwen3-8B-Q8_0.gguf --vae /models/flux2-klein_diffusion_pytorch_model.safetensors --steps 4 --cfg-scale 1.0 --img-cfg-scale 1.0 --guidance 4.0

Steps to reproduce

  1. install nvidia-container-toolkit
  2. run nvidia-ctk runtime configure --runtime=docker
  3. restart docker service
  4. run docker container with --gpus all

What you expected to happen

This steps I used to make llama.cpp ghcr.io/ggml-org/llama.cpp:server-vulkan to work with Nvidia GPU and expected it to also work for stable-diffusion.cpp ghcr.io/leejet/stable-diffusion.cpp:master-vulkan.

What actually happened

Doesn't see GPU and runs on CPU.

Logs / error messages / stack trace

ggml_vulkan: No devices found. load_backend: loaded Vulkan backend from /sd.cpp/bin/libggml-vulkan.so load_backend: loaded CPU backend from /sd.cpp/bin/libggml-cpu-haswell.so [INFO ] stable-diffusion.cpp:710 - loading diffusion model from '/models/flux-2-klein-9b-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/flux-2-klein-9b-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:772 - loading llm from '/models/Qwen3-8B-Q8_0.gguf' [INFO ] model_loader.cpp:236 - load /models/Qwen3-8B-Q8_0.gguf using gguf format [INFO ] stable-diffusion.cpp:786 - loading vae from '/models/flux2-klein_diffusion_pytorch_model.safetensors' [INFO ] model_loader.cpp:242 - load /models/flux2-klein_diffusion_pytorch_model.safetensors using safetensors format [INFO ] stable-diffusion.cpp:847 - Version: Flux.2 klein [INFO ] stable-diffusion.cpp:902 - Weight type stat: f32: 225 | q8_0: 365 | i32: 1 | bf16: 259 [INFO ] stable-diffusion.cpp:903 - Conditioner weight type stat: f32: 145 | q8_0: 253 [INFO ] stable-diffusion.cpp:904 - Diffusion model weight type stat: f32: 80 | q8_0: 112 | bf16: 9 [INFO ] stable-diffusion.cpp:905 - VAE weight type stat: i32: 1 | bf16: 250 [INFO ] stable-diffusion.cpp:1359 - using VAE for encoding / decoding [INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128 [INFO ] stable-diffusion.cpp:1463 - Using flash attention [INFO ] stable-diffusion.cpp:1477 - Using flash attention in the diffusion model [INFO ] stable-diffusion.cpp:1583 - total params memory size = 17346.24MB (VRAM 0.00MB, RAM 17346.24MB): text_encoders 7669.77MB(RAM), diffusion_model 9516.04MB(RAM), vae 160.43MB(RAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A) [INFO ] stable-diffusion.cpp:1699 - running in Flux FLOW mode [INFO ] main.cpp:148 - listening on: http://0.0.0.0:8081

Additional context / environment details

Compiling vulkan version locally works as expected, issue is only present with docker image.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions