[Bug]vllm backend doesn't work for wsl2 ubuntu22.04

[AirAgentSDE]

Systeminfo

Docker Desktop v4.71.0
WSL Ubuntu 22.04
Windows 11 Pro
Nvidia Geforce RTX 4090

Reproduce issue:

1. engine status

docker model status
Docker Model Runner is running
BACKEND    STATUS         DETAILS                                                                                       
llama.cpp  Running        llama.cpp latest-cuda (sha256:48883a67749cbe5343eacdc666bb1348ea70a55e3702b50059e20e6468ae7be1) e365e65
diffusers  Not Installed                                                                                                
vllm       Not Installed  only supported on Linux  

---

2. installation command

docker model install-runner --backend vllm --gpu cuda
Model Runner container docker-model-runner (85475cc8896c) is already running

---

3. model serving failed

docker model run hf.co/zed-industries/zeta-2 -d
Failed to load model: preload failed: status=500 body=unable to load runner: error waiting for runner to be ready: failed to get args for llama.cpp: GGUF file required by llama.cpp backend

---

4. gpu accessability possible issue

docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi
Unable to find image 'nvidia/cuda:12.0-base' locally
docker: Error response from daemon: manifest for nvidia/cuda:12.0-base not found: manifest unknown: manifest unknown

Run 'docker run --help' for more information

---

5. local gpu availability

nvidia-smi
Fri May  8 07:39:55 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.57                 Driver Version: 591.86         CUDA Version: 13.1     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:01:00.0 Off |                  Off |
| 35%   31C    P8             10W /  450W |     384MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

[AirAgentSDE]

docker model run hf.co/zed-industries/zeta-2 -d
Failed to load model: preload failed: status=500 body=unable to load runner: error waiting for runner to be ready: failed to get args for llama.cpp: GGUF file required by llama.cpp backend

Here I expect the model runner serve the model with vllm backend, but it failed. The GGUF model works well.

[YasharthPanwar-2003]

Hi, @ericcurtin and @AirAgentSDE I’d like to work on this issue. Could you please assign it to me?

I found that install-runner returns early if a container is already running, which skips the requested backend/GPU options. I'm working on a fix to add a helpful hint telling users to use reinstall-runner instead when this happens.I’ve already written unit tests and verified the CLI message behavior locally.

Just a heads-up: I don't have an NVIDIA GPU, so while I can validate the CLI and hint output, I won't be able to test the full CUDA/vLLM runtime path.