Docker and vLLM Quick Start Guide
Use this guide to prepare a host system with the Linux driver, driver-loaded firmware, and Docker, then start a vLLM container and verify the environment. Complete the setup by sending a prompt to the vLLM server and checking its response.
Prerequisites
Before you begin, make sure that your system meets the following requirements:
Supported hardware:
Intel® Arc™ Pro B50 Graphics
Intel® Arc™ Pro B60 Graphics
Intel® Arc™ Pro B65 Graphics
Intel® Arc™ Pro B70 Graphics
Intel® Arc™ B390 GPU
Intel® Arc™ B370 GPU
Intel® Arc™ 130V/140V Graphics
Supported operating system: Ubuntu 26.04.1.
Installation
To install Docker Engine with containerd and runc as the default runtimes, follow the Docker Engine installation instructions for Ubuntu.
If your system is behind a proxy, see the Docker proxy configuration documentation. Missing proxy configuration may prevent vLLM from loading the model.
Start a vLLM container
Pull the upstream vLLM container image with XPU support.
docker pull vllm/vllm-openai-xpu:latest
Run
xpu-smiinside the container to verify Intel GPU visibility.docker run --rm --device=/dev/dri --name test --entrypoint /usr/bin/xpu-smi vllm/vllm-openai-xpu:latest
If the configuration is successful, the output should resemble the following example:
+----------------------------------------------+----------------------------+------------------------+ | Intel XPU-SMI v2.1 Driver: ACC2C75180429CFD259EDF2 Level Zero: 1.32.0 | +----------------------------------------------+----------------------------+------------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | +----------------------------------------------+----------------------------+------------------------+ | 0 Intel(R) Arc(TM) Pro B Off | 0000:03:00.0 Off | Disabled | | 65 Graphics | | | | 0% 24C 9W / 200W | 51MiB / 32656MiB | N/A Default | +----------------------------------------------+----------------------------+------------------------+ +-------+-------+--------+----------------+----------------------------------------------------------+ | Processes: | | GPU PID Type Process Name GPU Memory Usage | +-------+-------+--------+----------------+----------------------------------------------------------+ | No running GPU processes found. | +-------+-------+--------+----------------+----------------------------------------------------------+
Successful output confirms that the Intel GPU is visible inside the Docker container and ready for workload execution.
Run a simple vLLM prompt and response
This test requires two shells: one to run the vLLM server and another to send requests to it.
In the first shell, start the vLLM server.
docker run --rm --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path:ro --ipc=host -e ZE_AFFINITY_MASK=0 -e ZE_AFFINITY_MASK=0 -p 8002:8000 --name test vllm/vllm-openai-xpu:latest Qwen/Qwen2.5-1.5B-Instruct
Output similar to the following indicates that the vLLM server has completed startup:
INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: 172.17.0.1:46366 - "GET /v1/models HTTP/1.1" 200 OK INFO: 172.17.0.1:46376 - "GET /v1/models HTTP/1.1" 200 OK
In the second shell, wait for the vLLM server to start. Startup can take one or two minutes. Optionally, run the following script to display progress while waiting for the server:
echo -n "waiting " for i in $(seq 1 60); do if curl -s -o /dev/null -w '%{http_code}' http://localhost:8002/v1/models | grep -q 200; then echo "vLLM ready at attempt $i (~$((i*2))s)" break else echo -n " $i " fi sleep 2 done curl -s http://localhost:8002/v1/models | head -c 400 echo
Example output:
waiting 1 2 3 4 5 6 7 8 9 10 11 12 13 14 vLLM ready at attempt 15 (~30s)
In the second shell, send a prompt to the vLLM server.
curl -s http://localhost:8002/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "Qwen/Qwen2.5-1.5B-Instruct", "messages": [{"role":"user","content":"In one sentence, explain recursion"}], "max_tokens": 300 }' | python3 -c 'import json,sys; r=json.load(sys.stdin); print(r["choices"][0]["message"]["content"])'
The response should resemble the following example:
Recursion is a programming technique where a function calls itself to solve a problem by breaking it down into smaller subproblems.