Docker and vLLM Quick Start Guide

Use this guide to prepare a host system with the Linux driver, driver-loaded firmware, and Docker, then start a vLLM container and verify the environment. Complete the setup by sending a prompt to the vLLM server and checking its response.

Prerequisites

Before you begin, make sure that your system meets the following requirements:

  • Supported hardware:

    • Intel® Arc™ Pro B50 Graphics

    • Intel® Arc™ Pro B60 Graphics

    • Intel® Arc™ Pro B65 Graphics

    • Intel® Arc™ Pro B70 Graphics

    • Intel® Arc™ B390 GPU

    • Intel® Arc™ B370 GPU

    • Intel® Arc™ 130V/140V Graphics

  • Supported operating system: Ubuntu 26.04.1.

Installation

To install Docker Engine with containerd and runc as the default runtimes, follow the Docker Engine installation instructions for Ubuntu.

If your system is behind a proxy, see the Docker proxy configuration documentation. Missing proxy configuration may prevent vLLM from loading the model.

Start a vLLM container

  1. Pull the upstream vLLM container image with XPU support.

    docker pull vllm/vllm-openai-xpu:latest
    
  2. Run xpu-smi inside the container to verify Intel GPU visibility.

    docker run --rm --device=/dev/dri --name test --entrypoint /usr/bin/xpu-smi vllm/vllm-openai-xpu:latest
    

    If the configuration is successful, the output should resemble the following example:

    +----------------------------------------------+----------------------------+------------------------+
    |            Intel XPU-SMI v2.1    Driver: ACC2C75180429CFD259EDF2    Level Zero: 1.32.0             |
    +----------------------------------------------+----------------------------+------------------------+
    | GPU  Name                    Persistence-M   | Bus-Id            Disp.A   |   Volatile Uncorr. ECC |
    | Fan              Temp  Pwr:Usage/Cap         | Memory-Usage               |   GPU-Util  Compute M. |
    +----------------------------------------------+----------------------------+------------------------+
    |   0  Intel(R) Arc(TM) Pro B  Off             | 0000:03:00.0      Off      |               Disabled |
    |      65 Graphics                             |                            |                        |
    | 0%                24C  9W / 200W             | 51MiB / 32656MiB           |      N/A       Default |
    +----------------------------------------------+----------------------------+------------------------+
    
    +-------+-------+--------+----------------+----------------------------------------------------------+
    | Processes:                                                                                         |
    |   GPU     PID    Type    Process Name                                             GPU Memory Usage |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    |                                  No running GPU processes found.                                   |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    

    Successful output confirms that the Intel GPU is visible inside the Docker container and ready for workload execution.

Run a simple vLLM prompt and response

This test requires two shells: one to run the vLLM server and another to send requests to it.

  1. In the first shell, start the vLLM server.

    docker run --rm --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path:ro --ipc=host -e ZE_AFFINITY_MASK=0 -e ZE_AFFINITY_MASK=0 -p 8002:8000 --name test   vllm/vllm-openai-xpu:latest Qwen/Qwen2.5-1.5B-Instruct
    

    Output similar to the following indicates that the vLLM server has completed startup:

    INFO:     Started server process [1]
    INFO:     Waiting for application startup.
    INFO:     Application startup complete.
    INFO:     172.17.0.1:46366 - "GET /v1/models HTTP/1.1" 200 OK
    INFO:     172.17.0.1:46376 - "GET /v1/models HTTP/1.1" 200 OK
    
  2. In the second shell, wait for the vLLM server to start. Startup can take one or two minutes. Optionally, run the following script to display progress while waiting for the server:

    echo -n "waiting "
    for i in $(seq 1 60); do
        if curl -s -o /dev/null -w '%{http_code}' http://localhost:8002/v1/models | grep -q 200; then
            echo "vLLM ready at attempt $i (~$((i*2))s)"
            break
        else
            echo -n " $i "
        fi
        sleep 2
    done
    curl -s http://localhost:8002/v1/models | head -c 400
    echo
    

    Example output:

    waiting  1  2  3  4  5  6  7  8  9  10  11  12  13  14 vLLM ready at attempt 15 (~30s)
    
  3. In the second shell, send a prompt to the vLLM server.

    curl -s http://localhost:8002/v1/chat/completions \
        -H 'Content-Type: application/json' \
        -d '{
            "model": "Qwen/Qwen2.5-1.5B-Instruct",
            "messages": [{"role":"user","content":"In one sentence, explain recursion"}],
            "max_tokens": 300
        }' | python3 -c 'import json,sys; r=json.load(sys.stdin); print(r["choices"][0]["message"]["content"])'
    

    The response should resemble the following example:

    Recursion is a programming technique where a function calls itself to solve a problem by breaking it down into smaller subproblems.