Running SGLang Inference Workloads in Docker Containers
Follow this guide to prepare an SGLang Docker container for inference on an Intel GPU and check a serving workload.
Prerequisites
Before you begin, make sure Docker and GPU drivers are installed on the host. For GPU driver installation, refer to the Installation Guide.
The public model used in this guide does not require a Hugging Face access token. If you choose a private or gated model instead, follow the Preparing a Hugging Face account and access token section of the vLLM guide and accept any required model access agreement before starting the container.
Running SGLang in Docker containers
These steps use a prebuilt XPU image from Docker Hub. Replace VERSION in both commands with the same current XPU image tag ending in -xpu; do not use the generic latest tag as a substitute.
Pull the prebuilt XPU image.
docker pull lmsysorg/sglang:VERSION
Start the container. This command opens a shell inside it; keep this terminal open while the server runs.
docker run --rm -it \ --name sglang-xpu \ --privileged \ --ipc=host \ --network=host \ --user root \ --group-add "$(getent group video | cut -d: -f3)" \ --device /dev/dri \ -v /dev/dri/by-path:/dev/dri/by-path \ -v /dev/shm:/dev/shm \ -v "${HOME}/.cache/huggingface:/root/.cache/huggingface" \ lmsysorg/sglang:VERSION /bin/bash
For a private or gated model, set
HF_TOKENon the host and add-e HF_TOKENto thedocker runcommand. Do not put the token value directly in the command; environment variables passed to Docker may still be visible to users with Docker access.Inside the container, start an SGLang server. The example uses
Qwen/Qwen3-0.6Bon one GPU.sglang serve \ --model-path Qwen/Qwen3-0.6B \ --device xpu \ --host 127.0.0.1 \ --port 30000
Wait for the server to report that it is ready before sending requests. Keep the server running in this terminal.
In another terminal on the host, enter the running container to run the client.
docker exec -it sglang-xpu /bin/bash
Inside the second shell, send a request to check that the server responds.
curl http://127.0.0.1:30000/v1/completions \ -H "Content-Type: application/json" \ -d '{"model":"Qwen/Qwen3-0.6B","prompt":"The capital of France is","max_tokens":32}'
A successful response contains generated text. If this request fails, confirm that the server has finished loading the model and is listening on port 30000.
Optionally, benchmark the running server from the second shell using the upstream XPU benchmark example.
python -m sglang.benchmark.serving \ --host 127.0.0.1 \ --port 30000 \ --dataset-name random \ --random-input-len 1024 \ --random-output-len 1024 \ --num-prompts 1 \ --request-rate inf \ --random-range-ratio 1.0
Optional: building an XPU image from the Dockerfile
Instead of pulling a prebuilt image, you can build one using the upstream XPU Dockerfile. To do it, run these commands on the host:
git clone https://github.com/sgl-project/sglang.git
cd sglang/docker
docker build -t sglang-xpu:latest -f xpu.Dockerfile .
Then follow the container and serving steps above, replacing lmsysorg/sglang:VERSION in the docker run command with sglang-xpu:latest. Skip the Docker Hub pull step.