Creating and Configuring Virtual Machines with GPU Virtual Functions
This guide explains how to create virtual machines with GPU virtual functions and configure GPU access in the guest.
Prerequisites
Complete Installing the Virtualization Stack and Preparing Cloud-init Configuration before continuing. Use the virtual function PCI addresses identified in Enabling SR-IOV Virtual Functions wherever <VF_PCI_ADDRESS> appears below.
Creating and configuring virtual machines with GPU virtual functions
Create and launch the virtual machine with
virt-installusing the following parameters:--name: The virtual machine name.--ram: The amount of memory.--vcpus: The number of virtual CPUs.--cpu host-passthrough: Host CPU features exposed to the guest.--disk(qcow2): The primary disk created from the cloud image.--disk(seed.iso): The cloud-init seed ISO attached as a CD-ROM.--import: Boot from the existing disk image instead of running an installer.--host-device <VF_PCI_ADDRESS>: The GPU virtual function passed through to the virtual machine.--boot uefi: UEFI boot mode for the guest.--noautoconsole: No automatic console attachment after creation.
sudo virt-install \ --name ubuntu-vm1 \ --ram 32768 --vcpus 8 \ --cpu host-passthrough \ --os-variant ubuntu24.04 \ --disk path=/var/lib/libvirt/images/ubuntu-vm1.qcow2,format=qcow2,bus=virtio \ --disk path=/var/lib/libvirt/images/vm1-seed.iso,device=cdrom,bus=sata \ --import \ --network network=default,model=virtio \ --graphics none --video none \ --console pty,target.type=serial \ --host-device <VF_PCI_ADDRESS> \ --boot uefi \ --noautoconsole
To create additional virtual machines, repeat the preparation steps starting from disk creation in Installing the Virtualization Stack and Preparing Cloud-init Configuration. For each additional virtual machine, use a unique name, hostname,
instance-id, disk path, seed ISO, and virtual function address.Wait for the virtual machine to boot and cloud-init to complete, which can take a minute or two. Then find the virtual machine’s IP address.
sudo virsh domifaddr ubuntu-vm1
If no address is listed, query the host ARP table instead with
sudo virsh domifaddr ubuntu-vm1 --source arp.Connect to the serial console of the running virtual machine, or connect over SSH if you know its IP address.
# Connect to the serial console sudo virsh console ubuntu-vm1 # Connect over SSH ssh vmadmin@<VM_IP_ADDRESS>
Inside the virtual machine, confirm that the passed-through GPU virtual function is visible.
lspci | grep -iE 'vga|display'
Inside the virtual machine, follow the requirements and installation guidance linked from Preparing the System for SR-IOV to install the GPU software packages and the
xpu-smitool.The guest also needs kernel version 6.17 or later to support Intel® GPU virtual functions. The Ubuntu 24.04 cloud image ships with kernel 6.8, on which the virtual function is visible in
lspcibut no GPU driver is loaded. If the GPU software installation does not upgrade the kernel, install the Ubuntu hardware enablement (HWE) kernel and reboot the virtual machine:sudo apt update && sudo apt install -y linux-generic-hwe-24.04 sudo reboot
After the virtual machine reboots, verify that the GPU is available in the guest, operates in VF mode, and has the expected VRAM allocation.
sudo xpu-smi discovery -d 0 sudo dmesg | grep -iE 'SR-IOV|VRAM\[0\]'
The kernel log should contain a
Running in SR-IOV VF modemessage.Ensure that every user who runs GPU workloads in the virtual machine is a member of the
rendergroup, and verify GPU access withoutsudo. The GPU device nodes in/dev/dri/are accessible only to therenderandvideogroups, so users outside these groups cannot use the GPU, andxpu-smifails withSysman driver initialization failed. Theuser-datafile prepared in Installing the Virtualization Stack and Preparing Cloud-init Configuration addsvmadminto both groups, but cloud-init applies these group memberships only when it creates the user. If the user already exists in the image, for example in an image created with the Ubuntu Server installer, add the user to the groups manually and log in again:sudo usermod -aG render,video $USER
id xpu-smi discovery -d 0
The output of
idshould include therendergroup, andxpu-smishould list the GPU. Runningxpu-smiwithoutsudocan also print aDMI reader failedmessage, which does not affect the device discovery.If you plan to use Docker in the virtual machine to run vLLM, SGLang, and similar software, install Docker Engine in the virtual machine. Pass the GPU to containers with
--device /dev/dri.