Managing GPU Virtual Machines

Use this guide with Creating and Configuring Virtual Machines with GPU Virtual Functions to configure full-GPU passthrough or manage virtual machine lifecycle operations.

Use the PCI addresses identified in Enabling SR-IOV Virtual Functions for the physical function placeholders in this guide.

Passing through the entire GPU

If a single virtual machine needs the whole GPU, including all of its VRAM, you can pass through the physical function instead of a virtual function. Remove all virtual functions from the GPU, re-enable driver auto-probe, and use the physical function address in --host-device:

echo 0 | sudo tee /sys/bus/pci/devices/<PF_PCI_ADDRESS>/sriov_numvfs
echo 1 | sudo tee /sys/bus/pci/devices/<PF_PCI_ADDRESS>/sriov_drivers_autoprobe

Create the virtual machine without an emulated display device by using --graphics none --video none, as shown in the virt-install command. When the entire GPU is passed through, an emulated display can cause GPU tools such as xpu-smi to hang in the guest.

While the physical function is assigned to a virtual machine, the host cannot use the GPU or create virtual functions on it.

Useful virtual machine lifecycle management commands

Use the following commands to manage existing virtual machines and the host GPU’s virtual functions:

  • List all virtual machines: sudo virsh list --all

  • Start a virtual machine: sudo virsh start ubuntu-vm1

  • Gracefully shut down a virtual machine: sudo virsh shutdown ubuntu-vm1

  • Forcefully stop a virtual machine: sudo virsh destroy ubuntu-vm1

  • Delete a virtual machine definition: sudo virsh undefine ubuntu-vm1 --nvram

  • Remove VFs on a card: echo 0 | sudo tee /sys/bus/pci/devices/<PF_PCI_ADDRESS_1>/sriov_numvfs

  • Return a GPU to the default state with no VFs: echo 0 | sudo tee /sys/bus/pci/devices/<PF_PCI_ADDRESS_1>/sriov_numvfs followed by echo 1 | sudo tee /sys/bus/pci/devices/<PF_PCI_ADDRESS_1>/sriov_drivers_autoprobe