Intel GPU Resource Driver (DRA)

The Intel GPU Resource Driver exposes Intel GPUs through the Kubernetes Dynamic Resource Allocation (DRA) framework, and is the preferred way to enable Intel GPUs on new clusters. Instead of advertising GPUs as an opaque extended-resource count, it publishes each device with its own attributes and capacity, which workloads then select with Common Expression Language (CEL) expressions. This lets workloads select GPUs by memory capacity or place a GPU and NIC behind the same PCIe root complex, rather than simply requesting any available GPU.

Warning

The Intel GPU Resource Driver for DRA is beta software and is not recommended for production deployments. Although DRA reached general availability with the v1 API in Kubernetes v1.34, OpenShift support depends on the Kubernetes version included in your RHOCP release.

Use the following procedure to install the Intel GPU Resource Driver and verify that GPU resources are published for workload scheduling.

Requirements

Before you install the Intel GPU Resource Driver, ensure that the following requirements are met:

  • An OpenShift release based on Kubernetes v1.34 or later.

  • A container runtime with CDI support, such as CRI-O v1.23.0 or later, which is included in OpenShift.

  • A functional xe kernel driver on the host.

  • A Helm client at version 4.0 or later.

  • A Node Feature Discovery deployment if you want to restrict the driver to GPU-labeled nodes.

Important

Starting with release 0.12.0, the chart and container image moved out of the per-repository path into the top-level intel namespace on GitHub Container Registry:

Artifact

Old location

New location

Helm chart

oci://ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver-chart

oci://ghcr.io/intel/intel-gpu-resource-driver-chart

Container image

ghcr.io/intel/intel-resource-drivers-for-kubernetes/intel-gpu-resource-driver

ghcr.io/intel/intel-gpu-resource-driver

Release 0.12.0 is published at both locations and is the last release to appear at the old one. Repoint existing deployments before you upgrade past 0.12.0.

Installing and verifying the Intel GPU Resource Driver

The chart carries native OpenShift support. Setting openshift.enabled=true makes it create a dedicated SecurityContextConstraints object and bind it to the driver’s service account, so no manual oc adm policy add-scc-to-user grants and no DaemonSet patching are needed.

  1. Create the driver namespace.

    oc create namespace intel-gpu-resource-driver
    
  2. Install the chart with OpenShift support enabled.

    helm install --namespace intel-gpu-resource-driver \
        intel-gpu-resource-driver \
        oci://ghcr.io/intel/intel-gpu-resource-driver-chart --version 0.12.0 \
        --set openshift.enabled=true \
        --set openshift.version=4.22 \
        --set nodeFeatureRules.enabled=true \
        --set kubeletPlugin.privileged=true \
        --set kubeletPlugin.healthMonitoring.enabled=false
    

    The following Helm values configure the driver for this OpenShift deployment:

    Value

    Purpose

    openshift.enabled=true

    Creates the intel-gpu-resource-driver SCC plus the ClusterRole and binding that let the kubelet-plugin service account use it.

    openshift.version

    Set this to your OpenShift minor version. The chart uses it to decide whether it must create the DeviceClass objects itself; on 4.21 and later the driver creates them at runtime.

    nodeFeatureRules.enabled=true

    Creates NodeFeatureRule objects so the DaemonSet only runs on nodes labeled intel.feature.node.kubernetes.io/gpu=true. This replaces the kustomize overlay that earlier releases required.

    kubeletPlugin.privileged=true

    Required on OpenShift. The kubelet-plugin writes CDI specifications to /etc/cdi on the host, and SELinux denies those writes unless the container runs with the spc_t type that privileged mode selects. It also lets the driver read full hardware details, including GPU memory size, directly from the devices.

    kubeletPlugin.healthMonitoring.enabled=false

    Required when XPU Manager is not deployed to prevent the driver from repeatedly crashing and restarting. See Health monitoring and XPU Manager.

  3. Wait for the kubelet-plugin pod to become ready, then confirm which SCC admitted it. The output should show 1/1 Running and scc=intel-gpu-resource-driver.

    oc get pods -n intel-gpu-resource-driver
    oc get pods -n intel-gpu-resource-driver \
      -o jsonpath='{range .items[*]}{.metadata.name}{" scc="}{.metadata.annotations.openshift\.io/scc}{"\n"}{end}'
    
  4. Verify that the GPU DeviceClass objects exist.

    oc get deviceclasses
    

    The driver publishes two classes:

    NAME                 AGE
    gpu-vfio.intel.com   113s
    gpu.intel.com        113s
    

    Use gpu.intel.com for regular containers. gpu-vfio.intel.com is for PCI passthrough to virtual machines. The chart enables kubeletPlugin.manageBinding.enabled by default, which lets the driver rebind a GPU between the xe and vfio-pci kernel drivers according to the device class a workload requests. Set it to false if you want only GPUs already bound to vfio-pci to be offered for passthrough.

  5. Verify that GPU ResourceSlice objects are published.

    oc get resourceslices
    
  6. Inspect the published per-device attributes and capacity.

    oc get resourceslices -o json | jq '.items[0].spec.devices[0]'
    

    On an Intel® Arc™ Pro B70, each device is reported as follows:

    {
      "name": "0000-1a-00-0-0xe223",
      "attributes": {
        "driver": { "string": "xe" },
        "family": { "string": "Arc Pro B-Series" },
        "health": { "string": "Healthy" },
        "model": { "string": "B70" },
        "pciId": { "string": "0xe223" },
        "resource.kubernetes.io/pciBusID": { "string": "0000:1a:00.0" },
        "resource.kubernetes.io/pcieRoot": { "string": "pci0000:14" },
        "sriov": { "bool": true },
        "subDeviceId": { "string": "0x1701" },
        "subVendorId": { "string": "0x8086" },
        "type": { "string": "gpu" }
      },
      "capacity": {
        "memory": { "value": "32656Mi" },
        "millicores": { "value": "1k" }
      }
    }
    

    The attributes available for CEL selectors are the following:

    Attribute

    Description

    driver

    Kernel driver bound to the device, xe or i915.

    family, model

    Marketing family and model, for example Arc Pro B-Series and B70.

    health

    Device health as reported by GPU monitoring.

    pciId

    PCI device ID, for example 0xe223.

    subVendorId, subDeviceId

    PCI subsystem IDs, added in 0.12.0, useful for telling board variants apart.

    resource.kubernetes.io/pciBusID

    PCI address of the device. Replaces the deprecated pciAddress attribute.

    resource.kubernetes.io/pcieRoot

    PCIe root complex the device sits under. Replaces the deprecated pciRoot attribute. Use it to co-locate GPUs behind the same root for peer-to-peer transfers.

    sriov

    Whether the device is SR-IOV capable.

    The pciAddress and pciRoot attributes are deprecated in favor of the resource.kubernetes.io/ equivalents. They are still published for compatibility, but write new selectors against the namespaced names.

  7. Create a ResourceClaimTemplate and reference it from a pod. The pod-level resourceClaims[].name and the container-level resources.claims[].name must match. The resourceClaimTemplateName points at the template.

    apiVersion: resource.k8s.io/v1
    kind: ResourceClaimTemplate
    metadata:
      name: gpu-claim
    spec:
      spec:
        devices:
          requests:
          - name: gpu
            exactly:
              deviceClassName: gpu.intel.com
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: gpu-workload
    spec:
      restartPolicy: Never
      containers:
      - name: compute
        image: docker.io/intel/vllm:0.21.0-xpu
        args:
        - bash
        - -c
        - |
          ls -la /dev/dri/
          python -c "import torch; print('XPU available:', torch.xpu.is_available()); print('XPU devices:', torch.xpu.device_count())"
        resources:
          claims:
          - name: gpu-resource
      resourceClaims:
      - name: gpu-resource
        resourceClaimTemplateName: gpu-claim
    
  8. Confirm that the claim was allocated and that the container received exactly one GPU.

    oc get resourceclaim
    oc logs gpu-workload
    

    The pod logs should show one GPU available to PyTorch, for example:

    card1
    renderD128
    XPU available: True
    XPU devices: 1
    
  9. Use CEL selectors to request specific GPU characteristics. For example, the following ResourceClaimTemplate requests two GPUs with at least 10 GiB of memory each:

    apiVersion: resource.k8s.io/v1
    kind: ResourceClaimTemplate
    metadata:
      name: multi-gpu-claim
    spec:
      spec:
        devices:
          requests:
          - name: gpu
            exactly:
              deviceClassName: gpu.intel.com
              count: 2
              selectors:
              - cel:
                  expression: device.capacity["gpu.intel.com"].memory.compareTo(quantity("10Gi")) >= 0
    

    A pod using this template reports two devices:

    card3
    card4
    renderD130
    renderD131
    XPU devices: 2
    

    Note

    Memory-based selectors only match when the driver can read the GPU memory size. If the driver runs with health monitoring enabled but no XPU Manager daemon is reachable, capacity.memory is reported as 0, every memory selector fails, and pods stay Pending with the claim in pending state. Running the kubelet-plugin with kubeletPlugin.privileged=true, as in step 2, avoids this because the driver then reads the hardware directly.

After completing this procedure, OpenShift can publish Intel GPU resources through DRA and schedule workloads by using ResourceClaim objects and CEL-based device selection.

Health monitoring and XPU Manager

Since release 0.10.0, the driver gets GPU health and hardware details from the Intel XPU Manager daemon (xpumd) over a gRPC socket rather than from a bundled library. Health monitoring is enabled by default, which makes the driver run with -m and dial that socket, but the chart does not deploy xpumd by default. Without xpumd, the driver cannot connect and exits after about five minutes.

E xpumd-client: error calling WatchDeviceHealth ... dial unix /run/xpumd/intelxpuinfo.sock: connect: no such file or directory
panic: xpumd-client: failed to connect to xpumd within expected time, exiting

The container then restarts and the cycle repeats indefinitely, which on a cluster left running overnight accumulates hundreds of restarts. This failure can go unnoticed because the driver publishes its ResourceSlice objects before it exits: GPUs appear allocatable, and short-lived test pods succeed. Check the restart count rather than the Running status:

oc get pods -n intel-gpu-resource-driver

Installing with kubeletPlugin.healthMonitoring.enabled=false, as in the procedure above, drops the -m flag and the driver stays up. Because kubeletPlugin.privileged=true lets it read the hardware directly, device attributes and capacity are unaffected: memory is still reported as 32656Mi per B70, and the health attribute is still published. What you give up is live health tracking, so a GPU that fails at runtime is not reported as unhealthy and is not tainted.

Warning

Do not try to fix this by deploying the chart’s bundled xpumd with xpumdEnabled=true. That DaemonSet does not start on OpenShift in release 0.12.0: its pod template requests the SYS_ADMIN capability, which the chart’s own SCC does not allow, and even after you grant the privileged SCC the daemon fails to bind its socket with listen unix /run/xpumd/intel_xpu_info.sock: bind: permission denied, because the container does not request the spc_t SELinux type. The xpumd pod template also requires the namespace to carry the resource.kubernetes.io/admin-access: "true" label, or the chart install fails outright.

If you need GPU telemetry and health-based device tainting on OpenShift, use the Intel GPU Base Operator in DRA mode instead. It deploys XPU Manager with the correct SELinux context and wires it to the driver automatically, so health monitoring stays enabled and the driver does not restart.

Uninstalling the Intel GPU Resource Driver

  1. Uninstall the driver chart. Helm removes the SCC, ClusterRole, and binding it created, but does not remove DeviceClass objects created by the driver at runtime.

    helm uninstall --namespace intel-gpu-resource-driver intel-gpu-resource-driver --wait
    
  2. Delete the driver namespace.

    oc delete namespace intel-gpu-resource-driver
    
  3. If you plan to reinstall with a different privilege configuration, delete any stale CDI specification left in /etc/cdi/intel.com-gpu.yaml on the nodes. The uninstall does not remove the file; leaving it behind can cause the new pod to fail with remove /etc/cdi/intel.com-gpu.yaml: permission denied.

    oc debug node/<gpu-node> -- chroot /host rm -f /etc/cdi/intel.com-gpu.yaml