Integration with Red Hat OpenShift AI

Red Hat OpenShift AI (RHOAI) can leverage Intel GPUs for inference and training workloads. This document covers the full installation from dependencies through workload deployment.

Validated test configuration

This guide was validated on a Single-Node OpenShift (SNO) environment on 24 September 2026:

Component

Version

OpenShift Container Platform

4.22.14

Kubernetes

1.35.6

RHEL CoreOS kernel

5.14.0-687.46.1.el9_8.x86_64

GPU enablement

Intel GPU Base Operator 0.4.0, verified in both DRA and device plugin mode, and Intel Device Plugins Operator 0.36.0

Node Feature Discovery Operator

4.22.0

cert-manager Operator for Red Hat OpenShift

1.20.0

LVM Storage Operator

4.22.0, providing the default dynamic StorageClass

Red Hat build of Kueue

1.4.2

RHOAI

3.5.1, stable-3.x channel

GPUs

Eight Intel® Arc™ Pro B70 (Battlemage G31, 8086:e223), 32656 MiB each

Test image

intel/vllm:0.21.0-xpu, PyTorch 2.11.0+xpu

For production multi-node clusters, adapt the following:

  • Use dedicated unformatted disks for LVM Storage or a CSI-based StorageClass.

  • Target GPU worker nodes rather than all workers.

  • Expect RHOAI dashboard and model controllers to schedule multiple replicas across different nodes.

  • Ensure etcd and control-plane quorum, because SNO has no redundancy.

Dependencies

Install the following components in this order before deploying GPU-accelerated workloads on OpenShift AI:

  1. Intel GPU Resource Driver (DRA) or the Intel GPU device plugin: Exposes GPU hardware to the Kubernetes scheduler. DRA is the preferred option for new deployments.

  2. cert-manager Operator for Red Hat OpenShift: Required by the kserve component. Install openshift-cert-manager-operator from the stable-v1 channel. If you already installed it for the Intel GPU Base Operator, there is nothing more to do.

  3. Storage: RHOAI expects a default StorageClass with dynamic provisioning.

  4. OpenShift AI Operator: Installs the RHOAI platform itself.

Optionally, install the Red Hat build of Kueue Operator if you want queue-based scheduling for distributed workloads. It is required by the kueue, ray, and trainingoperator components.

apiVersion: v1
kind: Namespace
metadata:
  name: openshift-kueue-operator
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-kueue-operator
  namespace: openshift-kueue-operator
spec:
  upgradeStrategy: Default
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: kueue-operator
  namespace: openshift-kueue-operator
spec:
  channel: stable-v1.4
  name: kueue-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  installPlanApproval: Automatic

Architecture notes

Keep the following in mind when you plan the deployment:

  • Model serving runs on KServe in RawDeployment mode. Neither OpenShift Serverless nor OpenShift Service Mesh is required, and you do not create a KnativeServing or ServiceMeshControlPlane resource.

  • The dashboard is published through a Gateway API Gateway named data-science-gateway and is reachable at https://rh-ai.apps.<cluster-domain>.

  • DSCInitialization and DataScienceCluster both use the v2 API. The operator creates DSCInitialization for you; you only create the DataScienceCluster.

  • Kueue is supplied by the Red Hat build of Kueue Operator and driven with kueue.managementState: Unmanaged. RHOAI does not embed its own copy.

  • Disable the GPU Base Operator’s bundled Kueue when installing RHOAI. Installing two copies of Kueue leaves the kueue.x-k8s.io custom resource definitions in a conflicting storage-version state. See the “Kueue conflicts with OpenShift AI” section of Intel GPU Base Operator.

  • If your cluster already has a default StorageClass, for example from an external CSI driver or cloud provider, you can skip the storage chapter. Use oc get storageclass to confirm.

  • OpenShift Service Mesh 3.x is only needed for the ogx component, which powers RAG and Llama Stack workloads. GPU-accelerated model serving does not need it.

The remaining procedures and reference sections are split into the following child chapters: