Intel® AI Software Stack Architecture

Intel’s AI software stack is designed as a layered architecture that helps developers move from solution discovery through application development to deployment and optimization on Intel hardware. This document explains how the hardware, system software, middleware, frameworks, and applications fit together.

Architecture overview

AI applications use GPU software components to develop and run workloads. Middleware provides libraries and development tools, while Intel GPU system software provides the host-side foundation and APIs for discovering and executing work on GPU hardware. Together, these layers connect application software to GPU execution.

Intel® Open Middleware Xe (Intel® OMIX) packages libraries and development tools that form the middleware layer, together with selected runtime and management components from the system software layer.

Architecture execution path
AI applications Build and run workloads
AI ecosystem enablement Frameworks, libraries, tools
Middleware Packages and containers
Intel GPU system software Runtime, APIs, drivers
Intel GPU hardware Workstation to datacenter

Recommended user journey

The diagram shows the software layers involved in running an AI application. This journey maps those layers to a developer workflow, from finding a solution and preparing the environment to optimizing and deploying it.

  1. Start with an AI use case.
  2. Find a blueprint or solution in the Intel AI software catalog.
  3. Choose a framework, such as PyTorch, vLLM, or SGLang.
  4. Prepare the environment using the Intel OMIX metapackage or containers.
  5. Prepare the deployment environment using the Docker, Kubernetes, or OpenShift setup guide.
  6. Optimize with Intel tools such as VTune Profiler, PTI, and Unitrace. See Profiling and Instrumentation.
  7. Deploy to production on a supported Intel GPU platform.

Intel AI software catalog

The Intel AI software catalog helps developers, solution architects, IT teams, and AI practitioners find resources for building applications on Intel platforms. It guides solution discovery and planning. Users can start with a goal, such as deploying an LLM or inference service, building an AI assistant, agentic workflow, or RAG solution, implementing computer vision, or deploying multimodal AI. Catalog resources include:

  • Solutions: Production-oriented software offerings and deployment stacks, such as enterprise AI, inference services, agentic AI platforms, and enterprise RAG.

  • Blueprints: Runnable reference implementations and example workloads, such as vLLM deployments, assistants, chatbots, vision applications, and multimodal workflows.

  • Optimizations: Performance replication recipes and tuning guidance for PyTorch, vLLM, TensorFlow, and Xeon.

  • Models: Validated and optimized AI models for Intel platforms.

AI ecosystem enablement

The AI ecosystem enablement includes frameworks, libraries, development technologies, tools, open-source integrations, and containers that support building and deploying AI applications. Intel generally prioritizes upstream enablement over developing proprietary frameworks. Examples include:

  • Frameworks and integrations: PyTorch, vLLM, SGLang, TensorFlow, ONNX Runtime, OpenVINO, and the Hugging Face ecosystem.

  • Agentic frameworks: LangChain and CrewAI.

  • Libraries: oneDNN, oneMKL, and oneCCL.

  • Development technologies: SYCL and Intel® oneAPI.

  • Tools: Intel® VTune™ Profiler, PTI, Unitrace, and Intel® Distribution for GDB.

  • Containers: Intel OMIX, PyTorch, vLLM, and GPU runtime software.

Intel OMIX middleware

Intel OMIX is a software distribution that includes middleware libraries and development tools alongside GPU runtime and components from the system software layer. Its component categories include:

  • AI development components: Intel SYCL toolchain, oneMKL, oneCCL, and oneDPL.

  • GPU runtime components: Intel Compute Runtime, Level Zero runtime, and OpenCL runtime.

  • Profiling components: PTI and GPU performance tooling support.

  • GPU management component: Intel® XPU System Management Interface.

Intel OMIX is distributed as a host-installable metapackage and as prebuilt container images. Development and runtime variants serve different purposes; their contents and supported configurations are release-specific. The installation guide covers host installation, while the container image guide covers container use.

System software

System software provides the low-level services required by frameworks and middleware and is generally invisible to most application developers. Key components and their roles include:

  • Compute Runtime: Device discovery, memory management, and command submission.

  • Level Zero: A low-level GPU execution API used by Intel GPU software components.

  • OpenCL: An industry-standard GPU compute interface.

  • SYCL runtime: SYCL execution support on Intel GPUs.

  • Driver software: Scheduling, memory management, device communication, and GPU execution services.

For deployment-environment setup guidance, see the Docker, Kubernetes, or OpenShift setup guide.

Supported platforms

This architecture is designed to provide a common software experience across Intel GPU generations.

  • Xe2 platforms: Intel® Arc™ Pro B50, B60, B65, and B70. These products are commonly targeted by Intel OMIX, PyTorch XPU, vLLM, and Intel AI software catalog solutions.

  • Datacenter platforms: Future Xe-based AI accelerators, extending the same software architecture to datacenter-class Intel GPUs.

The software stack is designed to provide a consistent experience across workstations, AI PCs, edge deployments, and datacenter deployments. Actual hardware, operating-system, and component combinations depend on the product and release. Use the Intel GPU support matrix and corresponding release notes to confirm a specific configuration. Support for one product or release does not imply support for another.