PyTorch Profiler

PyTorch Profiler is PyTorch’s built-in tool for collecting performance traces during model training and inference. On Intel GPUs, it uses Kineto to collect CPU and XPU activity, including host-side operator activity and device-side kernel execution.

This section covers profiling workflows for native PyTorch, Hugging Face Transformers and Accelerate, vLLM, and SGLang on Intel GPUs. Complete the following common setup before following the workflow guide for your framework.

Profiling outputs

Profiler traces show host-side operator activity alongside device-side XPU execution. The activities and additional details available in a trace depend on the selected collection settings. A torch.profiler session on XPU can collect:

  • CPU-side operator activity, including work that launches and coordinates XPU operations

  • Device-side XPU kernel activity

  • Self and total time summaries for operators and kernels

  • Optional tensor shapes, memory allocation and deallocation events, and Python or C++ stack traces

  • Operator summary tables, for example from key_averages()

  • Chrome Trace JSON files for timeline inspection in Perfetto

Options that collect additional detail can increase overhead and produce larger trace files. Treat profiler results as diagnostic data for locating and investigating bottlenecks, not as benchmark measurements.

Prerequisites

Before collecting an XPU profile, prepare the software environment and plan the capture.

Software requirements

Make sure you have the software and setup required for XPU profiling:

  • A working PyTorch build with Intel XPU support. For setup instructions, see Getting Started on Intel GPU. Verify that XPU is available with:

    import torch
    print(torch.xpu.is_available())
    
  • Profiling Tools Interface (PTI) is Intel’s low-level interface for collecting GPU runtime and device activity. The compatible intel-pti distribution is installed alongside the PyTorch XPU (torch-xpu) stack; PTI is not bundled with torch. In the documented pip/venv setup, the PTI runtime library is located at $VIRTUAL_ENV/lib/libpti_view.so. This path is specific to that environment, so verify the library location in your own setup. Kineto uses PTI to collect XPU activity. For XCCL host-call collection, see Distributed collective traces. You do not need a separate PTI source checkout.

Optional multi-device selection

To restrict which GPU devices are visible to the profiled process, set ZE_AFFINITY_MASK. If you leave it unset, all devices and sub-devices are visible. For example, set it to isolate a workload:

export ZE_AFFINITY_MASK=0      # single device
export ZE_AFFINITY_MASK=0,1    # multiple devices

Device ordinals follow driver enumeration order: 0 identifies the first device, then 1, and so on. Do not treat them as fixed hardware IDs. For sub-device syntax and complete behavior, see the Level Zero Affinity Mask specification.

Capture preparation

Before capturing a representative steady-state trace:

  • Warm up the model or workload outside the capture window, or use scheduled warm-up steps. First iterations include allocation, JIT/graph compilation, and cache-warming costs that can dominate the result.

  • Bound the capture duration or step count. Unbounded captures produce very large trace files.

  • Ensure adequate free disk space for trace files, especially with with_stack=True or multi-rank distributed runs.

  • Remember that profiling adds overhead. Enabling record_shapes, profile_memory, or with_stack increases that overhead and can distort timings. Use profiler output to identify and investigate performance issues, rather than to produce benchmark measurements.

Supported workloads

This guide covers the following frameworks and workflows. XPU support and configuration vary by framework and version. Other PyTorch-based frameworks may also work with XPU Kineto profiling, but are outside the scope of this guide.

PyTorch

PyTorch workloads use the torch.profiler API directly to collect CPU and XPU activity during training or inference. The PyTorch Profiling guide shows examples of minimal and scheduled captures. For additional PyTorch profiling and setup information, see:

Hugging Face Transformers and Accelerate

You can profile transformers workloads by wrapping model execution with torch.profiler. As Trainer does not expose profiler controls, it requires using manual instrumentation or Accelerate’s ProfileKwargs and Accelerator.profile() integration. For Transformers and Accelerate examples, see the Hugging Face Profiling guide. For framework-level profiling integration, see the Accelerate Profiler guide.

vLLM

vLLM exposes torch.profiler through server configuration and manual or benchmark-driven start/stop controls. It produces operator summaries and trace files for serving workloads. For server-side and benchmark-driven profiling examples, see the vLLM Profiling guide. For XPU setup and profiler configuration, see:

SGLang

SGLang exposes torch.profiler through server and benchmark profiling controls and writes trace files for timeline analysis. For server-side and benchmark-driven profiling examples, see the SGLang Profiling guide. For XPU setup and profiling options, see:

Advanced profiling

For device hardware performance metrics and deeper kernel analysis, see Scope Profiler and Hardware Metrics. For distributed XCCL host-call collection, see Distributed Collective Traces. For common collection problems, see Troubleshooting.

References