Intel® VTune™ Profiler

Intel® VTune™ Profiler collects GPU hotspot data to help analyze workload performance. Follow these procedures to set up the profiling environment and collect a GPU hotspots report.

This guide uses vLLM with the meta-llama/Llama-3.1-8B-Instruct model on an Intel® Arc™ Pro GPU as an example.

Set up the profiling container

Follow this procedure to prepare the container environment before you start profiling. This procedure selects the vLLM image, starts the container with GPU access, and installs Intel VTune Profiler.

  1. List the available vLLM XPU image tags on Docker Hub.

    curl -sS "https://hub.docker.com/v2/repositories/vllm/vllm-openai-xpu/tags/?page_size=100&ordering=last_updated" \
    | python3 -c "import sys, json; [print(t['name'], t['last_updated']) for t in json.load(sys.stdin)['results']]"
    
  2. Select the most recent release tag available in the repository, assign it to VLLM_VERSION, and pull the selected vLLM XPU container image for that tag. This example uses vLLM v0.28.0. Keep the variable defined in the same shell so it is available when you start the container.

    VLLM_VERSION=v0.28.0
    docker pull vllm/vllm-openai-xpu:${VLLM_VERSION}
    
  3. Gather information about the host GPUs. If the host has multiple GPUs, review the card numbers, memory usage, running processes, and PCI bus/device/function (BDF) addresses. Choose a card with sufficient available memory that is not running another workload.

    xpu-smi
    xpu-smi discovery
    
    Example output

    In this example, cards 0, 1, and 2 have low memory usage, while card 3 is already using more memory. Therefore, cards 0, 1, or 2 are suitable for profiling.

    +----------------------------------------------+----------------------------+------------------------+
    |            Intel XPU-SMI v2.0    Driver: 85B7CA089405934276CBAD3    Level Zero: 1.28.6             |
    +----------------------------------------------+----------------------------+------------------------+
    | GPU  Name                    Persistence-M   | Bus-Id            Disp.A   |   Volatile Uncorr. ECC |
    | Fan              Temp  Pwr:Usage/Cap         | Memory-Usage               |   GPU-Util  Compute M. |
    +----------------------------------------------+----------------------------+------------------------+
    |   0  Intel(R) Arc(TM) Pro B  Off             | 0000:17:00.0      Off      |               Disabled |
    |      70 Graphics                             |                            |                        |
    | N/A               N/A  45W / 230W            | 42MiB / 32656MiB           |      N/A       Default |
    +----------------------------------------------+----------------------------+------------------------+
    |   1  Intel(R) Arc(TM) Pro B  Off             | 0000:4e:00.0      Off      |               Disabled |
    |      70 Graphics                             |                            |                        |
    | N/A               N/A  45W / 230W            | 42MiB / 32656MiB           |      N/A       Default |
    +----------------------------------------------+----------------------------+------------------------+
    |   2  Intel(R) Arc(TM) Pro B  Off             | 0000:97:00.0      Off      |               Disabled |
    |      70 Graphics                             |                            |                        |
    | N/A               N/A  46W / 230W            | 42MiB / 32656MiB           |      N/A       Default |
    +----------------------------------------------+----------------------------+------------------------+
    |   3  Intel(R) Arc(TM) Pro B  Off             | 0000:ba:00.0      Off      |               Disabled |
    |      70 Graphics                             |                            |                        |
    | N/A               N/A  189W / 230W           | 4854MiB / 32656MiB         |      N/A       Default |
    +----------------------------------------------+----------------------------+------------------------+
    
    +-------+-------+--------+----------------+----------------------------------------------------------+
    | Processes:                                                                                         |
    |   GPU     PID    Type    Process Name                                             GPU Memory Usage |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    |                                  No running GPU processes found.                                   |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    
                                                                    
    Device ID     Device Information                                
    0             Device Name: Intel(R) Arc(TM) Pro B70 Graphics    
                    Device State: normal                              
                    Vendor Name: Intel(R) Corporation                 
                    SOC UUID: 00000000-0000-0017-0000-0000e2238086    
                    PCI BDF Address: 0000:17:00.0                     
                    DRM Device: /dev/dri/card1                        
                    Function Type: physical                           
                                                                    
    1             Device Name: Intel(R) Arc(TM) Pro B70 Graphics    
                    Device State: normal                              
                    Vendor Name: Intel(R) Corporation                 
                    SOC UUID: 00000000-0000-004e-0000-0000e2238086    
                    PCI BDF Address: 0000:4e:00.0                     
                    DRM Device: /dev/dri/card2                        
                    Function Type: physical                           
                                                                    
    2             Device Name: Intel(R) Arc(TM) Pro B70 Graphics    
                    Device State: normal                              
                    Vendor Name: Intel(R) Corporation                 
                    SOC UUID: 00000000-0000-0097-0000-0000e2238086    
                    PCI BDF Address: 0000:97:00.0                     
                    DRM Device: /dev/dri/card3                        
                    Function Type: physical                           
                                                                    
    3             Device Name: Intel(R) Arc(TM) Pro B70 Graphics    
                    Device State: normal                              
                    Vendor Name: Intel(R) Corporation                 
                    SOC UUID: 00000000-0000-00ba-0000-0000e2238086    
                    PCI BDF Address: 0000:ba:00.0                     
                    DRM Device: /dev/dri/card4                        
                    Function Type: physical                           
                                                                    
                                                                                    
    Metric                       Value                                               
    Device ID                    2                                                   
    PCI BDF                      0000:97:00.0                                        
    Device Type                  discrete                                            
    Start Time                   2026-08-28T12:16:43.244Z                            
    End Time                     2026-08-28T12:16:43.447Z                            
    Elapsed Time (seconds)       0.20                                                
    Energy Consumed (J)          9.23                                                
    GPU Utilization (%)          N/A                                                 
    Compute Engines Util (%)     N/A                                                 
    Render Engines Util (%)      N/A                                                 
    Media Engines Util (%)       N/A                                                 
    Copy Engines Util (%)        N/A                                                 
    GPU Power (W)                Tile 0: avg: 46, min: 46, max: 46, current: 46      
    GPU Frequency (MHz)          Tile 0: avg: 400, min: 400, max: 400, current: 400  
    Media Frequency (MHz)        Tile 0: avg: 400, min: 400, max: 400, current: 400  
    GPU Core Temperature         N/A                                                 
    (Degrees Celsius)                                                                
    GPU Memory Temperature       N/A                                                 
    (Degrees Celsius)                                                                
    GPU VR Temperature           N/A                                                 
    (Degrees Celsius)                                                                
    Fan Speed (%)                N/A                                                 
    GPU Memory Read (kB/s)       N/A                                                 
    GPU Memory Write (kB/s)      N/A                                                 
    GPU Memory Bandwidth (%)     N/A                                                 
    GPU Memory Used (MiB)        Tile 0: avg: 42, min: 42, max: 42, current: 42      
    GPU Memory Util (%)          Tile 0: avg: 0, min: 0, max: 0, current: 0          
    PCIe Read (kB/s)             N/A                                                 
    PCIe Write (kB/s)            N/A                                                 
    Compute Engine Util (%)      N/A                                                 
    Render Engine Util (%)       N/A                                                 
    Decoder Engine Util (%)      N/A                                                 
    Encoder Engine Util (%)      N/A                                                 
    Copy Engine Util (%)         N/A                                                 
    Media EM Engine Util (%)     N/A                                                 
    
  4. Select a host GPU for profiling. Assign the selected card number to GPU_CARD, then use it to query the card and select the GPU for the container. This example selects card 2.

    GPU_CARD=2
    xpu-smi stats -d "${GPU_CARD}"
    
  5. Start the vLLM XPU container with GPU access and profiling permissions. The GPU_CARD variable specifies which host GPU the container uses.

    docker run --rm -it \
    --network=host \
    --device /dev/dri:/dev/dri \
    -v /dev/dri/by-path:/dev/dri/by-path \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=$HF_TOKEN" \
    --ipc=host \
    --privileged \
    --pid=host \
    --cap-add=SYS_ADMIN \
    --cap-add=SYS_PTRACE \
    -e ZE_AFFINITY_MASK="${GPU_CARD}" \
    -e http_proxy="$http_proxy" \
    -e https_proxy="$https_proxy" \
    -e no_proxy="$no_proxy" \
    -e HTTP_PROXY="$HTTP_PROXY" \
    -e HTTPS_PROXY="$HTTPS_PROXY" \
    -e NO_PROXY="$NO_PROXY" \
    --entrypoint /bin/bash \
    vllm/vllm-openai-xpu:${VLLM_VERSION}
    
  6. Confirm that the selected GPU is visible inside the container. Check that the displayed PCI BDF address matches the host card selected earlier. The device number may change inside the container, but the PCI BDF address identifies the same physical GPU.

    xpu-smi
    xpu-smi discovery
    xpu-smi stats -d 0
    
    Example output
    +----------------------------------------------+----------------------------+------------------------+
    |            Intel XPU-SMI v2.1    Driver: 85B7CA089405934276CBAD3    Level Zero: 1.32.0             |
    +----------------------------------------------+----------------------------+------------------------+
    | GPU  Name                    Persistence-M   | Bus-Id            Disp.A   |   Volatile Uncorr. ECC |
    | Fan              Temp  Pwr:Usage/Cap         | Memory-Usage               |   GPU-Util  Compute M. |
    +----------------------------------------------+----------------------------+------------------------+
    |   0  Intel(R) Arc(TM) Pro B  Off             | 0000:97:00.0      Off      |               Disabled |
    |      70 Graphics                             |                            |                        |
    | 0%                N/A  50W / 230W            | 42MiB / 32656MiB           |      0%        Default |
    +----------------------------------------------+----------------------------+------------------------+
    
    +-------+-------+--------+----------------+----------------------------------------------------------+
    | Processes:                                                                                         |
    |   GPU     PID    Type    Process Name                                             GPU Memory Usage |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    |                                  No running GPU processes found.                                   |
    +-------+-------+--------+----------------+----------------------------------------------------------+
    
                                                                    
    Device ID     Device Information                                
    0             Device Name: Intel(R) Arc(TM) Pro B70 Graphics    
                    Device State: normal                              
                    Vendor Name: Intel(R) Corporation                 
                    SOC UUID: 00000000-0000-0097-0000-0000e2238086    
                    PCI BDF Address: 0000:97:00.0                     
                    DRM Device: /dev/dri/card3                        
                    Function Type: physical                           
                                                                    
    1             Device State: survivability mode                  
                    Recovery Action: Firmware update and GPU reset    
                    PCI BDF Address: 0000:4e:00.0                     
                                                                    
    2             Device State: survivability mode                  
                    Recovery Action: Firmware update and GPU reset    
                    PCI BDF Address: 0000:17:00.0                     
                                                                    
    3             Device State: survivability mode                  
                    Recovery Action: Firmware update and GPU reset    
                    PCI BDF Address: 0000:ba:00.0                     
                                                                    
                                                                                        
    Metric                       Value                                                 
    Device ID                    0                                                     
    PCI BDF                      0000:97:00.0                                          
    Device Type                  discrete                                              
    Start Time                   2026-08-28T12:16:48.308Z                              
    End Time                     2026-08-28T12:16:48.509Z                              
    Elapsed Time (seconds)       0.20                                                  
    Energy Consumed (J)          9.99                                                  
    GPU Utilization (%)          Tile 0: 0                                             
    Compute Engines Util (%)     Tile 0: 0                                             
    Render Engines Util (%)      Tile 0: 0                                             
    Media Engines Util (%)       Tile 0: 0                                             
    Copy Engines Util (%)        Tile 0: 0                                             
    GPU Power (W)                Tile 0: avg: 50, min: 50, max: 50, current: 50        
    GPU Frequency (MHz)          Tile 0: avg: 517, min: 517, max: 517, current: 517    
    Media Frequency (MHz)        Tile 0: avg: 400, min: 400, max: 400, current: 400    
    Memory Frequency (MHz)       N/A                                                   
    Memory Voltage (V)           N/A                                                   
    GPU Core Temperature         N/A                                                   
    (Degrees Celsius)                                                                  
    GPU Memory Temperature       N/A                                                   
    (Degrees Celsius)                                                                  
    GPU VR Temperature           N/A                                                   
    (Degrees Celsius)                                                                  
    Fan Speed (%)                Fan 0: 0%                                             
    GPU Memory Read (kB/s)       N/A                                                   
    GPU Memory Write (kB/s)      N/A                                                   
    GPU Memory Bandwidth (%)     N/A                                                   
    GPU Memory Used (MiB)        Tile 0: avg: 42, min: 42, max: 42, current: 42        
    GPU Memory Util (%)          Tile 0: avg: 0, min: 0, max: 0, current: 0            
    PCIe Read (kB/s)             N/A                                                   
    PCIe Write (kB/s)            N/A                                                   
    Compute Engine Util (%)      Engine 0: 0.0                                         
    Render Engine Util (%)       Engine 0: 0.0                                         
    Decoder Engine Util (%)      Engine 0: 0.0, Engine 1: 0.0                          
    Encoder Engine Util (%)      Engine 0: 0.0, Engine 1: 0.0                          
    Copy Engine Util (%)         Engine 0: 0.0                                         
    Media EM Engine Util (%)     Engine 0: 0.0, Engine 1: 0.0                          
    
  7. Install the required tools before installing Intel VTune Profiler.

    apt-get update
    apt-get install -y gnupg gpg-agent wget kmod pkg-config
    
  8. Add the Intel oneAPI APT repository to the container.

    cd /tmp
    wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
    apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
    rm GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
    echo "deb https://apt.repos.intel.com/oneapi all main" | tee /etc/apt/sources.list.d/oneAPI.list
    apt update
    
  9. Install Intel VTune Profiler in the container.

    apt install -y intel-oneapi-vtune
    
  10. Load the Intel VTune Profiler environment and confirm the installed version.

    source /opt/intel/oneapi/vtune/latest/env/vars.sh
    vtune --version
    
  11. Run the Intel VTune Profiler self-checker.

    mkdir -p /workspace/vtune-selfcheck
    vtune-self-checker.sh --log-dir /workspace/vtune-selfcheck 2>&1 | tee /workspace/selfcheck.log
    grep -E "The system is ready to be used for performance analysis|GPU HW event-based analysis" /workspace/selfcheck.log
    

Collect a GPU hotspots report

Follow this procedure to run the workload under Intel VTune Profiler when the container is ready. The server starts with collection paused so that model loading and server initialization do not fill the result with startup activity. The warmup benchmark then prepares the workload, and only the following benchmark run is measured.

  1. Start the vLLM server under Intel VTune Profiler with data collection paused. The command derives VTune’s target GPU value from the selected device’s PCI BDF address.

    BDF=$(xpu-smi stats -d 0 | awk '/PCI BDF/ {print $3}')
    TARGET_GPU=$(python3 -c "
    import sys
    d, b, rest = sys.argv[1].split(':')
    dev, fn = rest.split('.')
    print(f'{int(d,16)}:{int(b,16)}:{int(dev,16)}.{int(fn,16)}')" "$BDF")
    echo "BDF=$BDF TARGET_GPU=$TARGET_GPU"
    vtune -collect gpu-hotspots -start-paused \
    -knob target-gpu=$TARGET_GPU \
    -result-dir /workspace/profiling \
    -- /opt/venv/bin/python3 /opt/venv/bin/vllm serve meta-llama/Llama-3.1-8B-Instruct \
    --tensor-parallel-size 1 \
    --max-model-len 10240 \
    --max-num-batched-tokens 10240 \
    --enforce-eager \
    --no-enable-prefix-caching > /workspace/serve.log 2>&1 &
    VTUNE_PID=$!
    echo "VTUNE_PID=$VTUNE_PID"
    
  2. Wait until the vLLM server reports a healthy status before sending requests or starting the benchmark.

    until curl -sf http://localhost:8000/health > /dev/null; do sleep 5; done
    echo "server ready"
    
  3. Send a test request to the vLLM completions endpoint to confirm that the server can accept an inference request before you begin the benchmark.

    curl http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "meta-llama/Llama-3.1-8B-Instruct",
        "prompt": "San Francisco is a",
        "max_tokens": 7,
        "temperature": 0
    }'
    
  4. Run a warmup benchmark before collecting profiling data. This separates model and runtime warmup from the measured run and gives the server the same request pattern that will be profiled.

    vllm bench serve \
    --model meta-llama/Llama-3.1-8B-Instruct \
    --dataset-name sonnet \
    --dataset-path /workspace/vllm/benchmarks/sonnet.txt \
    --sonnet-prefix-len 100 \
    --sonnet-input-len 2048 \
    --sonnet-output-len 64 \
    --ignore-eos \
    --num-prompts 8 \
    --max-concurrency 8 > /workspace/warmup.log 2>&1
    tail -5 /workspace/warmup.log
    
  5. Resume data collection, run the benchmark, and pause data collection. Keep collection limited to the benchmark interval so the report represents the workload rather than server startup or unrelated activity.

    vtune -command resume -r /workspace/profiling
    vllm bench serve \
    --model meta-llama/Llama-3.1-8B-Instruct \
    --dataset-name sonnet \
    --dataset-path /workspace/vllm/benchmarks/sonnet.txt \
    --sonnet-prefix-len 100 \
    --sonnet-input-len 2048 \
    --sonnet-output-len 64 \
    --ignore-eos \
    --num-prompts 8 \
    --max-concurrency 8 > /workspace/bench.log 2>&1
    vtune -command pause -r /workspace/profiling
    
  6. Print the benchmark results from the benchmark log.

    sed -n '/Serving Benchmark Result/,/^=\+$/p' /workspace/bench.log
    
    Example output
    ============ Serving Benchmark Result ============
    Successful requests:                     8         
    Failed requests:                         0         
    Maximum request concurrency:             8         
    Benchmark duration (s):                  4.67      
    Total input tokens:                      14854     
    Total generated tokens:                  512       
    Request throughput (req/s):              1.71      
    Output token throughput (tok/s):         109.66    
    Peak output token throughput (tok/s):    184.00    
    Peak concurrent requests:                8.00      
    Total token throughput (tok/s):          3290.96   
    ---------------Time to First Token----------------
    Mean TTFT (ms):                          1905.71   
    Median TTFT (ms):                        1944.62   
    P99 TTFT (ms):                           1986.02   
    -----Time per Output Token (excl. 1st token)------
    Mean TPOT (ms):                          43.75     
    Median TPOT (ms):                        43.23     
    P99 TPOT (ms):                           48.40     
    ---------------Inter-token Latency----------------
    Mean ITL (ms):                           43.93     
    Median ITL (ms):                         43.07     
    P99 ITL (ms):                            51.13     
    
  7. Stop the vLLM engine core so Intel VTune Profiler can finalize the result. Wait for the VTune process to exit before generating the report.

    RUNSS_PID=$(pgrep -P $VTUNE_PID)
    SERVE_PID=$(pgrep -P $RUNSS_PID)
    ENGINE_PID=$(pgrep -P $SERVE_PID -f VLLM::EngineCore)
    kill $ENGINE_PID
    wait $VTUNE_PID
    
  8. Generate and print the GPU hotspots report.

    vtune -report hotspots -group-by computing-task -format csv -csv-delimiter=comma -sort-desc "Computing Task:Total Time:Self" -column="Work Size:Global:Self","Computing Task:Instance Count:Self","Computing Task:Total Time:Self","XVE Array:Active:Self","XVE Array:Stalled:Self","XVE Threads Occupancy:Self","XVE Stall Reasons:SBID:Self","GPU Memory Bandwidth" -result-dir /workspace/profiling/ -limit 10 | python3 -c "import sys, csv; r = csv.reader(sys.stdin); h = next(r); print(f'%-26.26s | %-16s | %-25s | %-29s | %-19s | %-20s | %-24s | %-25s | %-38s' % (h[0], h[1], h[2], h[3], h[4], h[5], h[6], h[7], h[8])); [print(f'%-26.26s | %-16s | %-25s | %-29s | %-19s | %-20s | %-24s | %-25s | %-38s' % (row[0], row[1], row[2], row[3], row[4], row[5], row[6], row[7], row[8])) for row in r if len(row) >= 9]"
    
    Example output
    Computing Task             | Work Size:Global | Computing Task:Total Time | Computing Task:Instance Count | XVE Array:Active(%) | XVE Array:Stalled(%) | XVE Threads Occupancy(%) | XVE Stall Reasons:SBID(%) | GPU Memory Bandwidth, GB/sec:Read     
    gemm_kernel                | 1024 x 2 x 8     | 1.633694                  | 8130                          | 5.2                 | 55.2                 | 59.1                     | 50.2                      | 404.170718                            
    gemm_kernel                | 2048 x 8         | 1.099761                  | 128                           | 49.8                | 49.6                 | 98.8                     | 5.0                       | 167.311040                            
    KernelFunctor<&(void cutla | 64 x 1 x 64      | 0.318105                  | 1984                          | 5.5                 | 58.0                 | 62.9                     | 52.6                      | 394.118999                            
    gemm_kernel                | 2048 x 8         | 0.254566                  | 64                            | 48.8                | 50.2                 | 98.0                     | 7.0                       | 202.714432                            
    gemm_kernel                | 4096 x 4         | 0.155946                  | 96                            | 47.5                | 48.0                 | 94.0                     | 8.7                       | 174.383451                            
    KernelFunctor<&(void cutla | 256 x 8 x 224    | 0.080996                  | 32                            | 57.0                | 40.2                 | 95.3                     | 37.7                      | 88.868125                             
    gemm_kernel                | 4096 x 4         | 0.062237                  | 64                            | 42.2                | 48.2                 | 85.9                     | 15.0                      | 194.418035                            
    act_and_mul_vec_kernel<syc | 10485760         | 0.052750                  | 32                            | 36.5                | 61.0                 | 84.5                     | 53.8                      | 299.637083                            
    fused_add_rms_norm_kernel< | 2621440          | 0.038600                  | 64                            | 40.1                | 57.3                 | 94.6                     | 26.5                      | 190.090231                            
    KernelFunctor<&(void cutla | 256 x 8 x 256    | 0.028913                  | 32                            | 48.3                | 38.7                 | 85.0                     | 27.9                      | 104.010733                            
    PASS: Step 18: hotspots report header carries all requested columns
    PASS: Step 18: hotspots report has 10 data rows
    
    =================== Preserve the VTune result directory ===================
    source: 2745 564696923 (files bytes)
    copy:   2745 564696923 (files bytes)
    PASS: Artifacts: VTune result copied intact — 2745 564696923 (files bytes)
    
    =================== SUMMARY ===================
    image            : vllm/vllm-openai-xpu:v0.28.0
    model            : meta-llama/Llama-3.1-8B-Instruct
    card             : 2 (0000:97:00.0), target-gpu=0:151:0.0
    port             : 18000
    container        : vllm-xpu-vtune-card2-p18000-3777355
    artifact dir     : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355
    serve log      : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/serve.log
    self-check log : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/selfcheck.log
    warmup log     : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/warmup.log
    bench log      : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/bench.log
    bench result   : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/bench_result_block.txt
    hotspots       : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/hotspots.txt
    vtune result   : /mnt/data/workspace/neelesh/applications.ai.aice.persona/persona/workflows/tmp/vllm-xpu-adv-profiling-3777355/profiling
    
    --- Serving Benchmark Result (profiled run) ---
    ============ Serving Benchmark Result ============
    Successful requests:                     8         
    Failed requests:                         0         
    Maximum request concurrency:             8         
    Benchmark duration (s):                  4.67      
    Total input tokens:                      14854     
    Total generated tokens:                  512       
    Request throughput (req/s):              1.71      
    Output token throughput (tok/s):         109.66    
    Peak output token throughput (tok/s):    184.00    
    Peak concurrent requests:                8.00      
    Total token throughput (tok/s):          3290.96   
    ---------------Time to First Token----------------
    Mean TTFT (ms):                          1905.71   
    Median TTFT (ms):                        1944.62   
    P99 TTFT (ms):                           1986.02   
    -----Time per Output Token (excl. 1st token)------
    Mean TPOT (ms):                          43.75     
    Median TPOT (ms):                        43.23     
    P99 TPOT (ms):                           48.40     
    ---------------Inter-token Latency----------------
    Mean ITL (ms):                           43.93     
    Median ITL (ms):                         43.07     
    P99 ITL (ms):                            51.13     
    ==================================================
    
    --- GPU hotspots (top 10 by Computing Task Total Time) ---
    Computing Task             | Work Size:Global | Computing Task:Total Time | Computing Task:Instance Count | XVE Array:Active(%) | XVE Array:Stalled(%) | XVE Threads Occupancy(%) | XVE Stall Reasons:SBID(%) | GPU Memory Bandwidth, GB/sec:Read     
    gemm_kernel                | 1024 x 2 x 8     | 1.633694                  | 8130                          | 5.2                 | 55.2                 | 59.1                     | 50.2                      | 404.170718                            
    gemm_kernel                | 2048 x 8         | 1.099761                  | 128                           | 49.8                | 49.6                 | 98.8                     | 5.0                       | 167.311040                            
    KernelFunctor<&(void cutla | 64 x 1 x 64      | 0.318105                  | 1984                          | 5.5                 | 58.0                 | 62.9                     | 52.6                      | 394.118999                            
    gemm_kernel                | 2048 x 8         | 0.254566                  | 64                            | 48.8                | 50.2                 | 98.0                     | 7.0                       | 202.714432                            
    gemm_kernel                | 4096 x 4         | 0.155946                  | 96                            | 47.5                | 48.0                 | 94.0                     | 8.7                       | 174.383451                            
    KernelFunctor<&(void cutla | 256 x 8 x 224    | 0.080996                  | 32                            | 57.0                | 40.2                 | 95.3                     | 37.7                      | 88.868125                             
    gemm_kernel                | 4096 x 4         | 0.062237                  | 64                            | 42.2                | 48.2                 | 85.9                     | 15.0                      | 194.418035                            
    act_and_mul_vec_kernel<syc | 10485760         | 0.052750                  | 32                            | 36.5                | 61.0                 | 84.5                     | 53.8                      | 299.637083                            
    fused_add_rms_norm_kernel< | 2621440          | 0.038600                  | 64                            | 40.1                | 57.3                 | 94.6                     | 26.5                      | 190.090231                            
    KernelFunctor<&(void cutla | 256 x 8 x 256    | 0.028913                  | 32                            | 48.3                | 38.7                 | 85.0                     | 27.9                      | 104.010733                            
    
    PASS: 37   FAIL: 0
    VERDICT: PASS
    

After you complete this procedure, Intel VTune Profiler has captured GPU hotspots data for the profiled vLLM benchmark run. The benchmark output shows serving performance, and the GPU hotspots report shows the GPU activity associated with that run.