VTune Profiler

VTune Amplifier XE is the profiler for use on Intel systems. v2015 is now the preferred version.

Supported Backend Nodes

This application is only available on the Xenomorph backend node and is mainly used for debugging and profiling on the Xeon Phi (MIC) accerlerator cards. Please see for how to access the node, compile your code and launch your executables on the Xeon Phi cards.

Set up procedure

To setup the VTune AmplXE profiler/debugger load the following modulefile

module load tools/intel/vtune/amplxe2015

or alternatively use the usual Intel dot file:

source /opt/intel/vtune_amplifier_xe/amplxe-vars.sh

Note that the previous version of VTune (2013) is available via the following modulefile but we no longer support this version and cannot guarantee it will work on the current Phi card linux installation.

# This version may no longer work as we upgrade the OS on the MIC cards
tools/intel/vtune/amplxe2013

Running the application

To start the GUI run:

amplxe-gui

Note, you will get the following error message when VTune starts:

sh: line 1:  2537 Segmentation fault      (core dumped)\
 /opt/intel/ism/bin/intel64/intelremotedialog > /dev/null 2>&1

This can be safely ignored.

Compilation Hints

It is recommended you compile your code with (at least) the following compiler flags when running a VTune analysis:

-g -debug inline-debug-info -O2

For example:

module load compiler/inte/14.0.3

# Native (runs entirely on MIC card)
icc -o matmul-mic-native.exe -mmic -openmp \
    -g -debug inline-debug-info -O2 matmul-mic.c

# Offload (runs partly on CPU, partly on MIC card)
icc -o matmul-mic-offload.exe -openmp \
    -g -debug inline-debug-info -O2 matmul-mic.c

# OpenCL (runs partly on CPU, partly on MIC card)
module load libs/intel/opencl/4.4.0
icpc -o matmul-mic-opencl.exe \
     -g -debug inline-debug-info -O2 -lOpenCL matmul-mic.c

The above examples assume the matmul-mic.c source code has suitable #ifdef ... #endif sections to allow the three forms of compilation. Alternatively you could use separate source files for each type.

Project Properties for MIC Analysis

Use the Project Properties dialogue to indicate you will be profiling a Xeon Phi (MIC) executable. There are now a couple of ways to launch the executable on the Xeon Phi (MIC) cards from within VTune, as described below.

Xeon Phi Host Launch

This is the method you mainly used in VTune 2013 and can be used in version 2015. It can be used to profile:

1. Native applications run entire on the Xeon Phi. You write a shell script to ssh on to the Phi and run your code. Vtune runs the shell script.
2. Offload or OpenCL applications which do some work on the host CPU and some work on the Xeon Phi. The executable will automatically offload the Xeon Phi code as and when required.

Xeon Phi Host Launch – Native Applications

In option 1 (Native applications) we write a shell script to ssh on to the MIC card and run the executable (the script can also set environment variables and pass arguments to your executable – whatever you put in the script). VTune will profile the executable on the MIC card when it runs.

In the Project Properties window set the Target system to be Intel Xeon Phi coprocessor (host launch) as shown below (click on image to enlarge it):

A suitable shell script would look something like:

#!/bin/bash
IROOT=/opt/intel/composerxe
ssh mic0 "export LD_LIBRARY_PATH=$IROOT/compiler/lib/mic:$IROOT/mkl/lib/mic; \
          export KMP_AFFINITY=granularity=thread,balanced; \
          cd $HOME/support/mycode;
          ./matmul-native-mic.exe 2048 244"

In the above script:

• We ssh on to mic0. You should ensure you ssh to the MIC card you have reserved on xenomorph. Check this by running the following on xenomorph:

  echo $OFFLOAD_DEVICES
  

• We must set LD_LIBRARY_PATH as shown. Otherwise you will get an error message about libiomp5.so.
• The KMP_AFFINITY setting is optional – please refer to the Intel documentation.
• When we ssh in to the MIC card we will be in our home directory. So we change to where our executable is.
• We then run our executable, giving it a couple of input parameters (your application will be different).

In the VTune Project Properties window we then set the Application to be this script, as shown below. In this example the script is named run-matmul-native-on-mic0.sh and you will need to use an absolute path – i.e., give the full path to the script (click on image to enlarge it):

Xeon Phi Host Launch – Offload Applications

For Option 2 (Offload or OpenCL applications) we do not need to use a shell script to launch the executable on the MIC cards. Instead, simply specify the full path to the executable in the Application field. This will be executed on the host CPU and the executable will automatically run parts of its code on the MIC card as required.

Analysis Type

Once your job is set up using either of the above methods you’ll need to select the Analysis Type (press the brown Play icon). As of v2015, when you’ve identified an executable to be a Xeon Phi executable as above (see Target System in the above settings) then you only get Analysis types suitable for MIC hardware in the GUI, as shown below (click on image to enlarge it):

Xeon Phi Native Launch

As of VTune 2015 it is possible for VTune to do the ssh on to the MIC cards to run a native executable. This means we no longer need to put ssh mic0 in a script. However, using a script is still advisable so that you can set any environment variables needed on the MIC cards and pass input args to your executable.

For example, if we have a native executable (i.e., one that runs entirely on the MIC cards) then in the Project Properties window set the Target system to be Intel Xeon Phi coprocessor (native) as shown below (click on image to enlarge it):

We then set the Application to be a shell script which does not ssh to the MIC card. It simply sets any required environment variables. For example, a suitable shell script would be

#!/bin/bash
### VTune will ssh to the MIC so we don't do it here
IROOT=/opt/intel/composerxe
export LD_LIBRARY_PATH=$IROOT/compiler/lib/mic:$IROOT/mkl/lib/
mic
export KMP_AFFINITY=granularity=thread,balanced
cd $HOME/support/mycode
./matmul-native-mic.exe 2048 244

In this script we’ve removed the ssh mic0Project Properties window we then set the Application to be this script, as shown below. In this example the script is named run-matmul-native-mic-no-ssh.sh and you will need to use an absolute path – i.e., give the full path to the script (click on image to enlarge it). We also set the Card number drop-down menu to be the require MIC card – either 0(mic0) as shown in the image below or 1(mic1) (not shown). As always, you should select the MIC card that has been assigned to you when you logged in to xenomorph (use echo $OFFLOAD_DEVICS to check). Click on the image below to enlarge it:

Analysis Type

Once your job is set up you’ll need to select the Analysis Type (press the brown Play icon). As of v2015, when you’ve identified an executable to be a Xeon Phi executable as above (see Target System in the above settings) then you only get Analysis types suitable for MIC hardware in the GUI, as shown below (click on image to enlarge it):

v2013

This version may fail – it reports errors when connecting to the MIC cards.

In v2013 when selecting an Analysis type in the user interface, you should ensure you select the Knights Corner Platform Analysis which is another Intel name for the MIC architecture. The VTune tool supports many Intel architectures (Intel Core 2, Nahalem, Sandybridge etc) but does not automatically select the Knights Corner platform when starting a new analysis.