VTune Profiler
VTune Amplifier XE is the profiler for use on Intel systems. v2015 is now the preferred version.
Supported Backend Nodes
This application is only available on the Xenomorph backend node and is mainly used for debugging and profiling on the Xeon Phi (MIC) accerlerator cards. Please see for how to access the node, compile your code and launch your executables on the Xeon Phi cards.
Set up procedure
To setup the VTune AmplXE profiler/debugger load the following modulefile
module load tools/intel/vtune/amplxe2015
or alternatively use the usual Intel dot file:
source /opt/intel/vtune_amplifier_xe/amplxe-vars.sh
Note that the previous version of VTune (2013) is available via the following modulefile but we no longer support this version and cannot guarantee it will work on the current Phi card linux installation.
# This version may no longer work as we upgrade the OS on the MIC cards tools/intel/vtune/amplxe2013
Running the application
To start the GUI run:
amplxe-gui
Note, you will get the following error message when VTune starts:
sh: line 1: 2537 Segmentation fault (core dumped)\ /opt/intel/ism/bin/intel64/intelremotedialog > /dev/null 2>&1
This can be safely ignored.
Compilation Hints
It is recommended you compile your code with (at least) the following compiler flags when running a VTune analysis:
-g -debug inline-debug-info -O2
For example:
module load compiler/inte/14.0.3 # Native (runs entirely on MIC card) icc -o matmul-mic-native.exe -mmic -openmp \ -g -debug inline-debug-info -O2 matmul-mic.c # Offload (runs partly on CPU, partly on MIC card) icc -o matmul-mic-offload.exe -openmp \ -g -debug inline-debug-info -O2 matmul-mic.c # OpenCL (runs partly on CPU, partly on MIC card) module load libs/intel/opencl/4.4.0 icpc -o matmul-mic-opencl.exe \ -g -debug inline-debug-info -O2 -lOpenCL matmul-mic.c
The above examples assume the matmul-mic.c
source code has suitable #ifdef ... #endif
sections to allow the three forms of compilation. Alternatively you could use separate source files for each type.
Project Properties for MIC Analysis
Use the Project Properties dialogue to indicate you will be profiling a Xeon Phi (MIC) executable. There are now a couple of ways to launch the executable on the Xeon Phi (MIC) cards from within VTune, as described below.
Xeon Phi Host Launch
This is the method you mainly used in VTune 2013 and can be used in version 2015. It can be used to profile:
- Native applications run entire on the Xeon Phi. You write a shell script to ssh on to the Phi and run your code. Vtune runs the shell script.
- Offload or OpenCL applications which do some work on the host CPU and some work on the Xeon Phi. The executable will automatically offload the Xeon Phi code as and when required.
Xeon Phi Host Launch – Native Applications
In option 1 (Native applications) we write a shell script to ssh on to the MIC card and run the executable (the script can also set environment variables and pass arguments to your executable – whatever you put in the script). VTune will profile the executable on the MIC card when it runs.
In the Project Properties window set the Target system to be Intel Xeon Phi coprocessor (host launch) as shown below (click on image to enlarge it):
A suitable shell script would look something like:
#!/bin/bash
IROOT=/opt/intel/composerxe
ssh mic0 "export LD_LIBRARY_PATH=$IROOT/compiler/lib/mic:$IROOT/mkl/lib/mic; \
export KMP_AFFINITY=granularity=thread,balanced; \
cd $HOME/support/mycode;
./matmul-native-mic.exe 2048 244"
In the above script:
- We
ssh
on tomic0
. You should ensure you ssh to the MIC card you have reserved on xenomorph. Check this by running the following on xenomorph:echo $OFFLOAD_DEVICES
- We must set
LD_LIBRARY_PATH
as shown. Otherwise you will get an error message aboutlibiomp5.so
. - The
KMP_AFFINITY
setting is optional – please refer to the Intel documentation. - When we ssh in to the MIC card we will be in our home directory. So we change to where our executable is.
- We then run our executable, giving it a couple of input parameters (your application will be different).
In the VTune Project Properties window we then set the Application to be this script, as shown below. In this example the script is named run-matmul-native-on-mic0.sh
and you will need to use an absolute path – i.e., give the full path to the script (click on image to enlarge it):
Xeon Phi Host Launch – Offload Applications
For Option 2 (Offload or OpenCL applications) we do not need to use a shell script to launch the executable on the MIC cards. Instead, simply specify the full path to the executable in the Application field. This will be executed on the host CPU and the executable will automatically run parts of its code on the MIC card as required.
Analysis Type
Once your job is set up using either of the above methods you’ll need to select the Analysis Type (press the brown Play icon). As of v2015, when you’ve identified an executable to be a Xeon Phi executable as above (see Target System in the above settings) then you only get Analysis types suitable for MIC hardware in the GUI, as shown below (click on image to enlarge it):
Xeon Phi Native Launch
As of VTune 2015 it is possible for VTune to do the ssh on to the MIC cards to run a native executable. This means we no longer need to put ssh mic0
in a script. However, using a script is still advisable so that you can set any environment variables needed on the MIC cards and pass input args to your executable.
For example, if we have a native executable (i.e., one that runs entirely on the MIC cards) then in the Project Properties window set the Target system to be Intel Xeon Phi coprocessor (native) as shown below (click on image to enlarge it):
We then set the Application to be a shell script which does not ssh to the MIC card. It simply sets any required environment variables. For example, a suitable shell script would be
#!/bin/bash ### VTune will ssh to the MIC so we don't do it here IROOT=/opt/intel/composerxe export LD_LIBRARY_PATH=$IROOT/compiler/lib/mic:$IROOT/mkl/lib/ mic export KMP_AFFINITY=granularity=thread,balanced cd $HOME/support/mycode ./matmul-native-mic.exe 2048 244
In this script we’ve removed the ssh mic0
Project Properties window we then set the Application to be this script, as shown below. In this example the script is named run-matmul-native-mic-no-ssh.sh
and you will need to use an absolute path – i.e., give the full path to the script (click on image to enlarge it). We also set the Card number drop-down menu to be the require MIC card – either 0(mic0)
as shown in the image below or 1(mic1)
(not shown). As always, you should select the MIC card that has been assigned to you when you logged in to xenomorph (use echo $OFFLOAD_DEVICS
to check). Click on the image below to enlarge it:
Analysis Type
Once your job is set up you’ll need to select the Analysis Type (press the brown Play icon). As of v2015, when you’ve identified an executable to be a Xeon Phi executable as above (see Target System in the above settings) then you only get Analysis types suitable for MIC hardware in the GUI, as shown below (click on image to enlarge it):
v2013
This version may fail – it reports errors when connecting to the MIC cards.
In v2013 when selecting an Analysis type in the user interface, you should ensure you select the Knights Corner Platform Analysis which is another Intel name for the MIC architecture. The VTune tool supports many Intel architectures (Intel Core 2, Nahalem, Sandybridge etc) but does not automatically select the Knights Corner platform when starting a new analysis.