Intel’s Math Kernel Library (MKL)

Overview

Intel’s Math Kernel Library contains the following:

• BLAS
• Sparse BLAS
• LAPACK
• PBLAS
• ScaLAPACK
• Sparse Solver routines
• Vector Mathematical Library functions
• Vector Statistical Library functions
• Fourier Transform functions (FFT)
• Cluster FFT
• Trigonometric Transform routines
• Poisson, Laplace, and Helmholtz Solver routines
• Optimization (Trust-Region) Solver routines
• GMP arithmetic functions

Versions of MKL installed on the CSF:

• MKL version 10.2u3 is supplied with Intel’s ifort compiler version 11.1.064
• MKL version 10.2u7 is supplied with Intel’s icc compiler version 11.1.075
• MKL version 10.3u5 is supplied with Intel ifort and icc compiler 12.0.5
• MKL version 11.1u3 is supplied with Intel ifort and icc compiler 14.0.3
• MKL version 11.2u3 is supplied with Intel ifort and icc compiler 15.0.3

Documentation can be found in the following directories according to version:

/opt/gridware/compilers/intel/11_1_064/Documentation/en_US/mkl
/opt/gridware/compilers/intel/11_1_075/Documentation/en_US/mkl
/opt/gridware/compilers/intel/12.0.5/composerxe/Documentation/en_US/mkl/
/opt/gridware/compilers/intel/14.0.3/composer_xe_2013_sp1.3.174/Documentation/en_US/mkl
/opt/gridware/compilers/intel/15.0.3/composer_xe_2015.3.187/Documentation/en_US/mkl

Restrictions on use

MKL on the CSF is supplied with the Intel icc and ifort compilers, these use the University’s Intel compiler licenses. The Intel compilers are available to any student or member of staff of the University of Manchester for the purpose of the normal business of the University. Such use includes research, personal development and administration and management of the university. The software may not be used for research, consultancy or services leading to commercial exploitation of the software.

Code may be compiled on the login node, but aside from very short test runs (e.g., one minute on fewer than 4 cores), executables must always be run by submitting to the batch system, SGE.

Set up procedure

Before compiling and using MKL you may need to load the appropriate module depending on which version of the Intel compiler you are going to use in which case you should choose one of the modulefiles:

module load libs/intel/mkl/10.2u3

or

module load libs/intel/mkl/10.2u7

or

module load libs/intel/mkl/10.3u5

For newer versions of the compiler:

11.1u3 is automatically available when loading version 14.0.3 of the Intel Compilers.
11.2u3 is automatically available when loading version 15.0.3 of the Intel Compilers.

Linking against MKL

There are various options to consider when linking against MKL, e.g. static or dynamic linking, serial or multi-threaded executables, Intel or Gnu compilers, etc. To obtain correct flags for linking the Intel Math Kernel Library Link Line Advisor can be used, alternatively read the documentation.

Note:

• The processor architecture on the CSF is Intel(R) 64
• The output from the link line advisor is meant to be copied in to a Makefile and so uses $(MKLROOT). If you are compiling on the command line you should change the brackets to be ${MKLROOT}.

Submission of batch jobs using serial MKL

First ensure you have loaded one of the modulefiles described above.

Jobs using serial MKL libraries can be submitted in the usual way. For example:

#!/bin/bash
#$ -S /bin/bash
#$ -cwd                    # Run the job in the current directory (where you run qsub)
#$ -V                      # Inherit current environment 
                           # (e.g., settings from any loaded modulefiles)
### No PE specified for serial jobs

./my_serial_mkl.exe

To submit:

qsub scriptname

Submission of batch jobs using multi-threaded MKL

First ensure you have loaded one of the modulefiles described above.

Jobs using multi-threaded MKL libraries should be submitted as per OpenMP based code:

1. The number of threads must be selected using the OMP_NUM_THREADS environment variable
2. An appropriate number of cores must be set in the SGE script
3. An appropriate parallel environment must be selected, e.g. smp.pe

Information on submitting OpenMP jobs to the CSF using SGE can be found here. An example jobscript might be:

#!/bin/bash
#$ -S /bin/bash
#$ -cwd                    # Run the job in the current directory (where you run qsub)
#$ -V                      # Inherit current environment
                           # (e.g., settings from any loaded modulefiles)
#$ -pe smp.pe 8            # Parallel environment and number of cores (8 in this example)

# NSLOTS is automatically set to the number specified on the -pe line.
# OMP_NUM_THREADS is used by the MKL to determine how many cores to use.
export OMP_NUM_THREADS=$NSLOTS

./my_openmp_mkl.exe

Submit in the usual way:

qsub jobscript

Updates

None at this time.