More MATLAB Compilation Hints & Tips

October 2023: It is NO LONGER necessary to compile your MATLAB code on the CSF. You may compile it if you wish to do so. Hence our compilation instructions are still provided below. Alternatively, see our instructions on running un-compiled MATLAB.

Introduction

Please first read the Basics of Compiling MATLAB Code if you haven’t already done so.

Recap: Single and multi-threaded compilation

The Basics of Compiling MATLAB Code page describes in details how to compile MATLAB code and the differences between a serial (1-core) app and a multi-core MATLAB app.

In summary, by default, MATLAB jobs are multithreaded so if you submit serial (1-core) jobs to the batch system you must also explicitly force MATLAB to perform serial (1-core) computations by compiling your MATLAB code with the -R -singleCompThread option  at compile time:

mcc -R -singleCompThread -m mycode.m

If, for any reason, you choose not to compile with this option, your program may utilize multiple threads and hence multiple cores on the CSF compute node where your batch job runs.  If you run a multi-threaded compiled MATLAB program using a serial (1-core) batch job, which is clearly incorrect, you can end up with situations where, for example, your MATLAB jobs are each trying to use 12 cores on a 12 core compute-node. But other jobs could be using those cores.  This results in slower run-times and hence reduced throughput.

So the correct method is:

  • If you want to submit a 1-core MATLAB job, you must compile the MATLAB code to use only 1 core.
  • If you want to submit a multi-core MATLAB job, you can compile the MATLAB code without forcing it to use 1 core (by default MATLAB will compile to use multi-cores). You should then submit the job to use the whole compute node in the smp.pe parallel environment.

Note: We have discovered certain cases where the compiler appears to ignore the -R -singleCompThread option leading to over subscription of processor cores.  This will slow down both your and other people’s jobs.  If your code is affected by this bug, you will need to use the smp.pe parallel environment.

Dealing with simple dependencies

Most MATLAB programs consist of several files.  For example, you may have main.m that uses a function defined in myfunc.m:

%contents of main.m
result = myfunc(1)

along with

%Contents of myfunc.m
function [out] = myfunc(in)
  out=in + sin(in);
end

In situations such as this, all you need to do is ensure that main.m and myfunc.m are in the same directory.  Only main.m need be compiled:

mcc -R -singleCompThread -m main.m

The compiler will automatically determine that main.m needs myfunc.m and include it in the resulting executable.

Dealing with a directory full of dependencies

More complicated code may include one or more directories containing functions that are required by your main program. You can tell the compiler to include these directories in its search path with the -I flag. For example, if your functions are in the directory myfuns, which is a sub-directory of the directory containing myprogram.m you can do

mcc -m myprogram.m -R -singleCompThread -I ./myfuns

Faster Compilation

It can often take a long time to compile large MATLAB programs, as much as 10 minutes or more.  It is sometimes possible to speed up this process using by using the -N flag which tells MATLAB not to search for functions from any of the built in toolboxes.  That is, the search path is limited to only these locations

  • matlabroot\toolbox\matlab
  • matlabroot\toolbox\local
  • matlabroot\toolbox\compiler\deploy

So, if your program does not use any toolbox functions at all, using the -N flag can improve compilation times without loss of functionality.

mcc -R -singleCompThread -N -m myprogram.m

If, however, your program does make use of toolbox functions, using the -N flag can result in runtime errors. For example, consider mystats.m

%contents of mystats.m
x=normpdf(1.0)
disp(x)

This will compile without error

mcc -R -singleCompThread -N -m mystats.m

It will, however, give an error message when run

Undefined function 'normpdf' for input arguments of type 'double'.
Error in mystats (line 1)

MATLAB:UndefinedFunction

This happens because normpdf is part of the statistics toolbox and we explicitly told MATLAB not to look there at compile time. To include the statistics toolbox we need to use the -p flag as follows

mcc -R -singleCompThread -N -p stats -m mystats.m

The resulting executable will now run without error.  You can add in as many toolboxes as you like using this mechanism.  For example, you could add optimisation as well as statistics with

mcc -R -singleCompThread -N -p stats -p optim -m mystats.m

The argument to -p is determined by the toolbox folder name within the MATLAB installation.

Producing image files

If your job runs and produces the expected output, but not the images/plots you are trying to create make sure that you have closed/saved them in your code. This is not strictly necessary for normal MATLAB usage, but it is essential for compiled code.

Not everything can be compiled

Unfortunately, not every MATLAB feature can be included in compiled code.  The following are the most commonly used features that cannot be compiled

  • Anything that uses the symbolic toolbox.
  • Most MATLAB ‘apps’ and user interfaces (e.g. optimtool from the optimisation toolbox)

Further details of supported MATLAB toolboxes (and exclusions) is available at

If your code relies on any non-compilable functionality, you will not be able to run many concurrent jobs and will need to contact us for advice.

The MATLAB Compiler will not make your code go any faster

A common misconception is that the MATLAB Compiler produces code that runs more quickly than in MATLAB itself.  This is not the case.  We use the MATLAB Compiler in order to ensure that large numbers of HPC MATLAB jobs do not use up all of the campus network licenses (remember: compiled MATLAB code only uses a license during compilation, not when the job runs).

Last modified on October 23, 2023 at 1:43 pm by George Leaver