High Throughput Computing using Condor

Matlab

Overview

Matlab is the computational software package developed by The MathWorks.

Restrictions on use

Matlab may not be used for commercial purposes including being used by consultants or subcontractors. A detailed list of the license restrictions is available on the IT Services Applications website.

Versions installed

Version installed HTCondor ClassAd Location of Matlab runtimes
2013a HAS_MATLAB_2013 /opt/MATLAB/MATLAB_Compiler_Runtime/v81
2015a HAS_MATLAB_2015 /opt/MATLAB/MATLAB_Compiler_Runtime/v851
2017a HAS_MATLAB_2017 /opt/MATLAB/MATLAB_Compiler_Runtime/v92
2019a HAS_MATLAB_2019 /opt/MATLAB/MATLAB_Compiler_Runtime/v96

Set up procedure on submit node

To access the software you must first run the command:

module load matlab/<version>

Running the application

In order to run Matlab jobs on HTCondor, it is first necessary to compile the Matlab script file using the Matlab compiler. Whilst this has the advantage that compiled Matlab programs do not use any Matlab licenses when running, some Matlab toolbox functionality may be unavailable.

Step 1 – Compiling your Matlab script file

To ensure that your compiled Matlab code will run on HTCondor, it is essential that you compile your Matlab code on the HTCondor submit node.

From a linux terminal prompt, run the command

mcc -R -singleCompThread -m <script.m>

where

  • mcc is the name of the Matlab compiler command.
  • -R -singleCompThread tells the compiler to produce a single-threaded executable.
  • -m <script.m> specifies the name of the Matlab script file to compile.

The reason for the -R -singleCompThread flag is to avoid problems with Matlab trying to use multiple threads. By default Matlab will detect the number of CPU cores on a PC and then attempt to use that many threads. This can cause problems unless your HTCondor submit file has requested to use all the available cores on the compute node. The maximum number of threads can be specified in your Matlab script file using the maxNumCompThreads command.

After successful compilation, a shell script run_<script>.sh and executable file <script> should exist in the current directory.

Step 2 – Submitting your job to run on HTCondor

The following lines are the contents of a simple generic HTCondor submit text file:

Universe = vanilla

Requirements = (Target.Opsys == "LINUX" && Target.Arch == "X86_64" && <HTCondor ClassAd>=?=True)
Request_memory = 1000

Log = matlab.log
Output = matlab.out
Error = matlab.error
Notification = Error

Should_Transfer_Files = Yes
When_To_Transfer_Output = ON_EXIT

Executable = run_<script>.sh
Transfer_Executable = True
Arguments = <Location of Matlab runtimes> <input arguments>
Transfer_Input_Files = <script>,run_<script>.sh

Queue

If your submit file is called submit.txt then your job can be submitted to HTCondor using the command condor_submit submit.txt.

Using MEX files with compiled Matlab

Whilst it is possible to use MEX files in compiled Matlab codes, each version of Matlab supports a very limited range of the GNU compiler suite. The supported versions are listed in the following table:

Matlab version GNU C/C++ version GNU gfortran version
2013a 4.4.x 4.3.x
2015a 4.7.x 4.7.x
2017a 4.9.x 4.9.x
2019a 6.3.x 6.3.x

Please be aware that even if the mex command produces a MEX file, this does not necessarily mean that it will work with the Matlab compiler unless it was compiled using a supported version of the GNU compiler suite.

Sample Matlab HTCondor jobs

  1. Matlab code for calculating Pi using n terms of the Gregory series.

Last modified on May 5, 2019 at 10:50 am by Chris Paul