Modules

Overview

Modules is the system by which most software is made available. Modules allow, via a simple interface, the update of various paths (and other environment settings) that allow access to software.

The use of modules also provides a convenient way to switch between different versions of the same software (such as compilers).

Commands

The environment is controlled using the module command with various arguments:

• module list – lists currently loaded modules
• module avail – lists all available modules
• module load modulename – loads module modulename
• module unload modulename – unloads module modulename
• module switch oldmodulename newmodulename – switches between two modules
• module initadd modulename – run this command once to automatically ensure that a module is loaded when you log in. (It creates a .modules file in your home dir which acts as your personal configuration.)
• module purge – unload all modules
• module help – may show longer description of the module if present in the modulefile
• man module – detailed explanation of the above commands and others

Categories

The software available of the CSF has been placed in categories:

• apps – for main applications which has subsections for the compiler used to build an application – for example, intel-11.1 – or where no compile was required – binapps.
• compilers
• libs – for example, NAG, mkl
• mpi

The module avail command will list everything available. You can view a single category at a time with:

module avail <category>
  #
  # replacing the optional <category> with one of the above

To use a module/piece of software:

module load <module name>
  #
  # replacing <module name> with the module you want

For example:

module load compilers/intel/fortran/11.1.064

It is possible, and often necessary, to have multiple modules loaded at once. For example the following commands are equivalent:

module load tools/gcc/autoconf/2.68
module load tools/gcc/cmake/2.8.7

or

module load tools/gcc/autoconf/2.68 tools/gcc/cmake/2.8.7

Some modulefiles have restricted access (you need to be in a certain unix group to load the modulefile). The module commands may report an error when trying to access these modules if you don’t have permission to use them. For example module avail will complain about not being able to read certain modulefiles. You can simply ignore these errors. For example:

[mxyzabc2@login2 ~]$ module avail apps
utility.c(2229):ERROR:50: Cannot open file '/opt/gridware/modulefiles/apps/binapps/starccm/7.04' for 'reading'
---------------------- /opt/gridware/modulefiles ---------------------
apps/amd-4.5.2/Code_Saturne/2.0.6                   apps/gcc/ipython/0.13.2
apps/amd-4.5.2/Code_Saturne/2.3.1                   apps/gcc/lens/2.64
apps/amd-4.5.2/Gerris/13.01.12                      apps/gcc/octave/3.6.4
   #
   # indicates the we are not in the unix group for the starccm software but we
   # simply ignore this error and can see all of the modulefiles available to us
   #

Bash Completion

The module paths on the CSF are lengthy, but you can use bash completion to help you. Press TAB part-way through typing a module name and the name will be completed for you or a list of possible completions will be given. Type a few more characters and hit TAB again to narrow the list down until you get the full module name.

Seeing module environment settings in batch

Modulefiles can be loaded:

• on the login node before you submit a job
• or from the jobscript when the job actually runs

There are pros and cons to each method but which method you choose is up to you – both are valid.

The advantages of loading the modulefile on the login node before you submit the job are that you can check the name of the modulefile is correct (any spelling mistakes and the modulefile won’t be loaded). You could also check the settings the modulefile makes (e.g., an environment variable) and then use that in your jobscript. The disadvantages are that the particular modulefile name used is not recorded in your jobscript. If you wish to run the job again (perhaps in a few months time) you may have forgotten which version of a modulefile you used.

The advantages of loading the modulefile in the jobscript are that the names of the modulefiles are recorded – this helps with reproducibility. However, if you spell a modulefile name incorrectly the mistake will only be detected when the job runs. Your job will probably fail and you’ll have to resubmit the job, waiting in the queue again.

Loading modulefiles on the login node

If you prefer to load your modulefiles on the login node before you submit a job then you should add the following line to your jobscript:

#$ -V                       # Inherit settings from the login node

Note: That is an uppercase V.

The batch system takes a copy of your environment at the time at which you run the qsub command, not when your job eventually runs. This means that after submitting your job with qsub you are free to change your environment or even log out and your job will still see the correct environment when it runs.

Loading modulefiles in jobscripts

If you prefer to load your modules within the batch script you must start the batch script with the following line:

#!/bin/bash --login

You can then do the usual jobscript commands and also include module commands. For example:

#!/bin/bash --login
#$ -cwd                     # Run from the current directory
                            # Notice there is no #$ -V flag now!!

module load compilers/intel/c/12.0.5
./my_app

We no longer use #$ -V to inherit the environment because we are loading modulefiles in the jobscript to set up the environment, so it should be removed from your jobscript if you have been using it previously.

Automatically loading modulefiles on log in

Note: We recommend that you only load modulefiles as and when you require them, not automatically on login. This way you avoid unexpected software clashes, interactions and environment problems that can cause job failure.

If you do wish to always have certain modulefiles loaded when you log into the CSF then best practice is to specify them in your .modules environment file, not the .bashrc, or .bash_profile. Add the module load commands to the end of the .modules file and leave any other content in that file untouched. However, bear in mind that the warning above still applies.

Private modules – software you have installed

If you have installed a piece of software in your home directory then you can create your own modulefile for it which saves you having to set variables on the command line, in your .bashrc, .bash_profile or in batch scripts.

• In your home dir create a directory called ‘privatemodules’ (the name is key)

  mkdir ~/privatemodules
  

• Within that dir create the required folders and modulefiles (see template below) so you get something like:

  cd ~/privatemodules
  pwd
  /mnt/iusers01/xy01/mxyzabc2/privatemodules
  
  # Create a directory named after my own application
  mkdir myapp
  
  # Go in to the dir and create a text file named using the application version number (e.g., 2.03):
  cd myapp
  gedit 2.03
         #
         # Write your modulefile in this plain text file which is named '2.03'
         # It is an unusual name for a text file but Linux allows any name to be used.
         # It will make the 'module load' command used later look like the others on the CSF.
         # See below for a template modulefile
  
  # You should now have a file in at the following location:
  ~/privatemodules/myapp/2.03   
  

• To load your new modulefile first load the ‘use.own’ modulefile (only need to do this once).

  module load use.own
  module load myapp/2.03
  

A template for a modulefile

#%Module1.0####################################################
##
## APP Modulefile
##    -- Replace APP with the name of your app
##
proc getenv {key {defaultvalue {}}} {
  global env; expr {[info exist env($key)]?$env($key):$defaultvalue}
}

## -- The 'module load' line below shoud be where you specify other modulefiles that may be required
## -- Remove the 'module load' line if you do not need it
## -- For example, compliers you used to build the software and mpi
module load compilers/gcc/x.x.x mpi/gcc/openmpi/1.x.x

## -- Now, specify the path to the top level of your software install and set that variable
set     APP_DIR     /PATH/TO/SOFTWARE_ROOT/
setenv  APP_DIR     $APP_DIR

## -- Next add required environment variables as detailed in the software manual
## -- Note, no need for 'export', the below examples add to your existing environment
prepend-path    PATH               $APP_DIR/bin
prepend-path	LD_LIBRARY_PATH    $APP_DIR/lib

Further Information

• man module
• Sourceforge Modules Website