The Computational Shared Facility 3

Software

Please do not run applications directly on the login node.
You must submit jobs to the batch system instead.

General Software Information

Many widely used pieces of software are installed centrally on the CSF by the Research Infrastructure team, so that you don’t need to use your own CSF storage to install applications. This also reduces duplication and allows you to spend your time on your research.

Software on the CSF is accessed through use of user environment modules which set up your environment to run a specific application – each application has its own modulefile which you load to give you access to that application. Applications should then be run via the batch system (SGE). Each application’s web-page, listed in the appropriate menu on the left, shows all the commands needed to load the modulefile and run the application in a batch job.

To run GUI/interactive software that can’t be run as a batch job, use qrsh to run interactively on a compute node, not directly on the login node.

Finding out what is available

New apps are (frequently) being compiled on the CSF3. To see what is available on the CSF, run the following commands on the login node:

module search keyword       # search for an app name, e.g., module search tensor   (try it)
module avail                # list all modulefiles!

A lot of software (mostly, but not only, bio-informatics apps) from the DPSF has also been transferred to the CSF3. To access this suite of applications please do

module load apps/dpsf                  # Can also use apps/bioinf

Now running

module search keyword       # search for an app name
module avail                # list all modulefiles!

will show you a much larger list of available applications. These are untested on CSF3 – please alert us to any issues using them.

Note that Module names for some applications may be different to those on CSF2/DPSF. Some older versions of software may not be available on the CSF3.

Requesting Additional Software

If you wish to use software we do not currently have on the system, or you require a newer version of an installed app, please contact us via its-ri-team@manchester.ac.uk. We will discuss the requirement with you and do our best to help.

Usually it is possible to add most freely available software and commercial packages, where licenses exist on campus. Commercial software which does not have a campus license can be more problematic to handle so please contact us before assuming that a purchase you wish to make will work on the CSF.

For software that only one user requires it might be deemed most suitable for you to install it in your home directory. Although we may be able to install software we cannot guarantee to provide in-depth support for every package – it is expected all users of the CSF take the time to learn how to use their required software packages effectively.

Please note that Linux has a lot of packages which we cannot list here, but which are available by default on the command line.

Installing your own Software

You may wish to download open-source applications and install them in your home directory, or additional Research Data Storage that your research group might have access to on the CSF. This is certainly possible – some hints and tips are given below.

We recommend creating a directory (folder) for all of your local software applications. Within there create a directory for the final installed version of an application (you may want to install several different versions over time) and also a directory for the source code to allow you to compile the application. For example:

mkdir p ~/software/app-name/
cd ~/software/app-name/
mkdir 1.2.3 build
        #     #
        #     # Repo or source code downloaded in to ~/software/app-name/build/
        #
        # Final install in ~/software/app-name/1.2.3/

All downloads from the web (including git repository clones) must be done via the University web-proxy. The CSF cannot access the outside world directly, for security. The proxy is an intermediate server than can access the outside world. Hence you will usually need to load the following modulefile before doing any downloads:

module load tools/env/proxy

To clone a git repository:

module load tools/gcc/git/2.24.0
module load tools/env/proxy
cd ~/software/app-name/build/
git clone https://github.com/project-name/repo-name.git

# You will now usually have a directory named after the repo
cd repo-name

# If you wish to switch to a known tagged version 
git checkout tags/1.2.3

The next step is often to configure the software so that it can be compiled correctly on the CSF. Typically GNU autoconf tools or cmake is used. Check the software’s installation notes. By default most software builds will assume you want to install the software in a central location. You don’t have the necessary permissions to do this on the CSF so you should specify a local installation directory. At this point, if you wish to use the Intel compilers you should load their modulefile.

# If using the intel compiler, load the modulefile (GNU compilers are default if no modulefile loaded)
module load compilers/intel/18.0.3

cd ~/software/app-name/build/repo-name

# GNU autoconf tools
./configure --prefix=~/software/app-name/1.2.3

# OR cmake
cmake -DCMAKE_INSTALL_PREFIX=~/software/app-name/1.2.3

It is then usually a case of running the make command to run the compilation.

# Compile the software in the 'build' area
make
# Copy the final version of the sofware to your 'install' area
make install

# Have a look in your 'install' area
ls ~/software/app-name/1.2.3

Last modified on September 28, 2020 at 11:14 am by George Leaver