Singularity

Overview

Singularity provides a mechanism to run containers where containers can be used to package entire scientific workflows, software and libraries, and even data.

Version 3.8.6-1 is installed on the iCSF.

Restrictions on use

The software is licensed under the BSD 3-clause “New” or “Revised” License.

Set up procedure

Please note: you no longer need to use a modulefile. We have installed singularity as a system-wide command. So you can simply run the singularity commands you wish to run without loading any modulefiles.

For example:

# Ensure you have NO singularity modulefiles loaded ('module purge' will unload all modulefiles)

[username@incline34 ~]$ singularity --version
singularity version 3.8.6-1.el7

Running the application

You may run the command on its own to obtain a list of flags:

singularity
USAGE: singularity [global options...]  [command options...] ...

ONTAINER USAGE COMMANDS:
    exec       Execute a command within container                               
    run        Launch a runscript within container                              
    shell      Run a Bourne shell within container                              
    test       Launch a testscript within container                             

CONTAINER MANAGEMENT COMMANDS:
    apps       List available apps within a container                           
    bootstrap  *Deprecated* use build instead                                   
    build      Build a new Singularity container                                
    check      Perform container lint checks                                    
    inspect    Display container's metadata                                     
    mount      Mount a Singularity container image                              
    pull       Pull a Singularity/Docker container to $PWD

Please note that users will not be permitted to run singularity containers in --writeable mode. You should build containers on your own platform where you have root access.

Running a container image

singularity run mystack.simg

Further info

• Singularity Documentation

Using your own containers

You may want to use your own containers on the iCSF – that’s fine. You will need to have the ‘/scratch’ directory within the container to bind to the ‘/scratch’ directory on the iCSF.

If building from a .def file, please include the line

%post
...
mkdir /scratch

in the “%post” section.

If using a prebuilt sif (or simg), then follow the steps below to rebuild with a ‘/scratch’ directory within:

# You must run these commands on your own Linux system (you don't have sudo rights on the iCSF)
sudo singularity build --sandbox mysandbox myimage.sif
sudo mkdir mysandbox/scratch
sudo singularity build myimage-icsf.sif mysandbox

Please note this will not work on the iCSF – as you cannot have sudo (admin) rights. The steps are necessary before the image is used on the iCSF. If you cannot, we can do this for you: its-ri-team@manchester.ac.uk.

Converting from a Docker container

Many Docker images exist that can be converted to singularity images, then uploaded to the iCSF and run. As before, these steps must be completed on your own machine with a singularity / apptainer install.
The example below uses https://hub.docker.com/r/cp2k/cp2k/.

# You must run these commands on your own Linux system (you don't have sudo rights on the iCSF)
sudo singularity build cp2k.sif docker://cp2k/cp2k
singularity run cp2k.sif cp2k --version | head -1
# returns 'CP2K version 2023.1', which we use to label
sudo singularity build --sandbox cp2k-2023.1-sandbox cp2k.sif 
sudo mkdir cp2k-2023.1-sandbox/scratch
sudo singularity build cp2k-2023.1-csf.sif cp2k-2023.1-sandbox

The cp2k-2023.1-csf.sif can then be uploaded and run on the iCSF.

Please remember to bind scratch (and also /mnt which will make your home directory available) and run your jobs from there:

singularity run --bind /scratch:/scratch,/mnt:/mnt my_container.sif arg1 arg2 ...

Alternatively, you can set the following environment variable:

export SINGULARITY_BINDPATH="/scratch,/mnt"
singularity run my_container.sif arg1 arg2 ...

Building your own Singularity image

You can build your own sifs for use on the iCSF via the online resource: https://cloud.sylabs.io/builder

Please remember to include

mkdir /scratch

in the definition instructions. Be aware also that this resource is not affiliated with The University of Manchester.

Updates

None.