{"id":33,"date":"2018-08-28T16:42:57","date_gmt":"2018-08-28T15:42:57","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=33"},"modified":"2026-01-27T14:48:50","modified_gmt":"2026-01-27T14:48:50","slug":"software","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/software\/","title":{"rendered":"Software"},"content":{"rendered":"
Please do not run applications directly on the login node.
\nYou must submit jobs to the batch system (Slurm) instead.<\/em>We are in the process of updating the application documentation to provide Slurm jobscript examples (the SGE batch system jobscripts are no longer valid – that batch system has been shutdown.)If you are running an app who’s page hasn’t been updated yet, please consult the SGE to Slurm<\/a> reference, for how to write a Slurm jobscript based on your SGE jobscript.<\/p>\n<\/div>\n

General Software Information<\/h2>\n

Many widely used pieces of software are installed centrally on the CSF by the Research Infrastructure team, so that you don’t need to use your own CSF storage to install applications. This also reduces duplication and allows you to spend your time on your research.<\/p>\n

Software on the CSF is accessed through use of user environment modules<\/a> which set up your environment to run a specific application – each application has its own modulefile<\/em> which you load<\/em> to give you access to that application. Applications should then be run via the batch system (Slurm)<\/a>. Each application’s web-page, listed in the appropriate menu on the left, shows all the commands needed to load the modulefile and run the application in a batch job.<\/p>\n

To run GUI\/interactive software that can’t be run as a batch job, use srun<\/a> to run interactively on a compute node<\/strong>, not directly on the login node.<\/p>\n

Finding out what is available<\/h2>\n

New apps are (frequently) being compiled on the CSF3. To see what is available on the CSF, run the following commands on the login node:<\/p>\n

module search keyword<\/em>       # search for an app name, e.g., module search tensor   (try it<\/strong>)\r\nmodule avail                # list all<\/em> modulefiles!\r\n<\/pre>\n
A lot of software (mostly, but not only, bio-informatics<\/a> apps) from the DPSF has also been transferred to the CSF3. To access this suite of applications please do<\/p>\n
module load apps\/dpsf       # Can also use apps\/bioinf\r\n<\/pre>\n
Now running<\/p>\n
module search keyword<\/em>       # search for an app name\r\nmodule avail                # list all<\/em> modulefiles!\r\n<\/pre>\n
will show you a much larger list of available applications. These are untested on CSF3 – please alert us to any issues using them.<\/p>\n
Note that Module names for some applications may be different to those on CSF2\/DPSF. Some older versions of software may not be available on the CSF3.<\/p>\n
Rule of thumb – don’t mix and match things! Finding out more about a piece of software<\/h2>\n
Most people don’t need to know the details of how a piece of software was compiled, or what environment it is setting. We will try and give useful details on this in our user docs if it is appropriate for a particular piece of software.<\/p>\n
You may want to check that multiple pieces of software you are loading are\/will be compatible. If you load something that has intel-17.0 in the name and something that has intel-18 in the name then this may cause an issue as you are loading software compiled with different versions of the intel compiler and thus different underlying libraries. However, something with gcc in the name and something with intel-1x.x in the name may be fine, it can be hard to tell. Note though, that a lot of our modulefiles load other modulefiles for you on which they are dependent and we will have ensured that these are compatible with one another.<\/p>\n
If you would like to see what a specific modulefile loads or what other settings\/variables are the default that you need to be careful not to override by any settings you make use the command:<\/p>\n
module show modulename<\/em>\r\n<\/pre>\n
where modulename<\/em> is the modulefile of the software you are interested in.<\/p>\n
Common examples we see:<\/p>\n
Loading a version of openmpi compiled with intel-1x.x and loading a different version of the intel compiler. The openmpi modulefile will load the correct intel compiler version for you.
\nLoading an application compiled with a specific version of python and then loading a different version of python.<\/p>\n
The above does mean that if you have a pipeline that uses several pieces of software then trying to handle it all within one batch job may be tricky. Consider using job dependencies<\/a> to run each element or piece of software.<\/p>\n
Requesting Additional Software<\/h2>\n
If you wish to use software we do not currently have on the system, or you require a newer version of an installed app, please contact us<\/a>. We will discuss the requirement with you and do our best to help.<\/p>\n
Usually it is possible to add most freely available software and commercial packages, where licenses exist on campus. Commercial software which does not have a campus license can be more problematic to handle so please contact us before assuming that a purchase you wish to make will work on the CSF.<\/p>\n
For software that only one user requires it might be deemed most suitable for you to install it in your home directory. Although we may be able to install software we cannot guarantee to provide in-depth support for every package – it is expected all users of the CSF take the time to learn how to use their required software packages effectively.<\/p>\n
Please note that Linux has a lot of packages which we cannot list here, but which are available by default on the command line.<\/p>\n
Installing your own Software<\/h2>\n
You may wish to download open-source applications and install them in your home directory, or additional Research Data Storage that your research group might have access to on the CSF. This is certainly possible – some hints and tips are given below.<\/p>\n
R and python<\/h3>\n
We have detailed instructions for R<\/a> and python<\/a> packages.<\/p>\n
Other applications<\/h3>\n
We recommend creating a directory (folder) for all of your local software applications. Within there create a directory for the final installed version of an application (you may want to install several different versions over time) and also a directory for the source code to allow you to compile the application. For example:<\/p>\n
mkdir -p ~\/software\/app-name<\/em>\/\r\ncd ~\/software\/app-name<\/em>\/\r\nmkdir 1.2.3<\/em> build\r\n        #     #\r\n        #     # Repo or source code downloaded in to ~\/software\/app-name<\/em>\/build\/\r\n        #\r\n        # Final install in ~\/software\/app-name<\/em>\/1.2.3<\/em>\/\r\n<\/pre>\n
To clone a git repository:<\/p>\n
\r\ncd ~\/software\/app-name<\/em>\/build\/\r\ngit clone https:\/\/github.com\/project-name<\/em>\/repo-name<\/em>.git\r\n\r\n# You will now usually have a directory named after the repo\r\ncd repo-name<\/em>\r\n\r\n# If you wish to switch to a known tagged version \r\ngit checkout tags\/1.2.3<\/em>\r\n<\/pre>\n
Further information about using git on the CSF<\/a>.<\/p>\n
Using wget to download:<\/p>\n
\r\ncd ~\/software\/app-name\/\r\nwget https:\/\/website.somewhere\/path\/to\/download.tar.gz\r\n    # Replace the https part of the above with the actual web address for your download\r\n<\/pre>\n
The next step is often to configure<\/em> the software so that it can be compiled correctly on the CSF. Typically GNU autoconf<\/em> tools or cmake<\/em> is used. Check the software’s installation notes. By default most software builds will assume you want to install the software in a central location. You don’t have the necessary permissions to do this on the CSF so you should specify a local installation directory. At this point, if you wish to use the Intel compilers you should load their modulefile.<\/p>\n
# If using the intel compiler, load the modulefile (GNU compilers are default if no modulefile loaded)\r\nmodule load compilers\/intel\/18.0.3\r\n\r\ncd ~\/software\/app-name<\/em>\/build\/repo-name<\/em>\r\n\r\n# GNU autoconf tools\r\n.\/configure --prefix=~\/software\/app-name<\/em>\/1.2.3<\/em>\r\n\r\n# OR cmake\r\ncmake -DCMAKE_INSTALL_PREFIX=~\/software\/app-name<\/em>\/1.2.3<\/em>\r\n<\/pre>\n
It is then usually a case of running the make<\/code> command to run the compilation.<\/p>\n
# Compile the software in the 'build' area\r\nmake\r\n# Copy the final version of the sofware to your 'install' area\r\nmake install\r\n\r\n# Have a look in your 'install' area\r\nls ~\/software\/app-name<\/em>\/1.2.3<\/em>\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
Please do not run applications directly on the login node. You must submit jobs to the batch system (Slurm) instead.We are in the process of updating the application documentation to provide Slurm jobscript examples (the SGE batch system jobscripts are no longer valid – that batch system has been shutdown.)If you are running an app who’s page hasn’t been updated yet, please consult the SGE to Slurm reference, for how to write a Slurm jobscript.. Read more »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-33","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=33"}],"version-history":[{"count":21,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/33\/revisions"}],"predecessor-version":[{"id":11774,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/33\/revisions\/11774"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}