| The CSF2 has been replaced by the CSF3 - please use that system! This documentation may be out of date. Please read the CSF3 documentation instead. To display this old CSF2 page click here. |
High Memory Jobs
Default Memory on the CSF
Intel nodes have 4GB of RAM per core, but the short area has only 2GB per core. Higher-memory nodes are described below.
All AMD nodes have 2GB per core.
Important information
- None of the high memory nodes have infiniband.
- There are no high memory options for AMD nodes and node in the
shortarea. - High memory resources are very limited so should only be used for jobs that have a genuine need for it.
How to check memory usage of your job
The batch system keeps track of the resources your jobs is using and also records statistics about the job once it has finished.
A currently running job
You can see the memory usage with the following command:
qstat -j jobid | grep mem
For example:
[username@login2 ~]$ qstat -j 123456 | grep maxvmem usage 1: cpu=7:01:59:41, mem=491221.37209 GBs, io=1.66402, vmem=15.448G, maxvmem=15.522G
You should ignore the value displayed after mem as this is a total aggregated over time. The value to check is maxvmem which shows what your job has peaked at. Divide that value by the number of cores your job is using to get a per core value.
A finished job
If your job has finished you can still check via:
qacct -j jobid | grep maxvmem
Depending on the software you are using you may also find memory usage reported in output files.
256GB Nodes
Two Intel nodes with 256GB of RAM (approx 21GB per core, Sandybridge) are available. Access to these nodes is restricted. If you think you need more than 4GB per core of memory for your jobs please email its-ri-team@manchester.ac.uk with details about which software you use and information about memory requirements (e.g. an estimate generated by your software, errors from not having enough memory).
Once you have been granted access your jobs that require this resource should include:
#$ -l mem256
Serial or SMP smp.pe jobs, with a maximum of 12 slots, may be run.
Note that your job will not run if you use the above flag without having been given access.
Changes to high memory node availability Jan 2019
We no longer have any 8GB per core nodes (-l vhighmem).
The 0.5TB westmere node (-l vvhighmem) has been retired.
A number of the 256GB nodes have now moved to CSF3.