{"id":9123,"date":"2025-03-17T17:58:43","date_gmt":"2025-03-17T17:58:43","guid":{"rendered":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=9123"},"modified":"2026-10-12T14:37:56","modified_gmt":"2026-10-12T13:37:56","slug":"partitions","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/batch-slurm\/partitions\/","title":{"rendered":"Slurm Partitions (Compute Resources)"},"content":{"rendered":"

In Slurm, jobs are submitted to partitions<\/em> – think of these as job queues for different types of jobs.<\/p>\n

In your jobscript, you must<\/em> specify the partition<\/em>, number of cores<\/em> and max “wallclock<\/em>” time your job is allowed to run for (up to a permitted maximum time.) There is now no<\/em> default wallclock time – you must specify it.<\/p>\n

You may also need to specify other flags depending on which parition you submit to. For example, if using one of the GPU partitions, you’ll need to say how many GPUs you want. The HPC Pool and H200 partitions also require an account code.<\/p>\n

The table below provides an overview of the partitions, and the compute resources, currently available in the upgraded CSF3 cluster (Slurm).<\/p>\n

The Slurm jobscript template<\/a> (shown further down this page) gives you a starting point for your jobscript. Much more detail on Slurm jobscript options is available in the sbatch command<\/a> page and, for users of previous CSF systems, the SGE to Slurm reference<\/a>.<\/p>\n

You will also need to use Slurm commands on the login node: sbatch<\/code>, squeue<\/code>, scancel<\/code> and srun<\/code>.<\/p>\n

Reminder: You must<\/em> specify a Slurm Partition in your jobscript for all<\/em> job types<\/strong>: -p name<\/em><\/code>You must also<\/em> specify a maximum wallclock time limit for your job (up to a permitted maximum time)<\/strong>: -t timelimit<\/em><\/code><\/p>\n

See table below.<\/strong><\/p>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
CSF3 Slurm Partitions and Compute Resources<\/caption>\n
Slurm Partition name<\/th>\nPartition Summary<\/th>\nCPU<\/th>\nMax Wallclock<\/acronym><\/th>\nCompute Resources<\/th>\nNotes<\/th>\n<\/tr>\n<\/thead>\n
Parallel Jobs (single node)<\/th>\n<\/tr>\n
-p multicore_small<\/code><\/td>\nSingle-node parallel (2-40 cores) batch<\/strong> jobs. -n numcores<\/em><\/code><\/td>\nIntel<\/td>\n7 days -t 7-0<\/code><\/td>\n21x Haswell nodes (24 cores, 4-5GB\/core) Optional: -C haswell<\/code><\/p>\n

28x Skylake nodes (32 cores, 4-5GB\/core) Optional: -C skylake<\/code><\/p>\n

NEW<\/strong><\/span>: 78x Cascadelake nodes (40 cores, 4GB\/core) Optional: -C cascadelake<\/code> or -C csf4<\/code><\/td>\n

1400 Haswell and Skylake cores available (shared with the serial partition below.)<\/p>\n

3120 Cascadelake cores now available.<\/strong>.<\/p>\n

Unless specified, 4GB\/core will be allocated no matter which node-type your job runs on.<\/p>\n

Recommended for jobs of 2 to 40 cores. multicore (see below) is suggested for jobs of 41 to 168 cores.<\/strong>\n<\/td>\n<\/tr>\n

-p multicore<\/code><\/td>\nSingle-node parallel (2-168 cores) batch<\/strong> jobs. -n numcores<\/em><\/code><\/td>\nAMD Genoa<\/td>\n7 days -t 7-0<\/code><\/td>\n84x AMD nodes (168 cores per node). 8GB RAM per core.<\/td>\n14,112 cores available in total.<\/p>\n

Recommended for jobs of 41 to 168 cores. multicore_small (see above) is suggested for jobs of 2 to 40 cores.<\/strong><\/td>\n<\/tr>\n

-p interactive<\/code><\/td>\nSerial (1-core) and small parallel (2–28 cores) interactive<\/em><\/strong> jobs. Optional: -n numcores<\/em><\/code><\/p>\n

Serial (1-core) batch<\/em><\/strong> jobs.<\/td>\n

AMD Genoa<\/td>\n6 hours<\/strong><\/span> -t 0-6<\/code><\/td>\n2x AMD nodes (168 cores per node). 8GB RAM per core.<\/td>\n336 cores available in total. srun<\/code> (interactive), srun-x11<\/code> (interactive GUIs) and also sbatch<\/code> (jobscripts) accepted.<\/p>\n

For testing apps on AMD “Genoa”.<\/strong><\/td>\n<\/tr>\n

Parallel Jobs (multinode)<\/th>\n<\/tr>\n-p multinode<\/code><\/td>\nMulti-node parallel (80-1280 cores, in multiples of 40 cores) batch<\/strong> jobs.
\nRequired:<\/p>\n
\r\n-N numnodes<\/em>\r\n-n numtasks<\/em><\/pre>\n
Optional:<\/p>\n
\r\n-c corespertask<\/em><\/pre>\n<\/td>\n
Intel Cascadelake<\/td>\n80-560<\/strong> cores: 7 days -t 7-0<\/code><\/p>\n

600-1280<\/strong> cores: 4 days -t 4-0<\/code><\/td>\n

Node spec: Cascadelake nodes (40 cores, 4GB\/core)<\/td>\n8,720 cores available in total.<\/p>\n

NOT available to free at point of use<\/em> users (fatpou01) due to core limits.<\/strong><\/p>\n

Job size limits will be reviewed in January 2027.<\/span><\/strong><\/td>\n<\/tr>\n

Serial Jobs<\/th>\n<\/tr>\n
-p serial<\/code><\/td>\nSerial (1-core) batch<\/strong> jobs.<\/p>\n

Serial (1-core) interactive<\/em><\/strong> jobs.<\/td>\n

Intel<\/td>\n7 days -t 7-0<\/code><\/p>\n

6 hours<\/strong><\/span> -t 0-6<\/code><\/td>\n

21x Haswell nodes (24 cores, 5GB\/core) Optional: -C haswell<\/code><\/p>\n

28x Skylake nodes (32 cores, 6GB\/core) Optional: -C skylake<\/code><\/td>\n

1400 cores available in total (shared with the multicore_small partition above).<\/p>\n

See also interactive<\/a> above.<\/td>\n<\/tr>\n

High Memory Jobs<\/th>\n<\/tr>\n
-p himem<\/code><\/td>\nHigh memory jobs up to 2TB (1-32 cores). Optional: -n numcores<\/em><\/code><\/td>\nIntel<\/td>\n7 days -t 7-0<\/code><\/td>\n1x 512GB node (16 cores, 32GB per core) Optional: -C haswell<\/code><\/p>\n

4x 1.5TB node (32 cores, 40GB per core) Optional: -C cascadelake<\/code><\/p>\n

9x 2TB nodes (32 cores, 64GB per core) Optional: -C icelake<\/code> or -C ssd<\/code><\/td>\n

Job default is 32GB\/core unless you ask for more<\/em> – even if you add a -C<\/code> constraint! Use --mem=total<\/em>G<\/code> or --mem-per-cpu=num<\/em>G<\/code> to request more.
\nHigh Memory Docs<\/a><\/td>\n<\/tr>\n
-p vhimem<\/code><\/td>\nHigh memory jobs requiring up to 4TB. Optional: -n numcores<\/em><\/code><\/td>\nIntel<\/td>\n7 days -t 7-0<\/code><\/td>\n1x 4tb node (32 cores, 128GB per core) Optional: -C icelake<\/code> or -C ssd<\/code><\/td>\nRestricted access<\/strong> – request access<\/a><\/td>\n<\/tr>\n
GPU Jobs<\/th>\n<\/tr>\n
-p gpuA<\/code><\/td>\nNvidia A100 GPU batch and interactive jobs. Optional: -n numcores<\/em><\/code>
\nRequired: -G NUM<\/em><\/code> or --gpus=NUM<\/em><\/code><\/td>\n		AMD Milan<\/td>\n4 days -t 4-0<\/code><\/td>\nNode spec: 4x Nvidia A100s, 80GB GPU RAM, 48x AMD “Milan” host cores, 512GB host RAM (10GB\/core)<\/td>\nMax 12 CPU-cores per GPU<\/p>\n

19 nodes available = 76 GPUs<\/td>\n<\/tr>\n

-p gpuL<\/code><\/td>\nNvidia L40S GPU batch and interactive jobs. Optional: -n numcores<\/em><\/code>
\nRequired: -G NUM<\/em><\/code> or --gpus=NUM<\/em><\/code><\/td>\n		Intel Xeon Sapphire Rapids<\/td>\n4 days -t 4-0<\/code><\/td>\nNode spec: 4x Nvidia L40S, 48GB GPU RAM, 2 x 24-core Intel “Sapphire Rapids” host cores, 512GB host RAM (10GB\/core)<\/td>\nMax 12 CPU-cores per GPU<\/p>\n

22 nodes available = 88 GPUs<\/td>\n<\/tr>\n

-p gpuA40GB<\/code><\/td>\nNvidia A100 (40GB) GPU batch and interactive jobs. Optional: -n numcores<\/em><\/code>
\nRequired: -G NUM<\/em><\/code> or --gpus=NUM<\/em><\/code><\/td>\n		AMD Milan<\/td>\n4 days -t 4-0<\/code><\/td>\nNode spec: 4x Nvidia A100s, 40GB GPU RAM, 48x AMD “Milan” host cores, 512GB host RAM (10GB\/core)<\/td>\nVery Restricted Access<\/strong>
\nMax 12 CPU-cores per GPU, 2 nodes available = 8 GPUs<\/td>\n<\/tr>\n
-p gpuH<\/code><\/td>\nNvidia H200 (141 GB) GPU batch jobs. Optional: -n numcores<\/em><\/code> or -c numcores<\/em><\/code>
\nRequired: -A accountcode<\/em><\/code> -G NUM<\/em><\/code> or --gpus=NUM<\/em><\/code><\/td>\n		Intel Xeon Emerald Rapids<\/td>\n4 days -t 4-0<\/code><\/td>\nNode spec: 8x Nvidia H200, 141GB GPU RAM, 64 “Emerald Rapids” host cores, 1.5 TB host RAM (24GB\/core)<\/td>\nRestricted Access<\/strong>
\nMax 8 CPU-cores per GPU, up to 3 nodes available = 24 GPUs.
\nH200 docs<\/a>.<\/td>\n<\/tr>\n
-p gpuH_short<\/code><\/td>\nNvidia H200 (141 GB) GPU batch and interactive jobs. Optional: -n numcores<\/em><\/code> or -c numcores<\/em><\/code>
\nRequired: -A accountcode<\/em><\/code> -G NUM<\/em><\/code> or --gpus=NUM<\/em><\/code><\/td>\n		Intel Xeon Emerald Rapids<\/td>\n1 day -t 1-0<\/code><\/td>\nNode spec: 8x Nvidia H200, 141GB GPU RAM, 64 “Emerald Rapids” host cores, 1.5 TB host RAM (24GB\/core)<\/td>\nRestricted Access<\/strong>
\nMax 8 CPU-cores per GPU, up to 2 nodes available = 16 GPUs.
\nH200 docs<\/a>.<\/td>\n<\/tr>\n
HPC Pool<\/th>\n<\/tr>\n
-p hpcpool<\/code><\/td>\nMulti-node parallel (120-1000 cores) batch<\/strong> jobs. Required:<\/p>\n
-A hpc-proj-code<\/em>\r\n-N numnodes<\/em>\r\n-n numtasks<\/em><\/pre>\n
Optional:<\/p>\n
\r\n-c corespertask<\/em><\/pre>\n<\/td>\n
Intel Cascadelake<\/td>\n4 days -t 4-0<\/code><\/td>\nCascadelake nodes (40 cores, 4GB\/core)<\/td>\nRestricted Access<\/strong>
\n2000 cores available in total.
\nHPC Pool docs<\/a>.<\/td>\n<\/tr>\n
Interactive Jobs<\/th>\n<\/tr>\n
See interactive<\/a> above.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

*Information is correct as of 1st October 2026<\/em><\/p>\n

Jobscript Template<\/h2>\n

All jobs must specify the partition name (see above) and wallclock runtime limit (see the various time formats<\/a>) for the job.<\/p>\n

Note that the Slurm lines begin with #SBATCH<\/code> – not #$<\/code> as used on SGE. A common mistake is to use a $<\/code> symbol in the Slurm special line. It is NOT #$<\/strong>BATCH<\/code>.<\/p>\n

Single-node CPU Jobs<\/h3>\n
#!\/bin\/bash --login\r\n#SBATCH -p partitionname<\/em>    # Required (all jobs)<\/strong> - see table above\r\n#SBATCH -n numcores<\/em>         # Required (parallel jobs)<\/strong>  - defaults to 1 for serial\r\n#SBATCH -t timelimit<\/em>        # Required (all jobs)<\/strong> wallclock timelimit. Max permitted is 7 days (7-0)\r\n\r\n#SBATCH flag resources<\/em>      # See table above for additional high-memory, CPU architecture\r\n                            # and GPU resource flags.\r\n\r\n# We recommend purging your env\r\nmodule purge\r\nmodule load apps\/gcc<\/em>\/someapp<\/em>\/1.2.3<\/em>\r\n\r\n# $SLURM_NTASKS will be set to the number of cores requested above, if your app wants to know.\r\nsomeapp.exe<\/em>\r\n\r\n# Or, if running an MPI app, Slurm knows to run $SLURM_NTASKS MPI processes if using mpirun\r\nmpirun some_mpi_app.exe<\/em>\r\n<\/pre>\n
For OpenMP parallel applications you can use:<\/p>\n
#!\/bin\/bash --login\r\n#SBATCH -n 1           # 1 task ($SLURM_NTASKS set to 1)\r\n#SBATCH -c numcores<\/em>    # cores-per-task ($SLURM_CPUS_PER_TASK set to this)\r\n#SBATCH -t timelimit<\/em>   # Max wallclock time\r\n\r\n# Inform OpenMP how many cores to use\r\nexport OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK\r\nsome_omp_app.exe<\/em>\r\n<\/pre>\n
Multi-node CPU Jobs<\/h3>\n
Please note: the multinode partition is NOT available to free at point of use users (those in fatpou01) due to core limits applied to those users.<\/strong><\/p>\n
You must<\/strong> specify the -N numnodes<\/em><\/code> and -n numtasks<\/em><\/code> flags. You may also add the -c corespertask<\/em><\/code> flag.<\/p>\n
For a MPI-only application (no OpenMP capability), where you run an MPI process on every core of each node assigned to your job:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#SBATCH -N numnodes<\/em>    # Number of compute nodes (minimum is 2)\r\n#SBATCH -n numtasks<\/em>    # Number of MPI processes ($SLURM_NTASKS set to this)\r\n#SBATCH -t timelimit<\/em>   # Max wallclock time\r\n\r\nmodule purge\r\nmodule load apps\/gcc<\/em>\/someapp<\/em>\/1.2.3<\/em>\r\n\r\n# Slurm knows to run $SLURM_NTASKS MPI processes if using mpirun.\r\nmpirun some_mpi_app.exe<\/em>\r\n<\/pre>\n
For a “mixed-mode” MPI+OpenMP capable application, where you run a small number of MPI processes but each uses OpenMP to do multicore computation on the compute node:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#SBATCH -N numnodes<\/em>    # Number of compute nodes (minimum is 2)\r\n#SBATCH -n numtasks<\/em>    # Number of MPI processes ($SLURM_NTASKS set to this)\r\n#SBATCH -c numcores<\/em>    # cores-per-task ($SLURM_CPUS_PER_TASK set to this)\r\n#SBATCH -t timelimit<\/em>   # Max wallclock time\r\n\r\nmodule purge\r\nmodule load apps\/gcc<\/em>\/someapp<\/em>\/1.2.3<\/em>\r\n\r\n# Say how many cores each<\/em> MPI process should use for OpenMP processing\r\nexport OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK\r\n\r\n# Slurm knows to run $SLURM_NTASKS MPI processes if using mpirun.\r\nmpirun mpi-mapping-flags<\/em> some_mpi_app.exe<\/em>\r\n          #\r\n          # For one MPI process per node, use:\r\n          # --map-by ppr:1:node --bind-to none\r\n          #\r\n          # For two MPI process per node (one per socket), use:\r\n          # --map-by ppr:1:socket\r\n<\/pre>\n
GPU Jobs<\/h3>\n
Please see the GPU Jobs<\/a> page.<\/p>\n
High-memory Jobs<\/h3>\n
Please see the High Memory Jobs<\/a> page.<\/p>\n
Submitting Jobs and Checking your Queue<\/h3>\n
Submit the job using<\/p>\n
sbatch jobscript<\/em>\r\n<\/pre>\n
Check on the status of the job using<\/p>\n
squeue\r\n<\/pre>\n
More jobscript options<\/h3>\n
For much more information on Slurm jobscript options, compared to SGE options, please see the SGE to Slurm<\/a> reference.<\/p>\n","protected":false},"excerpt":{"rendered":"
In Slurm, jobs are submitted to partitions – think of these as job queues for different types of jobs. In your jobscript, you must specify the partition, number of cores and max “wallclock” time your job is allowed to run for (up to a permitted maximum time.) There is now no default wallclock time – you must specify it. You may also need to specify other flags depending on which parition you submit to. For.. Read more »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":9105,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-9123","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/9123","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=9123"}],"version-history":[{"count":24,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/9123\/revisions"}],"predecessor-version":[{"id":12881,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/9123\/revisions\/12881"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/9105"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=9123"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}