{"id":8625,"date":"2025-01-28T10:36:10","date_gmt":"2025-01-28T10:36:10","guid":{"rendered":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=8625"},"modified":"2026-03-27T11:03:27","modified_gmt":"2026-03-27T11:03:27","slug":"sge-to-slurm","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/batch-slurm\/sge-to-slurm\/","title":{"rendered":"SGE to Slurm Reference"},"content":{"rendered":"
\nThis page is for users who have previously used the CSF3 when it was running the SGE batch system. It will help you to convert your SGE jobscripts to Slurm jobscripts.<\/p>\n

New users of the CSF from May 2025 onwards can ignore this – you have only ever been using the Slurm batch system.\n<\/p><\/div>\n

Introduction<\/h2>\n

The use of Slurm on CSF3 represents a significant change for CSF users who are used to using the SGE batch system.<\/p>\n

While SGE has served us well, Slurm has been widely adopted by many other HPC sites, is under active development and has features and flexibility that we need as we introduce new platforms for the research community at the University.<\/p>\n

This page shows the Slurm commands and jobscript options next to their SGE counterparts to help you move from SGE to Slurm.<\/p>\n

How can I tell which system I’m using?<\/h3>\n

It is now only possible to access to CSF3 Slurm system. <\/p>\n

On the login node, you will see the csf3<\/code> system name in green:<\/p>\n

\r\n[mabcxyz1<\/em>@login1[csf3] ~]$  # The upgraded CSF3 uses green in the prompt\r\n<\/pre>\n
What are the key differences?<\/h3>\n
    \n
  1. Slurm uses sbatch<\/code>, squeue<\/code>, scancel<\/code> and srun<\/code> commands, not qsub<\/code>, qstat<\/code>, qdel<\/code> and qrsh<\/code>. See below<\/a>.<\/li>\n
  2. The jobscript sentinel<\/em> is #SBATCH<\/code>, not #$<\/code>. See below<\/a>.<\/li>\n
  3. You must specify explicitly the serial<\/code> partition to run a 1-core job. See below<\/a>.<\/li>\n
  4. The parallel environments<\/em>, called “partitions” in Slurm, have new names. See below<\/a>.<\/li>\n
  5. You MUST specify a job wallclock time for your job! There is NO default value applied to a job. The maximum permitted is 7 days in most cases (GPU jobs can have a maximum of 4 days.) Your job will be rejected if you don’t give it a wallclock time<\/strong>. See below<\/a>.<\/li>\n
  6. Job output files (the .o and .e files in SGE) are merged by default, and have a different naming convention, in Slurm. See below<\/a>.<\/li>\n
  7. The jobscript flags (-pe, -l v100=1 and so on) have changed.<\/li>\n
  8. The jobscript variables ($JOB_ID<\/code>, $NSLOTS<\/code> and so on) have changed. See below<\/a>.<\/li>\n<\/ol>\n
    Please see below for more details on how to use Slurm, compared to SGE.<\/p>\n
    Jobscript Special Lines – SGE (#$) vs Slurm (#SBATCH)<\/h2>\n
    Your SGE jobscripts will not<\/em> run in the Slurm batch system. This is because the SGE jobscript special lines<\/em> beginning with #$<\/code> will be ignored by Slurm<\/strong>. Instead, you should use lines beginning with #SBATCH<\/code>, and will need to change the options you use on those lines.<\/p>\n
    Note that it is #S<\/strong>BATCH<\/code> (short for Slurm BATCH) and NOT #$<\/strong>BATCH<\/code>. This is an easy mistake to make when you begin to modify your SGE jobscripts. Do not use a $ (dollar) symbol in the Slurm special lines.<\/strong><\/div>\n
    Can I use a single jobscript for both SGE and Slurm?<\/h3>\n
    It is possible to have both SGE and Slurm lines in your jobscripts – they will each ignore the other’s special lines.<\/p>\n
    However, there are some differences in the way multi-core and multi-node jobs are run, so we advise writing new jobscripts for use on the upgraded CSF3.<\/p>\n
    One suggestion is to name your Slurm jobscripts jobscript<\/em>.sbatch<\/code> and your SGE jobscripts jobscript<\/em>.qsub<\/code>, but you can, of course, use any naming scheme you like.<\/p>\n
    Examples of SGE jobscripts and their equivalent Slurm jobscript are given below.<\/p>\n
    The commands used to submit jobs and check on the queue have also changed. See below for the equivalent commands.<\/p>\n
    Command-line tools – SGE (qsub, …) vs Slurm (sbatch, …)<\/h2>\n\n\n\n\n\n
    SGE Commands (old CSF3)<\/th>\nSlurm Commands (new CSF3)<\/th>\n<\/tr>\n<\/thead>\n
    \n
    # Batch job submission\r\nqsub jobscript<\/em>\r\nqsub jobscript<\/em> arg1 arg2 ...\r\nqsub options<\/em> -b y executable arg1 ...<\/em>\r\n\r\n# Job queue status\r\nqstat             # Show your jobs (if any)\r\nqstat -u \"*\"      # Show all jobs\r\nqstat -u username<\/em>\r\n\r\n# Cancel (delete) a job\r\nqdel jobid<\/em>\r\nqdel jobname<\/em>\r\nqdel jobid<\/em> -t taskid<\/em>\r\nqdel \"*\"             # Delete all my jobs\r\n\r\n# Interactive job\r\nqrsh -l short\r\n\r\n# Completed job stats\r\nqacct -j jobid<\/em>\r\n<\/pre>\n<\/td>\n
    		\n
    # Batch job submission\r\nsbatch jobscript<\/em>\r\nsbatch jobscript<\/em> arg1 arg2 ...\r\nsbatch options<\/em> --wrap=\"executable arg1 ...<\/em>\"\r\n\r\n# Job queue status\r\nsqueue      # An alias for \"squeue --me\"\r\n\\squeue     # Unaliased squeue shows all jobs\r\nsqueue -u username<\/em>\r\n\r\n# Cancel (delete) a job\r\nscancel jobid<\/em>\r\nscancel -n jobname<\/em>\r\nscancel jobid<\/em>_taskid<\/em>\r\nscancel -u $USER       # Delete all my jobs\r\n\r\n# Interactive job (1 hour session)\r\nsrun -p interactive -t 1:00:00 --pty bash\r\n\r\n# Completed job stats\r\nsacct -j jobid<\/em>\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    Job Output Files (stdout and stderr)<\/h2>\n\n\n\n\n\n
    SGE job output files, not merged by default (old CSF3)<\/th>\nSlurm job output files, merged by default (new CSF)<\/th>\n<\/tr>\n<\/thead>\n
    \n
    # Individual (non-array) jobs\r\njobscriptname<\/em>.oJOBID<\/em>\r\njobscriptname<\/em>.eJOBID<\/em>\r\n\r\n# Array jobs\r\njobscriptname<\/em>.oJOBID<\/em>.TASKID<\/em>\r\njobscriptname<\/em>.eJOBID<\/em>.TASKID<\/em>\r\n<\/pre>\n<\/td>\n
    		\n
    # Individual (non-array) jobs\r\nslurm-JOBID<\/em>.out\r\n\r\n\r\n# Array jobs (see later for more details)\r\nslurm-ARRAYJOBID_TASKID<\/em>.out\r\n\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    The Slurm files contain the normal and error output that SGE splits in to two files.<\/p>\n
    The naming and merging of the files can be changed using jobscript options (see below<\/a>) but for now, in the basic jobscripts shown next, we’ll just accept these default names to keep the jobscripts short.<\/p>\n
    Jobscripts<\/h2>\n
    You will need to rewrite your SGE jobscripts. You could name them somename<\/em>.sbatch<\/code> if you like, to make it obvious it is a Slurm jobscript.<\/p>\n
    Put #SBATCH lines in one block<\/h3>\n
    Please note: all Slurm special lines<\/em> beginning with #SBATCH<\/code> must come before ordinary lines that run Linux commands or your application. Any #SBATCH<\/code> lines appearing after the first non-#SBATCH<\/code> line will be ignored. For example:<\/p>\n
    #!\/bin\/bash --login\r\n\r\n# You may put comment lines before and after #SBATCH lines\r\n#SBATCH -n 4\r\n#SBATCH -c 2\r\n\r\n# Now the first 'ordinary' line. So no more #SBATCH lines allowed after here\r\nexport MY_DATA=~\/scratch\/data\r\n# We recommend that you purge your module environment before loading further modules in Slurm jobs\r\nmodule purge\r\nmodule load myapp<\/em>\/1.2.3<\/em>\r\n\r\n# Any SBATCH lines here will be ignored!<\/strong>\r\n#SBATCH --job-name  new_job_name\r\n\r\n.\/my_app dataset1.dat\r\n<\/pre>\n
    Partitions – required!<\/h3>\n
    On the upgraded CSF3 running Slurm, you are required<\/em> to specify, in your jobscript, the Slurm partition<\/em> in which your job will run. Think of these as like queues for different types of jobs.<\/p>\n
    For example, there is a partition for the AMD nodes running parallel jobs. There’s a partition for serial jobs, and so on.<\/p>\n
    The current list of Slurm partitions<\/em> (and the compute resources they offer) is available at the Partitions page<\/a>.<\/p>\n
    Job Wallclock (time) Limits – now required!<\/h3>\n
    On the upgraded CSF3 running Slurm, you are required<\/em> to specify the maximum wallclock time limit for your job.<\/p>\n
    Previously, on the old CSF3 running SGE, this was optional and would default to the maximum of 7 days (or 4 days for GPU and HPC Pool jobs.)<\/p>\n
    Why the change? In short, more accurate wallclock times, specified by you, will help with scheduling your job.<\/p>\n
    In particular, if you know your job will complete within 1 hour, or 1 day, say, by specifying this wallclock limit, the scheduler may be able to squeeze your job onto a compute node that is being held by the scheduler for a much larger, long running job. Slurm will try to use cores by running short jobs that it knows could be left idle while a larger job is waiting for resources.<\/p>\n
    Of course, you can still specify a maximum wallclock of 7 days, even if your job won’t actually take that long to complete. But you might wait longer in the queue.<\/p>\n
    You don’t need to be super-accurate<\/strong>, and it is much better to give your job plenty of time to complete<\/strong>. If you don’t give the job enough time, Slurm will kill the job before<\/em> it completes. So we recommend specifying a number of days.<\/p>\n
    To specify a wallclock time in a Slurm job:<\/p>\n
    #SBATCH -t timelimit<\/em>\r\n## OR\r\n#SBATCH --time=timelimit<\/em>\r\n<\/pre>\n
    where the following are examples of acceptable formats for the timelimit<\/code><\/p>\n
    #SBATCH -t 10            # minutes\r\n#SBATCH -t 10:30         # minutes:seconds\r\n#SBATCH -t 4:20:00       # hours:minutes:seconds\r\n#SBATCH -t 4-0           # days-hours              # Recommended format<\/strong>\r\n#SBATCH -t 4-10:00       # days-hours:minutes\r\n#SBATCH -t 5-8:15:30     # days-hours:minutes:seconds\r\n<\/pre>\n
    If you fail to specify a wallclock time limit, you will get the following error when submitting a job:<\/p>\n
    sbatch: error: Batch job submission failed: Requested time limit is invalid (missing or exceeds some limit)\r\n<\/pre>\n
    Add the -t<\/code> flag and resubmit the job.<\/p>\n
    Basic Serial (1-core) Josbscript<\/h3>\n
    A basic serial job requires the serial<\/code> partition to be specified, and a wallclock timelimt.<\/p>\n
    #!\/bin\/bash --login\r\n#SBATCH -p serial     # Partition name is required<\/strong> (serial will default to 1 core)\r\n#SBATCH -t 4-0        # Job \"wallclock\" limit is required<\/strong>. Max permitted is 7 days (7-0).\r\n                      # In this example 4-0 is 4 days (0 hours).\r\n                      # Other formats:  min:sec, hrs:min:sec, day-hours (to name a few)\r\n# Clean env\r\nmodule purge\r\nmodule load apps\/binapps\/someapp\/1.2.3\r\n\r\n# Use $SLURM_NTASKS if you need to say how many cores are being used (1 in this case)\r\nsomeapp.exe<\/em>\r\n<\/pre>\n
    Basic Multi-core (single AMD 168-core compute node) Parallel Jobscript<\/h3>\n
    The following will run on an AMD 168-core Genoa node, using 96 cores, for 4.5 days. Please see the full list of available partitions<\/em><\/a> for more details about the compute resources.<\/p>\n
    #!\/bin\/bash --login\r\n#SBATCH -p multicore   # Partition name is required<\/strong>. This gives you the AMD Genoa (168-core) nodes.\r\n#SBATCH -n 96          # (or --ntasks=) Number of cores (2--168 on AMD)\r\n#SBATCH -t 4-12        # Job \"wallclock\" limit is required<\/strong>. Max permitted is 7 days (7-0).\r\n                       # In this example 4-12 is 4 days 12 hours (4.5 days).\r\n\r\n# Clean env\r\nmodule purge\r\nmodule load apps\/binapps\/some_mpi_app\/1.2.3\r\n\r\n# Slurm knows how many cores to use for mpirun\r\nmpirun someMPIapp.exe<\/em>\r\n<\/pre>\n
    If running OpenMP parallel jobs you can also use the following jobscript, which requests one Slurm task<\/em> and several cores-per-task<\/em>:<\/p>\n
    #!\/bin\/bash --login\r\n#SBATCH -p multicore\r\n#SBATCH -n 1               # One \"task\" (sets $SLURM_NTASKS to 1)\r\n#SBATCH -c 84              # 84 cores-per-task (sets $SLURM_CPUS_PER_TASK to 84)\r\n#SBATCH -t 4-12            # 4 days 12 hours\r\n\r\nmodule purge\r\nmodule load apps\/binapps\/some_omp_app\/1.2.3\r\n\r\n# Inform OpenMP how many cores to use\r\nexport OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK\r\nsomeOMPapp.exe<\/em>\r\n<\/pre>\n
    Basic Multi-node Parallel Jobscript<\/h3>\n
    Multi-node jobs can only be run on the HPC Pool nodes. Please see how to run HPC Pool jobs<\/a>.<\/p>\n
    Basic Job Array Jobscript<\/h3>\n
    The following will run a 1000 task job array, where each task uses 2 cores on an AMD node, and each task can run for 2 days<\/p>\n
    #!\/bin\/bash --login\r\n#SBATCH -p multicore   # Partition name is required<\/strong>\r\n#SBATCH -n 2           # (or --ntasks=) Number of cores (2--168 on AMD)\r\n#SBATCH -t 2-0         # Job \"wallclock\" limit is required<\/strong>. Max permitted is 7 days (7-0).\r\n                       # In this example each<\/em> job array task gets 2 days (0 hours).\r\n#SBATCH -a 1-1000      # 1000 tasks in the job array, numbered 1..1000.\r\n                       # (NB: tasks can begin from 0 in Slurm, unlike SGE)\r\n\r\n# Clean env\r\nmodule purge\r\nmodule load apps\/gcc\/someapp\/1.2.3\r\n\r\n# Use ${SLURM_ARRAY_TASK_ID} to get the task number (1,2,...,1000 in this example)\r\n.\/myprog -in data.${SLURM_ARRAY_TASK_ID}.dat -out results.${SLURM_ARRAY_TASK_ID}.dat\r\n               #\r\n               # My input files are named: data.1.dat, data.2.dat, ..., data.1000.dat\r\n               # 1000 tasks (copies of this job) will run.\r\n               # Task 1 will read data.1.dat, task 2 will read data.2.dat, ... \r\n<\/pre>\n
    <\/a><\/p>\n
    More Jobscript options – SGE vs Slurm<\/h2>\n
    Here are some more example jobscripts special lines for achieving things in SGE and Slurm.<\/p>\n
    Renaming a job and the output .o and .e files<\/h3>\n\n\n\n\n\n
    SGE Jobscript<\/th>\nSlurm Jobscript<\/th>\n<\/tr>\n<\/thead>\n
    \n
    #!\/bin\/bash --login\r\n...\r\n# Naming the job is optional.\r\n# Default<\/strong> is name of jobscript<\/em>\r\n# DOES<\/strong> rename .o and .e output files.\r\n#$ -N jobname\r\n\r\n# Naming the output files is optional.\r\n# Default<\/strong> is separate<\/strong> .o and .e files:\r\n# jobname<\/em>.oJOBID<\/em><\/strong> and jobname<\/em>.eJOBID<\/em><\/strong>\r\n# Use of '-N jobname' DOES<\/strong> affect those defaults\r\n#$ -o myjob.out\r\n#$ -e myjob.err\r\n\r\n# To join .o and .e in to a single file\r\n# similar to Slurm's default behaviour:\r\n#$ -j y\r\n<\/pre>\n<\/td>\n
    		\n
    #!\/bin\/bash --login\r\n...\r\n# Naming the job is optional.\r\n# Default<\/strong> is name of jobscript<\/em>\r\n# Does NOT<\/strong> rename .out file.\r\n#SBATCH -J jobname\r\n\r\n# Naming the output files is optional.\r\n# Default<\/strong> is a single file<\/strong> for .o and .e:\r\n# slurm-JOBID<\/em>.out<\/strong>\r\n# Use of '-J jobname' does NOT<\/strong> affect the default\r\n#SBATCH -o myjob.out\r\n#SBATCH -e myjob.err\r\n\r\n# Use wildcards to recreate the SGE names\r\n#SBATCH -o %x.o%j      # %x = SLURM_JOB_NAME\r\n#SBATCH -e %x.e%j      # %j = SLURM_JOB_ID\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    The $SLURM_JOB_NAME<\/code> variable will tell you the name of your jobscript, unless the -J jobname<\/em><\/code> variable is used to rename your job. Then the env var is set to the value of jobname<\/em>.<\/p>\n
    If you wanted to use $SLURM_JOB_NAME<\/code> to always give you the name of the jobscript from within your job, you would have to remove the -J<\/code> flag. However, the following command run inside your jobscript will give you the name of the jobscript regardless of whether you use the -J<\/code> flag or not:<\/p>\n
    scontrol show jobid $SLURM_JOB_ID | grep Command= | awk -F\/ '{print $NF}'\r\n<\/pre>\n
    Renaming an array job output .o and .e files<\/h3>\n
    An array job uses slurm-ARRAYJOBID_TASKID<\/em>.out<\/code> as the default output file for each task in the array job. This can be renamed but you need to use the %A<\/code> and %a<\/code> wildcards (not %j<\/code>).<\/p>\n\n\n\n\n\n
    SGE Jobscript<\/th>\nSlurm Jobscript<\/th>\n<\/tr>\n<\/thead>\n
    \n
    #!\/bin\/bash --login\r\n...\r\n# An array job (cannot start at 0)\r\n#$ -t 1-1000\r\n\r\n# Naming the job is optional.\r\n# Default<\/strong> is name of jobscript<\/em>\r\n#$ -N jobname\r\n\r\n# Naming the output files is optional.\r\n# Default<\/strong> is separate .o and .e files:\r\n# jobname<\/em>.oJOBID<\/em><\/strong> and jobname<\/em>.eJOBID<\/em><\/strong>\r\n# Use of '-N jobname' DOES<\/strong> affect those defaults\r\n\r\n# To join .o and .e in to a single file\r\n# similar to Slurm's default behaviour:\r\n#$ -j y\r\n\r\n<\/pre>\n<\/td>\n
    		\n
    #!\/bin\/bash --login\r\n...\r\n# An array job (CAN start at 0)\r\n#SBATCH -a 0-999     # (or --array=0-999)\r\n\r\n# Naming the job is optional.\r\n# Default<\/strong> is name of jobscript<\/em>\r\n#SBATCH -J jobname\r\n\r\n# Naming the output files is optional.\r\n# Default<\/strong> is a single file<\/strong> for .o and .e:\r\n# slurm-ARRAYJOBID_TASKID<\/em>.out<\/strong>\r\n# Use of '-J jobname' does NOT<\/strong> affect the default\r\n\r\n# Use wildcards to recreate the SGE names\r\n#SBATCH -o %x.o%A.%a # %x=SLURM_JOB_NAME\r\n#SBATCH -e %x.e%A.%a # %A=SLURM_ARRAY_JOB_ID\r\n                     # %a=SLURM_ARRAY_TASK_ID\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    Emailing from a job<\/h3>\n
    SLURM can email you when your job begins, ends or fails.<\/p>\n\n\n\n\n\n
    SGE Jobscript<\/th>\nSlurm Jobscript<\/th>\n<\/tr>\n<\/thead>\n
    \n
    #!\/bin\/bash --login\r\n...\r\n# Mail events: b<\/strong>egin, e<\/strong>nd, a<\/strong>bort\r\n#$ -m bea\r\n#$ -M e.addr@manchester.ac.uk\t\r\n<\/pre>\n<\/td>\n
    		\n
    #!\/bin\/bash --login\r\n...\r\n# Mail events: NONE, BEGIN, END, FAIL, ALL\r\n#SBATCH --mail-type=BEGIN,END,FAIL\r\n#SBATCH --mail-user=e.addr@manchester.ac.uk\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    Note that in Slurm, array jobs only send one email, not an email per job-array tasks as happens in SGE. If you want an email from every job-array task, add ARRAY_TASKS<\/code> to the --mail<\/code> flag:<\/p>\n
    #SBATCH --mail-type=ALL,ARRAY_TASKS\r\n                            #\r\n                            # DO NOT USE IF YOUR ARRAY JOB CONTAINS MORE THAN\r\n                            # 20 TASKS!! THE UoM MAIL ROUTERS WILL BLOCK THE CSF!\r\n<\/pre>\n
    But please be aware that you will receive A LOT of email if you run a large job array with this flag enabled.<\/p>\n
    Job Environment Variables<\/h2>\n
    A number of environment variables are available for use in your jobscripts – these are sometimes useful when creating your own log files, for informing applications how many cores they are allowed to use (we’ve already seen $SLURM_NTASKS<\/code> in the examples above), and for reading sequentially numbered data files in job arrays.<\/p>\n\n\n\n\n\n
    SGE Environment Variables<\/th>\nSlurm Environment Variables<\/th>\n<\/tr>\n<\/thead>\n
    \n
    $NSLOTS             # Num cores reserved\r\n\r\n$JOB_ID             # Unique jobid number\r\n$JOB_NAME           # Name of job\r\n\r\n# For array jobs\r\n$JOB_ID             # Same for all tasks\r\n                    # (e.g, 2173<\/em>)\r\n\r\n\r\n$SGE_TASK_ID        # Job array task number*<\/sup>\r\n                    # (e.g., 1,2,3,...)\r\n$SGE_TASK_FIRST     # First task id\r\n$SGE_TASK_LAST      # Last task id\r\n$SGE_TASK_STEPSIZE  # Taskid increment\r\n                    # (default: 1)\r\n\r\n\r\n# Number of GPUs in GPU job\r\n$NGPUS\r\n\r\n# You will be unlikely to use these:\r\n$PE_HOSTFILE        # Multinode host list\r\n$NHOSTS             # Number of nodes in use\r\n$SGE_O_WORKDIR      # Submit directory\r\n<\/pre>\n<\/td>\n
    		\n
    $SLURM_NTASKS        # Num cores from -n flag\r\n$SLURM_CPUS_PER_TASK # Num cores from -c flag\r\n$SLURM_JOB_ID        # Unique job id number\r\n$SLURM_JOB_NAME      # Name of job\r\n\r\n# For array jobs\r\n$SLURM_JOB_ID        # NOT SAME FOR ALL TASKS<\/strong> \r\n                     # (eg: 2173<\/em>,2174<\/em>,2175<\/em>)\r\n$SLURM_ARRAY_JOB_ID  # SAME <\/strong>for all tasks\r\n                     # (eg: 2173<\/em>)\r\n$SLURM_ARRAY_TASK_ID # Job array task number*<\/sup>\r\n                     # (eg: 1,2,3,...)\r\n$SLURM_ARRAY_TASK_MIN   # First task id\r\n$SLURM_ARRAY_TASK_MAX   # Last task id\r\n$SLURM_ARRAY_TASK_STEP  # Taskid Increment\r\n                        # (default: 1)\r\n$SLURM_ARRAY_TASK_COUNT # Number of tasks\r\n\r\n# Number of GPUs in GPU job\r\n$SLURM_GPUS\r\n\r\n# You will be unlikely to use these:\r\n$SLURM_JOB_NODELIST  # Multinode host list\r\n$SLURM_JOB_NUM_NODES # Number of nodes in use\r\n$SLURM_SUBMIT_DIR    # Submit directory\r\n<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    Note (*)<\/strong>: SGE job-array task IDs are NOT allowed to start from zero – they must start from 1 (or higher). Slurm job array task IDs can<\/em> start from zero – it is up to you to specify the starting task id.<\/p>\n
    Many more environment variables are available for use in your jobscript. The Slurm sbatch manual<\/a> (also available on the CSF login node by running man sbatch<\/code>) documents Input<\/em> and Output<\/em> environment variables. The input<\/em> variables can be set by you before<\/em> submitting a job to set job options (although we recommend not<\/em> doing this – it is better to put all options in your jobscript so that you have a permanent record of how you ran the job). The output<\/em> variables can be used inside your jobscript to get information about the job (e.g., number of cores, job name and so on – we have documented several of these above.)<\/p>\n","protected":false},"excerpt":{"rendered":"
    This page is for users who have previously used the CSF3 when it was running the SGE batch system. It will help you to convert your SGE jobscripts to Slurm jobscripts. New users of the CSF from May 2025 onwards can ignore this – you have only ever been using the Slurm batch system. Introduction The use of Slurm on CSF3 represents a significant change for CSF users who are used to using the SGE.. Read more »<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":9105,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-8625","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/8625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=8625"}],"version-history":[{"count":21,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/8625\/revisions"}],"predecessor-version":[{"id":12192,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/8625\/revisions\/12192"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/9105"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=8625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}