{"id":11299,"date":"2025-11-03T10:47:02","date_gmt":"2025-11-03T10:47:02","guid":{"rendered":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=11299"},"modified":"2026-03-27T11:22:53","modified_gmt":"2026-03-27T11:22:53","slug":"job-dependencies","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/batch-slurm\/job-dependencies\/","title":{"rendered":"Job Dependencies (Slurm)"},"content":{"rendered":"

Slurm job dependencies allow you to specify that one job should not start until some other job(s) has met some condition. This is usually that the earlier job has finished successfully, but can be other things – see below.<\/p>\n

For example, this might be useful if ‘jobB’ relies on the output from ‘jobA’ – it saves you having to be aware of when individual jobs have started or completed. Instead, a job dependency allows Slurm to simply get on with running them in the order you specify. <\/p>\n

You can build up more complicated pipelines<\/em> of jobs, where you might be waiting for several jobs to finish before further jobs can run. Job dependencies will ensure jobs run in the correct order.<\/p>\n

Job Dependency via the sbatch command-line<\/h2>\n

To setup a job dependency, specify the type of dependency and the jobid of an already submitted<\/em> that you want the current<\/em> job submission to depend on.<\/p>\n

Note: If adding the -d, --dependency<\/code> flag to the sbatch<\/code> command-line, you must put the flags before<\/em> the jobscript name. Any flags after the jobscript are ignored:<\/p>\n

\r\nsbatch flags<\/em> jobscript\r\n        #\r\n        # Command-line flags must come before<\/em> the jobscript otherwise they will be ignored.\r\n<\/pre>\n
Simple Examples<\/h3>\n
Here we give some simply examples, as these cover many common situations. See below for a complete list of the available dependency types.<\/p>\n
\r\n# Note: You can use the short-form \"-d deplist<\/em>\" or long-form \"--dependency=<\/em>deplist<\/em>\" flag\r\n\r\n# A single job waiting for a single earlier job (123456) to finish successfully<\/em><\/strong>:\r\nsbatch --dependency=afterok<\/strong>:123456 myjobscript<\/em>\r\n\r\n# A single job waiting for a single earlier job (123456) to start<\/strong>:\r\nsbatch --dependency=after<\/strong>:123456 myjobscript<\/em>\r\n\r\n# A single job waiting for a single earlier job (123456) to finish with any state<\/strong> (success, error, ...)\r\nsbatch --dependency=afterany<\/strong>:123456 myjobscript<\/em>\r\n                       #\r\n                       # Note: afterany refers to the state<\/em> of the earlier job.\r\n                       # It is not referring to multiple dependencies. See below.\r\n\r\n# A single job waiting for BOTH earlier jobs (123456, 123789) to finish successfully<\/em><\/strong>:\r\nsbatch --dependency=afterok<\/strong>:123456:123789 myjobscript<\/em>\r\n\r\n# Multiple jobs waiting for a single earlier job (123456) to finish successfully<\/em><\/strong>:\r\nsbatch --dependency=afterok<\/strong>:123456 myjobscriptA<\/em>\r\nsbatch --dependency=afterok<\/strong>:123456 myjobscriptB<\/em>\r\n<\/pre>\n
Some more complex examples:<\/p>\n
\r\n# A single job waiting for several earlier jobs that all have the same job name (from the same user).\r\n# Note that the earlier jobs must only run one at a time, hence have their own dependencies.\r\nJOBA=$(sbatch --parsable --job-name=mypipeline<\/em><\/strong> jobscriptA)\r\nJOBB=$(sbatch --parsable --job-name=mypipeline<\/em><\/strong> --dependency=afterok:$JOBA<\/strong> jobscriptB)\r\nJOBC=$(sbatch --parsable --job-name=mypipeline<\/em><\/strong> --dependency=afterok:$JOBB<\/strong> jobscriptC)\r\nsbatch --dependency=singleton --job-name=mypipeline<\/em><\/strong> jobscriptD\r\n                       #\r\n                       # This one job will wait for the three earlier jobs\r\n\r\n# Wait for BOTH dependencies with different dependency types\r\nsbatch --dependency=afterok<\/strong>:123456,afternotok<\/strong>:123789 myjobscript<\/em>\r\n                                  #\r\n                                  # Use a , to mean both<\/em> dependencies\r\n\r\n# Wait for ANY dependencies for jobs (123456, 123789)\r\nsbatch --dependency=afterok<\/strong>:123456?afterok<\/strong>:123789 myjobscript<\/em>\r\n                                  #\r\n                                  # Use a ? to mean any<\/em> dependencies\r\n<\/pre>\n
Pipelines with Job Arrays<\/h3>\n
To construct dependencies between job arrays, to build up multi-task pipelines, please see the Slurm Job Arrays page<\/a>.<\/p>\n
Capturing the Job ID on the command line<\/h3>\n
You can specify the job dependency flag in a jobscript using #sbatch --dependency=...<\/code>, but will find it easier on the sbatch<\/code> command-line (due to the need to use a previous jobid). <\/p>\n
Add the --parsable<\/code> flag to your sbatch<\/code> command to report only the jobid:<\/p>\n
\r\n# Normally you get a user-friendly message when submitting a job:\r\n[username<\/em>@login1[csf3] ~]$  sbatch myjobscript\r\nSubmitted batch job 123456\r\n\r\n# Instead, we only want the jobid\r\n[username<\/em>@login1[csf3] ~]$ sbatch --parsable myjobscript\r\n123456\r\n<\/pre>\n
You can then capture the jobid using the BASH shell scripting language:<\/p>\n
\r\n# Submit the job on which you want later jobs to depend, capturing its jobid\r\nJOBID=$(sbatch --parsable firstjobscript<\/em>)\r\n\r\n# Now submit the job that will depend on the previous job\r\nsbatch --dependency=afterok:${JOBID} secondjobscript<\/em>\r\n<\/pre>\n
You can repeat the above procedure to submit multiple jobs as a pipeline:<\/p>\n
\r\n# A pipeline of jobs:\r\n#\r\n# JobA ------> JobB ------> JobC\r\n\r\nJOBID=$(sbatch --parsable JobA)\r\nJOBID=$(sbatch --parsable --dependency=afterok:${JOBID} JobB)\r\nJOBID=$(sbatch --parsable --dependency=afterok:${JOBID} JobC)\r\n<\/pre>\n
The Dependency Flag<\/h2>\n
The --dependency=deplist<\/em><\/code> flag is of the form:<\/p>\n