Job Dependencies

SGE job dependencies allow you to specify that one job should not start before another has completed. For example, this might be useful if ‘jobB’ relies on the output from ‘jobA’ and it saves you having to be aware of when individual jobs have started or completed because SGE will simply get on with running them in the order you specify. To use this functionality add this option to your submission script:

#$ -hold_jid jobid
               #
               # ...replacing 'jobid' with the number of the job to wait for.
               #

When you type qstat the jobs that are waiting on others will be listed as hqw.
It may be easier to set up job dependencies by naming jobs rather than specifying jobIDs. For example, the first job can have the following added to its jobscript:

#$ -N myFirstJob

This will name the job (and cause its output .o and .e files to use that name). The second job which is going to wait for the first job can now refer to it by name:

#$ -hold_jid myFirstJob

Our job dependency is now independent of the jobID number the batch system assigns to the first job.

Alternatively you can add the -hold_jid flag to the qsub command-line:

# Submit the first job
qsub jobscript1.sh
Your job 129673 ("jobscript1.sh") has been submitted
           #
           # Make a note of the job id

# Now submit the second job and make it wait for the first
qsub -hold_jid 129673 jobscript2.sh

By adding the flag to the qsub command-line we don’t have to edit the second jobscript to set the unique jobid to wait for. We don’t need to name the jobs either.

If your job needs to wait for multiple jobs to finish you may specify a comma separated list of jobIDs (or names) on the -hold_jid line. For example

# Wait for three earlier jobs (which were given -N names) to
# finish before we proceed

#$ -hold_jid myFirstJob,SimWork,dataJob

Jobarray Dependencies

There are a number of ways in which job dependencies can be used with job arrays. Please see our Job Array Information for further details.