Job Dependencies

SGE job dependencies allow you to specify that one job should not start before another has completed. For example, this might be useful if ‘jobB’ relies on the output from ‘jobA’. It saves you having to be aware of when individual jobs have started or completed because SGE will simply get on with running them in the order you specify. To use this functionality add this option to your submission script:

#$ -hold_jid jobid
               #
               # ...replacing jobid with the number of the job to wait for.
               #

When you type qstat the jobs that are waiting on others will be listed as hqw.

It may be easier to set up job dependencies by naming jobs rather than specifying jobIDs. For example, the first job can have the following added to its jobscript:

#$ -N myFirstJob

This will name the job (and cause its output .o and .e files to use that name). The second job which is going to wait for the first job can now refer to it by name:

#$ -hold_jid myFirstJob

Our job dependency is now independent of the jobID number the batch system assigns to the first job.

If your job needs to wait for multiple jobs to finish you may specify a comma separated list of jobIDs (or names) on the -hold_jid line. For example

# Wait for three earlier jobs (which were given -N names) to
# finish before we proceed

#$ -hold_jid myFirstJob,SimWork,dataJob