The Computational Shared Facility 3

Interactive Jobs (qrsh)

CPU-intensive interactive work must not be run on the login node at any time:

Interactive work, e.g., using CPU-intensive app through its GUI, must be scheduled using qrsh. This will log you in to a compute node where you can work at the command-line without using a batch script and without stressing the login node. The compute nodes have more memory and more CPU cores – your applications will run more responsively on a compute node.

Interactive jobs can also be used for quick pre/post processing commands where you may prefer to run an application directly rather than writing a job script. We do, however, recommend using batch jobs where possible. Interactive jobs require you to remain logged in to the CSF. A key advantage of batch jobs is that you can submit them to the batch system then log off and let the CSF get on with them.

It is possible to run serial core or multicore interactive sessions. The number of compute nodes available for interactive work is limited and so the qrsh command may not be able to give you an interactive session immediately. If asked, please try again later.

If you wish to run a GUI then you will need to ensure you have logged into the CSF with X11 enabled

Resource Limits

  • Interactive jobs can use up to 4GB of RAM per core.
  • The runtime limit of interactive jobs is 1 hour.

Serial (1-core) Interactive Jobs

To schedule a serial interactive job run the following command on the login node (not in a jobscript):

qrsh -l short

This will log you in to a compute node where you can run and the command-line. You will then need to use the module command to set up applications as usual.

Note: Unlike the CSF2, there is no need to add the -l inter flag to the qrsh command-line. The use of qrsh indicates you require an interactive job.

To run an application directly in an interactive session add its command to the qrsh line. For example:

# Run these commands directly on the login node:
module load name/of/app/1.2.3
qrsh -l short -V -cwd app.exe args...

This will run the application directly on a compute node in the current directory.

Parallel (multi-core) Interactive Jobs

To schedule a multi-core interactive job run the following command on the login node:

qrsh -l short -pe smp.pe 2
                         #
                         # Number of cores

You will be logged in to a compute node and will have access to 2 CPU cores (in this example) and 4GB per core. You should ensure any applications you run use only the number of cores reserved for you. For example you may have to use a command-line flag when running the application to inform it how many cores to use.

To run an application directly in a multi-core interactive session add its command to the qrsh line. For example:

# Run these commands directly on the login node:
module load name/of/app/1.2.3
qrsh -l short -V -cwd -pe smp.pe 2 app.exe args...

This will run the application directly on a compute node in the current directory with access to two cores (in this example).

Scheduling Messages/Errors

It is possible that you will see the following message when you run qrsh (see below for examples):

Your "qrsh" request could not be scheduled, try again later

This means either:

  • There are no cores free for interactive jobs (too many other interactive/short jobs are running)
  • You have reached the user job limit
  • You got something wrong on the command line (e.g. lower case v unstead of uppercase V)

This error often happens when the short environment is very busy, and you will need to try later. You should also consider whether you can convert your job to a batch job which requires no interaction or direct control from you while it runs.

Note: Unlike the CSF2, there is no need to add the -l inter flag to the qrsh command-line. The use of qrsh indicates you require an interactive job.

High Memory Interactive Jobs

If the normal qrsh nodes do not have sufficient memory for your work you can try adding

-l mem256

for 16GB per core or

-l mem512

for 32GB per core.

Note: due to the limited number of nodes of this type there is only one mem256 and one mem512 node available for this type of work.

Further Information

  • Further advice about running specific applications using qrsh may be available on the appropriate software page, for example, the idb and gdb debuggers.

Last modified on November 30, 2021 at 2:27 pm by Pen Richardson