Interactive Jobs (qrsh)

Introduction

CPU-intensive interactive work must not be run on the login node at any time:

Interactive work, e.g., using a CPU-intensive app through its GUI, must be scheduled using qrsh. This will log you in to a compute node where you can work at the command-line without using a batch script and without stressing the login node. The compute nodes have more memory and more CPU cores – your applications will run more responsively on a compute node.

Interactive jobs can also be used for quick pre/post processing commands where you may prefer to run an application directly rather than writing a job script. We do, however, recommend using batch jobs where possible. Interactive jobs require you to remain logged in to the CSF. A key advantage of batch jobs is that you can submit them to the batch system then log off and let the CSF get on with them.

It is possible to run serial core or multicore interactive sessions. The number of compute nodes available for interactive work is limited and so the qrsh command may not be able to give you an interactive session immediately. If asked, please try again later.

If you wish to run a GUI then you will need to ensure you have logged into the CSF with X11 enabled

Resource Limits

• Interactive jobs can use up to 4GB of RAM per core.
• The runtime limit of interactive jobs is 1 hour.
• There are only two nodes that can run interactive work. Please DO NOT run production work on the short queue.

See bottom of page for higher-memory options.

Serial (1-core) Interactive Jobs

To schedule a serial interactive job, run the following command on the login node (not in a jobscript):

# On the CSF login node:
qrsh -l short
  #
  # Wait to be logged in to a compute node:

# Your prompt will change to show a compute node number. For example:
[username@node694 [csf3] ~]$

Once you’ve been logged in to a compute node you can run applications interactively at the command-line. You will need to use the module command to set up applications as usual.

To run an application directly in an interactive session, add its command to the qrsh line. For example:

# Run these commands directly on the login node:
module load name/of/app/1.2.3
qrsh -l short -V -cwd app.exe args...
                        #
                        # If you specify the -V and -cwd flags you must also specify
                        # an executable name. If you just want a shell, use 'bash' as the
                        # the executable name.

This will run the application directly on a compute node in the current directory.

Parallel (multi-core) Interactive Jobs

To schedule a multi-core interactive job run the following command on the login node:

qrsh -l short -pe smp.pe 2
                         #
                         # Number of cores

You will be logged in to a compute node and will have access to 2 CPU cores (in this example) and 4GB per core. You should ensure any applications you run use only the number of cores reserved for you. For example you may have to use a command-line flag when running the application to inform it how many cores to use.

To run an application directly in a multi-core interactive session add its command to the qrsh line. For example:

# Run these commands directly on the login node:
module load name/of/app/1.2.3
qrsh -l short -V -cwd -pe smp.pe 2 app.exe args...
                                      #
                                      # If you specify the -V and -cwd flags you must also specify
                                      # an executable name. If you just want a shell, use bash as the
                                      # the executable name.

This will run the application directly on a compute node in the current directory with access to two cores (in this example).

Scheduling Messages/Errors

It is possible that you will see the following message when you run qrsh (see below for examples):

Your "qrsh" request could not be scheduled, try again later

This means either:

• There are no cores free for interactive jobs (too many other interactive/short jobs are running)
• You have reached the user job limit
• You got something wrong on the command line (e.g. lower case v unstead of uppercase V)

This error often happens when the short environment is very busy, and you will need to try later. You should also consider whether you can convert your job to a batch job which requires no interaction or direct control from you while it runs.

High Memory Interactive Jobs

If the normal qrsh nodes do not have sufficient memory for your work you can try using:

qrsh -l short -l mem512 .....

for 32GB per core.

Note: due to the limited number of nodes of this type there is only one mem512 node available for this type of work, with a maximum of 16 cores available.

GPU Interactive Jobs

Please see the GPU Jobs page.

Further Information

• Further advice about running specific applications using qrsh may be available on the appropriate software page, for example, the idb and gdb debuggers.