Interactive Jobs (qrsh)

CPU-intensive interactive work must not be run on the login node at any time:

• interactive work, e.g., that which is GUI-based, should be scheduled using qrsh. This will log you in to a backend-compute where you can work at the command-line without using a batch script.

Overview

qrsh is a command in SGE that allows users to obtain an interactive session on compute nodes. This is useful if you need to be able to run CPU or memory hungry sessions of GUI based applications, which are not permitted to run on the login node for more than a few minutes. It can also be used for debugging and pre/post processing.

Imporant Notes about qrsh

• This kind of job will not start if there are no suitable, free slots.
• Users are therefore strongly encouraged to use the command line alternative in their chosen application instead of the GUI, running it with a jobscript in the batch system, when ever possible.
• If you wish to run a GUI then you will need to ensure you have logged into the CSF with X11 enabled

Resource Limits

• Users may run single-core (i.e., serial) jobs or parallel jobs which require only a small number of cores intractively, via qrsh.
• There are no nodes dedicated to interactive work and batch jobs take priority.
• There is a maximum of 4 interactive cores per user.
• Interactive jobs are restricted to the “short” environments (each of which have 4GB memory per core)
• Interactive jobs are not permitted to use the Infiniband-connected nodes (with the exception of certain GPU-hosting nodes, which are not available to most users).

General Information About Running Interactive Jobs with qrsh

• Job scripts are not involved – you can run applications directly at the command-line.
• The instructions below are generic. You should check the relevant software pages to see if any qrsh advice specific to your application has been given.
• Interactive work is limited to one hour.
• Editors such as vi, nano, emacs and gedit are all available on the interactive nodes so you can edit files/code and then do simple, small test runs.
• If you wish to run a GUI then you will need to ensure you have logged into the CSF with X11 enabled
• If you need help running an application with qrsh please email its-ri-team@manchester.ac.uk

Scheduling Messages/Errors

It is possible that you will see the following message when you run qrsh (see below for examples):

Your "qrsh" request could not be scheduled, try again later

This means either:

• There are no cores free for interactive jobs (too many other interactive/short jobs are running)
• You have reached the user job limit
• You got something wrong on the command line (e.g. lower case v unstead of uppercase V)

This error often happens when the short environment is very busy, and you will need to try later. You should also consider whether you can convert your job to a batch job which requires no interaction or direct control from you while it runs.

Serial qrsh

Example: Serial Interactive job with a GUI application

• Log into the CSF with X11 enabled
• Load the module appropriate to your GUI application.
• On the command line enter:

  qrsh -cwd -V -l inter -l short myapp
                                  #
                                  # replace myapp with your
                                  # application/software executable

• When you have finished with your application, exit in the normal way.

Example: Serial Interactive job without GUI

Here we mean an application that uses a command-line interface. You still use it interactively.

• Load the appropriate module for your application
• On the command line enter:

  qrsh -cwd -V -l inter -l short myapp
                                  #
                                  # replace myapp with your
                                  # application/software executable

• When you have finished with your application, exit in the normal way.

Multi-core qrsh

• The following parallel environment is available for interactive jobs: smp.pe
• It is very important to make sure that the number you pass to the application matches the request you make to the parallel environment

Example: Interactive MPI job without GUI

Here we mean an application that uses a command-line interface. You still use it interactively.

• Load the appropriate module for your application
• We are going to request 4 cores from the batch system and so we must run our application with 4 cores.
• On the command line enter:

  qrsh -cwd -V -l inter -l short -pe smp.pe 4 mpirun -np 4 myapp
                                                             #
                        # replace myapp with your            #
                        # application/software executable.   #
                        # Notice the 4 used in two places.

• When you have finished with your application, exit in the normal way.

High Memory Interactive Jobs

The default memory setup of ‘short’ Intel nodes is 8GB of RAM per core. If your
job requires more than 2GB per core then you can add the -l highmem option to your command line:

qrsh -cwd -V -l inter -l highmem myapp

and SGE will run your job on the Intel nodes which have 8GB per core.

If you require more than 8GB per core than please add -l vhighmem. This will still use the Intel 8GB nodes, but the number of cores that can run under ‘vhighmem’ at any one time is lower hence your job gets access to more memory.

Further Information

• Further advice about running specific applications using qrsh may be available on the appropriate software page, for example, the idb and gdb debuggers.