Gaussian

Overview

Gaussian is a general purpose suite of electronic structure programs. Two versions are currently available on the system:

• g09d01 – installed 25/09/15
• g03d01 – this version is subject to review and may not always be available. Please consider moving to g09.We do not automatically grant access to this version.

Both are binary installs only. The source code is not available on the CSF.

Restrictions on use

The University of Manchester site license allows access for all staff and students of the university, however strict licensing restrictions require that all users who wish to use the software must complete some paperwork and be in a restricted unix group. There are two separate unix groups – one for g09 and one for g03. The procedure for getting access on the CSF is as follows:

1. Complete and submit the University’s gaussian paperwork. Note – you must submit a copy of the whole document, with the appropriate section(s) signed in ink (anything else will be rejected).
2. Email its-ri-team@manchester.ac.uk to request being added to the g03 or g09 gaussian group on CSF (attaching a copy of the completed paperwork can help speed the process up).
3. Once your paperwork appears in the ITS database of gaussian users (or if you have sent it to us directly) you will be added to the appropriate group on the CSF and we will send a confirmation to you.

Set up procedure

After being added to the relevant unix group, you will be able to access the executables loading one of the appropriate modulefiles:

For g09:

module load apps/binapps/gaussian/g09d01_em64t

module load apps/binapps/gaussian/g09b01_em64t

For g03 (by default we no longer grant access to this version unless it is essential to the work being undertaken):

module load apps/binapps/gaussian03/g03d01_em64t_amd

Gaussian Scratch

Gaussian uses an environment variable $GAUSS_SCRDIR to specify a directory path for where to write scratch (temporary) files (two-electron integral files, integral derivative files and a read-write file for temporary workings). It is set to your scratch directory (~/scratch) when you load the modulefile. This is a Lustre filesystem which provides good I/O performance. Do not be tempted to use your home directory for Gaussian scratch files – the files can be huge making the home area at risk of going over quota. We also recommend using a directory-per-job in your scratch area. See below for how to do this.

A faster, but smaller, local /tmp on each compute node is available should users prefer to use that. It can be more efficient if you have to a need to create lots of small files, but space is limited. The minimum /tmp on intel compute nodes is 800GB, the largest is 3.5TB.

Gaussian should delete scratch files automatically when a job completes successfully or dies cleanly. However, it often fails to do this. Scratch files are also not deleted when a job is killed externally or terminates abnormally so that you can use the scratch files to restart the job (if possible). Consequently, leftover files may accumulate in the scratch directory, and it is your responsibility to delete these files. Please check periodically whether you have a lot of temporary Gaussian files that can be deleted.

Using a Scratch Directory per Job

Update: We now recommend using a different scratch directory for each job. This improves file access times if you run many jobs – writing 1000s of scratch files to a single directory can slow down your jobs. It is much better to create a directory for each job within your scratch area. It is also then easy to delete the entire directory if Gaussian has left unwanted scratch files behind.

The example jobscripts below show how to use this method (it is simple – just two extra lines in your jobscript).

Very large Gaussian scratch files

Occasionally some jobs create .rwf files which are very large (several TB). The batch system will not permit a job to create files bigger than 4TB. If your gaussian job fails and the .rwf file is 4TB then it may be that this limit has prevented your job from completing. You should re-run the job and in your input file request that the .rwf file be split into multiple files. For example to split the file into two 3TB files:

%rwf=/scratch/$USER/myjob/one.rwf,3000GB,/scratch/$USER/myjob/two.rwf,3000GB

Running the application

Serial batch job

1. Make sure you have loaded the appropriate module.
2. It is recommended you run from within your scratch area:

   cd ~/scratch
   

3. Create a job script, for example:

   #!/bin/bash
   #$ -S /bin/bash
   #$ -cwd                       # Run job in directory you submitted from
   #$ -V                         # Use current environment settings, e.g. from 
                                 # loaded modulefiles, when the job runs.
   
   # We now recommend using a scratch directory per job
   export GAUSS_SCRDIR=/scratch/$USER/gau_temp_$JOB_ID
   mkdir -p $GAUSS_SCRDIR
   
   $g09root/g09/g09 < file.inp > file.out
   

4. Submit with the command:

   qsub scriptname

   where scriptname is the name of your job script.

5. When the job has finished check whether Gaussian has left behind unwanted scratch files (you’ll need to know the job id). For example, assuming my job id was 456789:

   cd ~/scratch/gau_temp_456789
   ls
   Gau-21738.inp  Gau-21738.chk  Gau-21738.d2e  Gau-21738.int  Gau-21738.scr
   
   # Example: Remove a specific scratch file
   rm Gau-21738.scr
   
   # Example: Remove all files in the directory (use with caution)
   rm Gau*
   
   # Example: go up and remove the empty directory
   cd ..
   rmdir gau_temp_456789
   

Parallel batch job

Gaussian is a multi-threaded application (shared memory) so a job can not run across nodes, and you are limited to a maximum of 24 processors. This means that you must run in smp.pe to confine your job to a single node. Follow the steps below to submit a parallel Gaussian job.

1. Make sure you have the appropriate module loaded
2. It is recommended you run from within your scratch area:

   cd ~/scratch
   

3. You must have a line in your Gaussian input file that tells Gaussian how many cores to use:

   %nprocshared=N              (or %nprocs=N)
   

   where N is the number of cores you request in the jobscript (see below). Failure to have these match can lead to two main issues a) nodes get overloaded and the efficient running of other users jobs on that node being impacted as a result b) without it jobs will only use one core leaving the others allocated to you idle and increasing queueing time for other people.

4. Create a job script, for example:

   #!/bin/bash
   #$ -S /bin/bash
   #$ -cwd                       # Run job in directory you submitted from
   #$ -V                         # Use current environment settings, e.g. from 
                                 # loaded modulefiles, when the job runs.
   #$ -pe smp.pe 12              # Run in SMP parallel environment with e.g. 12 cores
                                 # Ensure this matches '%nprocs=' line in .inp file.
   
   # We now recommend using a scratch directory per job
   export GAUSS_SCRDIR=/scratch/$USER/gau_temp_$JOB_ID
   mkdir -p $GAUSS_SCRDIR
   
   $g09root/g09/g09 < file.inp > file.out
                        #
                        # You MUST set %nprocs=N (12 in this example) in the .inp file!
   

5. Submit with the command:

   qsub scriptname

   where scriptname is the name of your job script.

6. See above for how to tidy up your scratch directory if Gaussian leaves scratch files behind once the job has finished.

Further info

• The IT Services Gaussian webpage contains important information applicable to all users of the software.
• Gaussian Inc. g09 reference pages.
• Gaussian Inc. Running Gaussian pages.

Updates

January 2015 – updated scratch dir advice.

March 2011 – g09b01 installed.

April 2011 – g03d01 installed.