Gaussian 09

Gaussian is a general purpose suite of electronic structure programs. Version g09d01 is installed. It is available as binaries only. The source code is not available on the CSF.

If you want information about Gaussian 16 on the CSF please see our G16 docs.

Restrictions on use – G09 ONLY

The University of Manchester site license allows access for all staff and students of the university, however strict licensing restrictions require that all users who wish to use the software must complete some paperwork and be in a restricted unix group.

The procedure for getting access on the CSF is as follows (please read carefully):

1. Print out and complete the University’s gaussian paperwork. Note – you must sign the appropriate section(s) in ink (anything else will be rejected). PLEASE NOTE: the PDF is only available if on campus or running GlobalProtect. If you cannot access the PDF please request a copy via our help form.)
2. Scan in the entire document (yes, even the pages you didn’t sign) and send it to us via our help form (please do not email gaussian@manchester.ac.uk as that address does not handle CSF access ) to request being added to the g09 gaussian group on CSF, attaching a scanned copy of the completed paperwork (the whole document). Anything other than the entire document will be rejected and lead to a delay in your access to Gaussian.
3. We will add your paperwork to the ITS database of Gaussian users, add you to the appropriate group on the CSF and then send a confirmation to you.

Please ensure you follow the above instructions carefully. We are required by the University License Manager to follow the above procedure. Failure to supply the requested information correctly will delay your access to the software.

This paperwork does not enable access to Gaussian 16. Please see the CSF gaussian 16 webpage for more info.

Set up procedure

After being added to the relevant unix group, you will be able to access the executables by loading the modulefile

For g09:

module load gaussian/g09d01_em64t

We recommend you do this in your jobscript, see examples below (not on the command line before job submission as per the previous CSF2).

Gaussian MUST ONLY be run in batch. Please DO NOT run g09 on the login nodes. Computational work found to be running on the login nodes will be killed WITHOUT WARNING.

Gaussian Scratch

Gaussian uses an environment variable $GAUSS_SCRDIR to specify a directory path for where to write scratch (temporary) files (two-electron integral files, integral derivative files and a read-write file for temporary workings).

If not already set by you, it is set to your scratch directory (~/scratch) when you load the modulefile. This is a Lustre filesystem which provides good I/O performance. Do not be tempted to use your home directory for Gaussian scratch files – the files can be huge making the home area at risk of going over quota. We also recommend using a directory-per-job in your scratch area. See below for how to do this.

A faster, but smaller, local /tmp on each compute node is available should users prefer to use that. It can be more efficient if you have a need to create lots of small files, but space is limited. The minimum /tmp on intel compute nodes is 800GB, the largest is 3.5TB.

Gaussian should delete scratch files automatically when a job completes successfully or dies cleanly. However, it often fails to do this. Scratch files are also not deleted when a job is killed externally or terminates abnormally so that you can use the scratch files to restart the job (if possible). Consequently, leftover files may accumulate in the scratch directory, and it is your responsibility to delete these files. Please check periodically whether you have a lot of temporary Gaussian files that can be deleted.

Using a Scratch Directory per Job

We now recommend using a different scratch directory for each job. This improves file access times if you run many jobs – writing 1000s of scratch files to a single directory can slow down your jobs. It is much better to create a directory for each job within your scratch area. It is also then easy to delete the entire directory if Gaussian has left unwanted scratch files behind.

The example jobscripts below show how to use this method (it is simple – just two extra lines in your jobscript).

Very large Gaussian scratch files

Occasionally some jobs create .rwf files which are very large (several TB). The batch system will not permit a job to create files bigger than 4TB. If your gaussian job fails and the .rwf file is 4TB then it may be that this limit has prevented your job from completing. You should re-run the job and in your input file request that the .rwf file be split into multiple files. For example to split the file into two 3TB files:

%rwf=/scratch/$USER/myjob/one.rwf,3000GB,/scratch/$USER/myjob/two.rwf,3000GB

Serial batch job

In the examples below we give example jobscripts using the BASH shell (the default used by most CSF users) and also the C shell, which is popular amongst computational chemists.

Example job submission

It is recommended you run from within your scratch area and use one directory per job:

cd ~/scratch
mkdir job1
cd job1

Create a job script, for example:

• BASH shell verison:

  #!/bin/bash --login
  #SBATCH -p serial      # (or --partition) Single-core job
  #SBATCH -n 1           # (or --ntasks=) Number of cores (always 1)
  
  # Load g09
  module load gaussian/g09d01_em64t
  
  ## Set up scratch dir (please do this!)
  export GAUSS_SCRDIR=/scratch/$USER/gau_temp_$SLURM_JOB_ID
  mkdir -p $GAUSS_SCRDIR
  
  ## Say how much memory to use (4GB per core)
  export GAUSS_MDEF=$((SLURM_NTASKS*4))GB
  
  $g09root/g09/g09 < file.inp > file.out
  

• C shell version:

  #!/bin/csh
  #SBATCH -p serial    # (or --partition) Single-core job
  #SBATCH -n 1         # (or --ntasks=) Number of cores (always 1)
  
  # Load g09 
  module load gaussian/g09d01_em64t
  
  # Set up scratch dir (please do this!)
  setenv GAUSS_SCRDIR /scratch/$USER/gau_temp_$SLURM_JOB_ID
  mkdir -p $GAUSS_SCRDIR
  
  ## Say how much memory to use (4GB per core)
  @ mem = ( $SLURM_NTASKS * 4 )
  setenv GAUSS_MDEF ${mem}GB
  
  $g09root/g09/g09 < file.inp > file.out
  

Submit with the command:

sbatch scriptname

where scriptname is the name of your job script.

When the job has finished check whether Gaussian has left behind unwanted scratch files (you’ll need to know the job id). For example, assuming your job id was 456789:

cd ~/scratch/gau_temp_456789
ls
Gau-21738.inp  Gau-21738.chk  Gau-21738.d2e  Gau-21738.int  Gau-21738.scr

# Example: Remove a specific scratch file
rm Gau-21738.scr

# Example: Remove all files in the directory (use with caution)
rm Gau*

# Example: go up and remove the empty directory
cd ..
rmdir gau_temp_456789

Parallel batch job

On the CSF Gaussian is a multi-threaded application (shared memory) only, so a job will not run across multiple compute nodes. Hence you are limited to a maximum of 40 cores. This means that you must run in the multicore partition to confine your job to a single node.

Follow the steps below to submit a parallel Gaussian job.

Important Information About Requesting cores

Old method: We used to advise setting the number of cores to use for a job in the Gaussian input file using %NProcsShared or %nprocs. But this can easily lead to mistakes – if you change the number of cores in the jobscript but forget to also change it in the Gaussian input file you will either use too few cores (some of the cores your job requested are sat idle) or too many cores (your job is trying to use cores it shouldn’t, possibly trampling on another user’s job).

New method: We now recommend setting the GAUSS_PDEF environment variable in your jobscript (set it to $SLURM_NTASKS) so that it always tells Gaussian the correct number of cores to use. This also means you don’t have to keep editing your Gaussian input file each time you want to run the input deck with a different number of cores.

For example, depending which shell you use (look at the first line of your jobscript to find out):

# If using BASH (the default shell used by most CSF users):
export GAUSS_PDEF=$SLURM_NTASKS

# If using CSH (the 'traditional' shell used by chemistry users):
setenv GAUSS_PDEF $SLURM_NTASKS

Remember that $SLURM_NTASKS is automatically set by the batch system to the number of cores you requested on the #SBATCH -n line in the jobscript. Hence there is only one number-of-cores to change if you want to run the job with a different number of cores.

Note: %NProcShared in the input file takes precedence over GAUSS_PDEF, so one could override the latter by setting %NProcShared in the input file. If you are using our recommended method of setting GAUSS_PDEF in the jobscript, please remove any %NProcShared line from your Gaussian input files.

Example job submission

It is recommended you run from within your scratch area and use one directory per job:

cd ~/scratch
mkdir job1
cd job1

Create a job script, for example:

• BASH shell verison:

  #!/bin/bash --login
  #SBATCH -p multicore      # (or --partition) Single-node multicore
  #SBATCH -n 20             # (or --ntasks=) Number of cores (2--40)
  
  # Load g09
  module load gaussian/g09d01_em64t
  
  ## Set up scratch dir (please do this!)
  export GAUSS_SCRDIR=/scratch/$USER/gau_temp_$SLURM_JOB_ID
  mkdir -p $GAUSS_SCRDIR
  
  ## Say how much memory to use (4GB per core)
  export GAUSS_MDEF=$((SLURM_NTASKS*4))GB
  
  ## Inform Gaussian how many cores to use
  export GAUSS_PDEF=$SLURM_NTASKS
  
  $g09root/g09/g09 < file.inp > file.out
  

• C shell version:

  #!/bin/csh         
  #SBATCH -p multicore      # (or --partition) Single-node multicore
  #SBATCH -n 20             # (or --ntasks=) Number of cores (2--40)
  
  # Load g09
  module load gaussian/g16c01_em64t_detectcpu
  
  # Set up scratch dir (please do this!)
  setenv GAUSS_SCRDIR /scratch/$USER/gau_temp_$SLURM_JOB_ID
  mkdir -p $GAUSS_SCRDIR
  
  ## Say how much memory to use (4GB per core)
  @ mem = ( $SLURM_NTASKS * 4 )
  setenv GAUSS_MDEF ${mem}GB
  
  ## Inform Gaussian how many cores to use
  setenv GAUSS_PDEF $SLURM_NTASKS
  
  $g09root/g09/g09 < file.inp > file.out
  

Submit with the command:

sbatch scriptname

where scriptname is the name of your job script.

GAUSS_PDEF vs GAUSS_CDEF

Gaussian has two environment variables that can be used to say how many cores to use. We saw the GAUSS_PDEF variable above. Alternatively the GAUSS_CDEF variable can be set but this must only be used when you are using all of the cores on a compute node. If you are unsure whether your job does this, please use the GAUSS_PDEF variable as shown above.

The GAUSS_CDEF variable may give increased performance because it pins g16 threads (used to do the parallel processing in Gaussian) to specific CPU cores. Without pinning Linux is free to move the threads between cores, although it tries not to do this. When a thread is moved it invalidates the low-level memory caches which may reduce performance.

The GAUSS_CDEF variable uses a slightly different format to the GAUSS_PDEF variable, as shown below:

#SBATCH -p multicore      # (or --partition) Single-node multicore
#SBATCH -n 40             # (or --ntasks=) Use all 40 cores in a cascade lake node

# Say which cores to use, e.g., 0-39 (BASH shell):
export GAUSS_CDEF=0-$((SLURM_NTASKS-1))

# Say which cores to use, e.g., 0-39 (C shell):
@ maxcore = ( $SLURM_NTASKS - 1 )
setenv GAUSS_CDEF 0-$maxcore

Reminder: the GAUSS_CDEF variable should only be used when you are using all cores on a compute node. Jobs found to be using this variable incorrectly will be killed without warning because you will be slowing down other users’ jobs.

Further info

• The IT Services Gaussian webpage contains important information applicable to all users of the software.
• Gaussian Inc.