AMBER

Overview

Amber (Assisted Model Building with Energy Refinement) is a general purpose molecular mechanics/dynamics suite which uses analytic potential energy functions, derived from experimental and ab initio data, to refine macromolecular conformations.

Three versions are available on the CSF:

• Version 16 with AmberTools 17 – detailed on this page
• Version 14 with AmberTools 14 – detailed on this page.
• Version 12
• Version 11

Basic Install Details

Version 16 – August 2017

• Installed with Intel Compilers 15.0.3 and Openmpi 1.8.3
• The compile flag ‘-axSSE4.2,AVX,CORE-AVX2’ was used. This builds multiple auto-dispatch ‘code paths’ in the executable for use on different Intel architectures. This is deemed beneficial given there are several different Intel CPUs available in the CSF.
• AmberTools 17 was patched to bugfix level 6 and Amber 16 to bugfix level 10.
• cpptraj, nab and saxs have been compiled with OpenMP capability.
• This version will not run on the AMD Magny-Cour nodes.
• DFTB files – All are available in $AMBERHOME/dat/slko
• This version has it’s own installation of Anaconda python (miniconda) and this is configured in the modulefile.
• Please note that the CUDA version of AMBER16 is not available – the hardware and drivers for the CSF GPUs are not current enough. We hope to purchase some more GPUs in late 2017/early 2017 – please get in touch if you would like to be involved in that procurement.

Version 14 – additional bugfixes – March 2015 – recommended version

• Installed with Intel Compilers 14.0.3 and Openmpi 1.8.3
• The compile flag ‘-axAVX’ was used. This builds multiple auto-dispatch ‘code paths’ in the executable for use on different Intel architectures. This is deemed beneficial given there are a couple of different Intel CPUs available in the CSF.
• AmberTools 14 was patched to bugfix level 26 and Amber 14 to bugfix level 11.

Version 14 – initial install – November 2014

• Installed with Intel Compilers 14.0.3 and Openmpi 1.8.3
• The compile flag ‘-axAVX’ was used. This builds multiple auto-dispatch ‘code paths’ in the executable for use on different Intel architectures. This is deemed beneficial given there are a couple of different Intel CPUs available in the CSF.
• AmberTools 14 was patched to bugfix level 19 and Amber 14 to bugfix level 6.
• Note: A typo was found in the config.h for the mpi part of this install that meant the software was not running as well as expected on SandyBridge and Ivybridge. A new version was installed with additional bugfixes and the typo corrected. This is the version above.

Restrictions on use – AMBER

This software is licensed for University of Manchester users. All users should familiarise themselves with the Terms and Conditions of the license and the University’s AMBER Usage Guidelines which are linked below:

• License Agreement V16
• University’s AMBER 16 Usage Guidelines
• License Agreement V14
• University’s AMBER 14 Usage Guidelines

You must confirm to its-ri-team@manchester.ac.uk that you have read these documents and that your usage will comply with the terms before you can be added to the amber unix group which restricts access to the software.

There is no access to the source code – groups that require source access should get in touch with us to see if this can be arranged.

Set up procedure

Amber 16

Version 16 – August 2017

• Depending on where you wish to run your jobs you will need to load the correct modulefile. Use the command module load with one of the below modulefiles as the option to the command

Modulefiles for August 2017 install	Type of Job	SGE/batch information
apps/intel-15.0/amber/16	For serial jobs and small parallel jobs up to 24 cores.	smp.pe
apps/intel-15.0/amber/16-ib	For parallel jobs which require 48 or more cores and are a multiple of 24. Uses Infinband nodes.	orte-24-ib.pe

Amber 14

• Depending on where you wish to run your jobs you will need to load the correct modulefile. Use the command module load with one of the below modulefiles as the option to the command

Modulefiles for Mar 2015 install	Type of Job	SGE/batch information
apps/intel-14.0/amber/14-bf	For serial jobs and small parallel jobs up to 24 cores.	smp.pe
apps/intel-14.0/amber/14-ib-bf	For parallel jobs which require 48 or more cores and are a multiple of 24. Uses Infinband nodes.	orte-24-ib.pe
apps/intel-14.0/amber/14-cuda-bf	For pmemd.cuda jobs.	-l nvidia_k20
Modulefiles for Nov 2014 install	Type of Job	SGE/batch information
apps/intel-14.0/amber/14	For serial jobs and small parallel jobs up to 24 cores.	smp.pe
apps/intel-14.0/amber/14-ib	For parallel jobs which require 48 or more cores and are a multiple of 24. Uses Infinband nodes.	orte-24-ib.pe
apps/intel-14.0/amber/14-cuda	For pmemd.cuda jobs.	-l nvidia_k20

Please see ‘Running the Application’ below for detailed job submission examples.

Note 1: the modulefiles load additional modules required to run amber, for example the correct openmpi, therefore unloading the amber module will also unload these additional modules.

Note 2: Amber 14 and the associated AmberTools will not run on AMD nodes.

Note 3: Amber16 and the associated AmberTools will not run on AMD Magny-Cour nodes.

Running Serial Amber

1. Make sure you have the correct modulefile from the table above loaded.

2. An example batch submission script:

### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V

sander -O \
       -i file.in
       -o file.out
       -c file.restart
       -r file.rst
       -p file.prmtop
       -x file.cmd
       -inf file.inf

3. Submit with the command: qsub scriptname

Running Parallel Amber

Important notes for all parallel jobs

• Please ensure that when running parallel amber you have selected the correct executable – normally with .MPI at the end of the name. Running a serial executable with multiple cores is inefficient and may give inaccurate results. If you in doubt about whether the part of amber you wish to use can be run in parallel please check the amber manual.
• Ensure you use $NSLOTS as shown in the examples – this ensures that the core request you have made is known to the application. Do not set the cores yourself on the application line as it can lead to errors and nodes being overlaoded.
• Where possible you should do scaling tests of your job to see how many cores suit it best. We have had reports that the same Amber12 job run on 24 Westmere cores with Infiniband and on 12 Sandybridge cores ran faster on the latter.

Any Intel node type up to 24 cores using MPI

1. Make sure you have the correct modulefile from the table above loaded.

2. An example batch submission script:

#!/bin/bash
### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V
### Submit to the pe and tell SGE how many processors are required. $NSLOTS picks up this number.
#$ -pe smp.pe 6
mpiexec -n $NSLOTS sander.MPI -O \
        -i file.in
        -o file.out
        -c file.restart
        -r file.rst
        -p file.prmtop
        -x file.cmd
        -inf file.inf

3. Submit with the command: qsub scriptname

More than one node – 48 or more cores in multiples of 24 – uses Infiniband connected Intel haswell nodes

1. Make sure you have the correct modulefile from the table above loaded.

2. An example batch submission script:

#!/bin/bash
### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V
### Submit to the pe and tell SGE how many processors are required. $NSLOTS picks up this number.
#$ -pe orte-24-ib.pe 48

mpiexec -n $NSLOTS sander.MPI -O \
        -i file.in
        -o file.out
        -c file.restart
        -r file.rst
        -p file.prmtop
        -x file.cmd
        -inf file.inf

3. Submit with the command: qsub scriptname

Please note – there are no Westmere, Sandybridge or Ivybridge nodes with Infiniband connections. Jobs submitted to orte-24-ib.pe will by default run only on Haswell nodes.

OpenMP components – AMBER 16 only

The following executables have been compiled with OpenMP (Multi-threaded) capability:

cpptraj.OMP
   ## The serial executable is called ccptraj and should be run as per section 5 of these notes. 
   ## The mpi executable is called cpptraj.MPI and should be run as per the MPI examples above.
nab
   ## There is no serial version of this.
saxs_rism
   ## No serial version.
saxs_md
   ## No serial version.

By default (due to a modulefile setting) they will only use one core. To use multiple cores you must request smp.pe and set the OMP_NUM_THREADS variable to $NSLOTS in your jobscript. For example:

#!/bin/bash
### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V
### Submit to the pe and tell SGE how many processors are required. $NSLOTS picks up this number.
#$ -pe smp.pe 4

export OMP_NUM_THREADS=$NSLOTS

ccptraj.OMP -p file.prmtop

Other components may be OpenMP capable, please consult the AMBER manual for further information.

pmemd.cuda example

Note 1: Serial only

Note 2: You need to request being added to the relevant group to access GPUs before you can run amber on the GPUs. Please note that as of May 2017 the only GPUs available in the CSF are K20s (a total of 4) and priority on these nodes is given to a particular group of people. Limited access can be granted to people outside that group – please contact us to discuss if you may be added.

Note 3: The CUDA version of AMBER16 is not available – the hardware and drivers for the CSF GPUs are not current enough.

Note 4: We hope to purchase some more GPUs in late 2017/early 2018 – please get in touch if you would like to be involved in that procurement which should be compatible with AMBER 16.

1. Make sure you have the correct modulefile from the table above loaded.

2. An example batch submission script:

#!/bin/bash
### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V
#$ -l nvidia_k20

pmemd.cuda -O -i mdin -o out.$JOB_ID

3. Submit with the command: qsub scriptname

• Please ensure that when running parallel amber you have selected the correct executable – normally with .MPI at the end of the name. Running a serial executable with multiple cores is inefficient and may give inaccurate results. If you in doubt about whether the part of amber you wish to use can be run in parallel please check the amber manual.

Further info

• Manuals are available on the system ($AMBERHOME/doc)
• The Amber web site also has manuals, tutorials, an FAQ and mailing list: http://ambermd.org