| The CSF2 has been replaced by the CSF3 - please use that system! This documentation may be out of date. Please read the CSF3 documentation instead. To display this old CSF2 page click here. |
AMBER 12
Overview
Amber (Assisted Model Building with Energy Refinement) is a general purpose molecular mechanics/dynamics suite which uses analytic potential energy functions, derived from experimental and ab initio data, to refine macromolecular conformations.
There are two installations of Amber 12 version available on the CSF:
- Version 12 patched to bugfix level 20 with AmberTools 13 patched to bugfix level 21— detailed below
- Version 12 patched to bugfix level 13 with AmberTools 12 patched to bugfix level 28— detailed below
Basic Install Details
- The software was installed with Intel Compilers 12.0.5 and Openmpi 1.6
- The compile flag ‘-axAVX’ was used. This builds multiple auto-dispatch ‘code paths’ in the executable for use on different Intel architectures. This is deemed beneficial given there are a couple of different Intel CPUs available in the CSF.
Restrictions on use
This software is licensed for University of Manchester users. All users should familiarise themselves with the License Agreement before using this software. A copy is also available on the system in $AMBERHOME/amber12.license .
Set up procedure
- Amber12 is accessed using modulefiles.
- Modulefiles with
at13at the end are Amber12 with AmberTools 13. - Depending on where you wish to run your jobs you will need to load the correct modulefile. So use the command
module loadwith one of the below modulefiles as the option to the command
| Modulefile | SGE/batch information | |
| apps/intel-12.0/amber/12-at13 | For serial jobs and parallel jobs up to 24 cores. Uses AmberTools 13 | smp.pe, orte.pe |
| apps/intel-12.0/amber/12-ib-at13 | For parallel job which are 48 cores or more and a multiple of 24. Uses Infinband nodes and gives better performance. Uses AmberTools 13 | orte-24-ib.pe |
| apps/intel-12.0/amber/12-cuda-at13 | For pmemd.cuda jobs. Uses AmberTools 13 | -l nvidia |
| apps/intel-12.0/amber/12 | For serial jobs and parallel jobs up to 24 cores. | smp.pe |
| apps/intel-12.0/amber/12-ib | For parallel job which are 48 cores or more and a multiple of 24. Uses Infinband nodes and gives better performance. | orte-24-ib.pe |
| apps/intel-12.0/amber/12-cuda | For pmemd.cuda jobs. | -l nvidia |
Please see ‘Running the Application’ below for detailed job submission examples.
Note 1: the modulefiles load additional modules required to run amber, for example the correct openmpi, therefore unloading the amber module will also unload these additional modules.
Note 2: Amber 12 and the associated AmberTools will not run on AMD nodes.
Running Serial Amber
1. Make sure you have the correct modulefile from the table above loaded.
2. An example batch submission script:
### SGE Job Stuff
### Set the directory you submit from to be the working directory
#$ -cwd
### Inherit the user environment settings from the login node (important so the job can find all commands)
#$ -V
sander -O \
-i file.in
-o file.out
-c file.restart
-r file.rst
-p file.prmtop
-x file.cmd
-inf file.inf
3. Submit with the command: qsub scriptname
Running Parallel Amber
Important notes for all parallel jobs
- Please ensure that when running parallel amber you have selected the correct executable – normally with .MPI at the end of the name. Running a serial executable with multiple cores is inefficient and may give inaccurate results. If you in doubt about whether the part of amber you wish to use can be run in parallel please check the amber manual.
- Ensure you use $NSLOTS as shown in the examples – this ensures that the core request you have made is known to the application. Do not set the cores yourself on the application line as it can lead to errors and nodes being overlaoded.
- Where possible you should do scaling tests of your job to see how many cores suit it best. We have had reports that the same Amber12 job run on 24 Westmere cores with Infiniband and on 12 Sandybridge cores ran faster on the latter.
SMP example – maximum of 24 cores
By default SMP jobs will run on either a Westmere, Sandybridge, Ivybridge or Haswell nodes depending on what is available at the time. If the job gets a Sandybridge node (max 12 cores) or Ivybridge node (max 16 cores) or Haswell node (max 24 cores) it will perform better. If you wish to ensure that your job runs on one of Sandybridge, Ivybridge or Haswell see the example further down the page. Otherwise the system will choose a node for you – whichever becomes free.
1. Make sure you have the correct modulefile from the table above loaded.
2. An example batch submission script:
#!/bin/bash ### SGE Job Stuff ### Set the directory you submit from to be the working directory #$ -cwd ### Inherit the user environment settings from the login node (important so the job can find all commands) #$ -V ### Submit to the pe and tell SGE how many processors are required. $NSLOTS is set to be this number. #$ -pe smp.pe 6 mpiexec -n $NSLOTS sander.MPI -O \ -i file.in -o file.out -c file.restart -r file.rst -p file.prmtop -x file.cmd -inf file.inf
3. Submit with the command: qsub scriptname
SMP specifically Sandybridge, Ivybridge or Haswell example – maximum of 12, 16 or 24 cores
By default SMP jobs will run on either a Westmere, Sandybridge, Ivybridge or Haswell node depending on what is available at the time. If you wish to ensure that your job can take advantage of the Sandybridge, Ivybridge or Haswell nodes specifically you need to add an option to your job – see below.
1. Make sure you have the correct modulefile from the table above loaded.
2. An example batch submission script:
#!/bin/bash ### SGE Job Stuff ### Set the directory you submit from to be the working directory #$ -cwd ### Inherit the user environment settings from the login node (important so the job can find all commands) #$ -V ### Submit to the pe and tell SGE how many processors are required. $NSLOTS picks up this number. #$ -pe smp.pe 12 #$ -l sandybridge mpiexec -n $NSLOTS sander.MPI -O \ -i file.in -o file.out -c file.restart -r file.rst -p file.prmtop -x file.cmd -inf file.inf
3. Submit with the command: qsub scriptname
To request an Ivybridge node (max 16 cores) use:
#$ -l ivybridge
instead of
#$ -l sandybridge
and specify up to 16 cores.
For Haswell nodes (max 24 cores) use
#$ -l haswell
Infiniband MPI example
1. Make sure you have the correct modulefile from the table above loaded.
2. An example batch submission script:
#!/bin/bash ### SGE Job Stuff ### Set the directory you submit from to be the working directory #$ -cwd ### Inherit the user environment settings from the login node (important so the job can find all commands) #$ -V ### Submit to the pe and tell SGE how many processors are required. $NSLOTS picks up this number. #$ -pe orte-24-ib.pe 48 mpiexec -n $NSLOTS sander.MPI -O \ -i file.in -o file.out -c file.restart -r file.rst -p file.prmtop -x file.cmd -inf file.inf
3. Submit with the command: qsub scriptname
pmemd.cuda example
Note 1: Serial only
Note 2: You need to request being added to the relevant group to access GPUs before you can run amber on the GPUs.
1. Make sure you have the correct modulefile from the table above loaded.
2. An example batch submission script:
#!/bin/bash ### SGE Job Stuff ### Set the directory you submit from to be the working directory #$ -cwd ### Inherit the user environment settings from the login node (important so the job can find all commands) #$ -V #$ -l nvidia pmemd.cuda -O -i mdin -o out.$JOB_ID
3. Submit with the command: qsub scriptname
- Please ensure that when running parallel amber you have selected the correct executable – normally with .MPI at the end of the name. Running a serial executable with multiple cores is inefficient and may give inaccurate results. If you in doubt about whether the part of amber you wish to use can be run in parallel please check the amber manual.
Further info
- Manuals are available on the system ($AMBERHOME/doc)
- The Amber web site also has manuals, tutorials, an FAQ and mailing list: http://ambermd.org
Updates
None since install in early Nov 2012.