| The CSF2 has been replaced by the CSF3 - please use that system! This documentation may be out of date. Please read the CSF3 documentation instead. To display this old CSF2 page click here. |
Chem Shell
Overview
ChemShell is a computational chemistry environment for standard quantum chemical or force field calculations, including hybrid QM/MM calculations. ChemShell includes interfaces to various QM and MM codes, and is designed to deal with the communication and data handling, whilst energy evaluations are undertaken by external codes.
Version 3.4.2 has been compiled on the CSF using Intel compilers v11.1 and OpenMPI v1.4.3
Version 3.5.0 has been compiled on the CSF using Intel compilers v12.0.5 and OpenMPI v1.6
Restrictions on use
Version 3.4.2 is for use by one research group only. Users need to be in the ‘chemshell34’ group.
Version 3.5.0 is available for all CSF users to use. Users need to be in the ‘chemshell’ group.
There is no access to the source code for either version.
Whilst the software is free for academic usage there are limitations within the licence agreement which must be strictly adhered to by users. The licence covers use on the CSF only.
A copy of the full licence for version 3.4.2 is available on the CSF in
/opt/gridware/apps/intel-11.1/chemshell/3.4.2/ChemShell_LicenseAgreement.pdf.
A full copy of the version 3.5.0 licence is in:
/opt/gridware/apps/intel-12.0/chemshell/3.5.0/chemshell_3.5.0_licence.pdf.
Important points to note are:
- All parts of Clause 2.1, in particular:
- The software is only available to Staff and Students of the University of Manchester. Visiting scientists/students who are collaborating with a member of Staff and who have been given an official University of Manchester IT username may use the software.
Users are reminded that they must not share their password with anyone, or allow anyone else to utilise their account.
- The software may only be used for academic purposes as defined in clause 1.1. No commercial work or commercially funded work permitted.
- Derived Work – Users must not modify, enhance or improve the software without consultation with the University Intellectual Property Team. Any derived works must be shared with appropriately cite CCLRC.
- The software is for use on the CSF only.
- Clause 4.2: the acknowledgements for the use of the software are set out in this clause and must be used for all employee and student publications.
All users who wish to use the software must request access by emailing its-ri-team@manchester.ac.uk.
Set up procedure
To use ChemShell first set up the environment by loading the appropriate module.
For jobs that are either
- Up to 24 cores (all on a single compute node) using the smp.pe parallel environment
module load apps/intel-11.1/chemshell/3.4.2 OR module load apps/intel-12.0/chemshell/3.5.0
To use more than one compute node for large jobs, where the number of cores to be a multiple of 24 and be a minimum of 48, using the orte-24-ib.pe parallel environment
module load apps/intel-11.1/chemshell/3.4.2-ib OR module load apps/intel-12.0/chemshell/3.5.0-ib
This will ensure the fast InfiniBand network between the compute nodes is used.
Running the application
Serial batch job submission
The following SGE script can be used to submit a single core Chemshell job with input data file test.chm:
### SGE script to submit a serial Chemshell computation #$ -V #$ -cwd chemsh test.chm
Parallel batch job submission
The Chemshell -p flag is used to set the number of cores in parallel jobs. The following example scripts can be modified to submit jobs to SGE.
Jobs that fit on a single Intel compute node (24 or fewer cores) can use the smp.pe parallel environment. For example the following script defines a 2 core job (with input file inner.chm) running on a single node:
### SGE script to submit a parallel single-node Chemshell computation #$ -cwd #$ -V #$ -pe smp.pe 2 chemsh -p $NSLOTS inner.chm
Larger jobs using multiples of 24 cores can run on the InfiniBand nodes. The following script defines a 48 core Chemshell job on the InfiniBand network, this uses the orte-24-ib.pe parallel environment and requires the InfiniBand module (described above).
### SGE script to submit a parallel multi-node Chemshell computation #$ -cwd #$ -V #$ -pe orte-24-ib.pe 48 chemsh -p $NSLOTS inner.chm
You can no longer run jobs of more than 24 cores that aren’t in multiples of 24.
For more information on submitting jobs to SGE, and on parallel environments, see here.
Serial ChemShell calling Parallel QM/MM Applications
A ChemShell script will often use other tools to complete its calculation. For example TurboMole and/or Charmm can be started from within your ChemShell script. These tools can often run in parallel.
On the CSF we have found that starting ChemShell in parallel and then trying to run other parallel tools from ChemShell can cause the ChemShell job to crash. The solution is to run ChemShell serially and let it run the other tools in parallel. There are two ways to do this as explained below.
Writing your own jobscript
To prevent Chemshell from crashing it should be run serially and then let the other tools (e.g., TurboMole or CHARMM) run in parallel. Hence we still require a PE in the jobscript to specify the number of cores that will be used by the parallel tools. But ChemShell can be run without any MPI commands using a custom
chemsh-serial-csf
script.
Note that you must also load the modulefile for the additional tools ChemShell will use. For example, if calling TurboMole from your ChemShell script ensure you have loaded the required TurboMole modulefile before submitting your ChemShell job. Otherwise ChemShell will not be able to find the additional tools.
For example, the following jobscript is for a ChemShell job which uses TurboMole to do some processing. TurboMole will run in parallel but ChemShell will be a serial process. The jobscript is:
#!/bin/bash #$ -S bash #$ -cwd #$ -V ### ChemShell is run serially but it starts TurboMole in parallel ### so we must reserve 4 (in this example) cores for TurboMole #$ -pe smp.pe 4 # PARNODES is used by Turbomole - set it to the number above using $NSLOTS export PARNODES=$NSLOTS # Run ChemShell serially (without any mpirun) chemsh-serial-csf < myinput.chm >& myoutput.log
If running larger multi-node TurboMole jobs via ChemShell, please use the hp-mpi-24.pe in your jobscript. This is needed by the Turbomole MPI implementation. For example:
#!/bin/bash #$ -S bash #$ -cwd #$ -V ### ChemShell is run serially but it starts TurboMole in parallel ### so we must reserve cores for TurboMole. To run multi-node ### parallel jobs we must use at least 48 cores (i.e., two 24-core nodes) ### and the number of cores must be a multiple of 24 (i.e., entire nodes only). #$ -pe hp-mpi-24.pe 48 # PARNODES is used by Turbomole - set it to the number above using $NSLOTS export PARNODES=$NSLOTS # Run ChemShell serially (without any mpirun) chemsh-serial-csf < myinput.chm >& myoutput.log
Using a helper script
To help with submitting QM/MM jobs that call other tools such as TurboMole and/or CHARMM, we have created a script that can be run on the CSF login nodes. It will submit jobs for you. After loading the chemshell modulefile, run on the login node:
qchemshell_csf
This allows you to specify the number of cores to use and the name of a .chm input file. If you are using TURBOMOLE and/or CHARMM you must load the modulefiles for those applications first. For example:
# Submit a Chemshell job that runs CHARMM and TURBOMOLE on a single compute node using 8 cores module load apps/intel-12.0/chemshell/3.5.0 module load apps/intel-12.0/charmm/c36b2 module load apps/binapps/turbomole/7.1.1/mpi qchemshell_csf -p 8 myinput.chm # # All of these commands are run on the login node. A batch job will be submitted for you.
If you wish to run larger multi-node jobs you should load the -ib versions of the modulefiles so that the fast InfiniBand networking between the compute nodes is used. The number of cores requested must be 48 or more and a multiple of 24. For example:
# Submit a Chemshell job that runs CHARMM and TURBOMOLE on multiple compute nodes using 48 cores and IB networking. module load apps/intel-12.0/chemshell/3.5.0-ib module load apps/intel-12.0/charmm/c36b2-ib module load apps/binapps/turbomole/7.1.1/mpi # This has no -ib version - it will use IB hardware if it detects it. qchemshell_csf -p 48 myinput.chm # # All of these commands are run on the login node. A batch job will be submitted for you.
Additional flags can be given on the qchemshell_csf command-line:
-ne # Turn off job email. By default the job will mail you when it has finished.
-long # Submit to the long queue. You must have been given permission to use it.
# The max number of cores is 12 in this case.
Further info
Further information can be found on the ChemShell web page including the Chemshell Manual, tutorials and details of the chemsh-users email list which can be used to post queries to seek help.
Updates
None at present.