CP2K
Overview
CP2K is a program to perform atomistic and molecular simulations of solid state, liquid, molecular, and biological systems. It provides a general framework for different methods such as e.g., density functional theory (DFT) using a mixed Gaussian and plane waves approach (GPW) and classical pair and many-body potentials.
Version 6.1.0 is installed on the CSF. It was compiled with Intel Compilers 17.0.7 & MKL. Serial, multi-core (OpenMP) and multi-node (MPI) versions are available, and all three are available with cuda support.
Restrictions on use
The software is open source under the GNU General Public License.
Set up procedure
We now recommend loading modulefiles within your jobscript so that you have a full record of how the job was run. See the example jobscript below for how to do this. Alternatively, you may load modulefiles on the login node and let the job inherit these settings.
To access the software you must first load the appropriate modulefile. Several different flavours are available:
The singularity modulefile allows access to more recent versions of the SMP (multicore) version. The singularity container was converted from a Docker container, where the Docker container was created by the CP2K developers.
# Features: TBC module load apps/singularity/cp2k/2023.2 # Version: cp2k.ssmp (multi-core parallel, OpenMP)
The binapps modulefile sets up to use the pre-compiled version built by the CP2K developers. It has more features enabled but may be less optimized for CSF hardware.
# Features: libint fftw3 libxc xsmm libderiv_max_am1=6 libint_max_am=7 max_contr=4
module load apps/binapps/cp2k/6.1.0 # Versions: cp2k.sopt (serial)
# cp2k.ssmp (multi-core parllel, OpenMP)
The intel-17.0 modulefile set up to use the version built by the Research Infrastructure team. It has fewer features enabled but may be more optimized for CSF hardware.
# Features: fftw3 parallel mpi3 scalapack mkl
module load apps/intel-17.0/cp2k/6.1.0 # Versions: cp2k.sopt (serial)
# cp2k.ssmp (multi-core parallel, OpenMP)
# cp2k.popt (multi-node parallel, MPI)
# Features: fftw3 parallel mpi3 scalapack acc pw_cuda dbcsr_acc mkl (must be run on a GPU node)
module load apps/intel-17.0/cp2k/6.1.0-cuda # Versions: cp2k.sopt (serial)
# cp2k.ssmp (multi-core parallel, OpenMP)
# cp2k.popt (multi-node parallel, MPI)
Running the application
Please do not run cp2k on the login node. Jobs should be submitted to the compute nodes via batch.
Serial batch job submission
Ensure you run the cp2k.sopt executable after loading one of the above modulefiles.
#!/bin/bash --login #$ -cwd # Load either the intel-17.0 version or the binapps version (see above). For example: module load apps/intel-17.0/cp2k/6.1.0 cp2k.sopt -i mysim.inp
Submit the jobscript using:
qsub scriptname
where scriptname is the name of your jobscript.
Small parallel batch job submission – 2 to 32 cores
Ensure you run the cp2k.ssmp executable after loading one of the above modulefiles.
This version will only run on 2 or more processors up to a maximum of 32.
#!/bin/bash --login #$ -cwd #$ -pe smp.pe 16 # Number of cores. Can be 2--32 in smp.pe # Load either the intel-17.0 version or the binapps version or the singularity version (see above). # For example: module load apps/intel-17.0/cp2k/6.1.0 # Inform cp2k how many cores it can use. $NSLOTS is automatically set to the number above. export OMP_NUM_THREADS=$NSLOTS cp2k.ssmp -i mysim.inp
Submit the jobscript using:
qsub scriptname
where scriptname is the name of your jobscript.
Large parallel batch job submission – 48 cores or more
Ensure you run the cp2k.popt executable after loading one of the above modulefiles.
This version will only run on 2 or more compute nodes – i.e, 48 cores or more in multiples of 24. Only the intel-17.0 version is available for this type of job.
#!/bin/bash --login #$ -cwd #$ -pe mpi-24-ib.pe 48 # Must be 48 cores or more in multiples of 24 # Must load the intel-17.0 version module load apps/intel-17.0/cp2k/6.1.0 # Inform cp2k how many cores it can use. $NSLOTS is automatically set to the number above. mpirun -n $NSLOTS cp2k.popt -i mysim.inp
Submit the jobscript using:
qsub scriptname
where scriptname is the name of your jobscript.
GPU Jobs
The GPU (CUDA) versions of CP2K are available by loading the modulefile:
module load apps/intel-17.0/cp2k/6.1.0-cuda
The executables use the same names as the CPU-only versions (cp2k.sopt, cp2k.ssmp, cp2k.popt).
You do not need to tell CP2K how manu GPUs you have requested in your jobscript. CP2K will get this from the CUDA_VISIBLE_DEVICES environment variable that is automatically set to indicate how many GPUs (and which particular GPUs in the compute node) you job should use.
Note that the number of GPUs you have access to is dependent on whether you are in a contributing research group or not.
Serial GPU job
The following example will use one GPU and one CPU core:
#!/bin/bash --login #$ -cwd #$ -l v100 # 1 GPU (and one CPU core since no -pe given) module load apps/intel-17.0/cp2k/6.1.0-cuda cp2k.sopt -i mysim.inp
Multi-core OpenMP GPU job
GPU jobs may use up to 8 cores per GPU:
#!/bin/bash --login #$ -cwd #$ -pe smp.pe 8 # Up to 8 cores per GPU permitted #$ -l v100 # 1 GPU module load apps/intel-17.0/cp2k/6.1.0-cuda # Inform cp2k how many CPU cores to use export OMP_NUM_THREADS=$NSLOTS cp2k.ssmp -i mysim.inp
Multi-core MPI GPU jobs
Note that it is not possible to run multi-node large parallel GPU jobs. Multi-GPU jobs should use a single compute node. CP2K will grab a GPU per MPI process so you should set the number of MPI processes to be the same as the number of GPUs you wish to use. For example:
#!/bin/bash --login #$ -cwd #$ -pe smp.pe 2 # 2 cores, one for each MPI process #$ -l v100=2 # 2 GPUs, one for each MPI process module load apps/intel-17.0/cp2k/6.1.0-cuda # Each MPI process will grab a GPU mpirun -n $NGPUS cp2k.popt -i mysim.inp
Further info
Updates
None.