{"id":1609,"date":"2018-12-14T11:52:05","date_gmt":"2018-12-14T11:52:05","guid":{"rendered":"http:\/\/ri.itservices.manchester.ac.uk\/csf3\/?page_id=1609"},"modified":"2024-01-03T16:14:42","modified_gmt":"2024-01-03T16:14:42","slug":"cp2k","status":"publish","type":"page","link":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/software\/applications\/cp2k\/","title":{"rendered":"CP2K"},"content":{"rendered":"

Overview<\/h2>\n

CP2K is a program to perform atomistic and molecular simulations of solid state, liquid, molecular, and biological systems. It provides a general framework for different methods such as e.g., density functional theory (DFT) using a mixed Gaussian and plane waves approach (GPW) and classical pair and many-body potentials.<\/p>\n

Version 6.1.0 is installed on the CSF. It was compiled with Intel Compilers 17.0.7 & MKL. Serial, multi-core (OpenMP) and multi-node (MPI) versions are available, and all three are available with cuda support.<\/p>\n

Restrictions on use<\/h2>\n

The software is open source under the GNU General Public License.<\/p>\n

Set up procedure<\/h2>\n

We now recommend loading modulefiles within your jobscript so that you have a full record of how the job was run. See the example jobscript below for how to do this. Alternatively, you may load modulefiles on the login node and let the job inherit these settings<\/abbr>.<\/p>\n

To access the software you must first load the appropriate modulefile. Several different flavours are available:<\/p>\n

The singularity<\/code> modulefile allows access to more recent versions of the SMP (multicore) version. The singularity container was converted from a Docker container, where the Docker container was created by the CP2K developers.<\/p>\n

\r\n# Features<\/strong>: TBC\r\nmodule load apps\/singularity<\/strong>\/cp2k\/2023.2          # Version: cp2k.ssmp (multi-core parallel, OpenMP)\r\n<\/pre>\n
The binapps<\/code> modulefile sets up to use the pre-compiled version built by the CP2K developers. It has more features enabled but may be less optimized for CSF hardware. <\/p>\n
\r\n# Features<\/strong>: libint fftw3 libxc xsmm libderiv_max_am1=6 libint_max_am=7 max_contr=4\r\nmodule load apps\/binapps<\/strong>\/cp2k\/6.1.0          # Versions: cp2k.sopt (serial)\r\n                                             #           cp2k.ssmp (multi-core parllel, OpenMP)\r\n<\/pre>\n
The intel-17.0<\/code> modulefile set up to use the version built by the Research Infrastructure team. It has fewer features enabled but may be more optimized for CSF hardware.<\/p>\n
\r\n# Features<\/strong>: fftw3 parallel mpi3 scalapack mkl\r\nmodule load apps\/intel-17.0<\/strong>\/cp2k\/6.1.0       # Versions: cp2k.sopt (serial)\r\n                                             #           cp2k.ssmp (multi-core parallel, OpenMP)\r\n                                             #           cp2k.popt (multi-node parallel, MPI)\r\n\r\n# Features<\/strong>: fftw3 parallel mpi3 scalapack acc pw_cuda dbcsr_acc mkl (must be run on a GPU node<\/strong>)\r\nmodule load apps\/intel-17.0<\/strong>\/cp2k\/6.1.0-cuda  # Versions: cp2k.sopt (serial)\r\n                                             #           cp2k.ssmp (multi-core parallel, OpenMP)\r\n                                             #           cp2k.popt (multi-node parallel, MPI)\r\n<\/pre>\n
Running the application<\/h2>\n
Please do not run cp2k on the login node. Jobs should be submitted to the compute nodes via batch.<\/p>\n
Serial batch job submission<\/h3>\n
Ensure you run the cp2k.sopt<\/code> executable after loading one of the above modulefiles.<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n\r\n# Load either the intel-17.0 version or the binapps version (see above). For example:\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0\r\n\r\ncp2k.sopt -i mysim.inp<\/em>\r\n<\/pre>\n
Submit the jobscript using: <\/p>\n
qsub scriptname<\/em><\/pre>\n
where scriptname<\/em> is the name of your jobscript.<\/p>\n
Small parallel batch job submission – 2 to 32 cores<\/h3>\n
Ensure you run the cp2k.ssmp<\/code> executable after loading one of the above modulefiles.
\nThis version will only run on 2 or more processors up to a maximum of 32.<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n#$ -pe smp.pe 16          # Number of cores. Can be 2--32 in smp.pe\r\n\r\n# Load either the intel-17.0 version or the binapps version or the singularity version (see above).\r\n# For example:\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0\r\n\r\n# Inform cp2k how many cores it can use. $NSLOTS is automatically set to the number above.\r\nexport OMP_NUM_THREADS=$NSLOTS\r\ncp2k.ssmp -i mysim.inp<\/em>\r\n<\/pre>\n
Submit the jobscript using: <\/p>\n
qsub scriptname<\/em><\/pre>\n
where scriptname<\/em> is the name of your jobscript.<\/p>\n
Large parallel batch job submission – 48 cores or more<\/h3>\n
Ensure you run the cp2k.popt<\/code> executable after loading one of the above modulefiles.
\nThis version will only run on 2 or more compute nodes<\/em> – i.e, 48 cores or more in multiples of 24. Only the intel-17.0<\/code> version is available for this type of job.<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n#$ -pe mpi-24-ib.pe 48         # Must be 48 cores or more in multiples of 24\r\n\r\n# Must load the intel-17.0 version\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0\r\n\r\n# Inform cp2k how many cores it can use. $NSLOTS is automatically set to the number above.\r\nmpirun -n $NSLOTS cp2k.popt -i mysim.inp<\/em>\r\n<\/pre>\n
Submit the jobscript using: <\/p>\n
qsub scriptname<\/em><\/pre>\n
where scriptname<\/em> is the name of your jobscript.<\/p>\n
GPU Jobs<\/h2>\n
The GPU (CUDA) versions of CP2K are available by loading the modulefile:<\/p>\n
\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0-cuda\r\n<\/pre>\n
The executables use the same names as the CPU-only versions (cp2k.sopt<\/code>, cp2k.ssmp<\/code>, cp2k.popt<\/code>).<\/p>\n
You do not need to tell CP2K how manu GPUs you have requested in your jobscript. CP2K will get this from the CUDA_VISIBLE_DEVICES<\/code> environment variable that is automatically set to indicate how many GPUs (and which particular GPUs in the compute node) you job should use.<\/p>\n
Note that the number of GPUs you have access to is dependent on whether you are in a contributing research group or not.<\/p>\n
Serial GPU job<\/h3>\n
The following example will use one GPU and one CPU core:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n#$ -l v100                # 1 GPU (and one CPU core since no -pe given)\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0-cuda\r\ncp2k.sopt -i mysim.inp<\/em>\r\n<\/pre>\n
Multi-core OpenMP GPU job<\/h3>\n
GPU jobs may use up to 8 cores per GPU:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n#$ -pe smp.pe 8           # Up to 8 cores per GPU permitted\r\n#$ -l v100                # 1 GPU\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0-cuda\r\n\r\n# Inform cp2k how many CPU cores to use\r\nexport OMP_NUM_THREADS=$NSLOTS\r\ncp2k.ssmp -i mysim.inp<\/em>\r\n<\/pre>\n
Multi-core MPI GPU jobs<\/h3>\n
Note that it is not<\/em> possible to run multi-node large parallel GPU jobs. Multi-GPU jobs should use a single compute node. CP2K will grab a GPU per MPI process so you should set the number of MPI processes to be the same as the number of GPUs you wish to use. For example:<\/p>\n
\r\n#!\/bin\/bash --login\r\n#$ -cwd\r\n#$ -pe smp.pe 2           # 2 cores, one for each MPI process\r\n#$ -l v100=2              # 2 GPUs, one for each MPI process\r\nmodule load apps\/intel-17.0\/cp2k\/6.1.0-cuda\r\n\r\n# Each MPI process will grab a GPU\r\nmpirun -n $NGPUS cp2k.popt -i mysim.inp<\/em>\r\n<\/pre>\n
Further info<\/h2>\n