incompact3d

Overview

incompact3d an efficient code to solve the incompressible Navier-Stokes equations on supercomputers

Version 2.0 is installed on the CSF. It was compiled with gcc 4.4.6, openmp 1.6 and fftw3 double-mpi 3.2.2

Restrictions on use

This software is open source under the GNU GPL v3 licence so all CSF users may use it. As with all software please ensure you acknowledge use of it appropriately in publications etc. Consult the incompact3d website for the relevent information.

Set up procedure

To access the software you must first load the modulefile:

To run on up to 24 cores (Intel none infiniband (ib) nodes):

module load apps/gcc/incompact3d/2.0

To run on two or more intel compute nodes (48 cores or more) or 2 or more AMD nodes (128 cores or more).

module load apps/gcc/incompact3d/2.0-ib

Running the application

Please do not run incompact3d on the login node. Jobs should be submitted to the compute nodes via batch.

Serial batch job submission

It is not recommended to run this application in serial.

Parallel batch job submission

Between 2 and 24 cores (intel)

Make sure you have the non-ib modulefile loaded (see above).

Write a job submission script and use smp.pe:

#!/bin/bash
#$ -cwd
#$ -V
#$ -pe smp.pe 8

mpirun -n $NSLOTS incompact3d

Submit with: qsub jobscript

48 cores or more, in multiples of 24 (intel)

Please DO NOT use this example to run 24 core jobs – such jobs can run in the smp.pe environment (i.e., the job will fit on a single compute node).

Make sure you have the -ib version of the modulefile loaded (see above).

Write a job submission script and use orte-24-ib.pe:

#!/bin/bash
#$ -cwd
#$ -V
#$ -pe orte-24-ib.pe 48

mpirun -n $NSLOTS incompact3d

Submit with: qsub jobscript

AMD nodes

Make sure you have the ib modulefile loaded (see above).

Write a job submission script based on those in the other examples but replace the pe with one of the following:

• smp-64bd.pe for jobs of 2-64 cores on bulldozer
• orte-64bd-ib.pe for jobs of 128 cores or more, in multiples of 64 on bulldozer
• smp-32mc.pe for jobs of 2 to 32 cores on magny-cours
• orte-32-ib.pe for jobs of 64 or more on magny-cours

Submit with: qsub jobscript

Further info

• Incompact3d website

Updates

None.