dfnWorks

Overview

dfnWorks is a parallelized computational suite to generate three-dimensional discrete fracture networks (DFN) and simulate flow and transport. Developed at Los Alamos National Laboratory, it has been used to study flow and transport in fractured media at scales ranging from millimeters to kilometers.

As stated in their documentation, dfnWorks currently runs on Macs and Unix machine running Ubuntu. Using Docker image is recommended method of using dfnWorks. CSF3 has Singularity installed for supporting containerised application therefore dfnWorks Singularity image was built from dfnWorks Docker image for users.

The dfnWorks Singularity image in CSF3 contains version 2.8 of dfnWorks program

Restrictions on use

dfnWorks is open source and freely distributed. It is released under GNU Lesser General Public License as published by the Free Software Foundation; either version 3.0 of the License, or (at your option)any later version.

Please follow this link for further information  – dfnWorks Copyright Information

Set up procedure

Users of dfnWorks must first create a file ~/.dfnworksrc in their CSf3 account containing the following before they are able to run dfnWorks:

{
    "dfnworks_PATH": "/dfnWorks/",
    "PETSC_DIR": "/dfnWorks/lib/petsc",
    "PETSC_ARCH": "arch-linux2-c-debug",
    "PFLOTRAN_EXE": "/dfnWorks/bin/pflotran",
    "PYTHON_EXE": "/usr/bin/python",
    "LAGRIT_EXE": "/dfnWorks/bin/lagrit",
    "FEHM_EXE": "/dfnWorks/bin/fehm"
}

We now recommend loading modulefiles within your jobscript so that you have a full record of how the job was run. See the example jobscript below for how to do this.

To use the software you must load ONE of the modulefiles:

dfnWorks v2.8 (Install date 15/08/2024)

module load apps/singularity/dfnWorks/2.8-new

dfnWorks v2.8 (Install date 15/06/2024)

module load apps/singularity/dfnWorks/2.8

Examples

Examples that came along with this version of dfnWorks are available under:

/opt/apps/apps/singularity/dfnWorks/2.8/examples

directory in CSF3.

Users can copy examples and run them for testing, which can help them get started.

Running the application

Please do not run dfnWorks on the login node. Jobs should be submitted to the compute nodes via batch.

Ideally, dfnWorks should be run in a parallel environment.

Users must specify the number of CPU cores they want to use for the job in these two places and they should be same:

(1) In their jobscript in the line #$ -pe smp.pe 8
(2) In their Python script (driver.py in this example) using the flag: ncpu=8

Excerpt from example Python script ‘driver.py’

...
DFN = DFNWORKS(jobname,
               dfnFlow_file=dfnFlow_file,
               dfnTrans_file=dfnTrans_file,
               ncpu=8)
...

Parallel batch job submission

Create a batch submission script (which will load the modulefile in the jobscript), for example:

#!/bin/bash --login
#$ -cwd             # Job will run from the current directory
#$ -pe smp.pe 8     # Set the number of CPU required for the job


# Load the version you require
module load apps/singularity/dfnWorks/2.8

singularity exec --bind /scratch:/scratch,/opt:/opt $DFNWORKS_SIF python driver.py
#                       ↑                 ↑                               ↑
#                       |                 |                               |
#                       |                 |           Should have flag ncpu set in this file.
#                       |                 Without this Parallel jobs will fail
#                       To bind /scratch directories

Submit the jobscript using:

qsub scriptname

where scriptname is the name of your jobscript.

Further info

dfnWorks

Updates

None.