ParaView

Overview

ParaView is an open-source, multi-platform data analysis and visualization application. ParaView users can quickly build visualizations to analyze their data using qualitative and quantitative techniques. The data exploration can be done interactively in 3D or programmatically using ParaView’s batch processing capabilities.

Version 5.9.0, 5.9.1, 5.10.1, 5.11.2 of the pvserver and pvbatch OSMesa server tools (binary-install) is installed on the CSF.

How ParaView runs on the CSF and your PC

On CSF4 two methods of running paraview are supported:

1. Client/server mode – the paraview GUI app runs on your PC or laptop – not the CSF. It connects to pvserver processes running in a batch job on the CSF. These do the hard work of processing and rendering your data. The final rendered image is sent back to your PC/laptop to be displayed in the ParaView GUI. The advantage of this is that you DO NOT need to download your data from the CSF.
2. In purely batch mode – pvbatch processes are run in a batch job on the CSF to process a ParaView script. No GUI is used. A final rendered image, movie or some other dataset may be generated and stored in new files on the CSF.

Note that it is not possible to run the Paraview GUI (paraview) directly on the CSF – it is not even installed! The client/server method is by far the most efficient method of using Paraview on the CSF and allows you to easily visualize huge datasets without needing to download the data to your PC and you also do not need a fancy GPU in your PC.

The remainder of this page describes the above two methods in detail. Firstly we cover the client/server method for interactive use of the ParaView GUI.

Summary of the Client / Server method

The first method (the client/server method) is the most common – you will use this to interactively examine your datasets.

It is a little more complicated to set up, and requires you to install the ParaView GUI app on your PC/laptop, but then allows you to visualize huge datasets stored on the CSF without needing a powerful local PC/workstation.

This is a four-step process. Detailed instructions are provided further down on this page, but a summary of the steps is as follows:

1. Firstly, you must install and run the ParaView GUI (paraview) on your local PC/laptop (see below for more details.) Please note: If you need help installing ParaView on a campus desktop or your laptop, please contact the Support Centre and they will arrange for this to be done. The Research Infrastructure team that look after the CSF cannot arrange this.
2. Then you run the parallel processing and rendering part of ParaView (pvserver processes) on the CSF in a batch job.
3. Then you create an SSH (secure shell) tunnel from your PC to the CSF batch job running the pvserver processes.
4. Then you run paraview on your PC/laptop and it will connect to the CSF, via the SSH tunnel, and use the pvserver processes. This is efficient because only the rendered images are sent to your PC over the network to be displayed in the paraview render window. Your datasets (which could be huge) are stored and processed entirely on the CSF.

While this may seem like a lot of steps, it is actually reasonably straight-forward and we provide complete instructions below. It is worth doing – using this method allows you to visualize very large datasets on the CSF from the comfort of your PC/laptop, without having to download those large datasets to your PC/laptop.

Note that the pvserver (or pvbatch) applications will not use any GPU hardware in the CSF. Instead all processing and rendering is performed in software on the compute nodes.

Restrictions on use

There are no restrictions on accessing the software on the CSF. The software is released under a BSD style license and all users should ensure they adhere to the terms of that license.

Installing ParaView on your local PC

You are going to run the paraview GUI on your PC/laptop and connect it to the CSF.

If using a University “public cluster” PC then ParaView might already be installed – check the Start Menu.

To install ParaView on your PC / laptop, please either

• Check the University “Software Centre” app running on managed Windows laptops / desktops – at the time of writing version 5.10.1 is available for install.
• Or, download ParaView with the same version number you will be using on the CSF from the ParaView Download page for your platform (versions are available for Windows, MacOS and Linux – choose whatever your PC/laptop runs). Please note that on Windows you should NOT download the “MPI” version – ignore the downloads that have “-MPI-” in their name.

If you need help installing ParaView on a campus desktop or your laptop, please contact the Support Centre and they will arrange for this to be done. The Research Infrastructure team that look after the CSF cannot arrange this.

Once installed on your PC, do not run ParaView just yet. You first need to start the pvserver job on the CSF and then set up an SSH tunnel on your PC. Full instructions are given below.

Running pvserver in batch on the CSF

You will now submit a batch job (just like any other batch job) to the CSF batch system to run ParaView’s pvserver app.

This is an application to perform processing and rendering as instructed by the ParaView GUI. For example, if you use the ParaView GUI to set up a pipeline that reads a data-file and then performs volume rendering, it is the pvserver processes running on the CSF that will do the actual file reading and volume rendering. The rendered image generated will be sent back to your ParaView GUI so that you can see it in the GUI.

The pvserver app can be run as a large CSF job, across multiple compute nodes, or as a smaller job on a few CPU cores in a single compute node. Examples of both types of jobs are given below.

Whether you submit a smaller single-node job or a much larger multi-node job mainly depends on how much data you will be loading in to paraview. Typically each pvserver process has access to approximately 4GB of RAM. So you may be able to estimate how many pvserver processes (and hence CPU cores) your job will need to be able to load and process your datasets. You may need to experiment – try submitting a smaller single-node job (e.g., 4 cores) to begin with. If your job runs out of memory, submit a new job with more CPU cores.

We now recommend loading modulefiles within your jobscript so that you have a full record of how the job was run. See the example jobscripts below for how to do this. Load the following modulefile:

# Note: The version of paraview installed on your PC must be the same as that used on the CSF.
# Load one of the following:
module load paraview/5.11.2
module load paraview/5.10.1
module load paraview/5.9.1
module load paraview/5.9.0

Small single-node (multicore) pvserver job

Create the following jobscript (e.g., naming it pvserver.sh). NOTE: the --force-offscreen-rendering rendering flag is required.

#!/bin/bash --login
#SBATCH -p multicore   # A single-node multicore job
#SBATCH -n 8           # (or --ntasks=8) number of cores - can be 2--40.

# We now recommend loading the modulefile in the jobscript
module purge
module load paraview/5.10.1

# mpiexec knows to use the number of cores requested above
mpiexec pvserver --system-mpi --force-offscreen-rendering
                     #
                     # Note: This is specific to the CSF4 (it isn't used on CSF3)

Then submit the above jobscript using

sbatch jobscript

where jobscript is the name of your file (e.g., pvserver.sh)

Large multi-node (multicore) pvserver job

Create the following jobscript (e.g., naming it pvserver-ib.sh). NOTE: the --force-offscreen-rendering rendering flag is required.

#!/bin/bash --login
#SBATCH -p multinode   # A multi-node job
#SBATCH -n 80          # (or --ntasks=80) number of cores, minimum is 80, must be in multiples of 40.

# We now recommend loading the modulefile in the jobscript
module load paraview/5.10.1

# mpiexec knows to use the number of cores requested above
mpiexec pvserver --system-mpi --force-offscreen-rendering
                     #
                     # Note: This is specific to the CSF4 (it isn't used on CSF3)

Then submit the above jobscript using

sbatch jobscript

where jobscript is the name of your file (e.g., pvserver-ib.sh)

Find where the pvserver batch job runs

You must now wait until the pvserver job is running. Use sbatch to check your job queue. Once the state reported by sbatch has changed from PD to R you will see something like:

JOBID PRIORITY PARTITION NAME     USER  ACC ST SUBMIT_TIME START_TIME  TIME NODES CPUS NODELIST(REA
24914 0.021350 multinode pvser... uname xy01 R 08/12/21... 08/12/21... 5:21     2   80 node[056-057]

Make a note of the NODELIST(REASON) value (yes really, please make a note if it – you need this info later), specifically the first node (if a range of node numbers is given) – e.g., node[056-057]. In this example we see node056 is the first node in the list of nodes your job is running on.

You should now check in the job output that pvserver is waiting for you to connect to it:

# Go to where you submit the job from. For example:
cd ~/scratch/data

# Check the end of the job output file for a message from paraview:
cat slurm-24914.out

# You should see something similar to
Waiting for client...
Connection URL: cs://node056.csf4.local:11111
Accepting connection(s): node056.csf4.local:11111

# If the above message is not visible, wait a minute and run
# the 'cat' command again. This shows that pvserver is ready
# for you to connect your paraview GUI to it.

It may take a minute or two before the pvserver job displays the above message in the slurm-NNNNNN.out file. Please wait until you see this message from the job.

Assuming you can see a message similar to the above, we are now going to do some steps on your local PC/laptop to run the paraview GUI there and connect it to the pvserver batch job.

Setting up the SSH tunnel from your PC to the CSF

We will now set up the SSH tunnel to the CSF from your PC/laptop.

This allows the ParaView GUI on your PC to communicate directly with your CSF batch job running the pvserver processes. The method used is very similar to how you log in to the CSF, just with some extra command-line flags, so don’t be put off about doing this step. You will need the information from the squeue command that we noted above.

Separate instructions are given for Windows and Linux/MacOS.

At this stage you must have ParaView installed on your local PC/laptop (see above). It doesn’t matter if the ParaView GUI is running or not yet on your PC.

SSH Tunnel on Linux or MacOS

In a shell (Terminal) window on your PC/laptop, create a tunnel between your local port 11111 and the CSF backend node’s port 11111 where the pvserver job is running. This will go via the CSF login node. In this example we assume the pvserver job is running on node node056.

ssh -L 11111:node056:11111 username@csf4.itservices.manchester.ac.uk
                 #             #
                 #             # Use your UoM username
                 #
                 # Use the node number that you made a note of earlier (see squeue output)

# Enter your University IT password when prompted and respond to any DUO prompt if shown.

where username is your CSF username and node056 is the node where your pvserver job is running (use squeue on the CSF to find out). Do not close this shell. This connection must remain open to use ParaView. You do not need to type any commands in to this shell.

SSH Tunnel on Windows

You must have an SSH client, such as MobaXterm (or PuTTY), installed. Note that we are going to run a locally-installed Windows version of the ParaView GUI. The SSH application is just to set up the SSH tunnel.

If using MobaXTerm, start MobaXTerm on your PC then simply copy the ssh command given in the Linux instructions above in to the black MobaXTerm window (don’t forget to use the appropriate node name – e.g., node056 – appropriate to your job). For example:

ssh -L 11111:node056:11111 username@csf4.itservices.manchester.ac.uk
                 #             #
                 #             # Use your UoM username
                 #
                 # Use the node number that you made a note of earlier (see squeue output)

# Enter your University IT password when prompted and respond to any DUO prompt if shown.

PuTTY instructions coming soon…

Run the Paraview GUI on your PC/laptop

Now start your local installation of paraview on your PC/laptop:

• Linux or MacOS: Run the following command in a Terminal (aka Shell) window:

  paraview --server-url=cs://localhost:11111

  ParaView will connect to the pvserver processes through your local port 11111. This has been tunnelled to the CSF backend node where the processes are actually running. You can now jump to Render Settings to ensure everything is working correctly.

• Windows: Run paraview by selecting it from the Start menu (or desktop icon if you have one). Then follow the steps below.

Connecting the ParaView GUI to your CSF job

Note that Linux and MacOS users do not need to do these steps if you ran paraview on your PC/laptop using:

paraview --server-url=cs://localhost:11111

because that command will have achieved what the following steps will do. The steps below are mostly for Windows users who will need to do the steps using the ParaView GUI.

Once the ParaView GUI has appeared, do the following:

1. Go to File > Connect
2. In the Choose Server Configuration window that pops up, select Add Server
• 
3. Fill in the form to connect to your local port 1111 using the following then hit Configure:
• Name: localhost
• Server Type: Client / Server
• Host: localhost
• Port: 11111
• 
4. In the Server Launch Configuration window ensure Manual is selected and hit Save
• 
5. Now select your saved configuration and hit Connect
• 

ParaView will connect to the pvserver processes through your local port 11111. This has been tunnelled to the CSF backend node where the processes are actually running.

Now jump to Render Settings to ensure everything is working correctly.

Render Settings

To ensure your local paraview client does no rendering work you should change the render settings in paraview. The default is to have your PC do a little bit of rendering work. We don’t want it to do any rendering otherwise it would have to transfer more data from the CSF to your PC.

1. In the paraview GUI go to Edit > Settings…
2. In the Render View tab, type in a zero 0 in the Remote Render Threshold box, as shown below.
3. 

This will ensure all geometry data and rendering tasks remain on the CSF and only the final rendered image is delivered to your PC.

Testing the Connection to the CSF

A useful way of checking what the remote pvserver processes are doing is to load a standard ParaView geometry (e.g., a sphere), or one of your own datasets, and then colour the data according to the processor ID on which the data is residing (ParaView distributes the gemetry amongst the pvserver processes). Hence you will see different colours for each pvserver used in the CSF batch job:

1. Go to the Sources > Geometric > Sphere menu item to add a sphere to your pipeline.
   
2. Hit the Apply button to display the sphere. The sphere’s geometry data will exist on the CSF (in the pvserver processes and be rendered by those processes. The final image of the sphere is then being sent back to your PC to be displayed in the ParaView GUI.
   
3. Change the Solid Color drop-down menu in the toolbar (above the pipeline browser) to be vtkProcessID. The will change how the sphere geometry is rendered by assigning the pvserver process ID to the parts of the sphere each process is responsible for.
   
4. You should now see that the sphere is rendered using 8 different colours. In this example we ran a CSF jobs with 8 pvserver processes. Hence 8 different colours are used to colour the sphere because each process is responsible for different sections of the sphere’s geometry.
   

Loading your own Datasets on the CSF

You can also load in your own dataset and apply the same technique as above. You will also notice that when you open the file browser, it will show you your files on the CSF, not those on your local PC. This is because the pvserver processes are running on the CSF and it is those processes that will read your data. If you want to visualize a dataset that is stored on your PC, you should transfer it to the CSF (using your preferred file transfer application) and then open the file in the ParaView GUI.

Exit and Clean up

Exiting from the paraview GUI will stop the backend pvserver processes, at which point you will have no ParaView jobs running in the batch system.

Close the SSH tunnel

Don’t forget to close the SSH tunnel. Otherwise you might not be able to make ParaView work correctly if you need to run it again using a new pvserver batch job.

Simply type

# In the Linux/MacOS shell or MobaXterm window where you ran the "ssh -L 11111:node056 ....." command:
exit

in the shell window that is logged in to the CSF with the tunnel enabled. It will log you out of the CSF.

Hints and Tips

If the paraview GUI freezes or hangs it usually indicates a problem with the backend pvserver processes. Check the pvserver.sh.oNNNNN or pvserver.sh.eNNNNN log files created by the batch job (the pvserver.sh name will actually be what ever your jobscript was called). The most common problem is that the pvserver processes run out of memory because you don’t have enough running for the size of your dataset.

If your local paraview GUI crashes you should run qstat on the CSF to check if the pvserver processes are still running. If so, use qdel NNNNNN to kill the batch job.

If you can’t create the SSH tunnel, check you haven’t already got an SSH tunnel still open from a previous run of PaeaView. Exit that logged-in session before starting a new one.

Running non-interactive (scripted) pvbatch (large datasets)

The pvbatch application allows you to run ParaView scripts without using the GUI. The entire job runs on the CSF, in batch, just like any other CSF job. The paraview GUI should not be running at this point.

After loading the ParaView modulefile use a jobscript to launch the pvbatch MPI processes. For example, to launch 4 backend processes, create the following jobscript (e.g., naming it pvbatch.sh):

#!/bin/bash --login
#SBATCH -p multicore     # Single-node multicore job
#SBATCH -n 8             # (or --ntasks=8) number of cores - can be 2-40.
### OR ###
#SBATCH -p multicore     # Multi-node job
#SBATCH -n 80            # (or --ntasks=80) number of cores - must be 80 or more in multiples of 40.

# Ensure the paraview modulefile is loaded
module purge
module load paraview/5.9.1

# Run pvbatch - you do not control this with the paraview GUI
mpiexec pvbatch --system-mpi scriptfile
                     #         #
                     #         # where scriptfile is the file
                     #         # containing your paraview script
                     #
                     # This flag is specific to the CSF4

Then submit the above jobscript using

sbatch jobscript

where jobscript is the name of your file (e.g., pvbatch.sh)

Please see Online Paraview Guide for more information on using pvbatch.

Further info

• Paraview website
• Online Paraview Guide

Updates

None.