StarCCM+<\/a> is a computation continuum mechanics application which can handle problems relating to fluid flow, heat transfer and stress. See below for the available versions on the CSF. When the job runs there may not be enough licenses for the number of cores you have requested. There is currently a high demand for licenses and they may run out. To check whether there are at least enough licenses for you job you can add the following line to your jobscript<\/strong>:<\/p>\n (that’s a full-stop followed by a space at the start of the line – please copy it carefully.)<\/p>\n If there are not<\/em> enough licenses for your job the job will automatically re-queue in the CSF batch system. If there are enough licenses the job will run StarCCM. If you omit this check your job will simply fail if there are not enough licenses.<\/p>\n Once you have been added to the StarCCM+ group, you will be able to access the modulefiles. We now recommend loading modulefiles within your jobscript so that you have a full record of how the job was run. See the example jobscript below for how to do this. Alternatively, you may load modulefiles on the login node and let the job inherit these settings<\/abbr>.<\/p>\n Load one<\/em> of the following modulefiles:<\/p>\n Currently, use of StarCCM+ in batch mode only is supported; no attempt should be made to use the StarCCM+ GUI directly on the CSF (however, see below for the client-server<\/a> method). Once you have loaded the modulefile for the version you wish to run please write a jobscript based on one of those below and then submit it to the batch system with this command:<\/p>\n replacing These are jobs that use a single AMD compute node – between 2 and 168 cores.<\/p>\n This example describes how to submit an SMP job (uses multiple cores in a single compute node):<\/p>\n If submitting a job with fewer than 168 cores, then you should add the following flag to the For example:<\/p>\n If your job uses exactly<\/em> 168 cores you must not<\/em> use this flag – it will slow down your job.<\/p>\n Only users who are members of a valid HPC Pool project, and who know their HPC Pool project code, can run multi-node jobs in the HPC Pool.<\/p>\n If you are not a member of an HPC Pool project you cannot<\/em> run in the Do not assume that using twice as many cores will run you job in half the time! Jobs do not always scale linearly. Users are encouraged to test how their problem scales when considering a new problem or mesh size. This can be achieved easily, using an inbuilt script provided by CD-Adapco, which can be invoked using the This will run a few iterations on 1, 2, 4, 8, 16, … and finally 128 cores, timing the runs. The output is a This is a multi-node example using four HPC Pool 32-core nodes (hence 128 cores in total). You can specify more nodes by using multiples of 32 for the number of cores (e.g., 256 for eight nodes).<\/p>\n Note that some users have found using IntelMPI faster on the HPC Pool. This has been tested in v18 (so should be OK in newer versions):<\/p>\n However larger jobs (320 cores and up, may not run with IntelMPI.) Please try openmpi first.<\/p>\n Checkpointing saves the current state of your simulation to file so that you can run the job again from the current state rather than from the beginning of the simulation. This is needed if your simulation is going to run for longer than the maximum runtime permitted (7 days on the Intel nodes in the CSF).<\/p>\n When asked to checkpoint, StarCCM+ will write out a new To force a checkpoint manually, leaving your batch job to carry on running the simulation after the checkpoint file has been written, run the following command on the login node in the directory from where you submitted the job:<\/p>\n StarCCM+ checks after every iteration for this file. Once it sees the file it will save the current state of the simulation then rename the If you wish to checkpoint and then terminate your simulation (which will end your CSF job), run the following on the login node in the directory where your simulation is running:<\/p>\n Your batch job will then terminate after it has written the checkpoint file.<\/p>\n It is possible to automatically checkpoint your job near the end of the job’s runtime and have it re-queue itself on the CSF using the jobscript as shown below.<\/p>\n In this example we run a simulation file named Note, it is important to change the Note that the script below does not delete any checkpoint files. If your simulation runs for a long time it may checkpoint a number of times and leave you with many large checkpoint files. You can periodically delete all but the most recent of these files.<\/p>\n <\/a><\/p>\n The StarCCM+ GUI running on a campus desktop PC can be connected to a batch simulation running on the CSF. This allows the GUI to display the current state of the simulation (for example you can see graphs showing how particular variables are converging).<\/p>\n Note that the method below will mean that the CSF job, once it is running, does NOT automatically start the simulation. Instead StarCCM+ will wait for you to connect the GUI to the job. But the CSF job is running and will be consuming its available runtime (max 7 days on the CSF).<\/p>\n Please follow the instructions below:<\/p>\n Submit the job using At the end of the file will be a message informing you where the job is running and on which port<\/em> the server is listening:<\/p>\n Enter your CSF password when asked. You must leave this window logged in all time while using the StarCCM+ GUI on your PC. The GUI will be communicating with the CSF through this log-in (tunnel<\/em>). You do not need to type any commands in to this login window.<\/li>\n Then hit OK. The GUI will connect to the job on the CSF.<\/li>\n It is possible to have STAR-CCM+ perform some calculations with Abaqus, exchanging data between the two applications. For example, in mechanical co-simulation STAR-CCM+ passes traction loads to Abaqus (pressure + wall shear stress), and Abaqus passes displacements to STAR-CCM+. In Abaqus, the traction loads are applied to the surface of the solid structure. In STAR-CCM+, the displacements are used as an input to the mesh morpher. Data is exchanged via the Co-Simulation module of STAR-CCM+.<\/p>\n You will need to set up the co-simulation in your STAR-CCM+ input file and also have available an Abaqus input file. You should also ensure the input files, when run in their respective applications, converge to solutions otherwise the co-simulation will not converge.<\/p>\n More information is available in the STAR-CCM+ user guide, available on the login node by running the following command after you’ve loaded the starccm modulefile:<\/p>\n In this example we will run a single-node 32-core job with 24 cores used by starccm and 8 cores used by Abaqus.<\/p>\n Create a directory structure for your co-simulation:<\/p>\n Now copy your input files to the respective directories. For example:<\/p>\n Now make some changes to the STAR-CCM+ input file to enable co-simulation. You can do this on your local workstation if you prefer but it is useful to be able to do this on the CSF when you want to change the settings before submitting a job – you will avoid transferring the input file back and forth between your workstation:<\/p>\n When the GUI starts, open the Save the input file and exit the STAR-CCM+ GUI.<\/p>\n Now create a jobscript to load the starccm and abaqus modulefiles, and run starccm with the correct number of cores:<\/p>\n The jobscript should contain the following:<\/p>\n Submit the job using<\/p>\n When the job runs you will see output files written to the The simulation should run until it is converged.<\/p>\n Further information on StarCCM+ and other CFD applications may be found by visiting the MACE CFD Forum<\/a>.<\/p>\n The STAR-CCM+ user guide is available on the CSF using the following command after<\/em> you have loaded the starccm modulefile:<\/p>\n Overview StarCCM+ is a computation continuum mechanics application which can handle problems relating to fluid flow, heat transfer and stress. See below for the available versions on the CSF. Please note that the CSF Team do not supply copies of STARCM+ for laptops\/desktops, you will need to request that via the Research Applications Team. Restrictions on Use Only users who have been added to the StarCCM group can run the application (run groups to see.. Read more »<\/a><\/p>\n","protected":false},"author":7,"featured_media":0,"parent":86,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-743","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/743","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/comments?post=743"}],"version-history":[{"count":23,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/743\/revisions"}],"predecessor-version":[{"id":11441,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/743\/revisions\/11441"}],"up":[{"embeddable":true,"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/pages\/86"}],"wp:attachment":[{"href":"https:\/\/ri.itservices.manchester.ac.uk\/csf3\/wp-json\/wp\/v2\/media?parent=743"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nPlease note that the CSF Team do not supply copies of STARCM+ for laptops\/desktops, you will need to request that via the<\/strong><\/em> Research Applications Team<\/a>.<\/p>\nRestrictions on Use<\/h2>\n
groups<\/code> to see your group memberships). Owing to licence restrictions, only users from the School of MACE can be added to this group. Requests to be added to the StarCCM group should be requested via the appropriate HPC Help Form<\/a>.<\/div>\n. $STARCCM_HOME\/liccheck.sh\r\n<\/pre>\n
Set Up Procedure<\/h2>\n
# Choose required precision (double is more accurate but slower)\r\nmodule load apps\/binapps\/starccm\/20.04-double # Also called v2025.06\r\nmodule load apps\/binapps\/starccm\/20.04-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/19.04-double # Also called v2024.06\r\nmodule load apps\/binapps\/starccm\/19.04-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/19.02-double # Also called v2024.02\r\nmodule load apps\/binapps\/starccm\/19.02-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/18.02-double # Also called v2023.02\r\nmodule load apps\/binapps\/starccm\/18.02-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/17.04-double \r\nmodule load apps\/binapps\/starccm\/17.04-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/17.02-double # Also called v2022.1\r\nmodule load apps\/binapps\/starccm\/17.02-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/15.04-double # Also called v2020.2.1\r\nmodule load apps\/binapps\/starccm\/15.04-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/15.02-double # Also called v2020.1\r\nmodule load apps\/binapps\/starccm\/15.02-mixed\r\n\r\nmodule load apps\/binapps\/starccm\/14.06-double \r\nmodule load apps\/binapps\/starccm\/14.06-mixed \r\n\r\nmodule load apps\/binapps\/starccm\/13.04-double \r\nmodule load apps\/binapps\/starccm\/13.04-mixed\r\n<\/pre>\n
Running the Application<\/h2>\n
sbatch myjobscript<\/em>\r\n<\/pre>\n
myjobscript<\/em><\/code> with the name of your file.<\/p>\nParallel Jobs<\/h2>\n
Jobs of up to 168 cores<\/h3>\n
#!\/bin\/bash --login\r\n#SBATCH -p multicore # (or --partition=) Run on the AMD 168-core nodes\r\n#SBATCH -n 16 # (or --ntasks=) Number of cores to use.\r\n#SBATCH -t 4-0 # Wallclock time limit. 4-0 is 4 days. Max permitted is 7-0.\r\n\r\n# We now recommend loading the modulefile in the jobscript. Use your required version\r\nmodule load apps\/binapps\/starccm\/17.04.008-mixed\r\n\r\n# Check for licenses, will requeue job if not enough available.\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\nstarccm+ -batch -pio -mpi openmpi -batchsystem slurm myinput.sim<\/em> \r\n # #\r\n # #\r\n # # Replace myinput.sim<\/em> with\r\n # # your input file name.\r\n #\r\n # Optional: may speed up your job by performing parallel file I\/O.\r\n # BUT: you MUST be running your job from the \"scratch\" filesystem\r\n # for this to work.\r\n\r\n\r\n<\/pre>\n
starccm+<\/code> command-line (e.g., at the end of the line):<\/p>\n-cpubind off\r\n<\/pre>\n
starccm+ -batch -mpi openmpi -batchsystem slurm myinput.sim<\/em> -cpubind off<\/strong>\r\n<\/pre>\n
Large Parallel Jobs – HPC Pool Users<\/h2>\n
hpcpool<\/code> partition.<\/p>\n<\/div>\nPlease check how your job scales before running large parallel jobs!<\/h3>\n
-benchmark<\/code> flag. For example:<\/p>\n#!\/bin\/bash --login\r\n#SBATCH -p hpcpool # The \"partition\" - named hpcpool\r\n#SBATCH -N 4 # (or --nodes=) Minimum is 4, Max is 32. Job uses 32 cores on each node.\r\n#SBATCH -n 128 # (or --ntasks=) TOTAL number of tasks. Max is 1024.\r\n#SBATCH -t 1-0 # Wallclock limit. 1-0 is 1 day. Maximum permitted is 4-0 (4-days).\r\n#SBATCH -A hpc-proj-name<\/em> # Use your HPC project code\r\n\r\n# We now recommend loading the modulefile in the jobscript. Use your required version\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008-mixed\r\n\r\n# Check for licenses, requeue if not enough available.\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\nstarccm+ -benchmark<\/strong> -mpi openmpi -batchsystem slurm myinput.sim<\/em>\r\n # #\r\n # Run the automatic benchmarks #\r\n # on your input model # Replace myinput.sim<\/em> with\r\n # your input file name.\r\n<\/pre>\n
.html<\/code> file which can be opened using a web browser. This file will show how your problem scales, which can be used to decide how many cores to use. The parallel efficiency can be computed as speedup \/ number of workers from the resulting table. The point at which efficiency drops below ~85% users should consider using fewer cores, which may actually increase the speed of your computations.<\/p>\nMulti-node jobs of 128 cores or more<\/h3>\n
#!\/bin\/bash --login\r\n#SBATCH -p hpcpool # The \"partition\" - named hpcpool\r\n#SBATCH -N 4 # (or --nodes=) Minimum is 4, Max is 32. Job uses 32 cores on each node.\r\n#SBATCH -n 128 # (or --ntasks=) TOTAL number of tasks. Max is 1024.\r\n#SBATCH -t 1-0 # Wallclock limit. 1-0 is 1 day. Maximum permitted is 4-0 (4-days).\r\n#SBATCH -A hpc-proj-name<\/em> # Use your HPC project code\r\n\r\n# We now recommend loading the modulefile in the jobscript. Use your required version\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008-mixed\r\n\r\n# Check for licenses, requeue if not enough available.\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\nstarccm+ -batch -pio -mpi openmpi -batchsystem slurm myinput.sim<\/em>\r\n #\r\n #\r\n # Replace myinput.sim<\/em> with\r\n # your input file name.\r\n<\/pre>\n
# Use Intel MPI\r\nstarccm+ -batch -pio -mpi intel -bs slurm myinput.sim<\/em>\r\n<\/pre>\n
Force a Checkpoint and optionally Abort<\/h2>\n
.sim<\/code> file with @iteration<\/em><\/code> in the name to indicate the iteration number at which the checkpoint file was made. For example myinput@25000.sim<\/code>. You can then use this as the input .sim<\/code> file for another job on the CSF.<\/p>\nManually<\/h3>\n
touch CHECKPOINT\r\n<\/pre>\n
CHECKPOINT<\/code> file to be CHECKPOINT~<\/code> (so that it doesn’t keep checkpointing) then carry on with the simulation. You can run touch CHECKPOINT<\/code> again at some time in the future to generate a new checkpoint file.<\/p>\ntouch ABORT\r\n<\/pre>\n
Automatically in your jobscript<\/h3>\n
myinput.sim<\/code>. When the job re-queues after a checkpoint, it will run myinput@nnnnn<\/em>.sim<\/code> where nnnnn<\/em><\/code> is the iteration number at which the checkpoint was written. The script will automatically use the most recent checkpoint file. Eventually, the simulation will converge and StarCCM will exit normally. When this happens no further checkpoints are made and the job will not re-queue.<\/p>\nSAVETIME<\/code> setting to be the duration you want StarCCM to run for before it checkpoints, aborts and requeues the job. Given that a checkpoint occurs at the end of an iteration, you need to allow enough time for the current iteration to finish and for the checkpoint file to be written. In the example below we run on the Intel nodes which have a maximum runtime of 7 days. Hence we set the checkpoint time to be at 6 days, 23 hours and 50 minutes. This give StarCCM 10 minutes to finish the current iteration and write the checkpoint file. Edit this setting as required<\/strong>.<\/p>\n#!\/bin\/bash --login\r\n#SBATCH -p multicore # (or --partition=) Run on the AMD 168-core nodes\r\n#SBATCH -n 16 # (or --ntasks=) Number of cores to use.\r\n#SBATCH -t 4-0 # Wallclock time limit. 4-0 is 4 days. Max permitted is 7-0.\r\n\r\n# We now recommend loading the modulefile in the jobscript. Use your required version\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008-mixed\r\n\r\n# ------ Edit these settings (carefully) -------\r\n\r\n# How long the job should run for before we checkpoint\r\n# and abort. Check the PE max runtime on the CSF webpages.\r\n# dd:hh:mm:ss\r\n\r\nSAVETIME=06:23:50:00\r\n\r\n# Name of simulation file - do not<\/strong> add .sim to end\r\n# of name. EG: Will use myinput.sim on first run or latest\r\n# checkpoint file myinput@nnnnn<\/em>.sim if available.\r\n\r\nSIMFILE=myinput\r\n\r\n# --------------------------------------------\r\n# Check for licenses, requeue if not enough available.\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\n# Find newest checkpoint file \r\nSIM=$(ls -t ${SIMFILE}*.sim | head -1)\r\n\r\nif [ -n \"$SIM\" ]; then\r\n if [[ \"$SIM\" =~ @[0-9] ]]; then\r\n echo \"Using checkpoint file $SIM\"\r\n else \r\n echo \"Using original simulation file $SIM\"\r\n fi\r\nelse\r\n echo \"Failed to find any .sim files. Job will exit!\"\r\n exit 1\r\nfi\r\n\r\n# Add a timer to create checkpoint file (the &<\/strong> is important!)\r\n(sleep $(echo $SAVETIME | awk -F: '{printf \"%dd %dh %dm %ds\",$1,$2,$3,$4}'); echo \"ABORT...\"; touch ABORT) &<\/strong>\r\nSLEEP_PID=$!\r\n\r\n# Run starccm as usual for a 32-core job (see earlier in this webpage for more details)\r\nstarccm+ -batch -mpi openmpi -batchsystem slurm $SIM -np $SLURM_NTASKS\r\n\r\n# Tidy up the sleep timer\r\npkill -P $SLEEP_PID\r\n\r\n# Automatically resubmit the job if we reached the checkpoint time?\r\nif [ -f ABORT ]; then\r\n CHKFILE=$(ls -t ${SIMFILE}@*.sim | head -1)\r\n if [ -f \"$CHKFILE\" ]; then \r\n echo \"Job checkpointed at $(date) - wrote $CHKFILE - job will be requeued automatically\"\r\n STATUS=99\r\n else\r\n echo \"No checkpoint file found (this is probably an error!) Job will not be requeued.\"\r\n fi\r\nfi\r\nrm -f ABORT\r\nexit $STATUS \r\n<\/pre>\nClient \/ Server Usage<\/h2>\n
\n
starccm+<\/code> command-line as show in this example:\n#!\/bin\/bash --login\r\n#SBATCH -p multicore # (or --partition=) Run on the AMD 168-core nodes\r\n#SBATCH -n 16 # (or --ntasks=) Number of cores to use.\r\n#SBATCH -t 4-0 # Wallclock time limit. 4-0 is 4 days. Max permitted is 7-0.\r\n\r\n# We now recommend loading the modulefile in the jobscript. Use your required version\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008-mixed\r\n\r\n# Check for licenses, requeue if not enough available.\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\n# Run starccm, but in server<\/em> mode instead of batch<\/em> mode\r\nstarccm+ -server<\/strong> -load myinput.sim<\/em> -mpi openmpi -batchsystem slurm -np $SLURM_NTASKS\r\n # #\r\n # # Replace myinput.sim<\/em> with your own input file.\r\n # \r\n # Now we use -server instead of -batch (as used previously)\r\n\r\n# If a different user will be connecting to the server, i.e usernames differ between the\r\n# client and server. Add the \"-collab\" option, e.g.\r\n# starccm+ -server -collab<\/strong> -load myinput.sim<\/em> -mpi openmpi -batchsystem slurm -np $SLURM_NTASKS\r\n<\/pre>\n<\/li>\n
<\/pre>\n
sbatch myjobscript<\/em><\/code> as usual.<\/li>\nslurm-12345<\/em>.out<\/code> where 12345<\/em><\/code> will be unique to your job. Have a look in this file:\ncat slurm-12345<\/em>.out\r\n<\/pre>\n
Server::start -host node770<\/em><\/strong>.pri.csf3.alces.network:47827<\/em><\/strong>\r\n # #\r\n # # The port number may<\/em> be different\r\n # # but it is often 47827 (the default).\r\n #\r\n # The node name will likely be different.\r\n # Make a note of your<\/em> node name.\r\n<\/pre>\n<\/li>\n
ssh -L 47827<\/em><\/strong>:node770<\/em><\/strong>:47827<\/em><\/strong> mxyzabc1<\/em><\/strong>@csf3.itservices.manchester.ac.uk\r\n # # # #\r\n # # # # Use your own username here\r\n # # #\r\n # # # Use the port number reported earlier\r\n # #\r\n # # Use the node name reported earlier\r\n #\r\n # Use the port number earlier\r\n<\/pre>\n
\n
Host: localhost\r\nPort: 47827 (or whatever number you got from above).\r\n<\/pre>\n
starccm+ -host localhost:47827<\/strong> # Change the port number to match given above\r\n<\/pre>\n<\/li>\n
CTRL+R<\/code> in the main window).<\/li>\nCo-Simulation with Abaqus<\/h2>\n
evince $STARCCM_UG\r\n<\/pre>\n
Example 32-core co-simulation job<\/h3>\n
cd ~\/scratch\r\nmkdir co-sim # Change the name as requried\r\ncd co-sim\r\nmkdir abaqus starccm # Two directories to hold the input and output files from each app\r\n<\/pre>\n
cp ~\/abq_cosim.inp ~\/scratch\/co-sim\/abaqus\r\ncp ~\/ccm_cosim.sim ~\/scratch\/co-sim\/starccm\r\n<\/pre>\n
# Load the required version of starccm, for example:\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008\r\n\r\n# Start an interactive job to run the starccm GUI:\r\ncd ~\/scratch\/co-sim\/starccm\r\nsrun -p interactive -t 0-1 --pty starccm+ ccm_cosim.sim\r\n<\/pre>\n
Co-Simulations<\/code> node in the tree viewer, expand Link1<\/code> then look for the following attributes and set their values as follows:<\/p>\nCo-Simulation Type = Abaqus Co-Simulation # Choose from the drop-down menu\r\n...\r\n### Note there are several simulation settings (e.g., ramping parameters) that control\r\n### the simulation. You will need to set these but they are beyond the scope of this\r\n### web-page. Please refer to the starccm user guide.\r\n...\r\nAbaqus Execution\r\n Current Job Name = my-abq-cosim<\/em> # Any name, used to name abaqus output files\r\n Input file = ..\/abaqus\/abq_cosim.inp # See directory structure created above\r\n Executable name = abq2018 # The Abaqus command used on the CSF (choose version)\r\n Number of CPUs = 8 # This MUST match the jobscript (see below)\r\n Remote shell = ssh # The default - leave set to this\r\n<\/pre>\n
cd ~\/scratch\/co-sim\/starccm\r\ngedit cosim.sbatch\r\n<\/pre>\n
#!\/bin\/bash --login\r\n#SBATCH -p multicore # (or --partition=) Run on the AMD 168-core nodes\r\n#SBATCH -n 32 # (or --ntasks=) Number of cores to use.\r\n#SBATCH -t 4-0 # Wallclock time limit. 4-0 is 4 days. Max permitted is 7-0.\r\n\r\n## Load the modulefiles in the jobscript so we always know which version we used\r\nmodule purge\r\nmodule load apps\/binapps\/starccm\/17.04.008\r\nmodule load apps\/binapps\/abaqus\/2018\r\n\r\n## We will use 24 (out of 32) cores for starccm and 8 (out of 32) cores for abaqus.\r\n## Manually set the special SLURM_NTASKS variable to these numbers so the license check\r\n## scripts test for the correct number of licenses. If there are not enough licenses\r\n## then the job will requeue.\r\nexport SLURM_NTASKS=8\r\n. $ABAQUS_HOME\/liccheck.sh\r\n\r\n## NOTE: We will leave SLURM_NTASKS set to the number needed for starccm after this license check\r\nexport SLURM_NTASKS=24\r\n. $STARCCM_HOME\/liccheck.sh\r\n\r\n## Now run starccm with 24 cores. It runs the 'abq2018' command with 8 cores (set in input file)\r\nstarccm+ -batch ccm_cosim.sim -mpi openmpi -batchsystem slurm -np $SLURM_NTASKS\r\n<\/pre>\n
sbatch cosim.sbatch<\/pre>\n
~\/scratch\/co-sim\/abaqus<\/code> and ~\/scratch\/co-sim\/starccm<\/code> directories. For example, to see what is happening in each simulation:<\/p>\ncd ~\/scratch\/co-sim\/abaqus\r\ntail -f my-abq-cosim<\/em>.msg # The abaqus output file name was set in the starccm input file above\r\n #\r\n # Press CTRL+C to exit out of the 'tail' command\r\n\r\ncd ~\/scratch\/co-sim\/starccm\r\ntail -f slurm-123456<\/em>.out # Replace 123456<\/em> with the job id number of your starccm job\r\n #\r\n # Press CTRL+C to exit out of the 'tail' command\r\n<\/pre>\n
Further Information<\/h2>\n
evince $STARCCM_UG\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"