Running the Solver¶
Using the M-Star Solver on Windows¶
Once a model is configured, users can choose to:
Export Solver Files,
Export Solver Files
Create a package of files that can be executed using M-Star Solve. This package contains and
.stl files that describing the runtime parameters and simulation geometry, respectively.
Executes the simulation on the same machine used to build the model.
There are two basic modes of executing the M-Star CFD Solver, CPU and GPU mode. In CPU mode, only the CPU processers are used. In GPU mode, the specified GPU accelerators will be used to execute the solver.
Note that the number of cores allocated to the simulation should be anchored to the number of physical processors, not the number of hyperthreaded cores.
Requirements for Multi GPU usage on Windows:
The GPU hardware must have “peer-access”. This lets the GPUs communicate with each other. The Solver GUI will check this requirement before running.
The GPU hardware must be configured in TCC mode which means any GPU used in a solver calculation cannot be currently used for any display. This may require using the
nvidia-smiutility to configure a given GPU.
The GPU hardware must have the same processor architecture
Using the M-Star Solver on Linux¶
In this sample, we will set the license file and execute the solver over 4 processors:
# Set the license file export mstar_LICENSE=/home/user/mstar.lic # Initialize the M-Star environment source /home/user/mstarcfd/mstar.sh # Go to a case folder cd /home/user/case1 # Run the solver in CPU mode on 4 processors, putting all output (standard and error stream) in a log file mpirun -np 4 mstar-cfd -i input.xml -o out > log.txt 2>&1 # Run the solver in GPU mode using 8 GPUs. Note that the -np argument to mpirun should match the number of physical GPU devices # Note that we are using the mstar-cfd-mgpu executable which requires MPI to be compiled with CUDA support # The GPUs are selected automatically with the --gpu-auto parameter # Putting all output (standard and error stream) in a log file mpirun -np 8 mstar-cfd-mgpu --gpu-auto -i input.xml -o out > log.txt 2>&1 # Run the solver in GPU mode using 2 GPUs. Note that the -np argument to mpirun should match the number of physical GPU devices # Note that we are using the mstar-cfd-gpu executable which does NOT require MPI to be compiled with CUDA support # The GPUs are selected automatically with the --gpu-auto parameter # Putting all output (standard and error stream) in a log file mpirun -np 2 mstar-cfd-gpu --gpu-auto -i input.xml -o out > log.txt 2>&1
Example PBS script¶
Here is an example of a PBS job script that executes the solver over 2 nodes with 16 processors each and a max walltime of 24 hours.:
#!/bin/bash #PBS -l nodes=2:ppn=16 #PBS -l walltime=24:00:00 #PBS -N dmt-job export mstar_LICENSE=/home/user/dmt.lic source /home/user/mstar/mstar.sh cd $PBS_O_WORKDIR mpirun mstar-cfd -i input.xml -o out > log.txt 2>&1
To verify the Solver is installed correctly, you will want to verify a few things and run a simple simulation. Here is a check list to run through -
Solver is installed¶
Verify the dmt executable and mpirun are present in the target install folder under the bin sub-directory.
Solver license has been setup¶
Check that your environment contains the definition of the mstar_LICENSE variable, or that the license file is in the same directory as the dmt executable.
Obtain a simple test case for testing¶
The download page for each released version includes a
cases.tgz file that contains prepared simulation files ready to execute. You may also request a test case by emailing mailto:email@example.com , orby generating one yourself with the M-Star GUI. The examples can be generated from the GUI by going to File-New from Template.
Run the test case¶
Execute the test case
View results in Paraview¶
If Paraview is installed to your Linux machine, you may open Paraview and immediately view results. Otherwise, copy the out folder to your local machine and open Paraview there.
Restarting a Simulation¶
Within Windows environments, open the “M-Star Solve” from the Windows start menu. You will be prompted to browse to the case directory corresponding to the simulation you want to restart. Once loaded, the interface will populate a list of all checkpoint files associated with this case, as stored in the associated checkpoint directory. Select the checkpoint file from which you would like to restart and execute by clicking the Run button.
Within Linux environments, you will need to give the solver a command line instruction to restart from a checkpoint. For example, consider the default submission command:
mpirun -np 8 mstar-cfd -i input.xml -o out > log.txt 2>&1
To run from the most recent check-point file, modify the command as follows:
mpirun -np 8 mstar-cfd -i input.xml -o out -r out --load-last > log.txt 2>&1
Alternatively, if you want to load from a checkpoint different from the last available checkpoint, you can execute:
mpirun -np 8 mstar-cfd -i input.xml -o out -r out -l N > log.txt 2>&1
where N is the specific checkpoint file you wish to evoke.