Linux (Single Node)

Software Prerequisites

  • CUDA 10.1+ (M-Star is always forward compatible with new drivers. We recommend installing the latest driver available from NVidia.)

  • Open MPI 3.1.3 – 4.1.5

  • Supported Linux x64 platform, including any of the following:

    • Redhat 7+ (or binary-compatible distribution such as Centos, Oracle, Alma, etc.)

    • Ubuntu 18.04+ (or binary-compatible distribution such as Debian, Mint, etc.)

    • Debian 10+

  • Graphical user interface

    • Desktop environment

    • Physical GPU supporting OpenGL 3.2+ (Alternatively, use software rendering on Linux.)

Hardware Prerequisites

  • X86 64-bit computer

  • NVLink (A wire-based communications link which allows several GPUs to interact directly through a high-speed interconnection.)

  • NVSwitch (if using three or more GPUs)

  • Supported GPU with compute capability 3.5+

Accessing the Software

Access the M-Star CFD Download Site. Note that only current licensed users are permitted access.

Installation

Install latest CUDA driver and package (minimum 10.1+)

Download the latest Nvidia CUDA package and install as root. The CUDA package typically comes with a driver update, which you will want to install. The CUDA SDK is required to build Open MPI with CUDA support. Refer to the CUDA installations provided by NVidia for further guidance. The CUDA version used to build Open MPI with CUDA support must match the version used at runtime.

Note

Using a CUDA version of 11+ will cause a short startup delay of the solver. This can be eliminated by installing the M-Star CFD solver package specific to your CUDA version.

Verify the Nvidia driver is loaded by running nvidia-smi. This should display the CUDA version and GPU name.

Install OpenMPI

Choose one of the options below.

Option 1: From Operating system package

Check your system package manager for an openmpi package. It is possible this component can be provided by your operating system package manager. For example on Ubuntu 20+, run sudo apt install openmpi-bin. Note that this option will not have CUDA-aware support. If you have more than 1 GPU, you should consider compiling OpenMPI from source to ensure CUDA aware support.

Option 2: From source

If your operating system package repository does not contain OpenMPI package compatible with M-Star CFD, you will need to compile OpenMPI per the instructions below.

Prerequisites:

  • GCC

  • Build environment tools

# Installs Open MPI to /opt/openmpi/4.1.5

cd $HOME
wget https://download.open-mpi.org/release/open-mpi/v4.1/openmpi-4.1.5.tar.gz
tar xzf openmpi-4.1.5.tar.gz
cd openmpi-4.1.5
mkdir build
cd build

../configure --prefix=/opt/openmpi/4.1.5 --with-cuda
make -j 8
make install

  1. Create an environment file for Open MPI at /opt/openmpi/4.1.5/env.sh.

#### Contents of /opt/openmpi/4.1.5/env.sh #######
installPath=/opt/openmpi/4.1.5
export PATH=$installPath/bin:$PATH
export LD_LIBRARY_PATH=$installPath/lib:$installPath/lib64:$LD_LIBRARY_PATH
###########################################

Important

If you have multiple versions of the CUDA toolkit installed, you must compile Open MPI against the CUDA version used at runtime. Use the --with-cuda=/usr/local/cuda-XX argument to choose a specific CUDA installation. Run nvidia-smi to see that your NVidia driver version matches the toolkit version.

Install M-Star CFD

  1. Go to M-Star CFD Downloads.

  2. Pick a package compatible with your operating system. Note that packages prefixed with mstarcfd-solver only contain the solver component. This package is usually relevant for single node Linux installations.

  3. Download and expand the M-Star CFD package.

mkdir -p /opt/mstar/3.9.54
cd /opt/mstar/3.9.54
wget -qO- ##COPY-PASTE LINK## | tar xzf

  1. Load the environment.

# Load the included environment file with M-Star
source /opt/mstar/3.9.65/mstar.sh

# if you compiled OpenMPI from source, load your environment file
source /opt/openmpi/4.1.5/env.sh

  1. Verify the M-Star Solver installation.

mstar-cfd-mgpu --version

  1. Verify the M-Star Pre-Processor installation. After running this command, the GUI should show up.

mstar

Note

If M-Star Pre-Processor does not start, run the Platform diagnositics script and send the output to support@mstarcfd.com . Also refer to Troubleshooting M-Star Pre/Post start up if having issues with program start up.

License Setup

Refer to the license setup guide at Workstation License.

Run Examples

Note the different executables used in each circumstance:

# Show standard help
mstar-cfd --help

# Single GPU
mstar-cfd-mgpu -i input.xml -o out --gpu-auto

# Multi GPU
mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto

# Single CPU run
mstar-cfd -i input.xml -o out

# Multi CPU run on 4 processes
mpirun -np 4 mstar-cfd -i input.xml -o out

# Note that you may specify GPU devices manually. Refer to the mstar-cfd-mgpu --help output for further guidance.

Multi-GPU Setup

The setup required to work on one GPU automatically works with multiple GPUs. However, if you employ three or more GPUs, you must also use an NVSwitch.


linux multi gpu 1

If using only two GPUs, an NVLink is a sufficient connection.


linux multi gpu 3

When using three or more GPUs, you need an NVLink and an NVSwitch. The monitor can be connected to any GPU on your network.

For more information on GPU peering, see: Understanding GPU peer access

OpenGL Compatibility on Linux

M-Star requires OpenGL v3.3+ compatibility.

To check for OpenGL compatibility, run (may require an operating system package such as mesa-utils):

glxinfo |grep OpenGL

You should see some output that lists the physical GPU that is currently driving the display. The below is an example of glxinfo output showing good OpenGL support.

OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: NVIDIA GeForce RTX 3090/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 530.30.02
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6.0 NVIDIA 530.30.02
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)
OpenGL extensions:

The output below is an example of glxinfo output that shows that we do not have OpenGL support. If you see terms like VMware, Mesa, llvmpipe, this indicates you are using software rendering driven by the CPU. The OpenGL version string also shows us we only have OpenGL v3.1 suport, which is not good enough for M-Star.

OpenGL vendor string: VMware, Inc.
OpenGL renderer string: llvmpipe (LLVM 10.0.0, 256 bits)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 20.0.8
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.1 Mesa 20.0.8
OpenGL shading language version string: 1.40
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 20.0.8
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
OpenGL ES profile extensions:

Related - Troubleshooting M-Star Pre/Post start up

Troubleshooting

Follow the instructions Linux and send your diagnostics output to support@mstarcfd.com .

Troubleshooting Pre and Post startup

See Troubleshooting M-Star Pre/Post start up for additional information