Singularity¶

These instructions were made and tested under the following conditions:

Singularity 3.9.1
Ubuntu 18.04
CUDA Version 11.0
M-Star CFD 3.3.122
4x NVidia V100 GPUs (AWS p3.8xlarge)

Please refer to Singularity Documentation for help installing and running Singularity.

Use one of the options below for creating a Singularity image:

Option 1: Build Image from example¶

See our Singularity container example on Github.

Option 2: Build Image from custom spec file¶

This approach allows you to customize the installed version.

Create a file called Singularity with the following content. Change the M-Star CFD tar gz package download link to adjust versions.

File Content: Singularity

Bootstrap: docker

From: mstarcfd/nvidia-mpi:1

%post

mkdir -p /usr/local/mstarcfd
cd /usr/local/mstarcfd
curl -s https://cdn.mstarcfd.com/3.3.122/mstarcfd-solver-3.3.122-oracle7_cuda-11.3_openmpi-4.1.tar.gz | tar xvz
echo "/usr/local/mstarcfd/lib" >> /etc/ld.so.conf.d/hpccm.conf
ldconfig

%environment

export LD_LIBRARY_PATH=/usr/local/mstarcfd/lib:$LD_LIBRARY_PATH
export PATH=/usr/local/mstarcfd/bin:$PATH

%runscript

exec /usr/bin/nventry

Build the singularity image:

sudo singularity build ~/mstar.img Singularity

Make sure the mstar_LICENSE variable is configured to properly setup the M-Star CFD license:

export mstar_LICENSE=5053@mylicenseserver

Option 2: Build Image from NVidia NGC¶

Build from an existing Docker image on NVidia NGC. You can pull an existing Docker image into Singularity. M-Star is available on NVidia’s container catalog (NVidia NGC). Note that the versions available this way are generally limited.

sudo singularity pull docker://nvcr.io/hpc/m-star:3.2.31

For more information, see M-Star NVidia NGC Catalog Page.

Run Singularity Image¶

Navigate to a test case directory and run the container.

Important

Note the use of the --nv argument to pass through access to the GPUs.

singularity exec --nv ~/mstar.img mstar-cfd-mgpu -i input.xml -o out --gpu-auto

Run on multiple GPUs:

singularity exec --nv ~/mstar.img mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto