Installation

Windows

Workflow

Run the msi file downloaded from the M-Star CFD Download Portal to install the software. This installation file contains:

  • M-Star Build

  • M-Star Solver

  • M-Star Post Scripts

ParaView (Optional)

M-Star recommends users post-process data generated by M-Star Solve using ParaView. ParaView is a free, open-source post-processing software developed by Kitware, Inc and Los Alamos National Laboratory. M-Star provides scripts, training and support on using ParaView.

Visit the ParaView Download portal to download the latest Windows version of the ParaView software.

The data generated by M-Star Solve takes the form of ASCII text files (for numerical data output) and binary VTI files (for graphics and visualization). ASCII and VTI files are common formats supported by most commercial and open-source scientific visualization packages. Although ParaView is recommended, it is not required to post-processes the output data.

Linux

System Requirements

  • GCC

  • Build environment tools

  • Bash terminal

  • CUDA 10.1

  • Desktop environment (Only needed for M-Star Build graphical Interface functionality)

  • Supported Linux x64 platform:

    • Centos 6 or 7

    • Ubuntu 18.04 (command line solver only)

    • Other Linux platforms may be able to support solver-only usage with the provided Centos packages

Installation

Download and extract mstarcfd tarball from download.mstarcfd.com. For the purposes of this documentation, it is assumed the tarball was extracted in the user’s home directory such that a directory ~/mstarcfd was created.:

# for example purposes only, adjust for your installation needs
cd $HOME
wget ##COPY-PASTE LINK##
tar xzf mstarcfd-centos7-2.0.X.tar.gz

Install CUDA 10.1

Only required if you want to do GPU runs. If you are only using CPUs, this is not necessary. Download CUDA 10.1 for your Linux OS, then install as root The CUDA package typically comes with a driver update, which you will want to install. The CUDA SDK is required to build OpenMPI with CUDA support. Refer to the CUDA installations provided by Nvidia for further guidence.

Install OpenMPI 3.1.3

This step is only necessary on new machines or when M-Star upgrades versions of OpenMPI. If your system already has OpenMPI 3.1.3 available, this step may be optional. If your system has 3.1.3 installed and you need to do multiple GPU runs, check with your system admin to ensure OpenMPI was compiled with the “–with-cuda” argument and the version of CUDA is 10.1.

# Download OpenMPI source
cd $HOME
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build

# Configuration:
# --prefix          argument to install location
# --with-cuda               argument to build openmpi with cuda support. This is required for multi GPU support.
# if specializations are required for your hardware, consult with your system admin for MPI configuration
../openmpi-3.1.3/configure --prefix=$HOME/openmpi-3.1.3 --with-cuda

# Compile with 4 processes
make -j 4

# Install
make install

Create environment file for custom OpenMPI build

Note that some users may need to use the lib64 library location:

#### Contents of ~/openmpi-3.1.3.sh #######
export PATH=$HOME/openmpi-3.1.3/bin:$PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3/lib64:$LD_LIBRARY_PATH
###########################################

Load Environment:

source ~/mstarcfd/mstar.sh
source ~/openmpi-3.1.3.sh

License Setup

Refer to the license setup guide on our documentation site

Run Examples

Note the different executables used in each circumstance:

# Show standard help
mstar-cfd --help

# single CPU run
mstar-cfd -i input.xml -o out

# multi CPU run on 4 processes
mpirun -np 4 mstar-cfd -i input.xml -o out

# single GPU
mstar-cfd-gpu -i input.xml -o out --gpu-auto

# multi GPU
mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto

*Note that you may specify GPU devices manually. Refer to the “mstar-cfd-gpu –help” output for further guidence. *

ParaView (Optional)

M-Star recommends users post-process data generated by M-Star Solve using ParaView. ParaView is a free, open-source post-processing software developed by Kitware, Inc and Los Alamos National Laboratory. M-Star provides scripts, training and support on using ParaView.

Visit the ‘ParaView Download portal <https://www.paraview.org/download/>`_ to download the latest Windows version of the ParaView software.

The data generated by M-Star Solve takes the form of ASCII text files (for numerical data output) and binary VTI files (for graphics and visualization). ASCII and VTI files are common formats supported by most commercial and open-source scientific visualization packages. Although ParaView is recommended, it is not required to post-processes the output data.

License Setup

M-Star Pre-Processor and Solver use RLM as the software licensing control mechanism. To determine your license type, please refer to your system admin or contract to determine your license type.

Short term license file issued by M-Star for testing and troubleshooting. Not locked to a specific machine.

A license locked to specific computer(s). Sharing of licenses is not permitted.

  1. Request a standard license file by sending your MAC address of the form 00:00:00:a1:2b:cc to M-Star support.

  2. Receive the license file and do one of the following to install it

  • Simply copy the license file to the application install folder. This will be c:\program files\M-Star on windows. On Linux, copy to the install folder , “bin” sub-directory.

  • Place the license file wherever you like, eg. c:\licenses\mstar.lic .

  • Set the environment variable mstar_LICENSE=c:\licenses\mstar.lic .

Important

Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration

To connect over the network to the license server, set the environment variable with the format, mstar_LICENSE=PORT@HOST . Refer to your operating system documentation for setting environment variables or consult your administrator.

mstar_LICENSE

Variable name

PORT

Port number. 5053 by default. Configured in license server file.

HOST

Host name of license server

Tip

For linux hosts: Add one of these statements to your $HOME/.bashrc file:

# Set to a license file
export mstar_LICENSE=/share/licenses/mstar.lic

# Set to a directory containing the file
export mstar_LICENSE=/share/licenses

# Set to a license server (PORT@HOST)
export mstar_LICENSE=5053@license-server.company

Download the license server for your platform

A license is locked to a specific license server. The floating server controls the number of open instances of Pre-Processors and Solvers on the network. For instance if you have 2 pre-processor and 5 solver floating licenses, you would be able to have 2 open instances of the GUI and 5 active simulations running at the same time. The floating server enforces a maximum concurrency on the license.

Obtain MAC address and send to M-Star support

To initialize a license server, you will need find the host ID of the machine so a license file may be generated for you. Send the host ID (MAC address) to M-Star support to receive the license file used for the server.

Configure the license file with your RLM server

Once you receive the license file, you should configure the host name in the license file as follows. Open the license file in a text editor and change the localhost entry to the hostname or IP address of the license server. You may also change the license server port number at this time.:

HOST localhost bc5f54ac312a 5053
ISV mstar
.. more text below defining the license

Tip

Changing the hostname and port in a floating server license file will not invalidate the license or require M-Star to re-sign it.

Important

Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration

  1. Download and extract the license server executables into an installation directory.

  2. Copy the license file to the install directory

  3. Start the server by executing the rlm exe from the command line.

  4. To configure the license server as a Windows service, follow the next steps

The license server can be installed as a Windows service so that it will start up automatically at boot up time, and run in the background. Follow these steps to set that up.

Verify the user you want to run the service has the proper security settings. Refer to https://blogs.msdn.microsoft.com/ablock/2008/09/18/setting-the-properties-log-on-as-a-service-and-allow-log-on-locally/

  1. Make a directory at C:\logs to house log files for the license server.

  2. Start a command window with administrator priveliges (Right click cmd.exe, Run As Administrator)

  3. Change the directory to the RLM install location.

  4. Execute the following command. Change the c:\licenses input to point to the directory containing your floating server license file:

    rlm -install_service -service_name rlm-mstar -dlog c:\logs\rlmmstar.log -c c:\licenses

Important

Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration

  1. Download and extract the license server executables into an installation directory.

  2. Copy the license file to the install directory

  3. Start the server by executing the rlm exe from the command line

  4. To configure the license server as a startup process, refer to your operating system instructions.

Refer to your operating system documentation for the correct procedure for adding a program to the boot up sequence. You may be able to simply use crontab to accomplish this, with something like:

crontab -e

# Add the following line to your crontab with paths modified to your machine
@reboot cd ~/rlm-mstar; ./rlm > log.txt 2>&1

Using the RLM web server

Open a browser and point it to http://localhost:5054 , replacing localhost with the host name of your license server if connecting remotely. You will be able to manage and monitor the license server.

Host Firewall (Optional)

Some hosts will require the firewall open up the license server port. Refer to your operating system documentation or administrator to open the necessary ports:

  • 5053 (TCP) (UDP optional): Default license server port, RLM will contact license server on this port to obtain a license. Change this port by editting the license file as described in the below section. UDP may optionally be enabled to allow automatic RLM server discovery.

  • 5054 (TCP) : Default license server web management portal. If you wish to use the web management portal, you should verify this port is open.

Important

Some systems require that an RLM ISV port be specified. Under default operation, the ISV port is randomly chosen at execution time, which may cause the RLM ISV port to be blocked if it is unknown to the firewall.

To address this issue, the ISV port can be specified in the license file:

HOST localhost xxxxxxxxxxx 5053
ISV mstar port=6000
LICENSE ......

Here the ISV port is specified as 6000. The server firewall would then be updated to allow traffic on this port.

For more information, see ` this article by RLM <http://www.reprisesoftware.com/blog/2012/10/using-rlm-across-a-firewall/>`__

Change license server port number (Optional)

This is an optional task in case you need to change the port the license server uses. Edit your license file, the first few 2 lines will be something like:

HOST user-pc bc5f54ac312a 5053
ISV mstar
.. more text below defining the license

  1. Change the port number 5053 to the port you want to use

  2. Save the file

  3. Restart RLM or have it reread licenses

Further Information Please refer to the RLM License Administration Manual, which provides in depth explanation of the license server - RLM License Administration PDF

Tip

Check that your license server firewall has the configured port open. Some systems also require that the HOST name be set to the IP address or actual host name of the server.

License Server Troubleshooting

Important

Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration

  • After starting the license server. Review the RLM server output and/or logs. Look for any messages that might indicate the license is invalid, server unable to start, etc.

  • Verify port on license server is open on the server’s firewall

  • Verify port on end user machines is open

  • Be aware of any other firewalls in between the license server and clients to allow RLM traffic through

  • Refer to the RLM Administrator Documentation

AWS (Windows)

This guide will setup a Windows based AWS instance that can be used for Pre-Processing, Solving and Post-Post Processing.

Start Instance Config

  1. To setup Amazon EC2, follow the instructions here <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-a-base-security-group>`__

  2. Login to AWS Management Console

  3. Open the Services -> EC2 link to display the dashboard

  4. Click Launch Instance

Step 1: Choose AMI

Select Microsoft Windows Server 2016 Base

Step 2: Choose an Instance Type

Scroll down to the Compute optimized instance types. Select the appropriate instance for your simulation. 16, 36, 72 core Compute Optimized instances are well suited for M-Star

Step 3: Configure Instance Details

If you will do post-processing on this instance, you should select Add GPU. Otherwise, click next.

Step 4: Add Storage

Increase the size of the Root volume. 500GB+.

Step 5: Add Tags

nothing to do here

Step 6: Configure Security Group

Create a rule with the following parameters:

  • Type: RDP

  • Protocol: TCP

  • Port Range: 3389

  • Source: My IP

Important

Only use My IP as the Source for RDP connections

Step 7: Review Instance Launch

Double check your instance configuration.

Click Launch.

On the dialog box that pops up: * Select Create a new key pair * Name the key pair * Download Key pair (save it to your computer)

This key pair will be needed to connect to the instance

Click Launch Instances

Connecting to your instance

After about 5 minutes your instance should be ready.

  • Browse to the EC2 dashboard

  • Click Instances to show list of active instances

  • Right click on table row of instance to display Connect to your instance dialog box

  • Click Get Password

  • Browse to the key pair file (.pem) you downloaded

  • Password should be decrypted and displayed in the dialog box

  • Copy the login information (DNS, User Name, Password)

On your local PC:

  • Start up Remote Desktop Connection

  • Enter the DNS to connect

  • Accept the security window that may pop up (related to unknown certificates)

  • Enter user and password dialog

  • Connect to the machine

Setting up the instance

You should now be logged into the AWS instance via RDP as the Administrator user. You should now install the M-Star software as described in this manual.

Note

To enable web browsing under windows server, you will need to open the Local Server configuration, and toggle the internet explorer setting

Azure (Multi-GPU)

This guide will walk through how to setup a single Azure instance with multiple GPUs. This guide is meant for testing purposes only.

Before starting, it is recommended to have the following items ready:

  • Azure account

  • Azure subscription and quota capable of starting GPU instances such as NC12 or better

  • SSH public key

  • SSH/SCP programs on local computer (eg. Putty/Winscp on windows)

  • M-Star license file

The basic tasks that need to be completed:

  • Start up a VM with GPU hardware

  • Install CUDA

  • Install OpenMPI

  • Install M-Star CFD

  • Run test case

Additional resources -

Create a virtual machine

Go to the azure portal to start the creation of a new virtual machine and fill out the form as follows. Some options may be different for you.

  • Subscription: Pay-As-You-Go (your choices may be different)

  • Resource group: Testing (your choices may be different)

  • Virtual machine name: testing

  • Image: Ubuntu 18.04

  • Size: Standard NC12 Select a multi-GPU node from the series NC, NCv2, or NCv3

  • Authentication Type: SSH public key

  • Username: test

  • SSH public key: (Paste your SSH public key here)

  • Public inbound ports: Allow selected ports

  • Select inbound ports: SSH

Click Review and create.

Click Create.

SSH to the new virtual machine.

CUDA

Install CUDA:

# Download CUDA 10.1 from here - https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804&target_type=debnetwork
# Choose Linux - x86_64 - Ubuntu - 18.04 - deb (network)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.1.105-1_amd64.deb

# Next few commands require root
sudo dpkg -i cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda

OpenMPI

Download and install openmpi:

cd $HOME
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build
../openmpi-3.1.3/configure --prefix=$HOME/openmpi-3.1.3-install --with-cuda
make -j 10 && make install

The following commands may be put into a ~/openmpi.sh file that you can source as needed:

export PATH=$HOME/openmpi-3.1.3-install/bin:$PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3-install/lib:$LD_LIBRARY_PATH

M-Star CFD

  1. Download the Centos 7 package and extract the files (Yes the Centos7 package should work okay on Ubuntu so long as you do not run any GUI programs).

  2. Upload your license file to a new file ~/mstarcfd/bin/license.lic

  3. Setup your environment:

    source ~/openmpi.sh
source ~/mstarcfd/mstar.sh

  4. Upload a test case to ~/test and run it on multiple GPU:

    mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto

AWS (Multi-GPU)

Important

This guide is meant for demonstration purposes only.

This guide will walk through how to setup a single AWS instance with multiple GPUs.

Before starting, it is recommended to have the following items ready:

  • AWS account

  • AWS subscription and quota capable of starting GPU instances such as p2.xlarge or better

  • SSH public key

  • SSH/SCP programs on local computer (eg. Putty/Winscp on windows)

  • M-Star license file (you may need to first obtain the MAC address of the instance before a license file will be issueed)

After finishing these steps, you will have a AWS GPU instance fully setup. The installation of the software will be done to the /usr/local folder for a system-wide installation.

Create a virtual machine

  • Go to the EC2 Dashboard

  • Launch Instance

  • Choose “Ubuntu Server 18.04 LTS (HVM), SSD Volume Type”. Click Select

  • Choose the Instance type: p2 or p3 instances are recommended. Click Next.

  • Configure Instance Details. (Setup as needed). Click Next.

  • Configure Add Storage

    • Setup Root device with 20GB

    • Click “Add New Volume”. Setup a data disk with the capacity you need for simulations. 100GB+

    • Click Next

  • Configure Tags. Setup as needed. Click Next.

  • Configure Security Group

    • Create or select a security group that has the following rule. This allows SSH traffic to the instance

      • Type: SSH

      • Source: Custom, 0.0.0.0/0

    • Click Review and Launch

  • Launch the instance

  • Proceed back to the instance dashboard

  • Wait for instance to come up

Install Software

  • SSH to the instance using the “ubuntu” username

  • Copy and paste the script shown below into a new file and execute it once. This script does the following:

    • Sets up the data disk you defined under the /data directory

    • Install CUDA

    • Install OpenMPI to /usr/local

  • Install M-Star CFD using the standard Linux instructions

Warning

  • This instance is configured open to the internet

  • Be aware of how storage volume persist after virtual machine termination

Ubuntu AWS Setup Script:

#!/bin/bash

function add_disk () {

partname=$1
partname+="1"

lsblk | grep -q $partname

if [ $? -eq 0 ]; then

echo "Existing partition found at $1, skipping fdisk, mkfs"

else

sudo fdisk $1 <<EOF
n
p
1


w
EOF

sudo mkfs.ext4 $partname
echo "$partname      $2   ext4    defaults        0 0" |sudo tee -a /etc/fstab
sudo mkdir -p $2
sudo mount -a
sudo chmod 777 $2

fi

}

function install_cuda () {
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install -y cuda vim nano git build-essential
}

function install_openmpi () {
sudo apt-get update
sudo apt-get install -y build-essential
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build
../openmpi-3.1.3/configure --prefix=$INSTALL_PATH --with-cuda
make -j 4 && sudo make install
cd ..
sudo ldconfig
}

add_disk /dev/xvdb /data
install_cuda
install_openmpi

# Now Install M-Star CFD using the standard Linux instructions