Installation¶
Windows¶
Workflow¶
Run the msi file downloaded from the M-Star CFD Download Portal to install the software. This installation file contains:
M-Star Build
M-Star Solver
M-Star Post Scripts
ParaView (Optional)¶
M-Star recommends users post-process data generated by M-Star Solve using ParaView. ParaView is a free, open-source post-processing software developed by Kitware, Inc and Los Alamos National Laboratory. M-Star provides scripts, training and support on using ParaView.
Visit the ParaView Download portal to download the latest Windows version of the ParaView software.
The data generated by M-Star Solve takes the form of ASCII text files (for numerical data output) and binary VTI files (for graphics and visualization). ASCII and VTI files are common formats supported by most commercial and open-source scientific visualization packages. Although ParaView is recommended, it is not required to post-processes the output data.
Linux¶
System Requirements¶
GCC
Build environment tools
Bash terminal
CUDA 10.1
Desktop environment (Only needed for M-Star Build graphical Interface functionality)
Supported Linux x64 platform:
Centos 6 or 7
Ubuntu 18.04 (command line solver only)
Other Linux platforms may be able to support solver-only usage with the provided Centos packages
Installation¶
Download and extract mstarcfd tarball from download.mstarcfd.com. For the purposes of this documentation, it is assumed the tarball was extracted in the user’s home directory such that a directory ~/mstarcfd was created.:
# for example purposes only, adjust for your installation needs
cd $HOME
wget ##COPY-PASTE LINK##
tar xzf mstarcfd-centos7-2.0.X.tar.gz
Install CUDA 10.1
Only required if you want to do GPU runs. If you are only using CPUs, this is not necessary. Download CUDA 10.1 for your Linux OS, then install as root The CUDA package typically comes with a driver update, which you will want to install. The CUDA SDK is required to build OpenMPI with CUDA support. Refer to the CUDA installations provided by Nvidia for further guidence.
Install OpenMPI 3.1.3
This step is only necessary on new machines or when M-Star upgrades versions of OpenMPI. If your system already has OpenMPI 3.1.3 available, this step may be optional. If your system has 3.1.3 installed and you need to do multiple GPU runs, check with your system admin to ensure OpenMPI was compiled with the “–with-cuda” argument and the version of CUDA is 10.1.
# Download OpenMPI source
cd $HOME
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build
# Configuration:
# --prefix argument to install location
# --with-cuda argument to build openmpi with cuda support. This is required for multi GPU support.
# if specializations are required for your hardware, consult with your system admin for MPI configuration
../openmpi-3.1.3/configure --prefix=$HOME/openmpi-3.1.3 --with-cuda
# Compile with 4 processes
make -j 4
# Install
make install
Create environment file for custom OpenMPI build
Note that some users may need to use the lib64 library location:
#### Contents of ~/openmpi-3.1.3.sh #######
export PATH=$HOME/openmpi-3.1.3/bin:$PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3/lib64:$LD_LIBRARY_PATH
###########################################
Load Environment:
source ~/mstarcfd/mstar.sh
source ~/openmpi-3.1.3.sh
License Setup
Refer to the license setup guide on our documentation site
Run Examples
Note the different executables used in each circumstance:
# Show standard help
mstar-cfd --help
# single CPU run
mstar-cfd -i input.xml -o out
# multi CPU run on 4 processes
mpirun -np 4 mstar-cfd -i input.xml -o out
# single GPU
mstar-cfd-gpu -i input.xml -o out --gpu-auto
# multi GPU
mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto
*Note that you may specify GPU devices manually. Refer to the “mstar-cfd-gpu –help” output for further guidence. *
ParaView (Optional)¶
M-Star recommends users post-process data generated by M-Star Solve using ParaView. ParaView is a free, open-source post-processing software developed by Kitware, Inc and Los Alamos National Laboratory. M-Star provides scripts, training and support on using ParaView.
Visit the ‘ParaView Download portal <https://www.paraview.org/download/>`_ to download the latest Windows version of the ParaView software.
The data generated by M-Star Solve takes the form of ASCII text files (for numerical data output) and binary VTI files (for graphics and visualization). ASCII and VTI files are common formats supported by most commercial and open-source scientific visualization packages. Although ParaView is recommended, it is not required to post-processes the output data.
License Setup¶
M-Star Pre-Processor and Solver use RLM as the software licensing control mechanism. To determine your license type, please refer to your system admin or contract to determine your license type.
Short term license file issued by M-Star for testing and troubleshooting. Not locked to a specific machine.
A license locked to specific computer(s). Sharing of licenses is not permitted.
Request a standard license file by sending your MAC address of the form
00:00:00:a1:2b:cc
to M-Star support.Receive the license file and do one of the following to install it
Simply copy the license file to the application install folder. This will be
c:\program files\M-Star
on windows. On Linux, copy to the install folder , “bin” sub-directory.
Place the license file wherever you like, eg.
c:\licenses\mstar.lic
.Set the environment variable
mstar_LICENSE=c:\licenses\mstar.lic
.
Important
Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration
To connect over the network to the license server, set the environment variable with the format, mstar_LICENSE=PORT@HOST
. Refer to your operating system documentation for setting environment variables or consult your administrator.
mstar_LICENSE
Variable name
PORT
Port number. 5053 by default. Configured in license server file.
HOST
Host name of license server
Tip
For linux hosts: Add one of these statements to your $HOME/.bashrc file:
# Set to a license file
export mstar_LICENSE=/share/licenses/mstar.lic
# Set to a directory containing the file
export mstar_LICENSE=/share/licenses
# Set to a license server (PORT@HOST)
export mstar_LICENSE=5053@license-server.company
Download the license server for your platform
Windows
sha1sum: 0195ec246e8721e01dd1b7d4902d21ad21702dbb
Linux 64bit
sha1sum: 07b7cf041971157c0342945f1a8dfbba7063fb9f
A license is locked to a specific license server. The floating server controls the number of open instances of Pre-Processors and Solvers on the network. For instance if you have 2 pre-processor and 5 solver floating licenses, you would be able to have 2 open instances of the GUI and 5 active simulations running at the same time. The floating server enforces a maximum concurrency on the license.
Obtain MAC address and send to M-Star support
To initialize a license server, you will need find the host ID of the machine so a license file may be generated for you. Send the host ID (MAC address) to M-Star support to receive the license file used for the server.
Configure the license file with your RLM server
Once you receive the license file, you should configure the host name in the license file as follows. Open the license file in a text editor and change the localhost entry to the hostname or IP address of the license server. You may also change the license server port number at this time.:
HOST localhost bc5f54ac312a 5053
ISV mstar
.. more text below defining the license
Tip
Changing the hostname and port in a floating server license file will not invalidate the license or require M-Star to re-sign it.
Important
Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration
Download and extract the license server executables into an installation directory.
Copy the license file to the install directory
Start the server by executing the
rlm
exe from the command line.To configure the license server as a Windows service, follow the next steps
The license server can be installed as a Windows service so that it will start up automatically at boot up time, and run in the background. Follow these steps to set that up.
Verify the user you want to run the service has the proper security settings. Refer to https://blogs.msdn.microsoft.com/ablock/2008/09/18/setting-the-properties-log-on-as-a-service-and-allow-log-on-locally/
Make a directory at
C:\logs
to house log files for the license server.Start a command window with administrator priveliges (Right click cmd.exe, Run As Administrator)
Change the directory to the RLM install location.
Execute the following command. Change the
c:\licenses
input to point to the directory containing your floating server license file:rlm -install_service -service_name rlm-mstar -dlog c:\logs\rlmmstar.log -c c:\licenses
Important
Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration
Download and extract the license server executables into an installation directory.
Copy the license file to the install directory
Start the server by executing the
rlm
exe from the command lineTo configure the license server as a startup process, refer to your operating system instructions.
Refer to your operating system documentation for the correct procedure for adding a program to the boot up sequence. You may be able to simply use crontab to accomplish this, with something like:
crontab -e
# Add the following line to your crontab with paths modified to your machine
@reboot cd ~/rlm-mstar; ./rlm > log.txt 2>&1
Using the RLM web server¶
Open a browser and point it to http://localhost:5054 , replacing localhost with the host name of your license server if connecting remotely. You will be able to manage and monitor the license server.
Host Firewall (Optional)¶
Some hosts will require the firewall open up the license server port. Refer to your operating system documentation or administrator to open the necessary ports:
5053 (TCP) (UDP optional): Default license server port, RLM will contact license server on this port to obtain a license. Change this port by editting the license file as described in the below section. UDP may optionally be enabled to allow automatic RLM server discovery.
5054 (TCP) : Default license server web management portal. If you wish to use the web management portal, you should verify this port is open.
Important
Some systems require that an RLM ISV port be specified. Under default operation, the ISV port is randomly chosen at execution time, which may cause the RLM ISV port to be blocked if it is unknown to the firewall.
To address this issue, the ISV port can be specified in the license file:
HOST localhost xxxxxxxxxxx 5053
ISV mstar port=6000
LICENSE ......
Here the ISV port is specified as 6000. The server firewall would then be updated to allow traffic on this port.
For more information, see ` this article by RLM <http://www.reprisesoftware.com/blog/2012/10/using-rlm-across-a-firewall/>`__
Change license server port number (Optional)¶
This is an optional task in case you need to change the port the license server uses. Edit your license file, the first few 2 lines will be something like:
HOST user-pc bc5f54ac312a 5053
ISV mstar
.. more text below defining the license
Change the port number 5053 to the port you want to use
Save the file
Restart RLM or have it reread licenses
Further Information Please refer to the RLM License Administration Manual, which provides in depth explanation of the license server - RLM License Administration PDF
Tip
Check that your license server firewall has the configured port open. Some systems also require that the HOST name be set to the IP address or actual host name of the server.
License Server Troubleshooting¶
Important
Please refer to the RLM’s support and documentation website for additional detail and troubleshooting tips. RLM License Administration
After starting the license server. Review the RLM server output and/or logs. Look for any messages that might indicate the license is invalid, server unable to start, etc.
Verify port on license server is open on the server’s firewall
Verify port on end user machines is open
Be aware of any other firewalls in between the license server and clients to allow RLM traffic through
Refer to the RLM Administrator Documentation
AWS (Windows)¶
This guide will setup a Windows based AWS instance that can be used for Pre-Processing, Solving and Post-Post Processing.
Start Instance Config
To setup Amazon EC2, follow the instructions here <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html#create-a-base-security-group>`__
Login to AWS Management Console
Open the Services -> EC2 link to display the dashboard
Click Launch Instance
Step 1: Choose AMI
Select Microsoft Windows Server 2016 Base
Step 2: Choose an Instance Type
Scroll down to the Compute optimized instance types. Select the appropriate instance for your simulation. 16, 36, 72 core Compute Optimized instances are well suited for M-Star
Step 3: Configure Instance Details
If you will do post-processing on this instance, you should select Add GPU. Otherwise, click next.
Step 4: Add Storage
Increase the size of the Root volume. 500GB+.
Step 5: Add Tags
nothing to do here
Step 6: Configure Security Group
Create a rule with the following parameters:
Type: RDP
Protocol: TCP
Port Range: 3389
Source: My IP
Important
Only use My IP as the Source for RDP connections
Step 7: Review Instance Launch
Double check your instance configuration.
Click Launch.
On the dialog box that pops up: * Select Create a new key pair * Name the key pair * Download Key pair (save it to your computer)
This key pair will be needed to connect to the instance
Click Launch Instances
Connecting to your instance
After about 5 minutes your instance should be ready.
Browse to the EC2 dashboard
Click Instances to show list of active instances
Right click on table row of instance to display Connect to your instance dialog box
Click Get Password
Browse to the key pair file (.pem) you downloaded
Password should be decrypted and displayed in the dialog box
Copy the login information (DNS, User Name, Password)
On your local PC:
Start up Remote Desktop Connection
Enter the DNS to connect
Accept the security window that may pop up (related to unknown certificates)
Enter user and password dialog
Connect to the machine
Setting up the instance
You should now be logged into the AWS instance via RDP as the Administrator user. You should now install the M-Star software as described in this manual.
Note
To enable web browsing under windows server, you will need to open the Local Server configuration, and toggle the internet explorer setting
Azure (Multi-GPU)¶
This guide will walk through how to setup a single Azure instance with multiple GPUs. This guide is meant for testing purposes only.
Before starting, it is recommended to have the following items ready:
Azure account
Azure subscription and quota capable of starting GPU instances such as NC12 or better
SSH public key
SSH/SCP programs on local computer (eg. Putty/Winscp on windows)
M-Star license file
The basic tasks that need to be completed:
Start up a VM with GPU hardware
Install CUDA
Install OpenMPI
Install M-Star CFD
Run test case
Additional resources -
M-Star README file documentation
Create a virtual machine
Go to the azure portal to start the creation of a new virtual machine and fill out the form as follows. Some options may be different for you.
Subscription: Pay-As-You-Go (your choices may be different)
Resource group: Testing (your choices may be different)
Virtual machine name: testing
Image: Ubuntu 18.04
Size: Standard NC12 Select a multi-GPU node from the series NC, NCv2, or NCv3
Authentication Type: SSH public key
Username: test
SSH public key: (Paste your SSH public key here)
Public inbound ports: Allow selected ports
Select inbound ports: SSH
Click Review and create.
Click Create.
SSH to the new virtual machine.
CUDA
Install CUDA:
# Download CUDA 10.1 from here - https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804&target_type=debnetwork
# Choose Linux - x86_64 - Ubuntu - 18.04 - deb (network)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
# Next few commands require root
sudo dpkg -i cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda
OpenMPI
Download and install openmpi:
cd $HOME
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build
../openmpi-3.1.3/configure --prefix=$HOME/openmpi-3.1.3-install --with-cuda
make -j 10 && make install
The following commands may be put into a ~/openmpi.sh file that you can source as needed:
export PATH=$HOME/openmpi-3.1.3-install/bin:$PATH
export LD_LIBRARY_PATH=$HOME/openmpi-3.1.3-install/lib:$LD_LIBRARY_PATH
M-Star CFD
Download the Centos 7 package and extract the files (Yes the Centos7 package should work okay on Ubuntu so long as you do not run any GUI programs).
Upload your license file to a new file
~/mstarcfd/bin/license.lic
Setup your environment:
source ~/openmpi.sh source ~/mstarcfd/mstar.sh
Upload a test case to ~/test and run it on multiple GPU:
mpirun -np 2 mstar-cfd-mgpu -i input.xml -o out --gpu-auto
AWS (Multi-GPU)¶
Important
This guide is meant for demonstration purposes only.
This guide will walk through how to setup a single AWS instance with multiple GPUs.
Before starting, it is recommended to have the following items ready:
AWS account
AWS subscription and quota capable of starting GPU instances such as p2.xlarge or better
SSH public key
SSH/SCP programs on local computer (eg. Putty/Winscp on windows)
M-Star license file (you may need to first obtain the MAC address of the instance before a license file will be issueed)
After finishing these steps, you will have a AWS GPU instance fully setup. The installation of the software will be done to the /usr/local folder for a system-wide installation.
Create a virtual machine
Go to the EC2 Dashboard
Launch Instance
Choose “Ubuntu Server 18.04 LTS (HVM), SSD Volume Type”. Click Select
Choose the Instance type: p2 or p3 instances are recommended. Click Next.
Configure Instance Details. (Setup as needed). Click Next.
Configure Add Storage
Setup Root device with 20GB
Click “Add New Volume”. Setup a data disk with the capacity you need for simulations. 100GB+
Click Next
Configure Tags. Setup as needed. Click Next.
Configure Security Group
Create or select a security group that has the following rule. This allows SSH traffic to the instance
Type: SSH
Source: Custom, 0.0.0.0/0
Click Review and Launch
Launch the instance
Proceed back to the instance dashboard
Wait for instance to come up
Install Software
SSH to the instance using the “ubuntu” username
Copy and paste the script shown below into a new file and execute it once. This script does the following:
Sets up the data disk you defined under the /data directory
Install CUDA
Install OpenMPI to /usr/local
Install M-Star CFD using the standard Linux instructions
Warning
This instance is configured open to the internet
Be aware of how storage volume persist after virtual machine termination
Ubuntu AWS Setup Script:
#!/bin/bash
function add_disk () {
partname=$1
partname+="1"
lsblk | grep -q $partname
if [ $? -eq 0 ]; then
echo "Existing partition found at $1, skipping fdisk, mkfs"
else
sudo fdisk $1 <<EOF
n
p
1
w
EOF
sudo mkfs.ext4 $partname
echo "$partname $2 ext4 defaults 0 0" |sudo tee -a /etc/fstab
sudo mkdir -p $2
sudo mount -a
sudo chmod 777 $2
fi
}
function install_cuda () {
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1804_10.1.105-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install -y cuda vim nano git build-essential
}
function install_openmpi () {
sudo apt-get update
sudo apt-get install -y build-essential
wget https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.gz
tar xzf openmpi-3.1.3.tar.gz
mkdir openmpi-3.1.3-build
cd openmpi-3.1.3-build
../openmpi-3.1.3/configure --prefix=$INSTALL_PATH --with-cuda
make -j 4 && sudo make install
cd ..
sudo ldconfig
}
add_disk /dev/xvdb /data
install_cuda
install_openmpi
# Now Install M-Star CFD using the standard Linux instructions