System Requirements

Operating systems supported

• Windows 10

• Windows Server 2016+

• Centos 7

• Redhat 7

• Oracle 7

• Ubuntu 18 and derivatives (eg. Linux Mint)

• Ubuntu 20 and derivatives (eg. Linux Mint)

GPU Hardware Support

The M-Star Solver requires GPUs with Compute Capability 3.5 or newer. Most NVidia GPUs produced since 2014 will be compatible. For a complete list of compute capability, refer to NVidia CUDA GPU Capability List .

When multiple GPUs are in use, NVlink should always be installed between the GPUs to ensure top performance.

To execute multi-GPU jobs on Windows, the GPUs must support a feature called TCC Mode. Generally this feature is limited NVidia Quadro, Tesla, and TITAN products. Geforce tends not to have this feature. You should verify independently this specification if you intend to run multi-GPU jobs on Windows. Additionally, TCC mode requires dedicated usage for CUDA jobs, and may not be used for display. For example, you cannot put 2x V100 GPUs in TCC mode and plug one of them into your monitor. You must have a third GPU dedicated for display in that case.

Multi node jobs that span multiple comptuers are currently not supported on Windows. This type of workload is only supported under linux.

Important

Multi-GPU jobs on Windows require TCC Mode. TCC Mode GPUs must only be used for CUDA jobs and may not be used for display purposes.

GPU Driver Support

• Linux - NVidia driver version 418.39 or newer

• Windows - NVidia driver version 418.96 or newer

NVidia Hardware Notes

Tesla based hardware

Whenever possible, we suggest using NVidia Tesla GPUs. This is enterprise grade equipment intended to be used for heavy computational loads. They have features such as TCC mode on windows, ECC memory, and higher memory capacity.

Quadro based hardware

These GPUs work fine and can be cost effective means to run M-Star

Geforce based hardware

These are the more consumer grade GPUs intended primarily for rendering 3D scenes. These are compatible with the solver and can be highly cost effective in regards to memory and CUDA cores. Multi-GPU use of Geforce hardware is generally limited to Linux.

Note

Multi-GPU jobs with NVidia RTX 3090 is currently only supported on Linux

Minimum Requirement

For smaller simulation domain sizes, a basic setup as follows can be used.

• CPU: Dual core or better

• Memory: 16GB

• Disk Space: 100GB

• GPU: NVIDIA Geforce RTX 2060 6GB

Recommended Workstation

• CPU: Quad core or better

• Memory: 128GB+

• Disk 1: 1TB SSD operating system drive

• Disk 2: 2TB SSD drive for additional simulation scratch space.

• Display GPU: Any recent/modern Geforce GPU 2GB+

• Accelerator GPU 1: NVIDIA Tesla V100 or A100

• Accelerator GPU 2: NVIDIA Tesla V100 or A100 (optional)

• Bridge Accelerator GPU 1 & 2 with NVLINK

Recommended Servers

M-Star is an GPU compute intensive application that requires NVidia Tesla based GPUs for optimum performance. Generally speaking you want the largest capacity GPUs (CUDA cores and GPU memory). On each compute node you should max out the available GPU slots. You must connect the GPU with NVLINK/NVSwitch to provide the maximum amount of inter-GPU bandwidth. Network connections should be selected to provide the maxium available bandwidth, typically using an Infiniband based network. Multi-node jobs require GPUDirect RDMA support for optimal performance. It is suggested to speak with a HPC representative from a reputable vendor to clarify capability and application requirements.

These general guidelines shown below are a starting point for that discussion:

• OS: Linux (Redhat/Centos, Ubuntu are supported by M-Star)

• CPU: Core count must be equal to or greater than number of GPUs in the node.

• Memory: 512GB+ . More is typically better generally speaking, although the M-Star solver is typically limited to the memory on the GPU.

• Disk: 200GB - 1TB per M-Star user attached over network to all compute nodes. This depends on specific needs of end users. Disk storage requirements vary based on simulation config. You may want to look into high performance file systems that are typically used in HPC setups.

• NVidia V100, A100, or newer GPUs with the maximum amount of Per-GPU memory available.

• Max out the number of GPU per compute node. New builds should consider 8x or 16x GPUs per compute node.

• Use NVLINK and NVSwitch in order to provide the maximum amount of bandwidth between GPUs on the compute nodes. See https://www.nvidia.com/en-us/data-center/nvlink/ .

• Compute Node Interconnect: Infiniband with GPUDirect RDMA support (200Gb/s or better). Required for multi-node compute jobs.

If multi-node jobs need to be supported, the system should be setup to support GPUDirect RDMA, permitting point-to-point GPU-to-GPU communication across the node interconnect without relying on GPU to CPU memory copies. Refer to the vendor documentation concerning this subject. On Mellanox, this requires loading the nv_peer_mem and gdr_copy kernel modules. See – https://docs.mellanox.com/display/hpcxv24/Unified+Communication+-+X+Framework+Library .

Running on CPUs

This method of execution is still supported. It is not recommended to run on CPU due to the fact that GPUs, even consumer grade level ones, will always beat CPUs in performance. Additionally many advanced features in M-Star that use custom UDFs are not supported on CPU.

Tip

Strongly consider aquiring dedicated GPU