To simplify using Google Cloud for CAE workflows, we have assembled the right cloud components to meet the requirements of computationally-intensive CAE workloads. Specifically, our CAE solution is architected around Google Cloud’s H3 and C3 VM families, based on the latest Intel Xeon platform. These VM families provide high memory bandwidth for a balanced memory/flop ratio, which is ideally suited for CAE. The solution caters to tightly-coupled MPI applications as well as memory-demanding workloads with up to 16GB/core. It includes various storage options addressing a broad range of IO requirements. For resource management, it supports schedulers such as SchedMD’s Slurm and Altair's PBS Professional. 

The following architecture diagram illustrates the solution:

When running CAE workloads in the cloud, there are a number of factors to consider, including:

• Compatibility: ensure that CAE applications have been tested for compatibility with the cloud-based HPC environment.
• VM Selection: choose the right VM type that runs ISV software optimally—many CAE workloads lend themselves to compute architectures with high amounts of memory bandwidth and an overall balanced flops:memory bandwidth ratio.
• Scalability: consider that larger CAE workloads may need to run across multiple VMs, requiring support for MPI and dense deployments of the VMs.
• Throughput: assess how many workloads need to run concurrently and consider auto-scaling scheduling models that can grow and shrink the cloud resources according to demand.
• Cost: commercial ISV software packages for CAE often are licensed “by core”. This is an important factor in the total cost and can influence the choice of a VM type.
• Usage/Consumption: cloud offers elasticity that on-prem installation cannot offer, explore the right balance between on-demand versus committed use or even spot VMs.
• Ease of use: make sure that the cloud HPC solution you choose is easy to use and manage.

As part of the Google Cloud CAE solution, we developed a general purpose CAE reference architecture and blueprint, which can readily be used with Google’s Cloud HPC Toolkit to provision the CAE architecture in Google Cloud. We have verified the compatibility and performance of several leading ISV applications, including: 

• Ansys Fluent 
• Ansys Mechanical 
• Ansys LS-DYNA 
• Siemens Simcenter STAR-CCM+ 
• Altair Radioss
• OpenFOAM

See the benchmarks section below for the performance of those software packages.

The general purpose CAE Reference Architecture blueprint enables users to instantly bring up a cloud environment that is compatible with a wide range of CAE applications and workflows. It is a good option for users who want to have flexibility in choosing their CAE software and who want to manage their own HPC environment. It also serves as a starting point for system integrators, leveraging Google’s best-practices for running CAE simulations on Google Cloud.

Google Cloud also offers a number of application-specific blueprints for popular CAE software. These blueprints are pre-configured to provide optimal performance for the specific CAE software. Software with specific blueprints include:

• Ansys Fluent
• Siemens SimCenter STAR-CCM+
• OpenFOAM

The application-specific blueprints are a good option for users who want to get started with CAE quickly and easily. The blueprints provide a pre-configured environment that is optimized for the specific CAE software, so users do not need to worry about configuring the environment themselves.

Google Cloud partners with a number of HPC-as-a-Service providers, such as TotalCAE, Rescale, Parallel Works, Eviden Nimbix, Penguin Computing, and NAG and also with CAE ISV vendors such as Altair. These providers offer a variety of managed HPC solutions for CAE, including pre-configured CAE software environments, support for specific CAE applications, and expert consulting services. These offerings are a good option for users who want a managed HPC solution for CAE. The providers offer a variety of services, including pre-configured CAE software environments, support for specific CAE applications, and expert consulting services.

While the CAE solution is built on Google Compute Engine, it is similarly possible to build it on top of other compute frameworks, such as Google Kubernetes Engine or Google Batch. Kubernetes Engine is a managed Kubernetes service that can be used to run CAE workloads in a containerized environment. Kubernetes Engine is a good choice for CAE workloads that require scalability and portability. Google Batch is a managed service for running batch jobs. Batch is a good choice for CAE workloads that are not containerized, and do not require significant customization or tuning.

You can read more about architecting HPC environments in our technical guide to Cluster Toolkit, which covers the breadth of options in infrastructure (compute, network, storage), system software (schedulers, storage), and architectural considerations.

There are a number of best practices you can follow to optimize the performance of your CAE workloads on Google Cloud. For example, you can use placement policies to ensure that your workloads are placed on compute resources that are close to each other, which can reduce latency and improve performance. You can also use the Cloud HPC Toolkit to optimize your workloads.

Our “Best practices for running HPC workloads” guide documents how to improve MPI performance. Both Open MPI and Intel MPI have been tuned and optimized for Google Cloud performance out of the box, in partnership with Google Cloud’s HPC networking engineers.

Google Cloud offers a wide range of machine types, each with different CPU, GPU, and memory configurations. Choosing the right machine type for your workload can have a significant impact on performance and cost. For example, the H3 VM is a good choice for per-core licensed CAE applications due to the high memory-bandwidth to core ratio and with 4 GB/core the H3 VM provides sufficient memory for diverse workloads. For particularly memory demanding workloads, such as structural mechanics, the C3 VM in its high memory configuration provides 16 GB/core.

Google Cloud offers a variety of storage options, each with different performance and cost characteristics. Choosing the right storage option for your workload can have a significant impact on performance and cost. There are also several types of storage in an HPC environment to consider.

Typical HPC environments host at least two types of storage with different requirements: home storage and scratch storage. The choice of storage type for each will depend on the specific needs of the HPC workload. For example, a workload that generates a large amount of scratch data may require a high-performance scratch storage solution, or a workload which accesses common data across multiple compute nodes simultaneously may require a parallel file system. The choice of storage type for a particular HPC workload will depend on the specific needs of the workload.

In addition to home storage and scratch storage, HPC environments may also use other types of storage, such as archive storage, which is used for storing data that is not frequently accessed. Archive storage can be provided most cost-effectively by Cloud Storage.

Home Storage

Home storage is typically used for storing shared user files, primarily in the “/home” directory, such as configurations, scripts, and post-processing data. This storage will be mounted in the same place across the cluster to allow common access to this namespace. Home storage is typically persistent. Home storage is typically built on the NFS protocol.

In a Google Cloud HPC environment, home storage can be provided by Google services like Filestore or partner offerings like NetApp.

• Google Cloud Filestore is a fully managed file storage service that provides high performance and low latency access to data. Filestore is a good choice for storing home storage data that is frequently accessed.
• NetApp Cloud Volumes Service is a fully managed file storage service that provides high performance and low latency access to data. NetApp Cloud Volumes Service is a good choice for storing home storage data that is frequently accessed.

Scratch Storage

Scratch storage is typically used for storing temporary files, such as intermediate results and simulation output data. These can be shared or not shared across different nodes in the HPC environment. Scratch storage is typically not persistent. Scratch storage is typically built on higher performance storage systems than home storage, such as local flash storage or Parallel File Systems.

In a Google Cloud HPC environment, scratch storage can be provided by Google services such as Persistent Disk, Local SSD, Cloud Filestore, or Parallelstore or by partner offerings like NetApp, DDN EXAScaler, Sycomp, and Weka.

• Google Cloud Persistent Disk is a block storage service that provides high performance and low latency access to data. Persistent Disk can be used to create a scratch directory for each user on an HPC cluster.
• Google Cloud Local SSD is a local SSD storage option that is available on some Compute Engine machine types. Local SSD can be used to create a scratch directory for each user on an HPC cluster.
• Google Cloud Filestore is a fully managed NFS file storage service that provides high performance and low latency access to data. Filestore can be used to create a scratch directory for each user on an HPC cluster.
• Google Cloud Parallelstore is a high-performance parallel file system that is optimized for HPC workloads. Parallelstore can be used to create a scratch directory for each user on an HPC cluster.
• NetApp Cloud Volumes Service is a fully managed file storage service that provides high performance and low latency access to data. NetApp Cloud Volumes Service can be used to create a scratch directory for each user on an HPC cluster.
• DDN EXAScaler Cloud is a high-performance parallel file system that is optimized for HPC workloads. DDN EXAScaler Cloud can be used to create a single or multiple scratch directories on an HPC cluster.
• WekaIO Matrix is a high-performance parallel file system that is optimized for HPC workloads. WekaIO Matrix can be used to create a scratch directory for each user on an HPC cluster.

Spot VMs

Spot VMs can be a cost-effective way to run CAE workloads. Spot VMs are available at a discounted price but can terminate anytime with a short notice period. Spot VMs are up to 91% off the cost of a standard instance and support the kind of features HPC users expect, including GPUs and Local SSDs. 

If your workflow can tolerate the chance of interruption (preemption), testing out the Spot model is a good idea if the application can run in a relatively short time frame (< 4 hours). Customers have found that the up to 90% cost saving versus on-demand saves them enough to tolerate minor interruptions.

Note that certain VM types, such as H3, do not support Spot.

AirShaper is an online aerodynamics platform that allows designers and engineers to run air flow simulations in a completely automated way, allowing users without expertise in aerodynamics to obtain reliable results and improve their design.

AirShaper migrated their HPC Computational Fluid Dynamics (CFD) workloads to the new C2D VM family from an older VM platform, saving them simulation time and cost per workload run compared to their previous cloud environment, and dramatically improving their time to results compared to their on-premises environment.

“At AirShaper, we offer CFD simulations at a fixed cost. More and faster cores usually means a higher overall cost, in part due to scaling issues. But with H3 we can cut simulation times in half while even reducing the overall cost."

- Wouter Remmerie, CEO, Airshaper

Altair is a global technology company that provides software and cloud solutions in the areas of product development, high performance computing (HPC), and data analytics. Altair's software is used by engineers, scientists, and data analysts to solve complex problems in a wide range of industries, including automotive, aerospace, manufacturing, and energy.

Altair is a Google Cloud partner, and its software is available on Google Cloud. Altair's software is optimized for Google Cloud, and it can be used to take advantage of the performance, scalability, and flexibility of Google Cloud. Altair is committed to helping customers achieve their HPC goals, and offers a wide range of software solutions for HPC. One of these is Radioss, a finite element analysis tool. Altair has been able to demonstrate significant improvements in cloud-based runtime of Radioss with the new H3 VM.

“At Altair, we're excited that initial testing indicates up to 3x reduction in simulation runtime for Radioss workloads running on H3 versus C2. These significantly faster runtimes on Google Cloud will help to increase engineering productivity for our joint customers.”

- Eric Lequiniou, Senior Vice President, Radioss Development and Altair Solver

TotalCAE is a leading provider of managed HPC solutions for engineering and scientific applications. TotalCAE's solutions are designed to be easy to use, and to help customers accelerate their time to results, reduce costs, and improve productivity. TotalCAE's solutions are used by customers around the world to solve complex engineering and scientific problems. For example, TotalCAE's solutions are used to design and simulate aircraft, cars, and other vehicles; to analyze the performance of buildings and bridges; and to develop new drugs and therapies.

TotalCAE is a Google Cloud partner, and its solutions support operating on Google Cloud. TotalCAE's solutions are optimized for Google Cloud, and they can be used to take advantage of the performance, scalability, and flexibility of Google Cloud. By using Google Cloud’s HPC infrastructure, TotalCAE has been able to offer their customers better performance at a lower cost. 

“With Google Cloud H3 instances, we have seen up to a 25% performance boost per core for CAE workloads at a 50% lower job cost than C2, enabling TotalCAE to offer customers up to 2.5x higher price performance, and scalability for CAE workloads on Google Cloud.”

- Rodney Mach, CEO, TotalCAE