High Performance Computing clusters can be created on the Google Cloud Platform by utilizing Google Compute Engine VMs and Google Cloud Storage. By running HPC workloads in Google’s Cloud, customers can augment on-premise HPC clusters or run all their jobs in the cloud.
The compute portion of the HPC cluster consists of a Head Node running scheduling and management software on a Google Compute Engine VM. The compute/worker nodes are also running on Google Compute Engine VMs. Instances sizes can be selected to match the workload. Choices include Standard, High Memory or High CPU instances in 1, 2, 4, 8 or 16 core sizes. Instances can also be added or deleted depending on the resources needed. The user has a choice of various commercial packages or open source software components to create the cluster.
Compute Engine VMs can also be used to create a file system for the cluster. Two popular options are NFS and Gluster. This is an optional component as the Compute Nodes can also access Google Cloud Storage directly.
Google Cloud Storage provides the backend storage for the cluster. This is a durable, highly available storage option making it an excellent choice for HPC work. Google Cloud SQL is also available for structured input or output data. The input data can be uploaded by the client directly into Cloud Storage or uploaded with the job. The resulting data can be downloaded to the client or left in the cloud for storage or further processing.