Cluster deployment overview

This document provides an overview of how to use Cluster Toolkit to deploy a cluster on Google Cloud.

Before you begin

  1. If you are using a Linux or macOS workstation to deploy your cluster, install dependencies.
  2. From either your workstation or Cloud Shell, configure your environment.
  3. Ensure that you have created a cluster blueprint or selected one from the Cluster Toolkit blueprint catalog.

Switch to the main working directory

After you have installed the Cluster Toolkit, ensure that you switch to the main working directory.

cd cluster-toolkit

Deploy the cluster

To deploy the cluster, complete the following steps.

  1. If you you might need to deploy the cluster multiple times for different purposes or with different configurations, create a deployment file.

  2. To deploy the cluster, run the ghpc deploy command. If you are using any of the example cluster blueprints provided by Cluster Toolkit, you need to set the project ID by using either the --vars flag or --deployment-file (or -d) flag.

    • Using the --vars flag.

      ./gcluster deploy --vars project_id=PROJECT_ID PATH_TO_BLUEPRINT_FILE
      
    • Using the --deployment-file (or -d) flag.

      ./gcluster deploy -d PATH_TO_DEPLOYMENT_FILE PATH_TO_BLUEPRINT_FILE
      

      If the --deployment-file and --vars flags are used at the same time, then the variables specified with --vars take precedence. If the --vars flag is specified multiple times, then later invocations take precedence.

      Replace the following:

      • PROJECT_ID: your project ID.
      • PATH_TO_BLUEPRINT_FILE: the path to your cluster blueprint file. For example, if you are in the main working directory and want to use the hpc-slurm.yaml blueprint, specify examples/hpc-slurm.yaml.
      • PATH_TO_DEPLOYMENT_FILE: the path to your deployment file.
  3. The ghpc command reports proposed changes for your cluster. To continue the deployment, accept the proposed changes by typing a and pressing enter. Optionally, you may review the proposed changes by typing d and pressing enter.

    Summary of proposed changes: Plan: 37 to add, 0 to change, 0 to destroy.
    (D)isplay full proposed changes,
    (A)pply proposed changes,
    (S)top and exit,
    (C)ontinue without applying
    Please select an option [d,a,s,c]:
    
  4. After accepting the changes, ghpc runs terraform apply automatically. This takes approximately 5 minutes while it displays progress. If the run is successful, the output is similar to the following:

    Apply complete! Resources: 37 added, 0 changed, 0 destroyed.
    

You are now ready to submit jobs to your cluster.

Get help at the command line

For a full list of flags that you can use, run the --help flag on ghpc or on any of the sub-commands.

./gcluster --help
./gcluster deploy --help