Set up Cluster Toolkit

To use Cluster Toolkit, you need a Google Cloud project. In your project, you need to enable APIs, permissions, and grant credentials to Terraform. You also need to clone and build the Cluster Toolkit repository.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Enable the Compute Engine, Filestore, Cloud Storage, and Service Usage APIs.

    Enable the APIs

    5.

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Make sure that billing is enabled for your Google Cloud project.

  7. Enable the Compute Engine, Filestore, Cloud Storage, and Service Usage APIs.

    Enable the APIs

    8.

Overview

To set up Cluster Toolkit, you need to complete the following steps:

Set up your CLI

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Set a default project

To set a default project, run the following command:

Replace PROJECT_ID with your project ID.

gcloud config set project PROJECT_ID

Ensure that the default Compute Engine service account is enabled

Cluster Toolkit requires that the default Compute Engine service account is enabled in your project and that the roles/editor IAM role is enabled on the service account. This is the default setting that is set up by Google Cloud for projects.

However, if you had disabled the default Compute Engine service account and the roles/editor IAM role in your project, complete the following steps to enable these settings:

  1. Enable the default Compute Engine service account.

    gcloud iam service-accounts enable \
   --project PROJECT_ID \
   PROJECT_NUMBER-compute@developer.gserviceaccount.com

  2. Grant the roles/editor IAM role to the service account.

    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \
    --role=roles/editor

Replace the following:

  • PROJECT_ID: your project ID

  • PROJECT_NUMBER: the automatically generated unique identifier for your project

    For more information, see Identifying projects.

Grant user access credentials to Terraform

To generate cloud credentials associated with your Google Cloud account, run the following command:

gcloud auth application-default login

This grants the Application Default Credentials (ADC) to Terraform.

Enable OS Login at the project-level

To be able to connect to a VM in your cluster by using ssh, you must enable OS Login. If OS Login is already enabled at the organization level, you don't need to enable it on your project.

gcloud compute project-info add-metadata \
     --metadata enable-oslogin=TRUE

Clone the Cluster Toolkit GitHub repository

  1. Clone the GitHub repository:

    git clone https://github.com/GoogleCloudPlatform/cluster-toolkit.git

  2. Go to the main working directory:

    cd cluster-toolkit/

Build the Cluster Toolkit binary

  1. To build the Cluster Toolkit binary from source, run the following command:

    make

  2. To verify the build, run the following command:

    ./gcluster --version

After building the binary, you are now ready to deploy clusters to run your jobs or workloads.

What's next