Scripting gcloud CLI commands

In addition to running gcloud CLI commands from the command line, you can also run them from scripts or other automations — for example, when using Jenkins to drive automation of Google Cloud Platform tasks.

Authorization

Google Cloud SDK tools support two authorization methods:

User account authorization
Service account authorization

User account authorization is recommended if you are running a script or other automation on a single machine.

To authorize access and perform other common Cloud SDK setup steps:

    gcloud init

Service account authorization is recommended if you are deploying a script or other automation across machines in a production environment. It is also the recommended authorization method if you are running gcloud CLI commands on a Google Compute Engine virtual machine instance where all users have access to root.

To use service account authorization, use an existing service account or create a new one through the Google Cloud Platform Console. From the options column of the service accounts table, create and download the associated private key as a JSON-formatted key file.

To run the authorization, use gcloud auth activate-service-account:

    gcloud auth activate-service-account --key-file [KEY_FILE]

You can SSH into your VM instance by using gcloud compute ssh, which takes care of authentication. SSH configuration files can be configured using gcloud compute config-ssh.

For detailed instructions regarding authorizing Cloud SDK tools, refer to this comprehensive guide.

Disabling prompts

Some gcloud CLI commands are interactive, prompting users for confirmation of an operation or requesting additional input for an entered command.

In most cases, this is not desirable when running commands in a script or other automation. You can disable prompts from gcloud CLI commands by setting the disable_prompts property in your configuration to True or by using the global --quiet or -q flag. Most interactive commands have default values when additional confirmation or input is required. If prompts are disabled, these default values are used.

For example:

    gcloud --quiet debug targets list

Note the --quiet flag is inserted right at the front.

Handling output

If you want a script or other automation to perform actions conditionally based on the output of a gcloud CLI command, observe the following:

Don't depend on messages printed to standard error.

These may change in future versions of the gcloud CLI and break your automation.
Don't depend on the raw output of messages printed to standard output.

The default output for any command may change in a future release. You can minimize the impact of those changes by using the --format flag to format the output with one of the following: --format=json|yaml|csv|text|list to specify values to be returned. Run $ gcloud topic formats for more options.

You can modify the default output from --format by using projections. For increased granularity, use the --filter flag to return a subset of the values based on an expression. You can then script against those returned values.

Examples of formatting and filtering output can be found in the section below.
Do depend on command exit status.

If the exit status is not zero, an error occurred and the output may be incomplete unless the command documentation notes otherwise. For example, a command that creates multiple resources may only create a few, list them on the standard output, and then exit with a non-zero status. Alternatively, you can use the show_structured_logs property to parse error logs. Run $ gcloud config for more details.

Examples of filtering and formatting

To work through an interactive tutorial about using the filter and format flags instead, launch the tutorial using the following button:

The following are examples of common uses of formatting and filtering with gcloud CLI commands:

List instances created in zone us-central1-a:

    gcloud compute instances list --filter="zone:us-central1-a"

List in JSON format those projects where the labels match specific values (e.g. label.env is 'test' and label.version is alpha):

    gcloud projects list --format="json" \
    --filter="labels.env=test AND labels.version=alpha"

List projects with their creation date and time specified in the local timezone:

    gcloud projects list \
    --format="table(name, project_id, createTime.date(tz=LOCAL))"

List projects that were created after a specific date in table format:

    gcloud projects list \
    --format="table(projectNumber,projectId,createTime)" \
    --filter="createTime.date('%Y-%m-%d', Z)='2016-05-11'"

Note that in the last example, a projection on the key was used. The filter is applied on the createTime key after the date formatting is set.

List a nested table of the quotas of a region:

    gcloud compute regions describe us-central1 \
    --format="table(quotas:format='table(metric,limit,usage)')"

Print a flattened list of global quotas in CSV format:

    gcloud compute project-info describe --flatten='quotas[]' \
    --format='csv(quotas.metric,quotas.limit,quotas.usage)'

List compute instance resources with box decorations and titles, sorted by name, in table format:

    gcloud compute instances list \
    --format='table[box,title=Instances](name:sort=1,zone:title=zone,status)'

List the project authenticated user email address:

    gcloud info --format='value(config.account)'

Examples of scripting

Using this functionality of format and filter, you can combine gcloud CLI commands into a script to easily extract embedded information.

If you were to list all the keys associated with all your projects' service accounts, you'd need to iterate over all your projects and for each project, get all the service accounts associated with it. For each service account, get all the keys. This can be accomplished as demonstrated below:

As a bash script:

#!/bin/bash
for project in  $(gcloud projects list --format="value(projectId)")
do
  echo "ProjectId:  $project"
  for robot in $(gcloud iam service-accounts list --project $project --format="value(email)")
   do
     echo "    -> Robot $robot"
     for key in $(gcloud iam service-accounts keys list --iam-account $robot --project $project --format="value(name.basename())")
        do
          echo "        $key"
     done
   done
done

Or as Windows PowerShell:

foreach ($project in gcloud projects list --format="value(projectId)")
{
  Write-Host "ProjectId: $project"
  foreach ($robot in  gcloud iam service-accounts list --project $project --format="value(email)")
  {
      Write-Host "    -> Robot $robot"
      foreach ($key in gcloud iam service-accounts keys list --iam-account $robot --project $project --format="value(name.basename())")
      {
        Write-Host "        $key"
      }
  }
}

Oftentimes, you'll need to parse output for processing. For example, it'd be useful to write the service account information into an array and segregate values in the multi-valued CSV-formatted serviceAccounts.scope() field. The script below does just this:

#!/bin/bash
for scopesInfo in $(
    gcloud compute instances list --filter=name:instance-1 \
        --format="csv[no-heading](name,id,serviceAccounts[].email.list(),
                      serviceAccounts[].scopes[].map().list(separator=;))")
do
      IFS=',' read -r -a scopesInfoArray<<< "$scopesInfo"
      NAME="${scopesInfoArray[0]}"
      ID="${scopesInfoArray[1]}"
      EMAIL="${scopesInfoArray[2]}"
      SCOPES_LIST="${scopesInfoArray[3]}"

      echo "NAME: $NAME, ID: $ID, EMAIL: $EMAIL"
      echo ""
      IFS=';' read -r -a scopeListArray<<< "$SCOPES_LIST"
      for SCOPE in  "${scopeListArray[@]}"
      do
        echo "  SCOPE: $SCOPE"
      done
done

More information

For a step-by-step guide to building basic scripts with the gcloud CLI, refer to this beginner's guide to automating GCP tasks.

More involved examples of the output configuring capabilities built into gcloud CLI's filters, formats, and projections flags can be found in this blog post about filtering and formatting.