Managing TPUs

You can use the gcloud commands described in this document with both TPU configurations: TPU VMs and TPU Nodes. The gcloud commands you use depend on the TPU configuration you are using. Each gcloud command is shown in a tabbed section. Choose the tab for the TPU configuration you want to use and the web page shows the appropriate gcloud command. Unless you know you need to use TPU Nodes, we recommend using TPU VMs. For more information about TPU configurations, see System Architecture.

Running a Machine Learning (ML) model requires a Compute Engine VM and Cloud TPU resources. This page describes how to manage these resources using:

The Google Cloud CLI, which provides the primary CLI to Google Cloud
The Google Cloud console, which provides an integrated management console for your Google Cloud resources.

Prerequisites

To run these procedures, you need to have a Google Cloud project set up. If you don't have a project, see Creating and managing projects to set one up.

If you are using the Google Cloud CLI you can use the Google Cloud Shell, a Compute Engine VM or install the Google Cloud CLI locally. The Google Cloud Shell allows you to interact with Cloud TPUs without having to install any software. The Google Cloud Shell may disconnect after a period of inactivity. If you're running long-running commands, we recommend installing the Google Cloud CLI on your local machine.

Install the Google Cloud CLI.
Configure gcloud to use your project.
```
gcloud config set project project-name
```
Configure gcloud to use the zone where you plan to create your Cloud TPU resources. For example, us-central1-b.
```
$ gcloud config set compute/zone zone
```

For more information on the gcloud command, see the gcloud Reference.

Creating a Cloud TPU

When you create a Cloud TPU, you create Compute Engine VM and TPU resources.

Creating a Cloud TPU with `gcloud`

If you want to use the Cloud Shell, click Open Cloud Shell. Otherwise, open a command prompt/terminal window on your local computer.

Create your Cloud TPU resources. The commands you use depend on whether you are using TPU VMs or TPU nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm create tpu-name \
  --zone=zone \
  --accelerator-type=v3-8 \
  --version=tpu-vm-tf-2.12.0

Command flag descriptions

zone: The zone where you plan to create your Cloud TPU.
accelerator-type: The type of the Cloud TPU to create.
version: The Cloud TPU software version.
shielded-secure-boot (optional): Specifies that the TPU instances are created with secure boot enabled. This implicitly makes them Shielded VM instances. See What is shielded VM? for more details.

TPU Nodes

$ gcloud compute tpus execution-groups create --name=tpu-name \
  --zone=zone \
  --tf-version=2.12.0 \
  --machine-type=n1-standard-1 \
  --accelerator-type=v3-8

Command flag descriptions

zone: The zone where you plan to create your Cloud TPU.
tf-version: The version of Tensorflow the gcloud command installs on your VM.
machine-type: The machine type of the Compute Engine VM to create.
accelerator-type: The type of the Cloud TPU to create.

Creating a Cloud TPU queued resource with `gcloud`

Using gcloud you can also create a Cloud TPU as a queued resource. When you make a request for a queued resource, your request is added to a queue managed by Cloud TPU. When a resource becomes available, the resource is allocated and is available for your exclusive use. For more information see, Cloud TPU queued resources.

Run standard installation scripts

You can run a startup script on each TPU VM by specifying the --metadata startup-script parameter when creating the TPU VM. The following is an example of using a startup-script for TPU VM.

$ gcloud compute tpus tpu-vm create tpu-name \
    --zone=zone \
    --accelerator-type=tpu-type \
    --version=tpu-vm-tf-2.12.0 \
    --metadata startup-script='#! /bin/bash
      pip3 install numpy
      EOF'

After the TPU VM is created, you can view the logs from the startup script by connecting to the TPU VM using SSH and running:

$ cat /var/log/syslog | grep startup-script

Creating a Cloud TPU in the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs.
Click CREATE TPU NODE.
In the Name box, type a TPU instance name.
In the Zone box, select the zone in which to create the TPU.
Under TPU Configuration, select either TPU VM or TPU Node. The TPU configuration determines whether you create the TPU as a TPU VM or a TPU Node. For more information, see System Architecture.
For TPU type, select the TPU type you want to create.
For TPU software version, select the software version. When creating a Cloud TPU VM, the TPU software version specifies the version of the TPU runtime to install. When creating a Cloud TPU Node, the TPU software version allows you to choose the ML framework installed on the node's VM. No other settings are required. For more information, see Supported Models.
Click CREATE to create your resources.

Connecting to a Cloud TPU VM

By default, the gcloud command you use to create TPU Nodes automatically attempts to SSH into your TPU node. If you are using TPU Nodes and are not connected to the Compute Engine instance by the gcloud command, you can connect by running the following TPU Nodes command. When using TPU VMs you must explicitly SSH into your TPU using the following TPU VM command.

TPU VMs

$ gcloud compute tpus tpu-vm ssh tpu-name \
  --zone=zone

TPU Nodes

$ gcloud compute ssh tpu-name\
  --zone=zone

Listing your Cloud TPU resources

You can list all of your Cloud TPU in a specified zone.

Listing your Cloud TPU resources using `gcloud`

The commands you use depend on whether you are using TPU VMs or TPU nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm list --zone=zone

TPU Nodes

$ gcloud compute tpus execution-groups list --zone=zone

This command lists the Cloud TPU resources in the specified zone. If no resources are currently set up, the output will just show dashes for the VM and TPU. If one resource is active and the other is not, you will see a message saying the status is unhealthy. You need to start or restart whichever resource is not running.

Listing your Cloud TPU resources using the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs. The console displays the TPUs page.

Retrieving information about your Cloud TPU

You can retrieve information about a specified Cloud TPU.

Retrieve information about a Cloud TPU using `gcloud`

The commands you use depend on whether you are using TPU VMs or TPU nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm describe tpu-name \
  --zone=zone

TPU Nodes

$ gcloud compute tpus execution-groups describe tpu-name \
  --zone=zone

Retrieve information about a Cloud TPU using the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs. The console displays the TPUs page.
Click the name of your Cloud TPU, the Cloud TPU detail page is displayed.

Stopping your Cloud TPU resources

You can stop a single Cloud TPU to stop incurring charges without losing your VM's configuration and software. Stopping TPU Pods or TPUs allocated through the queued resources API is not supported. To stop incurring charges for TPUs allocated through the queued resources API, you must delete the TPU.

Stopping a Cloud TPU with `gcloud`

The command you use for stopping a Cloud TPU depend on whether you are using TPU VMs or TPU Nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm stop tpu-name \
--zone=zone

TPU Nodes

$ gcloud compute tpus stop tpu-name \
--zone=zone

Stopping a Cloud TPU in the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs. The console displays the TPUs page.
Select the checkbox next to your Cloud TPU and click Stop.

Starting your Cloud TPU resources

You can start a Cloud TPU when it is stopped.

Starting a Cloud TPU with `gcloud`

You can start a stopped Cloud TPU to resume using it.

The command you use for starting a stopped Cloud TPU depend on whether you are using TPU VMs or TPU Nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm start tpu-name --zone=zone

TPU Nodes

$ gcloud compute tpus start tpu-name --zone=zone

Starting a Cloud TPU in the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs. The console displays the TPUs page.
Select the checkbox next to your Cloud TPU and click Start.

Deleting your Compute Engine VM and Cloud TPU resources

You can delete your Cloud TPU when you are done using them.

Deleting a Cloud TPU using `gcloud`

The command you use depends on whether you are using TPU VMs or TPU nodes. For more information, see System Architecture.

TPU VMs

$ gcloud compute tpus tpu-vm delete tpu-name \
  --zone=zone

Command flag descriptions

zone: The zone where you plan to create your Cloud TPU.

TPU Nodes

$ gcloud compute tpus execution-groups delete tpu-name \
  --zone=zone

Command flag descriptions

zone: The zone where you plan to create your Cloud TPU.

Deleting a Cloud TPU using the Google Cloud console

Navigate to the Google Cloud console.
From the navigation menu, select Compute Engine > TPUs. The console displays the TPUs page.
Select the checkbox next to your Cloud TPU and click Delete.

Advanced Configurations

Custom Network Resources

When you create the TPU, you can choose to specify the network and/or a subnetwork. You can do this either by submitting a gcloud command or a curl call.

To specify the network or subnetwork in the gcloud CLI, use:

--network [NETWORK] --subnetwork [SUBNETWORK]

To specify the network or subnetwork in a curl call, use:

network_config: {network: '[NETWORK]', subnet: '[SUBNETWORK]', enable_external_ips: true}

Network

You can optionally specify the network to use for the TPU. If not specified, the default network is used.

Valid network formats:

https://www.googleapis.com/compute/{version}/projects/{proj-id}/global/networks/{network}
compute/{version}/projects/{proj-id}/global/networks/{network}
compute/{version}/projects/{proj-##}/global/networks/{network}
projects/{proj-id}/global/networks/{network}
projects/{proj-##}/global/networks/{network}
global/networks/{network}
{network}

Subnetwork

You can specify the subnetwork to use a specific subnetwork. The specified subnetwork needs to be in the same region as the zone where the TPU runs.

Valid Formats:

https://www.googleapis.com/compute/{version}/projects/{proj-id}/regions/{region}/subnetworks/{subnetwork}
compute/{version}/projects/{proj-id}/regions/{region}/subnetworks/{subnetwork}
compute/{version}/projects/{proj-##}/regions/{region}/subnetworks/{subnetwork}
projects/{proj-id}/regions/{region}/subnetworks/{subnetwork}
projects/{proj-##}/regions/{region}/subnetworks/{subnetwork}
regions/{region}/subnetworks/{subnetwork}
{subnetwork}

Private Google Access

In order to SSH into the TPU VMs, you need to either add access configs for the TPU VMs, or turn on the Private Google Access for the subnetwork to which the TPU VMs are connected.

To add access configs, enable_external_ips must be set. When you create a TPU, enable_external_ips is set by default. If you want to opt out, specify the following command:

--internal-ips

Or use a curl call:

network_config: {enable_external_ips: true}

After you have configured Private Google Access, connect to the VM via SSH.

Custom Service Account

Each TPU VM has an associated service account it uses to make API requests on your behalf. TPU VMs use this service account to access files on Cloud Storage and access other services. See Service Accounts to learn more about them.

When you create a TPU node, you can choose to specify a custom service account for the TPU VM identities. By default, Google Compute Engine default service account is used. A custom service account needs to be in the project that you use to create the TPU. Use the following commands to specify a custom service account.

Specify in the gcloud CLI:

--service-account=[SERVICE_ACCOUNT]

Specify using curl:

service_account: {email: '[SERVICE_ACCOUNT]'}

To use a custom service account, you need to authorize the service account for your Google Cloud Storage buckets. See Connecting to Cloud Storage Buckets for instructions.

Custom VM SSH methods

Set up a firewall for SSH

The default network comes preconfigured to allow SSH access to all VMs. If you don't use the default network, or the default network was edited, you may need to explicitly enable SSH access by adding a firewall-rule:
```
$ gcloud compute firewall-rules create \
--network=NETWORK allow-ssh \
--allow=tcp:22
```
SSH into the TPU VMs
```
$ gcloud compute tpus tpu-vm ssh ${TPU_NAME} \
--zone ${ZONE} \
--project ${PROJECT_ID}
```
Required fields
- TPU_NAME: Name of the TPU node.
- ZONE: The location of the TPU node. Currently, only us-central2-b is supported.
- PROJECT_ID: The project you created above.
See the gcloud API documentation for a list of optional fields.

Managing TPUs

Prerequisites

Creating a Cloud TPU

Creating a Cloud TPU with gcloud

TPU VMs

Command flag descriptions

TPU Nodes

Command flag descriptions

Creating a Cloud TPU queued resource with gcloud

Run standard installation scripts

Creating a Cloud TPU in the Google Cloud console

Connecting to a Cloud TPU VM

TPU VMs

TPU Nodes

Listing your Cloud TPU resources

Listing your Cloud TPU resources using gcloud

TPU VMs

TPU Nodes

Listing your Cloud TPU resources using the Google Cloud console

Retrieving information about your Cloud TPU

Retrieve information about a Cloud TPU using gcloud

TPU VMs

TPU Nodes

Retrieve information about a Cloud TPU using the Google Cloud console

Stopping your Cloud TPU resources

Stopping a Cloud TPU with gcloud

TPU VMs

TPU Nodes

Stopping a Cloud TPU in the Google Cloud console

Starting your Cloud TPU resources

Starting a Cloud TPU with gcloud

TPU VMs

TPU Nodes

Starting a Cloud TPU in the Google Cloud console

Deleting your Compute Engine VM and Cloud TPU resources

Deleting a Cloud TPU using gcloud

TPU VMs

Command flag descriptions

TPU Nodes

Command flag descriptions

Deleting a Cloud TPU using the Google Cloud console

Advanced Configurations

Custom Network Resources

Network

Subnetwork

Private Google Access

Custom Service Account

Custom VM SSH methods

Creating a Cloud TPU with `gcloud`

Creating a Cloud TPU queued resource with `gcloud`

Listing your Cloud TPU resources using `gcloud`

Retrieve information about a Cloud TPU using `gcloud`

Stopping a Cloud TPU with `gcloud`

Starting a Cloud TPU with `gcloud`

Deleting a Cloud TPU using `gcloud`