Shared VPC networks

This guide explains how to set up Cloud TPUs that use a centrally managed Shared VPC network. A shared VPC network allows your organization to connect resources from multiple projects to a common network.

Before you begin

If you don't have the Google Cloud CLI installed, install it now.
Initialize the Google Cloud CLI.
Get familiar with Shared VPC concepts.
Enable a host project that contains one or more Shared VPC networks. This must be done by a Shared VPC admin.
Attach one or more service projects to your Shared VPC network. This must be done by a Shared VPC admin.
Create a VPC network in the host project.

Configure Private Service Access

Private Services Access is used to create a VPC peering between your network and the Cloud TPU service network. Before you use TPUs with Shared VPCs, you need to establish a private service access connection for the network.

Get the project ID for your Shared VPC host project, and then configure the Google Cloud CLI command with your project ID as shown below:
```
  gcloud config set project project-id
```
You can get the project ID from the Google Cloud console.
Enable the Service Networking API using the following Google Cloud CLI command.
```
  gcloud services enable servicenetworking.googleapis.com
```
You can also enable the Service Networking API from the Google Cloud console.

Allocate a reserved address range for use by Service Networking. The prefix-length needs to be 24 or less. For example:

gcloud compute addresses create SN-RANGE-1 --global \
--addresses=10.110.0.0 \
--prefix-length=16 \
--purpose=VPC_PEERING \
--network=network-name

Establish a private service access connection.

gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com --ranges=SN-RANGE-1 --network=network-name

Check if a Private Services Access connection has been established for the network. If it's already established, you can start using TPUs with the Shared VPC.

Verify Private Services Access

Check whether a Private Services Access connection has been established for your network by running following command:

gcloud services vpc-peerings list --network=network-name

Use TPUs with Shared VPC Networks

After you have established your VPC network with the Google service network, you can start provisioning Cloud TPUs.

Configure Google Cloud CLI

Create a variable for your project's ID.
```
export PROJECT_ID=project-id
```
Configure Google Cloud CLI to use the project where you want to create Cloud TPU.
```
gcloud config set project ${PROJECT_ID}
```

Activate TPU API

In order to use Cloud TPUs, you need to activate TPU API in your project. Run the following Google Cloud CLI command to enable the TPU API.

gcloud services enable tpu.googleapis.com

You may also enable it in the Google Cloud console.

Activate Service Networking API

In order to create TPUs with Shared VPC networks, you need to activate the Service Networking API in the service project. This only needs to be done once per Cloud Platform Project. Run the following Google Cloud CLI command to enable Service Networking API.

gcloud services enable servicenetworking.googleapis.com

You can also enable it from the Google Cloud console.

Grant Service Account Permissions in the host project

When you use the TPU VM architecture, you need to grant the service project's TPU Service Account permissions to manage resources in the host project. You do this using the "TPU Shared VPC Agent" (roles/tpu.xpnAgent) role. Run the following Google Cloud CLI commands to grant this role binding.

Create a variable for the project ID for your Shared VPC host project.
```
  export HOST_PROJECT_ID=host-project-id
```
Create a variable for the project number for your service project. This is the project that you create the TPUs in. You can find the project number in the Google Cloud Console using these instructions.
```
  export PROJECT_NUMBER=service-project-number
```

Add a role binding in the Shared VPC host project for the service project's TPU service account.

gcloud projects add-iam-policy-binding $HOST_PROJECT_ID \
  --member=service-${PROJECT_NUMBER}@gcp-sa-tpu.iam.gserviceaccount.com \
  --role=roles/tpu.xpnAgent

Create TPUs

You can launch a Compute Engine VM and Cloud TPU using the Google Cloud CLI command. The command you use depends on whether you are using TPU VMs or TPU nodes. For more information, see System Architecture.

TPU VM

$ gcloud compute tpus tpu-vm create tpu-name \
  --zone=zone \
  --accelerator-type=v3-8 \
  --version=tpu-vm-tf-2.7.0 \
  --network=network-name

TPU Node

$ gcloud compute tpus execution-groups create \
  --name=tpu-name \
  --zone=zone \
  --tf-version=2.7.0 \
  --machine-type=n1-standard-1 \
  --accelerator-type=v3-8 \
  --network=network-name \
  --use-service-networking

If using TPU node, set use-service-networking to true to create Cloud TPUs that can connect to Shared VPC networks.
The network field should include the host project ID or host project number and the network name. For example, projects/my-host-project-id/global/networks/my-network.
For information about what GCP zones support Cloud TPUs, see available zones.

For more information about working with TPUs see Cloud TPU User's Guide.

Delete the VPC peering

A peering connection can be disconnected using the compute networking API. These calls should be made in Shared VPC host projects.

List all VPC peerings.

gcloud compute networks peerings list --network=network-name

Delete a VPC peering.

gcloud compute networks peerings delete peering-name --network=network-name