Configuring Shared VPC

Cloud Composer 1 | Cloud Composer 2

This page describes the Shared VPC network and host project requirements for Cloud Composer.

Shared VPC enables organizations to establish budgeting and access control boundaries at the project level while allowing for secure and efficient communication using private IPs across those boundaries. In the Shared VPC configuration, Cloud Composer can invoke services hosted in other Google Cloud projects in the same organization without exposing services to the public internet.

Guidelines for Shared VPC

Service and Host Projects for Cloud Composer
Figure 1. Service and Host Projects for Cloud Composer
  • Shared VPC requires that you designate a host project to which networks and subnetworks belong and a service project, which is attached to the host project. When Cloud Composer participates in a Shared VPC, the Cloud Composer environment is in the service project.

  • To set up Shared VPC, select the following IP ranges in the host project:

    • Primary IP Range of the subnet used by GKE nodes that Cloud Composer uses as its Compute Engine layer.
    • Secondary IP Range for GKE services.
    • Secondary IP Range for GKE pods.
  • Secondary IP Ranges cannot overlap with any other secondary ranges in this VPC.

  • Ensure that secondary ranges are large enough to accommodate the cluster's size and anticipated growth.

    Network prefixes of secondary ranges cannot be increased above the following values. Specify network prefixes that are equal or below these values. Specifying lower values results in larger CIDR ranges.

    - Pods: `/21`
    - Services: `/27`
    

    See Creating a VPC-native cluster for guidelines on configuring secondary ranges for pods and services.

  • The primary address range of the subnet should accommodate anticipated growth and account for the reserved IP addresses.

    The network prefix of the subnet's primary address range cannot be increased above /29. Specify a network prefix that is equal or below this value. Specifying a lower value results in a larger CIDR range.

Preparation

  1. Find the following project IDs and project numbers:

    • Host project: The project that contains the Shared VPC network.
    • Service project: The project that contains the Cloud Composer environment.
  2. Prepare your organization.

  3. Enable the GKE API in your host and service projects.

Configure the host project

Configure the host project as described further.

Configure networking resources

Choose one of the following options to allocate and configure networking resources. For each option, you must name the secondary IP ranges for pods and services.

Set up Shared VPC and attach the service project

  1. If not already done, Set up Shared VPC. If you already have set up Shared VPC, skip to the next step.

  2. Attach the service project, which you use to host Cloud Composer environments.

    When attaching a project, leave the default VPC Network permissions in place.

Edit permissions for the Google APIs service account

In the host project, edit permissions for the Google APIs service account, SERVICE_PROJECT_NUMBER@cloudservices.gserviceaccount.com.

For this account, add another role, compute.networkUser at the project level. This is a requirement for managed instance groups used with Shared VPC because this type of service account performs tasks such as instance creation.

Edit permissions for GKE service accounts

In the host project, edit permissions for the GKE service accounts, service-SERVICE_PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com.

For each service account, add another role, compute.networkUser.

This permission must be granted at the subnet level to allow a service account to set up the VPC peerings required by Cloud Composer.

Edit permissions for the GKE Service Account of the service project

In the host project, edit permissions for the GKE Service Account of the service project.

For this account, add another role, Host Service Agent User.

This allows the GKE Service Account of the service project to use the GKE Service Account of the host project to configure shared network resources.

Configure connectivity to *.pkg.dev

In the host project, make sure that DNS *.pkg.dev resolves to 199.36.153.4/30.

To do that, create a new zone as: CNAME *.pkg.dev -> pkg.dev. A pkg.dev. -> 199.36.153.4, 199.36.153.5, 199.36.153.6, 199.36.153.7

Edit permissions for the Composer Agent Service Account

  1. In the host project, if this is the first Cloud Composer environment, then provision the Composer Agent Service Account: gcloud beta services identity create --service=composer.googleapis.com.

  2. In the host project:

    1. Edit permissions for the Composer Agent Service Account, service-SERVICE_PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com)

    2. For this account, add another role:

      • For Private IP environments, add the Composer Shared VPC Agent role.

      • For Public IP environments, add the Compute Network User role.

You've completed the Shared VPC network configuration for the host project.

What's next