Configure private IP networking

Cloud Composer 1 | Cloud Composer 2

This page provides information about configuring your Google Cloud project networking for Private IP environments.

For Private IP environments, Cloud Composer assigns only private IP (RFC 1918) addresses to the managed Google Kubernetes Engine and Cloud SQL VMs in your environment.

As an option, you can also use privately used public IP addresses and the IP Masquerade agent to save the IP address space.

For information about connecting to resources in your environment, see Private IP.

Before you begin

Make sure that you have the appropriate user and service account permissions to create an environment.

Step 1. Check network requirements

Verify that your project's VPC network meets the following requirements:

  • Make sure that there are no private IP block conflicts. If your VPC network and its established VPC peers have overlapping IP blocks with the VPC network in the Google-managed tenant project, Cloud Composer cannot create your environment. Consult the Web server IP range column in the default IP ranges table for the defaults used in each region.

  • Make sure that there are sufficient secondary IP ranges for the Cloud Composer GKE pods and services. GKE searches for secondary IP ranges for IP Aliasing. If GKE cannot find a range, Cloud Composer cannot create your environment.

  • Make sure that the number of VPC peering connections in your VPC network does not exceed 25. Consider the following:

    • The maximum number of Private IP environments you can create depends on the number of already existing VPC peering connections in your VPC network.
    • For each Private IP environment, Cloud Composer creates one peering connection for the tenant project network.
    • If your environment is located in a different zone than other environments in the VPC network, the private GKE cluster creates another VPC peering connection.
    • If all environments in the VPC network are in the same zone, the maximum number of Private IP environments that Cloud Composer can support is 23.
    • If all environments in the VPC network are in different zones, the maximum number of Private IP environments is 12.
  • Make sure that the number of secondary ranges in your subnetwork does not exceed 30. Consider the following:

    • The GKE cluster for your Private IP environment creates two secondary ranges in the subnetwork. You can create multiple subnetworks in the same region for the same VPC network.
    • The maximum number of supported secondary ranges is 30. Because each Private IP environment requires two secondary ranges for the Cloud Composer GKE pods and services, each subnetwork supports up to 15 Private IP environments.

Step 2. Choose a network, subnetwork, and network ranges

Choose the network ranges for your Private IP environment (or use the default ones). You use these network ranges later when you create a Private IP environment.

To create a Private IP environment, you need to have the following information:

  • Your VPC network ID
  • Your VPC subnetwork ID
  • Two secondary IP ranges in your VPC subnetwork:
    • Secondary IP range for pods
    • Secondary IP range for services
  • Three IP ranges for the components of the environment:
    • GKE Control Plane IP range. IP range for the GKE control plane.
    • Web server IP range. IP range for the Airflow web server instance
    • Cloud SQL IP range. IP range for the Cloud SQL instance.

Consult the default IP ranges table for the defaults used in each region.

Default IP ranges

Region GKE control plane IP range Web server IP range Cloud SQL IP range
asia-east2 172.16.0.0/23 172.31.255.0/24 10.0.0.0/12
asia-northeast1 172.16.2.0/23 172.31.254.0/24 10.0.0.0/12
asia-northeast2 172.16.32.0/23 172.31.239.0/24 10.0.0.0/12
asia-northeast3 172.16.30.0/23 172.31.240.0/24 10.0.0.0/12
asia-south1 172.16.4.0/23 172.31.253.0/24 10.0.0.0/12
australia-southeast1 172.16.6.0/23 172.31.252.0/24 10.0.0.0/12
europe-west1 172.16.8.0/23 172.31.251.0/24 10.0.0.0/12
europe-west2 172.16.10.0/23 172.31.250.0/24 10.0.0.0/12
europe-west3 172.16.12.0/23 172.31.249.0/24 10.0.0.0/12
europe-west6 172.16.14.0/23 172.31.248.0/24 10.0.0.0/12
europe-central2 172.16.36.0/23 172.31.237.0/24 10.0.0.0/12
northamerica-northeast1 172.16.16.0/23 172.31.247.0/24 10.0.0.0/12
southamerica-east1 172.16.18.0/23 172.31.246.0/24 10.0.0.0/12
us-central1 172.16.20.0/23 172.31.245.0/24 10.0.0.0/12
us-east1 172.16.22.0/23 172.31.244.0/24 10.0.0.0/12
us-east4 172.16.24.0/23 172.31.243.0/24 10.0.0.0/12
us-west2 172.16.34.0/23 172.31.238.0/24 10.0.0.0/12
us-west3 172.16.26.0/23 172.31.242.0/24 10.0.0.0/12
us-west4 172.16.28.0/23 172.31.241.0/24 10.0.0.0/12

Step 3. Configure firewall rules

Configure your VPC network to allow traffic from your Private IP environment to reach the required destinations.

  1. In the Google Cloud Console, go to the Firewall page.

    Go to Firewall

  2. Configure the following firewall rules:

    • Allow egress from GKE Node IP range to any destination (0.0.0.0/0), TCP/UDP port 53.
    • Allow egress from GKE Node IP range to GKE Node IP range, all ports.
    • Allow egress from GKE Node IP range to GKE Pods IP range, all ports.
    • Allow egress from GKE Node IP range to GKE Control Plane IP range, all ports.
    • Allow ingress from GCP Health Checks 130.211.0.0/22, 35.191.0.0/16 to GKE Node IP range, TCP ports 80 and 443.
    • Allow egress from GKE Node IP range to GCP Health Checks 130.211.0.0/22, 35.191.0.0/16, TCP ports 80 and 443.
    • Allow egress from GKE Node IP range to Web server IP range, TCP ports 3306 and 3307.

    See Using firewall rules to learn how to check, add, and update rules for your VPC network.

VPC Native configuration

Cloud Composer supports VPC-native GKE clusters in your environment.

During environment creation, you can enable VPC Native (using alias IP) and configure networking, such as IP allocation, without enabling private IP.

Because a VPC native cluster is required for Airflow tasks to communicate with other VMs that are reachable through private IPs, you must also enable VPC Native to configure a private IP environment.