Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3
This page provides information about Private IP Cloud Composer environments.
For Private IP environments, Cloud Composer assigns only private IP (RFC 1918) addresses to the managed Google Kubernetes Engine and Cloud SQL VMs in your environment, resulting in no inbound access to those managed VMs from the public internet. As an option, you can also use privately used public IP addresses and the IP Masquerade agent to save the IP address space and to use non-RFC 1918 addresses.
By default, in a Private IP environment, Cloud Composer workflows do not have outbound internet access. Access to Google Cloud APIs and services is not affected by routing over Google's private network.
VPC-native GKE cluster
When you create an environment, Cloud Composer distributes your environment's resources between a Google-managed tenant project and your customer project.
For a Private IP environment, Cloud Composer creates a VPC-native GKE cluster for your environment in your customer project.
VPC-native clusters use Alias IP routing built into the VPC network, enabling the VPC to manage routing for pods. When you use VPC-native clusters, GKE automatically chooses a secondary range. For specific networking requirements, you can also configure the secondary ranges for your GKE pods and GKE services when you create an environment.
Private IP Cloud Composer environment
You can select a Private IP environment when you create an environment. Using private IP means that the GKE and Cloud SQL VMs in your environment are not assigned public IP addresses and communicate only over Google's internal network.
When you create a Private IP environment, the GKE cluster for your environment is configured as a private cluster, and the Cloud SQL instance is configured for private IP.
If your Private IP environment uses Private Service Connect, your customer project's VPC network and your tenant project's VPC network connect through a PSC endpoint.
If your Private IP environment uses VPC peerings, Cloud Composer creates a peering connection between your customer project's VPC network and your tenant project's VPC network.
With private IP enabled for your environment, the IP traffic between your environment's GKE cluster and Cloud SQL database is private, thus isolating your workflows from the public internet.
This additional layer of security affects how you connect to these resources and how your environment accesses external resources. Using private IP does not affect how you access Cloud Storage or your Airflow webserver over the public IP.
GKE cluster
Using a private GKE cluster enables you to control access to the cluster's control plane (cluster nodes do not have public IP addresses).
When you create a private IP Cloud Composer environment, you specify whether or not access to the control plane is public and its IP range. The control plane IP range must not overlap with any subnetwork in your VPC network.
| Option | Description |
|---|---|
| Public endpoint access disabled | To connect to the cluster, you must
connect from a VM in the same region and same VPC
network of the Private IP environment.
The VM instance you are connecting from requires
the Access scope
Allow full access to all Cloud APIs. From that VM, you can run kubectl commands on your environment's
cluster |
| Public endpoint access enabled, master authorized networks enabled | In
this configuration, cluster nodes communicate with the control plane over
Google's private network. Nodes can access resources in your environment
and in authorized networks. You can
add authorized networks in
GKE. On authorized networks, you can run kubectl commands on your
environment's cluster |
Cloud SQL
Because the Cloud SQL instance does not have a public IP address, the Cloud SQL traffic inside your Private IP environment is not exposed to the public internet.
Cloud Composer configures Cloud SQL to accept incoming connections through private service access. You can access the Cloud SQL instance on your VPC network by using its private IP address.
Public internet access for your workflows
Operators and operations that require access to resources on unauthorized networks or on the public internet can fail. For example, the Dataflow Python operation requires a public internet connection to download Apache Beam from pip.
Allowing VMs without external IP addresses and private GKE clusters to connect to the internet requires Cloud NAT.
To use Cloud NAT, create a NAT configuration using Cloud Router for the VPC network and region that your private IP Cloud Composer environment is in.