Cloud Composer 1 is in the post-maintenance mode. Google does not release any further updates to Cloud Composer 1, including new versions of Airflow, bugfixes, and security updates. We recommend planning migration to Cloud Composer 2.

Configure VPC Service Controls

Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3

VPC Service Controls enable organizations to define a perimeter around Google Cloud resources to mitigate data exfiltration risks.

Cloud Composer environments can be deployed within a service perimeter. By configuring your environment with VPC Service Controls, you can keep sensitive data private while taking advantage of the fully-managed workflow orchestration capabilities of Cloud Composer.

VPC Service Controls support for Cloud Composer means that:

Cloud Composer can now be selected as a secured service inside a VPC Service Controls perimeter.
All underlying resources used by Cloud Composer are configured to support VPC Service Controls architecture and follow its rules.

Deploying Cloud Composer environments with VPC Service Controls gives you:

Reduced risk of data exfiltration.
Protection against data exposure due to misconfigured access controls.
Reduced risk of malicious users copying data to unauthorized Google Cloud resources, or external attackers accessing Google Cloud resources from the internet.

About VPC Service Controls in Cloud Composer

All VPC Service Controls network constraints also apply to your Cloud Composer environments. See the VPC Service Controls documentation for details.

If a Cloud Composer environment is protected by a perimeter, access to public PyPI repositories is restricted. See the Install PyPI packages in VPC Service Controls for more information.
If your environment uses Private IP networking, all internal traffic is routed to your VPC network, except the traffic to Google APIs, services, and domains that are available to Private IP environments through Private Google Access.
Depending on how you configure your VPC network, a Private IP environment can gain access to the internet through you VPC network.
In VPC Service Controls mode, access to web server is protected by the perimeter and access from outside of the perimeter is blocked. To allow access from outside the service perimeter, configure access levels or ingress and egress rules as required. In addition, you can restrict access to web server to specific IP ranges.

About connectivity to Google APIs and services in VPC Service Controls

Cloud Composer 3 routes traffic to Google services through restricted.googleapis.com, which enables access to Google APIs, services, and domains supported by this range.

For more information and the list of services and domains available through restricted.googleapis.com, see Network configuration in the Virtual Private Cloud documentation.

Cloud Composer 3 blocks calls to Google APIs, services, and domains that aren't in the list of required APIs and services. If you want to call an API from a DAG make sure to:

Add the service to restricted services
Add the service to VPC accessible services

For example, if you use a VertexAI Operator, add aiplatform.googleapis.com both to restricted services and to VPC accessible services.

For more information, see Manage service perimeters in the VPC Service Controls documentation.

Create a service perimeter

See Creating a service perimeter to learn how to create and configure service perimeters. Make sure to select Cloud Composer as one of the services secured within the perimeter.

Create environments in a perimeter

The following steps are required to deploy Cloud Composer inside a perimeter:

Enable Access Context Manager API and Cloud Composer API for your project. See Enabling APIs for reference.
Make sure that your service perimeter has the following VPC accessible Google services, otherwise your environment might fail to create:
- Cloud Composer API (composer.googleapis.com)
- Artifact Registry API (artifactregistry.googleapis.com)
- Compute Engine API (compute.googleapis.com)
- Kubernetes Engine API (container.googleapis.com)
- Container File System API (containerfilesystem.googleapis.com)
- Cloud DNS API (dns.googleapis.com)
- Service Account Credentials API (iamcredentials.googleapis.com)
- Cloud Logging API (logging.googleapis.com)
- Cloud Monitoring API (monitoring.googleapis.com)
- Cloud Pub/Sub API (pubsub.googleapis.com)
- Cloud SQL Admin API (sqladmin.googleapis.com)
- Cloud Storage API (storage.googleapis.com)
Create a new Cloud Composer environment:
1. Use Google Cloud CLI to create your environment.
2. Enable Private IP with the --enable-private-environment argument.
3. Specify access parameters for the web server with --web-server-allow-all, --web-server-allow-ip, or --web-server-deny-all arguments. For more information about using these arguments, see Create environments. To improve protection, only allow access to the web server from specific IP ranges.
4. Disallow the installation of packages from public internet repositories with the --enable-private-builds-only argument.
  
  Example:
```
gcloud composer environments create example-environment \
  --location us-central1 \
  --enable-private-environment \
  --web-server-allow-all \
  --enable-private-builds-only
```
By default, access to Airflow UI and API is only allowed from within the security perimeter. If you want to make it available outside of the security perimeter, then configure access levels or ingress and egress rules.

Add an existing environment to the perimeter

You can add the project containing your environment to a perimeter if your environments use Private IP and the installation of PyPI packages from public repositories is disabled.

To update an existing Cloud Composer 3 environment to this configuration:

Make sure that you already created or configured the perimeter as described in the previous section.
Use Google Cloud CLI to update your environment.
Enable Private IP with the --enable-private-environment argument.
Disallow the installation of packages from public internet repositories with the --enable-private-builds-only argument.
If required, configure access to the Airflow web server. To improve protection, only allow access to the web server from specific IP ranges.

Example:

gcloud composer environments update example-environment \
  --location us-central1 \
  --enable-private-environment \
  --enable-private-builds-only

Install PyPI packages in VPC Service Controls

In the default VPC Service Controls configuration, Cloud Composer only supports installing PyPI packages from private repositories that are reachable from the internal IP address space of the VPC network.

All Cloud Composer environments within a VPC Service Controls perimeter don't have access to public PyPI repositories by default.

Install from a private repository

The recommended configuration is to set up a private PyPI repository, populate it with vetted packages used by your organization, then configure Cloud Composer to install Python dependencies from a private repository.

Install from a public repository

To install PyPI packages from an external repository:

Create an Artifact Registry remote repository.
Grant this repository access to upstream sources.
Configure Airflow to install packages from an Artifact Registry repository.

The service account of your environment must have the following IAM roles:

Composer Worker (composer.worker)
Service Account User (iam.serviceAccountUser)

VPC Service Controls logs

When troubleshooting environment creation issues, you can analyze audit logs generated by VPC Service Controls.

In addition to other log messages, you can check logs for information about cloud-airflow-prod@system.gserviceaccount.com and service-PROJECT_ID@cloudcomposer-accounts.iam.gserviceaccount.com service accounts that configure components of your environments.

Cloud Composer service uses the cloud-airflow-prod@system.gserviceaccount.com service account to manage tenant project components of your environments.

The service-PROJECT_ID@cloudcomposer-accounts.iam.gserviceaccount.com service account, also known as Composer Service Agent Service Account manages environment components in service and host projects.