Cloud Composer security overview

Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3

Cloud Composer offers a handful of security features and compliances that are beneficial for enterprise companies with stricter security requirements.

These three sections present information about Cloud Composer security features:

Basic security features

This section lists security-related features provided by default for each Cloud Composer environment.

Encryption at rest

Cloud Composer utilizes encryption at rest in Google Cloud.

Cloud Composer stores data in different services. For example, the Airflow Metadata DB uses Cloud SQL database, DAGs are stored in Cloud Storage buckets.

By default, data is encrypted using Google-managed encryption keys.

If you prefer, you can configure Cloud Composer environments to be encrypted with customer-managed encryption keys.

Uniform bucket-level access

Uniform bucket-level access allows you to uniformly control access to your Cloud Storage resources. This mechanism also applies to your environment's bucket, which stores your DAGs and plugins.

User permissions

Cloud Composer has several features for managing user permissions:

  • IAM roles and permissions. Cloud Composer environments in a Google Cloud project can be accessed only by users whose accounts are added to IAM of the project.

  • Cloud Composer-specific roles and permissions. You assign these roles and permissions to user accounts in your project. Each role defines the types of operations that a user account can perform on Cloud Composer environments in your project.

  • Airflow UI Access Control. Users in your project can have different access levels in the Airflow UI. This mechanism is called Airflow UI Access Control (Airflow Role-Based Access Control, or Airflow RBAC).

  • Domain Restricted Sharing (DRS). Cloud Composer supports Domain Restricted Sharing organizational policy. If you use this policy, then only users from the selected domains can access your environments.

Private IP environments

You can create Cloud Composer environments in the Private IP networking configuration.

In the Private IP mode, nodes of your environment's cluster do not have external IP addresses and do not communicate through the public internet.

Your environment's cluster uses Shielded VMs

Shielded VMs are virtual machines (VMs) on Google Cloud hardened by a set of security controls that help defend against rootkits and bootkits.

Cloud Composer 1 environments that were created based on GKE versions 1.18 and later use Shielded VMs to run the nodes of their environment cluster.

Advanced security features

This section lists advanced security-related features for Cloud Composer environments.

Customer Managed Encryption Keys (CMEK)

Cloud Composer supports Customer Managed Encryption Keys (CMEK). CMEK provide you with more control over the keys used to encrypt data at rest within a Google Cloud project.

You can use CMEK with Cloud Composer to encrypt and decrypt data generated by a Cloud Composer environment.

VPC Service Controls (VPC SC) Support

VPC Service Controls is a mechanism to mitigate data exfiltration risks.

Cloud Composer can be selected as a secured service inside a VPC Service Controls perimeter. All underlying resources used by Cloud Composer are configured to support VPC Service Controls architecture and follow its rules. Only Private IP environments can be created in a VPC SC perimeter.

Deploying Cloud Composer environments with VPC Service Controls gives you:

  • Reduced risk of data exfiltration.

  • Protection against data exposure due to misconfigured access controls.

  • Reduced risk of malicious users copying data to unauthorized Google Cloud resources, or external attackers accessing Google Cloud resources from the internet.

Web server network access control levels (ACL)

Airflow web servers in Cloud Composer are always provisioned with an externally accessible IP address. You can control from which IP addresses the Airflow UI can be accessed. Cloud Composer supports IPv4 and IPv6 ranges.

You can configure web server access restrictions in Google Cloud console, gcloud, API, and Terraform.

Secret Manager as a storage for sensitive configuration data

In Cloud Composer, you can configure Airflow to use Secret Manager as a backend where Airflow connection variables are stored.

DAG developers can also read variables and connection stored in Secret Manager from the DAG code.

Compliance to standards

See the pages linked below to check Cloud Composer's compliance with various standards:

See also

Some of the security features mentioned in this article are discussed in the the Airflow Summit 2020 presentation: Run Airflow DAGs in a secure way.

What's next