Security best practices

Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3

This page describes one possible approach to organizing security for a team that works with a Cloud Composer environment.

An example of an Airflow CI/CD pipeline
Figure 1. An example of an Airflow CI/CD pipeline (click to enlarge)

Cloud Composer provides several security features that you can use when working with Airflow in a Cloud Composer environment. In addition to access control with Identity and Access Management and Airflow UI Access Control, you can set up a workflow for your team that prevents accidental modification of both environment's configuration and DAG code:

  1. Create your environment using Terraform. In this way, you can store environment's configuration as code in a repository.

  2. Assign IAM roles, so that only administrators can access environment's bucket and environment's cluster, and direct access is disabled for regular users. For example, the Composer User role enables access only to DAG UI and Airflow UI.

  3. Deploy DAGs in your environment with a CI/CD pipeline, so that DAG code is retrieved from a repository. In this way, DAGs are reviewed and approved before the changes are merged to the version control system. During the review process, approvers make sure that DAGs meet the security criteria established within their teams. The review step is critical to prevent deployment of DAGs modifying the content of the environment's bucket.

What's next