How-to guides

Authentication and access control

• Authentication

  Setting up authentication for a Cloud Composer environment

• Access control

  Controlling access to a Cloud Composer environment

Creating and configuring environments

• Enabling and disabling Cloud Composer

  Enabling the Cloud Composer API

• Configuring shared VPC

  Configuring a host project and networking resources for Cloud Composer

• Configuring private IP

  Enabling private IP in a Cloud Composer environment

• Creating environments

  Creating a new Cloud Composer environment

• Setting environment variables

  Requirements for setting environment variables

• Managing connections

  Using Airflow connections to access resources in a Google Cloud project

• Using the KubernetesPodOperator

  Launching GKE pods from Cloud Composer into an environment's GKE cluster

Accessing environments

• Airflow command-line interface

  Using the Apache Airflow command-line (CLI) interface with a Cloud Composer environment

• Airflow web interface

  Connecting to the Apache Airflow web interface for a Cloud Composer environment

Managing workflows

• Writing DAGs

  Writing DAGs (workflows) for a Cloud Composer environment

• Adding and updating DAGs

  Managing DAGs (workflows) in a Cloud Composer environment

• Installing custom plugins

  Installing Plugins in a Cloud Composer environment

• Installing Python dependencies

  Installing Python Package Index dependencies in a Cloud Composer environment

• Triggering DAGs

  Triggering DAGs in response to events

• Testing DAGs

  Testing DAGs locally

• Troubleshooting DAGs

  Troubleshooting common workflow issues

Monitoring environments

• Viewing Airflow logs

  Viewing Airflow task logs for a Cloud Composer environment

• Viewing audit logs

  Viewing audit logs for a Cloud Composer environment

• Monitoring environments in Stackdriver

  Using Stackdriver to monitor a Cloud Composer environment

Updating environments

• Updating and deleting environments

  Updating or deleting a Cloud Composer environment

• Upgrading environments

  Upgrade an existing Cloud Composer environment to a new version