Airflow Command-line Interface

Apache Airflow includes a command-line interface (CLI) that you can use to manage Airflow environments. The CLI is useful for tasks such as managing workflows, changing the Airflow environment, and obtaining log information.

Cloud Composer simplifies Airflow CLI commands with the Google Cloud SDK. Once installed, you can use the gcloud command line tool to run the gcloud composer environments run command to execute Airflow CLI sub-commands.

Before you begin

The following permissions are required to access the gcloud command-line tool (Airflow CLI) in the Cloud Composer environment:

• composer.environments.get
• container.clusters.get
• container.clusters.list
• container.clusters.getCredentials

For more information, see Cloud Composer Access Control.

Running Airflow CLI commands

You can run Airflow CLI commands on a Cloud Composer environment by using the following command:

gcloud composer environments run ENVIRONMENT_NAME \
    --location LOCATION SUBCOMMAND

Most gcloud composer commands require a location. You can specify the location by using the --location flag or by setting the default location.

For example, to trigger a DAG named sample_quickstart with the ID 5077 in your Cloud Composer environment:

gcloud composer environments run test-environment \
    --location us-central1 trigger_dag -- sample_quickstart \
    --run_id=1234

For example, to check for syntax errors in DAGs in a test/ directory:

gcloud composer environments run test-environment \
     --location us-central1 \
     list_dags -- -sd /home/airflow/gcs/data/test

Supported Airflow commands

Cloud Composer supports the following commands for environments running Airflow 1.9.x and Airflow 1.10.x:

• backfill
• clear
• connections
• dag_state
• delete_dag
• kerberos
• list_dags
• list_tasks
• pause
• pool
• render
• run
• task_failed_deps
• task_state
• test
• trigger_dag
• unpause
• variables
• version