Apache Airflow includes a command-line interface (CLI) that you can use to manage Airflow environments. The CLI is useful for tasks such as managing workflows, changing the Airflow environment, and obtaining log information.
Cloud Composer simplifies Airflow CLI commands with the Google Cloud SDK.
Once installed, you can use the gcloud
command line tool to run
the gcloud composer environments run
command to execute
Airflow CLI sub-commands.
Before you begin
The following permissions are required to access the gcloud
command-line tool (Airflow CLI)
in the Cloud Composer environment:
composer.environments.get
container.clusters.get
container.clusters.list
container.clusters.getCredentials
For more information, see Cloud Composer Access Control.
Limitations
Do not use the Airflow resetdb
command. The command clears the database,
including the connections that Cloud Composer sets.
Running Airflow CLI commands
You can run Airflow CLI commands on a Cloud Composer environment by using the following command:
gcloud composer environments run ENVIRONMENT_NAME \ --location LOCATION SUBCOMMAND
Most gcloud composer
commands require a location. You can specify the location by
using the --location
flag or by setting the default location.
For example, to trigger a DAG named SOME_DAG
with the ID FOO
in your Cloud Composer environment:
gcloud composer environments run ENVIRONMENT_NAME \ --location LOCATION \ trigger_dag -- SOME_DAG \ --run_id=FOO