Updating and Deleting Cloud Composer Environments

This page explains how to update, delete, and view the details of a Cloud Composer environment. It also explains how to update and view an environment's Apache Airflow configurations.

Before you begin

• The following permissions are required to update or delete environments:

  • composer.environments.update to update environments

  • composer.environments.delete to delete environments

    For more information, see Cloud Composer Access Control.

• Most gcloud composer commands require a location. You can specify the location by using the --location flag or by setting the default location.

• Some Airflow parameters are preconfigured for Cloud Composer, and you cannot change them. You can update the following Cloud Composer environment configurations: node count, Airflow config overrides, environment variables, labels, and PyPI packages.

• Updates to Airflow configurations take effect on all Airflow instances (worker, server, scheduler) approximately 5 minutes after you submit the update request.

Limitations

Cloud Composer doesn't accept concurrent environment update or delete requests. You must wait for an environment update or delete request to complete before issuing another environment update or delete request.

Updating environments

To update Cloud Composer environments, including Airflow configurations:

Console

1. Open the Environments page in the Google Cloud Platform.

   Open the Environments page

2. Click the environment Name to modify.

3. Select the tab corresponding to the property you want to update.

4. Click Edit.

5. After making updates, click Submit.

gcloud

To update Airflow configurations, run the gcloud composer environments update command with the --update-airflow-configs flag:

gcloud composer environments update ENVIRONMENT_NAME \
    --location LOCATION \
    --update-airflow-configs=KEY=VALUE,KEY=VALUE,... 

where:

• ENVIRONMENT_NAME is the name of the environment.
• LOCATION is the Compute Engine region where the environment is located.
• KEY=VALUE is the configuration section and the property name separated by a hyphen, such as core-print_stats_interval, and its corresponding value.

For example:

gcloud composer environments update test-environment \
    --location us-central1 \
    --update-airflow-configs=core-load_example=True,webserver-dag_orientation=TB 

See the 'gcloud composer environments update' reference page for additional examples.

API

To update a Cloud Composer environment, including Airflow configurations, using the Cloud Composer REST API, issue an environments.patch API request.

Restoring the default Airflow configuration

To restore all default Airflow configuration values:

gcloud composer environments update ENVIRONMENT_NAME \
    --location LOCATION \
    --clear-airflow-configs

To restore one or more default Airflow configuration values:

gcloud composer environments update ENVIRONMENT_NAME \
    --location LOCATION \
    --remove-airflow-configs=core-load_example,webserver-dag_orientation 

where:

• ENVIRONMENT_NAME is the name of the environment.
• LOCATION is the Compute Engine region where the environment is located.

For example:

gcloud composer environments update test-environment \
    --location us-central1 \
    --update-airflow-configs=core-load_example=True,webserver-dag_orientation=TB 

The command terminates when the operation is finished. To avoid waiting, use the --async flag.

Viewing environment details

To view Cloud Composer environment details, including Airflow configurations:

Console

1. Open the Environments page in the GCP Console.

   Open the Environments page

2. Click the Name of the environment.
3. Select the appropriate property tab.

gcloud

gcloud composer environments describe ENVIRONMENT_NAME \
    --location LOCATION
    --format="get(PROPERTY)"

where:

• ENVIRONMENT_NAME is the name of the environment.
• LOCATION is the Compute Engine region where the environment is located.
• --format is an option to specify an environment property, such as config.airflowUri for the Airflow web server URL.

For example:

gcloud composer environments describe test-environment 

  --location us-central1 --format="get(config.airflowUri)"


# Returns https://a6b3z6e3nnZZZnnd3a-tp.appspot.com 

API

To view Airflow configurations using the Cloud Composer REST API, make an environments.get API request, filling in the environment ID.

Deleting an environment

To delete a Cloud Composer environment:

Console

1. Open the Environments page in the GCP Console.

   Open the Environments page

2. Check the box next to the environment to delete.
3. Click Delete.

gcloud

Run the following command to delete a Cloud Composer environment:

gcloud composer environments delete ENVIRONMENT_NAME \
    --location LOCATION   

where:

• ENVIRONMENT_NAME is the name of the environment.
• LOCATION is the Compute Engine region where the environment is located.

Run gcloud composer environments describe --help for command parameter information.

API

To delete an environment using the Cloud Composer REST API, make an environments.delete API request, filling in the environment ID.

Note: Deleting your environment does not delete the storage bucket for the environment. To avoid incurring charges to your Google Cloud Platform account, download your data and then delete the Cloud Storage bucket.

What's next

• Installing Python Dependencies