This page describes the requirements for setting environment variables in Cloud Composer.
When you create or update an environment, you can add environment
variables, which Cloud Composer provides to the
Apache Airflow scheduler, worker, and webserver processes. For example, Cloud Composer
uses the Apache Airflow sendgrid module for email
notifications. To configure email notifications,
you need to set the SENDGRID_API_KEY
and SENDGRID_MAIL_FROM
environment variables.
Requirements
Environment variables must match the regular expression [a-zA-Z_][a-zA-Z0-9_]*
.
Environment variables must not specify Apache Airflow software configuration
overrides. This means the variables cannot not use the
format $AIRFLOW__{SECTION}__{KEY}
or match the regular
expression
.AIRFLOW__[A-Z0-9_]+__[A-Z0-9_]+
)
Reserved names
The following names are reserved and cannot be used:
AIRFLOW_HOME
C_FORCE_ROOT
COMPOSER_ENVIRONMENT
COMPOSER_GKE_NAME
COMPOSER_GKE_ZONE
COMPOSER_LOCATION
COMPOSER_PYTHON_VERSION
CONTAINER_NAME
DAGS_FOLDER
GCP_PROJECT
GCS_BUCKET
GKE_CLUSTER_NAME
GOOGLE_APPLICATION_CREDENTIALS
SQL_DATABASE
SQL_INSTANCE
SQL_PASSWORD
SQL_PROJECT
SQL_REGION
SQL_USER
Before you begin
The following permissions are required to set environment variables:
composer.environments.create
to set environment variables during environment creationcomposer.environments.update
to update environment variablesFor more information, see Cloud Composer Access Control.
Most
gcloud composer
commands require a location. You can specify the location by using the--location
flag or by setting the default location.
Adding and updating environment variables
To set environment variables:
Console
To add an environment variable:
- Open the Create Environment page in the Google Cloud Console.
Under Node configuration, click Add environment variable.
Enter the
NAME
andVALUE
for the environment variables.
gcloud
gcloud composer environments update ENVIRONMENT_NAME \ --location LOCATION \ --update-env-variables=KEY=VALUE,KEY=VALUE...
where:
ENVIRONMENT_NAME
is the name of the environment.LOCATION
is the Compute Engine region where the environment is located.KEY=VALUE
specifies the environment variables to create or update.
For example:
gcloud composer environments update test-environment
--location us-central1
--update-env-variables=SENDGRID_MAIL_FROM=updated-email@domain.com
API
To use the Cloud Composer REST API,
issue an environments.patch
.
Removing environment variables
To remove environment variables:
Console
- Open the Environments page.
- On the List Environments page, click the name of the environment to view its details.
- Under Environment Variables, click Edit
- To the right of the variable value, click the trash can icon.
gcloud
To remove all environment variables:
gcloud composer environments update ENVIRONMENT_NAME
--location LOCATION
--clear-env-variables
where:
ENVIRONMENT_NAME
is the name of the environmentLOCATION
is the Compute Engine region where the environment is located.
For example:
gcloud composer environments update test-environment
--location us-central1
--clear-env-variables
To remove a list of environment variables:
gcloud composer environments update ENVIRONMENT_NAMEwhere:
--location LOCATION
--remove-env-variables=NAME,NAME,...
ENVIRONMENT_NAME
is the name of the environment,LOCATION
is the Compute Engine region where the environment is located.NAME
is the environment variable to remove.
API
To use the Cloud Composer REST API,
issue an environments.patch
.