Cloud Composer 1 | Cloud Composer 2 | Cloud Composer 3
Follow the instructions on this page to override Airflow configuration options for new and existing environments.
When you create or update an environment, you can override Apache Airflow configuration options with different values. By doing so you can adjust the Airflow instance to your needs and requirements.
Override Airflow configuration options for new environments
You can override Airflow configuration options when you create your environment. For more information, see Create environments.
Override Airflow configuration options for existing environments
Console
To override Airflow configuration options for an existing environment:
In Google Cloud console, go to the Environments page.
In the list of environments, click the name of your environment. The Environment details page opens.
Go to the Airflow configuration overrides tab.
Click Edit.
Enter the Section, Key, and Value for the Airflow configuration option that you want to change.
For example:
Section | Key | Value |
---|---|---|
webserver |
dag_orientation |
RL |
gcloud
Following arguments override Airflow configuration options for an existing environment:
--update-airflow-configs
adds or changes specified Airflow configuration overrides.--remove-airflow-configs
removes specified Airflow configuration overrides.--clear-airflow-configs
removes all Airflow configuration overrides.
gcloud composer environments update ENVIRONMENT_NAME \
--location LOCATION \
--update-airflow-configs=KEY=VALUE,KEY=VALUE,...
Replace:
ENVIRONMENT_NAME
with the name of the environment.LOCATION
with the region where the environment is located.KEY
with the configuration section and the option name separated by a hyphen, for example,webserver-dag_orientation
.VALUE
with the corresponding value for an option.
For example:
gcloud composer environments update example-environment \
--location us-central1 \
--update-airflow-configs=webserver-dag_default_view=graph,webserver-dag_orientation=RL
API
To override Airflow properties for an existing environment:
Construct an
environments.patch
API request.In this request:
To replace all existing Airflow configuration overrides with the specified options, in the
updateMask
parameter, specify theconfig.softwareConfig.airflowConfigOverrides
mask.To override a specific Airflow configuration option, in the
updateMask
parameter, specify theconfig.softwareConfig.airflowConfigOverrides.KEY
mask. ReplaceKEY
with the configuration section and the option name separated by a hyphen, for example,webserver-dag_orientation
.If you want to override several Airflow configuration options, in the
updateMask
parameter, specify several masks separated by commas.
The request body must contain the list of Airflow configuration options. If you are replacing all existing overrides, make sure to include all overrides that you want to keep.
{
"config": {
"softwareConfig": {
"airflowConfigOverrides": {
"KEY": "VALUE"
}
}
}
}
Replace:
KEY
with the configuration section and the option name separated by a hyphen, for example,webserver-dag_orientation
.VALUE
with the corresponding value for an option.
The following example overrides two specific Airflow configuration options:
// PATCH https://composer.googleapis.com/v1/projects/example-project/
// locations/us-central1/environments/example-environment?updateMask=
// config.softwareConfig.airflowConfigOverrides.webserver-dag_default_view,
// config.softwareConfig.airflowConfigOverrides.webserver-dag_orientation
{
"config": {
"softwareConfig": {
"airflowConfigOverrides": {
"webserver-dag_default_view": "graph",
"webserver-dag_orientation": "RL"
}
}
}
}
Terraform
The airflow_config_overrides
block in the software_config
block
controls Airflow configuration overrides for your environment:
resource "google_composer_environment" "example" {
config {
# ... Other environment configuration parameters
software_config {
airflow_config_overrides = {
KEY = "VALUE"
}
}
}
}
Replace:
KEY
with the configuration section and the option name separated by a hyphen, for example,webserver-dag_orientation
.VALUE
with the corresponding value for an option.
Example:
resource "google_composer_environment" "example" {
provider = google-beta
name = "example-environment"
region = "us-central1"
# ... Other environment configuration parameters
config {
software_config {
airflow_config_overrides = {
webserver-dag_default_view = "graph"
webserver-dag_orientation = "RL"
}
}
}
}