This page lists the Apache Airflow configuration values associated with a Cloud Composer environment image version that are blocked and can't be overridden.
| Section and Configuration | Notes |
|---|---|
| api-* | The API config section is blocked. |
| celery-* | Cloud Composer configures Airflow to use Celery executor. In composer-0.5.x-airflow-1.9.0, the Celery config section is blocked. In composer-1.0.0-airflow-1.9.0, the following celery properties are blocked: In composer-1.3.0-airflow-1.10.0, the following celery properties are blocked: In composer-1.4.2-airflow-1.10.0, the following celery properties are blocked: Worker pods might require a restart for celery-related configurations to take effect. |
| cli-* | The CLI config section is blocked. |
| core-airflow_home | |
| core-base_log_folder | |
| core-dag_concurrency | DAG concurrency automatically scales to the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
| core-dags_folder | To allow Cloud Composer to upload DAGs, the dags folder must remain in the Google Cloud Storage bucket. |
| core-donot_pickle | Pickling poses security risks and has been disabled. |
| core-enable_xcom_pickling | Pickling poses security risks and has been disabled. |
| core-encrypt_s3_logs | Cannot be changed. Required for Cloud Storage logging. |
| core-executor | Cloud Composer configures Airflow to use the Celery executor. |
| core-fernet_key | Not supported yet in Cloud Composer as it may cause Airflow database encryption readback error. |
| core-logging_config_class | Overriding has no effect. The logs folder must remain in the Cloud Storage bucket. |
| core-max_active_runs_per_dag | Max active runs per DAG automatically scales with the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
| core-parallelism | Parallelism automatically scales to the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
| core-plugins_folder | To be able to upload plugins through Cloud Composer, the dags folder must remain in the Cloud Storage bucket. |
| core-remote_base_log_folder | The logs folder must remain in the Cloud Storage bucket. |
| core-remote_log_conn_id | Cannot be changed. Required for Cloud Storage logging. |
| core-sql_alchemy_conn | Changing sql alchemy settings can break the connection to the database. |
| core-sql_alchemy_pool_recycle | Changing sql alchemy settings can break the connection to the database. |
| core-sql_alchemy_pool_size | Changing sql alchemy settings can break the connection to the database. |
| core-task_log_reader | |
| smtp-smtp_password | |
| scheduler-authenticate | Cloud Composer configures the Airflow scheduler. The following scheduler properties are updatable: catchup_by_default, dag_dir_list_interval, max_tis_per_query, min_file_process_interval, scheduler_zombie_task_threshold, and all properties starting with statsd (statsd_*). |
| scheduler-child_process_log_directory | |
| scheduler-job_heartbeat_sec | |
| scheduler-max_threads | Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
| scheduler-print_stats_interval | |
| scheduler-run_duration | |
| scheduler-scheduler_heartbeat_sec | |
| webserver-access_logfile | Cloud Composer configures the Airflow webserver. The following webserver properties are updatable: dag_default_view, dag_orientation, demo_mode, expose_config, hide_paused_dags_by_default, and page_size. |
| webserver-authenticate | |
| webserver-base_url | |
| webserver-error_logfile | |
| webserver-filter_by_owner | |
| webserver-log_fetch_timeout_sec | |
| webserver-owner_mode | |
| webserver-secret_key | |
| webserver-web_server_host | |
| webserver-web_server_port | |
| webserver-web_server_ssl_cert | |
| webserver-web_server_ssl_key | |
| webserver-web_server_worker_timeout | |
| webserver-worker_refresh_batch_size | |
| webserver-worker_refresh_interval | Since composer-1.7.1-airflow-1.10.2, this property is updatable. The recommended value is 3600 when webserver-async_dagbag_loader is True. |
| webserver-workers | Since composer-1.7.2-airflow-1.10.2, this property is updatable. If your environment contains approximately 1000 DAG files or more, the recommended value is 1. |
