This page lists the Apache Airflow configuration values associated with a Cloud Composer environment image version that are blocked and can't be overridden.
Section and Configuration | Notes |
---|---|
api-* | The API config section is blocked. |
celery-* | Cloud Composer configures Airflow to use Celery executor. In composer-0.5.x-airflow-1.9.0, the Celery config section is blocked. In composer-1.0.0-airflow-1.9.0, the following celery properties are blocked: In composer-1.3.0-airflow-1.10.0, the following celery properties are blocked: In composer-1.4.2-airflow-1.10.0, the following celery properties are blocked: Worker pods might require a restart for celery-related configurations to take effect. |
cli-* | The CLI config section is blocked. |
core-airflow_home | |
core-base_log_folder | |
core-dag_concurrency | DAG concurrency automatically scales to the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
core-dags_folder | To allow Cloud Composer to upload DAGs, the dags folder must remain in the Google Cloud Storage bucket. |
core-donot_pickle | Pickling poses security risks and has been disabled. |
core-enable_xcom_pickling | Pickling poses security risks and has been disabled. |
core-encrypt_s3_logs | Cannot be changed. Required for Cloud Storage logging. |
core-executor | Cloud Composer configures Airflow to use the Celery executor. |
core-fernet_key | Not supported yet in Cloud Composer as it may cause Airflow database encryption readback error. |
core-logging_config_class | Overriding has no effect. The logs folder must remain in the Cloud Storage bucket. |
core-max_active_runs_per_dag | Max active runs per DAG automatically scales with the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
core-parallelism | Parallelism automatically scales to the number of nodes. Since composer-1.1.1-airflow-1.9.0, this property is updatable. |
core-plugins_folder | To be able to upload plugins through Cloud Composer, the dags folder must remain in the Cloud Storage bucket. |
core-remote_base_log_folder | The logs folder must remain in the Cloud Storage bucket. |
core-remote_log_conn_id | Cannot be changed. Required for Cloud Storage logging. |
core-security | Security is managed by your Cloud Composer environment. |
core-sql_alchemy_conn | Changing sql alchemy settings can break the connection to the database. |
core-sql_alchemy_pool_recycle | Changing sql alchemy settings can break the connection to the database. |
core-sql_alchemy_pool_size | Changing sql alchemy settings can break the connection to the database. |
core-task_log_reader | |
smtp-smtp_password | You can set an SMTP password in a more secure way. |
scheduler-authenticate | Cloud Composer configures the Airflow scheduler. The following scheduler properties are updatable: catchup_by_default , dag_dir_list_interval , max_tis_per_query , min_file_process_interval , scheduler_zombie_task_threshold , and all properties starting with statsd (statsd_* ). |
scheduler-auth_backend | |
scheduler-child_process_log_directory | |
scheduler-job_heartbeat_sec | |
scheduler-max_threads | Since composer-1.1.1-airflow-1.9.0 , this property is updatable. |
scheduler-print_stats_interval | |
scheduler-run_duration | |
scheduler-scheduler_heartbeat_sec | |
webserver-access_logfile | Cloud Composer configures the Airflow webserver. The following webserver properties are updatable: dag_default_view , dag_orientation , demo_mode , expose_config , hide_paused_dags_by_default , and page_size . |
webserver-authenticate | |
webserver-auth_backend | |
webserver-base_url | |
webserver-error_logfile | |
webserver-filter_by_owner | |
webserver-log_fetch_timeout_sec | |
webserver-owner_mode | |
webserver-secret_key | |
webserver-web_server_host | |
webserver-web_server_port | |
webserver-web_server_ssl_cert | |
webserver-web_server_ssl_key | |
webserver-web_server_worker_timeout | |
webserver-worker_refresh_batch_size | |
webserver-worker_refresh_interval | Since composer-1.7.1-airflow-1.10.2, this property is updatable. The recommended value is 3600 when webserver-async_dagbag_loader is True . |
webserver-workers | Since composer-1.7.2-airflow-1.10.2, this property is updatable. If your environment contains approximately 1000 DAG files or more, the recommended value is 1 . |