Specify maintenance windows

Cloud Composer 1 | Cloud Composer 2

This page explains how to define maintenance windows for your environments.

How to use maintenance windows

Maintenance operations might impact the execution of your DAGs and Airflow tasks, so we recommend you to do the following:

  1. Define maintenance windows for your Cloud Composer environments.

  2. Schedule DAG runs outside of specified maintenance windows by using start_date and schedule_interval parameters in your DAGs.

You can define maintenance windows for new and existing Cloud Composer environments. If you don't specify explicit maintenance windows, then time for maintenance operations is selected automatically without considering the schedule of your DAG runs.

By default, maintenance windows are from 00:00:00 to 04:00:00 (GMT) on Sunday, Friday, and Saturday every week.

Specify maintenance windows for new environments

You can specify maintenance windows when you create an environment. For more information, see Create environments.

Specify maintenance windows for existing environments

Console

To define or change maintenance windows for an existing environment, update the environment:

  1. In the Google Cloud Console, go to the Environments page.

    Go to Environments

  2. Select your environment.

  3. Go to the Environment configuration tab.

  4. Next to the Maintenance windows entry, click Edit.

  5. In the Maintenance windows dialog, select the Set custom time for maintenance windows checkbox.

  6. Set Start time, Timezone, Days, and Length, so that combined time for the specified schedule is at least 12 hours in a 7-day rolling window. For example, a period of 4 hours every Monday, Wednesday, and Friday provides the required amount of time.

  7. Click Save and wait until your environment updates.

gcloud

When you update an environment, the following arguments define maintenance windows parameters:

  • --maintenance-window-start sets the start time of a maintenance window.
  • --maintenance-window-end sets the end time of a maintenance window.
  • --maintenance-window-recurrence sets the maintenance window recurrence.
gcloud beta composer environments update ENVIRONMENT_NAME \
    --location LOCATION \
    --maintenance-window-start 'DATETIME_START' \
    --maintenance-window-end 'DATETIME_END' \
    --maintenance-window-recurrence 'MAINTENANCE_RECURRENCE'

Replace:

  • ENVIRONMENT_NAME with the name of the environment.
  • DATETIME_START with the start date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored.
  • DATETIME_END with the end date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored. The specified date and time must be after the start date.
  • MAINTENANCE_RECURRENCE with an RFC 5545 RRULE for maintenance windows recurrence. Cloud Composer supports two formats:

  • The FREQ=DAILY format specifies a daily recurrence.

  • The FREQ=WEEKLY;BYDAY=SU,MO,TU,WE,TH,FR,SA format specifies a recurrence on selected days of the week.

The following example specifies a 6-hour maintenance window between 01:00 and 07:00 (UTC) on Sundays, Wednesdays, and Saturdays. The 1 January, 2021 date is ignored.

gcloud beta composer environments update example-environment \
  --location us-central1 \
  --maintenance-window-start '2021-01-01T01:00:00Z' \
  --maintenance-window-end '2021-01-01T07:00:00Z' \
  --maintenance-window-recurrence 'FREQ=WEEKLY;BYDAY=SU,WE,SA'

API

  1. Construct an environments.patch API request.

  2. In this request:

    1. In the updateMask parameter, specify the config.maintenanceWindow mask.

    2. In the request body, specify the parameters for maintenance windows.

{
  "config": {
    "maintenanceWindow": {
      "startTime": "DATETIME_START",
      "endTime": "DATETIME_END",
      "recurrence": "MAINTENANCE_RECURRENCE"
    }
  }
}

Replace:

  • DATETIME_START with the start date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored.
  • DATETIME_END with the end date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored. The specified date and time must be after the start date.
  • MAINTENANCE_RECURRENCE with an RFC 5545 RRULE for maintenance windows recurrence. Cloud Composer supports two formats:

    • The FREQ=DAILY format specifies a daily recurrence.
    • The FREQ=WEEKLY;BYDAY=SU,MO,TU,WE,TH,FR,SA format specifies a recurrence on selected days of the week.

The following example specifies a 6-hour maintenance window between 01:00 and 07:00 (UTC) on Sundays, Wednesdays, and Saturdays. The 1 January, 2021 date is ignored.

  // PATCH https://composer.googleapis.com/v1beta1/projects/example-project/
  // locations/us-central1/environments/example-environment?updateMask=
  // config.maintenanceWindow

  {
    "config": {
      "maintenanceWindow": {
        "startTime": "2021-01-01T01:00:00Z",
        "endTime": "2021-01-01T07:00:00Z",
        "recurrence": "FREQ=WEEKLY;BYDAY=SU,WE,SA"
      }
    }
  }

Terraform

The maintenance_window block specifies the maintenance windows for your environment:

resource "google_composer_environment" "example" {
  provider = google-beta
  name = "ENVIRONMENT_NAME"
  region = "LOCATION"

  config {
    maintenance_window {
      start_time = "DATETIME_START"
      end_time = "DATETIME_END"
      recurrence = "MAINTENANCE_RECURRENCE"
    }
  }
}

Replace:

  • ENVIRONMENT_NAME with the name of the environment.
  • LOCATION with the Compute Engine region where the environment is located.
  • DATETIME_START with the start date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored.
  • DATETIME_END with the end date and time in the date/time input format. Only the specified time of the day is used, the specified date is ignored. The specified date and time must be after the start date.
  • MAINTENANCE_RECURRENCE with an RFC 5545 RRULE for maintenance windows recurrence. Cloud Composer supports two formats:

    • The FREQ=DAILY format specifies a daily recurrence.
    • The FREQ=WEEKLY;BYDAY=SU,MO,TU,WE,TH,FR,SA format specifies a recurrence on selected days of the week.

The following example specifies a 6-hour maintenance window between 01:00 and 07:00 (UTC) on Sundays, Wednesdays, and Saturdays. The 1 January, 2021 date is ignored.

resource "google_composer_environment" "example" {
  provider = google-beta
  name = "example-environment"
  region = "us-central1"

  config {
    maintenance_window {
      start_time = "2021-01-01T01:00:00Z"
      end_time = "2021-01-01T07:00:00Z"
      recurrence = "FREQ=WEEKLY;BYDAY=SU,WE,SA"
    }
  }
}

What's next