Configure maintenance windows

This page explains how to define maintenance windows for Cloud Data Fusion instances.

Maintenance windows are supported for Cloud Data Fusion instances in versions 6.8.0 and later.

About maintenance windows

A maintenance window in Cloud Data Fusion is a time period where you permit it to perform maintenance operations, reducing the risk of service disruptions during peak usage times. Scheduling maintenance windows during off-peak hours, or times when your activity is lower, minimizes their impact on your Cloud Data Fusion instances and your pipeline executions.

Configuring maintenance windows lets you control when maintenance operations occur in your instance:

  • If the maintenance window is defined, maintenance operations are performed by Cloud Data Fusion within the defined periods.
  • If the maintenance window isn't defined, maintenance can be performed at any time.

About maintenance operations

To ensure system security and compliance, Cloud Data Fusion conducts regular maintenance operations. These operations include applying critical updates and security patches to mitigate vulnerabilities and protect user data. Scheduled maintenance windows ensure that the updates are efficiently applied, preserving system integrity and minimizing disruption to instances and pipeline executions.

  • When you configure the maintenance windows, you must allocate at least 12 hours per week for maintenance operations. Each window must be between 4 and 12 hours.

  • During the maintenance window hours, certain operations might be impacted. We recommend having maintenance windows during non-peak usage times. The maintenance operation can affect Cloud Data Fusion in following ways:

    • The Cloud Data Fusion Studio interface might be temporarily unavailable.
    • Pipeline execution might be impacted, causing failure.
    • Cloud Data Fusion services might be down temporarily, causing API calls to fail.
    • Instances might not be fully functional for up to an hour during maintenance window hours. We recommend that you set up maintenance windows when your pipelines either aren't running or aren't scheduled to run.

Specify maintenance windows while creating an instance

Console

To set the maintenance window when you create an instance, follow these steps:

  1. Go to the Cloud Data Fusion Instances page, and click Create instance.

    Create an instance

  2. Enter an Instance name and Edition. For more information about each field, see Create an instance.
  3. Expand the Advanced options section.
  4. Expand the Maintenance section and follow these steps:
    1. Select the Enable maintenance window checkbox.
    2. In the Start time field, enter a time in a supported format.
    3. In the TimeZone list, select a time zone.
    4. In the Days and Length lists, select days of the week and duration of window. For more information about these settings, see About maintenance operations.
    5. Click Create.

For more information, see Configure maintenance window times and recurrence (RRULE).

REST API

  1. Set the Instance configuration using the instance.create API:
{
  "name": string,
  "description": string,
  …
  …
  "maintenancePolicy": {
    "maintenanceWindow": {
      "recurringTimeWindow": {
        "window": {
            "startTime": "START_TIME",
            "endTime": "END_TIME"
        },
        "recurrence": "MAINTENANCE_RECURRENCE"
      }
    },
  }
}

Replace the following:

  • START_TIME: time window start time, in the RFC 3339 format.
  • END_TIME: time window end time, in the RFC 3339 format.

  • MAINTENANCE_RECURRENCE: an RRULE for the maintenance window's recurrence. Cloud Data Fusion only supports the following formats:

    • FREQ=WEEKLY;BYDAY=SU,MO,TU,WE,TH,FR,SA or BYDAY=SU,MO,TU,WE,TH,FR,SA,MO;FREQ=WEEKLY. This specifies the weekly recurrence of the window on the selected days, starting at the startTime, and for the specified duration.

For more information, see Configure maintenance window times and recurrence (RRULE).

The following example creates an instance with a six-hour maintenance window between 10 AM to 4 PM (UTC) on Saturday and Sunday every week. For more information, see the instances API documentation.

// POST https://datafusion.googleapis.com/v1/projects/example-project/locations/us-central1/instances?instanceID=example-instance

{
  "maintenancePolicy": {
      "maintenanceWindow": {
          "recurringTimeWindow": {
              "window": {
                  "startTime": "2024-01-01T10:00:00Z",
                  "endTime": "2024-01-01T16:00:00Z",
              },
              "recurrence": "FREQ=WEEKLY;BYDAY=SA,SU"
          }
      },
  }
}

Update the maintenance windows for an instance

Console

To update the maintenance window in an existing instance in the Google Cloud console, follow these steps:

  1. Go to the Cloud Data Fusion Instances page.

    Go to Instances

  2. Click the instance name to view the instance details.
  3. On the Instance details page, navigate to the Maintenance window field and click Edit.
  4. Maintenance window fields are opened in the edit mode. Update the values, as needed:
    1. Select the Enable maintenance window checkbox.
    2. In the Start time field, enter a time in a supported format.
    3. In the TimeZone list, select a time zone.
    4. In the Days and Length lists, select days of week and duration of window. For more information about these settings, see About maintenance operations.
  5. Click Save.

For more information, see Configure maintenance window times and recurrence (RRULE).

REST API

For existing instances, update the maintenance windows using the instances.patch API request. This overrides the existing maintenance window configuration, if present. Otherwise, a new configuration is set, as mentioned in the request body.

Construct an API request as follows:

  1. In the updateMask parameter, specify the maintenance_policy mask.
  2. In the request body, specify the maintenance window in the maintenance policy configuration:
{
  "maintenancePolicy": {
    "maintenanceWindow": {
        "recurringTimeWindow": {
          "window": {
              "startTime": "START_TIME",
              "endTime": "END_TIME"
          },
          "recurrence": "MAINTENANCE_RECURRENCE"
        }
    },
  }
}

For more information, see Configure maintenance window times and recurrence (RRULE).

The following example updates the maintenance policy configuration of the instance with a four-hour maintenance window between 11 AM to 3 PM (UTC) on Friday, Saturday, and Sunday every week.

// PATCH https://datafusion.googleapis.com/v1/projects/example-project/locations/us-central1/instances/example-instance?updateMask=maintenance_policy

{
"maintenancePolicy": {
  "maintenanceWindow": {
    "recurringTimeWindow": {
      "window": {
          "startTime": "2024-01-01T11:00:00Z",
          "endTime": "2024-01-01T15:00:00Z",
        },
      "recurrence": "FREQ=WEEKLY;BYDAY=FR,SA,SU"
    }
  },
}
}

Configure maintenance window times and recurrence (RRULE)

When you configure start and end times for the maintenance window, the date components of the provided timestamps are ignored. Only the time of the day and duration between start and end time are considered. Times are stored in UTC, but you can use UTC or your time zone offset when scheduling maintenance windows using the API.

When you configure recurrence, keep in mind that RRULE follows UTC days, so adjust accordingly if your time zone differs. For example, if your time zone is PDT (UTC-7) and you want a maintenance window to start at 21:00 on Wednesdays in your time zone (which corresponds to 4:00 UTC on Thursdays), specify TH instead of WE in the BYDAY option of the RRULE.

Delete the maintenance window for an instance

Console

To delete the maintenance window in an existing instance in the Google Cloud console, follow these steps:

  1. Go to the Cloud Data Fusion Instances page.

    Go to Instances

  2. Click the instance name to view the instance details.
  3. On the Instance details page, navigate to the Maintenance window field and click Edit.
  4. To remove the maintenance window from the instance, clear the Enable maintenance window checkbox.
  5. Click Save.

REST API

To remove the maintenance window from an instance, pass an empty maintenance policy object in the request body.

// PATCH https://datafusion.googleapis.com/v1/projects/example-project/locations/us-central1/instances/example-instance?updateMask=maintenance_policy

{
    "maintenancePolicy": {}
}

What's next