Enabling and disabling high availability on an instance

MySQL   |  PostgreSQL   |  SQL Server

This page describes how to configure a Second Generation instance for high availability.

You can configure an instance for high availability when you create the instance, or you can enable high availability on an existing instance.

For more information about the high availability configuration, see Overview of the High Availability Configuration.

Creating a new instance configured for high availability

When you create an instance, and you configure it for high availability, Cloud SQL creates the instance as a regional instance.

To create an instance configured for high availability:

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Create an instance.
3. On the Instance ID page, select Show configuration options.
4. Open Auto backups and high availability.
5. Select the High availability (regional) checkbox.
   Leave the Automate backups and Enable binary logging checkboxes enabled.

gcloud

1. Create the regional instance:

   gcloud sql instances create [REGIONAL_INSTANCE_NAME] \
                        --availability-type=REGIONAL \
                        --database-version=[DATABASE_VERSION] \
                        --tier=[MACHINE_TYPE]
   

   For a complete list of available parameters, see the gcloud sql instances create reference page.
2. Configure the root user:

   gcloud sql users set-password root %\
              --instance=[MASTER_INSTANCE_NAME]
              --password=[PASSWORD]
   

For a complete list of available parameters, see the gcloud sql instances create reference page.

cURL

1. Create the regional instance:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"name":"[INSTANCE_NAME]", "region":"[REGION]",
                 "databaseVersion":"[DATABASE_VERSION]",
                 "settings": {
                     "tier":"[MACHINE_TYPE]",
                     "backupConfiguration": {"binaryLogEnabled": true, "enabled": true},
                     "availabilityType":"REGIONAL"}}' \
        -X POST \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances
   

   For the complete list of parameters for this request, see the instances:insert page.
2. When the instance finishes initializing, update the root password on the instance:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"name": "root", "host": "%", "password": "[ROOT_PASSWORD]"}' \
        'https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]/users?host=%25&name=root'
   

For more details about creating an instance, see Creating Instances.

Legacy configuration: Creating a new instance configured for high availability

When you create an instance, you can configure it for high availability; Cloud SQL creates the failover replica at the same time that it creates the master.

To create an instance configured for high availability:

gcloud

1. Create the master instance and its failover replica:

   gcloud sql instances create [MASTER_INSTANCE_NAME] \
                        --backup-start-time [BACKUP_WINDOW_START_TIME] \
                        --failover-replica-name [FAILOVER_REPLICA_NAME] \
                        --tier [MACHINE_TYPE] --enable-bin-log
   

   The backup window start time is in the format HH:MM.

   For a complete list of available parameters, see the gcloud sql instances create reference page.

2. Configure the root user on the master:

   gcloud sql users set-password root % \
                         --instance [MASTER_INSTANCE_NAME] --password [PASSWORD]
   

   The failover replica is created with the same machine type (tier) as the master instance. You can change the failover replica's machine type later, but it must be at least as large as the master instance.

cURL

1. Create the master instance and its failover replica:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"name":"[MASTER_INSTANCE_NAME]", "region":"[REGION]",
                 "settings": {
                              "tier":"[MACHINE_TYPE]",
                              "backupConfiguration": {"binaryLogEnabled": true, "enabled": true}},
                              "failoverReplica": {"name": "[FAILOVER_REPLICA_NAME]"}}' \
        -X POST \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances
   

   For the complete list of parameters for this request, see the instances:insert page.
2. When the master finishes initializing, update the root password on the master:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"name": "root", "host": "%", "password": "[ROOT_PASSWORD]"}' \
        'https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]/users?host=%25&name=root'
   

   This change is automatically propagated to the replica.

Configuring an existing instance for high availability

Configuring an existing instance for high availability causes a few minutes of downtime while the instance is reconfigured.

To configure an existing instance for high availability:

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Click the instance name to open its Overview page.
3. Click Edit.
4. Under the Auto backups and high availability configuration option section, select the Automate backups, Enable binary logging, and High availability (regional) checkboxes.
5. Click Save.

   You are returned to the instance page for the master while the instance is updated.

gcloud

Update the instance to be regional:

gcloud sql instances patch [INSTANCE_NAME] --availability-type REGIONAL --enable-bin-log --backup-start-time=[HH:MM]

For reference information, see gcloud sql instances patch.

cURL

1. Check the master instance to see if automatic backups and binary logging are enabled. These are required for high availability instances.

       gcloud auth login
       ACCESS_TOKEN="$(gcloud auth print-access-token)"
       curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
            -X GET \
            https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]?fields=settings
   

   The output should look similar to the following:

     "backupConfiguration": {
       "kind": "sql#backupConfiguration",
       "startTime": "12:00",
       "enabled": true,
       "binaryLogEnabled": true
       }
   

2. If either enabled or binaryLogEnabled are false, use the instances:patch method to enable them both. To enable backups, set enabled to true and startTime to a value which is the start of the backup window. To enable binary logging, set binaryLogEnabled to true.

     gcloud auth login
     ACCESS_TOKEN="$(gcloud auth print-access-token)"
     curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
          --header 'Content-Type: application/json' \
          --data '{"settings" :
                         {"backupConfiguration" :
                              {"startTime": "[HH:MM]", "enabled": true, "binaryLogEnabled": true}}}' \
          -X PATCH \
   

3. Update the instance to be a regional instance:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"settings": {"availabilityType":"REGIONAL"}}' \
        -X PATCH \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]
   

Legacy configuration: Configuring an existing instance for high availability

Configuring an existing instance for high availability causes a few minutes of downtime while the instance is reconfigured.

To configure an existing instance for high availability:

gcloud

1. Check the status of the master instance:

   gcloud sql instances describe [MASTER_INSTANCE_NAME]
   

   If the databaseReplicationEnabled property is true, the instance is a replica; you cannot create a failover replica for a replica.

2. If the enabled property under backupConfiguration is false, enable backups for the master instance now:

   gcloud sql instances patch [MASTER_INSTANCE_NAME] --backup-start-time [HH:MM]
   

   The backup-start-time parameter is specified in 24-hour time, in the UTC±00 time zone, and specifies the start of a 4-hour backup window. Backups can start any time during the backup window.

3. If the binaryLogEnabled property is false, enable binary logs:

   gcloud sql instances patch --enable-bin-log [MASTER_INSTANCE_NAME]
   

   Enabling binary logs causes the instance to be restarted.

4. Create the replica:

   gcloud sql instances create [FAILOVER_REPLICA_NAME] \
                        --master-instance-name=[MASTER_INSTANCE_NAME] \
                        --replica-type=FAILOVER
   

cURL

1. Check the status of the master instance:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        -X GET \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]?fields=settings
   

   The output will contain the backupConfiguration, for example:

   "backupConfiguration": {
       "kind": "sql#backupConfiguration",
       "startTime": "12:00",
       "enabled": true,
       "binaryLogEnabled": true
   }
   

2. If either enabled or binaryLogEnabled are false, use the patch method of the instances resource to enable them both. Specify the properties of the backup configuration you want to keep as is and those you want to change.

   To enable backups, set enabled to true and the startTime to a value which is the start of the backup window. To enable binary logging, set binaryLogEnabled to true.

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"settings" : {"backupConfiguration" : {"startTime": "[HH:MM]", "enabled": true, "binaryLogEnabled": true}}}' \
        -X PATCH \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]
   

3. Use the insert method of the instances resource to create the failover replica.

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"name": "[REPLICA_NAME]", "masterInstanceName": "[MASTER_INSTANCE_NAME]",
                 "region": "[MASTER_REGION]", "databaseVersion": "[MASTER_DATABASE_VERSION]",
                 "replicaConfiguration": {"failoverTarget": true},
                 "settings": {"tier":"[MACHINE_TYPE]"}}' \
        -X POST \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances
   

   MACHINE_TYPE must be a machine type valid for Second Generation instances and at least as large as the machine type of the master instance.

Updating an instance from legacy to current high availability

If you have an existing MySQL instance that use the legacy high availability and you want to configure it with the current version of high availability, do the following:

1. Delete the failover replica on the MySQL instances you want to update. See Legacy configuration: Disabling high availability on an instance.
2. Configure the instance to use the current version of high availability high availability. See Configuring an existing instance for high availability.

Initiating failover

Testing failover is optional, but is recommended so that you can see how your application responds in the event of a failover.

Note: This feature helps you test your application's response to an automatic failover. Make sure your instance has completed any previous failover (so it is responding to connection requests) before you trigger another failover.

To learn more about failovers, see the Failover overview.

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Click the master instance name to open its Instance details page.
3. Click Failover in the button bar.

gcloud

Initiate the failover:

gcloud sql instances failover [MASTER_INSTANCE_NAME]

cURL

1. Describe the master instance to get the value of the settingsVersion field.

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        -X GET \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]
   

2. Initiate the failover:

   gcloud auth login
   ACCESS_TOKEN="$(gcloud auth print-access-token)"
   curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
        --header 'Content-Type: application/json' \
        --data '{"failoverContext":{"settingsVersion":"[SETTINGS_VERSION]"}}' \
        -X POST \
        https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]/failover
   

The instance fails over and is not available to serve data for a few minutes.

Legacy configuration: Creating an alert for replication lag

The time required for a failover operation depends on the amount of replication lag when failover is initiated. You can use Stackdriver Monitoring in the Google Cloud Platform Console to alert you when replication lag exceeds a threshold.

For more information about replication lag, search for "seconds_behind_master" on the SHOW SLAVE STATUS Syntax page in the MySQL documentation.

For more information about Stackdriver Monitoring, see the Stackdriver Monitoring documentation.

Note: Newly created Cloud SQL instances can take some time to appear in the Stackdriver Monitoring interface. If you do not see an instance that was recently created, check back again in an hour or so.

Legacy configuration: Setting an alert for a specific failover replica

1. Click the Stackdriver logo in the upper-left corner to return to the main Stackdriver Monitoring page.
2. Click Create Alerting Policy.
3. Enter a descriptive name for your alert.
4. Select Metric Threshold and click Next to open the Target tab.
5. For Resource Type, select Cloud SQL.
6. For Applies To, select Single, then select your failover replica.
7. Click Next to open the Configuration tab.

8. Enter the following values:

   Field	Value
   If Metric	Seconds Behind Master
   Condition	above
   Thresholds	120 (or whatever value is appropriate for your environment)
   For	3 minutes

9. Select your desired Notification methods, and click Save Condition.

Legacy configuration: Setting an alert for a group of failover replicas

If you plan to create multiple failover replicas in the same project, it could be easier to set the alert for the entire group, rather than on each failover replica individually. To create a group for monitoring, use a suffix for the failover replicas' names, such as "-failover". Make sure you do not use this suffix for any other types of Cloud SQL instances.

Create your failover replica group:

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. From the left-hand menu, select Monitoring to open the Stackdriver Monitoring page, and log in.
3. From the upper menu bar, select Groups > Create....
4. Enter a descriptive name for your group.
5. Leave Name and Contains selected, and enter your name substring.
6. Click Save Group. The summary page for the new group opens.
7. Confirm that the expected instances are included in the group.

Note: If you open the Groups Overview page, where all existing groups are listed, your group shows 0 instances. This is because Stackdriver uses the term "instances" to mean VMs and VM clusters. Cloud SQL instances are shown under Resources.

Create the alert for the failover replica group:

1. Click the Stackdriver logo in the upper-left corner to return to the main Stackdriver Monitoring page.
2. Click Create Alerting Policy.
3. Enter a descriptive name for your group alert.
4. Select Metric Threshold and click Next to open the Target tab.
5. For Resource Type, select Cloud SQL.
6. For Applies To, select Group, then select your failover group.
7. Leave Any Member Violates selected and click Next to open the Configuration tab.

8. Enter the following values:

   Field	Value
   If Metric	Seconds Behind Master
   Condition	above
   Thresholds	120 (or whatever value is appropriate for your environment)
   For	3 minutes

9. Select your desired Notification methods, and click Save Condition.

Verifying an instance has high availability

To verify an instance has high availability:

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Click an instance name to open its Instance details page.
3. Under Configuration, it shows Highly available (regional).

gcloud

gcloud sql instances describe [INSTANCE_NAME]

The output indicates availabilityType is REGIONAL.

cURL

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     -X GET \
     https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[MASTER_INSTANCE_NAME]?fields=settings

The output indicates availabilityType is REGIONAL.

Disabling high availability on an instance

Before you perform this procedure, make sure there are no operations currently running on the instance.

To disable high availability:

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Select the instance you want to disable high availability on.
3. On the Instance details page, click EDIT.
4. Open Auto backups.
5. Under Availability click Single zone.
6. Click Save. This edit requires a restart of the instance.
7. When you see the message box, click SAVE AND RESTART.

gcloud

For reference information, see gcloud sql instances patch.

gcloud sql instances patch [INSTANCE_NAME]
           --availability-type ZONAL

cURL

The following request uses the instances:patch method to change the availability type on the instance to ZONAL.

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{"settings": {"availabilityType":"ZONAL"}}' \
     -X PATCH \
     https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]

Legacy configuration: Disabling high availability on an instance

You need to delete the failover replica instance to remove high availability from the master instance.

Before you perform this procedure, make sure there are no operations currently running on the master instance.

To disable high availability:

Console

1. Go to the Cloud SQL Instances page in the Google Cloud Platform Console.

   Go to the Cloud SQL Instances page

2. Select the failover instance associated with the instance you want to disable high availability on.
3. On the Instance details page, click DELETE.
4. In the Delete Replica window, retype the name of failover instance.
5. Click DELETE. The configuration for the original instance changes to zonal.

gcloud

For reference information, see gcloud sql instances delete.

gcloud sql instances delete [FAILOVER_REPLICA_NAME]

cURL

The following request uses the instances:delete method to delete the instance.

gcloud auth login
ACCESS_TOKEN="$(gcloud auth print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json'\
      -X DELETE\
https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]

What's next

• Learn more about how the high availability configuration works.
• Test how your application responds to lost connections by restarting your instance.
• Learn more about managing your database connections.
• Create read replicas for your instance.
• Learn more about Stackdriver Monitoring.