Use point-in-time recovery

Stay organized with collections Save and categorize content based on your preferences.

This page describes how to use point-in-time recovery to restore your Cloud SQL instance.

To learn more about point-in-time recovery, see Point-in-time recovery.

Log storage for point-in-time recovery

Point-in-time recovery uses binary logs. New logs are generated regularly and use storage space. The binary logs are automatically deleted with their associated automatic backup, which generally happens after the value set for transactionLogRetentionDays is met. This is the number of days of transaction logs that Cloud SQL retains for point-in-time recovery, from 1 to 7.

If the size of your binary logs is causing an issue for your instance:

  • You can increase the instance storage size. However, the binary log size increase in disk usage might be temporary.

  • We recommend enabling automatic storage increase to avoid unexpected storage issues.

  • You can disable point-in-time recovery if you want to delete logs and recover storage. Decreasing the storage used does not shrink the size of the disk provisioned for the instance.

  • Logs are purged once daily, not continuously. Setting log retention to two days means that at least two days of logs, and at most three days of logs, are retained. We recommend setting the number of backups to one more than the days of log retention to guarantee a minimum of specified days of log retention.

Enable point-in-time recovery

When you create a new instance in the Google Cloud console, both Automated backups and Enable point-in-time recovery are automatically enabled.

The following procedure enables point-in-time recovery on an existing instance.

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Open the more actions menu More actions icon. for the instance you want to enable point-in-time recovery on and click Edit.
  3. Under Customize your instance, expand the Data Protection section.
  4. Select the Enable point-in-time recovery checkbox.
  5. Expand Advanced options.
  6. Enter the number of days to retain logs, from 1-7.
  7. Click Save.

gcloud

  1. Display the instance overview: 
    gcloud sql instances describe INSTANCE_NAME
  2. If you see enabled: false in the backupConfiguration section, enable scheduled backups: 
    gcloud sql instances patch INSTANCE_NAME \
--backup-start-time=HH:MM

    Specify the backup-start-time parameter using 24-hour time in UTC±00 time zone.

  3. Enable point-in-time recovery: 
    gcloud sql instances patch INSTANCE_NAME \
--enable-bin-log

    If you're enabling point-in-time recovery on a primary instance, you can also configure the number of days for which you want to retain transaction logs by adding the following parameter:

    --retained-transaction-log-days=RETAINED_TRANSACTION_LOG_DAYS
  4. Confirm your change: 
    gcloud sql instances describe INSTANCE_NAME

    In the backupConfiguration section, you see binaryLogEnabled: true if the change was successful.

Terraform

To enable point-in-time recovery, use a Terraform resource.

cloud_sql/instance_pitr/main.tf 
resource "google_sql_database_instance" "default" {
  name             = "mysql-instance-pitr"
  region           = "asia-northeast1"
  database_version = "MYSQL_8_0"
  settings {
    tier = "db-f1-micro"
    backup_configuration {
      enabled                        = true
      binary_log_enabled             = true
      start_time                     = "20:55"
      transaction_log_retention_days = "3"
    }
  }
  deletion_protection = false # set to true to prevent destruction of the resource
}

Apply the changes

To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.

Prepare Cloud Shell

  1. Launch Cloud Shell.

  2. Set the default Google Cloud project where you want to apply your Terraform configurations.

    You only need to run this command once per project, and you can run it in any directory.

    export GOOGLE_CLOUD_PROJECT=PROJECT_ID

    Environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module).

  1. In Cloud Shell, create a directory and a new file within that directory. The filename must have the .tf extension—for example main.tf. In this tutorial, the file is referred to as main.tf. 
    mkdir DIRECTORY && cd DIRECTORY && touch main.tf

  2. If you are following a tutorial, you can copy the sample code in each section or step.

    Copy the sample code into the newly created main.tf.

    Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.

  3. Review and modify the sample parameters to apply to your environment.
  4. Save your changes.
  5. Initialize Terraform. You only need to do this once per directory. 
    terraform init

    Optionally, to use the latest Google provider version, include the -upgrade option: 

    terraform init -upgrade

Apply the changes

  1. Review the configuration and verify that the resources that Terraform is going to create or update match your expectations: 
    terraform plan

    Make corrections to the configuration as necessary.

  2. Apply the Terraform configuration by running the following command and entering yes at the prompt: 
    terraform apply

    Wait until Terraform displays the "Apply complete!" message.

  3. Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.

Delete the changes

To delete your changes, do the following:

  1. To disable deletion protection, in your Terraform configuration file set the deletion_protection argument to false. 
    deletion_protection =  "false"
  2. Apply the updated Terraform configuration by running the following command and entering yes at the prompt: 
    terraform apply

  1. Remove resources previously applied with your Terraform configuration by running the following command and entering yes at the prompt:

    terraform destroy

REST v1

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • instance-id: The instance ID (primary or replica)
  • start-time The time in the format "HH:MM"

HTTP method and URL: 

PATCH https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id

Request JSON body: 

{
  "settings":
  {
    "backupConfiguration":
    {
      "startTime": "start-time",
      "enabled": true,
      "binaryLogEnabled": true
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

REST v1beta4

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • instance-id: The instance ID (primary or replica)
  • start-time The time in the format "HH:MM"

HTTP method and URL: 

PATCH https://sqladmin.googleapis.com/v1beta4/projects/project-id/instances/instance-id

Request JSON body: 

{
  "settings":
  {
    "backupConfiguration":
    {
      "startTime": "start-time",
      "enabled": true,
      "binaryLogEnabled": true
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Perform point-in-time recovery

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Open the more actions menu More actions icon. for the instance you want to recover and click Create clone.
  3. Optionally, on the Create a clone page, update the ID of the new clone.
  4. Select Clone from an earlier point in time.
  5. Enter a point-in-time recovery time.
  6. Click Create clone.

gcloud

Create a clone using point-in-time recovery.

Replace the following:

  • SOURCE_INSTANCE_NAME - Name of the instance you're restoring from.
  • NEW_INSTANCE_NAME - Name for the clone.
  • TIMESTAMP - UTC timezone for the source instance in RFC 3339 format. For example, 2012-11-15T16:19:00.094Z.
gcloud sql instances clone SOURCE_INSTANCE_NAME \
NEW_INSTANCE_NAME \
--point-in-time 'TIMESTAMP'

REST v1

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • target-instance-id: The target instance ID
  • source-instance-id: The source instance ID
  • restore-timestamp The point-in-time to restore up to

HTTP method and URL: 

POST https://sqladmin.googleapis.com/v1/projects/project-id/instances/source-instance-id/clone

Request JSON body: 

{
  "cloneContext":
  {
    "kind": "sql#cloneContext",
    "destinationInstanceName": "target-instance-id",
    "pointInTime": "restore-timestamp"
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

REST v1beta4

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • target-instance-id: The target instance ID
  • source-instance-id: The source instance ID
  • restore-timestamp The point-in-time to restore up to

HTTP method and URL: 

POST https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/source-instance-id/clone

Request JSON body: 

{
  "cloneContext":
  {
    "kind": "sql#cloneContext",
    "destinationInstanceName": "target-instance-id",
    "pointInTime": "restore-timestamp"
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Disable point-in-time recovery

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Open the more actions menu More actions icon. for the instance you want to disable and select Edit.
  3. Under Customize your instance, expand the Data Protection section.
  4. Clear Enable point-in-time recovery.
  5. Click Save.
  6. In the Overview page for the instance, under Configuration, the point-in-time recovery setting is listed as disabled.

gcloud

  1. Disable point-in-time recovery: 
    gcloud sql instances patch INSTANCE_NAME \
--no-enable-bin-log
  2. Confirm your change: 
    gcloud sql instances describe INSTANCE_NAME

    In the backupConfiguration section, you see binaryLogEnabled: false if the change was successful.

REST v1

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • instance-id: The instance ID

HTTP method and URL: 

PATCH https://sqladmin.googleapis.com/v1/projects/project-id/instances/instance-id

Request JSON body: 

{
  "settings":
  {
    "backupConfiguration":
    {
      "enabled": false,
      "binaryLogEnabled": false
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

REST v1beta4

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • instance-id: The instance ID

HTTP method and URL: 

PATCH https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id

Request JSON body: 

{
  "settings":
  {
    "backupConfiguration":
    {
      "enabled": false,
      "binaryLogEnabled": false
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Set transaction log retention

To set the number of days to retain binary logs (from 1 to 7):

Console

  1. In the Google Cloud console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Open the more actions menu More actions icon. for the instance you want to set the transaction log on and select Edit.
  3. Under Customize your instance, expand the Data Protection section.
  4. In the Enable point-in-time recovery section, expand Advanced options.
  5. Enter the number of days to retain logs, from 1-7.
  6. Click Save.

gcloud

Edit the instance to set the number of days to retain binary logs.

Replace the following:

  • INSTANCE-NAME - The name of the instance you want to set the transaction log on.
  • DAYS-TO-RETAIN - The number of days of transaction logs to keep. The valid range is between 1 and 7. The default value is 7 if not specified. Only valid when point in time recovery is enabled. Keeping more days of transaction logs requires bigger storage size.
gcloud sql instances patch INSTANCE-NAME 

--retained-transaction-log-days=DAYS-TO-RETAIN

REST v1beta4

Before using any of the request data, make the following replacements:

  • days-to-retain: The number of days to retain transaction logs, from 1 to 7
  • project-id: The project ID
  • instance-id: The instance ID

HTTP method and URL: 

PATCH https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/instance-id

Request JSON body: 

{
  "settings":
  {
    "backupConfiguration":
    {
      "transactionLogRetentionDays": "days-to-retain"
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Perform the point-in-time recovery using binary log positions

While we recommend you perform point-in-time recovery using timestamps as described in the previous procedure, you can also perform point-in-time recovery by providing a specific binary log position in a binary log file.

For more information about point-in-time recovery using binary log positions, see the MySQL Reference, Point-in-Time Recovery Using the Binary Log.

Before you begin

Before completing this task, you must have:

Identify the recovery position

  1. Use the MySQL client to connect to the instance you want to restore to.

    To do so, use the Cloud Shell or your local client machine. For more information, see Connection options for external applications.

  2. Show the binary log files for the instance:

    SHOW BINARY LOGS;

  3. Display the first 100 events in the most recent binary log file:

    SHOW BINLOG EVENTS IN '<BINARY_LOG_FILE>' LIMIT 100;

    You can adjust the number of rows to show, but don't show all of the events in the file until you know how large the file is. Displaying a large number of events can affect system performance.

  4. If the event you're looking for isn't displayed, use the last position displayed as the starting point to search the next set of events:

    SHOW BINLOG EVENTS IN '<BINARY_LOG_FILE>' FROM <POSITION> LIMIT 100;

  5. When you find the event that marks the point in time you want to restore up to, record the position (shown as Pos) and the name of the binary log file.

    The binary log filename and the position are the values you use for the point-in-time recovery.

Below is some sample output from the SHOW BINLOG EVENTS command:

+------------------+-----+-------------+-----------+-------------+-----------------------------------------------------+
| Log_name         | Pos | Event_type  | Server_id | End_log_pos | Info                                                |
+------------------+-----+-------------+-----------+-------------+-----------------------------------------------------+
| mysql-bin.000011 |   4 | Format_desc |  88955285 |         120 | Server ver: 5.6.30-log, Binlog ver: 4               |
| mysql-bin.000011 | 120 | Query       |  88955285 |         211 | create database db1                                 |
| mysql-bin.000011 | 211 | Query       |  88955285 |         310 | use `db1`; CREATE TABLE t (c CHAR(20))              |
| mysql-bin.000011 | 310 | Query       |  88955285 |         381 | BEGIN                                               |
| mysql-bin.000011 | 381 | Table_map   |  88955285 |         426 | table_id: 18 (db1.t)                                |
| mysql-bin.000011 | 310 | Query       |  88955285 |         381 | BEGIN                                               |

| mysql-bin.000011 | 426 | Write_rows  |  88955285 |         464 | table_id: 18 flags: STMT_END_F                      |
| mysql-bin.000011 | 464 | Xid         |  88955285 |         495 | COMMIT /* xid=56 */                                 |
| mysql-bin.000011 | 495 | Query       |  88955285 |         566 | BEGIN                                               |
| mysql-bin.000011 | 566 | Table_map   |  88955285 |         611 | table_id: 18 (db1.t)                                |
| mysql-bin.000011 | 611 | Write_rows  |  88955285 |         649 | table_id: 18 flags: STMT_END_F                      |
| mysql-bin.000011 | 649 | Xid         |  88955285 |         680 | COMMIT /* xid=57 */                                 |
| mysql-bin.000011 | 680 | Query       |  88955285 |         751 | BEGIN                                               |
| mysql-bin.000011 | 751 | Table_map   |  88955285 |         796 | table_id: 18 (db1.t)                                |
| mysql-bin.000011 | 796 | Write_rows  |  88955285 |         834 | table_id: 18 flags: STMT_END_F                      |
| mysql-bin.000011 | 834 | Xid         |  88955285 |         865 | COMMIT /* xid=58 */                                 |
| mysql-bin.000011 | 865 | Query       |  88955285 |         977 | use `db1`; DROP TABLE `t` /* generated by server */ |
+------------------+-----+-------------+-----------+-------------+-----------------------------------------------------+
16 rows in set (0.04 sec)

To restore up to the DROP TABLE statement, bolded above, you would use "865" in "mysql-bin.000011" as the recovery position. The DROP TABLE statement and all operations after it are not reflected in the new instance.

Perform the point-in-time recovery using binary log event positions

gcloud

Use the gcloud sql instances clone command with the --bin-log-file-name and --bin-log-position flags.

  1. Create the new instance using the binary log filename and recovery position.

    Replace the following:

    • SOURCE_INSTANCE_NAME: Name of the instance you're restoring from.
    • NEW_INSTANCE_NAME: Name for the clone.
    • BINLOG_FILE_NAME: Name for the binary log, such as mysql-bin.187288.
    • POSITION: The position in the binary log to restore up to, such as 50001356.
    gcloud sql instances clone SOURCE_INSTANCE_NAME \
NEW_INSTANCE_NAME \
--bin-log-file-name="BINLOG_FILE_NAME" \
--bin-log-position=POSITION

    For example, a gcloud sql instances clone command might look similar to the following:

    gcloud sql instances clone instance1 \
instance1-clone \
--bin-log-file-name=mysql-bin.0000031 \
--bin-log-position=107 \
  2. Use the operation ID returned from the clone command to check the status of the restore operation. 
    gcloud sql operations describe OPERATION_ID

    When the operation is in progress, a state of RUNNING is returned. When the operation is complete, a state of DONE is returned.

REST v1

Create the new instance using the binary log filename and recovery position you have identified:

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • target-instance-id: The target instance ID
  • source-instance-id: The source instance ID
  • binary-log-file-name The name of the binary log file
  • binary-log-position The position within the binary log file

HTTP method and URL: 

POST https://sqladmin.googleapis.com/v1/projects/project-id/instances/source-instance-id/clone

Request JSON body: 

{
  "cloneContext":
  {
    "kind": "sql#cloneContext",
    "destinationInstanceName": "target-instance-id",
    "binLogCoordinates":
    {
      "kind": "sql#binLogCoordinates",
      "binLogFileName": "binary-log-file-name",
      "binLogPosition": "binary-log-position"
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

REST v1beta4

Create the new instance using the binary log file name and recovery position you have identified:

Before using any of the request data, make the following replacements:

  • project-id: The project ID
  • target-instance-id: The target instance ID
  • source-instance-id: The source instance ID
  • binary-log-file-name The name of the binary log file
  • binary-log-position The position within the binary log file

HTTP method and URL: 

POST https://sqladmin.googleapis.com/sql/v1beta4/projects/project-id/instances/source-instance-id/clone

Request JSON body: 

{
  "cloneContext":
  {
    "kind": "sql#cloneContext",
    "destinationInstanceName": "target-instance-id",
    "binLogCoordinates":
    {
      "kind": "sql#binLogCoordinates",
      "binLogFileName": "binary-log-file-name",
      "binLogPosition": "binary-log-position"
    }
  }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

Troubleshoot

Issue Troubleshooting

argument --point-in-time: Failed to parse date/time:
Unknown string format: 2021-0928T30:54:03.094;
received: 2021-0928T30:54:03.094Z

OR

Invalid value at 'body.clone_context.point_in_time'
(type.googleapis.com/google.protobuf.Timestamp), Field 'pointInTime',
Invalid time format: Failed to parse input,

 The timestamp you provided is invalid.

HTTP Error 400: Successful backup required for carrying out the operation was not found.

OR

Successful backup required for carrying out the operation was not found. or Time where no backups can be found.

 The timestamp that you provided is for a time where backups or when binlog coordinates could not be found.

What's next