This page explains how imports and exports work in Memorystore for Redis.
The import and export feature uses the native RDB snapshot feature of Redis to import data into or export data out of a Memorystore for Redis instance. The use of the native RDB format prevents lock-in and makes it very easy to move data within Google Cloud or outside of Google Cloud. Import and export uses Cloud Storage buckets to store RDB files.
The behavior of a Memorystore for Redis instance during export is very similar
BGSAVE command in open source Redis. Memorystore for Redis imports by
copying your RDB file into your instance's data directory.
For instructions on how to import and export RDB files, see Importing data to a Redis instance and Exporting data from a Redis instance.
For instructions on how to schedule RDB exports using Cloud Scheduler, see Scheduling Memorystore for Redis database exports using Cloud Scheduler.
Import feature behavior
For optimal performance, you should import from Cloud Storage buckets located in the same region as your Redis instance.
Your Redis instance is unavailable during an import operation. Your instance becomes available after the import operation finishes.
If an import is successful, the existing data in the instance is overwritten by the data in the RDB file.
If an import fails for any reason, the instance is brought online, but the data may be fully flushed. You can retry the import using the same RDB file, or you may use another RDB file to restore data.
Your Memorystore for Redis instance can import RDB files from the same Redis version and from a previous Redis version, but it cannot import RDB files from a newer version.
An import operation can be cancelled, but you should note that the cancellation results in your instance recovering with a fully flushed cache. This is the case for both Basic Tier and Standard Tier instances.
An instance can only import backups from older Redis versions. An instance running Redis 5.0 can import an RDB exported from Redis 4.0, but an instance running Redis 4.0 cannot import an RDB from Redis 5.0.
Export feature behavior
You can read and write data to your Redis instance during an export operation; however, you cannot perform any admin operations like scaling, updating, or configuring your instance.
Your instance may experience increased latency during the export.
For optimal performance, you should export your RDB backups to Cloud Storage buckets located in the same region as your Redis instance.
During an export, the RDB file is saved to a Cloud Storage bucket of your choosing.
You can cancel an export operation at any time. Cancelling does not impact the data or availability of an instance.
- Cancelling an export deletes the RDB file being written to the
Cloud Storage bucket, and frees up memory used by the
- Cancelling an export deletes the RDB file being written to the Cloud Storage bucket, and frees up memory used by the
An export can overwrite an existing file with the same name with no extra permissions when using the Google Cloud console. To export over an existing file with
gcloud, the user account needs the
Exporting under high write load
If you run an export during a period of high write-load your instance memory usage can increase up to 2X, causing the export operation to fail. You should perform export operations during periods of low writes. To monitor how much Redis data is stored in your instance you can create a dashboard to monitor your memory usage and monitor the instance memory usage metrics with Google Cloud's operations suite.
Memorystore for Redis export uses the
BGSAVE feature of Redis to take a
snapshot of the data in an instance. When executing
BGSAVE, Redis forks a new
child process to take the snapshot. Redis utilizes copy-on-write
during this process.
This means that the initial fork allocates minimal extra memory, but any pages changed by Redis server write traffic are duplicated. Therefore, under periods of high writes, in the worst case, the memory footprint of Redis doubles, causing the export operation to fail.
Exporting during a BGSAVE operation
You cannot export while a
BGSAVE operation is running. Standard Tier
Memorystore for Redis instances sometimes run full resynchronization
operations, or 'full syncs', to keep the primary and the replica synchronized. A
full resynchronization uses the
BGSAVE command, which prevents
Memorystore from performing an export.
To verify if a
BGSAVE operation is ongoing, view the
redis.googleapis.com/persistence/rdb/bgsave_in_progress API endpoint with
To view the metrics for a monitored resource by using Metrics Explorer, do the following:
- In the Google Cloud console, go to Monitoring or use the following button:
Go to Monitoring
- In the Monitoring navigation pane, click Metrics Explorer.
- In the Select a metric pane, expand the Metric menu, and then use the submenus to
select a resource type and metric type. For example, to chart the CPU utilization of a
virtual machine, do the following:
- (Optional) To reduce the menu's options, enter part of the metric name in the
Filter bar. For this example, enter
- In the Active resources menu, select VM instance.
- In the Active metric categories menu, select Instance.
- In the Active metrics menu, select CPU utilization and then click Apply.
- (Optional) To reduce the menu's options, enter part of the metric name in the Filter bar. For this example, enter
- By default, Metrics Explorer averages all time series. To change this behavior,
do one of the following:
- To group time series by resource or metric labels, expand the Labels menu in the Group by section, and then make your selections. You can also change the Grouping function.
- To view all time series, on the Group by entry, click deleteDelete.
Alternatively, you can use your own command line API tools.
Required permissions for importing and exporting
A user needs to configure sufficient IAM permissions to import or export RDB files. The simplest permissions setup is to apply the Cloud Memorystore Redis Admin and Storage Admin IAM roles to the user account, however this setup may be inappropriate for some user accounts because it gives them permissions to create, edit, or delete resources they shouldn't have access to. For instructions on setting minimally restricted permissions see Granting restricted permissions for import and export.
Duration of imports and exports
The duration of the import or export depends on the instance size and the size of the data set in the instance. Depending on the size, the imports and exports can take anywhere from a few minutes to a few hours.
For example, if you import or export a 10 GB instance with 8 GB of used memory, the operation can take 5 minutes. If you import or export a 120 GB instance with 100 GB of used memory, the operation can take an hour.
Stopping an import or export
To stop an import or export, refer to the following pages:
Monitoring progress of the operation
The time estimate for completion is not available while doing import and export. You can check whether the operation is in progress by checking the instance details page. To view the instance details page, click your instance ID on the Memorystore for Redis instances list page.
You can also monitor the progress of an import operation by running the following command:
gcloud redis instances describe [INSTANCE_ID] --region=[REGION]
[INSTANCE_ID]is the ID for your Memorystore for Redis instance.
[REGION]is the region your instance is in.
The command outputs
state: IMPORTING or
state: READY as one of the
Unsupported Cloud Storage buckets
Requester Pays Cloud Storage buckets are not supported for Memorystore imports and exports.
Securing your Cloud Storage buckets
You should ensure that your Cloud Storage bucket has the right restrictions to prevent unauthorized access to the Redis RDB backup it contains. Please refer to Cloud Storage Access Control Options on how to secure your Cloud Storage buckets.
Ensuring sufficient memory to complete an export
To prevent export from causing memory starvation, you should have 50% free memory in the instance during the export process. This may not be required all the time, depending on your write load, but having 50% free memory ensures that there is enough free space to accommodate copy-on-write overhead, if many pages are touched during the export process.
To understand the current memory usage of your instance, set up a Google Cloud's operations suite alert to notify you when your memory usage reaches a customizable threshold. Additionally you should set up a memory monitoring dashboard with Google Cloud's operations suite. These measures allow you to understand your instances memory usage patterns. Understanding memory usage patterns enables you to size your instance correctly when exporting at regular intervals.
Exporting and importing to instances with read replicas
For all Redis instances, including read replica enabled instances, data is exported from the primary node.
For read replica enabled instances, exporting can increase the CPU load and memory footprint on the primary node.
When importing data into a read replica enabled instance, the primary restarts during the import operation, and the instance is unavailable until the new data is fully loaded into the database.
Follow the guide for Importing data to a Redis instance.
Follow the guide for Exporting data from a Redis instance.