OpenStack users: Back up your Cinder volumes to Google Cloud Storage
Ben Chong
Product Manager, Transfer Appliance
Mark Lambert
Google Cloud Platform, Google Cloud Platform
OpenStack Mitaka has just launched and we’re super excited about it. In collaboration with Red Hat and Biarca, we’ve developed an OpenStack Cinder backup driver for Google Cloud Storage, available in the Mitaka release.
Google joined the OpenStack Foundation in July 2015, when we announced Kubernetes integration with OpenStack. Our work on Mitaka is the next step on our roadmap to making Google Cloud Platform a seamless public cloud complement for OpenStack environments.
Backup and recovery services represent one of the most costly and complex aspects of large scale infrastructure management. OpenStack provides an efficient mechanism for allocation and management of persistent block storage through Cinder. In an OpenStack deployment, Cinder volumes house virtual machine data at rest as well as, potentially, the operating system boot device. In production deployments, it’s critical that this persistent data is protected as part of a comprehensive business continuity and disaster recovery strategy. To satisfy this requirement, Cinder provides a backup service that includes a backup driver specification allowing storage vendors to add support for additional backup targets.
This is where we come in. The addition of highly durable and available cloud-scale object storage allows organizations to shift from bulk commodity storage for backup to a more operationally efficient and cost-effective architecture, all while avoiding additional capital expenditures and the complexity of managing storage device scale out. The traditional barrier to adoption for object storage is the engineering effort required to adapt existing software and systems, designed for either file or block storage access, to object store native REST interfaces. The Cinder backup driver model provides the potential to abstract this engineering complexity for OpenStack users. As long as an appropriate backup driver is installed, the backup target works with Cinder as intended.
Our OpenStack Cinder backup driver is included as part of the standard Cinder backup driver set in Mitaka and requires minimal setup to get up and running. Full Cinder backup functionality was successfully tested with the Cloud Storage driver against 1GB, 5GB and 10GB Cinder volume sizes. In addition, the driver provides the following user configurable parameters to allow administrators to tune the installation:
| Parameter | Purpose |
| backup_gcs_credential_file | Denotes the full path of the json file of the Google service account (downloaded from the Google Developer Console in step 3) |
| backup_gcs_bucket | GCS bucket name to use for backup. Please refer to the official bucket naming guidelines. |
| backup_gcs_driver | Used for selecting the Google backup driver |
| backup_gcs_project_id | Denotes the project ID where the backup bucket will be created |
| backup_gcs_object_size | The size in bytes of GCS backup objects. default: 52428800 bytes |
| backup_gcs_block_size | The change tracking size for incremental backup, in bytes. backup_gcs_object_size has to be a multiple of backup_gcs_block_size default: 327678 bytes |
| backup_gcs_user_agent | http user-agent string for the gcs API |
| backup_gcs_reader_chunk_size | Chunk size for GCS object downloads in bytes. default: 2097152 bytes |
| backup_gcs_writer_chunk_size | Chunk size for GCS object uploads in bytes. Pass in a value of -1 to cause the file to be uploaded as a single chunk. default: 2097152 bytes |
| backup_gcs_num_retries/td> | Number of times to retry transfers. default: 3 |
| backup_gcs_bucket_location | Location of GCS bucket. default: ‘US’ |
| backup_gcs_storage_class | Storage class of GCS bucket. default: ‘NEARLINE’ |
| backup_gcs_retry_error_codes | List of GCS error codes for which to initiate a retry. default: [‘429’] |
| backup_gcs_enable_progress_timer | Enable or Disable the timer to send the periodic progress notifications to Ceilometer when backing up the volume to the GCS backend storage. The default value is True to enable the timer. default: True |
The Cinder backup driver works with any class of Cloud Storage, including our Google Cloud Storage Nearline or Coldline archival options. Google Cloud’s archival storage options offer the same durability and low latency of hot storage. The low cost yet high performance of Nearline and Coldline makes backing up Cinder volumes economical while offering the ability to quickly restore if necessary.
If you’re running OpenStack, there’s no need to invest in additional storage systems or build out a second data center for backup and recovery. You can now use Cloud Storage in a hybrid scenario, optimized via the Cinder backup driver available in Mitaka.
Editor’s note: This post was updated in October 2018 to reflect updated Cloud Storage capacity information.
