Store it, analyze it, back it up: Cloud Storage updates bring new replication options

Google Cloud Storage is the unified object storage that powers many Google Cloud Platform (GCP) customers, letting you store and move data as needed. Today, we bring you some Cloud Storage updates that give you even more control over how and where your data is stored, plus a glimpse into the future look of our Cloud Storage offering.

In this post, you’ll find details on:

• The new dual-regional option for replicating data

• Nearline and Coldline data now geo-redundant in multi-regional locations

• A new look coming for Cloud Storage choices

• The new Cloud Storage C++ library

New dual-regional option available in beta

We’re pleased to announce that you can now choose to store your data in a dual-regional Cloud Storage location, which is a pair of specific GCP regions. By storing your data in a dual-regional location, you get the availability and business continuity characteristics of geo-redundant storage with predictable, low-latency storage access for co-located compute resources. When you’re creating a new multi-regional storage bucket, you can pick from a list of these dual-regional pairs, and data in that bucket will automatically be stored geo-redundantly across those specific two regions. Dual-regional redundancy type joins the other two options currently available from Cloud Storage, as shown here:

Dual-regional is especially beneficial for analytics or big data workloads. The closer together you can put your data and compute resources, the faster you can potentially run your analysis. It can also help with content serving. If you use a content delivery network (CDN) shield for delivering content, you may want to co-locate the content you’re serving geographically close to your CDN shield implementation location.

With this new option, you write to a single dual-regional bucket without having to manually copy data between primary and secondary locations. No replication tool is needed to do this and there are no network charges associated with replicating the data, which means less overhead for you storage administrators out there. In the event of a region failure, we transparently handle the failover and ensure continuity for your users and applications accessing data in Cloud Storage.  

Data in Cloud Storage is also strongly consistent, so when you create a dual-regional bucket, then follow that up with a request to fetch a list of resources, your resource will be in that response. That is critical for data analytics workloads, where a lack of consistency could mean missing files in your request.  

Today, the first dual regions are available as nam4 (combining us-central1 with us-east1) and eur4 (combining europe-north1 and europe-west4). To have your data stored with dual-regional replication, select your storage class and select nam4 or eur4 as your location. You can see here how to create a dual-regional bucket:

Availability raised for Nearline and Coldline data in multi-regional locations

In addition, Cloud Storage’s Nearline and Coldline tiers are now geo-redundant in multi-regional locations, raising the availability of archival data stored this way. Accompanying this change, Google is raising its availability SLA for Nearline and Coldline data in multi-regional locations from 99.0% to 99.9%, an industry-leading metric that we’re proud to stand behind.

François Fanuel, IT Operations Manager at Mailjet, believes that reliability is core to the service they provide their clients. Mailjet offers an ISO27001-certified and GDPR compliant email solution that lets teams create, send, and monitor marketing emails, transactional emails and SMS. That means storing a lot of client email templates, log and backup data in the cloud. “The reliability as well as price of Google’s archival storage is critical to us delivering a cost-effective service on GCP,” says Fanuel. “And the performance we achieve by running our compute in multiple locations with co-located storage means our customers, located in over 150 countries, experience low latency and send emails that hit the inbox instantaneously.”  

With multi-regional Nearline and Coldline storage, you can access your data with millisecond latency, it’s distributed redundantly across a multi-region (U.S., EU or Asia), and you pay archival prices. This is helpful when you have data that won’t be accessed very often, but still needs to be protected with geographically dispersed copies, like media archives or regulated content. It also simplifies management.

Most data has a lifecycle where its relevance (and subsequently its access frequency) is highest immediately after it’s been created, and then tails off as it ages. We believe that as content is accessed less frequently, you should benefit from lower-tier storage costs while maintaining a consistent customer experience and data availability and reliability. When you turn on object lifecycle management or move your data to a lower tier, you can have a consistent experience no matter how often data is being accessed.

To take advantage of this today, select the Nearline or Coldline storage class option and a multi-regional location: EU, U.S. or Asia. Your data will be distributed redundantly across that location.  

A new look coming for Cloud Storage choices

Cloud Storage has always been one product with a single API and four classes of storage. But until now, the naming of these classes blended access frequency and redundancy status of that data in a way that can sometimes be confusing. With the new addition of dual-regional storage and ability to deliver geo-redundancy of Nearline and Coldline data based on location, your choices for storage are now simpler.  

You will soon have just two dimensions to choose from for Cloud Storage:

• Redundancy type: Multi-regional, dual-regional or regional

• Access tier: Standard, Nearline and Coldline

You can see the breakdown of options here (prices in US dollars):

Your tier choice will be based primarily on how frequently you are accessing that data. Your choice of redundancy type will be driven by your business continuity goals, as well as the desire to co-locate storage and compute for best performance. With these new options, you can now assign data to a lower tier to save costs while maintaining consistency in API, performance and redundancy.

We’ll be rolling out these changes in the UI, our web pages and our documentation going forward.  

For now, the storage class options you’re already familiar with are still available in the console and UI. We’ve described above how to take advantage of the new dual-regional and multi-regional archive options in the existing console.

New Cloud Storage C++ client library

We’re also launching the new Cloud Storage C++ client library so developers can program applications that use Google Cloud Storage in C++. This is particularly useful if you’re running big data, high-intensity-compute apps in Cloud Storage. The Cloud Storage C++ client library implements best practices like application exponential backoff and retry logic, so application developers don’t have to worry about those tasks. Customers building low-latency, high-throughput applications in C++ can optimize object storage performance with the new library.

You can download the Cloud Storage C++ library here.   

Take advantage of these new features and options to create the flexible storage infrastructure to support your cloud. Learn more about GCP storage here.