Hide
Google Cloud Storage

Durable Reduced Availability Storage


Durable Reduced Availability Storage enables you to store data at lower cost, with the tradeoff of lower availability than standard Google Cloud Storage.

Contents

  1. Durable Reduced Availability Storage Overview
  2. Creating a Durable Reduced Availability Bucket
  3. Moving Data to a Durable Reduced Availability Bucket

Durable Reduced Availability (DRA) Storage Overview

Durable Reduced Availability storage buckets have lower costs and lower availability, but are designed to have the same durability as standard Google Cloud Storage buckets.

DRA storage is appropriate for applications that are particularly cost-sensitive, or for which some unavailability is acceptable. For example:

  • Data backup - Where high durability is critical, but the highest availability is not required.
  • Batch jobs - Batch jobs can recover from unavailable data, for example by keeping track of the last object that was processed and resuming from that point upon re-starting.

For information about DRA storage pricing, see Pricing and Support.

Google Cloud Storage allows you to enable DRA at the bucket level. As described below, you can specify DRA storage at the time of bucket creation. Once the DRA bucket is created, all bucket and object operations are exactly the same as with the standard storage class. New objects in the bucket are transparently stored with the lower-availability configuration, and your billing for these objects will automatically reflect the lowered prices.

There is currently no support for changing an existing standard bucket into a DRA bucket. To copy objects from a standard bucket to a DRA bucket or vice versa, objects must be downloaded and re-uploaded to the new bucket. Standard data egress/ingress fees apply. For more information, see Moving Data to a Durable Reduced Availability Bucket.

Creating a Durable Reduced Availability Bucket

You enable Durable Reduced Availability Storage at the bucket level. The following examples show different ways to create a Durable Reduced Availability bucket.

gsutil

Using gsutil mb.

$ gsutil mb -c DRA gs://<bucketname>/

JSON API

Using cURL and the JSON API.

$ curl -X POST --data-binary @dra.json -H "Authorization: Bearer <auth token>" \
       https://www.googleapis.com/storage/v1/b?project=<project-id>

Where the dra.json file contains the following information:

{
 "name": "<bucket-name>",
 "storageClass": "DURABLE_REDUCED_AVAILABILITY"
}

XML API

Using cURL and the XML API.

$ curl -X PUT --data-binary @dra.txt -H "Authorization: Bearer <auth token>" -H "x-goog-project-id: <project-id>" \
       https://storage.googleapis.com/<bucket name>

Where the dra.txt file contains the following information:

<CreateBucketConfiguration>
   <LocationConstraint>US</LocationConstraint>
   <StorageClass>DURABLE_REDUCED_AVAILABILITY</StorageClass>
</CreateBucketConfiguration>

Python

This example Python code uses the boto library that takes care of the PUT request details.

import logging
import boto

def createBucketDRA(bucketName, projectId):
   """
   Perform a PUT Bucket operation to create the specified bucket with
   Durable Reduced Availability storage in the specified project.
   """
   try:
      uri = boto.storage_uri(bucketName, "gs")

      # Assign the required header values.
      headerValues = {
         'x-goog-api-version': '2',
         'x-goog-project-id': projectId
      }

      # Issue a PUT Bucket request to create a Durable Reduced Availability storage bucket.
      uri.create_bucket(headers=headerValues, location='US', storage_class='DURABLE_REDUCED_AVAILABILITY')

   except Exception, e:
      logging.error("createBucketDRA, error occurred: %s", e)

To verify that Durable Reduced Availability Storage is enabled for a bucket, get the bucket's metadata and look for the Storage class property. For example, you can do this with gsutil using:

$ gsutil ls -L -b gs://<bucketname>/

gs://<bucketname>/ :
     Storage class:         DURABLE_REDUCED_AVAILABILITY
     Location constraint:   US
     ...

Moving Data to a Durable Reduced Availability Bucket

If you want to move data from a Standard Storage to a Durable Reduced Availability Storage bucket, you need to download the data from the Standard bucket to your computer and then upload it to the Durable Reduced Availability bucket. You can do this by using the gsutil cp command's "daisy chain" copy mode, as follows:

$ gsutil cp -D -R gs://<standard_bucket>/* gs://<durable_reduced_availability_bucket>

The daisy chain copy mode downloads the data from the source bucket and uploads it to the destination bucket. No copy of the data is saved locally to the machine where you run gsutil. When you perform a daisy chain copy, you may incur network and operation costs for moving the data because the data are downloaded and re-uploaded.

To preserve ACLs during a daisy chain copy, use the -p option of the gsutil cp command. You may want to set a default bucket ACL on the destination bucket before uploading the data.

Back to top