Use dual-region storage

Overview

This page describes how to use dual-region storage.

Required permissions

Console

In order to complete this guide using the Google Cloud console, you must have the proper IAM permissions. If the object you want to delete exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for the Google Cloud console.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Command line

In order to complete this guide using a command-line utility, you must have the proper IAM permissions. If the object you want to delete exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for gsutil commands.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Client libraries

In order to complete this guide using the Cloud Storage client libraries, you must have the proper IAM permissions. If the object you want to delete exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

Unless otherwise noted, client library requests are made through the JSON API and require permissions as listed in IAM permissions for JSON methods. To see which JSON API methods are invoked when you make requests using a client library, log the raw requests.

For a list of relevant IAM roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

REST APIs

JSON API

In order to complete this guide using the JSON API, you must have the proper IAM permissions. If the object you want to delete exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for JSON methods.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Create a dual-region bucket

Complete the following steps to create a dual-region bucket:

Console

  1. In the Google Cloud console, go to the Cloud Storage Buckets page.

    Go to Buckets

  2. Click Create.

  3. On the Create a bucket page, enter your bucket information. To go to the next step, click Continue.

    1. For Name your bucket, enter a name that meets the bucket naming requirements.

    2. For Choose where to store your data, next to Location type, choose Dual-region. Optional: You can combine the feature with turbo replication, by selecting the Add turbo replication checkbox.

    3. For Location, select the Continent and the associated Regions that you want to use.

    4. For Choose a default storage class for your data, select a storage class for the bucket. The default storage class is assigned by default to all objects uploaded to the bucket.

    5. For Choose how to control access to objects, select the public access prevention and access control options you want to use.

    6. For Choose how to protect object data, select the protection tools you want to use such as object versioning, a retention policy, and an encryption method.

  4. Click Create.

    To learn how to get detailed error information about failed Cloud Storage operations in the Google Cloud console, see Troubleshooting.

Command line

gcloud

Use the buckets create command with the --location and --placement flags:

gcloud storage buckets create gs://BUCKET_NAME --location=MULTI-REGION --placement=REGION_1,REGION_2

Where:

  • BUCKET_NAME is the name of the bucket you are creating. For example, my-bucket.

  • MULTI-REGION specifies the multi-region code associated with the underlying regions. For example, when choosing the regions ASIA-SOUTH1 (Mumbai) and ASIA-SOUTH2 (Delhi), use IN.

  • REGION_1 specifies the geographic location of a region for your bucket. For example, ASIA-EAST1.

  • REGION_2 specifies the geographic location of a second region for your bucket. For example, ASIA-SOUTHEAST1.

If the request is successful, the command returns the following message:

Creating gs://BUCKET_NAME/...

For a complete list of options available when creating buckets with gcloud storage, see buckets create options.

gsutil

Use the gsutil mb command with the -l and --placement flags:

gsutil mb -l MULTI-REGION --placement REGION_1,REGION_2 gs://BUCKET_NAME/

Where:

  • MULTI-REGION specifies the multi-region code associated with the underlying regions. For example, when choosing the regions ASIA-SOUTH1 (Mumbai) and ASIA-SOUTH2 (Delhi), use IN.

  • REGION_1 specifies the geographic location of a region for your bucket. For example, ASIA-EAST1.

  • REGION_2 specifies the geographic location of a second region for your bucket. For example, ASIA-SOUTHEAST1.

  • BUCKET_NAME is the name of the bucket you are creating. For example, my-bucket.

If the request is successful, the command returns the following message:

Creating gs://BUCKET_NAME/...

For a complete list of options available when creating buckets with gsutil, see mb options.

Client libraries

C++

For more information, see the Cloud Storage C++ API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name,
   std::string const& region_a, std::string const& region_b) {
  auto metadata = client.CreateBucket(
      bucket_name,
      gcs::BucketMetadata().set_custom_placement_config(
          gcs::BucketCustomPlacementConfig{{region_a, region_b}}));
  if (!metadata) throw std::move(metadata).status();

  std::cout << "Bucket " << metadata->name() << " created."
            << "\nFull Metadata: " << *metadata << "\n";
}

C#

For more information, see the Cloud Storage C# API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 


using Google.Apis.Storage.v1.Data;
using Google.Cloud.Storage.V1;
using System;

public class CreateDualRegionBucketSample
{
    public Bucket CreateDualRegionBucket(
        string projectId = "your-project-id",
        string bucketName = "your-unique-bucket-name",
        string location = "your-location",
        string region1 = "your-region1-name",
        string region2 = "your-region2-name")
    {
        var client = StorageClient.Create();

        var bucket = new Bucket
        {
            Name = bucketName,
            Location = location,
            CustomPlacementConfig = new Bucket.CustomPlacementConfigData
            {
                DataLocations = new[] { region1, region2 }
            }
        };

        var storageBucket = client.CreateBucket(projectId, bucket);

        Console.WriteLine($"Created storage bucket {storageBucket.Name}" +
            $" in {storageBucket.Location}" +
            $" with location-type {storageBucket.LocationType} and" +
            $" dataLocations {string.Join(",", storageBucket.CustomPlacementConfig.DataLocations)}.");

        return storageBucket;
    }

}

Go

For more information, see the Cloud Storage Go API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/storage"
)

// createBucketDualRegion creates a new dual-region bucket in the project in the
// provided location and regions.
// See https://cloud.google.com/storage/docs/locations#location-dr for more information.
func createBucketDualRegion(w io.Writer, projectID, bucketName string) error {
	// projectID := "my-project-id"
	// bucketName := "bucket-name"
	location := "US"
	region1 := "US-EAST1"
	region2 := "US-WEST1"

	ctx := context.Background()

	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*30)
	defer cancel()

	storageDualRegion := &storage.BucketAttrs{
		Location: location,
		CustomPlacementConfig: &storage.CustomPlacementConfig{
			DataLocations: []string{region1, region2},
		},
	}
	bucket := client.Bucket(bucketName)
	if err := bucket.Create(ctx, projectID, storageDualRegion); err != nil {
		return fmt.Errorf("Bucket(%q).Create: %w", bucketName, err)
	}

	attrs, err := bucket.Attrs(ctx)
	if err != nil {
		return fmt.Errorf("Bucket(%q).Attrs: %w", bucketName, err)
	}
	fmt.Fprintf(w, "Created bucket %v", bucketName)
	fmt.Fprintf(w, " - location: %v", attrs.Location)
	fmt.Fprintf(w, " - locationType: %v", attrs.LocationType)
	fmt.Fprintf(w, " - customPlacementConfig.dataLocations: %v", attrs.CustomPlacementConfig.DataLocations)
	return nil
}

Java

For more information, see the Cloud Storage Java API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 


import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.BucketInfo.CustomPlacementConfig;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.util.Arrays;

public class CreateBucketDualRegion {

  public static void createBucketDualRegion(
      String projectId,
      String bucketName,
      String location,
      String firstRegion,
      String secondRegion) {
    // The ID of your GCP project.
    // String projectId = "your-project-id";

    // The ID to give your GCS bucket.
    // String bucketName = "your-unique-bucket-name";

    // The location your dual regions will be located in.
    // String location = "US";

    // One of the regions the dual region bucket is to be created in.
    // String firstRegion = "US-EAST1";

    // The second region the dual region bucket is to be created in.
    // String secondRegion = "US-WEST1";

    // See this documentation for other valid locations and regions:
    // https://cloud.google.com/storage/docs/locations

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

    CustomPlacementConfig config =
        CustomPlacementConfig.newBuilder()
            .setDataLocations(Arrays.asList(firstRegion, secondRegion))
            .build();

    BucketInfo bucketInfo =
        BucketInfo.newBuilder(bucketName)
            .setLocation(location)
            .setCustomPlacementConfig(config)
            .build();

    Bucket bucket = storage.create(bucketInfo);

    System.out.println(
        "Created bucket "
            + bucket.getName()
            + " in location "
            + bucket.getLocation()
            + " with location type "
            + bucket.getLocationType()
            + " with Custom Placement Config "
            + bucket.getCustomPlacementConfig().toString());
  }
}

Node.js

For more information, see the Cloud Storage Node.js API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The bucket's pair of regions. Case-insensitive.
// See this documentation for other valid locations:
// https://cloud.google.com/storage/docs/locations
// const location = 'US';
// const region1 = 'US-EAST1';
// const region2 = 'US-WEST1';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
// The bucket in the sample below will be created in the project associated with this client.
// For more information, please see https://cloud.google.com/docs/authentication/production or https://googleapis.dev/nodejs/storage/latest/Storage.html
const storage = new Storage();

async function createDualRegionBucket() {
  // For regions supporting dual-regions see: https://cloud.google.com/storage/docs/locations
  const [bucket] = await storage.createBucket(bucketName, {
    location,
    customPlacementConfig: {
      dataLocations: [region1, region2],
    },
  });

  console.log(`Created '${bucket.name}'`);
  console.log(`- location: '${bucket.metadata.location}'`);
  console.log(`- locationType: '${bucket.metadata.locationType}'`);
  console.log(
    `- customPlacementConfig: '${JSON.stringify(
      bucket.metadata.customPlacementConfig
    )}'`
  );
}

createDualRegionBucket().catch(console.error);

PHP

For more information, see the Cloud Storage PHP API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

use Google\Cloud\Storage\StorageClient;

/**
 * Create a new bucket with a custom default storage class and location.
 *
 * @param string $bucketName The name of your Cloud Storage bucket.
 *        (e.g. 'my-bucket')
 * @param string $location Location for the bucket's regions. Case-insensitive.
 *        (e.g. 'US')
 * @param string $region1 First region for the bucket's regions. Case-insensitive.
 *        (e.g. 'US-EAST1')
 * @param string $region2 Second region for the bucket's regions. Case-insensitive.
 *        (e.g. 'US-WEST1')
 */
function create_bucket_dual_region(string $bucketName, string $location, string $region1, string $region2): void
{
    $storage = new StorageClient();
    $bucket = $storage->createBucket($bucketName, [
        'location' => $location,
        'customPlacementConfig' => [
            'dataLocations' => [$region1, $region2],
        ],
    ]);

    $info = $bucket->info();

    printf("Created '%s':", $bucket->name());
    printf("- location: '%s'", $info['location']);
    printf("- locationType: '%s'", $info['locationType']);
    printf("- customPlacementConfig: '%s'" . PHP_EOL, print_r($info['customPlacementConfig'], true));
}

Python

For more information, see the Cloud Storage Python API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

from google.cloud import storage


def create_bucket_dual_region(bucket_name, location, region_1, region_2):
    """Creates a Dual-Region Bucket with provided location and regions.."""
    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"

    # The bucket's pair of regions. Case-insensitive.
    # See this documentation for other valid locations:
    # https://cloud.google.com/storage/docs/locations
    # region_1 = "US-EAST1"
    # region_2 = "US-WEST1"
    # location = "US"

    storage_client = storage.Client()
    bucket = storage_client.create_bucket(bucket_name, location=location, data_locations=[region_1, region_2])

    print(f"Created bucket {bucket_name}")
    print(f" - location: {bucket.location}")
    print(f" - location_type: {bucket.location_type}")
    print(f" - customPlacementConfig data_locations: {bucket.data_locations}")

Ruby

For more information, see the Cloud Storage Ruby API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment. 

# The ID of your GCS bucket
# bucket_name = "your-bucket-name"

# The bucket's pair of regions. Case-insensitive.
# See this documentation for other valid locations:
# https://cloud.google.com/storage/docs/locations
# region_1 = "US-EAST1"
# region_2 = "US-WEST1"

require "google/cloud/storage"

storage = Google::Cloud::Storage.new
bucket  = storage.create_bucket bucket_name,
                                custom_placement_config: { data_locations: [region_1, region_2] }

puts "Bucket #{bucket.name} created:"
puts "- location: #{bucket.location}"
puts "- location_type: #{bucket.location_type}"
puts "- custom_placement_config:"
puts "  - data_locations: #{bucket.data_locations}"

REST APIs

JSON API

  1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials. For instructions, see API authentication.

  2. Create a .json file that contains the settings for the bucket, which must include a name and location. See the Buckets:Insert documentation for a complete list of settings. The following are common settings to include:

    {
  "name": "BUCKET_NAME",
  "location": "MULTI-REGION",
  "customPlacementConfig": {
    "dataLocations": ["REGION_1", "REGION_2"]
    },
  "storageClass": "STORAGE_CLASS"
}

    Where:

    • BUCKET_NAME is the name you want to give your bucket, subject to naming requirements. For example, my-bucket.
    • MULTI-REGION specifies the multi-region code associated with the underlying regions. For example, when choosing the regions ASIA-SOUTH1 (Mumbai) and ASIA-SOUTH2 (Delhi), use IN.
    • REGION_1 and REGION_2 are the regions where you want to store your bucket's object data. For example, ASIA-EAST1 and ASIA-SOUTHEAST1.
    • STORAGE_CLASS is the storage class of your bucket. For example, STANDARD.

  3. Use cURL to call the JSON API:

    curl -X POST --data-binary @JSON_FILE_NAME.json \
 -H "Authorization: Bearer OAUTH2_TOKEN" \
 -H "Content-Type: application/json" \
 "https://storage.googleapis.com/storage/v1/b?project=PROJECT_ID"

    Where:

    • JSON_FILE_NAME is name of the JSON file you created in Step 2.
    • OAUTH2_TOKEN is the access token you generated in Step 1.
    • PROJECT_ID is the ID of the project with which your bucket will be associated. For example, my-project.

XML API

  1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials. For instructions, see API authentication.

  2. Create an .xml file that contains the following information:

      <CreateBucketConfiguration>
     <LocationConstraint>MULTI-REGION</LocationConstraint>
        <CustomPlacementConfig>
           <DataLocations>
              <DataLocation>REGION_1</DataLocation>
              <DataLocation>REGION_2</DataLocation>
           </DataLocations>
        </CustomPlacementConfig>
     <StorageClass>STORAGE_CLASS</StorageClass>
  </CreateBucketConfiguration>

    Where:

    • MULTI-REGION specifies the multi-region code associated with the underlying regions. For example, when choosing the regions ASIA-SOUTH1 (Mumbai) and ASIA-SOUTH2 (Delhi), use IN.
    • REGION_1 and REGION_2 are the regions where you want to store your bucket's object data. For example, ASIA-EAST1 and ASIA-SOUTHEAST1.

    • STORAGE_CLASS is the default storage class of your bucket. For example, STANDARD.

  3. Use cURL to call the XML API:

      curl -X PUT --data-binary @XML_FILE_NAME.xml \
  -H "Authorization: Bearer OAUTH2_TOKEN" \
  -H "x-goog-project-id: PROJECT_ID" \
  "https://storage.googleapis.com/BUCKET_NAME"

    Where:

    • XML_FILE_NAME is the name of the XML file you created in Step 2.
    • OAUTH2_TOKEN is the access token you generated in Step 1.
    • PROJECT_ID is the ID of the project with which your bucket will be associated. For example, my-project.
    • BUCKET_NAME is the name you want to give your bucket, subject to bucket naming requirements. For example, my-bucket.

    If the request includes unsupported regions, an error message is returned. If the request was successful, a response is not returned.

What's next