Create and manage buckets with hierarchical namespace enabled

This page describes how to create, list, and delete buckets with hierarchical namespace enabled.

Create a bucket

This section describes how to create a bucket with hierarchical namespace enabled.

Roles required

In order to get the required permissions for creating a bucket , ask your administrator to grant you the Storage Admin (roles/storage.admin) role on the bucket.

The role contains the storage.buckets.create permission, which is required to create a bucket .

You might also be able to get this permission with other custom roles or predefined roles. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.

For instructions on granting roles on buckets, see Use IAM with buckets.

Console

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. Click Create.
3. On the Create a bucket page, enter your bucket information and then click Continue:
1. In the Get started section, do the following:
1. Enter a globally unique bucket name that meets the bucket name requirements.
2. To enable hierarchical namespace, click the expander arrow to expand the Optimize for file oriented and data-intensive workloads section, and then select Enable Hierarchical namespace on this bucket.

   Note: You cannot enable hierarchical namespace in an existing bucket.

3. Optional: To add a bucket label, click the expander arrow to expand the Labels section, and then click Add label, and specify a key and a value for your label.
2. In the Choose where to store your data section, do the following:
1. Select Region as the Location type.
2. In the Location list, select a region where the bucket data will be permanently stored.
3. In the Choose a storage class for your data section, either select a default storage class for the bucket, or select Autoclass for automatic storage class management of your bucket's data.

   Note: The Monthly cost estimate panel in the right pane estimates the bucket's monthly costs based on your selected storage class and location, as well as your expected data size and operations.

4. In the Choose how to control access to objects section, select whether or not your bucket enforces public access prevention, and for Access control, choose Uniform.

   Note: If public access prevention is already enforced by your project's organization policy, the Prevent public access toggle is locked.

5. In the Choose how to protect object data section, do the following:
   1. Optional: Select a Data protection method such as configuring a custom soft delete policy, object versioning, or a retention policy.
   2. To select a Data encryption method, click the expander arrow labeled Data encryption.
4. Click Create.

To learn how to get detailed error information about failed Cloud Storage operations in the Google Cloud console, see Troubleshooting.

Command line

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. In your development environment, run the gcloud storage buckets create command:

   gcloud storage buckets create gs://BUCKET_NAME --location=BUCKET_LOCATION --uniform-bucket-level-access --enable-hierarchical-namespace

   Where:

   • BUCKET_NAME is the name you want to give your bucket, subject to naming requirements. For example, my-bucket.
   • BUCKET_LOCATION is the location of your bucket. For example, us-east1.
   • --uniform-bucket-level-access: Enable uniform bucket-level access for the bucket.
     
     Note: Enabling uniform bucket-level access on a bucket is required for creating a bucket with hierarchical namespace enabled.
   • --enable-hierarchical-namespace: Enable hierarchical namespace for the bucket. You cannot enable hierarchical namespace in an existing bucket.

   If the request is successful, the command returns the following message:

   Creating gs://BUCKET_NAME/...

   Set the following flags to have greater control over the creation of your bucket:

   • --project: Specify the project ID or project number that your bucket will be associated with. For example, my-project.
   • --default-storage-class: Specify the default storage class of your bucket. For example, STANDARD.

   For a complete list of options for creating buckets using the Google Cloud CLI, see buckets create options.

   For example:

   gcloud storage buckets create gs://BUCKET_NAME --project=PROJECT_ID --default-storage-class=STORAGE_CLASS --location=BUCKET_LOCATION --uniform-bucket-level-access

Client Libraries

C++

For more information, see the Cloud Storage C++ API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name) {
  auto metadata = client.CreateBucket(
      bucket_name,
      gcs::BucketMetadata()
          .set_hierarchical_namespace(gcs::BucketHierarchicalNamespace{true})
          .set_iam_configuration(gcs::BucketIamConfiguration{
              gcs::UniformBucketLevelAccess{true, {}}, absl::nullopt}));
  if (!metadata) throw std::move(metadata).status();

  std::cout << "Bucket " << metadata->name() << " created."
            << "\nFull Metadata: " << *metadata << "\n";
}

Go

For more information, see the Cloud Storage Go API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/storage"
)

// createBucketHierarchicalNamespace creates a new bucket with hierarchical
// namespace features enabled.
func createBucketHierarchicalNamespace(w io.Writer, projectID, bucketName string) error {
	// projectID := "my-project-id"
	// bucketName := "bucket-name"

	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*30)
	defer cancel()

	attrs := &storage.BucketAttrs{
		HierarchicalNamespace: &storage.HierarchicalNamespace{
			Enabled: true,
		},
		// Hierarchical namespace buckets must use uniform bucket-level access.
		UniformBucketLevelAccess: storage.UniformBucketLevelAccess{
			Enabled: true,
		},
	}
	bucket := client.Bucket(bucketName)
	if err := bucket.Create(ctx, projectID, attrs); err != nil {
		return fmt.Errorf("Bucket(%q).Create: %w", bucketName, err)
	}
	fmt.Fprintf(w, "Created bucket %v with hierarchical namespace enabled\n", bucketName)
	return nil
}

Java

For more information, see the Cloud Storage Java API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.BucketInfo.HierarchicalNamespace;
import com.google.cloud.storage.BucketInfo.IamConfiguration;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public final class CreateHierarchicalNamespaceBucket {

  public static void createHierarchicalNamespaceBucket(String projectId, String bucketName)
      throws Exception {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID to give your GCS bucket
    // String bucketName = "your-unique-bucket-name";
    StorageOptions storageOptions = StorageOptions.newBuilder().setProjectId(projectId).build();
    try (Storage storage = storageOptions.getService()) {

      BucketInfo bucketInfo =
          BucketInfo.newBuilder(bucketName)
              .setIamConfiguration(
                  // Hierarchical namespace buckets must use uniform bucket-level access.
                  IamConfiguration.newBuilder().setIsUniformBucketLevelAccessEnabled(true).build())
              .setHierarchicalNamespace(HierarchicalNamespace.newBuilder().setEnabled(true).build())
              .build();

      Bucket bucket = storage.create(bucketInfo);

      System.out.printf(
          "Created bucket %s with Hierarchical Namespace enabled.%n", bucket.getName());
    }
  }
}

Node.js

For more information, see the Cloud Storage Node.js API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
// The bucket in the sample below will be created in the project associated with this client.
// For more information, please see https://cloud.google.com/docs/authentication/production or https://googleapis.dev/nodejs/storage/latest/Storage.html
const storage = new Storage();

async function createBucketWithHierarchicalNamespace() {
  const [bucket] = await storage.createBucket(bucketName, {
    iamConfiguration: {
      uniformBucketLevelAccess: {
        enabled: true,
      },
    },
    hierarchicalNamespace: {
      enabled: true,
    },
  });

  console.log(
    `Created '${bucket.name}' with hierarchical namespace enabled.`
  );
}

createBucketWithHierarchicalNamespace().catch(console.error);

PHP

For more information, see the Cloud Storage PHP API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

use Google\Cloud\Storage\StorageClient;

/**
 * Create a new bucket with Hierarchical Namespace enabled.
 *
 * @param string $bucketName The name of your Cloud Storage bucket.
 *        (e.g. 'my-bucket')
 */
function create_bucket_hierarchical_namespace(string $bucketName): void
{
    $storage = new StorageClient();
    $bucket = $storage->createBucket($bucketName, [
        'hierarchicalNamespace' => ['enabled' => true],
        'iamConfiguration' => ['uniformBucketLevelAccess' => ['enabled' => true]]
    ]);

    printf('Created bucket %s with Hierarchical Namespace enabled.', $bucket->name());
}

Python

For more information, see the Cloud Storage Python API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

from google.cloud import storage


def create_bucket_hierarchical_namespace(bucket_name):
    """Creates a bucket with hierarchical namespace enabled."""
    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    bucket.iam_configuration.uniform_bucket_level_access_enabled = True
    bucket.hierarchical_namespace_enabled = True
    bucket.create()

    print(f"Created bucket {bucket_name} with hierarchical namespace enabled.")

Ruby

For more information, see the Cloud Storage Ruby API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

def create_bucket_hierarchical_namespace bucket_name:
  # The ID to give your GCS bucket
  # bucket_name = "your-unique-bucket-name"

  require "google/cloud/storage"

  storage = Google::Cloud::Storage.new

  hierarchical_namespace = Google::Apis::StorageV1::Bucket::HierarchicalNamespace.new enabled: true

  storage.create_bucket bucket_name do |b|
    b.uniform_bucket_level_access = true
    b.hierarchical_namespace = hierarchical_namespace
  end

  puts "Created bucket #{bucket_name} with Hierarchical Namespace enabled."
end

REST APIs

JSON API

1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

2. Create a JSON file that contains the settings for the bucket, which must include a name for the bucket. See the Buckets: Insert documentation for a complete list of settings. The following are common settings to include:

{
  "name": "BUCKET_NAME",
  "location": "BUCKET_LOCATION",
  "storageClass": "STORAGE_CLASS",
  "hierarchicalNamespace": {
    "enabled": "BOOLEAN"
  },
  "iamConfiguration": {
    "uniformBucketLevelAccess": {
      "enabled": true
  },
},
}

Where:

• BUCKET_NAME is the name you want to give your bucket, subject to naming requirements. For example, my-bucket.
• BUCKET_LOCATION is the location where you want to store your bucket's object data. For example, US-EAST1.
• STORAGE_CLASS is the default storage class of your bucket. For example, STANDARD.
• hierarchicalNamespace.enabled is set to TRUE to enable hierarchical namespace for your bucket. You cannot enable hierarchical namespace in an existing bucket.
3. uniformBucketLevelAccess.enabled is set to TRUE to enable uniform bucket-level access for your bucket.
   
   Note: Enabling uniform bucket-level access on a bucket is required for creating a bucket with hierarchical namespace enabled.
4. Use cURL to call the JSON API:

   curl -X POST --data-binary @JSON_FILE_NAME \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json" \
     "https://storage.googleapis.com/storage/v1/b?project=PROJECT_IDENTIFIER"

   Where:

   • JSON_FILE_NAME is name of the JSON file that contains the bucket settings.
   • PROJECT_IDENTIFIER is the ID or number of the project that your bucket will be associated with. For example, my-project.

A bucket's storage layout describes how objects are arranged within a bucket, either in a flat namespace or a hierarchical namespace. For instructions on viewing a bucket's storage layout, see Get a bucket's storage layout.

To list all buckets, regardless of their storage layout, follow the instructions detailed in List buckets.

You can delete a bucket with hierarchical namespace enabled in the same manner as any other bucket. For the purposes of deletion, if a bucket enabled with hierarchical namespace only contains empty folders and no objects or managed folders, then the bucket is considered empty. For instructions about deleting buckets, see Delete buckets.

What's next

• Get information about a bucket's metadata.
• Get information about a bucket's storage layout.
• Understand folders.
• Create and manage folders.
• Rename folders.

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how Cloud Storage performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try Cloud Storage free