Configure access to a sink: Cloud Storage

Cloud Storage uses a Google-managed service account, known as a service agent, to move data to a Cloud Storage bucket. This service agent is created the first time that you call googleServiceAccounts.get.

The destination bucket does not need to belong to the same project as the service agent. The steps are the same regardless of which project the bucket is in.

User permissions

In order to grant the required permissions to the service agent, you must have the relevant permissions on the destination bucket:

• storage.buckets.getIamPolicy
• storage.buckets.setIamPolicy

The Storage Legacy Bucket Owner role (roles/storage.legacyBucketOwner) or the Storage Admin role (roles/storage.admin) provide the required permissions.

Auto-granting permissions in the Google Cloud console

If you're using the Google Cloud console to create your transfer and have the permissions listed in User permissions, the service agent will automatically be granted the required permissions on your destination bucket.

You can skip the steps on this page. If required, configure access to your source, then create a transfer.

Required permissions

The service agent must have the following permissions for the destination bucket:

Permission	Description
storage.buckets.get	Allows the service account to get the location of the bucket.
storage.objects.create	Allows the service account to add objects to the bucket.
storage.objects.delete	Allows the service account to delete objects in the bucket. Required if you set overwriteObjectsAlreadyExistingInSink or deleteObjectsUniqueInSink to true. Note that if the destination bucket has object versioning enabled, neither overwriteObjectsAlreadyExistingInSink nor deleteObjectsUniqueInSink permanently delete any objects. Instead, relevant live object versions become noncurrent.
storage.objects.list	Allows the service account to list objects in the bucket. Required if you set overwriteObjectsAlreadyExistingInSink to false or deleteObjectsUniqueInSink to true.

The following predefined role grants the required permissions:

• Storage Legacy Bucket Writer (roles/storage.legacyBucketWriter)

Any Cloud Storage role marked as a legacy role can only be granted at the bucket level.

For a complete list of Cloud Storage roles and the permissions they contain, see IAM roles.

Grant the required permissions

To grant the Storage Legacy Bucket Writer role to the service agent, follow the steps below.

Find the service agent's email

1. Go to the googleServiceAccounts.get reference page.

   An interactive panel opens, titled Try this method.

2. In the panel, under Request parameters, enter your project ID. The project you specify here must be the project that you're using to manage Storage Transfer Service, which might be different from the destination bucket's project.

3. Click Execute.

   Your service agent's email is returned as the value of accountEmail. Copy this value.

   The service agent's email uses the format [e-mail].

Add the service agent to a bucket-level policy

Console

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. Click the Bucket overflow menu () associated with the bucket to which you want to grant a principal a role.

3. Choose Edit access.

4. Click the + Add principal button.

5. In the New principals field, enter your service agent's account email.

6. Select Storage Legacy Bucket Writer from the Select a role drop-down menu.

7. Click Save.

gcloud

Use the gcloud storage buckets add-iam-policy-binding command:

gcloud storage buckets add-iam-policy-binding gs://BUCKET_NAME \
--member=serviceAccount:YOUR_AGENT_EMAIL --role=roles/storage.legacyBucketWriter

Where:

• BUCKET_NAME is the name of the bucket that you're granting the principal access to. For example, my-bucket.
• YOUR_AGENT_EMAIL is the agent account email that you copied in Find the service agent's email.

Code samples

C++

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage C++ API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name,
   std::string const& role, std::string const& member) {
  auto policy = client.GetNativeBucketIamPolicy(
      bucket_name, gcs::RequestedPolicyVersion(3));

  if (!policy) throw std::move(policy).status();

  policy->set_version(3);
  for (auto& binding : policy->bindings()) {
    if (binding.role() != role || binding.has_condition()) {
      continue;
    }
    auto& members = binding.members();
    if (std::find(members.begin(), members.end(), member) == members.end()) {
      members.emplace_back(member);
    }
  }

  auto updated = client.SetNativeBucketIamPolicy(bucket_name, *policy);
  if (!updated) throw std::move(updated).status();

  std::cout << "Updated IAM policy bucket " << bucket_name
            << ". The new policy is " << *updated << "\n";
}

C#

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage C# API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

using Google.Apis.Storage.v1.Data;
using Google.Cloud.Storage.V1;
using System;
using System.Collections.Generic;

public class AddBucketIamMemberSample
{
    public Policy AddBucketIamMember(
        string bucketName = "your-unique-bucket-name",
        string role = "roles/storage.objectViewer",
        string member = "serviceAccount:[e-mail]")
    {
        var storage = StorageClient.Create();
        var policy = storage.GetBucketIamPolicy(bucketName, new GetBucketIamPolicyOptions
        {
            RequestedPolicyVersion = 3
        });
        // Set the policy schema version. For more information, please refer to https://cloud.google.com/iam/docs/policies#versions.
        policy.Version = 3;

        Policy.BindingsData bindingToAdd = new Policy.BindingsData
        {
            Role = role,
            Members = new List<string> { member }
        };

        policy.Bindings.Add(bindingToAdd);
        var bucketIamPolicy = storage.SetBucketIamPolicy(bucketName, policy);
        Console.WriteLine($"Added {member} with role {role} " + $"to {bucketName}");
        return bucketIamPolicy;
    }
}

Go

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage Go API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/iam"
	"cloud.google.com/go/storage"
)

// addBucketIAMMember adds the bucket IAM member to permission role.
func addBucketIAMMember(w io.Writer, bucketName string) error {
	// bucketName := "bucket-name"
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*10)
	defer cancel()

	bucket := client.Bucket(bucketName)
	policy, err := bucket.IAM().Policy(ctx)
	if err != nil {
		return fmt.Errorf("Bucket(%q).IAM().Policy: %w", bucketName, err)
	}
	// Other valid prefixes are "serviceAccount:", "user:"
	// See the documentation for more values.
	// https://cloud.google.com/storage/docs/access-control/iam
	identity := "group:[e-mail]"
	var role iam.RoleName = "roles/storage.objectViewer"

	policy.Add(identity, role)
	if err := bucket.IAM().SetPolicy(ctx, policy); err != nil {
		return fmt.Errorf("Bucket(%q).IAM().SetPolicy: %w", bucketName, err)
	}
	// NOTE: It may be necessary to retry this operation if IAM policies are
	// being modified concurrently. SetPolicy will return an error if the policy
	// was modified since it was retrieved.
	fmt.Fprintf(w, "Added %v with role %v to %v\n", identity, role, bucketName)
	return nil
}

Java

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage Java API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

import com.google.cloud.Binding;
import com.google.cloud.Policy;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class AddBucketIamMember {
  /** Example of adding a member to the Bucket-level IAM */
  public static void addBucketIamMember(String projectId, String bucketName) {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // For more information please read:
    // https://cloud.google.com/storage/docs/access-control/iam
    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

    Policy originalPolicy =
        storage.getIamPolicy(bucketName, Storage.BucketSourceOption.requestedPolicyVersion(3));

    String role = "roles/storage.objectViewer";
    String member = "group:[e-mail]";

    // getBindingsList() returns an ImmutableList and copying over to an ArrayList so it's mutable.
    List<Binding> bindings = new ArrayList(originalPolicy.getBindingsList());

    // Create a new binding using role and member
    Binding.Builder newMemberBindingBuilder = Binding.newBuilder();
    newMemberBindingBuilder.setRole(role).setMembers(Arrays.asList(member));
    bindings.add(newMemberBindingBuilder.build());

    // Update policy to add member
    Policy.Builder updatedPolicyBuilder = originalPolicy.toBuilder();
    updatedPolicyBuilder.setBindings(bindings).setVersion(3);
    Policy updatedPolicy = storage.setIamPolicy(bucketName, updatedPolicyBuilder.build());

    System.out.printf("Added %s with role %s to %s\n", member, role, bucketName);
  }
}

Node.js

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage Node.js API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The role to grant
// const roleName = 'roles/storage.objectViewer';

// The members to grant the new role to
// const members = [
//   'user:[e-mail]',
//   'group:[e-mail]',
// ];

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function addBucketIamMember() {
  // Get a reference to a Google Cloud Storage bucket
  const bucket = storage.bucket(bucketName);

  // For more information please read:
  // https://cloud.google.com/storage/docs/access-control/iam
  const [policy] = await bucket.iam.getPolicy({requestedPolicyVersion: 3});

  // Adds the new roles to the bucket's IAM policy
  policy.bindings.push({
    role: roleName,
    members: members,
  });

  // Updates the bucket's IAM policy
  await bucket.iam.setPolicy(policy);

  console.log(
    `Added the following member(s) with role ${roleName} to ${bucketName}:`
  );

  members.forEach(member => {
    console.log(`  ${member}`);
  });
}

addBucketIamMember().catch(console.error);

PHP

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage PHP API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

use Google\Cloud\Storage\StorageClient;

/**
 * Adds a new member / role IAM pair to a given Cloud Storage bucket.
 *
 * @param string $bucketName The name of your Cloud Storage bucket.
 *        (e.g. 'my-bucket')
 * @param string $role The role to which the given member should be added.
 *        (e.g. 'roles/storage.objectViewer')
 * @param string[] $members The member(s) to be added to the role.
 *        (e.g. ['group:[e-mail]'])
 */
function add_bucket_iam_member(string $bucketName, string $role, array $members): void
{
    $storage = new StorageClient();
    $bucket = $storage->bucket($bucketName);

    $policy = $bucket->iam()->policy(['requestedPolicyVersion' => 3]);
    $policy['version'] = 3;

    $policy['bindings'][] = [
        'role' => $role,
        'members' => $members
    ];

    $bucket->iam()->setPolicy($policy);

    printf('Added the following member(s) to role %s for bucket %s' . PHP_EOL, $role, $bucketName);
    foreach ($members as $member) {
        printf('    %s' . PHP_EOL, $member);
    }
}

Python

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage Python API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

from google.cloud import storage


def add_bucket_iam_member(bucket_name, role, member):
    """Add a new member to an IAM Policy"""
    # bucket_name = "your-bucket-name"
    # role = "IAM role, e.g., roles/storage.objectViewer"
    # member = "IAM identity, e.g., user: [e-mail]"

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)

    policy = bucket.get_iam_policy(requested_policy_version=3)

    policy.bindings.append({"role": role, "members": {member}})

    bucket.set_iam_policy(policy)

    print(f"Added {member} with role {role} to {bucket_name}.")

Ruby

To learn how to install and use the client library for Cloud Storage, see Cloud Storage client libraries. For more information, see the Cloud Storage Ruby API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries.

def add_bucket_iam_member bucket_name:
  # The ID of your GCS bucket
  # bucket_name = "your-unique-bucket-name"

  require "google/cloud/storage"

  storage = Google::Cloud::Storage.new
  bucket = storage.bucket bucket_name

  role   = "roles/storage.objectViewer"
  member = "group:[e-mail]"

  bucket.policy requested_policy_version: 3 do |policy|
    policy.bindings.insert role: role, members: [member]
  end

  puts "Added #{member} with role #{role} to #{bucket_name}"
end

JSON

Note: This method replaces any existing IAM policy set on a bucket. If you want to modify the existing policy, retrieve it with the getIamPolicy method, edit it as needed, and use the edited version in your setIamPolicy request.

1. Have gcloud CLI installed and initialized, which lets you generate an access token for the Authorization header.

2. Create a JSON file that contains the following information:

   {
   "bindings":[
     {
       "role": "roles/storage.legacyBucketWriter",
       "members":[
         "YOUR_AGENT_EMAIL"
       ]
     }
   ]
   }

   Where:

   • YOUR_AGENT_EMAIL is the agent account email you copied in Find the service agent's email.

3. Use cURL to call the JSON API with a PUT setIamPolicy request:

   curl -X PUT --data-binary @JSON_FILE_NAME \
   -H "Authorization: Bearer OAUTH2_TOKEN" \
   -H "Content-Type: application/json" \
   "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/iam"

   Where:

   • JSON_FILE_NAME is the path for the file that you created in Step 2.
   • OAUTH2_TOKEN is the access token that you generated in Step 1.
   • BUCKET_NAME is the name of the bucket to which you want to give the principal access. For example, my-bucket.

For more information on assigning IAM roles to Cloud Storage resources, see the Cloud Storage IAM documentation.