Agentless transfer permissions

Agentless transfers include transfers within Cloud Storage, or from Amazon S3, Azure Blob Storage or Data Lake Storage Gen2, or URL lists of public objects. They do not require transfer agents or agent pools.

When transferring data between cloud providers in an agentless transfer, you must grant permissions to two entities:

  • The user or user-managed service account that's creating or managing the transfer.
  • A Google-managed service account that's automatically created the first time you interact with Storage Transfer Service.

In addition, you must configure access to the source files on AWS S3, Microsoft Azure, or in Cloud Storage.

To learn how to grant IAM roles, see Granting, changing, and revoking access to resources.

Simplest method of granting permissions

The gcloud CLI can be used to grant the required permissions to the user / user-managed service account and to the Google-managed service account. These permissions allow the user to create, edit, and delete transfer jobs, and set or modify bandwidth limits.

If these permissions are too broad for your organization's policies, refer to the later sections in this document for the minimum permissions required by Storage Transfer Service.

To inspect existing permissions and print out any missing roles, run the following command:

gcloud transfer authorize

To automatically apply those roles, use the --add-missing flag:

gcloud transfer authorize --add-missing

To grant permissions to a user-managed service account, pass the service account key file:

gcloud transfer authorize --add-missing --creds-file=path/to/key.json

For instructions on creating a service account, see Creating and managing service accounts.

The command grants the following permissions.

  • To the user / user-managed service account:

    • roles/owner
    • roles/storagetransfer.admin
    • roles/storagetransfer.transferAgent
    • roles/storage.objectAdmin
    • roles/pubsub.editor

  • To the Google-managed service account:

    • roles/storage.admin
    • roles/storagetransfer.serviceAgent

For instructions on installing the gcloud CLI, see the gcloud quickstart.

User or user-managed service account permissions

This section covers the roles required for the accounts managing and executing transfers. Your organization's requirements will dictate the exact roles for each persona; this section assumes that you will create an administrator and a user.

Administrator accounts

Storage Transfer Service administrator accounts set bandwidth usage limits and delete transfer jobs.

To set up an administrator account, assign the following IAM permissions and roles:

Role / Permission What it does Notes
resourcemanager.projects.getIamPolicy This permission is used to confirm that the Google-managed service account has the required permissions for a transfer. To grant this permission, grant the Role Viewer (roles/iam.roleViewer) predefined role, or create a custom role with this single permission and grant the custom role.
Storage Transfer Admin (roles/storagetransfer.admin) Enables administrative actions in the transfer project, such as project set up. For a detailed listing of permissions granted, see Storage Transfer Service Predefined roles.

User accounts

Storage Transfer Service user accounts are used to create and execute transfers. These accounts typically don't have access to delete transfer jobs.

A user account can be a Google Cloud console user, or a service account. If you're using a service account, the method you use to pass the credentials to Storage Transfer Service varies depending on the interface you use.

To set up a user account, assign the following permissions and roles to the account:

Role / Permission What it does Notes
resourcemanager.projects.getIamPolicy Used to confirm that the Google-managed service account has the required Pub/Sub permissions for a transfer. To grant this permission, grant the Role Viewer (roles/iam.roleViewer) predefined role, or create a custom role with this single permission and grant the custom role.
Storage Transfer User (roles/storagetransfer.user) Enables the user to create, get, update, and list transfers. For a detailed listing of permissions granted, see Storage Transfer Service Predefined roles.

Google-managed service account permissions

Storage Transfer Service uses a Google-managed service account to move your data. This service account is automatically created the first time you create a transfer job, call googleServiceAccounts.get, or visit the job creation page in the Google Cloud console.

The service account's format is typically project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com. To retrieve your service account identifier, use the googleServiceAccounts.get API call.

Automatically assigning roles

There are two ways to automatically assign the correct roles to the Google-managed service account:

  • Create your transfer using the Google Cloud console. The console automatically applies the required permissions.

  • Use gcloud transfer authorize --add-missing. See Simplest method of granting permissions.

Manually assigning roles

To allow the Google-managed service account access to resources needed to complete transfers, assign the following roles, or equivalent permissions, to the service account.

Role / Permission What it does Notes
Storage Legacy Bucket Writer (roles/storage.legacyBucketWriter) Enables Storage Transfer Service to read bucket metadata, list objects in the bucket, and write objects to the destination bucket. Grant on the Cloud Storage destination bucket.
Storage Object Viewer (roles/storage.objectViewer) Enables Storage Transfer Service to determine if a file has already been transferred to or from Cloud Storage. Grant on the Cloud Storage source bucket if your are transferring from Cloud Storage. If appropriate for your situation, you can grant the role on a project level to the project that Storage Transfer Service is running from.
Storage Legacy Bucket Reader (roles/storage.legacyBucketReader) Enables Storage Transfer Service to read Cloud Storage bucket metadata.

Grant on the Cloud Storage source bucket if your are transferring from Cloud Storage.

Cloud Storage legacy roles can only be granted at the bucket level.

See Grant the required permissions for instructions.

Cross-project Cloud Storage transfers

To transfer objects between Cloud Storage buckets in different projects, you must grant permission to a single Google-managed service account on both the source bucket and the destination bucket. The Google-managed service account belongs to the project from which you are creating the transfer.

The user or user-managed service account only needs permissions on the project that is creating the transfer.

To obtain the Google-managed service account's email, refer to Find the service agent's email.

On the source bucket, grant the following roles at the bucket level to the Google-managed service account:

  • Storage Legacy Bucket Reader
  • Storage Object Viewer

On the destination bucket, grant the following role at the bucket level to the same Google-managed service account:

  • Storage Legacy Bucket Writer

For instructions on granting roles at the bucket level, refer to the following pages:

Organizational restrictions

If your organization has applied organization policy constraints such as domain restrictions (constraints/iam.allowedPolicyMemberDomains) or disabling attachment of service accounts to resources in other projects (constraints/iam.disableCrossProjectServiceAccountUsage), these constraints must be updated before cross-project transfers can succeed.

You may also need to update any VPC service control settings to support cross-project transfers. See Configure VPC Service Controls for cloud object storage transfers.

AWS and Microsoft permissions

You must also configure access to your source files if they're in AWS S3 or Microsoft Azure Storage. The following documents detail the required steps: