Transfer Cloud Storage managed folders

Preview — Cloud Storage managed folders

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Cloud Storage managed folders provide fine-grained access control to objects in Cloud Storage buckets. Permissions can be set at a folder level within buckets that use universal bucket-level access. When transferring objects between Cloud Storage buckets with Storage Transfer Service, these managed folder permissions can be retained.

Limitations

The following limitations apply to transfers of managed folders:

• You must use the REST API to create the transfer.
• The destination bucket must use uniform bucket-level access.
• Managed folder transfers do not support the deleteObjectsUniqueInSink or deleteObjectsFromSourceAfterTransfer options.
• There must be no IAM Conditions on the destination bucket or its project that use the bucket resource type (storage.googleapis.com/Bucket) or the object resource type (storage.googleapis.com/Object). If any bucket within a project has an IAM Condition that uses either of these resource types, managed folders cannot be transferred to any of the buckets within that project, even if the condition is later removed.
• Event-driven transfers are not supported.
• Manifest transfers are not supported.
• Managed folder operations are not logged by Cloud Logging. Logging of objects is supported.

IAM permissions

The following Google Cloud Identity and Access Management (IAM) permissions are required by the Google-managed service account.

On the source bucket:

• storage.managedFolders.getIamPolicy
• storage.managedFolders.list
• storage.managedFolders.get

On the destination bucket:

• storage.managedFolders.setIamPolicy
• storage.managedFolders.list
• storage.managedFolders.create

These are in addition to the standard permissions required by Storage Transfer Service:

• Configure access to a source: Cloud Storage
• Configure access to a sink: Cloud Storage

To grant the required managed folder permissions, create a custom role with only the permissions required and assign it to the Google-managed service account, either at the bucket level (recommended) or at the project level. See Security considerations for additional info.

Create a managed folder transfer

To create a transfer containing a managed folder, specify managedFolderTransferEnabled: true in your transferSpec:

POST https://storagetransfer.googleapis.com/v1/transferJobs
{
  "name": "transferjobs/NAME",
  "projectId": "PROJECT_ID",
  "transferSpec": {
    "gcsDataSource": {
      "bucketName": "SOURCE_BUCKET",
      "managedFolderTransferEnabled": true
    },
    "gcsDataSink": {
      "bucketName": "DESTINATION_BUCKET"
    }
  },
  "status": "ENABLED"
}

See Create transfers for details on creating a transfer using the REST API, or refer to the transferJobs.create reference.

Security considerations

Granting managed folder permissions to a Google-managed service account enables the account to modify IAM policies on destination folders, or on all folders if the role is granted at the project level. This poses a security risk: a user with job edit permissions could exploit this to grant privileges to a malicious actor. To mitigate this risk, consider isolating managed folder transfers within a dedicated Google Cloud project.

Troubleshooting

For help creating and managing managed folders, refer to the Troubleshooting page.