Transfer Cloud Storage hierarchical namespace buckets

Cloud Storage hierarchical namespace stores data in a logical file system structure, using folders to organize objects. For details, see Hierarchical namespace.

Storage Transfer Service can transfer to and from Cloud Storage buckets that use hierarchical namespace. To successfully transfer the folders and folder metadata, additional permissions must be granted to the Storage Transfer Service service agent.

Required permissions

In addition to the standard permissions required for a transfer, the Storage Transfer Service service agent must be granted the following IAM permissions.

When the source is a hierarchical namespace bucket:

  • storage.folders.list on the source bucket. This permission is included in the Storage Object Viewer (roles/storage.objectViewer) role that is required for all transfers. No additional role is required.

When the destination is a hierarchical namespace bucket:

  • storage.folders.create on the destination bucket. This permission is included in the Storage Object User (roles/storage.objectUser) role. Grant this role in addition to the roles listed in Agentless transfer permissions.

For instructions on adding roles to the service agent, see:

Transfer behavior

When transferring between two hierarchical namespace buckets:

  • Objects and object metadata are transferred.
  • Folders and folder metadata are transferred.
  • Deletion of folders at source or sink is not supported.

When transferring from a hierarchical namespace bucket to a flat namespace bucket:

  • Objects and object metadata are transferred.
  • Folders are transferred as object prefixes. For example, object1 in folderA is transferred to the destination as folderA/object1.
  • Folder metadata is not transferred.
  • Deletion of folders at source is not supported.

When transferring from a flat namespace bucket to a hierarchical namespace bucket:

  • Objects and object metadata are transferred.
  • Object prefixes are used to create any missing folders. For example, when transferring folderA/object1, Storage Transfer Service creates a new folder called folderA and transfers object1 into that folder.
  • Deletion of folders at sink is not supported.