This page describes Storage Transfer Service.
Other Google Cloud transfer options include:
Transfer Appliance for moving offline data, large data sets, or data from a source with limited bandwidth
BigQuery Data Transfer Service to move data from SaaS applications to BigQuery.
What is Storage Transfer Service?
Storage Transfer Service is a product that enables you to:
Move or backup data to a Cloud Storage bucket either from other cloud storage providers or from your on-premises storage.
Move data from one Cloud Storage bucket to another, so that it is available to different groups of users or applications.
Periodically move data as part of a data processing pipeline or analytical workflow.
Storage Transfer Service provides options that make data transfers and synchronization easier. For example, you can:
Schedule one-time transfer operations or recurring transfer operations.
Delete data source objects after transferring them.
Schedule periodic synchronization from a data source to a data sink with advanced filters based on file creation dates, file-name filters, and the times of day you prefer to import data.
Storage Transfer Service does the following by default:
Storage Transfer Service copies a file from the data source if the file doesn't exist in the data sink or if it differs between the version in the source and the sink.
Retains files in the source after the transfer operation.
Permissions and role requirements for Storage Transfer Service
Storage Transfer Service uses Cloud Identity and Access Management to control and manage access. For more information about Cloud IAM, see Cloud IAM Overview.
To use Storage Transfer Service, you must be granted at least one of the following Cloud IAM roles, depending on the types of duties that you are performing:
|Access Type||IAM roles|
|Full access||Storage Transfer Admin|
|Submitting transfers||Storage Transfer User|
|Viewing or listing transfer jobs and operations||Storage Transfer Viewer
The project you use to create a transfer job doesn't have to be associated with the buckets that act as a data source or data sink, but additional permissions are required to configure and use data sources and data sinks.
For more information about Storage Transfer Service roles and permissions, see Configuring access to data sources and sinks.
There are a number of ways that you can work with Storage Transfer Service:
Use Google Cloud Console to create and manage transfer jobs. This is often the easiest and quickest way to start using Storage Transfer Service. For more information, see Creating and managing transfers with Console.
Use REST APIs to work directly with Storage Transfer Service API. See Creating a Storage Transfer Service client for more information about enabling the API and obtaining authentication tokens to use your requests.
Should you use gsutil or Storage Transfer Service?
The gsutil command-line tool also enables you to transfer data between Cloud Storage and other locations. While you can use gsutil to work with Amazon S3 buckets and transfer data from Amazon S3 to Cloud Storage, Storage Transfer Service is recommended for this use case.
Follow these rules of thumb when deciding whether to use gsutil or Storage Transfer Service:
|Transferring from another cloud storage provider||Use Storage Transfer Service|
|Transferring less than 1 TB from on-premises||Use gsutil|
|Transferring more than 1 TB from on-premises||Use Transfer Service for on-premises data(beta)|
Use this guidance as a starting point. The specific details of your transfer scenario will also help you determine which tool is more appropriate.
Service Level Agreement
Storage Transfer Service currently does not provide an SLA, and some performance fluctuations may occur. For example, we do not provide SLAs for transfer performance or latency.