Storage Transfer Service
Transfer data quickly and securely between object and file storage across Google Cloud, Amazon, Azure, on-premises, and more.
Enable your hybrid- or multicloud strategy
Move data from your private data centers, AWS, Azure, and Google Cloud, globally available through a single, easy-to-use interface.
Optimize infrastructure management and costs
Reduce your infrastructure costs by moving your data storage and application infrastructure to Google Cloud, while minimizing downtime.
Complete large-scale data transfers fast
Transfer petabytes of data from on-premises sources or other clouds over online networks— billions of files and 10s of Gbps. Optimize your network bandwidth and accelerate transfers with scale-out performance.
Migration to Google Cloud: transferring your large datasets
This document explores the process of moving data into Google Cloud, from planning a data transfer to using best practices in implementing a plan.
Storage Transfer Service overview
Access guides, references, and resources for learning more about Storage Transfer Service.
Managing transfers for on-premises jobs
This document describes how to create your transfer job, install transfer agents, and how to manage your transfer jobs.
Metadata preservation in Storage Transfer Service
This document describes metadata that is preserved when you use Storage Transfer Service for on-premises data to transfer data to Cloud Storage.
Performance, scaling and requirements for transfers
Understand the requirements for object size, object naming, and other scale and performance guidelines.
Explore common use cases for Storage Transfer Service
Move from your existing infrastructure to the cloud. The data you create and store on-premises demands time and significant resources to manage cost-effectively, securely, and reliably. Utilize our Storage Transfer Service solutions to migrate your workloads and datasets quickly to Google Cloud and start saving. Learn about all of Google Cloud’s migration services and solutions.
Storage Transfer Service enables you to push your media assets from other clouds or your private data center into multi-regional setups, designed for video streaming and frequently accessed content like websites and images. Moving data can enable more efficient content distribution and simpler publishing operations, with the help of products such as Cloud CDN.
With our data transfer services, you can schedule incremental syncs to enable disaster recovery for apps running in other clouds and on-premises, to meet your recovery goals. Learn about all of our backup and disaster recovery options, or read about how to use Cloud Storage for archiving your data.
Once transferred to Cloud Storage, your data is enabled for analytics or machine learning projects, via data pipelines that are hybrid or run across multiple clouds or cloud regions. Wherever your data is, you can take advantage of Google Cloud’s analytics and machine learning suite. Learn more about our innovative machine learning products and services.
Learn more about Storage Transfer Service features
|Monitoring and logging||Transfer service for on-premises data produces transfer logs so that you can verify the results of your transfer jobs and offers an option to monitor progress of transfer jobs via Cloud Monitoring. Storage Transfer Service could be configured to deliver Pub/Sub notification on transfer completion.|
|Data integrity||Storage Transfer Service uses metadata available from the source storage system, such as checksums and file sizes, to ensure that data written to Cloud Storage is the same data read from the source. In addition to data integrity provided by the TLS protocol, the Storage Transfer Service for on-premises data calculates a CRC32C checksum over each file that it copies as the file is being read.|
|Security and encryption||We use TLS encryption for HTTPs connections, through the public internet and private connections, and support transfers to Cloud Storage buckets protected by VPC Service Controls. The only exception to TLS encryption is if you specify an HTTP URL for a URL list transfer. If you are using interconnect, you can get another layer of security by accessing a private API endpoint.|
|Data transfer scheduling||You can schedule one-time transfer operations or recurring transfer operations. When transferring data from other cloud providers, you can schedule periodic synchronization with advanced filters based on file creation dates, file-names, and the times of day you prefer to import data. There is no manual intervention needed, and the results are recorded in the Cloud Console.|
|Bandwidth throttling||We’ve incorporated controls into Storage Transfer Service for on-premises data to avoid disruption of your day-to-day business operations. Configure a bandwidth limit for your Google Cloud project to limit the rate at which on-premises agents will copy files. The bandwidth limit is shared across all transfer jobs in your project.|
|Incremental transfer||By default all transfers begin by evaluating the data present at the source and destination. This process determines which source objects are new, updated, or deleted since the last transfer. With this initial step, you are able to minimize the amount of data that needs to be sent, use bandwidth effectively, and ensure transfers run quickly.|
|Flexibility, filtering, and controls||You can use includePrefix and excludePrefix when creating transfers to limit what objects Storage Transfer Service operates on. We also support modification-time-based data filtering options.|
|Dynamic scale out||When transferring data from on-premises locations, getting better performance is as easy as bringing up more Storage Transfer Service agents. No transfer config changes or re-submissions required. As soon as the agents are up, they will start performing transfer work.|
|Reliability and fault tolerance||We make reliability work out of the box. If some of your agents fail, the remaining ones will pick up their work. If all of your agents fail, no problem—as soon as you bring them back up, transfers will pick up where they had left off. No special recovery or retry logic required.|
|Simple pricing||When using Storage Transfer Service for on-premises data, we charge at a per-GB rate. When transferring data from other cloud providers, Storage Transfer Service is free of charge. After transferring data, you are charged for data stored on Cloud Storage as documented in Cloud Storage pricing, and external cloud providers' costs may also apply while you use Storage Transfer Service.|
|Transfer from a URL list||You can use Storage Transfer Service to transfer data from a list of public data locations to a Cloud Storage bucket.|
Storage Transfer Service pricing details
|Transfers using agents||$0.0125/GB|
|All other sources/sinks||Free|