출시 노트

이 페이지에서는 Storage Transfer Service의 프로덕션 업데이트를 설명합니다. Storage Transfer Service 개발자는 정기적으로 이 목록에서 새로운 공지 사항을 확인하는 것이 좋습니다.

Google Cloud 출시 노트 페이지에서 모든 Google Cloud의 최신 제품 업데이트를 확인할 수 있습니다.

최신 제품 업데이트를 받으려면 피드 리더에 이 페이지의 URL을 추가하거나 피드 URL을 다음과 같이 직접 추가하세요. https://cloud.google.com/feeds/storagetransfer-release-notes.xml

July 23, 2024

We've released a lighter weight (~50% reduction in image size) and more secure version of the transfer agent container image. If you're running agents that were installed on or before June 17th, 2024, we highly recommend that you delete those agents and install new agents into your agent pool. Any existing transfers using that pool will resume once the new agents are installed.

May 17, 2024

Storage Transfer Service now supports transfers from Amazon S3 over a Google-managed private network. Transfer jobs that select this option pay no AWS egress fees; instead, a flat per-GiB rate is charged by Google Cloud. This allows you to transfer data at a potentially lower overall cost.

Learn more about egress options for S3 transfers, including the managed private network.

Cloud Logging for Storage Transfer Service now supports transfers involving POSIX file systems.

See Cloud Logging for Storage Transfer Service for more details.

April 17, 2024

Storage Transfer Service has added support for Shared Keys as an authentication method when transferring from Microsoft Azure Storage.

To use an Azure Shared Key, you must store the key value in Secret Manager. See Save your Microsoft credentials in Secret Manager for details.

March 13, 2024

Support for transfers from cloud and on-premises Hadoop Distributed File System (HDFS) sources is now generally available (GA).

HDFS support allows for use cases such as migrating from on-premises storage to Cloud Storage, archiving data to free up on-premises storage space, replicating data to Google Cloud for business continuity, or transferring data to Google Cloud for analysis and processing.

See Transfer from HDFS to Cloud Storage for details.

February 22, 2024

Storage Transfer Service has added preview support for transferring managed folders between Cloud Storage buckets. Permissions on managed folders are copied between buckets when using this option.

See Transfer Cloud Storage managed folders for details.

November 30, 2023

Storage Transfer Service supports transfers from cloud and on-premises Hadoop Distributed File System (HDFS) sources. This feature is in limited release; complete the form linked from the user guide at Transfer from HDFS to Cloud Storage to request access.

This new feature supports use cases such as migrating from on-premises storage to Cloud Storage, archiving data to free up on-premises storage space, replicating data to Google Cloud for business continuity, or transferring data to Google Cloud for analysis and processing.

November 01, 2023

Storage Transfer Service has updated transfer agent behavior when transferring to Cloud Storage. To align with Cloud Storage auto-scaling, agents now gradually ramp up the number of requests being made to Cloud Storage. Customers who transfer many small files may notice initially slower transfer speeds while the requests are ramping up, but increased performance across the duration of the transfer.

October 12, 2023

You can now transfer data from Amazon S3 via your CloudFront domain. Learn more.

July 25, 2023

Event-driven transfers are now generally available (GA).

Event-driven transfers provide serverless, real-time replication from AWS S3 to Cloud Storage, and between Cloud Storage buckets. With this new capability, you can accelerate your event-driven analytics pipeline, enable automatic replication across Cloud Storage buckets, create a backup copy of data in a different region or project, or perform live migration.

Learn more about Event-driven transfers.

June 27, 2023

Transfers from Amazon S3 no longer require s3:GetBucketLocation permission on the source bucket.

For a full list of required permissions, see Configure access to a source: Amazon S3.

June 12, 2023

Storage Transfer Service now offers Preview support for providing a Secret Manager secret when creating transfer jobs from AWS S3 or Azure storage.

Secret Manager provides strong encryption, role-based access control, and audit logging to protect your secrets.

For details, see the Secret Manager section of the following documents:

• Configure access to a source: Amazon S3
• Configure access to a source: Microsoft Azure Storage

Cloud Monitoring for Storage Transfer Service is now Generally Available (GA). This integration allows you to monitor the number of objects and data being transferred and to compute transfer speeds. This GA launch also adds monitoring of errors and error codes.

See Monitor transfer jobs for details.

June 06, 2023

We discovered a security vulnerability in the Storage Transfer Service agent container. We've fixed this issue with a container update that is more secure.

If you're running agents that were installed on or before February 17, 2023, you should follow the instructions in the Action required email sent to your account email address to update the container image.

Agents installed after February 17, 2023 do not need to be updated.

April 25, 2023

Storage Transfer Service now publishes the IP ranges from which it makes requests to your AWS or Azure storage resources when performing a transfer. This allows you to restrict your resources by IP, and still allow Storage Transfer Service access.

For details, see the IP restrictions section of the following documents:

• Configure access to a source: Amazon S3
• Configure access to a source: Microsoft Azure Storage

April 14, 2023

Storage Transfer Service can now optionally preserve UID, GID, and mode metadata for folders, and recreate empty folders, when transferring between file systems.

See Metadata preservation for details.

April 11, 2023

Transfers from S3-compatible storage to Cloud Storage are now generally available (GA). This feature builds on support for Multipart upload and List Object V2, which makes Cloud Storage suitable for running applications written for the S3 API.

With this new feature, customers can seamlessly copy data from self-managed object storage to Google Cloud Storage. For customers moving data from AWS S3 to Cloud Storage, this feature provides an option to control network routes to Google Cloud, resulting in considerably lower egress charges.

See Transfer from S3-compatible sources for details.

April 06, 2023

Support for Manifest in Storage Transfer Service is now generally available (GA). You can use Manifest to transfer a specific list of objects, object versions, and files from cloud and on-premises sources. Programmatic users can use the output of an upstream operation generating a list of files and objects as an input for Storage Transfer Service to act upon.

January 17, 2023

Temporary pricing for Storage Transfer Service has expired; refer to Pricing for current charges.

Customers under contract can reach out to Google Cloud representatives for details.

December 19, 2022

Storage Transfer Service now offers Preview support for tracking progress of a Transfer Job using Cloud Monitoring, allowing you to monitor the number of objects and amount of data copied by Storage Transfer Service in near real-time.

See Monitor transfer jobs for details.

December 15, 2022

Storage Transfer Service now offers GA Support for transferring data between file systems, including on-premises file systems and Filestore instances. This allows you to use the Transfer Service API, gcloud command line tool, or the Cloud console to migrate data from a self-managed file system to Filestore; accelerate data transfer from an on-premise file system to a cloud file system; or move data between on-premises systems.

You can also transfer specific files or objects using a manifest for file system to file system transfers.

December 09, 2022

Storage Transfer Service offers Preview support for event-driven transfers - serverless, real-time replication from AWS S3 to Cloud Storage, and between Cloud Storage buckets. With this new capability, you can accelerate your event-driven analytics pipeline, enable automatic replication across Cloud Storage buckets, create a backup copy of data in a different region or project, or perform live migration.

Learn more about Event-driven transfers.

October 07, 2022

Multipart upload for transfers originating from a file system is now generally available (GA) and enabled by default.

Multipart uploads can speed up transfers that include large files. You must grant additional permissions on the destination or intermediate bucket; see Permissions for file system transfers for details.

October 03, 2022

The Storage Transfer Service REST API now provides a transferJobs.delete method. Deleting a transfer job stops all current and future operations associated with the job, and removes the job from all transfer job listings. Learn more on the Manage transfers page.

Support for exporting data from Cloud Storage to a file system is now generally available (GA).

You can use this bidirectional data movement capability to move data in and out of Cloud Storage, on-premises clusters, and edge locations including Google Distributed Cloud. To get started, see Requirements for file system transfers.

September 07, 2022

Storage Transfer Service now offers Preview support for moving data from S3-compatible storage to Cloud Storage. This feature builds on recent Cloud Storage launches, namely support for Multipart upload and List Object V2, which makes Cloud Storage suitable for running applications written for the S3 API. With this new feature, customers can seamlessly copy data from self-managed object storage to Google Cloud Storage. For customers moving data from AWS S3 to Cloud Storage, this feature provides an option to control network routes to Google Cloud, resulting in considerably lower egress charges.

See Transfer from S3-compatible sources for details.

August 30, 2022

Storage Transfer Service now offers preview support for multipart uploads for transfers originating from a file system, if the destination or intermediate bucket uses the Standard storage class.

Multipart uploads can speed up transfers that include large files. You must grant additional permissions on the destination or intermediate bucket; see Permissions for file system transfers for details.

July 18, 2022

Detailed logging for objects copied between AWS S3, Azure Blob Storage, ADLS Gen 2, and Cloud Storage with Storage Transfer Service is now generally available (GA).

With detailed logs of individual objects available in Cloud Logging, you can verify what was transferred and perform additional data integrity checks. This launch simplifies monitoring, reporting, and troubleshooting. Read Cloud Logging for Storage Transfer Service for details.

June 29, 2022

Expanded overwrite options are new generally available (GA). The overwriteWhen field can be used to specify whether data that already exists in the destination should be overwritten always, never, or only when ETags and checksum values indicate that the file has changed.

Metadata preservation options are now generally available (GA). This includes the option of preserving POSIX attributes and symlinks when transferring to, from, and between POSIX filesystems; as well as object ACLs, CMEK, temporary holds, and object creation time when transferring between Cloud Storage buckets.

See Metadata preservation for details.

June 06, 2022

Storage Transfer Service now offers a merged, unified console experience for cloud and file system transfers. All transfer jobs irrespective of source can be tracked though a single interface. This launch simplifies job creation, monitoring, and troubleshooting.

April 19, 2022

Storage Transfer Service now provides more options for when to overwrite files that already exist in the destination. The new overwriteWhen field provides three options, that apply to all transfers, including those to or from file systems.

• NEVER provides defense in depth for archival cases, where data is not intended to be overwritten. Users no longer need to rely on a retention policy to protect their data.
• DIFFERENT uses ETags and checksum values to only overwrite a file if the contents have changed.
• ALWAYS overwrites any existing files with the same name. Avoids LIST operations on the destination when transferring into Cloud Storage.

April 12, 2022

Storage Transfer Service now offers a predefined role to simplify permission assignment to transfer agents. The roles/storagetransfer.transferAgent role grants a minimum set of permissions required for the service to communicate with agents and eliminates the need to assign each permission individually.

The role should be granted to the user account or service account being used to authenticate the agents. See the On-premises agent account documentation for more details.

March 30, 2022

Creating and managing data transfers with the gcloud command-line tool is now generally available (GA).

You can use gcloud commands to perform agent installation, manage agent pool lifecycles, and orchestrate transfer jobs. This launch simplifies writing scripts to automate transfer workflow.

The GA launch adds support for transfers between file systems, metadata preservation, and manifests. It also introduces the gcloud transfer authorize command to inspect and grant required permissions for transfers.

March 23, 2022

Storage Transfer Service now enforces the Resource Location Restriction, which is part of the Org Policy Service. This allows an organization to define the allowed regions in which location-based Google Cloud resources, such as transfer jobs, can be created.

To learn how Storage Transfer Service chooses the region in which to run a transfer job, refer to Location of Storage Transfer Service jobs.

March 14, 2022

Between April 2, 2022 and December 31, 2022, transfers using Storage Transfer Service will not result in Cloud Storage charges for certain transfer scenarios.

For more details, refer to the Storage Transfer Service pricing page.

March 08, 2022

Storage Transfer Service now offers more control over preserving metadata when transferring between Cloud Storage buckets. Choose to retain or discard metadata including object ACLs, customer-managed encryption keys (CMEK), temporary holds, and object creation time. In addition, storage class can be set to any supported value, allowing you to change storage class at scale.

For details, refer to Metadata preservation.

March 03, 2022

Storage Transfer Service now supports Cloud Client Libraries, which are the recommended option for accessing Cloud APIs programmatically. This launch significantly reduces the amount of code you need to write; see Getting started and Migrating to the Cloud Client Library for more details.

February 08, 2022

Support for agent pools is now generally available (GA) .

You can use agent pools to create isolated groups of agents as a source or sink entity in a transfer job. This enables you to transfer data from multiple data centers and filesystems concurrently, without creating multiple projects for a large transfer spanning multiple filesystems and data centers.

January 27, 2022

Storage Transfer Service now offers Preview support for moving data between two filesystems and keeping them in sync on a periodic schedule. Transfers can be orchestrated by creating a transfer job through the API or gcloud command-line tool. See Transfer data between POSIX file systems for details.

This launch offers a managed way to migrate from a self-managed filesystem to Filestore. If you have on-premises systems generating massive amounts of data that needs to be processed in Google Cloud, you can now use Storage Transfer Service to accelerate data transfer from an on-prem filesystem to a cloud filesystem.

Storage Transfer Service now offers Preview support for preserving POSIX attributes and symlinks when transferring to, from, and between POSIX filesystems. Attributes include the user ID of the owner, the group ID of the owning group, the mode or permissions, the modification time, and the size of the file. See Metadata preservation for details.

December 13, 2021

Integration with AWS Security Token Service is now generally available (GA) for Storage Transfer Service.

Security conscious customers can use Storage Transfer Service to perform transfers from AWS S3 without passing long-term AWS S3 credentials, which have to be rotated or explicitly revoked when they are no longer needed. Refer to the Amazon S3 > Federated Identity tab when setting up access to your data source.

Creating and managing data transfers with the gcloud command-line tool is now available in Preview. You can use gcloud commands to perform agent installation, manage agent pool lifecycles, and orchestrate transfer jobs. This launch simplifies writing scripts to automate transfer workflow.

December 06, 2021

Storage Transfer Service now offers Preview support for detailed logging for objects copied between AWS S3, Azure Blob, ADLS Gen 2, and Cloud Storage. With detailed logs of individual objects available in Cloud Logging, you can verify what was transferred and perform additional data integrity checks. This launch simplifies monitoring, reporting, and troubleshooting. Read Configure transfer logs for details.

December 03, 2021

Support for transferring data from Azure ADLS Gen 2 to Cloud Storage with Storage Transfer Service is now generally available (GA).

December 02, 2021

The Storage Transfer Service API for managing on-premises transfers is now generally available (GA). Customers can use RESTful APIs to automate their on-prem to Cloud transfer workflow.

For more information, see Managing Transfer for on-premises jobs.

Storage Transfer Service now offers preview support for Manifest. You can use Manifest to transfer a specific list of objects, object versions, and files from cloud and on-premises sources. Programmatic users can use the output of an upstream operation generating a list of files and objects as an input for Storage Transfer Service to act upon.

November 04, 2021

Storage Transfer Service now offers Preview support for agent pools. You can use agent pools to create isolated groups of agents as a source or sink entity in a transfer job. This enables you to transfer data from multiple data centers and filesystems concurrently, without creating multiple projects for a large transfer spanning multiple filesystems and data centers.

November 01, 2021

Storage Transfer Service now offers Preview support for exporting data from Cloud Storage to a POSIX file system. You can use this bidirectional data movement capability to move data in and out of Cloud Storage, on-premises clusters, and edge locations including Google Distributed Cloud. For more information, see Download data from Cloud Storage.

July 26, 2021

Storage Transfer Service now offers Public Preview support for managing on-premises transfer via API. Customers can use RESTful APIs to automate their on-prem to Cloud transfer workflow.

For more information, see Managing Transfer for on-premises jobs.

July 01, 2021

Transfer service for on-premises data support for delete from source is now Generally Available. For more information, see Data consistency details.

Storage Transfer Service offers Preview for Integration with AWS Security Token Service. Security conscious customers can now use Storage Transfer Service to perform transfers from AWS S3 without passing any security credentials. This release will alleviate the security burden associated with passing long-term AWS S3 credentials, which have to be rotated or explicitly revoked when they are no longer needed. Refer Amazon Web Services (AWS) S3 Federated Identity credentials when setting up access to your data source.

June 18, 2021

Storage Transfer Service offers Preview support for transferring data from Azure ADLS Gen 2 to Cloud Storage.

June 01, 2021

As of June 1 2021 all Transfer service for on-premises projects added a per-project service account. Actions are required before September 1, 2021 to ensure minimal disruptions to your transfers. Please check your email for detailed instructions.

April 09, 2021

Obtaining the status of the latest transfer operation is in now generally available.

March 31, 2021

Storage Transfer Service support for hourly schedules and modifying a job's schedule is now generally available. For more information, see Schedule.

Storage Transfer Service support for specifying source and destination paths when creating a transfer is now generally available. For more information, see Specifying source and destination paths.

Storage Transfer Service now supports event-driven transfer workflows, a new capability that lets customers start Transfer jobs at once based on an event trigger. Customers can create Transfer jobs without specifying a schedule and start a TransferOperation using transferJobs.run from an existing TransferJob configuration.

January 20, 2021

Storage Transfer Service offers Preview support for specifying source and destination paths when creating a transfer. For more information, see Specifying source and destination paths.

January 19, 2021

Transfer service for on-premises data has updated the cost to $0.0125 per GB transferred to the destination successfully. For more information, see Transfer for on-premises pricing details.

November 09, 2020

Storage Transfer Service offers Preview support for hourly schedules and modifying a job's schedule. For more information, see Schedule.

October 13, 2020

Obtaining the status of the latest transfer operation is in Preview.

September 01, 2020

Storage Transfer Service integration for VPC Service Controls is now generally available.

July 31, 2020

Transfers from Microsoft Azure Blob Storage are now generally available.

March 31, 2020

Beta stage integration with VPC Service Controls

Transfer service on-premises: Beta stage integration with VPC Service Controls

March 23, 2020

Transfer Service for on-premises data is now generally available. For more information, see Transfer Service for on-premises data Overview.

Storage Transfer Service now offers Beta support for transfers from Microsoft Azure Blob Storage.

March 20, 2020

Storage Transfer Service supports Pub/Sub notifications for transfer jobs for external cloud provider to Cloud Storage transfers.

March 16, 2020

Storage Transfer Service now allows you to specify files to transfer based on their last modification times.

March 09, 2020

Storage Transfer Service supports custom job IDs.

December 04, 2019

Storage Transfer Service for on-premises is now in beta.

September 05, 2019

API documentation update to clarify transferJobs API descriptions, specifically: scheduleStartDate, scheduleEndDate, and startTimeOfDay.

August 05, 2019

Updates to API documentation.

March 20, 2019

Adding new IAM roles for Storage Transfer Service.

August 06, 2018

Updated documentation for configuring access to the data sink.

December 07, 2017

Documentation updates for IAM permissions.

September 18, 2017

Documentation update for access control.

September 07, 2017

Updates to API documentation.

August 31, 2016

Documentation update that provides clearer instructions for configuring a Cloud Storage data source.

August 25, 2016

Documentation update to add information for configuring access to a data sink and supported AWS regions.

June 15, 2016

API documentation update.

April 25, 2016

Documentation updates and content reorganization.

April 04, 2016

Updates to API documentation for scheduling jobs.

February 23, 2016

Documentation update to support scheduling recurring transfers with Cloud Console.

December 12, 2015

Updated Getting Started document for changes to the Create Transfer workflow.

September 23, 2015

Several updates to API documentation.

August 06, 2015

Documentation includes instructions for enabling API for non-UI users.

August 05, 2015

API documentation now includes "Try It" sections.

July 28, 2015

Documentation update for using prefixes in Storage Transfer Service user interface.

July 22, 2015

Storage Transfer Service, a cloud-to-cloud transfer tool, launched.