To successfully relocate buckets, define your objectives and understand your bucket's usage before initiating a bucket relocation. The following sections describe the key planning steps.
Analyze the bucket characteristics
To estimate your bucket relocation time, analyze your bucket's characteristics and usage, considering the following factors:
At-rest bytes: The total amount of data stored in the bucket impacts storage costs and transfer time.
Replication: Replicating the bucket to other regions, either synchronously or asynchronously, impacts data availability, durability, and cost. For details, see Data availability and durability.
Data transfer: The amount of data transferred out of the bucket during the relocation impacts the data transfer cost calculations. To calculate your bucket's data transfer costs, see Cloud Storage pricing.
Usage patterns: Understanding bucket activity levels, or how busy the bucket is, through usage patterns helps you prevent unexpected conflicts during the relocation. To understand your bucket's usage patterns, you can analyze your logs. For details, see Usage logs and storage logs.
Bucket write operations: Frequent bucket write operations during the relocation process increase the cost and duration. To understand how frequently objects are being written to your bucket, see Overview of monitoring in Cloud Storage.
Define your relocation goals
Based on your analysis of the bucket characteristics, identify the reasons for moving your bucket. The following are common goals for relocating a bucket:
Cost management: Reduce storage costs by moving to a lower-cost region or minimize data transfer costs by moving data closer to its access location. You'll need to calculate your Cloud Storage and data transfer costs and compare them to potential costs in different locations. For details about calculating costs for your Cloud Storage, see Cloud Storage pricing.
Performance enhancement: Improve data access speed and application performance by relocating the bucket closer to users or applications. To do so, identify the geographic regions where performance is critical and relocate your buckets.
Reliability improvement: Enhance data durability and disaster recovery capabilities by using dual- or multi-region configurations.
Decide on the bucket location
Based on your analysis and goals, choose the most suitable storage location for the bucket you're relocating from the following options:
Single-region: Store data in a single region that is cost-effective for applications with users concentrated in one geographic area.
Dual-region: Maintain two copies of your data in two regions within the same continent, providing higher availability and disaster recovery capabilities within a specific geographic area.
Multi-region: Distribute data across multiple regions, offering the highest level of availability and durability.
To learn more about choosing a location, see Considerations for choosing a location.
Understand the factors that affect the relocation time
Several factors affect the relocation time, and understanding them can help to estimate the required time. While these factors offer a helpful starting point for planning and scheduling your relocation, actual relocation times might be longer or shorter than the estimated time. Therefore, when scheduling your relocation, add buffer time to account for potential delays. The following sections describe the factors that affect the relocation time.
Relocation service limits
The following table describes the limits that affect the relocation time:
| Factor | Value | Description |
|---|---|---|
| Maximum request rate per job | 10,000 Objects per second |
These are the number of copy requests the service can handle per second.
A higher request rate means more files can be moved simultaneously. If your bucket has many small files, a high request rate speeds up the migration. If you have only a few large files, this factor has less impact. |
| Maximum overall bandwidth per project | 10 GBps |
This is the maximum speed or bandwidth at which you can transfer data for
a single project within a source location. If you're moving multiple buckets within the same project, then the buckets share the bandwidth.
A higher bandwidth means more data can be transferred at once. Even with a high request rate, if the bandwidth is small, the overall transfer is slow. |
| Maximum bandwidth per single object | 8 MBps |
This is the maximum speed at which you can transfer a single object.
A higher bandwidth per single object means you can transfer the objects at a faster rate. This is the speed limit for moving one object at a time. Even with a high request rate and high bandwidth per bucket, if individual objects have a speed limit, they can take more time to transfer. |
Relocation Time to Live limit
To ensure efficient resource utilization and prevent relocations from running indefinitely, a Time to Live (TTL) limit applies to all bucket relocations. TTL refers to the maximum time allowed for the entire relocation process to complete.
The maximum time allowed to complete a bucket relocation is 28 days and includes all phases of the relocation process, such as the initial copy, incremental updates, and final synchronization.
If the relocation process exceeds the 28-day TTL limit, the relocation operation fails.
Ongoing bucket activity
If you continue to write new objects, delete existing ones, or update objects in the bucket during the relocation, these operations compete for resources with the copy requests and can slow down the relocation process.
Lifecycle rules
If you have lifecycle rules configured for your bucket, such as automatically deleting or archiving objects after a specific time, these actions increase the overall relocation time.
Enable Management Hub
You must enable Management Hub for both the source and destination locations. You can enable Management Hub at different levels of your Google Cloud resource hierarchy. You can also use inclusion and exclusion filters to include relevant buckets to the Management Hub plan. For details, see Enable Management Hub.
Considerations with other features
When you relocate buckets, it has the following interactions with other Cloud Storage features:
Enable soft delete
Bucket relocation requires that you enable soft delete on the bucket and set the retention duration to at least seven days. The retention duration is the amount of time that soft delete keeps deleted objects before permanently deleting them. For information about how to configure the soft delete retention duration, see Use soft delete.
Check quotas and limits
Quotas and cloud capacity assessments are tied to specific regions or zones. As a result, when you move a bucket to a new location, you need to ensure that the new location has sufficient quotas to accommodate the bucket's data. For more information about quotas and limits, see Quotas and limits.
Determine your bucket relocation type
When relocating your bucket, it's important to understand that there might be a write downtime period during the final synchronization step where you cannot update or upload new objects. Additionally, you won't be able to change your bucket's configuration during the relocation process. To determine if your relocation involves downtime, see Determine your relocation type.
Remove existing bucket tags
You cannot relocate a bucket that has bucket tags attached to it. All existing tags must be removed before the bucket relocation. If any of the tags being removed from your source bucket are being used for access control, you'll need to use an alternative method to set up the IAM permissions to ensure the data in your bucket remains secure. To do so, complete the following steps:
Make a copy of your tag configuration and store it securely.
Detach any existing tags from the source bucket.
Configure IAM permissions to match your existing access control rules.
After you relocate the bucket, attach any existing tags to the relocated bucket.
Save the existing inventory report configurations
Existing inventory report configurations are not preserved during the relocation process. We recommend that you save your existing inventory report configurations manually before starting the relocation process, as you'll need to recreate them after the relocation process is complete. For information about managing inventory report configurations, see Create and manage inventory report configurations.
What's next
- Learn how to relocate buckets.