Bandwidth usage in Cloud Storage

This page discusses how to monitor data usage rates from your Cloud Storage buckets, as well as how to request additional egress bandwidth for different quotas if you need more than the default quota.

Introduction

• Egress bandwidth refers to the rate at which data is retrieved from your buckets in the course of completing requests made to Cloud Storage.

  • Google egress bandwidth refers to data retrieved from your buckets that is destined for other Google services, such as Compute Engine.

  • Internet egress bandwidth refers to data retrieved from your buckets that is served to the Internet, such as data downloaded directly to your local computer or data retrieved by Cloud CDN for serving to the Internet.

• Network egress refers to the rate at which data is sent over the network from your buckets to requesters.

In normal operation, egress bandwidth and network egress are typically about the same; however, in some cases egress bandwidth can be meaningfully greater than network egress. Broken connections, cancelled requests, and partial requests are cases where the data retrieved can be greater than the data that's ultimately sent.

Bandwidth monitoring

Cloud Storage provides bandwidth monitoring for you to track bandwidth usage from your project's buckets to other Google Cloud services. Bandwidth monitoring is aggregated by region and tracks usage for the last 6 weeks.

In order to be tracked by bandwidth monitoring:

• You must have the Cloud Storage service storage.googleapis.com enabled for your project.

• The usage must be by Google Cloud resources other than Cloud Storage buckets.

• If the bucket is located in a region, the usage must be by resources located in the same region.

• If the bucket is located in a dual-region, the usage must be by resources located in either of the regions that makes up the dual-region.

• The usage must be from either a JSON API GET Object request or a XML API GET Object request.

To monitor bandwidth usage:

Console

You can view the Google egress bandwidth usage of your project's region and dual-region buckets on the Cloud Storage monitoring page:

Open Cloud Storage bandwidth monitoring

Additionally, you can view this and other egress bandwidth usage for your project's buckets using the Quotas and system limits page with the appropriate name:

• Google Egress Bandwidth per second per region monitors Google egress bandwidth usage for your project's region and dual-region buckets.

• Dualregion Google Egress Bandwidth per second per region per dualregion_location monitors Google egress bandwidth usage for your project's dual-region buckets.

• MultiRegion Google Egress Bandwidth per second per region monitors Google egress bandwidth usage for your project's multi-region buckets.

• Internet Egress Bandwidth per second per region monitors Internet egress bandwidth usage for your project's region and dual-region buckets.

• Dualregion Internet Egress Bandwidth per second per region per dualregion_location monitors Internet egress bandwidth usage for your project's dual-region buckets.

• Multiregion Internet Egress Bandwidth per second per region monitors Internet egress bandwidth usage for your project's multi-region buckets.

For more information, see View and manage quotas.

Command line

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

1. Use the following command to return Cloud Storage quota information for the desired project:

   gcloud alpha services quota list --service=storage.googleapis.com --consumer=projects/PROJECT_ID

   Where PROJECT_ID is the desired project ID.

2. In the YAML response, look for the following:

   • storage.googleapis.com/google_egress_bandwidth, for Google egress usage from your region and dual-region buckets.

   • storage.googleapis.com/dualregion_google_egress_bandwidth, for Google egress usage from your dual-region buckets.

   • storage.googleapis.com/multiregion_google_egress_bandwidth, for Google egress usage from your multi-region buckets.

   • storage.googleapis.com/internet_egress_bandwidth for Internet egress usage from your region and dual-region buckets.

   • storage.googleapis.com/dualregion_internet_egress_bandwidth for Internet egress usage from your dual-region buckets.

   • storage.googleapis.com/multiregion_internet_egress_bandwidth for Internet egress usage from your multi-region buckets.

For more information, see Manage your quota using the Google Cloud CLI.

JSON API

For information on programmatically retrieving time series information for non dual-region quotas, see the ListTimeSeries command, using the following values for filter:

• metric.type="serviceruntime.googleapis.com/quota/rate/net_usage"

• resource.type="consumer_quota"

• One of the following:

  • metric.label.quota_metric="storage.googleapis.com/google_egress_bandwidth" for Google egress usage from your region and dual-region buckets.

  • metric.label.quota_metric="storage.googleapis.com/internet_egress_bandwidth" for Internet egress usage from your region and dual-region buckets.

  • metric.label.quota_metric="storage.googleapis.com/multiregion_google_egress_bandwidth" for Google egress usage from your multi-region buckets.

  • metric.label.quota_metric="storage.googleapis.com/multiregion_internet_egress_bandwidth" for Internet egress usage from your multi-region buckets.

For information on programmatically retrieving time series information for dual-region quotas, see the ListTimeSeries command, using the following values for filter:

• resource.type="storage.googleapis.com/Project"

• One of the following:

  • metric.type="storage.googleapis.com/quota/dualregion_google_egress_bandwidth/usage" for Google egress usage from your dual-region buckets.

  • metric.type="storage.googleapis.com/quota/dualregion_internet_egress_bandwidth/usage" for Internet egress usage from your dual-region buckets.

Monitoring alerts

You can use Monitoring alerting policies to inform you when metrics such as storage.googleapis.com/google_egress_bandwidth, storage.googleapis.com/dualregion_google_egress_bandwidth, and storage.googleapis.com/multiregion_google_egress_bandwidth approach a quota limit.

• For an overview of Monitoring alerts, see Introduction to alerting.

• To get started with Monitoring alerts using the Google Cloud console, see Create metric-based alert policies.

• For examples of Monitoring alerts, see Chart and monitor quota metrics.

Network monitoring

In addition to Cloud Storage bandwidth monitoring, you can use the network/sent_bytes_count metric to measure Cloud Storage network egress. This is useful for approximating bandwidth usage for specific buckets, because bandwidth monitoring tracks bandwidth usage by location and project.

When using this networking metric, keep in mind the following:

• network/sent_bytes_count uses bytes in its measurements, whereas typical networking metrics often use bits. For example, a 1 GiBps measurement for network/sent_bytes_count is equivalent to 8 Gibps.

• network/sent_bytes_count tracks all the traffic sent over the network, not only egress to co-located Google Cloud services. As such, network/sent_bytes_count is typically greater than bandwidth usage.

  • In some cases, network/sent_bytes_count can be less than bandwidth usage due to events such as broken connections, cancelled requests, and partial requests.

• network/sent_bytes_count metrics are sampled every 60 seconds. If the traffic is spiky, it's possible that requests are throttled for a short period of time even though the average egress over 60 seconds is below the limit.

Request more bandwidth

The following instructions apply when requesting egress bandwidth of up to 1 Tbps for Cloud Storage buckets:

1. Make sure you have enabled the Cloud Storage service storage.googleapis.com.

2. In the Google Cloud console, go to the IAM Quotas & System Limits page.

   Go to Browser

3. In the Filter search bar, select the Metric property, then search for the metric for which you are requesting an increase:

   • Use storage.googleapis.com/google_egress_bandwidth to increase Google egress for your region and dual-region buckets.

   • Use storage.googleapis.com/internet_egress_bandwidth to increase Internet egress for your region and dual-region buckets.

   • Use storage.googleapis.com/multiregion_google_egress_bandwidth to increase Google egress for your multi-region buckets.

   • Use storage.googleapis.com/multiregion_internet_egress_bandwidth to increase Internet egress for your multi-region buckets.

4. In the list of results, toggle the checkbox for the locations where you want to increase egress bandwidth.

5. Click Edit Quotas.

6. Enter your requested quota and business justification.

7. Click Submit Request.

For bandwidth requirements greater than 1 Tbps for egress from buckets in regions and dual-regions, contact your Technical Account Manager or Google representative. Increase requests greater than 1 Tbps for egress from buckets in multi-regions are not supported.

Important: Granting and continuation of bandwidth quota increases is dependent on your project's history. For example, a project might have an existing bandwidth quota increase reduced or rescinded if the project is not in good standing.

What's next

• Monitor who accesses your Cloud Storage resources.
• Create notifications when an object is created, updated, or deleted.