Bandwidth usage in Cloud Storage

This page discusses how to monitor data usage rates from your Cloud Storage buckets, as well as how to request additional egress bandwidth for different quotas if you need more than the default quota.

Introduction

  • Egress bandwidth refers to the rate at which data is retrieved from your buckets in the course of completing requests made to Cloud Storage.

    • Google egress bandwidth refers to data retrieved from your buckets that is destined for other Google services, such as Compute Engine.

    • Internet egress bandwidth refers to data retrieved from your buckets that is served to the Internet, such as data downloaded directly to your local computer or data retrieved by Cloud CDN for serving to the Internet.

  • Network egress refers to the rate at which data is sent over the network from your buckets to requesters.

In normal operation, egress bandwidth and network egress are typically about the same; however, in some cases egress bandwidth can be meaningfully greater than network egress. Broken connections, cancelled requests, and partial requests are cases where the data retrieved can be greater than the data that's ultimately sent.

Bandwidth monitoring

Cloud Storage provides bandwidth monitoring for you to track bandwidth usage from your project's buckets to other Google Cloud services. Bandwidth monitoring is aggregated by region and tracks usage for the last 6 weeks.

In order to be tracked by bandwidth monitoring:

  • You must have the Cloud Storage service storage.googleapis.com enabled for your project.

  • The usage must be by Google Cloud resources other than Cloud Storage buckets.

  • If the bucket is located in a region, the usage must be by resources located in the same region.

  • If the bucket is located in a dual-region, the usage must be by resources located in either of the regions that makes up the dual-region.

  • The usage must be from either a JSON API GET Object request or a XML API GET Object request.

To monitor bandwidth usage:

Console

You can view egress bandwidth usage for your project's buckets using the Quotas and system limits page with the appropriate name:

Open Cloud Storage Quotas and systems limits

  • Google Egress Bandwidth per second per region monitors Google egress bandwidth usage for buckets in your project that are located in regions.

  • Dualregion Google Egress Bandwidth per second per region per dualregion_location monitors Google egress bandwidth usage for buckets in your project that are located in dual-regions.

  • MultiRegion Google Egress Bandwidth per second per region monitors Google egress bandwidth usage for buckets in your project that are located in multi-regions.

  • Internet Egress Bandwidth per second per region monitors Internet egress bandwidth usage for buckets in your project that are located in regions.

  • Dualregion Internet Egress Bandwidth per second per region per dualregion_location monitors Internet egress bandwidth usage for buckets in your project that are located in dual-regions.

  • Multiregion Internet Egress Bandwidth per second per region monitors Internet egress bandwidth usage for buckets in your project that are located in multi-regions.

For more information, see View and manage quotas.

You can also view the Google egress bandwidth usage of buckets in your project that are located in regions on the Cloud Storage monitoring page.

Command line

  1. Use the following command to return Cloud Storage quota information for the desired project:

    gcloud alpha services quota list --service=storage.googleapis.com --consumer=projects/PROJECT_ID

    Where PROJECT_ID is the desired project ID.

  2. In the YAML response, look for the following:

    • storage.googleapis.com/google_egress_bandwidth, for Google egress usage from your buckets that are located in regions.

    • storage.googleapis.com/dualregion_google_egress_bandwidth, for Google egress usage from your buckets that are located in dual-regions.

    • storage.googleapis.com/multiregion_google_egress_bandwidth, for Google egress usage from your buckets that are located in multi-regions.

    • storage.googleapis.com/internet_egress_bandwidth for Internet egress usage from your buckets that are located in regions.

    • storage.googleapis.com/dualregion_internet_egress_bandwidth for Internet egress usage from your buckets that are located in dual-regions.

    • storage.googleapis.com/multiregion_internet_egress_bandwidth for Internet egress usage from your buckets that are located in multi-regions.

For more information, see Manage your quota using the Google Cloud CLI.

JSON API

For information on programmatically retrieving time series information for non dual-region quotas, see the ListTimeSeries command, using the following values for filter:

  • metric.type="serviceruntime.googleapis.com/quota/rate/net_usage"

  • resource.type="consumer_quota"

  • One of the following:

    • metric.label.quota_metric="storage.googleapis.com/google_egress_bandwidth" for Google egress usage from your buckets that are located in regions.

    • metric.label.quota_metric="storage.googleapis.com/internet_egress_bandwidth" for Internet egress usage from your buckets that are located in regions.

    • metric.label.quota_metric="storage.googleapis.com/multiregion_google_egress_bandwidth" for Google egress usage from your buckets that are located in multi-regions.

    • metric.label.quota_metric="storage.googleapis.com/multiregion_internet_egress_bandwidth" for Internet egress usage from your buckets that are located in multi-regions.

For information on programmatically retrieving time series information for dual-region quotas, see the ListTimeSeries command, using the following values for filter:

  • resource.type="storage.googleapis.com/Project"

  • One of the following:

    • metric.type="storage.googleapis.com/quota/dualregion_google_egress_bandwidth/usage" for Google egress usage from your buckets that are located in dual-regions.

    • metric.type="storage.googleapis.com/quota/dualregion_internet_egress_bandwidth/usage" for Internet egress usage from your buckets that are located in dual-regions.

Monitoring alerts

You can use Monitoring alerting policies to inform you when metrics such as storage.googleapis.com/google_egress_bandwidth, storage.googleapis.com/dualregion_google_egress_bandwidth, and storage.googleapis.com/multiregion_google_egress_bandwidth approach a quota limit.

Network monitoring

In addition to Cloud Storage bandwidth monitoring, you can use the network/sent_bytes_count metric to measure Cloud Storage network egress. This is useful for approximating bandwidth usage for specific buckets, because bandwidth monitoring tracks bandwidth usage by location and project.

When using this networking metric, keep in mind the following:

  • network/sent_bytes_count uses bytes in its measurements, whereas typical networking metrics often use bits. For example, a 1 GiBps measurement for network/sent_bytes_count is equivalent to 8 Gibps.

  • network/sent_bytes_count tracks all the traffic sent over the network, not only egress to co-located Google Cloud services. As such, network/sent_bytes_count is typically greater than bandwidth usage.

    • In some cases, network/sent_bytes_count can be less than bandwidth usage due to events such as broken connections, cancelled requests, and partial requests.

  • network/sent_bytes_count metrics are sampled every 60 seconds. If the traffic is spiky, it's possible that requests are throttled for a short period of time even though the average egress over 60 seconds is below the limit.

Request more bandwidth

The following instructions apply when requesting egress bandwidth of up to 1 Tbps for Cloud Storage buckets:

  1. Make sure you have enabled the Cloud Storage service storage.googleapis.com.

  2. In the Google Cloud console, go to the IAM Quotas & System Limits page.

    Go to Browser

  3. In the Filter search bar, select the Metric property, then search for the metric for which you are requesting an increase:

    • Use storage.googleapis.com/google_egress_bandwidth to increase Google egress for your buckets located in regions.

    • Use storage.googleapis.com/dualregion_google_egress_bandwidth to increase Google egress for your buckets located in dual-regions.

    • Use storage.googleapis.com/multiregion_google_egress_bandwidth to increase Google egress for your buckets located in multi-regions.

    • Use storage.googleapis.com/internet_egress_bandwidth to increase Internet egress for your buckets located in regions.

    • Use storage.googleapis.com/dualregion_internet_egress_bandwidth to increase Internet egress for your buckets located in dual-regions.

    • Use storage.googleapis.com/multiregion_internet_egress_bandwidth to increase Internet egress for your buckets located in multi-regions.

  4. In the list of results, toggle the checkbox for the locations where you want to increase egress bandwidth. For the storage.googleapis.com/dualregion_google_egress_bandwidth and storage.googleapis.com/dualregion_internet_egress_bandwidth metrics, there must be non-zero historical usage of the metric before you can request an increase.

  5. Click Edit Quotas.

  6. Enter your requested quota and business justification.

  7. Click Submit Request.

For bandwidth requirements greater than 1 Tbps for egress from buckets in regions and dual-regions, contact your Technical Account Manager or Google representative. Increase requests greater than 1 Tbps for egress from buckets in multi-regions are not supported.

What's next