Viewing Usage Reports

Compute Engine lets you export detailed reports of your Compute Engine usage to a Google Cloud Storage bucket using the usage export feature. Usage reports provide information about the lifetime of your resources. For example, you can see how many of your instances are running an n1-standard-1 image and how long each instance has been running. Usage reports do not provide billing or activity information, such as information about API requests. For billing information, see the billing export feature. For activity logs, see Activity logs.

Usage logs are available on the Compute Engine settings page as CSV files. The logs describe usage at the resource level, such as the running time of a virtual machine instance, the storage space of a persistent disk, and information about other Compute Engine features.

Before you begin

If you want to use the command-line examples in this guide:
1. Install or update to the latest version of the gcloud command-line tool.
2. Set a default region and zone.
If you want to use the API examples in this guide, set up API access.

Overview

When you enable usage reports, Compute Engine delivers two types of reports to the Google Cloud Storage bucket you specify:

Daily usage reports

These reports are delivered daily and include usage data from the preceding day. Each report is a separate file that contains data from the last period. Data in these reports are immutable, meaning that Compute Engine will not update or rewrite the log file if there are inaccuracies. Instead, the data is corrected in the next new report that is delivered to the bucket.

Daily usage reports have the following name format:
```
<bucket>/<report_prefix>_<numeric_project_id>_<YYYYMMDD>.csv
```
Monthly rollup report

A single monthly rollup report is delivered daily, which contains monthly usage data for that project up to, but not including, that day. The monthly usage report is overwritten each day with new data that reflects the monthly usage of resources up to that date. There will only be one monthly usage data file per project, per month.

Monthly rollup reports have following name format:
```
<bucket>/<report_prefix>_<numeric_project_id>_<YYYYMM>.csv
```
The daily and monthly report files look very similar, except for the difference in date format, where the monthly rollup reports are dated using the year and month (YYYYMM), and the daily usage reports are dated using the year, month, and date (YYYYMMDD).

All usage reports are delivered in comma-separated values (CSV) format and usage report files are prefixed using <report_prefix>. The <report_prefix> is a customizable value chosen by the user. If you don't specify a report prefix, the prefix usage_gce will be used by default. All times are given in Pacific time (PST).

Supported metrics

Daily usage reports provide usage information about the following resources:

Virtual machines
Persistent disks
Images
Snapshots
Static IP addresses
Load balancers

Each resource is described using the following metrics:

Metric Name	Metric Properties
Report Date	Metric Type: string Description: The date of the usage. Example: 2013-08-15
MeasurementId	Metric Type: string Description: The ID of the type of resource that is being measured. For example, `VmimageN1Standard_1` to represent an `n1-standard-1` machine type. Example: `com.google.cloud/services/compute‑engine/VmimageN1Standard_1`
Quantity	Metric Type: integer Description: The amount of usage for the specified date. Example: 86400
Unit	Metric Type: string Description: The unit type, such as count, seconds, hours, and so on. Example: seconds
Resource URI	Metric Type: string Description: The URI of the specified resource. Example: `https://compute.googleapis.com/compute/v1/projects/<project‑id>/zones/<zone>/instances/<instance>`
ResourceId	Metric Type: integer Description: A numeric ID that identifies the specific resource. Example: 16557630484925648021
Location	Metric Type: string Description: The location of the resource. Either a zone or region name, or `GLOBAL` for global resources. Example: `us-central1-a`

An example entry in the report would look like the following:

Report Date	MeasurementId	Quantity	Unit	Resource URI	Resource ID	Location
02/13/2014	`com.google.cloud/services/compute-engine/VmimageN1Standard_1`	86400	seconds	`https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/instances/my-instance`	16557630484	us-central1-a

Access control

When you enable the usage export feature for a Cloud Storage bucket, Compute Engine automatically adds itself to the bucket with write access in order to deliver usage reports. As long as Compute Engine has access to the bucket and the usage export feature is enabled, Compute Engine will continue to export usage reports to the specified Cloud Storage bucket.

You can identify that Compute Engine has access to a bucket if you see the following identity added to the bucket access control list:

cloud-cluster-analytics-export@google.com

Any user who is an owner of the project has full access to the Google Cloud Storage bucket. Other users, such as writers and readers, have different degrees of access to the bucket. To learn about ACLs for a bucket, read the Cloud Storage access control documentation.

If you disable the usage export feature, Compute Engine will automatically remove write access from Compute Engine to the bucket. If you modify the permissions on the cloud-cluster-analytics-export@google.com account and then disable the usage export feature, Compute Engine will disable the usage export feature but won't remove the account from the project access list. You can choose to remove the account manually if desired.

Prerequisites

Before you can start using Compute Engine usage export, you must sign up for Google Cloud Storage if you haven't already.

Setting up usage export

When you first enable the usage export feature, the first report will be sent the following day, detailing the previous day's usage. Afterwards, you will receive reports in 24 hour intervals.

To set up the usage export feature, enable the feature in the gcloud compute tool. When you enable the feature, you must define two properties:

The Google Cloud Storage bucket where you would like your reports to be delivered.

You can select any Cloud Storage bucket for which you are an owner, including buckets that are from different projects. This bucket must exist before you can start exporting reports and you must have owner access to the bucket. Google Cloud Storage charges for usage, so you should review the Cloud Storage pricesheet for information on how you might incur charges for the service.

Any user who has read access to the Cloud Storage bucket will be able to view the usage reports in the bucket. Any user who has write access to the bucket will be able to create, view, and modify existing files in the bucket. For more information, see the Access control section.
The report prefix for your files.

You can specify the report prefix to use for your usage reports. Your usage reports will have file names that contain this prefix. For example, specifying "my-cool-project-report" as your report prefix results in a file name similar to the format my-cool-project-report_1234567890_20131230.csv. If you do not specify a report prefix, the default prefix usage_gce is used.

After you decide on these two properties, you can enable the usage export feature in the following ways:

Console

Go to the Compute Engine Settings page.
Click Enable usage export.
Fill in the fields asking for a Bucket name and Report prefix, if desired. If you leave the report prefix empty, the default prefix usage_gce is used.
Click Save.

gcloud

In gcloud compute, use the gcloud compute project-info set-usage-bucket command to enable this feature:

 gcloud compute project-info set-usage-bucket --bucket URI [--prefix PREFIX]

The URI to the bucket can be in the format gs://<bucket-name> or https://storage.googleapis.com/<bucket-name>.

API

Client libraries

Using the client libraries, set the setUsageExportBucket() method in the Projects collection to enable usage export. For example, in the Python Client Library, you can enable the feature like so:

def setUsageExport(gce_service, auth_http):
  body = {
    "bucketName": "https://storage.googleapis.com/usage-export-sample",
    "reportNamePrefix": "exampleprefix"
  }

request = gce_service.projects().setUsageExportBucket(project=PROJECT_ID, body=body)
response = request.execute(http=auth_http)

print response

HTTP

You can also directly make a HTTP request to the setUsageExportBucket method as well. The following example uses the httplib2 library to enable usage export:

#!/usr/bin/python

import urllib
import argparse
import logging
import sys
import random
from json import load,dumps
...

PROJECT_ID = "myproject"
API_VERSION = "v1"
API_URL = "https://compute.googleapis.com/compute/" + API_VERSION + "/projects/" + PROJECT_ID
OAUTH_FILE = "oauth-dev.dat"

def main(argv):
  logging.basicConfig(level=logging.INFO)

  parser = argparse.ArgumentParser(
    description=__doc__,
    formatter_class=argparse.RawDescriptionHelpFormatter,
    parents=[tools.argparser])

  # Parse the command-line flags.
  flags = parser.parse_args(argv[1:])

  http = httplib2.Http()

  # Add code to authenticate to the service
  ....

  url = API_URL + "/setUsageExportBucket"

  body = {
    "bucketName": "https://storage.googleapis.com/usage-export-sample",
    "reportNamePrefix": "exampleprefix"
  }

  formattedBody = dumps(body)
  resp,content = http.request(url,
                              "POST",
                              headers=listOfHeaders,
                              body=formattedBody)

  print str(content)

if __name__ == "__main__":
    main(sys.argv)

For more information, see the API reference documentation.

Check if usage reports are enabled

You can check on a project's usage export settings by getting information about the project:

gcloud compute project-info describe

Look for the usageExportLocation field:

+-------------------------+----------------------------------------------------+
| name                    | myproject                                          |
| description             |                                                    |
| creation-time           | 2012-10-18T16:31:52.308-07:00                      |
| usage                   |                                                    |
|   snapshots             | 1.0/1000.0                                         |
|   networks              | 2.0/2.0                                            |
|   firewalls             | 3.0/10.0                                           |
|...                      |                                                    |
| usageExportLocation     |                                                    |
|   bucketName            | https://storage.googleapis.com/usage-export-sample |
|   reportNamePrefix      |                                                    |
+-------------------------+----------------------------------------------------+

Disable usage reports

When you disable usage reports, Compute Engine automatically removes write access for Compute Engine to your Cloud Storage bucket and discontinues sending any new reports.

Console

Go to the Compute Engine Settings page.
Click Disable usage export.

gcloud

In gcloud compute, you can disable the usage export feature by using the gcloud compute project-info set-usage-bucket command with the --no-bucket flag:

gcloud compute project-info set-usage-bucket --no-bucket

API

Client libraries

To use the client libraries, make a request to the setUsageExportBucket() method, with a blank bucket name. For example, in the Python Client Library, you can disable the feature like so:

def disableUsageExport(gce_service, auth_http):
  body = {
    "bucketName": "",
}

request = gce_service.projects().setUsageExportBucket(project=PROJECT_ID, body=body)
response = request.execute(http=auth_http)

print response

HTTP

You can also directly make a HTTP request to the setUsageExportBucket method as well. The following example uses the httplib2 library to make a request to disable the usage export bucket:

#!/usr/bin/python

import urllib
import argparse
import logging
import sys
import random
from json import load,dumps
...

PROJECT_ID = "myproject"
API_VERSION = "v1"
API_URL = "https://compute.googleapis.com/compute/" + API_VERSION + "/projects/" + PROJECT_ID
OAUTH_FILE = "oauth-dev.dat"

def main(argv):
  logging.basicConfig(level=logging.INFO)

  parser = argparse.ArgumentParser(
    description=__doc__,
    formatter_class=argparse.RawDescriptionHelpFormatter,
    parents=[tools.argparser])

  # Parse the command-line flags.
  flags = parser.parse_args(argv[1:])

  http = httplib2.Http()

  # Add code to authenticate to the service
  ....

  url = API_URL + "/setUsageExportBucket"

  body = {
     "bucketName": ""
  }

  formattedBody = dumps(body)
  resp,content = http.request(url,
                              "POST",
                              headers=listOfHeaders,
                              body=formattedBody)

  print str(content)

if __name__ == "__main__":
    main(sys.argv)

Was this page helpful? Let us know how we did:

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated December 5, 2019.

Viewing Usage Reports

Before you begin

Overview

Supported metrics

Access control

Prerequisites

Setting up usage export

Console

gcloud

API

Client libraries

HTTP

Check if usage reports are enabled

Disable usage reports

Console

gcloud

API

Client libraries

HTTP

Send feedback about...