Understand the Cloud Billing data tables in BigQuery

Stay organized with collections Save and categorize content based on your preferences.

This page provides reference information for the schema of the Cloud Billing data that is exported to each table in BigQuery.

Billing data tables

Shortly after enabling Cloud Billing export to BigQuery, billing data tables are automatically created in the BigQuery dataset.

  • Standard usage cost table – In your BigQuery dataset, this table is named gcp_billing_export_v1_<BILLING_ACCOUNT_ID>.
  • Detailed usage cost table – This includes all of the data fields from the standard usage cost table, along with additional fields that provide resource-level cost data, such as a virtual machine or SSD that generates service usage. In your BigQuery dataset, this table is named gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>.
  • Pricing data table – In your BigQuery dataset, this table is named cloud_pricing_export.

Frequency of data loads

Standard and detailed usage cost export:

  • When you first enable the standard or detailed usage cost export to BigQuery, it might take a few hours to start seeing your Google Cloud cost data.
  • Google Cloud services report usage and cost data to Cloud Billing processes at varying intervals.
  • Cloud Billing exports usage and cost data to your BigQuery dataset at regular intervals (there are no delivery or latency guarantees for the export to BigQuery).
  • Because the usage reporting frequency varies by Google Cloud service, you might not see Cloud Billing usage and cost data right away for all recently used Google Cloud services.

Pricing export:

  • When you first enable the pricing export to BigQuery, it might take up to 48 hours to start seeing your Google Cloud pricing data.
  • After you enable pricing export, the pricing data applicable to your Cloud Billing account is exported to BigQuery once each day.

Data availability:

  • For customers who enable the standard usage cost or pricing data exports, your BigQuery dataset only reflects Cloud Billing data incurred from the date when you first enabled the export, and after. Cloud Billing data is not added retroactively, so you won't see Cloud Billing data for usage that occurred before you enabled export.
  • For customers who enable the detailed usage cost data export for the first time, Cloud Billing data is available from the start of the previous month. For example, if you enable detailed usage cost data export on September 23rd, your export will include data beginning August 1st.
    • For customers who have enabled, disabled, and subsequently re-enabled the detailed usage cost data export, the Cloud Billing data might not be available for the period it was explicitly disabled.
  • If you delete any exported data (such as BigQuery export records), we cannot backfill the deleted records.
  • BigQuery loads are ACID compliant, so if you query the BigQuery Cloud Billing export dataset while data is being loaded into it, you will not encounter partially loaded data. Learn more about BigQuery characteristics.
  • See other limitations that might impact exporting your billing data to BigQuery, such as datasets with customer-managed encryption keys (CMEK) enabled.

Limitations

Exporting Cloud Billing data to BigQuery is subject to the following limitations.

  • Your BigQuery datasets containing standard usage cost and pricing data only reflect Google Cloud billing data incurred from the date you set up Cloud Billing export, and after. That is, Google Cloud billing data is not added retroactively, so you won't see Cloud Billing data from before you enable export. For more details, see Data availability.
  • Your BigQuery datasets containing detailed usage cost data reflect Google Cloud billing data incurred from the start of the previous month from when you first enabled the export, unless you are re-enabling it. That is, Google Cloud billing data is added retroactively only for the current and previous month. For more details, see Data availability.
  • When exporting detailed usage cost data, you must export to a dataset configured with a multi-region location, such as United States (US) or European Union (EU). If you select a dataset that is not a multi-region location, this prevents Cloud Billing from writing detailed billing data to the appropriate tables within that dataset.
  • When exporting detailed usage cost data, the detailed export automatically includes resource-level information about Compute Engine. To view a breakdown of Google Kubernetes Engine (GKE) cluster costs in a detailed data export, you must also enable cost allocation for GKE.
  • Customer Managed Encryption Keys (CMEK) are not supported when exporting billing data to BigQuery. If you enable CMEK for your billing data dataset, this prevents Cloud Billing from writing billing data to the appropriate tables within that dataset. Instead, you need to configure the dataset to use Google-managed key encryption.
  • Tags are limited to Compute Engine instances and might take up to an hour to propagate to BigQuery exports. If a tag has been added or removed within an hour, or if a resource has existed for less than an hour, it might not appear in the export.

Cost and pricing reports available in the Google Cloud console