Cloud Billing export to BigQuery enables you to export detailed Google Cloud billing data (such as usage, cost estimates, and pricing data) automatically throughout the day to a BigQuery dataset that you specify. Then you can access your Cloud Billing data from BigQuery for detailed analysis, or use a tool like Google Data Studio to visualize your data. You can also use this export method to export data to a JSON file.
Timing is important. To have access to a more comprehensive set of Google Cloud billing data for your analysis needs, we recommend that you enable Cloud Billing data export to BigQuery at the same time that you create a Cloud Billing account. Be aware that your BigQuery dataset only reflects Google Cloud billing data incurred from the date you set up Cloud Billing export, and after. That is, Google Cloud billing data is not added retroactively, so you won't see Cloud Billing data from before you enable export.
Managing and reporting costs effectively is a critical part of financial stewardship, whether you're running a multi-billion-dollar enterprise business or small household budget. Making data-driven decisions about your Google Cloud costs and usage starts with collecting the data you'll need to inform those decisions.
Refer to the guides in this section to learn about the following tasks:
- Set up Cloud Billing data export to BigQuery
- Understand the Cloud Billing data tables
- Find example queries for Cloud Billing data export
Set up Cloud Billing data export to BigQuery
To start collecting your Cloud Billing data, you must enable Cloud Billing data export to BigQuery.
The setup guide provides best practice recommendations and detailed instructions for enabling Cloud Billing data export to BigQuery. There are two types of Cloud Billing data you can enable for export:
- Daily cost detail data
- Pricing data
Understand the Cloud Billing data tables
To understand the data schema of your exported content, see the reference information for the contents of the Cloud Billing data that is exported to each table in the BigQuery dataset.
Find example queries for Cloud Billing data
For tips and guidance for using SQL to run queries on your Cloud Billing data, view the example queries.
On the example queries page, you'll find various SQL examples, including the following examples:
- Return the total costs on an invoice
- Query your data using labels
- Return data about your committed use discounts
- Query costs and credits by project for a specified invoice month
Cost of use
Using BigQuery to store usage data will incur minimal fees.
- Loading data into the designated dataset is free; this action takes advantage of BigQuery's pool of shared resources to load data in batches.
- When exporting and analyzing Cloud Billing data with BigQuery, the associated cost will depend on the amount of data you stream, store, and query.
To get an idea of what charges to expect, see Estimating storage and query costs.
Summary of BigQuery pricing
Here is a summary of the BigQuery costs that you'll likely incur when exporting and analyzing your Cloud Billing data with BigQuery. For more details, review BigQuery pricing.
|Active Storage||$0.020 per GB||The first 10 GB is free each month.|
|Long Term Storage||$0.010 per GB||The first 10 GB is free each month.|
|Analysis||$5.00 per TB on-demand pricing||First 1 TB per month is free.
Flat-rate pricing is available for high-volume customers.
|Streaming Inserts||$0.010 per 200 MB||Charged per row using a 1 KB minimum size.
*NO FREE Usage for this SKU
Many partitioned table operations are free, including loading data into partitions, copying partitions, and exporting data from partitions. Though free, these operations are subject to BigQuery's quotas and limits. For information on all free operations, see Free operations on the BigQuery pricing page.
For more information on best practices for controlling costs in BigQuery, see Controlling costs in BigQuery.