View detailed breakdown of cluster costs


This page shows you how to view a breakdown of your Google Kubernetes Engine (GKE) cluster costs in Cloud Billing BigQuery export using GKE cost allocation.

You can use this information to distribute the costs of a cluster to its users.

Potential data volume increase

Enabling GKE cost allocation can increase your storage and querying costs in BigQuery. The exact increase depends on the number of distinct label and namespace combinations you use across Pods and clusters.

Enabling GKE cost allocation does not change the total cost of your GKE usage. The sum of the cost line items in your export is the same and existing queries and reports you have built on the billing export return the same values.

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
  • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI.

Limitations and restrictions

  • You can only view GKE cost allocation data in the Cloud Billing BigQuery detailed usage cost data export.
  • You must have Google Cloud CLI version 392.0.0 or later
  • If you enable GKE cost allocation, your billing export starts including additional line items for your GKE resources from that date. The billing export does not backfill data.
  • If you disable GKE cost allocation, your billing export stops including additional line items, but does not modify or remove previous line items that were generated while the feature was enabled.
  • GKE cost allocation data is based on resource requests, not resources consumed. To learn more about the differences between resource requests, resource limits, and resource consumption, see Use GKE usage metering to combat over-provisioning.
  • GKE cost allocation supports the following resource SKU types:
    • Compute Engine VM Instance Core SKUs
    • Compute Engine VM Instance RAM SKUs
    • Compute Engine VM Instance GPU SKUs

View GKE cluster costs

When you enable GKE cost allocation, the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery:

Metadata Description Cluster name Namespace
Resource label key (labels.key) Kubernetes labels have the following format: k8s-label/${k8s-label-key}, where ${k8s-label-key} is the key of the Pod's Kubernetes label. The billing label k8s-label/${k8s-label-key} has the same value as the corresponding Kubernetes Pod label. goog-k8s-cluster-name k8s-namespace
Resource label value (labels.value) If a Kubernetes label key collides with a VM label key, the Kubernetes label value overrides the VM label value. [cluster-name] [namespace]

For more information, see Schema of the detailed usage cost data.

Additionally, the following namespaces are used to track overhead and unallocated resources:

  • kube:system-overhead: Resources are reserved for system overhead.
  • kube:unallocated: Resources are neither reserved by pods nor reserved for system overhead.

If GKE cannot determine the cost allocation for a resource, the export includes one of the following values:

  • goog-k8s-unknown: Cloud Billing could not process the SKU. This can occur when a new Compute Engine instance is being provisioned.
  • goog-k8s-unsupported-sku: Cloud Billing does not support the SKU.

Verify that GKE cost allocation is enabled

gcloud beta container clusters describe CLUSTER_NAME \
    --region=COMPUTE_REGION \
    --format="value(costManagementConfig.enabled)"

Replace the following:

  • CLUSTER_NAME: the name of the cluster.
  • COMPUTE_REGION: the Compute Engine region for your cluster. For zonal clusters, use --zone=COMPUTE_ZONE.

The output is True if GKE cost allocation is enabled.

Enable GKE cost allocation

To enable GKE cost allocation on a new or existing cluster, use the --enable-cost-allocation flag:

Create a new cluster

gcloud beta container clusters create CLUSTER_NAME \
    --enable-cost-allocation \
    --region=COMPUTE_REGION

Replace the following:

  • CLUSTER_NAME: the name of the new cluster.
  • COMPUTE_REGION: the Compute Engine region for your cluster. For zonal clusters, use --zone=COMPUTE_ZONE.

Update a cluster

gcloud beta container clusters update CLUSTER_NAME \
    --enable-cost-allocation

Replace CLUSTER_NAME with the name of the cluster.

Verify GKE cost allocation is enabled

Describe the cluster:

gcloud beta container clusters describe CLUSTER_NAME

The output is similar to the following:

...
costManagementConfig:
  enabled: true
...

In this output, costManagementConfig indicates that GKE cost allocation is enabled.

Filter GKE cost data

The following example queries show you how to filter and group your GKE costs for supported resource types by cluster name, namespace, and label.

Replace BILLING_DATASET_TABLE with the name of the dataset in BigQuery. The table name is similar to gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this page, disable GKE cost allocation for the cluster:

gcloud beta container clusters update CLUSTER_NAME \
    --no-enable-cost-allocation

You can also delete the cluster that you created.

What's next