This document explains how to set up programmatic budget and cost anomaly (Preview) notifications using Pub/Sub.
Budgets and cost anomalies are typically configured to send email notifications. If you use budgets or cost anomaly detection as a cost control tool, email notifications might not be the best method to use to ensure timely action to control your costs. You can set up and use programmatic notifications to automate cost control responses. Examples of automated cost control responses include the following:
- Integrate with Slack for faster response
- Manage resource consumption to reduce costs
- Limit spending by disabling billing when you reach your budget
Limitations
Budgets use estimated Cloud Billing data which is subject to change until your invoice is finalized.
If you set organization policies to limit resource sharing by domain (for example, enabling
enforceInTransit
on Pub/Sub topics), you might encounter an error when you try to set up or connect Pub/Sub topics. In this situation, you might need to force account access to successfully connect a Pub/Sub topic to a budget. To force account access, complete the following steps:- Remove the organization policy containing the domain restriction constraint
using one of the following methods:
- Temporarily remove the organization policy.
- Override the organization policy for the project that contains the Pub/Sub topic.
- Follow the steps to connect a Pub/Sub topic.
- Reinstate the organization policy for the organization or project (optional).
- Remove the organization policy containing the domain restriction constraint
using one of the following methods:
Permissions required for this task
Budget notifications
To set up programmatic budget notifications, you need permissions to create or modify budgets and Pub/Sub topics. You need a different set of permissions depending on your level of access.
Users with Cloud Billing account permissions | Users with project-level permissions only (Preview) |
---|---|
If your organization uses custom roles for authentication, you need the following permissions in the custom roles:
For custom roles to modify an existing Cloud Billing budget to connect it to an existing Pub/Sub topic, you need the following permissions:
To gain the necessary permissions using predefined roles, ask your administrator to grant you one of the following predefined Cloud Billing IAM roles on your Cloud Billing account: And ask your administrator to grant you the following role on the target project that contains the Pub/Sub topics: |
If your organization uses custom roles for authentication, you need the following permissions in the custom roles:
To gain the necessary permissions using predefined roles, ask your administrator to grant you one of the following predefined Cloud Billing IAM roles on the project: And ask your administrator to grant you the following role on the target project that contains the Pub/Sub topics: |
Anomaly notifications
To set up programmatic anomaly notifications, you need permissions to create or modify anomalies and Pub/Sub topics. You must have Cloud Billing account permissions.
Users with Cloud Billing account permissions (Preview) |
---|
To gain the necessary permissions using predefined roles, ask your administrator to grant you one of the following predefined Cloud Billing IAM roles on your Cloud Billing account: And ask your administrator to grant you the following role on the target project that contains the Pub/Sub topics: |
Create a Pub/Sub topic
To set up programmatic budget or anomaly notifications, you must first create a Pub/Sub topic. If you already configured a Pub/Sub topic for budgets or cost anomalies, you can skip this step.
Go to the Google Cloud console Pub/Sub page.
Select a project to contain the Pub/Sub topic.
When configuring Pub/Sub topics for billing notifications, we recommend you use a FinOps administration project to contain your billing-related Pub/Sub topics.
Click
Create topic.In the Topic ID field, provide a topic name.
Retain the default subscription and encryption settings, and click Create.
Connect your Pub/Sub topic
To enable programmatic notifications to receive Pub/Sub messages with the current status of your budget or anomaly, you must connect your budget or anomaly to your Pub/Sub topic.
Budget notifications
To connect a Pub/Sub topic to a Cloud Billing budget, complete the following steps:
Users with Cloud Billing account permissions | Users with project-level permissions only (Preview) |
---|---|
|
|
Anomaly notifications
To connect a Pub/Sub topic to a cost anomaly, complete the following steps:
Users with Cloud Billing account permissions (Preview) |
---|
|
Notification format
Notifications sent to the Pub/Sub topic consist of two parts:
- Attributes: A set of key-value pairs describing the event.
Data: A string containing a JSON object that describes the budget alert details.
Budget notifications
Attributes
Attributes are key-value pairs contained in all notifications sent by Cloud Billing to your Pub/Sub topic. Notifications always contain the following set of key-value pairs, regardless of the notification payload.
Attribute name | Example | Description |
---|---|---|
billingAccountId |
01D4EE-079462-DFD6EC
|
The identifier for the Cloud Billing account owning the budget. |
budgetId |
de72f49d-779b-4945-a127-4d6ce8def0bb
|
The identifier for the budget alert within the Cloud Billing account. |
schemaVersion |
1.0
|
The notifications schema version. |
Data
Budget alert details are returned in a base64-encoded UTF-8 string containing a JSON object with the following properties:
Data | Example | Description |
---|---|---|
budgetDisplayName
|
"My Personal Budget"
|
The human-readable name assigned to the budget. |
costAmount
|
140.321
|
The amount of costs accrued. The type of costs tracked depends on budget filters & settings. |
costIntervalStart
|
"2021-02-01T08:00:00Z"
|
The start of the budget alert period. `costAmount` includes costs for usage starting at this time. This is the first day of the budget's time period (month, quarter, year, or a custom date range) during which the budget usage occurred. |
budgetAmount
|
152.557
|
The amount allocated in the budget. |
budgetAmountType
|
"SPECIFIED_AMOUNT"
|
The budget amount type. This can be one of the following values:
|
alertThresholdExceeded
|
0.9
|
The highest configured actual cost alert threshold exceeded. Value is a percent in decimal format (for example, 0.9 is 90%). This key isn't present if the actual cost doesn't exceed any threshold. |
forecastThresholdExceeded
|
0.2
|
The highest forecasted cost alert threshold exceeded. Value is a percent in decimal format (for example, 0.2 is 20%). This key isn't present if the forecasted cost doesn't exceed any threshold. |
currencyCode
|
"USD"
|
The budget alert currency. All costs and budget alert amounts are in this currency. |
Anomaly notifications
Attributes
Attributes are key-value pairs contained in all notifications sent by Cloud Billing to your Pub/Sub topic. Notifications always contain the following set of key-value pairs, regardless of the notification payload.
Attribute name | Example | Description |
---|---|---|
scope |
scope_project
|
The scope of the anomaly. |
billingAccountName |
billingAccounts/01D4EE-079462-DFD6EC
|
The identifier of the Cloud Billing account where the anomaly occurred. |
resourceDisplayName |
My project
|
The display name of the resource that the anomaly occurred in. |
resourceName |
projects/123456
|
The identifier of the resource that the anomaly occurred in. |
SchemaVersion |
1.0
|
The Pub/Sub schema version. |
Data
Anomaly alert details are returned in a base64-encoded UTF-8 string containing a JSON object with the following properties:
Data | Example | Description |
---|---|---|
anomalyName
|
billingAccounts/01D4EE-079462-DFD6EC/anomalies/aaa
|
The resource name for the anomaly. |
billingAccountName
|
billingAccounts/01D4EE-079462-DFD6EC
|
The identifier of the billing account. |
resourceName
|
"projects/12345"
|
The identifier of the resource that the anomaly occurred in. |
resourceDisplayName
|
"My project"
|
The display name of the resource that the anomaly occurred in. |
detectionDate
|
"2024-02-01T08:00:00Z"
|
The date the cost anomaly was detected. |
Scope
|
SCOPE_PROJECT
|
Indicates the scope of the anomaly. |
expectedSpendAmount
|
{
|
The expected spend for the resource. |
actualSpendAmount
|
196.22
|
The actual spend for the resource. |
deviationAmount
|
31.01
|
The difference between the actual spend and expected spend. |
deviationPercentage
|
18.77
|
The percentage of deviation amount from expected spend. |
Root cause
|
root_causes { resource: "services/XXX-XXXX-XXX" display_name: "BigQuery" cause_type: CAUSE_TYPE_SERVICE deviation { expected_spend { currency_code: "USD" units: "5000" } actual_spend { currency_code: "USD" units: "5000" } deviation_amount { currency_code: "USD" units: "5000" } deviation_percentage: "Infinity" } sub_causes { resource: "locations/us-west1", display_name: "us-west1", cause_type: CAUSE_TYPE_LOCATION, deviation { expected_spend { currency_code: "USD", } actual_spend { currency_code: "USD", units:"5000" } deviation_amount { currency_code: "USD", units: "5000" } deviation_percentage: "Infinity" } sub_causes { resource: "services/XXXX-XXXX-XXXX/skus/XXXX-XXXX-XXXX" display_name: "Some display name" cause_type: CAUSE_TYPE_SKU deviation { expected_spend { currency_code: "USD" } actual_spend { currency_code: "USD" units: 5000 } deviation_amount { currency_code: "USD" units: 5000 } deviation_percentage: "Infinity" } } sub_causes { resource: "services/XXXX-XXXX-XXXX/skus/XXXX-XXXX-XXXX" display_name: "Some display name" cause_type: CAUSE_TYPE_SKU deviation { expected_spend { currency_code: "USD" units: "1" nanos: 5413077 } actual_spend { currency_code: "USD" units: "1", nanos: 122606000 } deviation_amount { currency_code: "USD" nanos: 117192923 } deviation_percentage:11.656196419841217 } } } } |
A repeated field that contains information about the root cause of the anomaly. |
Delivery guarantees for notifications to Pub/Sub
After you connect your budget or anomaly to a Pub/Sub topic, you can expect to receive programmatic notifications as follows:
Budget notifications
- Budget notifications are sent to the Pub/Sub topic multiple times per day with the current status of your budget. This cadence is different than the cadence for budget alert emails, which are only sent when a budget threshold is met.
- Pub/Sub notifications are sent even if the Cloud Billing account has no usage.
- It may take several hours before you receive the first Pub/Sub notification.
- Pub/Sub only guarantees at-least-once delivery. You might receive a message multiple times, and messages might arrive out of order.
- If delivery fails due to Pub/Sub topic misconfiguration, you won't be notified.
Delivery might fail for one of the following reasons:
- The Pub/Sub topic no longer exists.
- The Cloud Billing no longer has permission to publish to the Pub/Sub topic.
- You exceeded your notification quota.
- You have organization policies in place that limit resource sharing by domain.
Anomaly notifications
- Anomaly notifications are sent to the Pub/Sub topic only when the threshold is reached.
- Pub/Sub only guarantees an at-least-once delivery. You might receive a message multiple times, and messages might arrive out of order.
- If delivery fails due to Pub/Sub topic misconfiguration, you won't be notified.
Delivery might fail for one of the following reasons:
- The Pub/Sub topic no longer exists.
- The Cloud Billing account no longer has permission to publish to the Pub/Sub topic.
- You exceeded your notification quota.
- You have organization policies in place that limit resource sharing by domain.
For more information, see Publish and receive messages in Pub/Sub.
What's next
To learn more about how you can use programmatic notifications, review the following examples of programmatic cost control responses:
- Listen to your notifications
- Send notifications to Slack
- Control resource usage with notifications
- Disable billing usage with notifications