Manage programmatic budget alert notifications

Overview

The typical behavior of a budget is to send alert emails when the budget alert threshold rules trigger an email notification.

Figure 1: Illustrates the default email functionality for budget alert notifications and the optional method to automate cost control responses using Pub/Sub for programmatic notifications.

However, if you're using budgets as a cost control tool, email notifications might not be the best notification method to use to ensure timely action to control your Google Cloud costs. Beyond email notifications, you can use programmatic notifications to automate cost control responses. Examples of cost control responses include:

• Capping your Google Cloud spend by automatically disabling billing or terminating resources.
• Forwarding your budget messages to other mediums (such as Slack).
• Controlling your resource usage by adjusting quotas.

For more information, see Examples of automated cost control responses.

Note: Budgets use estimated Cloud Billing data which is subject to change until your invoice is finalized. In rare cases you may see a cost decrease between notifications.

Permissions required for this task

Depending on whether you're managing budgets at the Cloud Billing account level, or you're a project owner without any Cloud Billing account roles, you need a different set of permissions, described in the following sections.

If you're monitoring costs at the billing account level

If you're primarily monitoring costs for your Cloud Billing account, you need permissions at the Cloud Billing account level.

If you have limited or no access to the Cloud Billing account, you might have sufficient permissions for projects that you own.

• To create or modify a budget for your Cloud Billing account, you need one of the following roles:

  • Billing Account Costs Manager
  • Billing Account Administrator on the Cloud Billing account.

• To set or modify Pub/Sub topics, you need the Pub/Sub Admin role on the target project that contains the Pub/Sub topics.

If your organization uses custom roles for authentication, you need these specific permissions in the custom roles:

• billing.budgets.create on the Cloud Billing account, to create a new budget.
• billing.budgets.get and billing.budgets.list on the Cloud Billing account, to view all budgets for the Cloud Billing account.
• billing.budgets.update on the Cloud Billing account, to modify an existing budget.
• pubsub.topics.setIamPolicy and pubsub.topics.list on the target project that contains the Pub/Sub topics that you want to use.

For custom roles to modify an existing Cloud Billing budget to connect it to an existing Pub/Sub topic, you need these specific permissions:

• billing.budgets.get and billing.budgets.list on the Cloud Billing account, to view all budgets for the Cloud Billing account.
• billing.budgets.update on the Cloud Billing account, to modify an existing budget.
• pubsub.topics.list on the target project that contains the Pub/Sub topics.

If you're monitoring costs at the project level

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

If you have limited or no access to a Cloud Billing account, you can manage budgets that are scoped only to the projects that you own.

• To create, modify, or view budgets for your project, you need one of the following roles on the project:

  • Project Owner
  • Project Editor

• To set or modify Pub/Sub topics, you need the Pub/Sub Admin role on the target project that contains the Pub/Sub topics.

If your organization uses custom roles for authentication, you need these specific permissions in the custom roles:

• resourcemanager.projects.get on your project, to view the project associations (such as the linked billing account) for your project.
• billing.resourceCosts.get on your project (optional), to view costs and usage for your project.
• billing.resourcebudgets.read on your project, to view a billing account's single-project budgets that are scoped to your project.
• billing.resourcebudgets.write on your project, to create or modify single-project budgets that are scoped to your project.
• pubsub.topics.setIamPolicy and pubsub.topics.list on the target project that contains the Pub/Sub topics that you want to use.

Permissions related to a domain restriction constraint

If you set organization policies to limit resource sharing by domain (for example, enabling enforceInTransit on Pub/Sub topics), you might encounter an error when attempting to set up Pub/Sub topics or connect them to a Cloud Billing budget.

In this situation, you might need to force account access to successfully connect a Pub/Sub topic to a budget. To force account access, do the following:

1. Remove the organization policy containing the domain restriction constraint using one of the following methods:

   • Temporarily remove the organization policy.

     OR

   • Override the organization policy for the project that contains the Pub/Sub topic.

2. Follow the steps to connect a Pub/Sub topic to a Cloud Billing budget.

3. Reinstate the organization policy for the organization or project (optional).

About Google Cloud permissions

For more information about Cloud Billing permissions, see:

• Overview of Cloud Billing access control
• Create custom roles for Cloud Billing
• Understanding predefined Identity and Access Management roles for Cloud Billing

For more information about Google Cloud project permissions, see:

• Access control for projects with IAM
• IAM basic and predefined roles reference

How to setup and enable Pub/Sub programmatic notifications

To enable programmatic notifications to receive Pub/Sub messages with the current status of your budget, connect a Pub/Sub topic to your Cloud Billing budget.

Note: You must have permission to grant the Pub/Sub Publisher role on the topic to allow Cloud Billing to publish messages to it.

To connect, edit, or remove a Pub/Sub topic linked to your budget, do the following:

1. Create a Pub/Sub topic

Configure a Pub/Sub topic for Cloud Billing to publish your budget alerts to. If you already have a Cloud Billing budgets Pub/Sub topic configured, you can reuse it for all budgets, or you can create a unique topic per budget.

1. Go to the Google Cloud console Pub/Sub page.

   Sign in to Google Cloud console Pub/Sub

2. Select a project.

   We recommend that you create and use a separate Google Cloud project to contain all of your FinOps and billing administration needs for a Cloud Billing account, including your Cloud Billing-related Pub/Sub topics. Learn more about the benefits to using a FinOps-focused project for your billing administration use cases.

3. Click Create topic.

4. In the Topic ID field, provide a topic name, and click Create topic.

2. Connect a Cloud Billing budget to a Pub/Sub topic

To connect a Cloud Billing budget to a Pub/Sub topic, do the following:

1. Using the procedure that fits your level of access to Cloud Billing accounts, sign in to the Budgets & alerts page in the Billing section of the Google Cloud console:

   Users with Cloud Billing account permissions

   Users with project-level permissions only (Preview)

   If you have Cloud Billing account permissions, you can select from a list of billing accounts that you have permissions to access. Sign in to the Budgets & alerts page in the Google Cloud console. Open Budgets & alerts At the prompt, choose the Cloud Billing account for which you'd like to manage budgets. The Budgets & alerts page opens for the selected billing account.

   If you only have project permissions, but don't have any permissions on your project's Cloud Billing account, you'll need to select your project before you navigate to the Billing section. Sign in to the Google Cloud console dashboard and select a project. Open Google Cloud console Select the project for which you want to manage budgets. Next, navigate to Billing: Open the Google Cloud console Navigation menu , and then select Billing. If you're prompted to choose which billing account you want to view and manage, click Go to linked billing account to view the billing account that's linked to your selected project. The Billing Overview page opens for the selected billing account. In the Cost management section of the Billing navigation menu, select Budgets & alerts.

2. If necessary, create a budget. Or, to edit an existing budget and alert rules, click a budget name.

3. In the Actions section, under Manage notifications, select Connect a Pub/Sub topic to this budget.

4. In the Project field, click Select a project and choose the project that contains the Pub/Sub topics you've set up. If you followed the recommendation in the previous section, this is your FinOps or billing administration Google Cloud project.

5. Under Select a Pub/Sub topic, select an existing topic, or create a new topic by selecting Create a topic.

   • If creating a new topic, in the Topic ID field, provide a topic name, and click Create topic.
   • After it's created, the topic will be selected.

6. Click Save.

Notification format

Notifications sent to the Pub/Sub topic consist of two parts:

• Attributes: A set of key-value pairs describing the event.
• Data: A string containing a JSON object that describes the budget alert details.

Note: You can find and test an example notification by reading Examples of automated cost control responses and setting up a Cloud Run function.

Attributes

Attributes are key-value pairs contained in all notifications sent by Cloud Billing to your Pub/Sub topic. Notifications always contain the following set of key-value pairs, regardless of the notification payload.

Attribute name	Example	Description
billingAccountId	01D4EE-079462-DFD6EC	The identifier for the Cloud Billing account owning the budget.
budgetId	de72f49d-779b-4945-a127-4d6ce8def0bb	The identifier for the budget alert within the Cloud Billing account.
schemaVersion	1.0	The notifications schema version.

Data

For schema version 1.x, budget alert details are returned in a base64-encoded UTF-8 string containing a JSON object with the following properties:

Data	Example	Description
budgetDisplayName	"My Personal Budget"	The human-readable name assigned to the budget.
costAmount	140.321	The amount of costs accrued. The type of costs tracked depends on budget filters & settings.
costIntervalStart	"2021-02-01T08:00:00Z"	The start of the budget alert period. `costAmount` includes costs for usage starting at this time. This is the first day of the budget's time period (month, quarter, year, or a custom date range) during which the budget usage occurred.
budgetAmount	152.557	The amount allocated in the budget.
budgetAmountType	"SPECIFIED_AMOUNT"	The budget amount type. This can be one of the following values: "SPECIFIED_AMOUNT": A fixed amount. "LAST_MONTH_COST": Total costs incurred during the last calendar month. "LAST_PERIODS_COST": The costs incurred during the last, non-monthly, calendar period, such as a quarter or a year.
alertThresholdExceeded	0.9	The highest configured actual cost alert threshold exceeded. Value is a percent in decimal format (for example, 0.9 is 90%). This key isn't present if the actual cost doesn't exceed any threshold.
forecastThresholdExceeded	0.2	The highest forecasted cost alert threshold exceeded. Value is a percent in decimal format (for example, 0.2 is 20%). This key isn't present if the forecasted cost doesn't exceed any threshold.
currencyCode	"USD"	The budget alert currency. All costs and budget alert amounts are in this currency.

Delivery guarantees for notifications to Pub/Sub

After you create a budget and connect it to a Pub/Sub topic, you can expect to receive programmatic notifications as described here:

• Budget notifications are sent to the Pub/Sub topic multiple times per day with the current status of your budget. This is a different cadence than the budget alert emails, which are only sent when a budget threshold is met.
• Pub/Sub notifications are sent even if the Cloud Billing account has no usage.
• It may take several hours before receiving the first Pub/Sub notification.
• Pub/Sub only guarantees at-least-once delivery. You might receive a message multiple times, and messages might arrive out of order.

• If delivery fails due to Pub/Sub topic misconfiguration, you won't be notified.

  Delivery might fail for one of the following reasons:

  • The Pub/Sub topic no longer exists.
  • The Cloud Billing no longer has permission to publish to the Pub/Sub topic.
  • You exceeded your notification quota.
  • You have organization policies in place that limit resource sharing by domain.