If you are cost conscious and need to control your environment relative to your budget, then you can use programmatic budget notifications to automate your cost control response based on the budget notification.
Budget notifications use Pub/Sub topics to provide a real-time status of the Cloud Billing budget, using the scalability, flexibility, and reliability of enterprise message-oriented middleware for the cloud.
This page has examples and step-by-step instructions on how to use budget notifications with Cloud Run functions to automate cost management.
Set up budget notifications
The first step is to enable a Pub/Sub topic for your budget. This is described in detail at Manage programmatic budget alert notifications.
After enabling budget notifications, note the following:
- Pub/Sub Topic: This is the configured notifications endpoint for the budget.
- Budget ID: This is a unique ID for your budget that is included in all notifications. You can locate the budget's ID in your budget under Manage notifications. The ID is displayed after you select Connect a Pub/Sub topic to this budget.
Listen to your notifications
The next step is to listen to your notifications by subscribing to your Pub/Sub topic. If you don't have a subscriber, then Pub/Sub will drop published messages and you cannot retrieve them later.
Although there are many ways you can subscribe to your topic, for these examples we will use Cloud Run function triggers.
Create a Cloud Run function
To create a new Cloud Run function:
In the Google Cloud console, go to the Cloud Run functions page.
Click
CREATE FUNCTION and give the function a name that is meaningful to your budget.Under Trigger, select Pub/Sub topic.
Select the topic that you configured on your budget.
Provide source code and dependencies for the function to run.
Ensure you set the Function to execute to the correct function name.
Describe your Cloud Run function
To tell your Cloud Run function what you want it to do with the notification, you can either write code using the inline editor or you can upload a file. For details about the notifications your code will receive, see Notification Format.
For example, a function might log received Pub/Sub notifications, attributes, and data when triggered by a budget notification. To learn more, see Pub/Sub Triggers.
View your Cloud Run function events
After you save the Cloud Run function, you can click VIEW LOGS to view your logged budget notifications. This shows the logs from your function invocations.
Test your Cloud Run function
Notifications are sent to Pub/Sub and subscribers receive the messages. To test a sample notification to make sure your function is working as expected, publish a message in Pub/Sub using this object as the message body:
{ "budgetDisplayName": "name-of-budget", "alertThresholdExceeded": 1.0, "costAmount": 100.01, "costIntervalStart": "2019-01-01T00:00:00Z", "budgetAmount": 100.00, "budgetAmountType": "SPECIFIED_AMOUNT", "currencyCode": "USD" }
You can also add message attributes such as the billing account ID. See the full notification format documentation for more information.
Send notifications to Slack
Email isn't always the best way to stay up to date on your cloud costs, particularly if your budget is critical and time sensitive. With notifications you can forward your budget messages to other mediums.
In this example we describe how to forward budget notifications to Slack. This way, every time Cloud Billing publishes a budget notification, a Cloud Run function uses a bot to post a message to a Slack channel of the bot's workspace.
Set up a Slack channel and permissions
The first step is to create your Slack workspace and the bot user tokens that are used to call the Slack API. API tokens can be managed at https://api.slack.com/apps. For detailed instructions, see Bot Users on the Slack site.
Write a Cloud Run function
Create a new function following the steps in Create a Cloud Run function. Ensure that the trigger is set to the same Pub/Sub topic that your budget is set to use.
Add dependencies:
Node.js
Copy the following to your
package.json
:Python
Copy the following to your
requirements.txt
:Write code or use the example below to post budget notifications to a Slack chat channel using the Slack API.
Ensure the following Slack API postMessage parameters are set correctly:
- Bot User OAuth access token
- Channel name
Example code:
Node.js
Python
Now you can Test your Cloud Run function to see a message show up in Slack.
Cap (disable) billing to stop usage
This example shows you how to cap costs and stop usage for a project by disabling Cloud Billing. Disabling billing on a project will cause all Google Cloud services in the project to terminate, including free-tier services.
Why disable billing?
You might cap costs because you have a hard limit on how much money you can spend on Google Cloud. This is typical for students, researchers, or developers working in sandbox environments. In these cases you want to stop the spending and might be willing to shutdown all your Google Cloud services and usage when your budget limit is reached.
For our example, we use acme-backend-dev as a non-production project for which Cloud Billing can be safely disabled.
Before you start working with this example, ensure you have done the following:
Enable the Cloud Billing API. Your Cloud Run function needs to call the Cloud Billing API to disable Cloud Billing for a project.
Set up a budget to monitor project costs and enable budget notifications.
Write a Cloud Run function
Next you need to configure your Cloud Run function to call the Cloud Billing API. This enables the Cloud Run function to disable Cloud Billing for our example project acme-backend-dev.
Create a new function following the steps in Create a Cloud Run function. Ensure that the trigger is set to the same Pub/Sub topic that your budget is set to use.
Add the following dependencies:
Node.js
Copy the following to your
package.json
:Python
Copy the following to your
requirements.txt
:Copy the code below into the Cloud Run function.
Set the function to execute to "stopBilling" (Node) or "stop_billing" (Python).
Depending on your runtime, the
GOOGLE_CLOUD_PROJECT
environment variable might be set automatically. Review the list of environment variables set automatically and determine if you need to manually set theGOOGLE_CLOUD_PROJECT
variable to the project for which you want to cap (disable) Cloud Billing.
Node.js
Python
Configure service account permissions
Your Cloud Run function is run as an automatically created service account. So that the service account can disable billing, you need to grant it the correct permissions, such as Billing Admin.
To identify the correct service account, view your Cloud Run function details. The service account is listed at the bottom of the page.
You can manage Billing Admin permissions on the Billing page in the Google Cloud console.
To grant Billing Account Administrator privileges to the service account, select the service account name.
Validate that Cloud Billing is disabled
When the budget sends out a notification, the specified project will no longer have a Cloud Billing account. If you want to test the function, publish a sample message with the testing message above. The project will no longer be visible under the Cloud Billing account and resources in the project are disabled, including the Cloud Run function if it's in the same project.
You can manually re-enable Cloud Billing for your project in the Google Cloud console.
Selectively control usage
Capping (disabling) Cloud Billing as described in the previous example is binary and terminal. Your project is either enabled or disabled. When it is disabled all services are stopped and all resources are eventually deleted.
If you require a more nuanced response, you can selectively control resources. For example, if you want to stop some Compute Engine resources but leave Cloud Storage intact, then you can selectively control usage. This reduces your cost per hour without completely disabling your environment.
You can write as nuanced a policy as you like. However, for our example, our project is running research with a number of Compute Engine virtual machines, and is storing results in Cloud Storage. This Cloud Run function example will shut down all Compute Engine instances, but will not impact our stored results after the budget is exceeded.
Write a Cloud Run function
Create a new function following the steps in Create a Cloud Run function. Ensure that the trigger is set to the same Pub/Sub topic that your budget is set to use.
Make sure that you have added the dependencies described in Cap (disable) billing to stop usage.
Copy the code below into the Cloud Run function.
Set the function to execute to "limitUse" (Node) or "limit_use" (Python).
Depending on your runtime, the
GCP_PROJECT
environment variable might be set automatically. Review the list of environment variables set automatically and determine if you need to manually set theGCP_PROJECT
variable to the project running the virtual machines.Set the ZONE parameter. This is the zone where instances will be stopped for this sample.
Node.js
Python
Configure service account permissions
- Your Cloud Run function is run as an automatically created service account. To control usage you need to grant the service account permissions to any services on the project that it needs to make changes.
- To identify the correct service account, view the details of your Cloud Run function. The service account is listed at the bottom of the page.
- In the Google Cloud console, go to the IAM page to set the appropriate permissions.
Go to the IAM page
Validate that instances are stopped
When the budget sends out a notification, the Compute Engine virtual machines are stopped.
To test the function, publish a sample message with the testing message above. To validate that the function ran successfully, check your Compute Engine virtual machines in the Google Cloud console.