Programmatic Budgets Notification Examples

If you are cost conscious and need to control your environment relative to your budget, then you can use programmatic budget notifications to automate your budget actions.

Budget notifications use Cloud Pub/Sub notifications to provide real time budget state, using the scalability, flexibility, and reliability of enterprise message-oriented middleware for the cloud.

This page has examples and step by step instructions on how to use budget notifications with Cloud Functions to automate cost management.

Set up budget notifications

The first step is to enable a Cloud Pub/Sub notification for your budget. This step is described at Manage Notifications.

After enabling budget notifications, note the following:

  • Pub/Sub Topic—This is the configured notifications endpoint for the budget.
  • Budget ID—This is a unique ID for your budget that is included in all notifications. You can locate the budget's ID in your budget under Manage notifications. The ID is displayed after you select Connect a Pub/Sub topic to this budget.

Manage billing notifications in the GCP console

Listen to your notifications

The next step is to listen to your notifications by subscribing to your Pub/Sub topic. If you don't have a subscriber, then Pub/Sub will drop published messages and you cannot retrieve them later.

Although there are many ways you can subscribe your topic, for these examples we will use Cloud Function triggers.

Create a Cloud Function

To create a new Cloud Function:

  1. Go to the Cloud Functions page of the Google Cloud Platform Console. Create a new function and give it a name that is meaningful to your budget.
  2. Under Trigger, select Cloud Pub/Sub topic.
  3. Select the topic that you configured on your budget.
  4. Provide source code and dependencies for the function to run.

Create a new function in the GCP console

Describe your Cloud Function

To tell your Cloud Function what you want it to do with the notification, you can either write code using the inline editor or you can upload a file. For details about the notifications your code will receive, see Notification Format.

For example, a function might log received Pub/Sub notifications, attributes, and data when triggered by a budget notification. To learn more, see Google Cloud Pub/Sub Triggers.

View your Cloud Function events

After you save the Cloud Function, you can click VIEW LOGS to view your logged budget notifications. This shows the logs from your function invocations.

View Cloud Function events in the GCP console

Send notifications to Slack

Email isn't always the best way to stay up to date on your cloud costs, particularly if your budget is critical and time sensitive. With notifications you can forward your budget messages to other mediums.

In this example we describe how to forward budget notifications to Slack. This way, every time Cloud Billing publishes a budget notification, a Cloud Function uses a bot to post a message to a Slack channel of the bot's workspace.

Set up a Slack channel and permissions

The first step is to create your Slack workspace and the bot user tokens that are used to call the Slack API. API tokens can be managed at https://api.slack.com/apps. For detailed instructions, see Bot Users on the Slack site.

Configure Slack notifications

Write a Cloud Function

  1. Create a new function following the steps in Create a Cloud Function.

  2. Add dependencies:

    Node.js 8 (Beta)

    Add a dependency on slack npm package to your function's package.json file.
    {
      "name": "cloud-functions-billing",
      "version": "0.0.1",
      "dependencies": {
        "slack": "^11.0.1"
      }
    }
    

    Python (Beta)

    Add slackclient==1.3.0 to your function's requirements.txt file:
    slackclient==1.3.0

  3. Write code to post budget notifications to a Slack chat channel using the Slack API.

  4. Set the following Slack API postMessage parameters in your code:

    • Bot User OAuth access token
    • Channel
    • Text

For example:

Node.js 8 (Beta)

const slack = require('slack');

const BOT_ACCESS_TOKEN = 'xxxx-111111111111-abcdefghidklmnopq';
const CHANNEL = 'general';

exports.notifySlack = async (data, context) => {
  const pubsubMessage = data;
  const pubsubAttrs = JSON.stringify(pubsubMessage.attributes);
  const pubsubData = Buffer.from(pubsubMessage.data, 'base64').toString();
  const budgetNotificationText = `${pubsubAttrs}, ${pubsubData}`;

  const res = await slack.chat.postMessage({
    token: BOT_ACCESS_TOKEN,
    channel: CHANNEL,
    text: budgetNotificationText
  });
  console.log(res);
};

Python (Beta)

from slackclient import SlackClient

# See https://api.slack.com/docs/token-types#bot for more info
BOT_ACCESS_TOKEN = 'xxxx-111111111111-abcdefghidklmnopq'

CHANNEL = 'general'

slack_client = SlackClient(BOT_ACCESS_TOKEN)


def notify_slack(data, context):
    pubsub_message = data

    notification_attrs = json.dumps(pubsub_message['attributes'])
    notification_data = base64.b64decode(data['data']).decode('utf-8')
    budget_notification_text = f'{notification_attrs}, {notification_data}'

    res = slack_client.api_call(
      'chat.postMessage',
      channel=CHANNEL,
      text=budget_notification_text)
    print(res)

Cap (disable) billing to stop usage

This example shows you how to cap costs and stops usage for a Google Cloud project by disabling billing. This will cause all Google Cloud services to terminate non-free tier services for the project.

You might cap costs because you have a hard limit on how much money you can spend on Google Cloud Platform. This is typical for students, researchers, or developers working in sandbox environments. In these cases you want to stop the spending and might be willing to shutdown all your Google Cloud Platform services and usage when your budget limit is reached.

For our example, we use acme-backend-dev as a non-production project for which billing can be safely disabled.

Configure the budget cap in the GCP console

Before you start, you need to set up a budget to monitor project costs and enable budget notifications.

Manage billing alerts in the GCP console

Configure service account permissions

Your Cloud Function is run as an automatically created service account. So that the service account can disable billing, you need to grant it Billing Admin permissions.

To identify the correct service account, view your Cloud Function details. The service account is listed at the bottom of the page.

Identify the service account in the GCP console

You can manage Billing Admin permissions on the Google Cloud Platform Console Billing page.

To grant Billing Account Administrator privileges to the service account, select the service account name.

Manage permissions in the GCP console

Write a Cloud Function

Next you need to configure your Cloud Function to call the Cloud Billing API. This enables the Cloud Function to disable billing for our example project acme-backend-dev.

  1. Create a new function using the steps in Create a Cloud Function.

  2. Add the following dependencies:

    Node.js 8 (Beta)

    Add dependencies on googleapis and google-auth-library to your function's package.json file:
    {
     "name": "cloud-functions-billing",
     "version": "0.0.1",
     "dependencies": {
        "google-auth-library": "^2.0.0",
        "googleapis": "^33.0.0"
     }
    }

    Python (Beta)

    Add oauth2client==4.1.3 and google-api-python-client==1.7.4 to your function's requirements.txt file:
    oauth2client==4.1.3
    google-api-python-client==1.7.4
    

  3. Write code to disable billing on the project by removing it from the billing account.

  4. Set the PROJECT_NAME parameter. This is the project that you want to cap (disable) billing for.

Node.js 8 (Beta)

const { google } = require('googleapis');
const { auth } = require('google-auth-library');

const PROJECT_ID = process.env.GCP_PROJECT;
const PROJECT_NAME = `projects/${PROJECT_ID}`;
const compute = google.compute('v1');
const ZONE = 'us-west1-b';

exports.limitUse = async (data, context) => {
  const pubsubData = JSON.parse(Buffer.from(data.data, 'base64').toString());
  if (pubsubData.costAmount <= pubsubData.budgetAmount) {
    return `No action necessary. (Current cost: ${pubsubData.costAmount})`;
  }

  await _setAuthCredential();
  const instanceNames = await _listRunningInstances(PROJECT_ID, ZONE);
  await _stopInstances(PROJECT_ID, ZONE, instanceNames);
};

/**
 * @return {Promise} Array of names of running instances
 */
const _listRunningInstances = async (projectId, zone) => {
  const res = await compute.instances.list({
    project: projectId,
    zone: zone
  });

  const instances = res.data.items || [];
  const ranInstances = instances.filter(item => item.status === 'RUNNING');
  return ranInstances.map(item => item.name);
};

/**
 * @param {Array} instanceNames Names of instance to stop
 * @return {Promise} Response from stopping instances
 */
const _stopInstances = async (projectId, zone, instanceNames) => {
  if (!instanceNames.length) {
    return 'No running instances were found.';
  }
  await Promise.all(instanceNames.map(instanceName => {
    return compute.instances.stop({
      project: projectId,
      zone: zone,
      instance: instanceName
    }).then((res) => {
      console.log('Instance stopped successfully: ' + instanceName);
      return res.data;
    });
  }));
};

Python (Beta)

import base64
import json
import os
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials

PROJECT_ID = os.getenv('GCP_PROJECT')
PROJECT_NAME = f'projects/{PROJECT_ID}'
def stop_billing(data, context):
    pubsub_data = base64.b64decode(data['data']).decode('utf-8')
    pubsub_json = json.loads(pubsub_data)
    cost_amount = pubsub_json['costAmount']
    budget_amount = pubsub_json['budgetAmount']
    if cost_amount <= budget_amount:
        print(f'No action necessary. (Current cost: {cost_amount})')
        return

    billing = discovery.build(
        'cloudbilling',
        'v1',
        cache_discovery=False,
        credentials=GoogleCredentials.get_application_default()
    )

    projects = billing.projects()

    if __is_billing_enabled(PROJECT_NAME, projects):
        print(__disable_billing_for_project(PROJECT_NAME, projects))
    else:
        print('Billing already disabled')


def __is_billing_enabled(project_name, projects):
    """
    Determine whether billing is enabled for a project
    @param {string} project_name Name of project to check if billing is enabled
    @return {bool} Whether project has billing enabled or not
    """
    res = projects.getBillingInfo(name=project_name).execute()
    return res['billingEnabled']


def __disable_billing_for_project(project_name, projects):
    """
    Disable billing for a project by removing its billing account
    @param {string} project_name Name of project disable billing on
    @return {string} Text containing response from disabling billing
    """
    body = {'billingAccountName': ''}  # Disable billing
    res = projects.updateBillingInfo(name=project_name, body=body).execute()
    print(f'Billing disabled: {json.dumps(res)}')


ZONE = 'us-west1-b'


def limit_use(data, context):
    pubsub_data = base64.b64decode(data['data']).decode('utf-8')
    pubsub_json = json.loads(pubsub_data)
    cost_amount = pubsub_json['costAmount']
    budget_amount = pubsub_json['budgetAmount']
    if cost_amount <= budget_amount:
        print(f'No action necessary. (Current cost: {cost_amount})')
        return

    compute = discovery.build(
        'compute',
        'v1',
        cache_discovery=False,
        credentials=GoogleCredentials.get_application_default()
    )
    instances = compute.instances()

    instance_names = __list_running_instances(PROJECT_ID, ZONE, instances)
    __stop_instances(PROJECT_ID, ZONE, instance_names, instances)


def __list_running_instances(project_id, zone, instances):
    """
    @param {string} project_id ID of project that contains instances to stop
    @param {string} zone Zone that contains instances to stop
    @return {Promise} Array of names of running instances
    """
    res = instances.list(project=project_id, zone=zone).execute()

    items = res['items']
    running_names = [i['name'] for i in items if i['status'] == 'RUNNING']
    return running_names


def __stop_instances(project_id, zone, instance_names, instances):
    """
    @param {string} project_id ID of project that contains instances to stop
    @param {string} zone Zone that contains instances to stop
    @param {Array} instance_names Names of instance to stop
    @return {Promise} Response from stopping instances
    """
    if not len(instance_names):
        print('No running instances were found.')
        return

    for name in instance_names:
        instances.stop(
          project=project_id,
          zone=zone,
          instance=name).execute()
        print(f'Instance stopped successfully: {name}')

Validate that billing is disabled

When the function is triggered, you can validate that your Cloud Function was successful. The project will no longer be visible under the billing account and resources in the project are disabled.

Validate that billing is disabled

You can manually re-enable billing for your project in the GCP Console.

Selectively control usage

Capping (disabling billing) as described in the previous example is binary and terminal. Your project is either enabled or disabled. When it is disabled all services are stopped and all resources are eventually deleted.

If you require a more nuanced response, you can selectively control resources. For example, if you want to stop some Compute resources but leave Storage intact, then you can selectively control usage. This will reduce your cost per hour without completely disabling your environment.

You can write as nuanced a policy as you like. However, for our example, our project is running research with a number of Compute virtual machines, and is storing results in Cloud Storage. This Cloud Function example will shut down all Compute instances, but will not impact our stored results after the budget is exceeded.

Configure service account permissions

  1. Your Cloud Function is run as an automatically created service account. To control usage you need to grant the service account permissions to any services on the project that it needs to make changes.
  2. To identify the correct service account, view the details of your Cloud Function. The service account is listed at the bottom of the page.
  3. Go to the IAM page of the GCP Console to set the appropriate permissions.
     
    Manage billing alerts in the GCP console

Write a Cloud Function

  1. Create a new function using the steps in Create a Cloud Function.

  2. Make sure that you have added the dependencies described in Cap (disable) billing to stop usage.

  3. Write code to shut down resources in the project.

  4. Set the PROJECT_NAME parameter. This is the project you want to configure for capping.

Node.js 8 (Beta)

const { google } = require('googleapis');
const { auth } = require('google-auth-library');

const PROJECT_ID = process.env.GCP_PROJECT;
const PROJECT_NAME = `projects/${PROJECT_ID}`;
const billing = google.cloudbilling('v1').projects;

exports.stopBilling = async (data, context) => {
  const pubsubData = JSON.parse(Buffer.from(data.data, 'base64').toString());
  if (pubsubData.costAmount <= pubsubData.budgetAmount) {
    return `No action necessary. (Current cost: ${pubsubData.costAmount})`;
  }

  await _setAuthCredential();
  if (await _isBillingEnabled(PROJECT_NAME)) {
    return _disableBillingForProject(PROJECT_NAME);
  } else {
    return 'Billing already disabled';
  }
};

/**
 * @return {Promise} Credentials set globally
 */
const _setAuthCredential = async () => {
  const res = await auth.getApplicationDefault();

  let client = res.credential;
  if (client.hasScopes && !client.hasScopes()) {
    client = client.createScoped([
      'https://www.googleapis.com/auth/cloud-billing',
      'https://www.googleapis.com/auth/cloud-platform'
    ]);
  }

  // Set credential globally for all requests
  google.options({
    auth: client
  });
};

/**
 * Determine whether billing is enabled for a project
 * @param {string} projectName Name of project to check if billing is enabled
 * @return {bool} Whether project has billing enabled or not
 */
const _isBillingEnabled = async (projectName) => {
  const res = await billing.getBillingInfo({ name: projectName });
  return res.data.billingEnabled;
};

/**
 * Disable billing for a project by removing its billing account
 * @param {string} projectName Name of project disable billing on
 * @return {string} Text containing response from disabling billing
 */
const _disableBillingForProject = async (projectName) => {
  const res = await billing.updateBillingInfo({
    name: projectName,
    resource: { 'billingAccountName': '' } // Disable billing
  });
  return `Billing disabled: ${JSON.stringify(res.data)}`;
};

Python (Beta)

import base64
import json
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials

PROJECT_ID = os.getenv('GCP_PROJECT')
PROJECT_NAME = f'projects/{PROJECT_ID}'

Validate that billing is disabled

You can validate that the function ran successfully by checking your Compute virtual machines were stopped in the GCP Console.

Was this page helpful? Let us know how we did: