Programmatic Budgets Notification Examples

If you are cost conscious and need to control your environment relative to your budget, then you can use programmatic budget notifications to automate your budget actions.

Budget notifications use Cloud Pub/Sub notifications to provide real time budget state, using the scalability, flexibility, and reliability of enterprise message-oriented middleware for the cloud.

This page has examples and step by step instructions on how to use budget notifications with Cloud Functions to automate cost management.

Set up budget notifications

The first step is to enable a Cloud Pub/Sub notification for your budget. This step is described at Manage Notifications.

After enabling budget notifications, note the following:

  • Pub/Sub Topic—This is the configured notifications endpoint for the budget.
  • Budget ID—This is a unique ID for your budget that is included in all notifications. You can locate the budget's ID in your budget under Manage notifications. The ID is displayed after you select Connect a Pub/Sub topic to this budget.

Manage billing notifications in the GCP console

Listen to your notifications

The next step is to listen to your notifications by subscribing to your Pub/Sub topic. If you don't have a subscriber, then Pub/Sub will drop published messages and you cannot retrieve them later.

Although there are many ways you can subscribe your topic, for these examples we will use Cloud Function triggers.

Create a Cloud Function

To create a new Cloud Function:

  1. Go to the Cloud Functions page of the Google Cloud Platform Console. Create a new function and give it a name that is meaningful to your budget.
  2. Under Trigger, select Cloud Pub/Sub topic.
  3. Select the topic that you configured on your budget.
  4. Provide source code and dependencies for the function to run.

Create a new function in the GCP console

Describe your Cloud Function

To tell your Cloud Function what you want it to do with the notification, you can either write code using the inline editor or you can upload a file. For details about the notifications your code will receive, see Notification Format.

In this example, the function will log received Pub/Sub notifications, attributes, and data when triggered by a budget notification.

/**
 * Triggered from a message on a Cloud Pub/Sub topic.
 *
 * @param {!Object} event The Cloud Functions event.
 * @param {!Function} The callback function.
 */
exports.subscribe = (event, callback) => {
  const pubsubMessage = event.data;
  console.log(JSON.stringify(pubsubMessage.attributes));
  console.log(Buffer.from(pubsubMessage.data, 'base64').toString());

  // Don't forget to call the callback to acknowledge the message.
  callback();
};

View your Cloud Function events

After you save the Cloud Function, you can click VIEW LOGS to view your logged budget notifications. This shows the logs from your function invocations.

View Cloud Function events in the GCP console

Send notifications to Slack

Email isn't always the best way to stay up to date on your cloud costs, particularly if your budget is critical and time sensitive. With notifications you can forward your budget messages to other mediums.

In this example we describe how to forward budget notifications to Slack. This way, every time Cloud Billing publishes a budget notification, a Cloud Function uses a bot to post a message to a Slack channel of the bot's workspace.

Set up a Slack channel and permissions

The first step is to create your Slack workspace and the bot user tokens that are used to call the Slack API. API tokens can be managed at https://api.slack.com/apps. For detailed instructions, see Bot Users on the Slack site.

Configure Slack notifications

Write a Cloud Function

  1. Create a new function following the steps in Create a Cloud Function.

  2. Add a dependency on slack npm package to your function's package.json.

    {
         "name": "sample-pubsub",
         "version": "0.0.1",
         "dependencies": {
           "slack": "11.0.0"
         }
       }
       

  3. Write code to post budget notifications to a Slack chat channel using the Slack API.
  4. Set the following Slack API postMessage parameters in your code:
    • Bot User OAuth access token
    • Channel
    • Text

For example, your index.js might look like the following sample.

const slack = require('slack');

const BOT_ACCESS_TOKEN = 'xxxx-111111111111-abcdefghidklmnopq';
const CHANNEL = 'general';

exports.subscribe = (event) => {
  const pubsubMessage = event.data;
  const budgetNotificationText = JSON.stringify(pubsubMessage.attributes)
    + ', '
    + Buffer.from(pubsubMessage.data, 'base64').toString();

  return slack.chat.postMessage({
    token: BOT_ACCESS_TOKEN,
    channel: CHANNEL,
    text: budgetNotificationText,
  }).finally(console.log);
};

Cap (disable) billing to stop usage

This example shows you how to cap costs and stops usage for a Google Cloud project by disabling billing. This will cause all Google Cloud services to terminate non-free tier services for the project.

You might cap costs because you have a hard limit on how much money you can spend on Google Cloud Platform. This is typical for students, researchers, or developers working in sandbox environments. In these cases you want to stop the spending and might be willing to shutdown all your Google Cloud Platform services and usage when your budget limit is reached.

For our example, we use acme-backend-dev as a non-production project for which billing can be safely disabled.

Configure the budget cap in the GCP console

Before you start, you need to set up a budget to monitor project costs and enable budget notifications.

Manage billing alerts in the GCP console

Configure service account permissions

Your Cloud Function is run as an automatically created service account. So that the service account can disable billing, you need to grant it Billing Admin permissions.

To identify the correct service account, view your Cloud Function details. The service account is listed at the bottom of the page.

Identify the service account in the GCP console

You can manage Billing Admin permissions on the Google Cloud Platform Console Billing page.

To grant Billing Account Administrator privileges to the service account, select the service account name.

Manage permissions in the GCP console

Write a Cloud Function

Next you need to configure your Cloud Function to call the Cloud Billing API. This enables the Cloud Function to disable billing for our example project acme-backend-dev.

  1. Create a new function using the steps in Create a Cloud Function.

  2. Add dependencies on googleapis and google-auth-library by editing the function's package.json.

       {
         "name": "sample-pubsub",
         "version": "0.0.1",
         "dependencies": {
            "google-auth-library": "1.5.0",
            "googleapis": "30.0.0"
         }
       }

  3. Write code to disable billing on the project by removing it from the billing account.
  4. You need to set the Project name parameter. This is the project that you want to cap (disable) billing for.
const {google} = require('googleapis');
const cloudbilling = google.cloudbilling('v1');
const {auth} = require('google-auth-library');
const PROJECT_NAME = 'projects/acme-backend-dev';

exports.subscribe = (event) => {
  const pubsubData = JSON.parse(Buffer.from(event.data.data, 'base64').toString());
  if (pubsubData.costAmount <= pubsubData.budgetAmount) {
    return Promise.resolve('No action shall be taken on current cost ' +
      pubsubData.costAmount);
  }

  return setAuthCredential()
    .then(() => isBillingEnabled(PROJECT_NAME))
    .then((enabled) => {
      if (enabled) {
        return disableBillingForProject(PROJECT_NAME);
      }
      return Promise.resolve('Billing already in disabled state');
    });
};

/**
 * @return {Promise} Credentials set globally
 */
function setAuthCredential() {
  return auth.getApplicationDefault()
    .then((res) => {
      let client = res.credential;
      if (client.createScopedRequired && client.createScopedRequired()) {
        client = client.createScoped([
          'https://www.googleapis.com/auth/cloud-billing'
        ]);
      }

      // Set credential globally for all requests
      google.options({
        auth: client
      });
    });
}

/**
 * @param {string} projectName Name of project to check if billing is enabled
 * @return {Promise} Whether project has billing enabled or not
 */
function isBillingEnabled(projectName) {
  return cloudbilling.projects.getBillingInfo({
    name: projectName
  }).then((res) => res.data.billingEnabled);
};

/**
 * @param {string} projectName Name of project disable billing on
 * @return {Promise} Text containing response from disabling billing
 */
function disableBillingForProject(projectName) {
  return cloudbilling.projects.updateBillingInfo({
    name: projectName,
    // Setting this to empty is equivalent to disabling billing.
    resource: {
      'billingAccountName': ''
    }
  }).then((res) => {
    return 'Billing disabled successfully: ' + JSON.stringify(res.data);
  });
}

Validate that billing is disabled

When the function is triggered, you can validate that your Cloud Function was successful. The project will no longer be visible under the billing account and resources in the project are disabled.

Validate that billing is disabled

You can manually re-enable billing for your project in the GCP Console.

Selectively control usage

Capping (disabling billing) as described in the previous example is binary and terminal. Your project is either enabled or disabled. When it is disabled all services are stopped and all resources are eventually deleted.

If you require a more nuanced response, you can selectively control resources. For example, if you want to stop some Compute resources but leave Storage intact, then you can selectively control usage. This will reduce your cost per hour without completely disabling your environment.

You can write as nuanced a policy as you like. However, for our example, our project is running research with a number of Compute virtual machines, and is storing results in Cloud Storage. This Cloud Function example will shut down all Compute instances, but will not impact our stored results after the budget is exceeded.

Configure service account permissions

  1. Your Cloud Function is run as an automatically created service account. To control usage you need to grant the service account permissions to any services on the project that it needs to make changes.
  2. To identify the correct service account, view the details of your Cloud Function. The service account is listed at the bottom of the page.
  3. Go to the IAM page of the GCP Console to set the appropriate permissions.
     
    Manage billing alerts in the GCP console

Write a Cloud Function

  1. Create a new function using the steps in Create a Cloud Function.

  2. Add dependencies on googleapis and google-auth-library by editing the function's package.json.

       {
         "name": "sample-pubsub",
         "version": "0.0.1",
         "dependencies": {
            "google-auth-library": "1.5.0",
            "googleapis": "30.0.0"
         }
       }

  3. Write code to shut down resources in the project.
  4. Set the following parameters:
    • Project name—the project you would like to configure for capping
    • Zone name—name of zone that contains your instances to shutdown.
const {google} = require('googleapis');
const compute = google.compute('v1');
const {auth} = require('google-auth-library');
const PROJECT_ID = 'acme-research';
const ZONE = 'us-west1-a';

exports.subscribe = (event) => {
  const pubsubData = JSON.parse(Buffer.from(event.data.data, 'base64').toString());
  if (pubsubData.costAmount <= pubsubData.budgetAmount) {
    return Promise.resolve('No action shall be taken on current cost ' +
      pubsubData.costAmount);
  }

  return setAuthCredential()
    .then(() => listRunningInstances(PROJECT_ID, ZONE))
    .then((instanceNames) => stopInstances(PROJECT_ID, ZONE, instanceNames));
};

/**
 * @return {Promise} Credentials set globally
 */
function setAuthCredential() {
  return auth.getApplicationDefault()
    .then((res) => {
      let client = res.credential;
      if (client.createScopedRequired && client.createScopedRequired()) {
        client = client.createScoped([
          'https://www.googleapis.com/auth/cloud-platform',
        ]);
      }

      // Set credential globally for all requests
      google.options({
        auth: client
      });
    });
}

/**
 * @param {string} projectId ID of project that contains instances to stop
 * @param {string} zone Zone that contains instances to stop
 * @return {Promise} Array of names of running instances
 */
function listRunningInstances(projectId, zone) {
  return compute.instances.list({
    project: PROJECT_ID,
    zone: ZONE,
  }).then((res) => {
    return res.data.items.filter(item => item.status == 'RUNNING').map(
      item => item.name);
  });
};

/**
 * @param {string} projectId ID of project that contains instances to stop
 * @param {string} zone Zone that contains instances to stop
 * @param {Array} instanceNames Names of instance to stop
 * @return {Promise} Response from stopping instances
 */
function stopInstances(projectId, zone, instanceNames) {
  if (instanceNames.length == 0) {
    return Promise.resolve('No running instances were found.');
  }
  return Promise.all(instanceNames.map(instanceName => {
    return compute.instances.stop({
      project: projectId,
      zone: zone,
      instance: instanceName,
    }).then((res) => {
      console.log('Instance stopped successfully: ' + instanceName);
      return res.data;
    })
  }));
}

Validate that billing is disabled

You can validate that the function ran successfully by checking your Compute virtual machines were stopped in the GCP Console.

Was this page helpful? Let us know how we did: