Control resource usage with notifications

This document explains how to use budget notifications to selectively control resource usage.

When you disable billing on a project, all services stop and all resources are eventually deleted. If you need a more nuanced response, you can selectively control resources. For example, you can stop some Compute Engine resources while leaving Cloud Storage resources intact. Stopping only some resources reduces your costs without completely disabling your environment.

In the following example, the project runs research with a number of Compute Engine virtual machines (VMs) and stores results in Cloud Storage buckets. Using the budget notifications as the trigger, after the budget is exceeded, this Cloud Run function shuts down all Compute Engine instances, but doesn't affect the stored results.

Before you begin

Before you begin, you must complete the following tasks:

1. Enable the Cloud Billing API
2. Create a budget
3. Set up programmatic budget notifications

Set up a Cloud Run function

1. Complete the steps in Create a Cloud Run function. Ensure that you set the Trigger type to the same Pub/Sub topic that your budget will use.

2. Add the following dependencies:

   Node.js

   Copy the following to your package.json file:

   {
     "name": "cloud-functions-billing",
     "private": "true",
     "version": "0.0.1",
     "description": "Examples of integrating Cloud Functions with billing",
     "main": "index.js",
     "engines": {
       "node": ">=16.0.0"
     },
     "scripts": {
       "compute-test": "c8 mocha -p -j 2 test/periodic.test.js --timeout=600000",
       "test": "c8 mocha -p -j 2 test/index.test.js --timeout=5000 --exit"
     },
     "author": "Ace Nassri <[e-mail]>",
     "license": "Apache-2.0",
     "dependencies": {
       "@google-cloud/billing": "^4.0.0",
       "@google-cloud/compute": "^4.0.0",
       "google-auth-library": "^9.0.0",
       "googleapis": "^143.0.0",
       "slack": "^11.0.1"
     },
     "devDependencies": {
       "@google-cloud/functions-framework": "^3.0.0",
       "c8": "^10.0.0",
       "gaxios": "^6.0.0",
       "mocha": "^10.0.0",
       "promise-retry": "^2.0.0",
       "proxyquire": "^2.1.0",
       "sinon": "^18.0.0",
       "wait-port": "^1.0.4"
     }
   }
   

   Python

   Copy the following to your requirements.txt file:

   slackclient==2.9.4
   google-api-python-client==2.131.0
   

3. Copy the following code into your Cloud Run function:

   Node.js

   const {CloudBillingClient} = require('@google-cloud/billing');
   const {InstancesClient} = require('@google-cloud/compute');
   
   const PROJECT_ID = process.env.GOOGLE_CLOUD_PROJECT;
   const PROJECT_NAME = `projects/${PROJECT_ID}`;
   const instancesClient = new InstancesClient();
   const ZONE = 'us-central1-a';
   
   exports.limitUse = async pubsubEvent => {
     const pubsubData = JSON.parse(
       Buffer.from(pubsubEvent.data, 'base64').toString()
     );
     if (pubsubData.costAmount <= pubsubData.budgetAmount) {
       return `No action necessary. (Current cost: ${pubsubData.costAmount})`;
     }
   
     const instanceNames = await _listRunningInstances(PROJECT_ID, ZONE);
     if (!instanceNames.length) {
       return 'No running instances were found.';
     }
   
     await _stopInstances(PROJECT_ID, ZONE, instanceNames);
     return `${instanceNames.length} instance(s) stopped successfully.`;
   };
   
   /**
    * @return {Promise} Array of names of running instances
    */
   const _listRunningInstances = async (projectId, zone) => {
     const [instances] = await instancesClient.list({
       project: projectId,
       zone: zone,
     });
     return instances
       .filter(item => item.status === 'RUNNING')
       .map(item => item.name);
   };
   
   /**
    * @param {Array} instanceNames Names of instance to stop
    * @return {Promise} Response from stopping instances
    */
   const _stopInstances = async (projectId, zone, instanceNames) => {
     await Promise.all(
       instanceNames.map(instanceName => {
         return instancesClient
           .stop({
             project: projectId,
             zone: zone,
             instance: instanceName,
           })
           .then(() => {
             console.log(`Instance stopped successfully: ${instanceName}`);
           });
       })
     );
   };

   Python

   import base64
   import json
   import os
   
   from googleapiclient import discovery
   
   PROJECT_ID = os.getenv("GCP_PROJECT")
   PROJECT_NAME = f"projects/{PROJECT_ID}"
   ZONE = "us-west1-b"
   
   
   def limit_use(data, context):
       pubsub_data = base64.b64decode(data["data"]).decode("utf-8")
       pubsub_json = json.loads(pubsub_data)
       cost_amount = pubsub_json["costAmount"]
       budget_amount = pubsub_json["budgetAmount"]
       if cost_amount <= budget_amount:
           print(f"No action necessary. (Current cost: {cost_amount})")
           return
   
       compute = discovery.build(
           "compute",
           "v1",
           cache_discovery=False,
       )
       instances = compute.instances()
   
       instance_names = __list_running_instances(PROJECT_ID, ZONE, instances)
       __stop_instances(PROJECT_ID, ZONE, instance_names, instances)
   
   
   def __list_running_instances(project_id, zone, instances):
       """
       @param {string} project_id ID of project that contains instances to stop
       @param {string} zone Zone that contains instances to stop
       @return {Promise} Array of names of running instances
       """
       res = instances.list(project=project_id, zone=zone).execute()
   
       if "items" not in res:
           return []
   
       items = res["items"]
       running_names = [i["name"] for i in items if i["status"] == "RUNNING"]
       return running_names
   
   
   def __stop_instances(project_id, zone, instance_names, instances):
       """
       @param {string} project_id ID of project that contains instances to stop
       @param {string} zone Zone that contains instances to stop
       @param {Array} instance_names Names of instance to stop
       @return {Promise} Response from stopping instances
       """
       if not len(instance_names):
           print("No running instances were found.")
           return
   
       for name in instance_names:
           instances.stop(project=project_id, zone=zone, instance=name).execute()
           print(f"Instance stopped successfully: {name}")
   
   

4. Set the Entry point to the correct function to execute:

   Node.js

   Set the Entry point to limitUse.

   Python

   Set the Entry point to limit_use.

5. Review the list of environment variables set automatically and determine if you need to manually set the GCP_PROJECT variable to the project running the virtual machines.

6. Set the ZONE parameter. This parameter is the zone where instances are stopped when the budget is exceeded.

   Note: This code sample only stops instances in a single zone in the specified project. Depending on your needs, you can change the code inside your Cloud Run function to handle other projects and zones.

7. Click DEPLOY.

Configure service account permissions

Your Cloud Run function runs as an automatically created service account. To control usage, you need to grant the service account permissions to any services on the project that it needs to modify by completing the following steps:

1. Identify the correct service account by viewing the details of your Cloud Run function. The service account is listed at the bottom of the page.

2. Go to the IAM page in the Google Cloud console to set the appropriate permissions.

   Go to the IAM page

Note: We recommend that you grant the lowest required permissions for the services that the function manages. The Compute Instance Admin (v1) role will grant you the appropriate permissions. However, to be more secure, you might want to create a custom role that only grants the necessary permissions.

Test that instances are stopped

To ensure your function works as expected, follow the steps in Test a Cloud Run function.

If successful, your Compute Engine VMs in the Google Cloud console are stopped.

What's next

Review other programmatic notification examples to learn how to do the following:

• Listen to your notifications
• Disable billing usage with notifications
• Send notifications to Slack