Setting up automated chat messages with your Cloud Billing summary

In this tutorial, you export Cloud Billing data to BigQuery and create a Cloud Function and a Cloud Scheduler job to send daily chat messages summarizing your Cloud Billing data. By exporting all charges to BigQuery, you can query the data to aggregate your costs. This configuration provides insight into the daily charges of your Google Cloud usage without you needing to look up the charges every day.

This document is intended for developers and billing administrators who want more insight into their Google Cloud spending. It assumes that you're familiar with creating a Cloud Billing account and that you are already using Hangouts Chat.

The following diagram illustrates the Google Cloud components used in this tutorial.

Architectural diagram showing how data flows from Cloud Billing to Hangouts Chat.

The data flows from Cloud Billing to BigQuery, between Cloud Functions and BigQuery, to the Hangouts Chat webhook. Cloud Scheduler triggers the Cloud Function.

Objectives

  • Enable Cloud Billing data export to BigQuery.
  • Create a Cloud Function and send a chat message with the query results.
  • Test the service.
  • Create a Cloud Scheduler job to invoke the Cloud Function daily.

Costs

This tutorial uses the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Cloud Console, on the project selector page, select or create a Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

  4. In the Cloud Console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Cloud Console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

Enabling Cloud Billing data export to BigQuery

To export your Cloud Billing data to BigQuery, you create a target dataset table in BigQuery and then configure Cloud Billing to export billing data to that dataset.

Create a dataset in BigQuery

In this tutorial, you create a dataset in the same Cloud project as the app.

If you already have a dataset in a different Cloud project, you can share the dataset with your service account and skip to the next section.

  1. In the Cloud Console, go to the BigQuery page.

    Go to the BigQuery page

  2. For Resources, click your Cloud project name.

  3. To create a target table in BigQuery to store your Cloud Billing data, click Create dataset.

  4. In the Dataset ID field, enter billing_data.

  5. Click Create dataset.

Export Cloud Billing data to BigQuery

  1. In Cloud Console, go to the Billing page.

    Go to the Billing page

  2. Select your Cloud Billing account.

  3. Click Billing export.

  4. Click Edit settings.

  5. Select the Cloud project that you created for this tutorial.

  6. Select the billing_data dataset that you previously created.

  7. Click Save.

Your Cloud Billing account is configured to export data to BigQuery. The export process can take up to 24 hours.

Get the name of the dataset table

  1. In the Cloud Console, go to the BigQuery page.

    Go to the BigQuery page

  2. For Resources, click your Cloud project name.

  3. To expand, click billing_data.

  4. Click the name of the table that the export process created. The name starts with gcp_billing_export_v1.

  5. Click Query Table.

  6. Save the dataset table name, which you use to customize the index.php file later in this tutorial.

The SQL statement contains the dataset table name after the FROM keyword in the format of project_name.dataset.table_name.

  • project_name represents the name of your Cloud project.
  • dataset represents the name of your dataset.
  • table_name represents the name of your table.

The following is an example dataset table name: myproject.billing_data.gcp_billing_export_v1_000000_000000_000000

Creating a Hangouts Chat room

  1. Go to your Hangouts Chat application.
  2. Click the Find people, rooms, bots search box, and then click Create Room.
  3. In the Room name field, enter GCP_Billing.
  4. Click Create.
  5. Click the GCP_Billing room menu, and then select Configure webhooks.

    Drop-down menu to configure webhooks for the `GCP_Billing` room.

  6. In the Name field, enter gcp_billing_webhook.

  7. Click Save. Make a note of the URL of the webhook because you need it later in the tutorial.

Creating a Cloud Function

  1. In the Cloud Console, go to Cloud Functions.

    Go to Cloud Functions

  2. Click Create Function.

  3. In the Name field, enter getBillingAlert.

  4. In the Memory allocated drop-down menu, select 128 MiB.

  5. In the Runtime drop-down menu, select Python 3.7.

  6. In the main.py tab, paste the following code:

    from google.cloud import bigquery
    from httplib2 import Http
    from json import dumps
    from datetime import datetime
    
    def run_rule(request):
        client = bigquery.Client()
        query_job = client.query("""
                SELECT sum(cost) as total_cost FROM
    `project_name.dataset.tablename`""")
    
        for row in query_job:
            tcost = row["total_cost"]
    
        url = 'hangout_room_url'
        now = datetime.now()
        msg_content = "GCP Cost: {}".format(tcost) + " [date]: " + now.strftime("%m/%d/%Y, %H:%M:%S")
        bot_message = {
            'text' : msg_content}
    
        message_headers = { 'Content-Type': 'application/json; charset=UTF-8'}
    
        http_obj = Http()
    
        response = http_obj.request(
            uri=url,
            method='POST',
            headers=message_headers,
            body=dumps(bot_message),
        )
    

    Replace the following:

    • project_name: the name of your project.
    • dataset: the name of your dataset
    • tablename: the name of your table
    • hangout_room_url: the webhook URL that you previously copied
  7. In the requirements.txt tab, paste the following code:

    # Function dependencies, for example:
    # package>=version
    google-cloud-pubsub==0.34.0
    google-cloud-storage==1.13.1
    google-cloud-bigquery==1.8.1
    google-cloud-core==0.29.1
    pytz==2018.7
    
  8. In the Function to execute field, enter run_rule.

  9. Click Create.

  10. Click the getBillingAlert function.

  11. Click the Trigger tab.

  12. Copy the Cloud Function URL. You use this value later to schedule the Cloud Function to run.

Testing the Cloud Function

  1. In the Cloud Console, go to Cloud Functions.

    Go to Cloud Functions

  2. For the getBillingAlert Cloud Function, click Actions.

  3. Click Test function.

  4. Click Test the function.

    A message with the Cloud Billing data is sent to your Hangouts Chat room.

Setting up Cloud Scheduler

After you verify that you can run the function, you can use Cloud Scheduler to run the function on scheduled intervals.

In this tutorial, you run the function daily at 10 AM. The Cloud Scheduler uses cron format for the schedule.

  1. In the Cloud Console, go to Cloud Scheduler.

    Go to Cloud Scheduler

  2. Click Create Job.

  3. In the Name field, enter Billing-BigQuery.

  4. In the Description field, enter Send Hangout message with Billing summary.

  5. In the Frequency field, enter 0 10 * * *.

  6. Select your timezone.

  7. For Target, select HTTP.

  8. In the URL field, enter the functions URL endpoint.

  9. Leave the Service field blank.

  10. Click Create.

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Delete the project

The easiest way to eliminate billing is to delete the Cloud project you created for the tutorial. Alternatively, you can delete the individual resources.

  1. In the Cloud Console, go to the Manage resources page.

    Go to the Manage resources page

  2. In the project list, select the project that you want to delete and then click Delete .
  3. In the dialog, type the project ID and then click Shut down to delete the project.

What's next