Using Cloud Bigtable with Cloud Functions

Objectives

Write, deploy, and trigger an HTTP Cloud Function that accesses Cloud Bigtable.

Costs

This topic uses Cloud Bigtable and Cloud Functions, which are billable components of Google Cloud.

Before you begin

  1. This topic assumes you have a Cloud Bigtable instance named test-instance and a table named test-table. You can create these resources by following the steps on Creating a test table. Be sure to delete the resources when you are finished to avoid incurring unnecessary costs.

  2. Enable the Cloud Functions API.

    Enable the API

  3. Install and initialize the Cloud SDK.

  4. Update gcloud components:

    gcloud components update
    
  5. Prepare your development environment:

Prepare the application

  1. Clone the sample app repository to your local machine:

    Node.js

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Python

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Go

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

  2. Change to the directory that contains the Cloud Functions sample code for accessing Cloud Bigtable:

    Node.js

    cd nodejs-docs-samples/functions/bigtable/

    Python

    cd python-docs-samples/functions/bigtable/

    Go

    cd golang-samples/functions/bigtable/
  3. Take a look at the sample code:

    Node.js

    // Imports the Google Cloud client library
    const {Bigtable} = require('@google-cloud/bigtable');
    
    // Instantiates a client
    const bigtable = new Bigtable();
    
    exports.readRows = async (req, res) => {
      // Gets a reference to a Cloud Bigtable instance and database
      const instance = bigtable.instance(req.body.instanceId);
      const table = instance.table(req.body.tableId);
    
      // Execute the query
      try {
        const prefix = 'phone#';
        const rows = [];
        await table
          .createReadStream({
            prefix,
          })
          .on('error', err => {
            res.send(`Error querying Bigtable: ${err}`);
            res.status(500).end();
          })
          .on('data', row => {
            rows.push(
              `rowkey: ${row.id}, ` +
                `os_build: ${row.data['stats_summary']['os_build'][0].value}\n`
            );
          })
          .on('end', () => {
            rows.forEach(r => res.write(r));
            res.status(200).end();
          });
      } catch (err) {
        res.send(`Error querying Bigtable: ${err}`);
        res.status(500).end();
      }
    };
    

    Python

    from google.cloud import bigtable
    from google.cloud.bigtable.row_set import RowSet
    
    client = bigtable.Client()
    
    
    def bigtable_read_data(request):
        instance = client.instance(request.headers.get("instance_id"))
        table = instance.table(request.headers.get("table_id"))
    
        prefix = 'phone#'
        end_key = prefix[:-1] + chr(ord(prefix[-1]) + 1)
    
        outputs = []
        row_set = RowSet()
        row_set.add_row_range_from_keys(prefix.encode("utf-8"),
                                        end_key.encode("utf-8"))
    
        rows = table.read_rows(row_set=row_set)
        for row in rows:
            output = 'Rowkey: {}, os_build: {}'.format(
                row.row_key.decode('utf-8'),
                row.cells["stats_summary"]["os_build".encode('utf-8')][0]
                .value.decode('utf-8'))
            outputs.append(output)
    
        return '\n'.join(outputs)
    

    Go

    
    // Package bigtable contains an example of using Bigtable from a Cloud Function.
    package bigtable
    
    import (
    	"context"
    	"fmt"
    	"log"
    	"net/http"
    	"sync"
    
    	"cloud.google.com/go/bigtable"
    )
    
    // client is a global Bigtable client, to avoid initializing a new client for
    // every request.
    var client *bigtable.Client
    var clientOnce sync.Once
    
    // BigtableRead is an example of reading Bigtable from a Cloud Function.
    func BigtableRead(w http.ResponseWriter, r *http.Request) {
    	clientOnce.Do(func() {
    		// Declare a separate err variable to avoid shadowing client.
    		var err error
    		client, err = bigtable.NewClient(context.Background(), r.Header.Get("projectID"), r.Header.Get("instanceId"))
    		if err != nil {
    			http.Error(w, "Error initializing client", http.StatusInternalServerError)
    			log.Printf("bigtable.NewClient: %v", err)
    			return
    		}
    	})
    
    	tbl := client.Open(r.Header.Get("tableID"))
    	err := tbl.ReadRows(r.Context(), bigtable.PrefixRange("phone#"),
    		func(row bigtable.Row) bool {
    			osBuild := ""
    			for _, col := range row["stats_summary"] {
    				if col.Column == "stats_summary:os_build" {
    					osBuild = string(col.Value)
    				}
    			}
    
    			fmt.Fprintf(w, "Rowkey: %s, os_build:  %s\n", row.Key(), osBuild)
    			return true
    		})
    
    	if err != nil {
    		http.Error(w, "Error reading rows", http.StatusInternalServerError)
    		log.Printf("tbl.ReadRows(): %v", err)
    	}
    }
    

    The function sends a read request to the table to fetch all stats_summary data for rows with a row key prefix of phone. The function is executed when you make an HTTP request to the function's endpoint.

Deploy the function

To deploy the function with an HTTP trigger, run the following command in the bigtable directory:

Node.js

gcloud functions deploy get \
--runtime nodejs10 --trigger-http
You can use the following values for the --runtime flag to specify your preferred Node.js version:
  • nodejs10
  • nodejs12

Python

gcloud functions deploy bigtable_read_data \
--runtime python37 --trigger-http
You can use the following values for the --runtime flag to specify your preferred Python version:
  • python37
  • python38

Go

gcloud functions deploy HelloBigtable \
--runtime go111 --trigger-http
You can use the following values for the --runtime flag to specify your preferred Go version:
  • go111
  • go113

Function deployment may take up to two minutes.

When your function finishes deploying, it returns the url value. You will use that value when you trigger the function.

You can view your deployed functions on the Cloud Functions page in the Google Cloud Console. You can also create and edit functions on that page, and get details and diagnostics for your functions.

Trigger the function

Make an HTTP request to your function:

Node.js

curl "https://REGION-PROJECT_ID.cloudfunctions.net/get" --trigger-http

Python

curl "https://REGION-PROJECT_ID.cloudfunctions.net/bigtable_read_data" --trigger-http

Go

curl "https://REGION-PROJECT_ID.cloudfunctions.net/HelloBigtable" --trigger-http

where REGION and PROJECT_ID match the values that are visible in your terminal when your function finishes deploying. You should see output that shows the results of the read request.

You can also visit the function's URL in your browser to see the result of your read request.

Clean up

To avoid incurring additional charges to your Google Cloud account for the Cloud Bigtable and Cloud Functions resources used in this topic:

  1. Delete the instance:

    gcloud bigtable instances delete test-instance
    
  2. Delete the function that you deployed:

    Node.js

    gcloud functions delete get --trigger-http

    Python

    gcloud functions delete bigtable_read_data --trigger-http

    Go

    gcloud functions delete HelloBigtable --trigger-http

What's next