Capturing Cloud Bigtable tracing and metrics using Stackdriver and OpenCensus

This tutorial shows how to implement client-side tracing and metrics recording in your Cloud Bigtable workloads by using OpenCensus and Stackdriver. Although Cloud Bigtable surfaces several helpful server-side metrics using Stackdriver, apps can realize added benefits by implementing client-side tracing, instrumenting, and app-defined metrics. For example, server-side metrics don't give you a window into the round-trip latency of calls made to your Cloud Bigtable endpoint and can only be surfaced using client-side tracing.

OpenCensus is an open source library that you can use to provide observability in your apps. The library is vendor agnostic and integrates with several backends, such as Prometheus and Zipkin. In this tutorial, you use Stackdriver as the backend for tracing and metrics.

To complete the steps in this tutorial, you should be familiar with the Linux command line. Though it's not required, knowledge of the Java programming language helps you understand the example code.

Objectives

  • Deploy a Cloud Bigtable instance.
  • Deploy a Compute Engine virtual machine (VM) for running a sample OpenCensus-instrumented Java client.
  • Download, deploy, and run the instrumented Java client app.
  • View OpenCensus traces in Stackdriver Trace.
  • View OpenCensus metrics in Stackdriver Metrics Explorer.

Costs

Neste tutorial, usamos os seguintes componentes faturáveis do Google Cloud Platform:

Use a calculadora de preços para gerar uma estimativa de custo com base no uso do projeto. Novos usuários do GCP são qualificados para uma avaliação gratuita.

Before you begin

  1. Faça login na sua Conta do Google.

    Se você ainda não tiver uma, inscreva-se.

  2. Selecione ou crie um projeto do Google Cloud Platform.

    Acessar a página Gerenciar recursos

  3. Verifique se o faturamento foi ativado no projeto do Google Cloud Platform.

    Saiba como ativar o faturamento

  4. Ativar Compute Engine, Cloud Bigtable, and Stackdriver Logging APIs.

    Ativar as APIs

  5. Instale e inicialize o SDK do Cloud.

Reference architecture

For simplicity, in this tutorial you implement all of your client-side logic in a Java console app. For the datastore tier, you use Cloud Bigtable, which allows you to focus on the key aspects of client-side tracing and metrics without having to worry about things like database deployments and related configuration.

The following architecture diagram shows a Java console app and a datastore tier.

Reference architecture showing a Java console app and a datastore tier.

Creating a Cloud Bigtable instance

In this section, you create a Cloud Bigtable instance that your Java app uses later in the tutorial.

  • In Cloud Shell, create a Cloud Bigtable development instance:

    gcloud bigtable instances create cbt-oc \
      --cluster=cbt-oc \
      --cluster-zone=us-central1-c \
      --display-name=cbt-oc \
      --instance-type=DEVELOPMENT
    

This command might take a few minutes to complete.

Creating and configuring a Compute Engine VM

  • In Cloud Shell, create a Compute Engine VM with the security scopes necessary for 0Auth 2.0:

    gcloud compute instances create trace-client \
        --zone=us-central1-c \
        --scopes="https://www.googleapis.com/auth/bigtable.admin.table,\
    https://www.googleapis.com/auth/bigtable.data,\
    https://www.googleapis.com/auth/logging.write,\
    https://www.googleapis.com/auth/monitoring.write,\
    https://www.googleapis.com/auth/trace.append"
    

Sample Java app

This section uses a sample Java app that generates transactions in order to demonstrate the tracing capabilities of OpenCensus and Stackdriver.

App flow

The sample Java app running on the Compute Engine VM does the following:

  1. Creates a table in the Cloud Bigtable instance.
  2. For a series of 10,000 transactions sets, does the following:
    1. Writes a small set of rows.
    2. Reads a single row.
    3. Performs a table scan for those rows.
  3. Deletes the table.

Deploying the sample app

In this section, you download the Java app containing the instrumented code, modify it to reflect your environment, and then run it.

  1. In the GCP Console, go to the VM instances page:

    GO TO THE VM INSTANCES PAGE

  2. Use SSH to connect to the VM by clicking the SSH button (highlighted in the following screenshot):

    Using SSH to connect to the VM.

  3. In the VM instance, install Git, the Java 8 JDK, and Maven:

    sudo apt-get install git openjdk-8-jdk maven -y
    
  4. In the instance, clone the source repository for this tutorial:

    git clone https://github.com/GoogleCloudPlatform/community.git
    

    You can now update the Java app with some configuration settings specific to your project.

  5. Navigate to the folder containing the Java source:

    cd community/tutorials/bigtable-oc/java/
    
  6. Configure the app code to use the cbt-oc Cloud Bigtable instance:

    export INSTANCE_ID=cbt-oc
    
  7. Now run the Maven commands to build and run the program:

    mvn package -DskipTests --quiet
    mvn exec:java -Dexec.mainClass=com.example.bigtable.App --quiet
    

    The output looks similar to the following:

    ...
    2019-05-13 23:31:54 INFO  BigtableSession:89 - Opening connection for projectId your-project, instanceId cbt-oc, on data host bigtable.googleapis.com, admin host bigtableadmin.googleapis.com.
    2019-05-13 23:31:54 INFO  BigtableSession:89 - Bigtable options: {......}
    2019-05-13 23:31:54 INFO  OAuthCredentialsCache:89 - Refreshing the OAuth token
    2019-05-13 23:31:55 INFO  App:170 - Create table Hello-Bigtable
    2019-05-13 23:35:36 INFO  App:209 - Delete the table
    2019-05-13 23:35:36 WARN  BigtableAdmin:116 - Table Hello-Bigtable was disabled in memory only.
    

Sample app code highlights

In the following code segment, the Cloud Bigtable instance is provided to the Java runtime using the environment variable INSTANCE_ID that you set previously.

private static final String PROJECT_ID = ServiceOptions.getDefaultProjectId();
private static final String INSTANCE_ID = System.getenv( "INSTANCE_ID");

The code segment below shows how to define a manually labeled tracing scope:

try (Scope ss = tracer.spanBuilder("opencensus.Bigtable.Tutorial").startScopedSpan()) {

    // generate unique UUID
    UUID uuid = UUID.randomUUID();
    String randomUUIDString = uuid.toString();

    startRead = System.currentTimeMillis();
    // write to Bigtable
    writeRows(table, randomUUIDString);
    endRead = System.currentTimeMillis();


    startWrite = System.currentTimeMillis();
    // read from Bigtable
    readRows(table, randomUUIDString);
    endWrite = System.currentTimeMillis();

Notice the try block with the call to spanBuilder. This illustrates how the program uses OpenCensus to perform tracing. The call chain that performs the table writes and reads in the doBigTableOperations function is instrumented in this way.

The program also configures Stackdriver Trace as the tracing backend:

private static void configureOpenCensusExporters(Sampler sampler) throws IOException {
    TraceConfig traceConfig = Tracing.getTraceConfig();

    // For demo purposes, lets always sample.

    traceConfig.updateActiveTraceParams(
      traceConfig.getActiveTraceParams().toBuilder().setSampler(sampler).build());

    // Create the Stackdriver trace exporter
    StackdriverTraceExporter.createAndRegister(
      StackdriverTraceConfiguration.builder()
        .setProjectId(PROJECT_ID)
        .build());

    // [Start Stackdriver Monitoring]
    StackdriverStatsExporter.createAndRegister();

Viewing traces in Stackdriver Trace UI

The sample program performs 10,000 transaction sets: three writes and a range read. The exporter is configured to record one trace sample for every 1,000 transaction sets. As a result, 10 or 11 traces are captured over the run of the program.

After the program has been running for a short time, do the following:

  1. Navigate to the Stackdriver Trace console under Stackdriver:

    Stackdriver Trace console.

  2. Click Trace List.

    On the right, you should see a table similar to the following:

    Trace list table.

    The sampling rate for the traces is set to record one trace for every 1,000 transactions.

    The tracing label opencensus.Bigtable.Tutorial in the Timeline is the name of the outermost tracing scope that is defined in the following code snippet.

    // sample every 1000 transactions
    configureOpenCensusExporters(Samplers.probabilitySampler(1/1000.0));
  3. Select opencensus.Bigtable.Tutorial. This opens a drill-down view that shows more information about the call chain, along with other useful information such as tracing scopes for discrete API calls instrumented in the client library and operation-level call latencies.

    For instance, each of the series of write and read rows are encapsulated by the lower-level, user-defined WriteRows and ReadRows tracing spans.

    Below ReadRows, you can see the get operation, followed by the table scan operations.

    Get operation and table scan operations.

    The other items included in the trace list, such as Operation.google.bigtable.admin.v2.BigtableTableAdmin.CreateTable, occurred outside of the manually defined tracing scope. As a result, these items are included as separate operations in the list.

Viewing metrics in Stackdriver Metrics Explorer UI

The app code shows how to measure and record latency and transaction counts.

For metrics, there is no sampling. Every recorded value is included in the metric representations. Each metric is defined by the type of measurement to be performed. In this example, write latency is recorded with microsecond units:

// The write latency in milliseconds
private static final MeasureDouble M_WRITE_LATENCY_MS = MeasureDouble.create("btapp/write_latency", "The latency in milliseconds for write", "ms");

The distribution is aggregated and stored using these buckets: 0–5 ms, 5–10ms, 10–25 ms, and so on.

Aggregation latencyDistribution = Distribution.create(BucketBoundaries.create(
        Arrays.asList(
            0.0, 5.0, 10.0, 25.0, 100.0, 200.0, 400.0, 800.0, 10000.0)));
View.create(Name.create("btappmetrics/write_latency"),
            "The distribution of the write latencies",
            M_WRITE_LATENCY_MS,
            latencyDistribution,
            Collections.singletonList(KEY_LATENCY)),

All three metrics—write latency, read latency, and transaction counts—are recorded using a single call to the recorder:

// record read, write latency metrics and count
STATS_RECORDER.newMeasureMap()
              .put(M_READ_LATENCY_MS, endRead - startRead)
              .put(M_WRITE_LATENCY_MS, endWrite - startWrite)
              .put(M_TRANSACTION_SETS, 1)
              .record();

You can review the captured metrics:

  1. In the GCP Console, go to the Metrics Explorer page:

    Go to Metrics Explorer

  2. In the left navigation pane, click Dashboards/Create Dashboard.

  3. In the dashboard title, replace Untitled Dashboard with Cloud Bigtable Metrics.

  4. In the left navigation pane, click Resources/Metrics Explorer.

  5. In Metrics Explorer, search for the metrics by using Find resource type and metric. Enter opencensus/btappmetrics/write_latency, and then click Enter.

  6. From the list at the top, select Heatmap.

    The distribution Heatmap graph is shown in the right pane.

  7. Click Save Chart.

  8. Under Select Dashboard, select Cloud Bigtable Metrics.

  9. Click Save.

  10. In the left pane, next to the Metric label, click next to the selected metric.

  11. Repeat the steps for the metrics: opencensus/btappmetrics/read_latency and opencensus/btappmetrics/transaction_set_count.

  12. In the left navigation pane, select Dashboards/Cloud Bigtable Metrics.

    The three metrics charts are displayed.

  13. Zoom in by selecting a time range, clicking in one of the charts, and dragging the cursor to the right edge of the graph.

    For the heatmaps, you can see more details about the distribution by holding the cursor over the various colored blocks.

    Distribution details for heatmaps.

    Metrics Explorer shows a heatmap of the write latency. As the image shows, 2108 metrics samples fall into the 5–10 ms bucket.

Cleaning up

  1. No Console do GCP, acesse a página "Projetos".

    Acessar a página Projetos

  2. Na lista de projetos, selecione um e clique em Excluir projeto.
  3. Na caixa de diálogo, digite o código do projeto e clique em Encerrar para excluí-lo.

What's next

Esta página foi útil? Conte sua opinião sobre:

Enviar comentários sobre…