Set up client-side metrics

This page describes how to install, set up, and use Bigtable client-side metrics.

Client-side metrics are available for users of the following client libraries:

• Bigtable client library for Java
• Bigtable HBase client for Java
• Bigtable client library for Go

For an overview, see the Client-side metrics overview. For a detailed list of the metrics, see Client-side metric descriptions.

Add or confirm service account permissions

Note: Metrics writer IAM permissions are included in the Bigtable predefined roles by default. This step is only required if you're using a custom role.

To add IAM permissions to the service account, complete the following:

1. In the Google Cloud console, go to the IAM page.

   Go to IAM

2. Select the project where you created the Bigtable cluster.

3. Search for the service account you want to add the permission to.

4. Click Edit principal.

5. Click Add another role.

6. Search for "monitoring metric writer" and select the result.

7. Click Save.

Enable the Cloud Monitoring API

1. In the Google Cloud console, go to the APIs & Services page.

   Go to APIs & Services

2. Select the project where you created the Bigtable cluster.

3. Click Enable APIs and Service.

4. Search for "Monitoring".

5. In the search results, click through to "Stackdriver Monitoring API".

6. If "API enabled" is displayed, then the API is already enabled. If not, then click Enable.

Set up your Bigtable client library

After you enable the Cloud Monitoring API, configure your Bigtable client library.

Go

Follow these steps if you use the Bigtable client library for Go, which calls the Bigtable APIs.

Install and configure the client library for Go

Update the Bigtable client library for Go to version 1.27.0 or later. Your go.mod should look similar to the following:

require (
  cloud.google.com/go/bigtable v1.27.2-0.20240725222120-ce31365acc54

Optional: Opt out of client-side metrics

Client-side metrics are enabled by default in the Go client library. To opt out, pass NoopMetricsProvider in the ClientConfig:

client, err := bigtable.NewClientWithConfig(ctx, project, instance,
bigtable.ClientConfig{MetricsProvider: bigtable.NoopMetricsProvider{},
})

HBase

Follow these steps if you use the Bigtable HBase client for Java, which calls the HBase API, to connect your application to Bigtable.

Install and configure the HBase client library for Java

1. Download and install the latest Bigtable HBase client for Java. To enable client-side metrics, you must use version 2.6.4 or later.

2. Create an empty Maven project.

3. Determine the artifact dependency that's right for your use case. The 2.x versions include an HBase async client that the 1.x versions do not have.

   • bigtable-hbase-1.x or bigtable-hbase-2.x: Use for standalone applications where you are in control of your dependencies.
   • bigtable-hbase-1.x-hadoop or bigtable-hbase-2.x-hadoop: Use in Hadoop environments.
   • bigtable-hbase-1.x-shaded or bigtable-hbase-2.x-shaded: Use in environments other than Hadoop that require earlier versions of components such as protobufs or Guava.

4. Update the pom.xml file:

   <dependencyManagement>
       <dependencies>
           <dependency>
                 <groupId>com.google.cloud.bigtable</groupId>
                 <artifactId>BIGTABLE_HBASE_ARTIFACT</artifactId>
                 <version>VERSION_NUMBER</version>
             </dependency>
       </dependencies>
   </dependencyManagement>
   

   Provide the following:

   • BIGTABLE_HBASE_ARTIFACT: the artifact dependency for your project, formatted like bigtable-hbase-1.x or bigtable-hbase-2.x-hadoop.
   • VERSION_NUMBER: the client library version that you are using, formatted like 2.6.4

Enable client-side metrics in your application

Note: Client-side metrics are enabled by default since version 2.14.1. The following steps are only required when you're using a version between 2.6.4 and 2.14.1.

Using bigtable-hbase-1.x or bigtable-hbase-2.x

When you create a connection using the HBase client library and either the bigtable-hbase-1.x or bigtable-hbase-2.x artifacts, your code to enable the built-in client-side metrics looks similar to the following:

  import com.google.cloud.bigtable.data.v2.BigtableDataSettings;

  Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
  Connection connection = new BigtableConnection(configuration);

  BigtableDataSettings.enableBuiltinMetrics();

Using -hadoop or -shaded artifacts

When you create a connection using the HBase client library and one of the -hadoop or -shaded artifacts, your code to enable the built-in client-side metrics looks similar to the following:

  import com.google.bigtable.repackaged.com.google.cloud.bigtable.data.v2.BigtableDataSettings;

  Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
  Connection connection = new BigtableConnection(configuration);

  BigtableDataSettings.enableBuiltinMetrics();

Optional: Opt out of client-side metrics

If you are using version 2.14.1 or later, client-side metrics are enabled by default. You can opt out.

Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
configuration.setBoolean(BigtableOptionsFactory.BIGTABLE_ENABLE_CLIENT_SIDE_METRICS, false);
Connection connection = new BigtableConnection(configuration);

Java

Follow these steps if you use the Bigtable client library for Java, which calls the Bigtable APIs.

Before you begin

Optional: If you use an OpenCensus Stats integration to get Bigtable metrics, revert the dependency and StackdriverStatsExporter changes in your application. To understand why, see When to upgrade.

Install and configure the client library for Java

1. Download and install the latest Bigtable client library for Java. You must have version 2.16.0 or higher to use client-side metrics.

2. Update the Java client library's pom.xml file to use the latest version. Your pom should look similar to the following:

   <dependencyManagement>
       <dependencies>
           <dependency>
                 <groupId>com.google.cloud</groupId>
                 <artifactId>google-cloud-bigtable-bom</artifactId>
                 <version>2.38.0</version>
                 <scope>import</scope>
                 <type>pom</type>
             </dependency>
       </dependencies>
   </dependencyManagement>
   
   <dependencies>
       <dependency>
             <groupId>com.google.cloud</groupId>
             <artifactId>google-cloud-bigtable</artifactId>
       </dependency>
   </dependencies>
   

Enable client-side metrics in your application

Note: Client-side metrics are enabled by default since version 2.38.0. This step is only required when you're using a version between 2.16.0 and 2.38.0.

When you create a client using the client library, enable the built-in metrics. Your code should look similar to the following:

  BigtableDataSettings settings = BigtableDataSettings.newBuilder()
              .setProjectId("our-project-2-12")
              .setInstanceId("our-instance-85")
              .setAppProfileId("our-appProfile-06")
              .build();

  BigtableDataSettings.enableBuiltinMetrics();

  try (BigtableDataClient client = BigtableDataClient.create(settings)) {
          // Application logic
  }

Set up a custom OpenTelemetry instance

If you are using version 2.38.0 or later, and you want to publish client-side metrics to Cloud Monitoring and custom sinks, you can set up a custom OpenTelemetry instance.

  SdkMeterProviderBuilder meterProviderBuilder = SdkMeterProvider.builder();

  // register client side metrics on your meter provider
  BuiltinMetricsView.registerBuiltinMetrics("my-project-id", meterProviderBuilder);

  // register other metric reader and views
  meterProviderBuilder.registerMetricReader().registerView();

  // create the OTEL instance
  OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
          .setMeterProvider(meterProviderBuilder.build())
          .build();

  // Override MetricsProvider in BigtableDataSettings
  BigtableDataSettings settings = BigtableDataSettings.newBuilder()
          .setProjectId("my-project-id")
          .setInstanceId("my-instance-id")
          .setMetricsProvider(CustomOpenTelemetryMetricsProvider.create(openTelemetry))
          .build();

  try (BigtableDataClient client = BigtableDataClient.create(settings)) {
        // Application logic
  }

Optional: Opt out of client-side metrics

If you are using version 2.38.0 and later, you can opt out of using client-side metrics.

BigtableDataSettings settings = BigtableDataSettings.newBuilder()
        .setProjectId("my-project")
        .setInstanceId("my-instance")
        .setMetricsProvider(NoopMetricsProvider.INSTANCE)
        .build();

View metrics in Metrics Explorer

1. In the Google Cloud console, go to the Metrics Explorer page.

   Go to Metrics Explorer

2. Select the project.

3. Click Select a metric.

4. Search for bigtable.googleapis.com/client.

5. Select a metric, a group by method, and status, and pick an Aggregator. To explore more options, see Select metrics when using Metrics Explorer.

After you enable client-side metrics, let your application run for at least a minute before you check for any published metrics.

View metrics in Bigtable monitoring

Charts for the following client-side metrics are displayed on the Bigtable monitoring page:

• Client-side read latency
• Client-side write latency
• Client-side read attempt latency
• Client-side write attempt latency

To view client-side metrics on the Monitoring page, follow these steps:

1. Open the list of Bigtable instances in the Google Cloud console.

   Open the instance list

2. Click the instance whose metrics you want to view.

3. In the left pane, click Monitoring. The Google Cloud console displays a series of charts for the instance.

4. Scroll through the pane to view the client-side metrics.

5. Optional: To view metrics at the table level, click Tables.

6. Optional: To view metrics for individual app profiles, click Application Profiles.

For more information, see Bigtable monitoring.

What's next

• Read an overview of client-side metrics.
• View complete descriptions of client-side metrics.