Set up client-side metrics

This page describes how to install, set up, and use Bigtable client side metrics.

To enable and view client-side metrics, you must use either the Cloud Bigtable client library for Java, which calls the Cloud Bigtable service APIs, or the Cloud Bigtable HBase client for Java, which uses the open-source HBase API.

For an overview, see the Client-side metrics overview. For a detailed list of the metrics, see Client-side metric descriptions.

Note: Client-side metrics are not supported for applications that integrate the Dataflow or Apache Beam client libraries with the Cloud Bigtable HBase client for Java.

Before you begin

Optional: If you currently use an OpenCensus Stats integration to get Bigtable metrics, revert the dependency and StackdriverStatsExporter changes in your application. To understand why, see When to upgrade.

Add or confirm service account permissions

Note: Metrics writer IAM permissions are included in the Bigtable predefined roles by default. This step is only required if you're using a custom role.

To add IAM permissions to the service account, complete the following:

1. In the Google Cloud console, go to the IAM page.

   Go to IAM

2. Select the project where you created the Bigtable cluster.

3. Search for the service account you want to add the permission to.

4. Click Edit principal.

5. Click Add another role.

6. Search for "monitoring metric writer" and select the result.

7. Click Save.

Enable the Cloud Monitoring API

1. In the Google Cloud console, go to the APIs & Services page.

   Go to APIs & Services

2. Select the project where you created the Bigtable cluster.

3. Click Enable APIs and Service.

4. Search for "Monitoring".

5. In the search results, click through to "Stackdriver Monitoring API".

6. If "API enabled" is displayed, then the API is already enabled. If not, then click Enable.

Set up and use the Cloud Bigtable client library for Java

Follow these steps if you use the Cloud Bigtable client library for Java, which calls the Cloud Bigtable APIs.

Install and configure the client library for Java

1. Download and install the latest Cloud Bigtable client library for Java. You must have version 2.16.0 or higher to use client-side metrics.

2. Update the Java client library's pom.xml file to use the latest version. Your pom should look similar to the following:

   <dependencyManagement>
       <dependencies>
           <dependency>
                 <groupId>com.google.cloud</groupId>
                 <artifactId>google-cloud-bigtable-bom</artifactId>
                 <version>2.38.0</version>
                 <scope>import</scope>
                 <type>pom</type>
             </dependency>
       </dependencies>
   </dependencyManagement>
   
   <dependencies>
       <dependency>
             <groupId>com.google.cloud</groupId>
             <artifactId>google-cloud-bigtable</artifactId>
       </dependency>
   </dependencies>
   

Enable client-side metrics in your application

Note: Client-side metrics are enabled by default since version 2.38.0. This step is only required when you're using a version between 2.16.0 and 2.38.0.

When you create a client using the client library, enable the built-in metrics. Your code should look similar to the following:

  BigtableDataSettings settings = BigtableDataSettings.newBuilder()
              .setProjectId("our-project-2-12")
              .setInstanceId("our-instance-85")
              .setAppProfileId("our-appProfile-06")
              .build();

  BigtableDataSettings.enableBuiltinMetrics();

  try (BigtableDataClient client = BigtableDataClient.create(settings)) {
          // Application logic
  }

Set up a custom OpenTelemetry instance

If you are using version 2.38.0 or later, and you want to publish client-side metrics to Cloud Monitoring and custom sinks, you can set up a custom OpenTelemetry instance.

  SdkMeterProviderBuilder meterProviderBuilder = SdkMeterProvider.builder();

  // register client side metrics on your meter provider
  BuiltinMetricsView.registerBuiltinMetrics("my-project-id", meterProviderBuilder);

  // register other metric reader and views
  meterProviderBuilder.registerMetricReader().registerView();

  // create the OTEL instance
  OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
          .setMeterProvider(meterProviderBuilder.build())
          .build();

  // Override MetricsProvider in BigtableDataSettings
  BigtableDataSettings settings = BigtableDataSettings.newBuilder()
          .setProjectId("my-project-id")
          .setInstanceId("my-instance-id")
          .setMetricsProvider(CustomOpenTelemetryMetricsProvider.create(openTelemetry))
          .build();

  try (BigtableDataClient client = BigtableDataClient.create(settings)) {
        // Application logic
  }

Optional: Opt out of client-side metrics

If you are using version 2.38.0 and later, you can opt out of using client-side metrics.

BigtableDataSettings settings = BigtableDataSettings.newBuilder()
        .setProjectId("my-project")
        .setInstanceId("my-instance")
        .setMetricsProvider(NoopMetricsProvider.INSTANCE)
        .build();

Set up and use the Cloud Bigtable HBase client for Java

Follow these steps if you use the Cloud Bigtable HBase client for Java, which calls the HBase API, to connect your application to Bigtable.

Install and configure the HBase client library for Java

1. Download and install the latest Cloud Bigtable HBase client for Java. To enable client-side metrics, you must use version 2.6.4 or later.

2. Create an empty Maven project.

3. Determine the artifact dependency that's right for your use case. The 2.x versions include an HBase async client that the 1.x versions do not have.

   • bigtable-hbase-1.x or bigtable-hbase-2.x: Use for standalone applications where you are in control of your dependencies.
   • bigtable-hbase-1.x-hadoop or bigtable-hbase-2.x-hadoop: Use in Hadoop environments.
   • bigtable-hbase-1.x-shaded or bigtable-hbase-2.x-shaded: Use in environments other than Hadoop that require older versions of components such as protobufs or Guava.

4. Update the pom.xml file:

   <dependencyManagement>
       <dependencies>
           <dependency>
                 <groupId>com.google.cloud.bigtable</groupId>
                 <artifactId>BIGTABLE_HBASE_ARTIFACT</artifactId>
                 <version>VERSION_NUMBER</version>
             </dependency>
       </dependencies>
   </dependencyManagement>
   

   Provide the following:

   • BIGTABLE_HBASE_ARTIFACT: the artifact dependency for your project, formatted like bigtable-hbase-1.x or bigtable-hbase-2.x-hadoop.
   • VERSION_NUMBER: the client library version that you are using, formatted like 2.6.4

Enable client-side metrics in your application

Note: Client-side metrics are enabled by default since version 2.14.1. The following steps are only required when you're using a version between 2.6.4 and 2.14.1.

Using bigtable-hbase-1.x or bigtable-hbase-2.x

When you create a connection using the HBase client library and either the bigtable-hbase-1.x or bigtable-hbase-2.x artifacts, your code to enable the built-in client-side metrics looks similar to the following:

  import com.google.cloud.bigtable.data.v2.BigtableDataSettings;

  Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
  Connection connection = new BigtableConnection(configuration);

  BigtableDataSettings.enableBuiltinMetrics();

Using -hadoop or -shaded artifacts

When you create a connection using the HBase client library and one of the -hadoop or -shaded artifacts, your code to enable the built-in client-side metrics looks similar to the following:

  import com.google.bigtable.repackaged.com.google.cloud.bigtable.data.v2.BigtableDataSettings;

  Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
  Connection connection = new BigtableConnection(configuration);

  BigtableDataSettings.enableBuiltinMetrics();

Optional: Opt out of client-side metrics

If you are using version 2.14.1 or later, client-side metrics are enabled by default. You can opt out.

Configuration configuration = BigtableConfiguration.configure("my-project-id", "my-instance-id");
configuration.setBoolean(BigtableOptionsFactory.BIGTABLE_ENABLE_CLIENT_SIDE_METRICS, false);
Connection connection = new BigtableConnection(configuration);

View metrics in Metrics Explorer

1. In the Google Cloud console, go to the Metrics Explorer page.

   Go to Metrics Explorer

2. Select the project.

3. Click Select a metric.

4. Search for bigtable.googleapis.com/client.

5. Select a metric, a group by method, and status, and pick an Aggregator. To explore more options, see Select metrics when using Metrics Explorer.

After you enable client-side metrics, let your application run for at least a minute before you check for any published metrics.

What's next

• Read an overview of client-side metrics.
• View complete descriptions of client-side metrics.
• Learn more about Bigtable monitoring.