Monitor topics within Pub/Sub

You can use the Google Cloud console to access the monitoring dashboard for a single topic or all the topics available in your project.

Key Point: Learn how to access the monitoring dashboard for Pub/Sub topics, understand some key metrics, and learn how to create a custom monitoring experience.

Before you begin

Ensure that you've prepared the following:

• A Cloud Billing account

• A Pub/Sub project with billing enabled

• At least one topic in your Pub/Sub project

• Familiarity with Cloud Monitoring and Metrics Explorer

View the monitoring dashboard for all topics

To view the monitoring dashboard for all the topics in a project, perform the following steps:

1. In the Google Cloud console, go to the Pub/Sub Topics page.

   Go to Topics

2. Click the Metrics tab.

   The monitoring dashboard is displayed for all topics in the project.

The monitoring dashboard has two sections: Overview and Quota.

View the monitoring dashboard for a single topic

To view the monitoring dashboard for a single topic in a project, perform the following steps:

1. In the Google Cloud console, go to the Pub/Sub Topics page.

   Go to Topics

2. In the Topics page, click any topic.

   The topic details page is displayed.

3. Within the topic details page, click the Metrics tab.

   The monitoring dashboard is displayed for the single topic.

The monitoring dashboard has three sections: Overview, Subscriptions, and Retention. You can see the Retention section only if you enable message retention for the topic. For more information about message retention, see Properties of a topic.

Customize the monitoring dashboard

You can create a custom dashboard or add individual charts to an existing custom dashboard.

To create a custom dashboard, perform the following steps:

1. Follow the steps to go to the monitoring page for all topics or for a single topic.

2. Choose one of the following options:

   • To create a new dashboard containing all the charts, or to add all the charts to an existing custom dashboard, click Save as custom dashboard.

   • To add a single chart to a new or custom dashboard, in an individual chart, expand the More Actions option and click Add to custom dashboard.

You can view all your dashboards in the Monitoring console.

Modify a chart

To modify a chart, perform the following steps:

1. Follow the steps to go to the monitoring page for all topics or for a single topic.

2. In an individual chart, expand the More Actions option and click View in Metrics Explorer.

   The chart opens in Metrics Explorer.

3. You can use Monitoring Query Language queries to make changes to your chart and test the output.

4. Click Save chart to save the changes.

Create an alerting policy

Creating an alerting policy for a chart lets you set up an alert monitoring metric in the chart, configure the circumstances under which you want to be alerted, and configure how you want to be notified.

For more information about how to create alerts, see Create metric-based alert policies.

Some charts might not offer a Create alerting policy option. In this case, perform the following steps:

1. Save the chart to a custom dashboard.

2. In the custom dashboard, for the individual chart, expand the More Actions option.

3. Click Convert to alert chart.

Key charts to monitor topics

Although there are a number of charts available to monitor a single topic, the following charts in the Overview section are a good starting point:

• The Publish requests chart lets you check if there are sudden spikes in the topic for receiving publish requests.

• The Average number of messages per batch chart lets you check if the batching size is adequate for your requirements. A smaller batch size leads to reduced publish latency.

The following charts in the Quota section for all topics help you monitor quotas for all the topics in your project:

• The Administrator operations chart lets you monitor administrative operations regarding topics. Each administrator operation charges one unit against this quota.

For more information, see Pub/Sub quotas and limits.

Metrics to monitor import topics

You can use the following metrics to monitor import topics:

• Ingestion byte count: This metric displays the total volume of data (in bytes) ingested from an import source into an import topic. It provides a breakdown of byte counts for each import source and partition (shard).

• Ingestion message count: This metric displays the total number of import messages for each import source and partition.

• Ingestion data source state: This metric displays the state of the external source from which the import topic is ingesting data. The error codes and their workarounds are listed in the troubleshooting section of the import topics. For more information, see the following:

  • Troubleshooting Amazon Kinesis Data Streams import topics.
  • Troubleshooting Cloud Storage import topics.

• Ingestion failure count. Used only for Cloud Storage import topics. This metric displays the number of failures encountered during ingestion. This metric includes the Cloud Storage bucket name and the failure reason.

To check these metrics, follow the steps described in View the monitoring dashboard for a single topic.

What's next

• To monitor subscriptions, see Monitor subscriptions in Pub/Sub.

• To learn more about using MQL to build monitoring charts, see Using the Query Editor.