Cloud Monitoring

Cloud Monitoring provides visibility into the performance, uptime, and overall health of cloud-powered applications. Google Cloud's operations suite collects and ingests metrics, events, and metadata from Dataproc Metastore services to generate insights via dashboards and charts.

Access service metrics in Monitoring

Dataproc Metastore service resource metrics are automatically enabled on Dataproc Metastore services. Use Monitoring to view these metrics.

You can access Monitoring from the Google Cloud console or by using the Monitoring API.

Console

  1. In the Google Cloud console, go to the Cloud Monitoring > Metrics explorer page:

    Go to the Metrics explorer page

  2. From the "Find resource type and metric" drop-down list, select the resource Dataproc Metastore Service.

  3. Click again in the input box, and then select a metric from the drop-down list. Hovering over the metric name displays information about the metric.

You can select filters, group by metric labels, perform aggregations, and select chart viewing options.

REST

You can use the Monitoring timeSeries.list API to capture and list metrics defined by a filter expression. Use the Try this API template on the API page to send an API request and display the response.

Dataproc Metastore service metrics in Monitoring

Dataproc Metastore service metrics include service health and request count.

Metric Name Monitored Resource Status
metastore.googleapis.com/service/health metastore.googleapis.com/Service GA
metastore.googleapis.com/service/request_count metastore.googleapis.com/Service GA

Build a custom Monitoring dashboard

You can build a custom Monitoring dashboard that displays charts of selected Dataproc Metastore service metrics.

  1. Select + CREATE DASHBOARD from the Monitoring Dashboards Overview page.

  2. Provide a name for the dashboard.

  3. In the upper-right menu, click Add Chart to open the Add Chart window.

  4. Select Dataproc Metastore Service as the resource type.

  5. Select one or more metrics and metric and chart properties, and then click Save to save the chart.

  6. Add additional charts to your dashboard if you'd like.

  7. Click Save to save the dashboard. Its title appears in the Monitoring Dashboards Overview page.

You can view, update, and delete dashboard charts from the dashboard display page.

Use Monitoring alerts

You can create a Monitoring alert that notifies you when a Dataproc Metastore service or job metric crosses a specified threshold.

Create an alert

  1. In the Google Cloud console, open Monitoring Alerting.

  2. Click + CREATE POLICY to open the Create new alerting policy form.

  3. Define an alert by adding alert conditions, policy triggers, notification channels, and documentation.

  4. Select ADD CONDITION to open the alert condition form with the Metric tab selected.

  5. Fill in the fields to define an alert condition, and then click ADD.

  6. After adding the alert condition, complete the alert policy by setting notification channels, policy triggers, documentation, and the alert policy name.

View alerts

When an alert is triggered by a metric threshold condition, Monitoring creates an incident and a corresponding event. You can view incidents from the Monitoring Alerting page in the Google Cloud console. If you defined a notification mechanism in the alert policy, such as an email or SMS notification, Monitoring also sends a notification of the incident.

Whats next