Logs-based Metrics

This guide explains how to create metrics that count the number of log entries that match given filters. You can use these metrics to create charts and alerting policies in Stackdriver Monitoring.

Overview

Applications typically output log messages when various events occur, such as when they receive a request or when they experience a problem. An easy way to monitor those events is to keep track of the number of associated log messages by using logs-based metrics.

  1. Use the filters in the Logs Viewer to show the log entries for an event of interest.
  2. Create a Stackdriver Monitoring metric that counts the number of log entries matching the filter.
  3. Create Stackdriver Monitoring charts and alerting policies using the logs-based metric.

Stackdriver Logging also provides some predefined logging system metrics that you can use immediately to answer questions like the following:

  • How many log entries with severity ERROR were received in the App Engine request log?
  • How many bytes of log entries were received in syslog from your Compute Engine instances?

Before you start

  1. Sign in to your Google account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Cloud Platform project.

    Go to the Projects page

  3. Enable billing for your project.

    Enable billing

  4. Create or use a Stackdriver account for your project.

Creating a metric

Follow these steps to create a logs-based metric in the Logs Viewer:

  1. Go to the Logs Viewer in the Cloud Platform Console. Use the logs selector menus and the search bar to create a filter that shows only the log entries you want to count in your metric. Alternatively, you can change to an advanced logs filter.

  2. Select the bar-graph icon, "Create Metric," which is either next to the search bar in the basic filter or underneath the advanced filter. You will see a new section:

    Create a logs-based metric

  3. Give the metric a name and a description. Metric names can include only the following characters: A-Z, a-z, 0-9, and the special characters _-.,+!*',()%/. The slash character (/) denotes a hierarchy of name pieces and cannot be the first character of the name. Metric names must not be more than 100 characters long.

  4. Select Save.

  5. Read the message that appears and select OK.

  6. If you wish, select the Metrics panel to see your new metric. The settings menu for each metric offers several options. See Managing logs-based metrics

Creating a chart

See the Stackdriver Monitoring Quickstart for a general introduction to creating dashboards and charts. Use logs-based metrics in your chart by using the following instructions:

  1. In the Add Chart dialog, select Log Metrics in the Resource Type menu.
  2. In the Metric menu, select your logs-based metric. If you named your metric error_count in Stackdriver Logging then it will appear as user/error_count in the Metric menu.

If you have a problem, see the Troubleshooting section.

Stackdriver Monitoring charts now have a View Logs option in their settings menu. This is not specifically related to logs-based metrics, but rather lets you look at log entries that are relevant to whatever metrics are being displayed. For more information, see View logs in the Stackdriver Monitoring Quickstart.

Creating an alerting policy

See the Stackdriver Monitoring Quickstart for a general introduction to creating alerting policies. Use logs-based metrics in your policy by using the following instructions:

  1. In the Add Condition > Target dialog, select Log Metrics in the Resource Type menu.
  2. In the Add Condition > Configuration dialog, select your logs-based metric. If you named your metric error_count in Stackdriver Logging then it will appear as user/error_count in the Metric menu.

Alternatively, you can quickly create an alerting policy from a logs-based metric. Go to the Logs Viewer in the Cloud Platform Console and select the Metrics panel. Find your logs-based metric in the list and select Create alert from metric from the metric's settings menu.

To test the new alerting policy, be sure Stackdriver Logging receives enough log entries of the right kind within the alert's time window. If you have a problem, see the Troubleshooting section.

Reading logs-based metrics

The Stackdriver Logging API has methods for creating, listing, modifying, and deleting logs-based metrics. For more information, see the project.metrics collection in the API and the Stackdriver Logging command-line tool.

To read time series data from your logs-based metrics, you use the Stackdriver Monitoring API V3. The Stackdriver Monitoring API v2 cannot access logs-based metrics.

Managing logs-based metrics

Using the Metrics panel in the Logs Viewer, you can edit or delete your metrics.

Go to the Logs Viewer in the Cloud Platform Console. Select Logs-based Metrics in the left-side navigation panel. You see all the logs-based metrics you have created:

User-defined metrics

To delete a single metric, select Delete metric from the settings menu for the metric. To delete several metrics, mark the checkbox next to the metrics you want to delete and then choose Delete above the list of metrics.

To edit a metric, select Edit metric from the settings menu for your metric. You can edit the advanced logs filter or the description for your metric, but you cannot change the metric's name. Select Submit filter after editing your filter to see what log entries are selected. Select Save to save the changes to your metric's filter and description.

Using predefined logging metrics

In addition to the user-defined logs-based metrics described previously in this guide, Stackdriver Logging provides some predefined logs-based system metrics for your project. These are delta metrics, meaning they record the number of events that occurred within a specific time period. Each of the metrics has labels that partition the data. For example, all the system metrics group their data by log.

The following metrics appear in Stackdriver Monitoring under the Log Metrics resource type along with the logs-based metrics you create. The names of the metrics in the menus do not include logging.googleapis.com/, which appears in the following table. You can use these metrics as described in Create a chart and Create an alerting policy.

Metric name and description Type Value Labels
logging.googleapis.com/log_entry_count

The total number of log entries received.

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".

severity: The severity of the log entries. Example: "ERROR".

logging.googleapis.com/byte_count

The total number of bytes received in log entries.

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".
logging.googleapis.com/dropped_log_entry_count

The number of late-arriving log entries.

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".

Troubleshooting

Metric doesn't appear in Stackdriver Monitoring menus

Logs-based metrics appear under the Log Metrics resource type in Stackdriver Monitoring, and user-defined metric names are prefixed with user/.

Metric is missing logs data

There are a few possible reasons for missing data in logs-based metrics:

  • Logs-based metrics starts counting matching log entries that are received after the metric is created. It currently does not support backfilling the metric with existing data.

  • It takes a few minutes to update the logs-based metrics in Stackdriver Monitoring, so you may see log entries in Stackdriver Logging that are not yet counted in Stackdriver Monitoring.

  • If a log entry arrives late to Stackdriver Logging, it will appear in the Logs Viewer but will not be counted in the logs-based metrics. A log entry is considered late if it arrives more than two minutes after the timestamp included in the log entry. The number of late-arriving entries is recorded for each log in the system metric, logging.googleapis.com/dropped_log_entry_count.

Metric doesn't appear in the Stackdriver Monitoring API

Only the Stackdriver Monitoring API V3 can access logs-based metrics.

Send feedback about...

Stackdriver Logging