Overview of Logs-based Metrics

Logs-based metrics are Stackdriver Monitoring metrics that are based on the content of log entries. For example, the metrics can record the number of log entries containing particular messages, or they can extract latency information reported in log entries. You can use logs-based metrics in Stackdriver Monitoring charts and alerting policies.

Logs-based metrics that are predefined by Stackdriver Logging are called system logs-based metrics. Logs-based metrics that are created by the user are called user-defined logs-based metrics.

To see the list of logs-based metrics in your GCP project, click the following button:

Go to Logs-based Metrics

Before you start

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a GCP project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your project.

    Learn how to enable billing

  4. Create or use a Stackdriver account for your project.

Overview

Stackdriver Logging logs-based metrics can be one of two metric types: counter or distribution. All system logs-based metrics are the counter type. User-defined logs-based metrics can be either the counter type or the distribution type.

The following sections describe the characteristics of counter and distribution type metrics.

Counter metrics

Counter metrics count the number of log entries matching an advanced logs filter. For example, you can do the following:

  • Create a metric that counts log entries containing a certain specific error message.
  • Count the number of times each user invokes an operation, by looking for log messages like

    ... user [USERNAME] called  [OPERATION] ...
    

    By extracting [USERNAME] and [OPERATION] and using them as values for two labels, you can later ask, "How many times did sally call the UPDATE operation?", "How many people called the READ operation?", "How many times did george call an operation?", and so on.

For more information, see Creating Counter Metrics.

Distribution metrics

Distribution metrics accumulate numeric data from log entries matching a filter. The metrics contain a time series of Distribution objects, each of which contains the following:

  • A count of the number of values in the distribution.
  • The mean of the values.
  • The sum of squared deviations: Sumi=1..n(xi–mean)2
  • A set of histogram buckets with the count of values in each bucket. You can use the default bucket layout or choose your own.

A common use for distribution metrics is to track latencies. As each log entry is received, a latency value is extracted from somewhere in the log entry and is added to the distribution. At regular intervals, the accumulated distribution is written to Stackdriver Monitoring.

For more information, see Creating Distribution Metrics.

Common properties of metrics

A logs-based metric is a delta metric. Each data point in the metric's time series represents only the additional information received since the previous data point.

Stackdriver Logging accumulates information for logs-based metrics every time a matching log entry is received. On a regular schedule, Stackdriver Logging writes a new data point to the metric's time series, making the data available to Stackdriver Monitoring.

Labels

Logs-based metrics can optionally have labels, which allow a single metric to hold multiple time series. Values for the labels are extracted from fields in the matching log entries. Stackdriver Logging records separate time series for each different value of your label.

The system logs-based metrics have predefined labels. You can define your own labels for your user-defined metrics. For more information, see Logs-based Metric Labels.

System logs-based metrics

Stackdriver Logging provides some predefined counter metrics that track the number and volume of log entries received. The metrics have labels that record the counts by log name and severity level. The following table lists the metrics:

Metric name and description Type Value Labels
logging.googleapis.com/log_entry_count

The total number of log entries received.

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".

severity: The severity of the log entries. Example: "ERROR".

logging.googleapis.com/byte_count

The total number of bytes received in log entries.

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".
logging.googleapis.com/excluded_log_entry_count

The total number of log entries that were excluded.

delta int64
logging.googleapis.com/excluded_byte_count

The total number of bytes in log entries that were excluded.

delta int64
logging.googleapis.com/logs_based_metrics_error_count

The number of late-arriving log entries.1

delta int64 log: The name of the log. Example: "appengine.googleapis.com/request_log".

1 The late-arriving log entries are not included in the log_entry_count or byte_count metrics.

See Logging metrics for a full list of system logs-based metrics.

Stackdriver Monitoring

You can use both system and user-defined logs-based metrics in Stackdriver Monitoring to create charts and alerting policies. Your user-defined logs-based metric names are prefixed by user/; the system logs-based metrics are not.

If you are using the Stackdriver Monitoring API, the logs-based metrics names are shown below:

logging.googleapis.com/user/[USER_METRIC_NAME]
logging.googleapis.com/[SYSTEM_METRIC_NAME]

For more information, see Creating Charts and Alerts.

Troubleshooting

Metric is missing logs data

There are a few possible reasons for missing data in logs-based metrics:

  • New log entries might not match your metric's logs filter. A logs-based metric gets data from matching log entries that are received after the metric is created. Stackdriver Logging does not backfill the metric from previous log entries.

  • New log entries might not contain the correct field, or the data might not be in the right format for extraction by your distribution metric. Check that your field names and regular expressions are correct.

  • Your metric counts may be delayed. Even though you see countable log entries in the Logs Viewer, it takes up to a minute to update the logs-based metrics in Stackdriver Monitoring.

  • The log entries you see might be counted late or might not be counted at all, because they are time-stamped too far in the past or future. If a log entry is received more than 10 minutes in the past or future, then the log entry will not be counted in the logs-based metric.

    The number of late-arriving entries is recorded for each log in the system logs-based metric, logging.googleapis.com/logs_based_metrics_error_count.

    Example: A log entry matching a logs-based metric arrives late. It has a timestamp of 2:30 PM and a receivedTimestamp of 2:45 PM. This entry will not be counted in the logs-based metric.

Metric has too many time series

You might have a user-defined label in a logs-based metric that has a large number of unique values. The number of time series in a metric depends on the number of different combinations of label values. When the cardinality of the label values is very high, the metric can get throttled and some points may not be written to the metric. The metric will also be slow to load in charts due to the large number of timeseries it has to process. The number of timeseries also determines overage charges. To avoid creating high cardinality metrics:

  • Check that your label fields and extractor regular expressions match values that have a limited cardinality.

  • Avoid extracting long text messages that can change unboundedly as label values.

  • Avoid extracting numerical values with unbounded cardinality.

  • Only extract values that you know the cardinality of. For example, status codes which have a known set of values.

Label values are truncated

Values for user-defined labels must not exceed 1,024 bytes.

Was this page helpful? Let us know how we did:

Send feedback about...

Stackdriver Logging