Log-based metrics overview

This page provides a conceptual overview of log-based metrics.

Log-based metrics can help you observe trends and patterns in a large volume of log entries. These metrics are derived from the log entries that originate in your Google Cloud project or that are routed to your Google Cloud project by a sink in another project. For example, you can use a log-based metric to count the number of log entries that contain a particular message or to extract latency information recorded in log entries. You can display log-based metrics in Cloud Monitoring charts, and alerting policies can monitor these metrics.

If you plan to create an alerting policy to monitor a log-based metric, then ensure sure that you understand the limitations of this approach. In particular, review Troubleshoot: Metric is missing logs data.

Sources of log-based metrics

You can use the metrics defined by Cloud Logging to collect general usage information, and you can define your own log-based metric to capture information specific to your application or business.

Log-based metrics can apply within a single Google Cloud project or within a log bucket. You can't create log-based metrics for other Google Cloud resources such as Cloud Billing accounts or organizations.

For information about the differences between project-based log-based metrics and bucket-based log-based metrics, see Bucket-scoped log-based metrics.

System-defined metrics

Logging provides a set of metrics for usage values such as the number of log entries stored in log buckets in your project, or the number of bytes you've exported. For a complete list of system-defined metrics, see Google Cloud metrics: logging.

User-defined metrics

You can create user-defined log-based metrics to track other metrics that are important for your project. For example, you might create a log-based metric to count the number of log entries that match a given filter.

By default, user-defined log-based metrics are calculated from all logs received by the Logging API for the Google Cloud project, regardless of any inclusion filters or exclusion filters that might apply to the Google Cloud project.

You can also create user-defined log-based metrics for a specific log bucket in a Google Cloud project. Bucket-scoped log-based metrics are calculated from all logs destined for the bucket, regardless of where they originated. For more information see Log-based metrics on log buckets.

Data types for log-based metrics

Log-based metrics can extract data from logs to create metrics of the following types:

Counter: these metrics count the number of log entries that match a specified filter within a specific period. Use counters when you want to keep track of the number of times a value or string appears in your logs.
Distribution: these metrics also count values, but they collect the counts into ranges of values (histogram buckets). Use distributions when you want to extract values like latencies.
Boolean: these metrics capture whether or not a log entry matches a specified filter.

User-defined log-based metrics can be of the counter or distribution metric types. Most system-defined log-based metrics are counters, but some are of the boolean type. The characteristics of counters and distributions are described in more detail in subsequent sections.

The data for a user-defined log-based metric comes only from log entries received after the metric is created. A metric isn't retroactively populated with data from log entries that are already in Logging.

System log-based metrics are calculated from included logs only. User-defined log-based metrics are calculated from both included and excluded logs.

Logging accumulates information for a log-based metric each time it receives a matching log entry. Logging writes a new data point to the metric's time series at the rate of 1 datapoint per minute, making the data available to Cloud Monitoring.

Each data point in a log-based metric's time series represents only the additional information (the delta) received since the previous data point.

The following sections describe the characteristics of counter-type and distribution-type metrics.

Counter metrics

Counter metrics count the number of log entries matching a given filter. For example, you can do the following:

Count the log entries that contain a certain specific error message.
Count the number of times each user invokes an operation, by looking for log messages that match this pattern:

... user USERNAME called OPERATION ...

By extracting USERNAME and OPERATION and using them as values for two labels, you can later ask, "How many times did sally call the update operation?", "How many people called the read operation?", "How many times did george call an operation?", and so on.

For more information, see Configure counter metrics.

Distribution metrics

Distribution metrics accumulate numeric data from log entries matching a filter. The metrics contain a time series of distribution objects, each of which contains the following:

A count of the number of values in the distribution.
The mean of the values.
The sum of squared deviations: Sum_i=1..n(x_i–mean)²
A set of histogram buckets with the count of values in each bucket. You can use the default bucket layout or choose your own.

A common use for distribution metrics is to track latencies. As each log entry is received, a latency value is extracted from somewhere in the log entry and is added to the distribution. At regular intervals, the accumulated distribution is written to Cloud Monitoring.

For information about distributions, including their format within a time series and how they are visualized, see Charting distribution metrics.

For information about creating distribution log-based metrics, see Configure distribution metrics.

Resource types

We recommend that your log-based metrics specify a single resource type. This specification ensures that the time-series data generated for your log-based metric matches the data model expected by Cloud Monitoring. It also simplifies the process of creating charts and alerting policies, as there is no ambiguity as to which time-series data is charted or monitored.

Consider a log-based metric with the following filter:

severity>="ERROR"
resource.type="gce_instance"

Cloud Monitoring recognizes that this data is for a Compute Engine VM instance. Therefore, when you create a chart for a VM instance, your log-based metric is listed as an option in the menus. When you create a chart for a different resource type, your log-based metric isn't listed as an option.

If you create a log-based metric without specifying a resource type, then Monitoring lists your log-based metric as an option for multiple resource types, even though you don't count entries from those resources.

Labels

Log-based metrics can have labels, which allow multiple time series to be collected for the metric. Values for the labels are extracted from fields in the matching log entries. Logging records separate time series for each combination of label values.

All log-based metrics provide a default log label. This label holds the value of the LOG_ID portion of the logName field in the log entry, for example, cloudresourcemanager.googleapis.com%2Factivity.

System log-based metrics have predefined labels that can't be changed. However, you can customize labels for user-defined metrics. For example, you can use labels in the log-based metric to record the folder or organization from which a log entry originated, even though bucket-scoped log-based metrics can't be created for folders or organizations. For more information, see Log-based metric labels.

Charts and alerting policies in Cloud Monitoring

You can use both system and user-defined log-based metrics in Cloud Monitoring to create charts and alerting policies. For more information, see List and chart log-based metrics and Alert on log-based metrics.

In Cloud Monitoring, log-based metrics use the following naming patterns:

System: logging.googleapis.com/SYSTEM_METRIC_NAME
User-defined: logging.googleapis.com/user/USER_METRIC_NAME

Note that user-defined log-based metrics include the string user.

Visibility to Monitoring metrics scopes

Log-based metrics are ingested by Cloud Monitoring, and the visibility of metric data to a Google Cloud project is determined by a metrics scope. A metrics scope is a list of projects that are monitored by the project that hosts the metrics scope; the hosting project is called a scoping project.

By default, each project hosts a metrics scope that includes only itself, so a project is a scoping project for itself. Therefore, your metrics, including log-based metrics, are visible only to your Google Cloud project.

You can also create a multi-project metrics scope for the scoping project. With a multi-project metrics scope, the scoping project can see the metrics from all the projects in the metrics scope. What is visible to the individual projects in a multi-project metrics scope is determined by the metrics scope hosted by each of those projects. The fact that two projects are in a multi-project metrics scope does not mean that each project has access to the metric or configuration data in the other project.

A single project can also appear in multiple metrics scopes. The metrics from such a project are visible to the scoping projects of each of those metrics scopes.

Metrics, including log-based metrics, are defined within a specific project. When that project appears in multiple metrics scopes, the metrics are visible to projects other than the one in which they are defined. For information about how metrics scopes and bucket-scoped log-based metrics interact, see Projects and metrics scopes.

For more information about metrics scopes, including multi-project metrics scopes, and about scoping projects, see the following:

Pricing

All user-defined log-based metrics are a class of Cloud Monitoring custom metrics and are chargeable. For pricing information, see Cloud Logging pricing: Log-based metrics.

Quota

For information about the quotas and limits associated with user-defined log-based metrics, see Quotas and limits.

Troubleshooting

If you encounter issues when using log-based metrics, see Troubleshoot log-based metrics.