Structure of time series

This page is an in-depth continuation of the discussion and examples presented in Metrics, time series, and resources and Components of the metric model. To get the most from this page, read those pages first.

This page is intended for readers who want a deeper understanding of time series and the implementation of the Cloud Monitoring metric model, particularly those readers who are using the Monitoring API. This includes users of custom (user-defined) metrics.

  • This page maps the general metric model to the Cloud Monitoring API, so it is useful for anyone using the Monitoring API directly. Users whose primary interaction is with the Google Cloud Console probably don't need this level of detail.

  • It is also useful to anyone using a third-party integration like OpenCensus. As data from an external source reaches Cloud Monitoring through an integration, it is mapped to the Cloud Monitoring metric model and treated like any other metric data.

For more on collecting your own metrics, see Using custom metrics.

Review of the metric model

A prerequisite for understanding the structure of time series is familiarity with the metric model used by Cloud Monitoring. This model is described in Metrics, time series, and resources. This section provides a brief review.

In general terms, monitoring data is recorded in time series. Each individual time series includes three pieces of information relevant to this discussion:

  • A set of time-stamped data points.
  • A reference to the metric type that tells you how to interpret the data points.
  • A reference to the monitored resource from which the data originated.

    Structure of a time series

Each time series holds the measurements from a single physical or logical source. If your environment has hundreds of resources, each contributes at least one time series. In fact, if your metric type and monitored-resource type have labels, you get one time series per combination of label values; see Cardinality for more information.

A sample time series

The following is a complete instance of a single time series (type TimeSeries):

    {
      "metric": {
        "labels": {
          "log": "kubelet",
          "severity": "DEFAULT"
        },
        "type": "logging.googleapis.com/log_entry_count"
      },
      "resource": {
        "type": "gce_instance",
        "labels": {
          "instance_id": "5106847938295940291",
          "zone": "us-central1-a",
          "project_id": "a-gcp-project"
        }
      },
      "metricKind": "DELTA",
      "valueType": "INT64",
      "points": [
        {
          "interval": {
            "startTime": "2019-12-20T20:25:38Z",
            "endTime": "2019-12-20T20:26:38Z"
          },
          "value": {
            "int64Value": "20"
          }
        }
      ]
    }

Most time series include a lot more data points; this one covers a one-minute interval. All time series have the same structure, with the following fields:

  • The metric field records

    • The metric-label values for this particular time series, representing one combination of label values.
    • The metric type with which the data is associated. The metric type specifies the available labels and describes what is represented by the data points.

    The information in this field is described in more detail in Metrics.

  • The resource field records:

    • The resource-label values for this particular time series, representing one combination of label values.
    • The specific monitored resource from which the data was collected.

    The information in this field is described in more detail in Monitored resources.

  • The metricKind and valueType fields tell you how to interpret the values. See Metric kinds and types for more information on these.

  • The points field is an array of timestamped values. The metric type tells you what the values represent. The sample time series has an array with a single data point; in most time series, the array has many more values.

For a live example that retrieves time series data, see Time series: data from a monitored resource.

This rest of this page looks at the information in the metric and resource fields in more detail.

Monitored resources

Cloud Monitoring collects data from monitored resources and records it in time series. Each time series created includes a description of the monitored resource from which the data was collected.

Monitored-resource objects

Each time series records the source of the data points as a monitored-resource object (type MonitoredResource) This object is embedded into the time series and identifies a specific instance of a monitored-resource type known to Monitoring: it points to a physical or logical entity.

For example, here is a monitored-resource object describing a particular Compute Engine instance, extracted from the example time series:

      "resource": {
        "type": "gce_instance",
        "labels": {
          "instance_id": "5106847938295940291",
          "zone": "us-central1-a",
          "project_id": "a-gcp-project"
        }
      }

This instance of a monitored resource is of the type gce_instance, it has a specific instance_id value (5106847938295940291) in the project a-gcp-project. The instance is located in the US. Another time series might originate from the a different instance of this monitored resource. Each combination of label values identifies a unique resource from which data is collected.

Different sets of labels make sense for different resources. The set of labels for a resource is determined by the monitored-resource descriptor for that type.

Monitored-resource descriptors

The information about a monitored-resource type is held in a data structure called a monitored-resource descriptor (type MonitoredResourceDescriptor).

A monitored-resource descriptor is like a schema or specification for a record structure. It doesn't contain the data about a particular instance of the monitored resource; it tells you how to describe a monitored resource of a specific type. For example, here is the monitored-resource descriptor for the gce_instance resource:

{
  "type": "gce_instance",
  "displayName": "GCE VM Instance",
  "description": "A virtual machine instance hosted in Google Compute Engine (GCE).",
  "name": "projects/[PROJECT_ID]/monitoredResourceDescriptors/gce_instance"
  "labels": [
    {
      "key": "project_id",
      "description": "The identifier of the GCP project associated with this resource, such as \"my-project\"."
    },
    {
      "key": "instance_id",
      "description": "The numeric VM instance identifier assigned by Compute Engine."
    },
    {
      "key": "zone",
      "description": "The Compute Engine zone in which the VM is running."
    }
  ],
}

Compare this monitored-resource descriptor to the specific instance of a gce_instance resource shown in Monitored-resource objects. This descriptor doesn't describe a particular monitored resource; instead, the descriptor specifies the labels, and each combination of label values identifies a particular monitored resource.

You can retrieve the monitored-resource descriptors in your Google Cloud project by using the Cloud Monitoring API. See, for example, the reference pages for the monitoredResourceDescriptors.list and monitoredResourceDescriptors.get methods.

Live example: If you have a Google Cloud project, you can retrieve this descriptor by using the API Explorer widget, located on the reference page for the monitoredResourceDescriptors.get method in the Monitoring API.

When you run this example, you must change [PROJECT_ID] in the name field to your project ID. Then click the Execute button at the bottom of the widget panel. The request returns the descriptor above.

Try It!

For more examples using the Monitoring API methods, see Browsing metric and resource types.

At last count, Cloud Monitoring can monitor approximately over 100 resource types; see the Resource list for details. The list is derived directly from the set of available monitored-resource descriptors.

Metrics

Each time series created by Cloud Monitoring records a set of data points as well as information about the organization and meaning of those data points.

Metric objects

Each time series includes a reference to the description of the data being recorded in a metric object.

A metric object (type Metric) is a collection of information that indicates the metric type and the values of metric type's labels for the data points in this time series. For example, here is the metric object extracted from the time series in A sample time series:

    {
      "metric": {
        "labels": {
          "log": "kubelet",
          "severity": "DEFAULT"
        },
        "type": "logging.googleapis.com/log_entry_count"
      }

This object tells you that the time series contains measurements for the logging.googleapis.com/log_entry_count metric type. The label values tell you that this specific time series counts only log entries of severity DEFAULT in the log file called kubelet.

There is one time series for each combination of label values, so INFO entries to the same log file appear in a different time series.

The set of labels collected in a metric object is specified in the descriptor for that metric type.

Metric descriptors

The information about a metric type is held in a data structure called a metric descriptor (type MetricDescriptor).

A metric descriptor is a schema or specification for a record structure. It doesn't contain the data about a particular metric; rather, it tells you how to interpret the data associated with a specific metric type.

Here is the metric descriptor for the metric type logging.googleapis.com/log_entry_count:

{
  "type": "logging.googleapis.com/log_entry_count",
  "name": "projects/a-gcp-project/metricDescriptors/logging.googleapis.com/log_entry_count",
  "labels": [
    {
      "key": "log",
      "description": "Name of the log."
    },
    {
      "key": "severity",
      "description": "Severity of the log entry."
    }
  ],
  "metricKind": "DELTA",
  "valueType": "INT64",
  "unit": "1",
  "description": "Number of log entries that contributed to user-defined metrics.",
  "displayName": "Log entries",
  "metadata": {
    "launchStage": "GA",
    "samplePeriod": "60s"
  },
  "launchStage": "GA"
}

Compare this metric descriptor to the metric object shown in Metric objects. The descriptor tells you what the label values mean, as well as how to interpret the values of the data points.

You can retrieve metric descriptors by using the Cloud Monitoring API. See, for example, the reference pages for the metricDescriptors.list and metricDescriptors.get methods.

Live example: If you have a Google Cloud project, you can retrieve this descriptor by using the API Explorer widget, located on the reference page for the metricDescriptors.get method in the Monitoring API.

When you run this example, you must change [PROJECT_ID] in the name field to your project ID. Then click the Execute button at the bottom of the widget panel. The request returns the descriptor above.

Try It!

For more examples using the Monitoring API methods, see Browsing metric and resource types.

Most of the fields in the metric descriptor are self-explanatory. The two likely to need further explanation are the metric kind and value type, which are described further in Metric kinds and types.

At last count, Cloud Monitoring has approximately over 1,500 built-in metric types; see the Metrics list for details. You can also create your own metric descriptors to capture custom metrics. For more information, see Using Custom Metrics.