Structure of time series

This page is an in-depth continuation of the discussion and examples presented in Metrics, time series, and resources and Components of the metric model. To get the most from this page, read those pages first.

This page is intended for readers who want a deeper understanding of time series and the implementation of the Cloud Monitoring metric model, particularly those readers who are using the Monitoring API. This includes users of custom (user-defined) metrics.

  • This page maps the general metric model to the Cloud Monitoring API, so it is useful for anyone using the Monitoring API directly. Users whose primary interaction is with the Google Cloud console probably don't need this level of detail.

  • It is also useful to anyone using a third-party integration like OpenTelemetry. As data from an external source reaches Cloud Monitoring through an integration, it is mapped to the Cloud Monitoring metric model and treated like any other metric data.

For more on collecting your own metrics, see User-defined metrics overview.

Review of the metric model

A prerequisite for understanding the structure of time series is familiarity with the metric model used by Cloud Monitoring. This model is described in Metrics, time series, and resources. This section provides a brief review.

In general terms, monitoring data is recorded in time series. Each individual time series includes three pieces of information relevant to this discussion:

  • A set of time-stamped data points.
  • A reference to the metric type that tells you how to interpret the data points.

  • A reference to the monitored resource from which the data originated.

    Structure of a time series.

Each time series holds the measurements from a single physical or logical source. If your environment has hundreds of resources, each contributes at least one time series. In fact, if your metric type and monitored-resource type have labels, you get one time series per combination of label values; see Cardinality for more information.

A sample time series

A time series is represented as an instance of a TimeSeries object. The following is a complete instance of a single time series:

    {
      "metric": {
        "labels": {
          "log": "kubelet",
          "severity": "DEFAULT"
        },
        "type": "logging.googleapis.com/log_entry_count"
      },
      "resource": {
        "type": "gce_instance",
        "labels": {
          "instance_id": "5106847938295940291",
          "zone": "us-central1-a",
          "project_id": "a-gcp-project"
        }
      },
      "metricKind": "DELTA",
      "valueType": "INT64",
      "points": [
        {
          "interval": {
            "startTime": "2019-12-20T20:25:38Z",
            "endTime": "2019-12-20T20:26:38Z"
          },
          "value": {
            "int64Value": "20"
          }
        }
      ]
    }

Most time series include a lot more data points; this one covers a one-minute interval. All time series have the same structure, with the following fields:

  • The metric field records

    • The metric-label values for this particular time series, representing one combination of label values.
    • The metric type with which the data is associated. The metric type specifies the available labels and describes what is represented by the data points.

    The information in this field is described in more detail in Metrics.

  • The resource field records:

    • The resource-label values for this particular time series, representing one combination of label values.
    • The specific monitored resource from which the data was collected.

    The information in this field is described in more detail in Monitored-resource objects.

  • The metricKind and valueType fields tell you how to interpret the values. For more information, see Metric kinds and types.

  • The points field is an array of timestamped values. The metric type tells you what the values represent. The sample time series has an array with a single data point; in most time series, the array has many more values.

For a live example that retrieves time series data, see Time series: data from a monitored resource.

This rest of this page looks at the information in the metric and resource fields in more detail.

Monitored resources

Cloud Monitoring collects data from monitored resources and records it in time series. Each time series created includes a description of the monitored resource from which the data was collected.

Monitored-resource objects

A monitored resource is represented by an instance of a MonitoredResource object. The monitored resource describes the source of the values in a time series. The MonitoredResource object is embedded into the time series and identifies a specific instance of a monitored-resource type known to Monitoring: it points to a physical or logical entity.

For example, here is a monitored-resource object describing a particular Compute Engine instance, extracted from the example time series:

      "resource": {
        "type": "gce_instance",
        "labels": {
          "instance_id": "5106847938295940291",
          "zone": "us-central1-a",
          "project_id": "a-gcp-project"
        }
      }

This instance of a monitored resource is of the type gce_instance, it has a specific instance_id value (5106847938295940291) in the project a-gcp-project. The instance is located in the US. Another time series might originate from a different instance of this monitored resource. Each combination of label values identifies a unique resource from which data is collected.

Different sets of labels make sense for different resources. The set of labels for a resource is determined by the monitored-resource descriptor for that type.

Monitored-resource descriptors

The information about a monitored-resource type is held in a data structure called a monitored-resource descriptor. For a definition of this data structure, see MonitoredResourceDescriptor.

A monitored-resource descriptor is like a schema or specification for a record structure. It doesn't contain the data about a particular instance of the monitored resource; it tells you how to describe a monitored resource of a specific type. For example, here is the monitored-resource descriptor for the gce_instance resource:

{
  "type": "gce_instance",
  "displayName": "VM Instance",
  "description": "A virtual machine instance hosted in Compute Engine.",
  "name": "projects/[PROJECT_ID]/monitoredResourceDescriptors/gce_instance"
  "labels": [
    {
      "key": "project_id",
      "description": "The identifier of the GCP project associated with this resource, such as \"my-project\"."
    },
    {
      "key": "instance_id",
      "description": "The numeric VM instance identifier assigned by Compute Engine."
    },
    {
      "key": "zone",
      "description": "The Compute Engine zone in which the VM is running."
    }
  ],
}

Compare this monitored-resource descriptor to the specific instance of a gce_instance resource shown in Monitored-resource objects. This descriptor doesn't describe a particular monitored resource; instead, the descriptor specifies the labels, and each combination of label values identifies a particular monitored resource.

You can retrieve the monitored-resource descriptors in your Google Cloud project by using the Cloud Monitoring API. See, for example, the reference pages for the monitoredResourceDescriptors.list and monitoredResourceDescriptors.get methods.

Live example: If you have a Google Cloud project, you can retrieve this descriptor by using the APIs Explorer widget as follows:

  1. Open the monitoredResourceDescriptors.list reference page.

  2. In the pane labeled Try this method, enter the following:

    • name: projects/PROJECT_ID/monitoredResourceDescriptors/gce_instance

      Replace PROJECT_ID with the ID of your Google Cloud project.

  3. Click Execute.

The request returns the preceding descriptor.

For examples using the Monitoring API methods, see List metric and resource types.

Metrics

Each time series created by Cloud Monitoring records a set of data points as well as information about the organization and meaning of those data points.

Metric objects

Each time series includes a reference to the description of the data being recorded in a metric object.

The metric object contained in a time series specifies the type of measurements and metric-specific information about those measurements. For a definition of the metric object data structure, see Metric. For example, here is the metric object extracted from the time series in A sample time series:

    {
      "metric": {
        "labels": {
          "log": "kubelet",
          "severity": "DEFAULT"
        },
        "type": "logging.googleapis.com/log_entry_count"
      }

This object tells you that the time series contains logging.googleapis.com/log_entry_count measurements. The label values tell you that this specific time series counts only log entries of severity DEFAULT in the log file called kubelet.

There is one time series for each combination of label values, so INFO entries to the same log file appear in a different time series.

The set of labels collected in a metric object is specified in the descriptor for that metric type.

Metric descriptors

The information about a metric type is held in a data structure called a metric descriptor. For a definition of this data structure, see MetricDescriptor.

A metric descriptor is a schema or specification for a record structure. It doesn't contain the data about a particular metric; rather, it tells you how to interpret the data associated with a specific metric type.

Here is an example metric descriptor:

{
  "type": "logging.googleapis.com/log_entry_count",
  "name": "projects/a-gcp-project/metricDescriptors/logging.googleapis.com/log_entry_count",
  "labels": [
    {
      "key": "log",
      "description": "Name of the log."
    },
    {
      "key": "severity",
      "description": "Severity of the log entry."
    }
  ],
  "metricKind": "DELTA",
  "valueType": "INT64",
  "unit": "1",
  "description": "Number of log entries that contributed to user-defined metrics.",
  "displayName": "Log entries",
  "metadata": {
    "launchStage": "GA",
    "samplePeriod": "60s"
  },
  "launchStage": "GA"
}

Compare this metric descriptor to the metric object shown in Metric objects. The descriptor tells you what the label values mean, as well as how to interpret the values of the data points.

You can retrieve metric descriptors by using the Cloud Monitoring API. See, for example, the reference pages for the metricDescriptors.list and metricDescriptors.get methods.

Live example: If you have a Google Cloud project, you can retrieve this descriptor by using the APIs Explorer widget as follows:

  1. Open the metricDescriptors.list reference page.

  2. In the pane labeled Try this method, enter the following:

    • name: projects/PROJECT_ID/metricDescriptors/logging.googleapis.com/log_entry_count Replace PROJECT_ID with the ID of your Google Cloud project.

  3. Click Execute.

The request returns the preceding descriptor.

For examples using the Monitoring API methods, see List metric and resource types.

Most of the fields in the metric descriptor are self-explanatory. The two likely to need further explanation are the metric kind and value type, which are described further in Metric kinds and types.

Cloud Monitoring has approximately 6,500 built-in metric types; see the Metrics list for details. You can also create your own metric descriptors to capture custom metrics. For more information, see User-defined metrics overview.