Working with the API

Service Monitoring adds the following resources to the Monitoring API:

This page illustrates primary uses of the new API resources. For a general overview of the structures used here, see Constructs in the API. Comprehensive reference material appears under Cloud Monitoring API v3.

This document collectively refers to these additions as the SLO API.

When to use the API

The SLO API can be used to manage services and service-level objectives (SLOs). This page focuses on custom services and service-level indicators (SLIs). Services running on Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine are automatically detected, and the SLIs for them are predefined:

To define SLOs for Anthos Service Mesh services, you can use the Anthos Service Mesh console or the SLO API. For more information on creating SLOs by using the Anthos Service Mesh dashboard, see Anthos Service Mesh documentation: Creating SLOs.
For other services and for custom services, you must define SLOs by using the SLO API. There is currently no Google Cloud Console support available.

The examples on this page focus primarily on custom services and SLIs.

Valid identifiers

Several methods in the SLO API use identifiers for different elements, particularly identifiers for project and services. These IDs adhere to the following rules:

Length: 1–63 characters
Characters: only lower-case letters, number, and hyphens
Initial character: lower-case letter
Terminal character: lower-case letter or a number, but not a hyphen

The regular expression for these rules is as follows: [a-z](?:[-a-z0-9]{0,61}[a-z0-9])?

Examples using `curl`

This section describes the conventions and setup used for invoking the SLO API by using the curl tool. If you are using this API through a client library, you can skip this section.

The examples on this page access the SLO API by using the curl tool to send HTTP requests to REST endpoints. Use the following information about authentication and about invoking curl to set the variables used in the invocations.

Authentication

Create an environment variable to hold the ID of your Cloud Monitoring Workspace. These examples use PROJECT_ID:
```
PROJECT_ID=my-test-service
```
Authenticate to the Cloud SDK:
```
gcloud auth login
```
To avoid having to specify your project ID with each command, set it as the default by using Cloud SDK:
```
gcloud config set project ${PROJECT_ID}
```
Create an authorization token and capture it in an environment variable. These examples call the variable ACCESS_TOKEN:
```
ACCESS_TOKEN=`gcloud auth print-access-token`
```
You have to periodically refresh the access token. If commands that worked suddenly report that you are unauthenticated, re-issue this command.
To verify that you got an access token, echo the ACCESS_TOKEN variable:
```
echo ${ACCESS_TOKEN}
ya29.GluiBj8o....
```

Invoking `curl`

Each curl invocation includes a set of arguments, followed by the URL of a SLO API resource. The common arguments include values specified by the PROJECT_ID and ACCESS_TOKEN environment variables. You might also have to specify other arguments, for example, to specify the type of the HTTP request (for example, -X DELETE). The default request is GET, so the examples don't specify it.

Each curl invocation has this general structure:

curl --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" <other_args> https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/<request>

For example, to list all the services in your project, issue the following GET request:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services

This request returns an array of service descriptions, with entries like the following, an Istio service called “currencyservice”:

{
  "services": [
    {
      "name": "projects/[PROJECT_NUMBER]/services/[PROJECT_ID]-zone-us-central1-c-csm-main-default-currencyservice",
      "displayName": "[PROJECT_ID]/us-central1-c/csm-main/default/currencyservice",
      "clusterIstio": {
        "location": "us-central1-c",
        "clusterName": "csm-main",
        "serviceNamespace": "default",
        "serviceName": "currencyservice"
      },
      "telemetry": {
        "resourceName": "//container.googleapis.com/projects/[PROJECT_ID]/zones/us-central1-c/clusters/csm-main/k8s/namespaces/default/services/currencyservice"
      }
    },
   ...
  ]
}

See Service for more information about the structure of a service.

Managing services

The Service resource acts as the root element for organizing your services. Aspects of a particular service, like its SLOs for example, are organized under the name of the service.

Services on Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine are created automatically for you. You can do things like add SLOs to these services by using the Anthos Service Mesh console or the SLO API.

Services you create manually are called custom services. Custom services can be created, deleted, retrieved, and updated only by using the SLO API.

After you have identified or created a service, you can add SLOs to it. Managing SLOs covers commands to manipulate SLOs.

Service IDs

Each service has a fully qualified identifier called the resource name. This identifier is stored in the name field of the service description, for example:

"name": "projects/[PROJECT_NUMBER]/services/[PROJECT_ID]-zone-us-central1-a-cloudrun-istio-system-cluster-local-gateway",

Embedded in the resource name is a shorter ID for the service, the part of the resource name after the substring projects/[PROJECT_NUMBER]/services/

If you created your own service with the resource name projects/[PROJECT_NUMBER]/services/my-test-service, the service ID is my-test-service.

For brevity and accuracy, many curl examples assume the service ID is held in the environment variable SERVICE_ID so you can refer to it repeatedly.

Listing services

To retrieve information about all the services in your project, invoke the services.list method. To retrieve information about a specific service by service ID, use the services.get method:

Protocol

To list information about services by using curl, invoke the services.list or services.get method by sending a GET request to:

https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services to list all services
https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID] to get a specific service

For example, the following request retrieves information about the service identified by the value stored in the environment variable SERVICE_ID:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}

Creating a service

If you are not using an environment where services are automatically created (that is, Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine), then you can create services by using the services.create method.

If you manually create services, you then have to manually add the SLOs and other service-monitoring artifacts to the service structure by using the SLO API. For an overview of these structures, see Constructs in the API.

To create a service, you must specify a display name for the service and a field named custom with an empty object. You can optionally specify the service ID you want the service to have.

Protocol

To create the service by using curl, send a POST message to the https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services endpoint:

If you want to provide an ID for the service, create a variable for the service ID:
```
SERVICE_ID=my-test-service
```
This value is supplied in the URL query parameter service_id.

Create a variable to hold the request body that describes the service:

CREATE_SERVICE_POST_BODY=$(cat <<EOF
{
    "displayName": "My Test Service",
    "custom": {}
}
EOF
)

Post the request body to the endpoint, and specify the service ID as the value of the service_id query parameter:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" --header "Content-Type: application/json" -X POST -d "${CREATE_SERVICE_POST_BODY}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services?service_id=${SERVICE_ID}

Deleting a service

To delete a custom service, invoke the services.delete method and specify the service ID.

Protocol

To delete a service by using curl, send a DELETE request to the https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID] endpoint:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" -X DELETE https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}

Managing SLOs

SLOs are associated with services. You can create SLOs using the Anthos Service Mesh console for Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine, or by using the SLO API. You must use the SLO API to create SLOs for custom services.

You can also use the SLO API to retrieve the SLOs associated with a service, and to delete SLOs from a service. You can have up to 500 SLOs for each service.

To manage SLOs for a service, you must have the service ID. SLOs are named based on the service they belong to. Service IDs describes how to recognize service IDs.

Listing SLOs

To retrieve information about all the SLOs associated with a service, invoke the services.serviceLevelObjectives.list method. To retrieve information about a specific SLO by name, use the services.serviceLevelObjectives.get method:

Protocol

To list information about services by using curl, invoke the services.serviceLevelObjectives.list or services.serviceLevelObjectives.get method by sending a GET request to:

https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID]/serviceLevelObjectives to list all SLOs
https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID]/serviceLevelObjectives/[SLO_ID] to get a specific SLO

For example, the following request lists all SLOs associated with the service ID stored in the environment variable SERVICE_ID:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}/serviceLevelObjectives

The following is an availability SLO retrieved from the “currencyservice” service:

{
  "name": "projects/[PROJECT_NUMBER]/services/[PROJECT_ID]-zone-us-central1-c-csm-main-default-currencyservice/serviceLevelObjectives/3kavNVTtTMuzL7KcXAxqCQ",
  "displayName": "98% Availability in Calendar Week"
  "serviceLevelIndicator": {
    "basicSli": {
      "availability": {},
      "location": [
        "us-central1-c"
      ]
    }
  },
  "goal": 0.98,
  "calendarPeriod": "WEEK",
},

This SLO is built on an availability SLI, it sets a target performance goal of 98 percent, and it measures compliance over a calendar week. For more information on availability SLIs, see Service-level indicators.

See ServiceLevelObjective for more information about the structure of SLOs.

Retrieving a specific SLO

Each SLO has a unique identifier within the service, consisting of the string following serviceLevelObjectives in the SLO's name field. In the following example:

"name": "projects/[PROJECT_NUMBER]/services/[PROJECT_ID]-zone-us-central1-c-csm-main-default-currencyservice/serviceLevelObjectives/3kavNVTtTMuzL7KcXAxqCQ",

the SLO ID is the string 3kavNVTtTMuzL7KcXAxqCQ.

To retrieve information about this particular SLO, request the SLO by ID.

Protocol

To get a specific SLO by using curl, invoke the services.serviceLevelObjectives.get method by sending a GET request to https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID]/serviceLevelObjectives/[SLO_ID]:

SLO_ID=dhefHDM4TwSRZEKIV4ZYEg

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}/serviceLevelObjectives/${SLO_ID}

Creating an SLO

To create an SLO by using the SLO API, you must create a ServiceLevelObjective object and pass it to the serviceLevelObjectives.create method.

The structure of an SLO has many embedded structures, including one for the value of the serviceLevelIndicator field.

For Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine services, the SLIs are pre-defined. You can use the Anthos Service Mesh console to create SLOs; all you have to do is specify the performance goals and compliance periods.

You can also define SLOs by using the SLO API.
For custom services, you must define SLOs by using the SLO API. To specify an SLO, you must create a value for the serviceLevelIndicator field, as well. See Creating a service-level indicator for some techniques.

Skeleton of a SLO

The basic skeleton for building the SLO is as follows:

{
  "displayName": string,
  "serviceLevelIndicator": {
    object (ServiceLevelIndicator)
  },
  "goal": number,

  // Union field period can be only one of the following:
  "rollingPeriod": string,
  "calendarPeriod": enum (CalendarPeriod)
  // End of list of possible types for union field period.
}

You must specify the following:

Display name: a description of the SLO
A service-level indicator, which is one of the three types:
The performance goal (a percentage)
The compliance period, which one of two types:
- A rolling period of some length (in seconds)
- A calendar period

For more information about SLIs, performance goals, and compliance periods, see Concepts in service monitoring.

For Anthos Service Mesh, Istio on Google Kubernetes Engine, and App Engine services, the SLI type is the basic SLI.

For custom services, you have to create a request-based SLI or a windows-based SLI. See Creating a service-level indicator for some techniques.

Example

After you have the SLI, you can build the SLO. The following example defines an SLO for a service that uses a basic SLI. You might have several SLOs on a single SLI, for example, one for 95% availability and one for 99% availability. The following example is an SLO for 95% availability over a calendar week:

{
  "serviceLevelIndicator": {
    "basicSli": {
      "availability": {},
      "location": [
        "us-central1-c"
      ]
    }
  },
  "goal": 0.95,
  "calendarPeriod": "WEEK",
  "displayName": "95% Availability in Calendar Week"
}

This example specifies an SLO for 75% availability over a rolling 3-day period:

{
  "serviceLevelIndicator": {
    "basicSli": {
      "availability": {},
      "location": [
        "us-central1-c"
      ]
    }
  },
  "goal": 0.75,
  "rollingPeriod": "259200s",
  "displayName": "75% Availability in Rolling 3 Days"
}

Protocol

To create an SLO by using curl, send a POST message to the https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID]/serviceLevelObjectives endpoint:

Create a variable for the service ID:
```
SERVICE_ID=custom:my-test-service
```

Create a variable to hold the request body, an instance of the ServiceLevelObjective object. This example uses one of the SLOs described previously:

CREATE_SLO_POST_BODY=$(cat <<EOF
{
  "serviceLevelIndicator": {
    "basicSli": {
      "availability": {},
      "location": [
        "us-central1-c"
      ]
    }
  },
  "goal": 0.75,
  "rollingPeriod": "259200s",
  "displayName": "75% Availability in Rolling 3 Days"
}
EOF
)

Post the request body to the endpoint:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" --header "Content-Type: application/json" -X POST -d "${CREATE_SLO_POST_BODY}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}/serviceLevelObjectives

Optionally, you can also specify a desired ID for the SLO as the value of the service_level_objective_id query parameter:

SLO_ID=test-slo-id

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" --header "Content-Type: application/json" -X POST -d "${CREATE_SLO_POST_BODY}" https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}/serviceLevelObjectives?service_level_objective_id=${SLO_ID}

Deleting an SLO

To delete an SLO, invoke the services.serviceLevelObjectives.delete method and specify the ID of the SLO in your project:

Protocol

To delete an SLO by using curl, send a DELETE request to the https://monitoring.googleapis.com/v3/projects/[PROJECT_ID]/services/[SERVICE_ID]/serviceLevelObjectives/[SLO_ID] endpoint:

curl  --http1.1 --header "Authorization: Bearer ${ACCESS_TOKEN}" -X DELETE https://monitoring.googleapis.com/v3/projects/${PROJECT_ID}/services/${SERVICE_ID}/serviceLevelObjectives/${SLO_ID}

Accessing SLO time series

SLO data is stored in time series, so you can use the timeSeries.list method to retrieve it. However, this data isn't stored in standard metric types, so you can't use the standard mechanism of specifying a metric-type filter to the timeSeries.list method.

Instead, SLO time series are retrieved by specifying another type of filter, a time-series selector, to the timeSeries.list method in the filter parameter. See Retrieving SLO data for information on using these selectors.

You also use time-series selectors to set up alerting policies programmatically. See Alerting on your burn rate for more information.

Working with the API

When to use the API

Valid identifiers

Examples using curl

Authentication

Invoking curl

Managing services

Service IDs

Listing services

Protocol

Creating a service

Protocol

Deleting a service

Protocol

Managing SLOs

Listing SLOs

Protocol

Retrieving a specific SLO

Protocol

Creating an SLO

Skeleton of a SLO

Example

Protocol

Deleting an SLO

Protocol

Accessing SLO time series

Examples using `curl`

Invoking `curl`