Setting up alerts and notifications

You're viewing Apigee X documentation.
View Apigee Edge documentation.

One of the main purposes of API Monitoring is to keep you informed of unusual events or patterns, such as spikes in traffic or latencies. To track events like these, you can set up alerts, which are triggered when a specified event occurs. An actual event that triggers an alert is called an incident.

You can set specific rules for an alert—for example, it might be triggered when the number of API requests exceeds 3600 over a period of 1 minute. When an incident for the alert occurs, you can view its details in the Events pane of the Cloud Console Alerting dashboard.

You define an alert rule using two main properties:

  • The metric: The quantity that triggers the alert—for example, number of requests.
  • The threshold: The level that the metric must reach to trigger the alert.

If you want to be notified when a particular alert is triggered, you can set up a notification for the alert. When an incident occurs, Apigee sends you a notification with a summary of the event, including links to the Cloud Console and the Apigee UI, for further investigation.

To learn more about alerts, see Introduction to alerting.

Initial steps for creating an alert

To create an alert, start by doing the steps below.

  1. Open the Create alerting policy dashboard in the Google Cloud Console.

    Note: Open the Create alerting policy dashboard in a new tab if you want to follow the steps below.

    Create alerting policy page

  2. Click ADD CONDITION. This opens a new window where you can add the alert condition.

    Configure an alert.

  3. Optional: Add a title for the condition in the Untitled Condition field. We'll leave it blank for this example, in which case the condition is given a default title.
  4. Copy and paste the following in the Find resource type and metric field:

    Apigee proxy (v2)

    The Target pane now appears as shown below.

    Select a metric.

    The pane displays the Select a metric field, where you can specify the metric, such as total traffic or latency percentile.

After completing the initial steps above, you can continue by selecting a metric and adding conditions for the alert in the Configuration pane.

The next two sections present examples that show how to do this:

Traffic spike alert

The following sections show how to create an alert that is triggered when the total API traffic is above 3600 for 1 minute.

Initial steps

To set up the alert, start by doing the initial steps described in the preceding section. When you are done, add the settings described below.

Target settings

In the Target pane:

  1. Copy one of the code samples below, depending on whether you want an alert for requests or responses, and paste it in the Select a metric field.

    Requests

    apigee.googleapis.com/proxyV2/request_count

    Responses

    apigee.googleapis.com/proxyV2/response_count
  2. Click the Select a metric field and select either Apigee proxy request cumulative or Apigee proxy response cumulative.

    Find resource type and metric.

    This displays the fields shown below.

    Select target options.

  3. In the Aggregator field, select sum. This sums the total traffic over the time span specified by Period.
  4. Set Period to 1 minute.

    These settings will trigger an alert based on the total number of requests in each 1 minute interval of the time range.

Configuration settings

The Configuration settings specify the conditions that trigger the alert. The Threshold is the minimum value of the quantity specified in the Target section—which in this case is requests per second—that will set off the alert.

For example, suppose you want the alert to be triggered if the total traffic in any 1 minute interval is above 36. Since the threshold is measured in requests per second, you divide 3600 by 60 to get a threshold of 60.

Enter the following in the Configuration pane:

  • Set Threshold to 60.
  • Set For to 1 minute.

The Configuration pane now appears as shown below.

Configuration settings

Finally, click ADD to add the alert condition. This takes you back to the Create alerting policy dashboard, which displays the new condition, as shown below.

New condition added.

From here you can choose to either:

  • Click ADD Condition to create another condition for the alert, or
  • Click NEXT to go to Step 2, where you will have the option to [create a notification for the alert](#creating-a-notification-for-an-alert), as described in the next section.

If you don't want to create a notification, click NEXT twice without entering anything to bypass Steps 2 and 3.

Click SAVE to save the alert.

Viewing incident details

An API event that triggers an alert is called an incident. You can view the details of recent incidents in the Cloud Console Alerting dashboard.

Metrics and filters

This section describes the metrics and filters you can use to create alerts.

Metrics

A metric is the quantity that triggers an alert, for example, number of requests. The table below shows the available metrics.

Metric Description
apigee.googleapis.com/proxyv2/request_count Total number of requests received by the Apigee API proxy.
apigee.googleapis.com/proxyv2/response_count Total number of responses returned by the Apigee API proxy.
apigee.googleapis.com/proxyv2/latencies_percentile Percentile of all API proxy responses to a request.

Filters

Filters enable you to create alerts for subsets of API data. A filter specifies a condition that defines the subset. For example, you can create an alert based on data for a single proxy.

You can choose from the following filters:

Filter Description Metrics for which filter can be applied
env Environment for the data. All
location Location for the data. All
proxy_name Proxy for the data. All
runtime_version Runtime version for the data. All
instance_id Instance ID for the data. All
method HTTP method for the data. All
response_code HTTP response code for the data. apigee.googleapis.com/proxyv2/response_count
fault_code Fault code for the data. apigee.googleapis.com/proxyv2/response_count
fault_source Fault source for the data. apigee.googleapis.com/proxyv2/response_count
percentile Apigee proxy response latencies percentile. apigee.googleapis.com/proxyv2/latencies_percentile

Creating a notification for an alert

If you want to be notified immediately when an incident occurs, you can create a notification for the alert. When the alert is triggered, Apigee sends you a notification. You can choose any of the following channels to receive the notification:

  • Email
  • PagerDuty
  • Slack
  • Webhooks

To create a notification for an alert:

  1. If you just created the alert (as shown in the preceding example) and are currently viewing the Create alerting policy dashboard, skip to the next step.

    Otherwise, open the Policies pane in the Cloud Console Alerting dashboard, and in the row for the alert you created, click the three dots icon at the end of the row and select Edit.

    Notification options

    This opens the Edit alerting policy dashboard:

    Notification options
  2. In the What do you want to track? section, click NEXT.

    Notification options
  3. In the Who should be notified section, select Notification Channels, where you can choose from existing channels for receiving notifications.

    If you want to create a new notification channel, select MANAGE NOTIFICATION CHANNELS. This opens the Notification channels dashboard in the Cloud Console, where you can add channels. See Managing notification channels for more details.

  4. Click NEXT. This displays the options shown below.

    Notification options
  5. In the Alert name field, enter a name for the alert.
  6. In the Documentation field, enter the URL for a web page that provides information on how to fix the issue that caused the alert. The URL will be included in the notification.
  7. Click Save to create the alert.

When an alert is triggered, you will receive a notification providing a summary of the incident and when it occurred. The notification also contains two links to help you investigate the incident:

Latency alert

The following section shows how to create an alert that is triggered when the 90th percentile of API latency is above 600 ms for 10 minutes.

Initial steps

To create the alert, start by doing the Initial steps described in the first section. After doing so, the Target pane appears as shown below.

Select a metric.

Target settings

Next, do the following steps in the Target pane:

  1. Copy the code below and paste it in the Select a metric field.
    apigee.googleapis.com/ProxyV2/latencies-percentile

    or select Percentile of Apigee proxy response.

    Select latency metric.

  2. Under Filter, click in the Add a filter field and select percentile.

    Select a metric.

  3. In the Value field that appears below Select a metric, select 90.

    Select a metric.

  4. Click Apply.

Configuration settings

In the Configuration pane, do the following steps:

  1. Set Threshold to 600.

    Select threshold.

With these settings, an alert will be triggered when the 90th percentile of the API's latency is above 600 for 10 minutes.

The UI displays the latency graph with the threshold, as shown below.

Select a metric.

To create the alert, click Add at the bottom of the UI. This takes you back to the Create alerting policy dashboard.

See Creating a notification to learn how to create a notification for the alert.