Managing metric-based alerting policies

This page illustrates how to use the Google Cloud Console to create and manage an alerting policy based on metrics. This content does not apply to log-based alerting policies. For information about log-based alerting policies, which notify you when a particular message appears in your logs, see Monitoring your logs.

An alerting policy describes a set of conditions that you want to monitor. These conditions might relate to the state of an unhealthy system or to resource consumption. For example, you might want to create a policy to monitor an uptime check, or to create a policy that monitors your Cloud Monitoring API usage. In addition to conditions, in an alerting policy, you can specify how you want to be notified and what documentation is included in that notification.

You can also use the Cloud Monitoring API to create and manage alerting policies. For more information about this approach, see Managing alerting policies by API. To see policies represented in JSON, see Sample policies.

Before you begin

Before creating alerting policies, you should be familiar with the general concepts and terminology in alerting policies. This includes understanding the components of a policy, the concept of an incident, and pricing and limitations. See Introduction to alerting for more information.

Creating an alerting policy

To create an alerting policy, do the following:

  1. In the Cloud Console, select Monitoring:

    Go to Monitoring

  2. Select Alerting.

  3. Click Create Policy to see the Create alerting policy page:

    Create an alerting policy dialog is displayed.

    1. Click Add condition and complete the dialog. For information on the fields in a condition, see Specifying conditions.

      A condition describes a monitored resource, a metric for that resource, and when the condition is met. An alerting policy must have at least 1 condition, however alerting policies can contain up to 6 conditions. If an alerting policy has exactly 1 condition and if the condition is met, then an incident is created. If an alerting policy has multiple conditions, then you specify how these conditions are combined. For more information, see Policies with multiple conditions.

    2. Click Next to advance to the notifications section.

    3. To be informed when an incident is created, add a notification channel to your alerting policy. You can add multiple notification channels. For details on your choices of notification channels, see Notification options.

      To add a notification channel, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

      Notification dialog displaying the refresh and manage channels buttons.

      If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. When you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy. For more information, see Creating a channel on demand.

    4. (Optional) If you want to be notified when an incident is opened and closed, then check Notify on incident closure. By default, notifications are sent only when an incident is opened.

    5. Click Next to advance to the documentation section.

    6. Click Name and enter a policy name. This name is included in notifications and it is displayed in the Policies page.

    7. (Optional) Specify the documentation to be included in notifications. To format your documentation, you can use Markdown. To pull information out of the policy itself to tailor the content of your documentation, you can use variables. For more information about how you can format and tailor the content of this field, see Using Markdown and variables in documentation templates.

      For example, documentation might include a title such as Addressing High CPU Usage and details that identify the project:

      ## Addressing High CPU Usage
      
      This note contains information about high CPU Usage.
      
      You can include variables in the documentation. For example:
      
      This alert originated from the project ${project}, using
      the variable $${project}.
      

      The variables are replaced by their values only in notifications. The Preview Markdown pane, and the other places in the Cloud Console that show the documentation, reflect only the Markdown formatting:

      Example writing a documentation note using markdown.

      You can also include channel-specific tagging to control notifications. For more information, see Using channel controls.

  4. Click Save.

Adding an alerting policy to a dashboard

When an alerting policy contains one condition, you can display a summary of that alerting policy on a custom dashboard. The summary includes the time series that are monitored by the alerting policy, the threshold, and the number of open incidents.

To display a summary of an alerting policy on a custom dashboard, do the following:

  1. In the Cloud Console, select Monitoring:

    Go to Monitoring

  2. Select Dashboards and open the custom dashboard that you want to modify.

  3. If Editing isn't shown, then click Viewing and select Switch to Editing mode.

  4. Select Alert chart from the widget library, or click Add chart and then select Alert chart from the menu.

  5. In the configuration pane of the Alert chart, use the the Alert policy menu to select an alerting policy. Only single-condition alerting policies can be selected from the Alert policy menu.

The following screenshot illustrates an alert chart:

Example of an alert chart.

In this example, the alerting policy is monitoring the CPU usage of two different virtual machines. The condition threshold, which is set to 50%, is shown by the dashed red line. The green chip with the label No incidents indicates that there are no open incidents for the alerting policy. If you place your pointer on the incidents chip, then a dialog opens that links to the underlying alerting policy.

For more information, see Using dashboards and charts.

Managing policies

To list all alerting policies, do the following:

  1. In the Cloud Console, select Monitoring:

    Go to Monitoring

  2. Select Alerting.

  3. A partial list of policies is shown in the Policies pane. To see all policies and to enable filtering, click See all policies.

To view the details of an alerting policy, click on its name.

To restrict the alerting policies that are listed, add filters. Each filter is composed of a name and a value. You can set the value to be an exact match for a policy name, or a partial match. Matches are not case sensitive. If you have multiple filters, then the filters are automatically joined by a logical AND unless you insert an OR filter. The following screenshot lists all alerting policies that match test or My Uptime Check Alert Policy:

Sample alerts overview with filters.

From the Policies page you can edit, delete, copy, enable, or disable an alerting policy:

  • To edit or copy a policy, click More options , and select the corresponding option. Editing and copying a policy is similar to Creating an alerting policy. You can change, and in some cases, delete the values in the fields. When done, click Save.

  • To delete a policy, click More options and select Delete. In the confirmation dialog, select Delete.

  • To changed the enabled status of the alerting policy, click the toggle located under the heading Enabled.

If you select an alerting policy, then the Policy details are displayed. For example, the following screenshot illustrates the details for the alerting policy named Test staging:

Sample alert-policy details page.

The Notifications section lists if you've configured this policy to notify you when incidents are opened and closed, or if you've configured it to notify you only when incidents are opened. To change the notification behavior, edit the alerting policy.

You can edit, delete, copy, enable, or disable an alerting policy from the Policy details page. For example, to disable an alerting policy that is currently enabled, click Enabled and select Turn off. Similarly, to enable a policy that is currently disabled, click Disabled.