This page illustrates using the Cloud Monitoring API to create and manage alerting policies programmatically. It also illustrates the use of the Cloud SDK command-line interface for managing alerting policies.
Many of these tasks can also be performed by using the Cloud Monitoring console; see Using Alerting Policies for an introduction to creating and managing alerting policies with the Cloud Monitoring console.
Before you begin
Before writing code against the API, you should:
- Be familiar with the general concepts and terminology used with alerting policies; see Introduction to Alerting for more information.
- Ensure that the Cloud Monitoring API is enabled for use; see Enabling the API for more information.
-
Configure a Cloud Monitoring Workspace for your project by doing the following:
- In the Cloud Console, select your Google Cloud project.
Go to Cloud Console - In the navigation pane, select Monitoring.
If you have never used Cloud Monitoring, then on your first access of Monitoring in the Google Cloud Console, a Workspace is automatically created and your project is associated with that Workspace. Otherwise, if your project isn't associated with a Workspace, then a dialog appears and you can either create a Workspace or add your project to an existing Workspace. We recommend that you create a Workspace. After you make your selection, click Add.
- In the Cloud Console, select your Google Cloud project.
- Install the client libraries for the languages you want to use; see Client Libraries for details. Currently, API support for alerting is available only for C#, Go, Java, Node.js, and Python.
Install Cloud SDK. This gives you the access to the
gcloud
command-line interface, from which you can also do these tasks. If you use Cloud Shell, you use that instead of installing Cloud SDK.Examples using the
gcloud
interface are also provided here. Note that thegcloud
examples all assume that the current project has already been set as the target (gcloud config set project [PROJECT_ID]
) so invocations omit the explicit--project
flag. The ID of the current project in the examples isa-gcp-project
.Ensure that you have the proper permissions for the Google Cloud project. For more information, see Permissions.
About alerting policies
An alerting policy is represented by an AlertPolicy
object,
which describes a set of conditions indicating a potentially
unhealthy status in your system. Alerting policies reference
notification channels, which let you specify how you want to be informed
that an alerting policy has been triggered.
Each alerting policy belongs to a scoping project of a metrics scope (or the Workspace's host project). Each
project can contain up to 500 policies.
For API calls, you must provide a “project ID”; use the
ID of the scoping project of a metrics scope (or the Workspace's host project) as the value. In these examples,
the ID of the scoping project of a metrics scope (or the Workspace's host project) is a-gcp-project
.
The AlertPolicy
resource supports five operations:
- Creating new policies
- Deleting existing policies
- Retrieving specific policies
- Retrieving all policies
- Modifying existing policies
Alerting policies can be expressed in JSON or YAML, which lets you record policies in files, and use files to back up and restore policies. With Cloud SDK, you can create policies from files in either format. With the REST API, you can create policies from JSON files. See Sample policies for a selection of alerting policies in JSON format.
The following examples use the gcloud
interface and the API
to illustrate these basic use cases. The API samples are excerpted
from a sample program that uses the API to implement a backup and restore
system for alerting policies. Fuller samples are shown in Example: backup
and restore.
Creating policies
To create an alerting policy in a project, use the
alertPolicies.create
method.
You can create policies from JSON or YAML files.
The gcloud
command-line interface accepts these files as arguments, and
you can programmatically read JSON files, convert them to AlertPolicy
objects, and create policies from them
by using the alertPolicies.create
method.
The following examples illustrate the creation of alerting policies.
gcloud
To create an alerting policy in a project, use the gcloud alpha monitoring
policies create
command. The following example creates an alerting policy in
a-gcp-project
from the rising-cpu-usage.json
file:
gcloud alpha monitoring policies create --policy-from-file="rising-cpu-usage.json"
If successful, this command returns the name of the new policy, for example:
Created alert policy [projects/a-gcp-project/alertPolicies/12669073143329903307].
The file rising-cpu-usage.json
file contains the JSON for a policy with
the display name “High CPU rate of change”. This policy is described in
A rate-of-change policy.
See the
gcloud alpha monitoring policies create
reference for more information.
C#
Go
Java
Node.js
PHP
Python
The created AlertPolicy
object will have additional fields.
The policy itself will have name
, creationRecord
, and mutationRecord
fields. Additionally, each condition in the policy is also given a name
.
These fields cannot be modified externally, so there is no need to set them
when creating a policy. None of the JSON examples used for creating
policies include them, but if policies created from them are retrieved after
creation, the fields will be present.
Retrieving policies
To retrieve a list of the policies in a project, use the
alertPolicies.list
method.
Use this method to retrieve policies and apply some action to each of
them, for example, backing them up. This method also supports
filter
and orderBy
options to restrict and sort the results;
see Sorting and Filtering.
If you are looking for a specific policy and you know its name,
you can use the alertPolicies.get
method
to retrieve only that policy. The name of a policy is the value of the name
field, not the displayName
, in the AlertPolicy
object. The name of
a policy has the format projects/[PROJECT_ID]/alertPolicies/[POLICY_ID]
,
for example:
projects/a-gcp-project/alertPolicies/12669073143329903307
gcloud
To list all alerting policies in a project, use the gcloud alpha monitoring
policies list
command:
gcloud alpha monitoring policies list
If successful, the list
command provides a listing of all the policies in the
specified project, formatted as YAML. For example, the policy with the display
name “High CPU rate of change” in the project
a-gcp-project
is listed like this, among the other policies listed:
---
combiner: OR
conditions:
- conditionThreshold:
aggregations:
- alignmentPeriod: 900s
perSeriesAligner: ALIGN_PERCENT_CHANGE
comparison: COMPARISON_GT
duration: 180s
filter: metric.type="compute.googleapis.com/instance/cpu/utilization" AND resource.type="gce_instance"
thresholdValue: 0.5
trigger:
count: 1
displayName: CPU usage is increasing at a high rate
name: projects/a-gcp-project/alertPolicies/12669073143329903307/conditions/12669073143329903008
creationRecord:
mutateTime: '2018-03-26T18:52:39.363601689Z'
mutatedBy: [USER@DOMAIN]
displayName: High CPU rate of change
enabled: true
mutationRecord:
mutateTime: '2018-03-26T18:52:39.363601689Z'
mutatedBy: [USER@DOMAIN]
name: projects/a-gcp-project/alertPolicies/12669073143329903307
---
To list a single alerting policy, use gcloud alpha monitoring policies
describe
, instead, and specify the name of the policy. For example, this
command returns only the listing above:
gcloud alpha monitoring policies describe projects/a-gcp-project/alertPolicies/12669073143329903307
See the gcloud alpha monitoring policies list
and describe
references for more information. The describe
command corresponds to the
alertPolicies.get
method in the API.
C#
Go
Java
Node.js
PHP
Python
Deleting policies
To delete a policy from a project, use the
alertPolicies.delete
method and supply the name of the
alerting policy to delete.
gcloud
To delete an alerting policy, use gcloud alpha monitoring policies
delete
, and specify the name of the policy to delete. For example, the
following command deletes the “High CPU rate of change” policy:
gcloud alpha monitoring policies delete projects/a-gcp-project/alertPolicies/12669073143329903307
See the
gcloud alpha monitoring policies delete
reference for more information.
Modifying policies
To modify an alerting policy, use the
alertPolicies.patch
method (in the REST API).
Other API implementations and the gcloud
interface call this update
instead of patch
.
An update operation can wholly replace the existing policy, or it can modify
a subset of fields. An update operation takes a new AlertPolicy
object and
an optional field mask.
If a field mask is specified, then any field listed in the field mask is updated with the value in the supplied policy. If the supplied policy does not include a field mentioned in the field mask, then that field is cleared and set to its default value. Any field not listed in the mask retains its previous value.
If no field mask is specified, then the existing policy is replaced by the
supplied one, but the name (projects/[PROJECT_ID]/alertPolicies/[POLICY_ID]
)
is reused. Any conditions in the new policy that have name
values that include
a CONDITION_ID
will keep those names. If not, new condition and policy names
are created.
When using the gcloud
command-line to update policies, command-line flags
rather than a field mask are used to specify fields to update.
See the gcloud alpha monitoring policies update
for details.
Enabling or disabling a policy
To enable or disable a policy, change the value of the boolean
enabled
field in the AlertPolicy
object. Note that after you enable
a policy, it can still be triggered by data collected while it was disabled.
gcloud
To disable an alerting policy, use the
gcloud alpha monitoring policies update
command and provide the --no-enabled
flag. The following command disables the “High CPU rate of change”
alerting policy in the project a-gcp-project
:
gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 --no-enabled
To enable the policy, use the same command and provide the --enabled
flag.
See the gcloud alpha monitoring policies update
reference for more information. The update
command corresponds to the
alertPolicies.patch
method in the REST API.
C#
Go
Java
Node.js
PHP
Python
Updating notification channels in a policy
You can also update the notification channels referenced by an alerting policy. Alerting policies refer to notification channels by name. The channels must exist before they can be used in an alerting policy.
You create and manage notification channels programmatically by using
the NotificationChannel
and
NotificationChannelDescriptors
resources.
These examples in this
section assume these channels already exist, and uses of these APIs also
appear in the programmatic samples.
For more discussion of the notification-channel objects, see Managing Channels by API.
gcloud
To modify the notification channels in an alerting policy, use the
gcloud alpha monitoring policies update
command. There are several
flags related to notification channels, letting you remove notification
channels, replace notification channels, and add new notification channels.
For example, the policy with the display name “High CPU rate of change” in the project a-gcp-project was created with no notification channels.
To add a notification channel to this policy, use the gcloud alpha monitoring
policies update
command, and specify the channel to add with the
--add-notification-channels
flag:
gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 \
--add-notification-channels="projects/a-gcp-project/notificationChannels/1355376463305411567"
See the gcloud alpha monitoring policies update
reference for more information. The update
command corresponds to the
alertPolicies.patch
method in the REST API.
The notification channel added here must already exist; see Creating channels for more information.
C#
Go
Java
Node.js
PHP
Python
Modifying documentation in a policy
A policy can include documentation that is included with incidents and notifications associated with the policy. Use this field to include information to help responders understand and handle the problem indicated by the alert policy. Documentation is included in email notifications and notification types that allow for it; other channel types may omit it.
gcloud
To add documentation to a policy, or replace existing documentation, use thegcloud alpha monitoring policies update
command and provide the
--documentation-format="text/markdown"
flag (the only supported format)
and either the --documentation
flag (to enter the value from the command line)
or the --documentation-from-file
flag (to read the value from a file).
For example, the policy with the display name “High CPU rate of change” in the project a-gcp-project was created with no documentation.
The following command sets the documentation
field in the specified policy to
the contents of the cpu-usage-doc.md
file:
gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 \
--documentation-format="text/markdown" \
--documentation-from-file="cpu-usage-doc.md"
See the gcloud alpha monitoring policies update
reference for more information. The update
command corresponds to the
alertPolicies.patch
method in the REST API.
Example: backup and restore
All of the API examples shown are pulled from a larger application that can back up the alerting policies in a project to a file and can restore the policies, possibly to another project. If the projects used for backup and restore are different, the application effectively exports and imports policies from one project to another.
This section shows the code for backup and restore in context rather than as a set of small, isolated excerpts.
Backing up policies
The back-up operation is straight-forward. The set of alerting policies and the set of notification channels in each project is collected and saved to external storage in JSON.
C#
Go
Java
Node.js
PHP
Python
Restoring the backed-up policies
The restoration process is more complex than the original back-up. You can restore to the project you originally backed up. You can also restore to a different project, effectively providing import of alerting policies.
If restoring to the same project, any existing channels or policies are updated if they still exist. If they do not, they are recreated. Read-only fields, like the creation and mutation records, in the backed-up policies are cleared by the restoration process before policies and notifications are recreated.
You can use a policy saved in one project to create a new, or similar, policy in another project. However, you must first make the following changes in a copy of the saved policy:
- Remove the following fields from any notification channels:
name
verificationStatus
- Create notification channels before referring to the channels in alerting policies (you need the new channel identifiers).
- Remove the following fields from any alerting policies you are recreating:
name
condition.name
creationRecord
mutationRecord
If the policy is being recreated in a new project, the names of any conditions in the backed-up policies are cleared along with the creation and mutation records.
Additionally, when a notification channel is recreated in a different project, it gets a different name, so the restore process has to map the names of channels in backed-up alerting policies to their new names, and replace the old names with the new ones.
In addition to the names of notification channels, the value of the
verificationStatus
field cannot be set when the channel is created or
updated, so a sentinel value, unspecified
is used. After channels have
been restored into a new project, they must be explicitly verified.
C#
Go
Java
Node.js
PHP
Python
Alerting and Cloud SDK
In Cloud SDK, the command group for managing alerting policies and
notification channels is monitoring
, which is in alpha release.
The monitoring
group is available in the alpha
component.
That is, these commands will all begin with:
gcloud alpha monitoring
To check if you have the alpha
component installed, run this command:
gcloud components list
If you do not have the alpha
component installed, run this command to
install it:
gcloud components install alpha
If you do have the alpha
component, check for the monitoring
group by
running this command:
gcloud alpha monitoring --help
If the monitoring
group is not included, Cloud SDK will prompt you to
add it:
You do not currently have this command group installed.
[...]
Do you want to continue (Y/n)? y