This document describes how to configure notification channels by using the Google Cloud console. Cloud Monitoring uses these channels to notify you, or your on-call team, when an alerting policy fires. When you create an alerting policy, you select who is notified by making selections from the list of configured notification channels. For example, you might configure alerts that monitor Compute Engine instances to publish a Pub/Sub topic and to notify the Slack channel for the on-call team.
If your preferred notification channel isn't supported, consider creating a pipeline that relies on sending your notifications to Pub/Sub. For a Python example that uses Flask, see Creating custom notifications with Cloud Monitoring and Cloud Run. For other examples, see the cloud-alerting-notification-forwarding Git repository.
To configure notification channels by using the Cloud Monitoring API, see Create and manage notification channels by API.
For information about alerting-policy notifications, see the following pages:
- For information on the delay between a problem starting and an alert being created, see Notification latency.
- For information on the number of notifications generated for an alerting policy, see Notifications per incident.
By default, an alerting policy sends a notification only when an incident is created. To receive a notification when the incident is opened and when it is closed, edit the alerting policy and, in the notifications section, select Notify on incident closure.
Before you begin
To get the permissions that you need to view and configure notification channels by using the Google Cloud console,
ask your administrator to grant you the
Monitoring Editor (roles/monitoring.editor
) IAM role on your project.
For more information about granting roles, see
Manage access.
You might also be able to get the required permissions through custom roles or other predefined roles.
For more information about Cloud Monitoring roles, see Control access with Identity and Access Management.
Create a notification channel
When you are creating an alerting policy, you can select any configured notification channel and add it to your policy. You can pre-configure your notification channels, or you can configure them as part of the process of creating an alerting policy. For more information, see Create a channel on demand.
To create a notification channel by using the Google Cloud console, follow the channel-specific instructions contained in the following table:
To add an email notification channel, do the following:
- In the Google Cloud console, select Monitoring
- Click Alerting and then click Edit notification channels.
- In the Email section, click Add new.
- Enter a single email address and a description.
- Click Save.
If you use a group email address as the notification channel for
an alerting policy, then configure the group to accept mail from
alerting-noreply@google.com
.
You can create email channels during the creation of an alerting policy. For more information, see Create a channel on demand.
Mobile App
Use the Google Cloud console Mobile App to monitor your Google Cloud console resources and Monitoring information from anywhere. Google Cloud console Mobile App notifications are either sent to a specific device or to a specific user:
- Device indicates that notifications are sent only to the specific device that created the notification channel. For device-scoped notification channels, the Display name field includes device information.
- User indicates that the notifications are sent to all of your devices that have the Google Cloud console Mobile App installed.
Cloud Monitoring determines the notification scope when the channel is created. You can't select or change the scope.
To configure a Google Cloud console Mobile App notification channel for a specific Google Cloud project, do the following:
- Install the Google Cloud console Mobile App from your mobile device's app store.
Select a project for viewing in the Google Cloud console Mobile App.
After you select a project, a data exchange between the app and the selected Google Cloud project occurs. A notification channel is created when one doesn't exist, and after a few minutes, this channel is listed under the Mobile Devices section of the Notification channels page.
To add your mobile device as a notification channel for an alerting policy, in the alerting Notifications section, select Google Cloud console (mobile) and then choose your mobile device from the list.
PagerDuty
Integration with PagerDuty allows for one-way or two-way synchronization with Monitoring. Independent of your configuration, the following are true:
- If an incident is created in Monitoring, then an incident is opened in PagerDuty.
- You can't use PagerDuty to close an incident in Monitoring.
If you use one-way synchronization and if you resolve the incident in PagerDuty, then the state of the incident in PagerDuty is decoupled from the state of the incident in Monitoring. In effect, if you resolve an incident in PagerDuty, then the incident is permanently closed in PagerDuty and can't be reopened.
If you use two-way synchronization, then Monitoring controls the state shown by PagerDuty. If you resolve the incident in PagerDuty and if Monitoring has the incident open, then the incident is reopened in PagerDuty.
To set up PagerDuty notifications, do the following:
- In PagerDuty: Create a PagerDuty account at the PagerDuty site.
- Add the PagerDuty notification channel:
- In the Google Cloud console, select Monitoring
Go to Monitoring - Click Alerting and then click Edit notification channels.
- In the PagerDuty section, click Add new.
- Enter the Display Name. This name should match the name provided to PagerDuty when you added the integration.
- Enter the Integration Service Key generated by PagerDuty into the Service Key field.
- Click Save.
- In the Google Cloud console, select Monitoring
- (Optional) If you want to configure two-way synchronization, then
do the following:
- Open PagerDuty.
- Select Configuration, select Services, and then select the service name you entered when configuring the integration.
- Click Edit Settings, select Create incidents, and then clear Create alerts and incidents.
When you create an alerting policy, select PagerDuty in the Notifications section and choose your PagerDuty configuration.
The JSON packet for PagerDuty has the following format:
{
"description": A string of various fields in the incident,
"details": JSON payload with schema version 1.2.
}
To view an example and the schema for the details
field,
expand the following sections.
Deprecation policy
The payload schema is subject to the Google Cloud deprecation policy outlined in Section 1.4(d) of the Google Cloud Platform Terms of Service. Note that the schema does not control the formats of generated field values, and these formats can change without notice. For example,incident.summary
,
incident.documentation.content
, and incident.url
are meant to include data pertaining to their fields, but the schema doesn't
have constraints to guarantee accurate parsing of these fields. You can
consume the value as a whole and expect that it adheres to the deprecation
policy, but don't rely on parsing the generated fields.SMS
To configure SMS notifications, do the following:
- In the Google Cloud console, select Monitoring
- Click Alerting and then click Edit notification channels.
- In the SMS section, click Add new.
- Complete the dialog and click Save.
When you set up your alerting policy, select the SMS notification type and choose a verified phone number from the list.
Slack
The Monitoring Slack integration allows your alerting policies to post to a Slack channel when a new incident is created. To set up Slack notifications, do the following:
In Slack: Create a Slack workspace and channel at the Slack site. Record the channel URL.
In the Google Cloud console, select Monitoring:
Click Alerting and then click Edit notification channels.
In the Slack section, click Add new to open the Slack sign-in page:
- Select your Slack workspace.
- Click Allow to enable Cloud Monitoring access to your Slack workspace. This action takes you back to the Monitoring configuration page for your notification channel.
- In the Slack Channel Name field, enter the name of the Slack channel you want to use for notifications.
- In the Cloud Alerting Display Name field, enter a short descriptive statement. Monitoring displays the value of this field on the Notifications channel page.
- (Optional) To test the connection between
Cloud Monitoring and your Slack workspace, click
Send test notification. If the connection is successful, then
you see a message
This is a test alert notification...
in the Slack notification channel that you specified. Check the notification channel to confirm receipt.
If the Slack channel you want to use for notifications is a private channel, then you must manually invite the Monitoring app to the channel:
- Open Slack.
Go to the channel you specified as your Monitoring notification channel.
Invite the Monitoring app to the channel by entering and sending the following message in the channel:
/invite @Google Cloud Monitoring
Be sure you invite the Monitoring app to the private channel you specified when creating the notification channel in Monitoring. Inviting the Monitoring app to public channels is optional.
When you create an alerting policy, select Slack in the Notifications section and choose your Slack configuration.
Webhooks
To configure Webhooks notifications, do the following:
- The webhook handler: Identify the public endpoint URL to receive webhook data from Monitoring.
- In the Google Cloud console, select Monitoring
- Click Alerting and then click Edit notification channels.
- In the Webhook section, click Add new.
- Complete the dialog.
- Click Test Connection to send a test payload to the Webhook endpoint. You can go to the receiving endpoint to verify delivery.
- Click Save.
When you create an alerting policy, select Webhook in the Notifications section and choose your webhook configuration. The notifications sent by Error Reporting follow the 1.0 schema while notifications sent by Monitoring follow the 1.2 schema:
Basic authentication
In addition to the webhook request sent by Cloud Monitoring, basic
authentication utilizes the HTTP specification for the username and
password. Cloud Monitoring requires your server to return a 401
response with the proper WWW-Authenticate
header. For more
information about basic authentication, see the following:
Token authentication
Token Authentication requires a query string parameter in the endpoint URL and a key that the server expects to be secret between itself and Monitoring. The following is a sample URL that includes a token:
https://www.myserver.com/stackdriver-hook?auth_token=1234-abcd
If Monitoring posts an incident to the endpoint URL, your server can validate the attached token. This method of authentication is most effective when used with SSL/TLS to encrypt the HTTP request preventing snoopers from learning the token.
For an example server in Python, see this sample server.
Deprecation policy
The payload schema is subject to the Google Cloud deprecation policy outlined in Section 1.4(d) of the Google Cloud Platform Terms of Service. Note that the schema does not control the formats of generated field values, and these formats can change without notice. For example,incident.summary
,
incident.documentation.content
, and incident.url
are meant to include data pertaining to their fields, but the schema doesn't
have constraints to guarantee accurate parsing of these fields. You can
consume the value as a whole and expect that it adheres to the deprecation
policy, but don't rely on parsing the generated fields.
Pub/Sub
This section describes how to configure Pub/Sub notification channels. For redundancy, we recommend that you create multiple notification channels. Pairing Pub/Sub with Google Cloud console Mobile App, PagerDuty, Webhooks, or Slack is recommended, because Pub/Sub uses a different delivery mechanism.
To configure a Pub/Sub notification channel:
- Enable the Pub/Sub API and create a topic
- Configure the notification channel for a topic
- Authorize a service account
- Set the notification channel in an alerting policy
To understand the data schema, see Schema example.
To receive the alert notifications, you must also
create a Pub/Sub subscription.
When the subscription is in a different project than the topic,
create a service account in the subscriber project, and grant it the
role of
roles/pubsub.subscriber
for the topic.
Before you begin
When you add the first notification channel for a Google Cloud project, Cloud Monitoring creates a service account for that project. It also grants the Identity and Access Management role Monitoring Notification Service Agent to the service account. This service account lets Monitoring send notifications to Pub/Sub-based notification channels in this project.
The service account has the following format:
service-PROJECT_NUMBER@gcp-sa-monitoring-notification.iam.gserviceaccount.com
The name of the notifications service account includes a unique Google Cloud project number. You can find the project name, ID, and number on the project's dashboard in the Google Cloud console, or you can retrieve it with the following command:
gcloud projects describe PROJECT_ID --format="value(project_number)"
You can view and edit the service account by using the Google Cloud console or the Google Cloud CLI.
Enable the Pub/Sub API and create a topic
Enable the Pub/Sub API for your Google Cloud project:
Enable Pub/Sub API- Select the Google Cloud project in which you plan to create the Pub/Sub topic.
- Click Enable when shown. No action is required when API enabled is shown.
Create a Pub/Sub topic. For information about how to create a topic, see Create a topic. For example, to create a topic called
notificationTopic
by using the Google Cloud CLI, run the following command:gcloud pubsub topics create notificationTopic
Next, configure your Pub/Sub notification channels.
Configure notifications for a topic
To create a Pub/Sub notification channel, you can use the Monitoring API, the Google Cloud CLI, or the Google Cloud console. After you create the notification channel, authorize the notifications service account to publish each topic that you are using as a notification channel.
For information about using the Monitoring API or the Google Cloud CLI to create the notification channel, see Creating channels.
To use the Google Cloud console to create the notification channel, do the following:
- In the Google Cloud console, select Monitoring
- Select the Google Cloud project that contains the Pub/Sub topic you created.
- Click Alerting and then click Edit notification channels.
In the Pub/Sub section, click Add new.
The Created Pub/Sub Channel dialog displays the name of the service account that Monitoring created.
Authorize the service account. Your service account can publish all topics or specific topics:
To publish all topics, select Manage roles, and add the role of Pub/Sub Publisher.
After you complete these steps that configure the notification channel, skip the next section, which is titled Authorize service account, and proceed to Set the notification channel in an alerting policy.
- To publish specific topics, proceed to the next step and authorize the service account to publish specific topics after you complete configuring the notification channel. For instructions to authorize the service account, see Authorize service account.
- Enter a display name for your channel and the Pub/Sub topic name.
- (Optional) To verify that the channel is correctly configured, click Send test notification.
- Select Add channel.
Next, authorize the service account.
Authorize service account
Authorization lets the notifications service account publish each Pub/Sub topic that you are using as a notification channel. This section describes how you can do the following:
- Authorize a service account for a specific topic.
- Authorize a service account for all topics.
Authorize a service account for a specific topic
You can authorize a service account to publish a specific topic by using the Google Cloud console and the Google Cloud CLI. This section describes both approaches.
To authorize your service account for a specific topic by using the Google Cloud console, do the following:
- Go to the Topics page for Pub/Sub:
Go to Topics - Select the topic.
- In the Permissions tab, select Add principal.
- In the New principal field, enter the name of the notifications
service account. The service account has the following naming
convention:
service-PROJECT_NUMBER@gcp-sa-monitoring-notification.iam.gserviceaccount.com
- Select the role Pub/Sub Publisher, and then select Save.
To authorize your service account for a specific topic by using
the Google Cloud CLI, grant the pubsub.publisher
IAM role for the topic to the service account.
For example, the following command configures the
IAM role for the notificationTopic
topic:
gcloud pubsub topics add-iam-policy-binding \ projects/PROJECT_NUMBER/topics/notificationTopic --role=roles/pubsub.publisher \ --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-monitoring-notification.iam.gserviceaccount.com
A response to a successful execution of the
add-iam-policy-binding
command is like the following:
Updated IAM policy for topic [notificationTopic]. bindings: ‐ members: ‐ serviceAccount:service-PROJECT_NUMBER@gcp-sa-monitoring-notification.iam.gserviceaccount.com role: roles/pubsub.publisher etag: BwWcDOIw1Pc= version: 1
For more information, see the pubsub topics add-iam-policy-binding
reference page.
Authorize a service account for all topics
To authorize your service account for all topics, do the following:
- Go to the IAM page:
Go to IAM - Select Include Google-provided role grants as shown in
the following image:
- Search for the service account that has the following format:
service-PROJECT_NUMBER@gcp-sa-monitoring-notification.iam.gserviceaccount.com
Reload the page when the service account isn't listed.
- Select edit Edit for the notifications service account, select Add another role, and then add the role Pub/Sub Publisher.
Next, set the notification channel in an alerting policy.
Set the notification channel in an alerting policy
To use a Pub/Sub notification channel in an alerting policy, select Pub/Sub as the channel type, and then select the topic.
Schema example
To view an example JSON packet and the schema, expand the following sections.
Deprecation policy
The payload schema is subject to the Google Cloud deprecation policy outlined in Section 1.4(d) of the Google Cloud Platform Terms of Service. Note that the schema does not control the formats of generated field values, and these formats can change without notice. For example,incident.summary
,
incident.documentation.content
, and incident.url
are meant to include data pertaining to their fields, but the schema doesn't
have constraints to guarantee accurate parsing of these fields. You can
consume the value as a whole and expect that it adheres to the deprecation
policy, but don't rely on parsing the generated fields.
Create a channel on demand
When adding a notification channel to an alerting policy, you must select a channel from a list. To update the list of options when you are creating an alerting policy, use the following process:
- In the notification dialog, click Manage Notification Channels. You're taken to the Notification channels window in a new browser tab.
- To add a new notification channel, locate the channel type, click Add new, and then follow the channel-specific instructions contained in the previous table.
Return to the original tab, and in the notification dialog, click refresh Refresh .
Select the notification channel from the updated list.
Test a notification channel
Cloud Monitoring doesn't support a test option for notification channels. However, you can verify that a notification channel is properly configured by creating an alerting policy and configuring the condition to trigger.
For example, you can verify your notification channel configurations by using a procedure similar to the following:
- If you don't have a Compute Engine instance in your Google Cloud project, then create one.
Create an alerting policy to monitor the CPU utilization of that instance.
If your Google Cloud project contains multiple instances, then add a filter to select one instance.
Configure the Condition trigger fields as follows:
- Select Threshold for the Condition type field.
- Select Below threshold for the Threshold position field.
- Select a value that is larger than the CPU utilization for your instance. For example, if the chart displays the CPU utilization to be about 5%, then set the threshold to 10%.
- Expand Advanced Options and select No retest for the Restest window field.
These settings configure the policy to trigger when the CPU utilization of a Compute Engine instance is less than the specified threshold. Because you selected the threshold to be larger than the CPU utilization of your instance, the condition triggers.
Add the notification channels to your policy.
Expand Incident autoclose duration and select 30 min.
Enter a policy name and click Create policy.
After the condition to triggers, verify that you received a notification on each notification channel.
Delete the alerting policy and the Compute Engine instance that you created. The incident is closed automatically after the autoclose duration expires.
If you created a VM for this test, then delete it.
Edit and delete notification channels
To edit or delete a notification channel by using the Google Cloud console, do the following:
In the Google Cloud console, select Monitoring, or click the following button:
In the Monitoring navigation pane, click notifications Alerting.
Click Edit notification channels.
The Notification channels dashboard contains a section for each type of notification channel. Each section lists all configurations for that type:
- To modify an entry, click edit Edit . Click Save after your changes are complete.
- To delete an entry, click delete Delete . Click Delete in the confirmation dialog.
View notification channel logs
You can use the Logs Explorer to view notification channel errors:
To begin using the Google Cloud console to view the errors, navigate to the Logs Explorer:
Enter and run your query. For queries specific to notification channel errors, see Cloud Monitoring queries queries.