Settings for common alerting policies

To create an alerting policy, you must describe what is to be monitored, when the alerting policy is triggered, and how you want to be notified. This page contains settings that you can use to create alerting policies. Most sections in this page have the following elements:

• Title: Lists the relevant product name and a brief description of the alerting policy.
• Summary: A brief description of the alerting policy. For full information, see the product documentation.
• Steps to create an alerting policy: Outline of the steps required to create an alerting policy. For detailed information on these steps, see Creating an alerting policy.

• Target-pane fields: These fields specify what is being monitored and how the data is aggregated. If you are using the Google Cloud Console, these are the fields in the Target pane of the Add Condition dialog.

• Configuration fields: These fields specify when the alerting policy triggers. By changing the duration window, you can reduce how often a policy triggers. If you are using the Google Cloud Console, these are the fields in the Configuration pane of the Add Condition dialog.

When you only want to configure a chart, use the table with the target-pane fields to help you configure the chart. Alerting conditions, Metrics Explorer and the Legacy dashboard editor use the same terminology. However, the Dashboard editor uses slightly different notation. If you are using the Dashboard editor to configure a chart, then select Advanced mode and make the following substitutions:

Field name in the target pane	Field name in Dashboard editor
Aligner	Alignment function
Aggregator	Group by function

Billing

To be notified if your billable or forecasted charges exceed a budget, create an alert by using the Budgets and alerts page of the Google Cloud Console:

1. Sign in to the Google Cloud Console:
   Sign in to Cloud Console
2. Open the console Navigation menu , and then select Billing. If you have more than one Cloud Billing account, then do one of the following:
   • To manage Cloud Billing for the current project, select Go to linked billing account.
   • To locate a different Cloud Billing account, select Manage billing accounts and choose the account for which you'd like to set a budget.
3. In the Billing navigation menu, select Budgets & alerts.
4. Click Create budget.
5. Complete the budget dialog. In this dialog, you select Google Cloud projects and products, and then you create a budget for that combination. By default, you are notified when you reach 50%, 90%, and 100% of the budget. For complete documentation, see Set budgets and budget alerts.
   Note: In the billing dialog, the Google Cloud's operations suite products are prefixed with "Stackdriver". If you want an alert on Cloud Monitoring usage, then select Stackdriver Monitoring.

BigQuery execution time

To create an alerting policy that triggers when the 99th percentile of the execution time of a BigQuery query exceeds a user-defined limit, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	BigQuery Project (you can also use the Global resource)
Metric	Query execution times
Filter
Group by	priority
Aggregator	99th percentile
Period	5 m
Advanced Aggregation	Aligner: sum

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	You determine this value; however, a threshold of 60 seconds is recommended.
For	most recent value

BigQuery usage

To create an alerting policy that triggers when the ingested BigQuery metrics exceed a user-defined level, do the following:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Enter BigQuery Dataset
Metric	Metrics specific to usage include Stored bytes, Uploaded bytes, and Uploaded bytes billed. However, this in only a partial list. For a full list of available metrics, see BigQuery metrics.
Filter	project_id: Your Google Cloud project ID. dataset_id: Your dataset ID.
Group By	dataset_id: Your dataset ID.
Aggregator	sum
Period	1 m
Advanced Aggregation	Aligner: mean

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	You determine the acceptable value.
For	1 minute

Compute Engine early boot validation

Early Boot Validation shows the pass/fail status of the early boot portion of the last boot sequence. Early boot is the boot sequence from the start of the UEFI firmware until it passes control to the bootloader.

To create an alerting policy that triggers when the early boot sequence fails for any of your Compute Engine VM instances, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	GCE VM Instance
Metric	Early boot validation
Filter	status = failed
Group By	status
Aggregator	sum
Period	Use default.
Advanced Aggregation	Use defaults.

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	0
For	1 minute

Compute Engine late boot validation

Late Boot Validation shows the pass/fail status of the late boot portion of the last boot sequence. Late boot is the boot sequence from the bootloader until completion. This includes the loading of the operating system kernel.

To create an alerting policy that triggers when the late boot sequence fails for any of your Compute Engine VM instances, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	GCE VM Instance
Metric	late boot validation
Filter	status = failed
Group By	status
Aggregator	sum
Period	Use default.
Advanced Aggregation	Use defaults.

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	0
For	1 minute

Logging monthly log bytes ingested

To create an alerting policy that triggers when your monthly log bytes ingested exceeds your user-defined limit for Cloud Logging, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Global
Metric	Monthly log bytes ingested
Filter
Aggregator	sum
Period	60 m
Advanced Aggregation	Aligner: max

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	You determine the acceptable value.
For	Minimum acceptable value is 30 minutes.

Recommendations AI prediction

To set up a Recommendations AI prediction alert, use the following settings in the alerting policy:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Consumed API
Metric	Request count
Filter	service = recommendationengine.googleapis.com method = google.cloud.recommendationengine.v1beta1.PredictionService.Predict response_code != 200
Aggregator	sum
Period	1 m
Advanced Aggregation	Aligner: sum

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	0
For	5 minutes

Recommendations AI user event recording reduction

To set up a Recommendations AI event recording reduction alert, use the following settings in the alerting policy:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Consumed API
Metric	Request count
Filter	service = recommendationengine.googleapis.com method = google.cloud.recommendationengine.v1beta1.PredictionService.CollectUserEvent response_code != 200
Aggregator	sum
Period	1 m
Advanced Aggregation	Aligner: sum

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is absent
Threshold
For	10 minutes

Spanner high priority CPU usage

To create an alerting policy that triggers when your high priority cpu utilization for Cloud Spanner is above a recommended threshold, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type^*	Cloud Spanner Instance
Metric	CPU Utilization by priority
Filter	instance_id = YOUR_INSTANCE_ID priority = high
Aggregator	max
Period	10 m
Advanced Aggregation	Aligner: mean

^* Selecting this resource and metric type is equivalent to entering the following value in the Find resource type and metric pane: spanner.googleapis.com/instance/cpu/utilization_by_priority.

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	45% for multi-region instances; 65% for regional instances.
For	10 minutes

Spanner 24 hour rolling usage

To create an alerting policy that triggers when the 24 hour rolling average of your cpu utilization for Cloud Spanner is above a recommended threshold, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type^*	Cloud Spanner Instance
Metric	Smoothed CPU utilization
Filter	instance_id = YOUR_INSTANCE_ID
Aggregator	sum
Period	10 m
Advanced Aggregation	Aligner: mean

^* Selecting this resource and metric type is equivalent to entering the following value in the Find resource type and metric pane: spanner.googleapis.com/instance/cpu/smoothed_utilization.

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	90%
For	10 minutes

Spanner storage

To create an alerting policy that triggers when your storage for your Cloud Spanner instance is above a recommended threshold, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type^*	Cloud Spanner Instance
Metric	Storage used
Filter	instance_id = YOUR_INSTANCE_ID
Aggregator	sum
Period	10 m
Advanced Aggregation	Aligner: max

^* Selecting this resource and metric type is equivalent to entering the following value in the Find resource type and metric pane: spanner.googleapis.com/instance/storage/used_bytes.

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	Set the threshold to 75% of the maximum storage per node, multiplied by the number of nodes. For the current node limits, see Cloud Spanner Quotas and limits. For a 2 TB limit per node, the recommended threshold is the number of bytes in 1.5 TB: 1649267441664 multiplied by the number of nodes in your instance.
For	10 minutes

Trace over quota on API usage

To create an alerting policy that triggers when your monthly Cloud Trace spans ingested exceeds your quota, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Consumed API
Metric	Request count
Filter	service = cloudtrace.googleapis.com response_code = 429
Aggregator	sum
Period	1 m
Advanced Aggregation	Aligner: sum

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	0
For	1 minute

Trace monitor monthly span-usage

To create an alerting policy that triggers when your monthly Cloud Trace spans ingested exceeds a user-defined limit, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	global
Metric	Monthly trace spans ingested
Filter
Aggregator	sum
Period	60 m
Advanced Aggregation	Aligner: max

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	You determine the acceptable value.
For	Minimum acceptable value is 30 minutes.

Trace export errors

To create an alerting policy that triggers if there are errors exporting Cloud Trace data to BigQuery, use the following settings:

Steps to create an alerting policy.

To create an alerting policy, do the following:

1. In the Google Cloud Console, go to Monitoring or use the following button:
   Go to Monitoring
2. In the Monitoring navigation pane, select Alerting and then select Create Policy.
3. Click Add Condition, and then complete the Target and Configuration sections of the dialog by using the settings in the following tables. Click Add. To monitor another resource, repeat this step.
4. To advance to the notifications section, click Next.
5. (Optional) To add notifications to your alerting policy, click Notification channels. In the dialog, select one or more notification channels from the menu and then click OK.

   If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click Refresh , and then select the notification channels to add to the alerting policy.

6. To advance to the documentation section, click Next.
7. Click Name and enter a name for the alerting policy.
8. (Optional) Click Documentation and add any information that you want included in a notification message.
9. Click Save.

Target pane Field	Value
Resource type	Cloud Trace
Metric	Spans Exported to BigQuery
Filter	status != ok
Group by	status
Aggregator	sum
Period	1 m
Advanced Aggregation	Aligner: rate

Configuration pane Field	Value
Condition triggers if	Any time series violates
Condition	is above
Threshold	0
For	1 minute

Uptime check monitoring

To create an alerting policy for an uptime check, or to create a chart that displays the success or latency status of an uptime check, see Alerting on uptime checks.