Configure notification channels for alerts

The Observability platform lets you configure notification channels for the alerts you set up. The system sends an alert to these customized channels whenever an event causes an alert. This way, you can notify the recipients you need according to your preferred methods.

Google Distributed Cloud (GDC) air-gapped appliance does not predefine notification channels intended for sending alerts. You must define at least one channel to receive alerts. If you don't define a notification channel configuration, the GDC Observability system takes the configuration from the platform resources as a default. Consequently, if you don't configure notification channels but an administrator did it for the organization, then the system sends alerts to those channels.

Before you begin

To get the permissions you need to configure or view notification channels, ask your Project IAM Admin to grant you one of the following roles in your project namespace:

• Observability Pipeline Editor: edits or modifies ObservabilityPipeline custom resources. Request the Observability Pipeline Editor (observabilitypipeline-editor) role.
• Observability Pipeline Viewer: views ObservabilityPipeline custom resources. Request the Observability Pipeline Viewer (observabilitypipeline-viewer) role.

Configure notification channels

Work through the following steps to configure notification channels for the alerts you set up for system monitoring:

1. Define the Observability alerting configuration following the specification in https://prometheus.io/docs/alerting/latest/configuration/.

2. Create a ConfigMap object and deploy it into your project namespace. The object must contain the configuration rules in the alertmanager.yml file.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     # The name must match the ConfigMap name specified in the ObservabilityPipeline custom resource.
     name: CUSTOMIZED_CONFIGMAP_NAME
     # Choose namespace that matches the project's namespace
     namespace: PROJECT_NAMESPACE
   data:
     # The file name must be alertmanager.yml.
     alertmanager.yml: |
       # Add the customized Alertmanager configuration here.
   

   Replace the following:

   • CUSTOMIZED_CONFIGMAP_NAME: The name of your configuration file to use it in the ObservabilityPipeline CR during the following step.
   • PROJECT_NAMESPACE: The namespace of your project.
   Note: Follow the same syntax as described in https://prometheus.io/docs/alerting/latest/configuration to define the notification channels in the alertmanager.yml file.

3. Configure the ObservabilityPipeline custom resource in a command-line editor to import the configuration in the alertmanagerConfig field nested in the alerting field.

     # Configure observability pipeline
   apiVersion: observability.gdc.goog/v1
   kind: ObservabilityPipeline
   metadata:
     # Choose namespace that matches the project's namespace
     namespace: PROJECT_NAMESPACE
     name: observability-config
   spec:
     # Configure Alertmanager
     alerting:
       # Permission: PA and AO
       # The alerts configuration must be in the key "alertmanager.yml" of the configMap
       alertmanagerConfig: CUSTOMIZED_CONFIGMAP_NAME
     ...
   

   Replace the following:

   • PROJECT_NAMESPACE: The namespace of your project.
   • CUSTOMIZED_CONFIGMAP_NAME: The name you used for your configuration file in the ConfigMap object. The name must match exactly.

4. Save the changes in the ObservabilityPipeline CR and exit your command-line editor.