Configure notification channels for PA alerts

Important: Throughout this document, replace PROJECT_NAMESPACE with the project namespace where you want to apply changes.

The Observability platform lets you configure notification channels for the alerts you set up. The Observability system sends an alert to these customized channels whenever an event causes an alert. This way, you can notify the recipients you need according to your preferred methods.

Google Distributed Cloud (GDC) air-gapped does not predefine notification channels intended for sending alerts. You must define at least one channel to receive Observability alerts.

Before you begin

To get the permissions you need to configure or view notification channels, ask your Organization IAM Admin to grant you one of the following roles in your project namespace:

• Observability Pipeline Editor: edits or modifies ObservabilityPipeline custom resources. Request the Observability Pipeline Editor (observabilitypipeline-editor) role.
• Observability Pipeline Viewer: views ObservabilityPipeline custom resources. Request the Observability Pipeline Viewer (observabilitypipeline-viewer) role.

If you want to configure notification channels in the platform-obs namespace, the role names are different. Ask your Organization IAM Admin to grant you one of the following roles in the platform-obs namespace:

• Observability Pipeline PA Creator: creates ObservabilityPipeline custom resources. Request the Observability Pipeline PA Creator (observabilitypipeline-pa-creator) cluster role.
• Observability Pipeline PA Editor: edits or modifies ObservabilityPipeline custom resources. Request the Observability Pipeline PA Editor (observabilitypipeline-pa-editor) cluster role.
• Observability Pipeline PA Viewer: views ObservabilityPipeline custom resources. Request the Observability Pipeline PA Viewer (observabilitypipeline-pa-viewer) cluster role.

Additionally, to get the permissions you need to create ConfigMap objects in the project namespace, ask your Organization IAM Admin to grant you the ConfigMap Creator (configmap-creator) role.

For more information about roles and access, see Grant and revoke access.

Configure notification channels

Work through the following steps to configure notification channels for the alerts you set up for system monitoring:

1. Define the Observability alerting configuration following the specification in https://prometheus.io/docs/alerting/latest/configuration/.

2. Create a ConfigMap object and deploy it into your project namespace. The object must contain the configuration rules in the alertmanager.yml file.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     # The name must match the ConfigMap name specified in the ObservabilityPipeline custom resource.
     name: CUSTOMIZED_CONFIGMAP_NAME
     # The namespace must match the project namespace
     namespace: PROJECT_NAMESPACE
   data:
     # The file name must be alertmanager.yml
     alertmanager.yml: |
       # Add the customized configuration in the following lines of this file.
   

   Replace the following:

   • CUSTOMIZED_CONFIGMAP_NAME: the name of your configuration file. You use this name in the ObservabilityPipeline custom resource during the following step.
   • PROJECT_NAMESPACE: the namespace of your project
   Note: Follow the same syntax as described in https://prometheus.io/docs/alerting/latest/configuration to define the notification channels in the alertmanager.yml file.

3. Configure the ObservabilityPipeline custom resource in a command-line editor to import the configuration to the alertmanagerConfig field.

    # Configure observability pipeline
   apiVersion: observability.gdc.goog/v1
   kind: ObservabilityPipeline
   metadata:
     # The namespace must match the project namespace
     namespace: PROJECT_NAMESPACE
     name: observability-config
   spec:
     # Configure alerts
     alerting:
       # The alerting configuration must be in the key "alertmanager.yml" file of the configMap object
       alertmanagerConfig: CUSTOMIZED_CONFIGMAP_NAME
     ...
   

   Replace the following:

   • PROJECT_NAMESPACE: the namespace of your project
   • CUSTOMIZED_CONFIGMAP_NAME: the name you used for your configuration file in the ConfigMap object. The name must match exactly.

4. Save the changes in the ObservabilityPipeline custom resource and exit your command-line editor.

Note: In the alerting field of the ObservabilityPipeline custom resource, you can also define other configurations for the project namespace such as volumes (volumes) and volume mounts (volumeMounts). You reference volume mount names in the alertmanager.yml file.