Subscribe to events from Datadog

Datadog supports sending events to Eventarc for delivery to interested subscribers.

Datadog is an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services, through a SaaS-based data analytics platform. Learn more about using Datadog.
Datadog logo

To subscribe to events from Datadog, complete the following steps:

Before you begin

  1. Ensure that you have enabled the Eventarc and Eventarc Publishing APIs:

    Console

    Enable the APIs

    gcloud

    gcloud services enable \
      eventarc.googleapis.com \
      eventarcpublishing.googleapis.com
    
  2. If you are enabling a customer-managed encryption key (CMEK) for a channel, enable the Cloud Key Management Service API.

    Console

    Enable the API

    gcloud

    gcloud services enable cloudkms.googleapis.com
    

Retrieve the provider name used by Datadog

The provider name is required when creating and using a channel. You can list all Eventarc providers and retrieve their details, including supported event types, by running gcloud commands.

For example, you can list all third-party providers in any location:

gcloud eventarc providers list --filter='eventTypes.type!~^google*' --location=-

The output should include something similar to the following:

NAME: datadog
LOCATION: us-central1

For details, see List event providers.

Retrieve the events supported by Datadog

An event type is needed when creating an Eventarc trigger. You can list Datadog's supported event types, by running a gcloud command:

gcloud eventarc providers describe datadog \
  --location=LOCATION

Replace LOCATION with a supported Eventarc region.

For example:

gcloud eventarc providers describe datadog --location=us-central1

The output should resemble the following:

displayName: Datadog
eventTypes:
- type: datadog.v1.alert
name: projects/project-id/locations/us-central1/providers/datadog
[...]

Make note of the event type to use when creating an Eventarc trigger.

Create a channel for Datadog events

To subscribe to events from Datadog, you must create a channel:

Console

  1. In the console, go to the Eventarc > Channels page.

    Go to Eventarc

  2. Click Create channel.

  3. In the Provider list, select a provider.

  4. In the Channel name field, enter a name for the channel.

  5. In the Region list, select a supported Eventarc region.

  6. Optionally, select the Use a customer-managed encryption key (CMEK) checkbox and, in the CMEK encryption key list, select a key ring that you have created for the region.

    For more information, see Use customer-managed encryption keys.

  7. Click Create.

gcloud

gcloud eventarc channels create CHANNEL_ID \
  --provider datadog \
  --project PROJECT_ID \
  --location LOCATION

Replace the following:

  • CHANNEL_ID: a name for the channel
  • PROJECT_ID: your Google Cloud project ID

    You can find your project ID on the Dashboard page of the Google Cloud console.

  • LOCATION: a supported Eventarc region

You can optionally use a --crypto-key flag to specify a fully qualified Cloud KMS key name in the format projects/PROJECT_NAME/locations/LOCATION/keyRings/RING_NAME/cryptoKeys/KEY_NAME.

For more information, see Use customer-managed encryption keys.

For details, see Create a channel.

Retrieve specific details to convey to Datadog

After creating a channel, you can retrieve specific details to convey to Datadog:

Console

  1. In the console, go to the Eventarc > Channels page.

    Go to Eventarc

  2. From the list of channels, click a channel to view its details.

    The channel status should be Pending.

  3. Copy the Channel full name and Activation token value to convey to Datadog.

gcloud

gcloud eventarc channels describe CHANNEL_ID \
  --project PROJECT_ID \
  --location LOCATION

The output is similar to the following:

activationToken: aS7dXs1b79AcXsf
createTime: '2021-11-15T15:20:31.582356065Z'
name: projects/project-id/locations/us-central1/channels/datadog-channel
provider: projects/project-id/locations/us-central1/providers/datadog
pubsubTopic: projects/project-id/topics/eventarc-channel-us-central1-datadog-channel-077
state: PENDING
uid: c6703a91-ccd3-4c32-a729-967393f23a29
updateTime: '2021-11-15T15:21:03.689597653Z'

Note the activationToken value and the channel name.

Learn more about the channel state.

Send the channel details to Datadog

Before you can start consuming events, Datadog must initialize the connection between their ecosystem and the channel.

  1. Log into Datadog at https://www.datadoghq.com.
  2. In the navigation menu, select Integrations.
  3. From the list of integrations, select Google Eventarc.

    A green checkmark confirms the installation:

    Installed
  4. Click Configuration.
  5. In the Add New Channels section, click Add New.
  6. Enter the following channel details:
    • Channel full name: the channel ID, including the Google Cloud project ID and location of the resource.

      For example:

      projects/PROJECT_ID/locations/LOCATION/channels/CHANNEL_ID
    • Activation token: the token is valid for 24 hours after the channel's creation. After 24 hours, the channel becomes INACTIVE.

      For example: aS7dXs1b79AcXsf

  7. To save the configuration, click the checkmark Save.

    The new channel is now listed.

Once Datadog receives the required information, and has initialized the connection between their ecosystem and the channel, the state of the channel changes from PENDING to ACTIVE. Datadog can now send events to your project and you can start consuming the events.

Deploy an event receiver

To test the configuration, deploy an event receiver—in this case, a Workflows workflow that logs received events—and create an Eventarc trigger that routes events from Datadog to the event receiver. You must use a service account that has been granted the appropriate roles to access your Google Cloud project resources.

  1. Enable the Workflows and Workflow Executions APIs:

    Console

    Enable the APIs

    gcloud

    gcloud services enable \
      workflows.googleapis.com \
      workflowexecutions.googleapis.com
    
  2. Create a service account and grant it the roles/logging.logWriter and roles/workflows.invoker roles:

    Console

    1. In the console, go to the Create service account page.

      Go to Create service account

    2. Select your project.

    3. In the Service account name field, enter a name. The console fills in the Service account ID field based on this name. Optionally, add a description.

    4. Click Create and continue.

    5. In the Select a role list, select Logging > Logs Writer.

    6. Click Add another role and select Workflows > Workflows Invoker.

    7. Click Continue.

    8. To finish creating the service account, click Done.

    gcloud

    gcloud iam service-accounts create MY_SERVICE_ACCOUNT
    gcloud projects add-iam-policy-binding PROJECT_ID \
      --role="roles/workflows.invoker" \
      --member="serviceAccount:MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
    gcloud projects add-iam-policy-binding PROJECT_ID \
      --role="roles/logging.logWriter" \
      --member="serviceAccount:MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
    

    Replace MY_SERVICE_ACCOUNT with a name for your service account.

    For more information about how to control access to Workflows resources, see Use IAM to control access.

  3. Create and deploy a workflow that logs a received event:

    Console

    1. In the console, go to the Workflows page.

      Go to Workflows

    2. Click Create.

    3. Name your workflow logEventsWorkflow.

    4. Choose a supported Workflows region; for example, us-central1.

    5. Select the service account you previously created.

    6. Click Next.

    7. In the workflow editor, enter the following definition for your workflow:

      main:
        params: [event]
        steps:
        - logStep:
            call: sys.log
            args:
              data: ${event}
      
    8. Click Deploy.

    gcloud

    1. Open a terminal or Cloud Shell.
    2. In your home directory, create a new file called logEventsWorkflow.yaml.
    3. Copy and paste the following into the new file and save it:

      main:
        params: [event]
        steps:
        - logStep:
            call: sys.log
            args:
              data: ${event}
      
    4. Deploy the workflow:

      gcloud workflows deploy logEventsWorkflow \
        --source=logEventsWorkflow.yaml \
        --location LOCATION
      

      Replace LOCATION with a supported Workflows region; for example, us-central1.

  4. Create an Eventarc trigger to route events from Datadog to the target workflow:

    Console

    1. In the console, go to the Eventarc page.

      Go to Eventarc

    2. Click Create trigger.

    3. Type a Trigger name.

      This is the ID of the trigger and it must start with a letter. For example, my-event-channel-trigger.

    4. Select an Event provider.

      This is the partner service that is the source of events. For example, select Partner sources > Datadog.

    5. Select the appropriate Channel.

    6. Select an Event.

      For example: datadog.v1.alert

    7. Optionally, in the Filters section, specify an appropriate attribute and value.

    8. Select the Service account you previously created to invoke your workflow.

    9. For the Event destination, select Workflows, and then select the workflow you previously created.

    10. Click Create.

    gcloud

    gcloud eventarc triggers create TRIGGER \
      --location=LOCATION \
      --destination-workflow=logEventsWorkflow  \
      --event-filters=type=datadog.v1.alert \
      --channel=CHANNEL_NAME \
      --project=PROJECT_ID \
      --service-account="MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
    

    Replace TRIGGER with the ID of the trigger or a fully qualified identifier. For example, my-event-channel-trigger.

    For more information, see Create a trigger for a third-party provider.

Test the routing of events

You can now test the entire configuration and ensure that an event from Datadog is routed to the workflow event receiver which logs the received event.

  1. To create a Datadog monitor, log into Datadog:

    1. In the navigation menu, select Monitors > New Monitor.
    2. From the list of Custom monitor types, select Metric.
    3. Accept the following default settings:
      1. Choose the detection method
      2. Define the metric
    4. Set alert conditions: for the Alert threshold, enter 1.
    5. Notify your team: give your monitor a name and, from the Notify your services and your team members list, select the appropriate value. The message should be similar to the following:

      @eventarc-PROJECT_ID_LOCATION_CHANNEL_ID

    6. Define permissions and audit notifications: select the appropriate values.

    7. Click Test Notifications.

    8. In the Test notifications for this monitor dialog, click Run Test.

      This triggers an execution of the workflow.

    For more information about configuring a Datadog monitor, see the Datadog documentation.

  2. To verify that a workflow execution was triggered, list the last five executions:

    gcloud workflows executions list ${MY_WORKFLOW} --limit=5
    

    The output should be similar to the following, listing a NAME and STATE equal to SUCCEEDED for each workflow execution.

    NAME: projects/1051295516635/locations/us-central1/workflows/logEventsWorkflow/executions/674b5783-deec-4d1b-be1d-b067f7b32971
    STATE: SUCCEEDED
    START_TIME: 2022-07-13T22:01:06.314980819Z
    END_TIME: 2022-07-13T22:01:06.637825944Z
    
  3. Every time a workflow executes a single run of the logic contained in a workflow's definition, it automatically triggers at least two execution logs: one at the start of an execution and one at the end. To view the logs do the following:

    1. In the console, go to the Workflows page:
      Go to Workflows

    2. To access a workflow's Details page, click the workflow's name.

    3. To view the logs, click Logs. The output should be similar to the following:

    {
      "alert_type":"error",
      "data":{
          "message": "@eventarc-test-project_us-central1_datadog-channel\nTest notification"
      },
      "datacontenttype":"application/json",
      "id":6601612988428280602,
      "monitor":"monitor",
      "monitor_id": "1234567",
      "msg_title":[
          "Triggered on"
      ][
          "TEST"
      ]"Test monitor for Eventarc",
      "priority":"None",
      "source": "//datadog/alert",
      "specversion":1.0,
      "time":"2022-07-13T22":"01":03Z,
      "type":datadog.v1.alert
    }
    

What's next