Route audit log events to Workflows

An Eventarc trigger declares your interest in a certain event or set of events. You can configure event routing by specifying filters for the trigger, including the event source, and the target workflow.

Note: For the event that you want to route, if both direct and audit log options are supported, we recommend that you create a trigger to route the direct event. Direct events offer a number of advantages over audit log events. For more information, see Event routing options.

Eventarc delivers triggered events from the sources over Cloud Pub/Sub. The delivered events are transformed and passed to Workflows as runtime arguments to execute the workflow. Make sure that the event size does not exceed 512 KB. Events larger than the maximum Workflows arguments size will not trigger workflow executions.

These instructions show you how a new execution of your workflow is triggered when an audit log is created that matches the trigger's filter criteria. This type of event applies to all event providers. For a list of the audit log events supported by Eventarc, including serviceName and methodName values, see Event types supported by Eventarc.

For more information about capturing events that are triggered when an audit log is created that matches the trigger's filter criteria, see Determine event filters for Cloud Audit Logs.

Prepare to create a trigger

Before creating an Eventarc trigger for a target workflow, complete the following tasks.

Console

1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

2. Enable the Eventarc, Eventarc Publishing, Workflows, and Workflow Executions APIs.

   Enable the APIs

3. If applicable, enable the API related to the direct events. For example, for Cloud Functions events, enable cloudfunctions.googleapis.com.

4. If you don't already have one, create a user-managed service account, then grant it the roles and permissions necessary so that Eventarc can manage events for a target workflow.

   1. In the Google Cloud console, go to the Service Accounts page.

      Go to Service Accounts

   2. Select your project and then click Create service account.

   3. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.

      In the Service account description field, enter a description. For example, Service account for event trigger.

   4. Click Create and continue.

   5. To provide appropriate access, in the Select a role list, select the following roles to grant to your service account.

      • Eventarc > Eventarc Event Receiver
      • Workflows > Workflows Invoker

      For additional roles, click Add another role and add each additional role.

      Note: The Role field affects which resources your service account can access in your project. You can revoke these roles or grant additional roles later. In production environments, do not grant the Owner, Editor, or Viewer roles. Instead, grant a predefined role or custom role that meets your needs.

      For more information about how to control access to Workflows resources, see Use IAM to control access.

   6. Click Continue.

   7. To finish creating the account, click Done.

gcloud

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. Enable the Eventarc, Eventarc Publishing, Workflows, and Workflow Executions APIs:

   gcloud services enable eventarc.googleapis.com \
       eventarcpublishing.googleapis.com \
       workflows.googleapis.com \
       workflowexecutions.googleapis.com
   

3. If applicable, enable the API related to the direct events. For example, for Cloud Functions events, enable cloudfunctions.googleapis.com.

4. Grant the Eventarc Admin role (roles/eventarc.admin) to the user:

   gcloud projects add-iam-policy-binding PROJECT_ID \
      --member=PRINCIPAL \
      --role="roles/eventarc.admin"
   

   Replace the following values:

   • PROJECT_ID: the Google Cloud project ID.
   • PRINCIPAL: the principal to add the binding for. Should be of the form user|group|serviceAccount:email or domain:domain.

     Examples: user:[e-mail], group:[e-mail], serviceAccount:[e-mail], or domain:example.domain.com

5. If you don't already have one, create a user-managed service account, then grant it the roles and permissions necessary so that Eventarc can manage events for a target workflow.

   1. Create the service account:

      TRIGGER_SA=MY_SERVICE_ACCOUNT
      gcloud iam service-accounts create ${TRIGGER_SA}

      Replace MY_SERVICE_ACCOUNT with the name of the service account. It must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you cannot change its name.

   2. Grant role(s) to the service account:

      Run the following command once for each of the following IAM roles:

      • roles/eventarc.eventReceiver
      • roles/workflows.invoker

      gcloud projects add-iam-policy-binding PROJECT_ID \
        --member="serviceAccount:${TRIGGER_SA}@PROJECT_ID.iam.gserviceaccount.com" \
        --role="ROLE"

      Replace the following:

      • PROJECT_ID: the project ID where you created the service account
      • ROLE: the role to grant to the service account
      Note: The --role flag affects which resources your service account can access in your project. You can revoke these roles or grant additional roles later. In production environments, do not grant the Owner, Editor, or Viewer roles. Instead, grant a predefined role or custom role that meets your needs.

      For more information about how to control access to Workflows resources, see Use IAM to control access.

Create a trigger

You can create an Eventarc trigger with a deployed workflow as the event receiver by using the Google Cloud CLI (gcloud or Terraform), or through the Google Cloud console.

Note: Filtering is done through an exact match and does not support wildcards or regular expressions. However, you can use the --event-filters-path-pattern flag in the following gcloud CLI command, or define a path pattern through the console. For more information, see Understand path patterns.

Console

1. In the Google Cloud console, go to the Eventarc Triggers page.

   Go to Triggers

2. Click Create trigger.
3. Type a Trigger name.

   This is the ID of the trigger and it must start with a letter. It can contain up to 63 lowercase letters, numbers, or hyphens.

4. For the Trigger type, select Google sources.
5. Select an Event provider.

   This is the Google service that is the source of events through its audit logs. For example, select BigQuery.

6. In the Event list, under via Cloud Audit Logs, select an event.
7. Select one of the following:
   • Any resource—This is the default and includes dynamically created resources that have identifiers generated at creation time.
   • Specific resource—You must provide the full resource name.
   • Path pattern—You can filter for resources using a path pattern. For example, type projects/_/buckets/eventarc-bucket/objects/random.txt or type projects/_/buckets/**/r*.txt.
8. In the Region list, select a region.

   Cloud Audit Logs triggers for Eventarc are available in specific regions and in the global region, but are not available in dual-region and multi-region locations. To avoid any performance and data residency issues caused by a global trigger, Google recommends that the location match that of the Google Cloud service that is generating events. For more information, see Eventarc locations.

   If you specify the global location, you will receive events from all locations yielding matches for the event filters. For example, by creating a global Eventarc trigger, you can receive events from resources in the EU and US multi-regions.

   Note that there is a known issue with Cloud Audit Logs triggers for Compute Engine that results in events originating from a single region: us-central1. This is regardless of where the virtual machine instance is actually located. When creating your trigger, set the trigger location to either us-central1 or global.

9. Select the Service account that will invoke your service or workflow.

   Or, you can create a new service account.

   This specifies the Identity and Access Management (IAM) service account email associated with the trigger and to which you previously granted specific roles required by Eventarc.

10. In the Event destination list, select Workflows.
11. Select a workflow.
    

    This is the name of the workflow to pass events to. Events for a workflow execution are transformed and passed to the workflow as runtime arguments.

    For more information, see Create a trigger for Workflows.

12. Click Create.

After a trigger is created, the event source filters cannot be modified. Instead, create a new trigger and delete the old one. For more information, see Manage triggers.

gcloud

You can create a trigger by running a gcloud eventarc triggers create command along with required and optional flags.

gcloud eventarc triggers create TRIGGER \
    --location=LOCATION \
    --destination-workflow=DESTINATION_WORKFLOW \
    --destination-workflow-location=DESTINATION_WORKFLOW_LOCATION \
    --event-filters="type=google.cloud.audit.log.v1.written" \
    --event-filters="serviceName=SERVICE_NAME" \
    --event-filters="methodName=METHOD_NAME" \
    --service-account="MY_SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"

Replace the following:

• TRIGGER: the ID of the trigger or a fully qualified identifier.
• LOCATION: the location of the Eventarc trigger. Alternatively, you can set the eventarc/location property; for example, gcloud config set eventarc/location us-central1.

  Eventarc is available in specific locations and in the global location, but it is not available in dual-region and multi-region locations. To avoid any performance and data residency issues caused by a global trigger, we recommend that the location match that of the Google Cloud service that is generating events.

  If you specify the global location, you will receive events from all locations for which the event filters match. For example, by creating a global Eventarc trigger, you can receive events from resources such as Cloud Storage buckets in the EU and US multi-regions.

• DESTINATION_WORKFLOW: the ID of the deployed workflow that receives the events from the trigger. The workflow can be in any of the Workflows supported locations and does not need to be in the same location as the trigger. However, the workflow must be in the same project as the trigger.
• DESTINATION_WORKFLOW_LOCATION (optional): the location in which the destination workflow is deployed. If not specified, it is assumed that the workflow is in the same location as the trigger.

• SERVICE_NAME: the identifier of the Google Cloud service
• METHOD_NAME: the identifier of the operation
• MY_SERVICE_ACCOUNT: the name of the IAM service account you created to which you granted specific roles required by Workflows.
• PROJECT_ID: your Google Cloud project ID

Notes:

• These flags are required:
  • --event-filters="type=google.cloud.audit.log.v1.written"
  • --event-filters="serviceName=VALUE"
  • --event-filters="methodName=VALUE"
• After a trigger is created, --event-filters="type=google.cloud.audit.log.v1.written" can't be changed. For a different event type, you must create a new trigger.
• --service-account: The IAM service account email that your Eventarc trigger will use to invoke the workflow executions, and to receive cloud audit logs. We strongly recommend using a service account with the least privileges necessary to access the required resources. To learn more about service accounts, see Create and manage service accounts.
• For a list of the audit log events supported by Eventarc, including serviceName and methodName values, see Events supported by Eventarc.
• Each trigger can have multiple event filters, comma delimited in one --event-filters=[ATTRIBUTE=VALUE,...] flag, or you can repeat the flag to add more filters. Only events that match all the filters are sent to the destination. Wildcards and regular expressions are not supported.
See Determine event filters for Cloud Audit Logs.
• Optionally, you can filter events for a specific resource by using the --event-filters="resourceName=VALUE" flag and specifying the complete path to the resource. Omit the flag for dynamically created resources that have identifiers generated at creation time. Or, you can filter events for a set of resources by using the --event-filters-path-pattern="resourceName=VALUE" flag and specifying the resource path pattern.
• By default, Pub/Sub subscriptions created for Eventarc persist regardless of activity and do not expire. To change the inactivity duration, see Subscription properties.

Example:

gcloud eventarc triggers create helloworld-trigger \
    --location=us-central1 \
    --destination-workflow=my-workflow \
    --destination-workflow-location=europe-west4 \
    --event-filters="type=google.cloud.audit.log.v1.written" \
    --event-filters="serviceName=bigquery.googleapis.com" \
    --event-filters="methodName=jobservice.jobcompleted" \
    --service-account="${TRIGGER_SA}@${PROJECT_ID}.iam.gserviceaccount.com"

This creates a trigger called helloworld-trigger for audit logs that are written by bigquery.googleapis.com and for the operation identified as jobservice.jobcompleted.

Terraform

You can create a trigger for a workflow using Terraform. For details, see Trigger a workflow using Eventarc and Terraform.

Note: Although your trigger is created immediately, it can take up to two minutes for a trigger to propagate and filter events.

List a trigger

You can confirm the creation of a trigger by listing Eventarc triggers using the Google Cloud CLI or through the Google Cloud console.

Console

1. In the Google Cloud console, go to the Eventarc Triggers page.

   Go to Triggers

   This page lists your triggers in all locations, and includes details such as names, regions, event providers, destinations, and more.

2. To filter your triggers:

   1. Click Filter or the Filter triggers field.
   2. In the Properties list, select an option to filter the triggers by.

   You can select a single property or use the logical operator OR to add more properties.

3. To sort your triggers, beside any supported column heading, click Sort.

gcloud

Run the following command to list your triggers:

gcloud eventarc triggers list --location=-

This command lists your triggers in all locations, and includes details such as names, types, destinations, and statuses.

What's next

• Trigger a workflow using Eventarc and Terraform.

• Trigger Workflows with events from Cloud Storage

• Trigger Workflows with Pub/Sub messages

• Learn more about Eventarc