Route Cloud Storage events to Cloud Run

Standard

An Eventarc trigger declares your interest in a certain event or set of events. You can configure event routing by specifying filters for the trigger, including the event source, and the target Cloud Run service.

Eventarc delivers events to the event receiver in a CloudEvents format through an HTTP request.

These instructions show you how to configure event routing to your Cloud Run service that is triggered by a direct Cloud Storage event. For more details, see the list of supported direct events.

Note: If both direct and audit log events are supported for a Google provider, we recommend that you create a trigger that filters for the direct event. Direct events offer a number of advantages over audit log events. For more information, see Event routes.

Prepare to create a trigger

Before you create a trigger, complete these prerequisites:

Console

1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

2. Enable the Cloud Logging, Eventarc, and Eventarc Publishing APIs.

   Enable the APIs

3. If applicable, enable the API related to the direct events. For example, for Cloud Storage events, enable the Cloud Storage API.

4. If you don't already have one, create a user-managed service account, then grant it the roles and permissions necessary so that Eventarc can manage events for your target service.

   1. In the Google Cloud console, go to the Create service account page.

      Go to Create service account

   2. Select your project.

   3. In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.

      In the Service account description field, enter a description. For example, Service account for event trigger.

   4. Click Create and continue.

   5. To provide appropriate access, in the Select a role list, select the required Identity and Access Management (IAM) roles to grant to your service account for authenticated or unauthenticated invocations. For more information, see Roles and permissions for Cloud Run targets.

      For additional roles, click Add another role and add each additional role.

      Important: To accept authenticated invocations to your Cloud Run service, you must associate your trigger with a service account that has the roles/run.invoker role. When creating or updating your trigger, select that service account.

   6. Click Continue.

   7. To finish creating the account, click Done.

5. Grant the Pub/Sub Publisher role to the Cloud Storage service agent. Typically, this is [e-mail]. You can retrieve the email address for the Cloud Storage service agent.

   1. In the Google Cloud console, go to the IAM page.

      Go to IAM

   2. In the row for the Cloud Storage service agent, click Edit principal. (If the service agent is not listed, proceed to the next step.) The Edit access pane opens.

      1. Click Add another role and then search for the Pub/Sub Publisher role.
      2. Select the role.
      3. Click Save.

   3. If the service agent is not listed, click Grant access. The Grant access pane opens.

      1. In the New principals field, enter the email address for the service agent.
      2. In the Select a role list, search for the Pub/Sub Publisher role.
      3. Select the role.
      4. Click Save.

gcloud

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. Enable the Cloud Logging, Eventarc, and Eventarc Publishing APIs.

   gcloud services enable logging.googleapis.com \
     eventarc.googleapis.com \
     eventarcpublishing.googleapis.com

3. If applicable, enable the API related to the direct events. For example, for Cloud Storage events, enable storage.googleapis.com.

4. If you don't already have one, create a user-managed service account, then grant it the roles and permissions necessary so that Eventarc can manage events for your target service.

   1. Create the service account:

      gcloud iam service-accounts create SERVICE_ACCOUNT_NAME

      Replace SERVICE_ACCOUNT_NAME with the name of the service account. It must be between 6 and 30 characters, and can contain lowercase alphanumeric characters and dashes. After you create a service account, you cannot change its name.

   2. Grant the required Identity and Access Management (IAM) roles or permissions for authenticated or unauthenticated invocations. For more information, see Roles and permissions for Cloud Run targets.

   Important: To accept authenticated invocations to your Cloud Run service, you must associate your trigger with a service account that has the roles/run.invoker role. When creating or editing your trigger, use the --service-account flag to associate that service account with the trigger.

5. Grant the pubsub.publisher role to the Cloud Storage service account:

   SERVICE_ACCOUNT="$(gsutil kms serviceaccount -p PROJECT_ID)"
   
   gcloud projects add-iam-policy-binding PROJECT_ID \
     --member="serviceAccount:${SERVICE_ACCOUNT}" \
     --role="roles/pubsub.publisher"
   

Create a trigger

You can create an Eventarc trigger using the Google Cloud CLI or through the Google Cloud console.

Note: Filtering is done through an exact match and does not support wildcards or regular expressions.

Console

1. In the Google Cloud console, go to the Eventarc Triggers page.

   Go to Triggers

2. Click Create trigger.
3. Type a Trigger name.

   This is the ID of the trigger and it must start with a letter. It can contain up to 63 lowercase letters, numbers, or hyphens.

4. For the Trigger type, select Google sources.
5. In the Event provider list, select Cloud Storage.

   Note that the event provider name used in the associated Google Cloud documentation might not have a prefix of Cloud or Google Cloud. For example, on the console, Memorystore for Redis is referred to as Google Cloud Memorystore for Redis.

6. In the Event type list, from the Direct events, select an event type:
   • google.cloud.storage.object.v1.archived: event is sent when a live version of an object is archived or deleted. This event is only sent for versioning buckets.
   • google.cloud.storage.object.v1.delete: event is sent when an object is permanently deleted. Depending on the object versioning setting for a bucket, this means:
     • For versioning buckets, this is only sent when a version is permanently deleted (but not when an object is archived).
     • For non-versioning buckets, this is sent when an object is deleted or overwritten.
   • google.cloud.storage.object.v1.finalized: event is sent when a new object is created (or an existing object is overwritten, and a new generation of that object is created) in the bucket.
   • google.cloud.storage.object.v1.metadataUpdated: event is sent when the [metadata](/storage/docs/metadata) of an existing object changes.
7. In the Event data content type list, select the encoding of the event payload.

   For direct events from Cloud Storage, this must be application/json.

8. Specify or browse for the globally unique identifier of the Cloud Storage Bucket.

   The Cloud Storage bucket must reside in the same Google Cloud project and region or multi-region as the Eventarc trigger.

9. Select a Region.

   Cloud Storage triggers for Eventarc are available in single-region, dual-region, and multi-region locations]. Note that the Cloud Storage bucket must reside in the same Google Cloud project and region or multi-region as the Eventarc trigger.

   Events are delivered using Pub/Sub notifications from Cloud Storage. Setting up too many notifications registered against the same bucket might exhaust the notification limit for the bucket as indicated by the error Cloud Storage bucket ...: Pub/Sub notification limit reached. The bucket can have up to 10 notification configurations set to trigger for a specific event. See more quotas and limitations in the Cloud Storage quotas and limits page .

10. Select the Service account that will invoke your service or workflow.

    Or, you can create a new service account.

    This specifies the Identity and Access Management (IAM) service account email associated with the trigger and to which you previously granted specific roles required by Eventarc.

11. In the Event destination list, select Cloud Run.
12. Select a service.
    

    This is the name of the service that receives the events for the trigger. The service must be in the same project as the trigger and will receive events as HTTP POST requests sent to its root URL path (/), whenever the event is generated.

13. Optionally, you can specify the Service URL path to send the incoming request to.

    This is the relative path on the destination service to which the events for the trigger should be sent. For example: /, /route, route, route/subroute.

14. Click Create.

After a trigger is created, the event source filters cannot be modified. Instead, create a new trigger and delete the old one. For more information, see Manage triggers.

gcloud

You can create a trigger by running a gcloud eventarc triggers create command along with required and optional flags.

gcloud eventarc triggers create TRIGGER \
    --location=LOCATION \
    --destination-run-service=DESTINATION_RUN_SERVICE \
    --destination-run-region=DESTINATION_RUN_REGION \
    --event-filters="type=EVENT_FILTER_TYPE" \
    --event-filters="bucket=BUCKET" \
    --service-account=SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com

Replace the following:

• TRIGGER: the ID of the trigger or a fully qualified identifier.
• LOCATION: the location of the Eventarc trigger. Alternatively, you can set the eventarc/location property; for example gcloud config set eventarc/location us-central1.

  The Cloud Storage bucket must reside in the same Google Cloud project and region or multi-region as the Eventarc trigger. For more information, see Eventarc locations.

• DESTINATION_RUN_SERVICE: the name of the Cloud Run service that receives the events for the trigger. The service can be in any of the Cloud Run supported locations and does not need to be in the same location as the trigger. However, the service must be in the same project as the trigger and will receive events as HTTP POST requests sent to its root URL path (/), whenever the event is generated.
• DESTINATION_RUN_REGION: (optional) the region in which the destination Cloud Run service can be found. If not specified, it is assumed that the service is in the same region as the trigger.
• EVENT_FILTER_TYPE: the identifier of the Cloud Storage event and can be one of the following:
  • google.cloud.storage.object.v1.finalized: event is sent when a new object is created (or an existing object is overwritten, and a new generation of that object is created) in the bucket
  • google.cloud.storage.object.v1.archived: event is sent when a live version of an object is archived or deleted. This event is only sent for versioning buckets.
  • google.cloud.storage.object.v1.deleted: event is sent when an object is permanently deleted. Depending on the object versioning setting for a bucket this means:
    • For versioning buckets, this is only sent when a version is permanently deleted (but not when an object is archived).
    • For non-versioning buckets, this is sent when an object is deleted or overwritten.
  • google.cloud.storage.object.v1.metadataUpdated: event is sent when the metadata of an existing object changes.
• BUCKET is the globally unique identifier of the Cloud Storage bucket.
• SERVICE_ACCOUNT_NAME: the name of your user-managed service account.
• PROJECT_ID: your Google Cloud project ID.

Notes:

• For direct events from Cloud Storage, the encoding of the event payload is application/json.
• These flags are required:
  • --event-filters="type=EVENT_FILTER_TYPE"
  • --event-filters="bucket=BUCKET"
• Each trigger can have multiple event filters, comma delimited in one --event-filters=[ATTRIBUTE=VALUE,...] flag, or you can repeat the flag to add more filters. Only events that match all the filters are sent to the destination. Wildcards and regular expressions are not supported.
• After a trigger is created, the event filter type can't be changed. For a different event type, you must create a new trigger.
• Events are delivered using Pub/Sub notifications from Cloud Storage. Setting up too many notifications registered against the same bucket might exhaust the notification limit for the bucket as indicated by the error Cloud Storage bucket ...: Pub/Sub notification limit reached. The bucket can have up to 10 notification configurations set to trigger for a specific event. See more quotas and limitations in the Cloud Storage quotas and limits page.
• The --service-account flag is used to specify the Identity and Access Management (IAM) service account email associated with the trigger.
• Optionally, specify a relative path on the destination Cloud Run service to which the events for the trigger should be sent by using the --destination-run-path flag.

Example:

gcloud eventarc triggers create helloworld-trigger \
    --location=us-central1 \
    --destination-run-service=helloworld-events \
    --destination-run-region=us-central1 \
    --event-filters="type=google.cloud.storage.object.v1.finalized" \
    --event-filters="bucket=my-bucket" \
    --service-account=${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com

This command creates a trigger called helloworld-trigger for the Cloud Storage bucket my-bucket and the event identified as google.cloud.storage.object.v1.finalized.

Terraform

You can create a trigger for a Cloud Run destination using Terraform. For details, see Create a trigger using Terraform.

Note: Although your trigger is created immediately, it can take up to two minutes for a trigger to propagate and filter events.

List a trigger

You can confirm the creation of a trigger by listing Eventarc triggers using the Google Cloud CLI or through the Google Cloud console.

Console

1. In the Google Cloud console, go to the Eventarc Triggers page.

   Go to Triggers

   This page lists your triggers in all locations, and includes details such as names, regions, event providers, destinations, and more.

2. To filter your triggers:

   1. Click Filter or the Filter triggers field.
   2. In the Properties list, select an option to filter the triggers by.

   You can select a single property or use the logical operator OR to add more properties.

3. To sort your triggers, beside any supported column heading, click Sort.

gcloud

Run the following command to list your triggers:

gcloud eventarc triggers list --location=-

This command lists your triggers in all locations, and includes details such as names, types, destinations, and statuses.

What's next

• Learn more about Eventarc.
• Learn how to manage triggers.