Create triggers from Cloud Storage events

In Cloud Run, you can trigger services from Cloud Storage using Eventarc, in response to changes in Cloud Storage.

When you specify a Cloud Storage trigger for your service, you choose an event type and specify a Cloud Storage bucket. As a result of this trigger, your service is called whenever a change occurs on an object (file) within the specified bucket.

For your service to be triggered by an event on a Cloud Storage bucket, the service and the bucket must be in the same Google Cloud project.

For detailed instructions and code samples for creating triggers from Cloud Storage events, refer to the following tutorials:

Supported Cloud Storage event types

Cloud Run supports the following Cloud Storage event types:

Event Event type Description
Object finalized
  • google.cloud.storage.object.v1.finalized (through Eventarc)
 Occurs when a you create a new object, or overwrite an existing object, and Cloud Storage creates a new generation of that object.
Object deleted
  • google.cloud.storage.object.v1.deleted (through Eventarc)
 Occurs you permanently delete an object.
Object archived
  • google.cloud.storage.object.v1.archived (through Eventarc)
 Occurs when a live version of an object becomes a noncurrent version. See Object versioning for more information.
Object metadata updated
  • google.cloud.storage.object.v1.metadataUpdated (through Eventarc)
 Occurs when you change the metadata of an existing object.

The Google Events repository contains additional resources for working with event data.

Before you begin

  1. Make sure you have set up a new project for Cloud Run as described in the setup page.

  2. Enable the Artifact Registry, Cloud Build, Cloud Run Admin API, Eventarc, Cloud Logging, Pub/Sub, and Cloud Storage APIs:

    Enable the APIs

Set required roles

You or your administrator must grant the deployer account, the trigger identity, and optionally, the Pub/Sub service agent the following IAM roles.

Required roles for the deployer account

To get the permissions that you need to trigger from Cloud Storage events, ask your administrator to grant you the following IAM roles on your project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Note that by default, Cloud Build permissions include permissions to upload and download Artifact Registry artifacts.

Required roles for the trigger identity

  1. Make note of the Compute Engine default service account as you will you attach it to an Eventarc trigger to represent the identity of the trigger for testing purposes. This service account is automatically created after enabling or using a Google Cloud service that uses Compute Engine, and with the following email format:

    PROJECT_NUMBER-compute@developer.gserviceaccount.com

    Replace PROJECT_NUMBER with your Google Cloud project number. You can find your project number on the Welcome page of the Google Cloud console or by running the following command:

    gcloud projects describe PROJECT_ID --format='value(projectNumber)'

    For production environments, we strongly recommend creating a new service account and granting it one or more IAM roles that contain the minimum permissions required and follow the principle of least privilege.

  2. By default, Cloud Run services are only callable by Project Owners, Project Editors, and Cloud Run Admins and Invokers. You can control access on a per-service basis; however, for testing purposes, grant the Cloud Run Invoker role (run.invoker) on the Google Cloud project to the Compute Engine service account. This grants the role on all Cloud Run services and jobs in a project. 
    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \
    --role=roles/run.invoker

    Note that if you create a trigger for an authenticated Cloud Run service without granting the Cloud Run Invoker role, the trigger is created successfully and is active. However, the trigger will not work as expected and a message similar to the following appears in the logs:

    The request was not authenticated. Either allow unauthenticated invocations or set the proper Authorization header.
  3. Grant the Eventarc Event Receiver role (roles/eventarc.eventReceiver) on the project to the Compute Engine default service account so that the Eventarc trigger can receive events from event providers. 
    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:PROJECT_NUMBER-compute@developer.gserviceaccount.com \
    --role=roles/eventarc.eventReceiver

Required role for the Pub/Sub service agent

  • If you enabled the Cloud Pub/Sub service agent on or before April 8, 2021, to support authenticated Pub/Sub push requests, grant the Service Account Token Creator role (roles/iam.serviceAccountTokenCreator) to the service agent. Otherwise, this role is granted by default: 
    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \
    --role=roles/iam.serviceAccountTokenCreator

Create a trigger for services

You can specify a trigger after you deploy a service.

Click the tab for instructions using the tool of your choice.

Console

  1. Deploy your Cloud Run service using containers or from source.

  2. In the Google Cloud console, go to Cloud Run:

    Go to Cloud Run

  3. From the list of services, click an existing service.

  4. On the Service details page, navigate to the Triggers tab.

  5. Click Add trigger, and select Cloud Storage trigger.

  6. In the Eventarc trigger pane, modify the trigger details as follows:

    1. In the Trigger name field, enter a name for the trigger, or use the default name.

    2. Select a Trigger type from the list to specify one of the following trigger types:

      • Google Sources to specify triggers for Pub/Sub, Cloud Storage, Firestore, and other Google event providers.

      • Third-party to integrate with non-Google providers that offer an Eventarc source. For more information, see Third-party events in Eventarc.

    3. Select Cloud Storage from the Event provider list, to select a product that provides the type of event for triggering your service. For the list of event providers, see Event providers and destinations.

    4. Select google.cloud.storage.object.v1.finalized from the Event type list. Your trigger configuration varies depending on the supported event type. For more information, see Event types.

    5. In the Bucket field, click Browse to select a Cloud Storage bucket for the trigger to monitor. Changes to objects within this bucket will trigger calls to your function.

    6. If the Region field is enabled, select a location for the Eventarc trigger. In general, the location of an Eventarc trigger should match the location of the Google Cloud resource that you want to monitor for events. In most scenarios, you should also deploy your service in the same region. See Understand Eventarc locations for more details about Eventarc trigger locations.

    7. In the Service account field, select a service account. Eventarc triggers are linked to service accounts to use as an identity when invoking your service. Your Eventarc trigger's service account must have the permission to invoke your service. By default, Cloud Run uses the Compute Engine default service account.

    8. Optionally, specify the Service URL path to send the incoming request to. This is the relative path on the destination service to which the events for the trigger should be sent. For example: /, /route, route, and route/subroute.

    9. Once you've completed the required fields, click Save trigger.

  7. After creating the trigger, verify its health by ensuring that there is a checkmark on the Triggers tab.

gcloud

  1. Deploy your Cloud Run service using containers or from source.

  2. Run the following command to create a trigger that filters events:

    gcloud eventarc triggers create TRIGGER_NAME  \
    --location=EVENTARC_TRIGGER_LOCATION \
    --destination-run-service=SERVICE  \
    --destination-run-region=REGION \
    --event-filters="type=google.cloud.storage.object.v1.finalized" \
    --event-filters="bucket=PROJECT_ID-bucket" \
    --service-account=PROJECT_NUMBER-compute@developer.gserviceaccount.com

    Replace:

    • TRIGGER_NAME with the name for your trigger.

    • EVENTARC_TRIGGER_LOCATION with the location for the Eventarc trigger. In general, the location of an Eventarc trigger should match the location of the Google Cloud resource that you want to monitor for events. In most scenarios, you should also deploy your service in the same region. For more information, see Eventarc locations.

    • SERVICE with the name of the service you are deploying.

    • REGION with the Cloud Run region of the service.

    • PROJECT_NUMBER with your Google Cloud project number. Eventarc triggers are linked to service accounts to use as an identity when invoking your service. Your Eventarc trigger's service account must have the permission to invoke your service. By default, Cloud Run uses the Default compute service account.

    • The event-filters flag specifies the event filters that the trigger monitors. An event that matches all the event-filters, filters triggers calls to your service. Each trigger must have a supported event type. You can't change the event filter type after creation. To change the event filter type, you must create a new trigger and delete the old one. Optionally, you can repeat the --event-filters flag with a supported filter in the form ATTRIBUTE=VALUE to add more filters.

Terraform

To create an Eventarc trigger for a Cloud Run service, see Create a trigger using Terraform.

Create a trigger for functions

Click the tab for instructions using the tool of your choice.

Console

When you use the Google Cloud console to create a function, you can also add a trigger to your function. Follow these steps to create a trigger for your function:

  1. In the Google Cloud console, go to Cloud Run:

    Go to Cloud Run

  2. Click Write a function, and enter the function details. For more information about configuring functions during deployment, see Deploy functions.

  3. In the Trigger section, click Add trigger.

  4. Select Cloud Storage trigger.

  5. In the Eventarc trigger pane, modify the trigger details as follows:

    1. Enter a name for the trigger in the Trigger name field, or use the default name.

    2. Select a Trigger type from the list to specify one of the following trigger types:

      • Google Sources to specify triggers for Pub/Sub, Cloud Storage, Firestore, and other Google event providers.

      • Third-party to integrate with non-Google providers that offer an Eventarc source. For more information, see Third-party events in Eventarc.

    3. Select Cloud Storage from the Event provider list, to select a product that provides the type of event for triggering your function. For the list of event providers, see Event providers and destinations.

    4. Select google.cloud.storage.object.v1.finalized from the Event type list. Your trigger configuration varies depending on the supported event type. For more information, see Event types.

    5. In the Bucket field, click Browse to select a Cloud Storage bucket for the trigger to monitor. Changes to objects within this bucket will trigger calls to your function.

    6. If the Region field is enabled, select a location for the Eventarc trigger. In general, the location of an Eventarc trigger should match the location of the Google Cloud resource that you want to monitor for events. In most scenarios, you should also deploy your function in the same region. See Understand Eventarc locations for more details about Eventarc trigger locations.

    7. In the Service account field, select a service account. Eventarc triggers are linked to service accounts to use as an identity when invoking your function. Your Eventarc trigger's service account must have the permission to invoke your function. By default, Cloud Run uses the Compute Engine default service account.

    8. Optionally, specify the Service URL path to send the incoming request to. This is the relative path on the destination service to which the events for the trigger should be sent. For example: /, /route, route, and route/subroute.

  6. Once you've completed the required fields, click Save trigger.

gcloud

When you create a function using the gcloud CLI, you must first deploy your function, and then create a trigger. Follow these steps to create a trigger for your function:

  1. Run the following command in the directory that contains the sample code to deploy your function:

    gcloud run deploy FUNCTION \
        --source . \
        --function FUNCTION_ENTRYPOINT \
        --base-image BASE_IMAGE_ID \
        --region REGION

    Replace:

    • FUNCTION with the name of the function you are deploying. You can omit this parameter entirely, but you will be prompted for the name if you omit it.

    • FUNCTION_ENTRYPOINT with the entry point to your function in your source code. This is the code Cloud Run executes when your function runs. The value of this flag must be a function name or fully-qualified class name that exists in your source code.

    • BASE_IMAGE_ID with the base image environment for your function. For more details about base images and the packages included in each image, see Runtimes base images.

    • REGION with the Google Cloud region where you want to deploy your function. For example, europe-west1.

  2. Run the following command to create a trigger that filters events:

    gcloud eventarc triggers create TRIGGER_NAME  \
    --location=EVENTARC_TRIGGER_LOCATION \
    --destination-run-service=FUNCTION  \
    --destination-run-region=REGION \
    --event-filters="type=google.cloud.storage.object.v1.finalized" \
    --event-filters="bucket=PROJECT_ID-bucket" \
    --service-account=PROJECT_NUMBER-compute@developer.gserviceaccount.com

    Replace:

    • TRIGGER_NAME with the name for your trigger.

    • EVENTARC_TRIGGER_LOCATION with the location for the Eventarc trigger. In general, the location of an Eventarc trigger should match the location of the Google Cloud resource that you want to monitor for events. In most scenarios, you should also deploy your function in the same region. For more information, see Eventarc locations.

    • FUNCTION with the name of the function you are deploying.

    • REGION with the Cloud Run region of the function.

    • PROJECT_NUMBER with your Google Cloud project number. Eventarc triggers are linked to service accounts to use as an identity when invoking your function. Your Eventarc trigger's service account must have the permission to invoke your function. By default, Cloud Run uses the Default compute service account.

    • The event-filters flag specifies the event filters that the trigger monitors. An event that matches all the event-filters, filters triggers calls to your function. Each trigger must have a supported event type. You can't change the event filter type after creation. To change the event filter type, you must create a new trigger and delete the old one. Optionally, you can repeat the --event-filters flag with a supported filter in the form ATTRIBUTE=VALUE to add more filters.

Terraform

To create an Eventarc trigger for a Cloud Run function, see Create a trigger using Terraform.

Event delivery

Cloud Storage triggers are implemented with Pub/Sub notifications for Cloud Storage. Events are subject to Pub/Sub notification delivery guarantees.

A Cloud Storage bucket can have up to 10 notification configurations set to trigger for a specific event. Exceeding the bucket's notifications limits will cause further function deployments to fail with an error like the following:

Cloud Storage bucket ...: Pub/Sub notification limit reached

You can work around this limitation by managing your own Pub/Sub topic and registering it for Cloud Storage notifications of the type you want your function to respond to. For example, you could configure a single notification for your chosen event (such as object creation) to a Pub/Sub topic. Then, have as many function subscribers as needed to the topic process the event and fan it out to multiple destinations. For example:

  1. Create your own Pub/Sub topic.
  2. Create a single notification configuration on your Cloud Storage bucket that triggers on a specific event (such as OBJECT_FINALIZE for object creation) and sends the event to your single Pub/Sub topic.
  3. Create a function (or functions) triggered by your Pub/Sub topic.

This results in one Cloud Storage notification configuration, one Pub/Sub topic, and as many functions as you need (within the very high delivery limits allowed by Pub/Sub).

See Cloud Storage Quotas and limits to learn more about Cloud Storage limitations.

Next steps