Trigger Workflows using Eventarc

This tutorial page shows how to execute a workflow using an authenticated Cloud Run service that receives events using Pub/Sub. Pub/Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.

Here are a few example scenarios you could use to integrate Eventarc and Workflows:

• An organization sends a message to a Pub/Sub topic to indicate the hiring of an employee. This triggers the Cloud Run service to execute a workflow that assigns appropriate access and permissions to that employee.

• A service sends a message to a Pub/Sub topic that it isn't working as expected. This event triggers the Cloud Run service which executes a workflow that includes automated troubleshooting mechanisms.

Objectives

In this tutorial, you will:

1. Use Workflows to deploy a workflow that receives Pub/Sub messages.

2. Deploy an event receiver service to Cloud Run that requires authenticated invocations.

3. Create an Eventarc trigger that connects a Pub/Sub topic to the Cloud Run service.

4. Publish a message to the Pub/Sub topic to generate an event. This executes the workflow and logs the message in the Workflows' log.

The following diagram shows the high-level architecture:

Note: For projects with Domain Restricted Sharing organization policies, use an authenticated Cloud Run target service. For more information about required roles, permissions, and flags for authenticated invocations, see Eventarc roles and permissions.

Costs

This tutorial uses billable components of Google Cloud, including:

• Cloud Build
• Cloud Run
• Container Registry
• Eventarc
• Pub/Sub
• Workflows

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

4. Install and initialize the Cloud SDK.
5. Update gcloud components:

   gcloud components update

6. Log in using your account:

   gcloud auth login

7. Enable the Cloud Run, Eventarc, Pub/Sub, Workflows, Cloud Build APIs:

   gcloud services enable run.googleapis.com eventarc.googleapis.com pubsub.googleapis.com workflows.googleapis.com cloudbuild.googleapis.com

8. Set the configuration variables used in this tutorial:

   export REGION=us-central1
   gcloud config set project PROJECT_ID
   gcloud config set run/region ${REGION}
   gcloud config set run/platform managed
   gcloud config set eventarc/location ${REGION}
   

   Where PROJECT_ID is your Google Cloud project ID.
9. Create a service account for the project:

   gcloud iam service-accounts create sample-service-account \
     --description="A sample service account" \
     --display-name="Sample service account"

   After you create a service account, it can take up to 7 minutes before you can use the service account. If you try to use a service account immediately after you create it, and you receive an error, wait at least 60 seconds and try again.
10. To confirm that sample-service-account has been created, run:

    gcloud iam service-accounts list

    The output should be similar to the following:

    DISPLAY NAME                     EMAIL                                                               DISABLED
    Default compute service account  [e-mail]                False
    Sample service account           sample-service-account@PROJECT_ID.iam.gserviceaccount.com           False

11. Grant the run.invoker role to the service account:

    gcloud projects add-iam-policy-binding PROJECT_ID \
      --member="serviceAccount:SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com" \
      --role="roles/run.invoker"

    Replace the following values:

    • PROJECT_ID: The project ID.
    • SERVICE_ACCOUNT_ID: The service account ID.
12. Grant the following IAM roles to the user:

eventarc.admin

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="user:USER_EMAIL" \
  --role='roles/eventarc.admin'

Replace the following values:

• PROJECT_ID: the Google Cloud project ID.
• USER_EMAIL: the email address for the user.

  Examples: user:[e-mail], group:[e-mail], serviceAccount:[e-mail], or domain:example.domain.com

iam.serviceAccountUser

gcloud iam service-accounts add-iam-policy-binding \
  SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com \
  --member="user:USER_EMAIL" \
  --role="roles/iam.serviceAccountUser"

Replace the following values:

• PROJECT_ID: the Google Cloud project ID.
• SERVICE_ACCOUNT_ID: the service account ID.
• USER_EMAIL: the email address for the user.

  Examples: user:[e-mail], group:[e-mail], serviceAccount:[e-mail], or domain:example.domain.com

Deploy a workflow

Deploy a workflow that gets executed when a Pub/Sub message to a topic triggers a Cloud Run service with an HTTP request.

1. Clone the repository:

   git clone https://github.com/GoogleCloudPlatform/eventarc-samples.git
   

2. Change to the directory that contains the Workflows sample code:

   cd eventarc-samples/eventarc-workflows-integration/eventarc-pubsub

3. Deploy the workflow defined in the eventarc-pubsub/workflow.yaml file:

   export WORKFLOW_NAME=workflow-pubsub
   gcloud workflows deploy ${WORKFLOW_NAME} --source=workflow.yaml --location=${REGION}
   

   This workflow decodes and logs the received Pub/Sub message.

   eventarc-workflows-integration/eventarc-pubsub/workflow.yaml

   View on GitHub

   main:
     params: [args]
     steps:
       - init:
           assign:
             - headers: ${args.headers}
             - body: ${args.body}
       - log1:
           call: sys.log
           args:
               text: Workflows received request
               severity: INFO
       - log2:
           call: sys.log
           args:
               text: ${args}
               severity: INFO
       - pubSubMessageStep:
           call: sys.log
           args:
               text: ${"Decoded Pub/Sub message data is " + text.decode(base64.decode(args.body.message.data))}
               severity: INFO

Deploy an event receiver to Cloud Run

Deploy a Cloud Run service to execute the workflow.

1. Change to the directory that contains the Cloud Run sample code:

   cd trigger-workflow

2. Build the container for the service that will execute the workflow:

   export PROJECT_ID=$(gcloud config get-value project)
   export SERVICE_NAME=trigger-workflow-pubsub
   gcloud builds submit --tag gcr.io/${PROJECT_ID}/${SERVICE_NAME}
   

3. Respond n, "No" to the Allow unauthenticated invocations to trigger-workflow-pubsub (y/N)? prompt.

4. Deploy the container image to Cloud Run:

   gcloud run deploy ${SERVICE_NAME} \
     --image gcr.io/${PROJECT_ID}/${SERVICE_NAME} \
     --region=${REGION} \
     --update-env-vars GOOGLE_CLOUD_PROJECT=${PROJECT_ID},WORKFLOW_REGION=${REGION},WORKFLOW_NAME=${WORKFLOW_NAME}
   

   This Cloud Run service executes the workflow with the HTTP request.

   eventarc-workflows-integration/eventarc-pubsub/trigger-workflow/app.js

   View on GitHub

   console.log(`Workflow path: ${GOOGLE_CLOUD_PROJECT}, ${WORKFLOW_REGION}, ${WORKFLOW_NAME}`);
   const execResponse = await client.createExecution({
     parent: client.workflowPath(GOOGLE_CLOUD_PROJECT, WORKFLOW_REGION, WORKFLOW_NAME),
     execution: {
       argument: JSON.stringify({headers: req.headers, body: req.body})
     }
   });
   console.log(`Execution response: ${JSON.stringify(execResponse)}`);
   
   const execName = execResponse[0].name;
   console.log(`Created execution: ${execName}`);
   
   res.status(200).send(`Created execution: ${execName}`);

Note: Based on the required level of network access to the Cloud Run service, you can also set the ingress flag.

When you see the service URL, the deployment is complete.

Create an Eventarc trigger

When a message is published to the Pub/Sub topic, the event triggers the authenticated Cloud Run service. This event should be triggered by a caller that has a service account with required IAM roles and permissions to use the resource.

1. Create a trigger to listen for Pub/Sub messages:

   New Pub/Sub topic

     gcloud eventarc triggers create ${SERVICE_NAME} \
       --destination-run-service=${SERVICE_NAME} \
       --destination-run-region=${REGION} \
       --location=${REGION} \
       --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
       --service-account=SERVICE_ACCOUNT_USER_EMAIL
   

   Replace SERVICE_ACCOUNT_USER_EMAIL with the email address for the service account.
   This creates a new Pub/Sub topic and a trigger for it called trigger-workflow-pubsub.

   Existing Pub/Sub topic

     gcloud eventarc triggers create ${SERVICE_NAME} \
       --destination-run-service=${SERVICE_NAME} \
       --destination-run-region=${REGION} \
       --location=${REGION} \
       --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
       --transport-topic=projects/PROJECT_ID/topics/TOPIC_ID \
       --service-account=SERVICE_ACCOUNT_USER_EMAIL
   

   Replace the following:

   • PROJECT_ID is your Google Cloud project ID.
   • TOPIC_ID is the ID of the existing Pub/Sub topic.

   • SERVICE_ACCOUNT_USER_EMAIL is the email address for the service account.
     

     This creates a trigger called trigger-workflow-pubsub for the existing Pub/Sub topic.

   Note: By default, Pub/Sub subscriptions created for Eventarc persist regardless of activity and do not expire. To change the inactivity duration, see Managing subscriptions.

2. Confirm the trigger was successfully created:

     gcloud eventarc triggers list --location=us-central1
   

Generate and view an event

Publish a message to a Pub/Sub topic to generate an event and trigger the Cloud Run service. The Cloud Run service then executes the workflow that logs the messages on the Workflows service logs.

1. Find and set the Pub/Sub topic created, as an environment variable:

   export TOPIC_ID=$(basename $(gcloud eventarc triggers describe ${SERVICE_NAME} --format='value(transport.pubsub.topic)'))
   

2. Send a message to the Pub/Sub topic to trigger the workflow:

   gcloud pubsub topics publish $TOPIC_ID --message "Hello there"
   

   The generated event is logged in the Workflows service logs.

3. To view the event message, go to the Workflows service logs:

   1. Open the Workflows page in the Google Cloud Console.

      Go to Workflows

   2. Click the workflow-pubsub workflow.

   3. Select the Logs tab.

      Logs might take a few moments to appear. If you don't see them immediately, check again after a few moments.

4. Look for the "Hello there!" message.

Clean up

If you created a new project for this tutorial, delete the project. If you used an existing project and wish to keep it without the changes added in this tutorial, delete resources created for the tutorial.

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Cloud Console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete tutorial resources

1. Delete the Cloud Run service you deployed in this tutorial:

   gcloud run services delete SERVICE-NAME

   Where SERVICE-NAME is your chosen service name.

   You can also delete Cloud Run services from the Google Cloud Console.

2. Remove the gcloud default configurations you added during tutorial setup.

   Remove the region setting:

    gcloud config unset run/region
   

3. Remove the project configuration:

    gcloud config unset project
   

4. Delete other Google Cloud resources created in this tutorial:

   • Delete the trigger:

     gcloud eventarc triggers delete TRIGGER_NAME
     

     Replace TRIGGER_NAME with the name of your trigger.
   • Delete the container image named gcr.io/PROJECT_ID/events-pubsub from Container Registry.

What's next

• Receive a Pub/Sub event using an unauthenticated Cloud Run service
• Build containers
• Create an Eventarc trigger
• Develop Cloud Run services
• Learn more about Workflows