Trigger Workflows using Eventarc

This tutorial page shows how to execute a workflow using an authenticated Cloud Run service that receives events using Pub/Sub. Pub/Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.

Here are a few example scenarios you could use to integrate Eventarc and Workflows:

  • An organization sends a message to a Pub/Sub topic to indicate the hiring of an employee. This triggers the Cloud Run service to execute a workflow that assigns appropriate access and permissions to that employee.

  • A service sends a message to a Pub/Sub topic that it isn't working as expected. This event triggers the Cloud Run service which executes a workflow that includes automated troubleshooting mechanisms.

Objectives

In this tutorial, you will:

  1. Use Workflows to deploy a workflow that receives Pub/Sub messages.

  2. Deploy an event receiver service to Cloud Run that requires authenticated invocations.

  3. Create an Eventarc trigger that connects a Pub/Sub topic to the Cloud Run service.

  4. Publish a message to the Pub/Sub topic to generate an event. This executes the workflow and logs the message in the Workflows' log.

The following diagram shows the high-level architecture:

Eventarc-Workflows integration

Costs

This tutorial uses billable components of Google Cloud, including:

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Google Cloud users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Install and initialize the Cloud SDK.
  5. Update gcloud components:
    gcloud components update
  6. Log in using your account:
    gcloud auth login
  7. Enable the Cloud Run, Eventarc, Pub/Sub, Workflows, Cloud Build APIs:
    gcloud services enable run.googleapis.com eventarc.googleapis.com pubsub.googleapis.com workflows.googleapis.com cloudbuild.googleapis.com
  8. Set the configuration variables used in this tutorial:
    export REGION=us-central1
    gcloud config set project PROJECT_ID
    gcloud config set run/region ${REGION}
    gcloud config set run/platform managed
    gcloud config set eventarc/location ${REGION}
    
    Where PROJECT_ID is your Google Cloud project ID.
  9. Create a service account for the project:
    gcloud iam service-accounts create sample-service-account \
      --description="A sample service account" \
      --display-name="Sample service account"
    After you create a service account, it can take up to 7 minutes before you can use the service account. If you try to use a service account immediately after you create it, and you receive an error, wait at least 60 seconds and try again.
  10. To confirm that sample-service-account has been created, run:
    gcloud iam service-accounts list
    The output should be similar to the following:
    DISPLAY NAME                     EMAIL                                                               DISABLED
    Default compute service account  PROJECT_NUMBER-compute@developer.gserviceaccount.com                False
    Sample service account           sample-service-account@PROJECT_ID.iam.gserviceaccount.com           False
  11. Grant the run.invoker role to the service account:
    gcloud projects add-iam-policy-binding PROJECT_ID \
      --member="serviceAccount:SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com" \
      --role="roles/run.invoker"

    Replace the following values:

    • PROJECT_ID: The project ID.
    • SERVICE_ACCOUNT_ID: The service account ID.
  12. Grant the following IAM roles to the user:
    • eventarc.admin

      gcloud projects add-iam-policy-binding PROJECT_ID \
        --member="user:USER_EMAIL" \
        --role='roles/eventarc.admin'

      Replace the following values:

      • PROJECT_ID: the Google Cloud project ID.
      • USER_EMAIL: the email address for the user.

        Examples: user:test-user@gmail.com, group:admins@example.com, serviceAccount:test123@example.domain.com, or domain:example.domain.com

      iam.serviceAccountUser

      gcloud iam service-accounts add-iam-policy-binding \
        SERVICE_ACCOUNT_ID@PROJECT_ID.iam.gserviceaccount.com \
        --member="user:USER_EMAIL" \
        --role="roles/iam.serviceAccountUser"

      Replace the following values:

      • PROJECT_ID: the Google Cloud project ID.
      • SERVICE_ACCOUNT_ID: the service account ID.
      • USER_EMAIL: the email address for the user.

        Examples: user:test-user@gmail.com, group:admins@example.com, serviceAccount:test123@example.domain.com, or domain:example.domain.com

Deploy a workflow

Deploy a workflow that gets executed when a Pub/Sub message to a topic triggers a Cloud Run service with an HTTP request.

  1. Clone the repository:

    git clone https://github.com/GoogleCloudPlatform/eventarc-samples.git
    
  2. Change to the directory that contains the Workflows sample code:

    cd eventarc-samples/eventarc-workflows-integration/eventarc-pubsub
  3. Deploy the workflow defined in the eventarc-pubsub/workflow.yaml file:

    export WORKFLOW_NAME=workflow-pubsub
    gcloud workflows deploy ${WORKFLOW_NAME} --source=workflow.yaml --location=${REGION}
    

    This workflow decodes and logs the received Pub/Sub message.

    main:
      params: [args]
      steps:
        - init:
            assign:
              - headers: ${args.headers}
              - body: ${args.body}
        - log1:
            call: sys.log
            args:
                text: Workflows received request
                severity: INFO
        - log2:
            call: sys.log
            args:
                text: ${args}
                severity: INFO
        - pubSubMessageStep:
            call: sys.log
            args:
                text: ${"Decoded Pub/Sub message data is " + text.decode(base64.decode(args.body.message.data))}
                severity: INFO

Deploy an event receiver to Cloud Run

Deploy a Cloud Run service to execute the workflow.

  1. Change to the directory that contains the Cloud Run sample code:

    cd trigger-workflow
  2. Build the container for the service that will execute the workflow:

    export PROJECT_ID=$(gcloud config get-value project)
    export SERVICE_NAME=trigger-workflow-pubsub
    gcloud builds submit --tag gcr.io/${PROJECT_ID}/${SERVICE_NAME}
    
  3. Respond n, "No" to the Allow unauthenticated invocations to trigger-workflow-pubsub (y/N)? prompt.

  4. Deploy the container image to Cloud Run:

    gcloud run deploy ${SERVICE_NAME} \
      --image gcr.io/${PROJECT_ID}/${SERVICE_NAME} \
      --region=${REGION} \
      --update-env-vars GOOGLE_CLOUD_PROJECT=${PROJECT_ID},WORKFLOW_REGION=${REGION},WORKFLOW_NAME=${WORKFLOW_NAME}
    

    This Cloud Run service executes the workflow with the HTTP request.

    console.log(`Workflow path: ${GOOGLE_CLOUD_PROJECT}, ${WORKFLOW_REGION}, ${WORKFLOW_NAME}`);
    const execResponse = await client.createExecution({
      parent: client.workflowPath(GOOGLE_CLOUD_PROJECT, WORKFLOW_REGION, WORKFLOW_NAME),
      execution: {
        argument: JSON.stringify({headers: req.headers, body: req.body})
      }
    });
    console.log(`Execution response: ${JSON.stringify(execResponse)}`);
    
    const execName = execResponse[0].name;
    console.log(`Created execution: ${execName}`);
    
    res.status(200).send(`Created execution: ${execName}`);

When you see the service URL, the deployment is complete.

Create an Eventarc trigger

When a message is published to the Pub/Sub topic, the event triggers the authenticated Cloud Run service. This event should be triggered by a caller that has a service account with required IAM roles and permissions to use the resource.

  1. Create a trigger to listen for Pub/Sub messages:

    New Pub/Sub topic

      gcloud eventarc triggers create ${SERVICE_NAME} \
        --destination-run-service=${SERVICE_NAME} \
        --destination-run-region=${REGION} \
        --location=${REGION} \
        --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
        --service-account=SERVICE_ACCOUNT_USER_EMAIL
    

    Replace SERVICE_ACCOUNT_USER_EMAIL with the email address for the service account.
    This creates a new Pub/Sub topic and a trigger for it called trigger-workflow-pubsub.

    Existing Pub/Sub topic

      gcloud eventarc triggers create ${SERVICE_NAME} \
        --destination-run-service=${SERVICE_NAME} \
        --destination-run-region=${REGION} \
        --location=${REGION} \
        --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
        --transport-topic=projects/PROJECT_ID/topics/TOPIC_ID \
        --service-account=SERVICE_ACCOUNT_USER_EMAIL
    

    Replace the following:

    • PROJECT_ID is your Google Cloud project ID.
    • TOPIC_ID is the ID of the existing Pub/Sub topic.
    • SERVICE_ACCOUNT_USER_EMAIL is the email address for the service account.

      This creates a trigger called trigger-workflow-pubsub for the existing Pub/Sub topic.

  2. Confirm the trigger was successfully created:

      gcloud eventarc triggers list --location=us-central1
    

Generate and view an event

Publish a message to a Pub/Sub topic to generate an event and trigger the Cloud Run service. The Cloud Run service then executes the workflow that logs the messages on the Workflows service logs.

  1. Find and set the Pub/Sub topic created, as an environment variable:

    export TOPIC_ID=$(basename $(gcloud eventarc triggers describe ${SERVICE_NAME} --format='value(transport.pubsub.topic)'))
    
  2. Send a message to the Pub/Sub topic to trigger the workflow:

    gcloud pubsub topics publish $TOPIC_ID --message "Hello there"
    

    The generated event is logged in the Workflows service logs.

  3. To view the event message, go to the Workflows service logs:

    1. Open the Workflows page in the Google Cloud Console.

      Go to Workflows

    2. Click the workflow-pubsub workflow.

    3. Select the Logs tab.

      Logs might take a few moments to appear. If you don't see them immediately, check again after a few moments.

  4. Look for the "Hello there!" message.

Clean up

If you created a new project for this tutorial, delete the project. If you used an existing project and wish to keep it without the changes added in this tutorial, delete resources created for the tutorial.

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete tutorial resources

  1. Delete the Cloud Run service you deployed in this tutorial:

    gcloud run services delete SERVICE-NAME

    Where SERVICE-NAME is your chosen service name.

    You can also delete Cloud Run services from the Google Cloud Console.

  2. Remove the gcloud default configurations you added during tutorial setup.

    Remove the region setting:

     gcloud config unset run/region
    
  3. Remove the project configuration:

     gcloud config unset project
    
  4. Delete other Google Cloud resources created in this tutorial:

    • Delete the trigger:
      gcloud eventarc triggers delete TRIGGER_NAME
      
      Replace TRIGGER_NAME with the name of your trigger.

What's next