Receive events using Pub/Sub messages (Terraform)

Receive events using Pub/Sub messages (Terraform)

Standard

This quickstart shows you how to use Terraform to create an Eventarc trigger that receives direct events from Pub/Sub and that routes the events to a Cloud Run service. For more information about using Terraform to create Eventarc triggers, see Create a trigger using Terraform.

In this quickstart, you will do the following:

1. Prepare to deploy Terraform.

2. Define a Terraform configuration that does the following:

   1. Enable APIs.
   2. Create a service account and grant it the necessary Identity and Access Management (IAM) roles.
   3. Deploy a service to Cloud Run as an event destination.
   4. Create a Pub/Sub topic as an event provider.
   5. Create an Eventarc trigger.

3. Apply your Terraform configuration.

4. Publish a message to a Pub/Sub topic to generate an event, and view it in the Cloud Run logs.

Before you begin

Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
2. Install the Google Cloud CLI.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

4. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Google Cloud project:

     gcloud projects create PROJECT_ID

     Replace PROJECT_ID with a name for the Google Cloud project you are creating.

   • Select the Google Cloud project that you created:

     gcloud config set project PROJECT_ID

     Replace PROJECT_ID with your Google Cloud project name.

5. Make sure that billing is enabled for your Google Cloud project.

6. Enable the Cloud Resource Manager and IAM APIs:

   gcloud services enable cloudresourcemanager.googleapis.com iam.googleapis.com

7. If you're using a local shell, then create local authentication credentials for your user account:

   gcloud auth application-default login

   You don't need to do this if you're using Cloud Shell.

8. Install the Google Cloud CLI.

9. To initialize the gcloud CLI, run the following command:

   gcloud init

10. Create or select a Google Cloud project.

    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    • Create a Google Cloud project:

      gcloud projects create PROJECT_ID

      Replace PROJECT_ID with a name for the Google Cloud project you are creating.

    • Select the Google Cloud project that you created:

      gcloud config set project PROJECT_ID

      Replace PROJECT_ID with your Google Cloud project name.

11. Make sure that billing is enabled for your Google Cloud project.

12. Enable the Cloud Resource Manager and IAM APIs:

    gcloud services enable cloudresourcemanager.googleapis.com iam.googleapis.com

13. If you're using a local shell, then create local authentication credentials for your user account:

    gcloud auth application-default login

    You don't need to do this if you're using Cloud Shell.

14. If you are the project creator, you are granted the basic Owner role (roles/owner). By default, this Identity and Access Management (IAM) role includes the permissions necessary for full access to most Google Cloud resources and you can skip this step.

    If you are not the project creator, required permissions must be granted on the project to the appropriate principal. For example, a principal can be a Google Account (for end users) or a service account (for applications and compute workloads). For more information, see the Roles and permissions page for your event destination.

    Required permissions

    To get the permissions that you need to complete this quickstart, ask your administrator to grant you the following IAM roles on your project:

    • Cloud Run Admin (roles/run.admin)
    • Eventarc Admin (roles/eventarc.admin)
    • Logs View Accessor (roles/logging.viewAccessor)
    • Project IAM Admin (roles/resourcemanager.projectIamAdmin)
    • Pub/Sub Publisher (roles/pubsub.publisher)
    • Service Account Admin (roles/iam.serviceAccountAdmin)
    • Service Account User (roles/iam.serviceAccountUser)
    • Service Usage Admin (roles/serviceusage.serviceUsageAdmin)

    For more information about granting roles, see Manage access to projects, folders, and organizations.

    You might also be able to get the required permissions through custom roles or other predefined roles.

Prepare to deploy Terraform

Prepare to deploy Terraform resources by creating a Terraform configuration file. A Terraform configuration file lets you define your preferred end-state for your infrastructure using the Terraform syntax.

1. If you are using a local shell, install and configure Terraform.

   Terraform is already integrated into the Cloud Shell environment and you can use Cloud Shell to deploy your Terraform resources without having to install Terraform.

2. In Cloud Shell or your local shell, set the default Google Cloud project where you want to apply your Terraform configurations. You only need to run this command once per project, and you can run it in any directory:

   export GOOGLE_CLOUD_PROJECT=PROJECT_ID

   Replace PROJECT_ID with the ID of your Google Cloud project.

Note that environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module). Create a directory and create a new file within that directory:

mkdir DIRECTORY && cd DIRECTORY && touch main.tf

The filename must have the .tf extension—for example, in this quickstart, the file is referred to as main.tf.

Define your Terraform configuration

Copy the following Terraform code snippets into your newly created main.tf file. Optionally, you can copy the code from GitHub. (In the top right corner of the code snippet, click > View on GitHub.)

Enable APIs

Terraform samples typically assume that the required APIs are enabled in your Google Cloud project. Use the following code snippet to enable the APIs needed for this quickstart:

# Enable Cloud Run API
resource "google_project_service" "run" {
  service            = "run.googleapis.com"
  disable_on_destroy = false
}

# Enable Eventarc API
resource "google_project_service" "eventarc" {
  service            = "eventarc.googleapis.com"
  disable_on_destroy = false
}

# Enable Pub/Sub API
resource "google_project_service" "pubsub" {
  service            = "pubsub.googleapis.com"
  disable_on_destroy = false
}

Create a service account and configure its access

Every Eventarc trigger is associated with an IAM service account. To complete this quickstart, you must grant a user-managed service account the following IAM roles:

• Cloud Run Invoker role (roles/run.invoker)
• Pub/Sub Publisher role (roles/pubsub.publisher)

Use the following code snippet to create a dedicated service account and grant it specific IAM roles to manage events:

# Used to retrieve project information later
data "google_project" "project" {}

# Create a dedicated service account
resource "google_service_account" "eventarc" {
  account_id   = "eventarc-trigger-sa"
  display_name = "Eventarc trigger service account"
}

# Grant permission to invoke Cloud Run services
resource "google_project_iam_member" "runinvoker" {
  project = data.google_project.project.id
  role    = "roles/run.invoker"
  member  = "serviceAccount:${google_service_account.eventarc.email}"
}

# Grant permission to publish messages to a Pub/Sub topic
resource "google_project_iam_member" "pubsubpublisher" {
  project = data.google_project.project.id
  member  = "serviceAccount:${google_service_account.eventarc.email}"
  role    = "roles/pubsub.publisher"
}

If you enabled the Pub/Sub service agent on or before April 8, 2021, grant the Service Account Token Creator role (roles/iam.serviceAccountTokenCreator) to the service agent.

resource "google_project_iam_member" "tokencreator" {
  project  = data.google_project.project.id
  role     = "roles/iam.serviceAccountTokenCreator"
  member   = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-pubsub.iam.gserviceaccount.com"
}

Deploy an event receiver to Cloud Run

Create a Cloud Run service as an event destination for the Eventarc trigger using the google_cloud_run_v2_service Terraform resource:

# Deploy a Cloud Run service
resource "google_cloud_run_v2_service" "default" {
  name     = "hello-events"
  location = "us-central1"

  deletion_protection = false # set to "true" in production

  template {
    containers {
      # This container will log received events
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
    service_account = google_service_account.eventarc.email
  }

  depends_on = [google_project_service.run]
}

Create a Pub/Sub topic as an event provider

Create a Pub/Sub topic using the google_pubsub_topic Terraform resource:

# Create a Pub/Sub topic
resource "google_pubsub_topic" "default" {
  name = "pubsub_topic"
}

Create an Eventarc trigger

Create an Eventarc trigger to listen for Pub/Sub messages using the google_eventarc_trigger Terraform resource:

# Create an Eventarc trigger, routing Pub/Sub events to Cloud Run
resource "google_eventarc_trigger" "default" {
  name     = "trigger-pubsub-cloudrun-tf"
  location = google_cloud_run_v2_service.default.location

  # Capture messages published to a Pub/Sub topic
  matching_criteria {
    attribute = "type"
    value     = "google.cloud.pubsub.topic.v1.messagePublished"
  }

  # Send events to Cloud Run
  destination {
    cloud_run_service {
      service = google_cloud_run_v2_service.default.name
      region  = google_cloud_run_v2_service.default.location
    }
  }

  transport {
    pubsub {
      topic = google_pubsub_topic.default.id
    }
  }

  service_account = google_service_account.eventarc.email
  depends_on = [
    google_project_service.eventarc,
    google_project_iam_member.pubsubpublisher
  ]
}

Note: Although your trigger is created immediately, it can take up to two minutes for triggers to be fully functional.

Apply Terraform

Use the Terraform CLI to provision infrastructure based on the configuration file.

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

1. Initialize Terraform. You only need to do this once per directory.

   terraform init

   Optionally, to use the latest Google provider version, include the -upgrade option:

   terraform init -upgrade

2. Review the configuration and verify that the resources that Terraform is going to create or update match your expectations:

   terraform plan

   Make corrections to the configuration as necessary.

3. Apply the Terraform configuration by running the following command and entering yes at the prompt:

   terraform apply

   Typically, you apply the entire configuration at once. However, you can also target a specific resource. For example:

   terraform apply -target="google_eventarc_trigger.default"

   After enabling the APIs, it might take a few minutes for the action to propagate and before you can deploy any further resources. If you run into an issue, try applying the Terraform configuration again.

   Wait until Terraform displays the "Apply complete!" message.

Verify the creation of resources

1. Confirm that the Cloud Run service has been created:

   gcloud run services list --region us-central1
   

   The output should be similar to the following:

   SERVICE: hello-events
   REGION: us-central1
   URL: https://hello-events-13335919645.us-central1.run.app
   LAST DEPLOYED BY: ...
   LAST DEPLOYED AT: 2024-12-16T15:00:52.606160Z
   

2. Confirm that the Eventarc trigger has been created:

   gcloud eventarc triggers list --location us-central1
   

   The output should be similar to the following:

   NAME: trigger-pubsub-cloudrun-tf
   TYPE: google.cloud.pubsub.topic.v1.messagePublished
   DESTINATION: Cloud Run service: hello-events
   ACTIVE: Yes
   LOCATION: us-central1
   

Generate and view a Pub/Sub topic event

You can generate an event by publishing a message to the Pub/Sub topic. The Eventarc trigger routes the message to the event receiver service deployed on Cloud Run and the service logs the event message.

1. Find and set the Pub/Sub topic as an environment variable:

   gcloud config set eventarc/location us-central1
   export RUN_TOPIC=$(gcloud eventarc triggers describe trigger-pubsub-cloudrun-tf \
       --format='value(transport.pubsub.topic)')
   

2. Publish a message to the Pub/Sub topic to generate an event:

   gcloud pubsub topics publish $RUN_TOPIC --message "Hello World!"
   

   The event is routed to the Cloud Run service, which logs the event message.

3. To view the event-related log entries created by your service, run the following command:

   gcloud logging read 'jsonPayload.message: "Received event of type google.cloud.pubsub.topic.v1.messagePublished"'
   

4. Look for a log entry similar to:

   jsonPayload:
   ...
   message: 'Received event of type google.cloud.pubsub.topic.v1.messagePublished.
       Event data: Hello World!'
   

   Note: If you don't see a log entry, wait a few minutes and retry the gcloud logging read command. The trigger can take up to two minutes after creation to start filtering events.

You have successfully used Terraform to deploy an event receiver service to Cloud Run and create an Eventarc trigger. After generating an event from Pub/Sub, you are able to view it in the Cloud Run logs.

Clean up

When you finish the tasks that are described in this quickstart, you can avoid continued billing by deleting the resources that you created.

Remove resources previously applied with your Terraform configuration by running the following command and entering yes at the prompt:

terraform destroy

Alternatively, you can delete your Google Cloud project to avoid incurring charges. Deleting your Google Cloud project stops billing for all the resources used within that project.

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

Delete a Google Cloud project:

gcloud projects delete PROJECT_ID

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

What's next

• Develop services for Cloud Run
• Terraform on Google Cloud documentation