Migrate Pub/Sub triggers from Events for Cloud Run for Anthos to Eventarc

You can migrate event triggers so that a Cloud Run for Anthos service can receive Pub/Sub events using Eventarc. This guide assumes that you are migrating existing Events for Cloud Run for Anthos triggers and that your Cloud Run for Anthos service is running in a Google Kubernetes Engine (GKE) cluster. A migration needs to be run for each cluster.

This migration path avoids the loss of events while minimizing their duplication.

The migration involves the following:

  1. To allow Eventarc to manage resources in the GKE cluster, enable GKE destinations.
  2. If necessary, enable Workload Identity for the GKE cluster running your target Cloud Run for Anthos service.
  3. Identify existing Events for Cloud Run for Anthos triggers.
  4. Create an Eventarc trigger that points to the same Cloud Run for Anthos service as the existing trigger.
  5. Confirm that the Eventarc trigger is delivering events as expected.
  6. Delete the original event trigger and clean up after migration.

Before you begin

This guide assumes that you already have:

Set up the command-line environment

To set up the gcloud and kubectl command-line tools:

  1. Install and initialize the Google Cloud CLI.

  2. Log in using your account:

    gcloud auth login
  3. Set the default settings for the gcloud CLI:

    gcloud config set project PROJECT_ID
    gcloud config set run/cluster CLUSTER_NAME
    gcloud config set run/cluster_location CLUSTER_LOCATION
    gcloud config set run/platform gke
    gcloud config set eventarc/location LOCATION
    

    Replace:

    • PROJECT_ID with your project ID
    • CLUSTER_NAME with the name of your cluster
    • CLUSTER_LOCATION with any zone where GKE is supported; for example: us-central1-a
    • LOCATION with a location for the Eventarc trigger; for example: us-central1
  4. Enable the following APIs for the project:

    gcloud services enable cloudapis.googleapis.com
    gcloud services enable cloudbuild.googleapis.com
    gcloud services enable cloudresourcemanager.googleapis.com
    gcloud services enable container.googleapis.com
    gcloud services enable containerregistry.googleapis.com
    gcloud services enable eventarc.googleapis.com
    
  5. Install the kubectl command-line tool:

    gcloud components install kubectl
  6. Update installed gcloud CLI components:

    gcloud components update

Enable GKE destinations

To allow Eventarc to manage resources in the GKE cluster, enable GKE destinations, and bind the Eventarc service account with the required roles.

  1. Enable GKE destinations for Eventarc:

    gcloud eventarc gke-destinations init
    
  2. At the prompt to bind the required roles, enter y.

    The following roles are bound:

    • roles/compute.viewer
    • roles/container.developer
    • roles/iam.serviceAccountAdmin

Enable Workload Identity on a cluster

If Workload Identity is already enabled on your cluster, you can skip this step.

Workload Identity is the recommended way to access Google Cloud services from applications running within Google Kubernetes Engine (GKE) due to its improved security properties and manageability. It is also needed to forward Cloud Run for Anthos events using Eventarc.

To enable Workload Identity on an existing cluster, refer to Using Workload Identity.

Identify existing Events for Cloud Run for Anthos triggers

Before migrating any existing Events for Cloud Run for Anthos triggers, you must retrieve the trigger details.

  1. List the existing Events for Cloud Run for Anthos triggers for the cluster:

    gcloud beta events triggers list --namespace EVENTS_NAMESPACE
    

    Replace EVENTS_NAMESPACE with the namespace of your events broker.

    The output is similar to the following:

       TRIGGER             EVENT TYPE                                     TARGET
    ✔  trigger-id          google.cloud.pubsub.topic.v1.messagePublished  cloud-run-service-name
    
  2. You will need the topic ID to create the Eventarc trigger. Retrieve the topic ID for the existing Events for Cloud Run for Anthos trigger:

    gcloud beta events triggers describe TRIGGER_ID \
    --platform gke --namespace EVENTS_NAMESPACE \
    --format="flattened(serialized_source.spec.topic,serialized_trigger.spec.filter.attributes.type,serialized_trigger.spec.subscriber.ref.name,serialized_trigger.spec.subscriber.ref.namespace)"
    

    Replace TRIGGER_ID with the ID of the existing Events for Cloud Run for Anthos trigger or a fully qualified identifier.

    The output is similar to the following:

    topic:     topic-id
    type:      google.cloud.pubsub.topic.v1.messagePublished
    name:      cloud-run-service-name
    namespace: events
    

Create an Eventarc trigger to replace the existing trigger

Before creating the Eventarc trigger, set up a user-managed service account and grant it specific roles so that Eventarc can manage events for Cloud Run for Anthos destinations.

  1. Create a Google Cloud service account (GSA):

    TRIGGER_GSA=SERVICE_ACCOUNT_NAME
    gcloud iam service-accounts create $TRIGGER_GSA

    Replace SERVICE_ACCOUNT_NAME with a name that is between 6 and 30 characters. It can contain lowercase alphanumeric characters and dashes.

  2. Grant the pubsub.subscriber and monitoring.metricWriter roles to the service account:

    PROJECT_ID=$(gcloud config get-value project)
    
    gcloud projects add-iam-policy-binding $PROJECT_ID \
      --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
      --role "roles/pubsub.subscriber"
    
    gcloud projects add-iam-policy-binding $PROJECT_ID \
      --member "serviceAccount:$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com" \
      --role "roles/monitoring.metricWriter"
  3. Create a new Eventarc trigger based on the existing Events for Cloud Run for Anthos trigger's configuration. All the arguments, including the destination service, cluster, and topic ID, should match that of the existing Events for Cloud Run for Anthos trigger.

    gcloud eventarc triggers create EVENTARC_TRIGGER_NAME \
      --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
      --location LOCATION \
      --destination-gke-service=DESTINATION_SERVICE \
      --destination-gke-cluster=CLUSTER_NAME  \
      --destination-gke-location=CLUSTER_LOCATION  \
      --destination-gke-namespace=EVENTS_NAMESPACE  \
      --destination-gke-path=/ \
      --service-account=$TRIGGER_GSA@$PROJECT_ID.iam.gserviceaccount.com \
      --transport-topic=projects/PROJECT_ID/topics/TOPIC_ID
    

    Replace EVENTARC_TRIGGER_NAME with a name for the new Eventarc trigger and TOPIC_ID with the Pub/Sub topic ID that you retrieved previously.

    This creates an Eventarc trigger for the Pub/Sub topic ID.

  4. Confirm that the trigger was successfully created:

    gcloud eventarc triggers list
    

    The output is similar to the following:

    NAME                  TYPE                                           DESTINATION_RUN_SERVICE  DESTINATION_RUN_PATH  ACTIVE
    eventarc-trigger-name google.cloud.pubsub.topic.v1.messagePublished                                                 Yes
    

Test the migration and verifying the output

To test your migration, trigger a test event by publishing to a Pub/Sub topic, and then verify that there is at least one event delivered for each of the existing Events for Cloud Run for Anthos and Eventarc triggers, respectively.

For example, assuming that you completed the Receiving events from Pub/Sub tutorial and are now migrating that Events for Cloud Run for Anthos trigger, the output after triggering a test event would be duplicated, and would be similar to the following:

  Hello, World! ID: 2896291125947199
  Hello, World! ID: 2896291125947199

You can also use the following scenario for guidance in which the event receiver can display the event data:

  1. Publish a message to the Pub/Sub topic:

    gcloud pubsub topics publish TOPIC_ID \
      --message "Hello, World!"
    
  2. Verify the event and its successful delivery. For example, look at the logs of the receiving Cloud Run for Anthos service:

    kubectl logs \
      --selector serving.knative.dev/service=DESTINATION_SERVICE \
      -c user-container \
      -n EVENTS_NAMESPACE \
      --tail=100
    

    The output from the two triggers is similar to the following, with differences only in the Context Attributes and Extensions. The actual message data, with the exception of the subscription data, should be identical so that no event receiver changes are required after migrating to Eventarc.

    Eventarc event:

    ☁️  cloudevents.Event
    Validation: valid
    Context Attributes,
      specversion: 1.0
      type: google.cloud.pubsub.topic.v1.messagePublished
      source: //pubsub.googleapis.com/projects/project-id/topics/topic-name
      id: 2759155988927083
      time: 2021-07-22T17:47:19.964Z
      datacontenttype: application/json
    Data,
      {
        "subscription": "projects/project-id/subscriptions/eventarc-us-central1-eventarc-trigger-name-sub-567",
        "message": {
          "data": "V29ybGQ=",
          "messageId": "2759155988927083",
          "publishTime": "2021-07-22T17:47:19.964Z"
        }
      }
    

    Events for Cloud Run for Anthos event:

    ☁️  cloudevents.Event
    Validation: valid
    Context Attributes,
      specversion: 1.0
      type: google.cloud.pubsub.topic.v1.messagePublished
      source: //pubsub.googleapis.com/projects/project-id/topics/topic-name
      id: 2759155988927083
      time: 2021-07-22T17:47:19.964Z
      dataschema: https://raw.githubusercontent.com/googleapis/google-cloudevents/master/proto/google/events/cloud/pubsub/v1/data.proto
      datacontenttype: application/json
    Extensions,
      knativearrivaltime: 2021-07-22T17:47:20.723302362Z
      knsourcetrigger: link0.09767362059083662
      traceparent: 00-f3cc6d754d361a0e49e83e5973fa3565-b4a20ef4ecffe96b-00
    Data,
      {
        "subscription": "cre-src_events_source-for-trigger-name_5ffa9638-8cab-4010-900a-2edb275d9eaf",
        "message": {
          "messageId": "2759155988927083",
          "data": "V29ybGQ=",
          "publishTime": "2021-07-22T17:47:19.964Z"
        }
      }
    

Clean up after the migration

After testing and verifying the migration of the trigger to Eventarc, you can delete the original, Events for Cloud Run for Anthos trigger.

  1. Delete the Events for Cloud Run for Anthos trigger:

    gcloud beta events triggers delete TRIGGER_NAME \
      --platform gke \
      --namespace EVENTS_NAMESPACE \
      --quiet
    

You have migrated an existing Events for Cloud Run for Anthos trigger to Eventarc.

What's next