Troubleshoot Eventarc for Workflows

This page shows you how to resolve issues that you might encounter when using Eventarc for Workflows.

For other issues that you might also encounter, refer to troubleshooting for:

Trigger creation fails because workflow target does not exist

You receive an error message similar to the following: 

cloud workflow "projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_ID" does not exist
This output includes the following values:

  • PROJECT_ID: your Google Cloud project ID
  • LOCATION: the location of your workflow
  • WORKFLOW_ID: the name of your workflow

This occurs when Eventarc is not able to find the target workflow. To resolve this:

  1. Ensure that the target workflow exists and is ACTIVE: 

    gcloud workflows list --location -

    The output should be similar to the following: 

    NAME                                                          STATE   REVISION_ID  UPDATE_TIME
projects/PROJECT_ID/locations/LOCATION/workflows/WORKFLOW_ID  ACTIVE  000004-c0c   2021-11-19T14:29:27.530185556Z

  2. Ensure that you have provided the correct workflow ID and workflow location when creating the trigger.

Trigger is created successfully but target is not receiving events

Check that enough time has elapsed since creating the trigger. It can take up to two minutes before events are sent.

If the trigger still doesn't work and events are not being delivered:

  1. Events sent from Pub/Sub to the target might be getting dropped. To ensure that events aren't dropped, set up a Pub/Sub subscription retry policy or forward undelivered messages to a dead-letter topic (also known as a dead-letter queue).

    Before setting the dead-letter topic, retrieve the trigger's topic and subscription:

    gcloud eventarc triggers describe TRIGGER \
--location=LOCATION

    Replace the following:

    • TRIGGER: the ID of the trigger or a fully qualified identifier.
    • LOCATION: the location of the Eventarc trigger.

  2. Use the Google Cloud console to monitor if messages are published to the Pub/Sub topic with the metric: topic/send_message_operation_count.

  3. If messages are not being published to Pub/Sub topic, verify that the source is generating events:

    • For events from Cloud Audit Logs, check the logs and make sure that the monitored service is writing logs. If logs are recorded but events are not delivered, contact support.

    • For events from Cloud Storage, check notifications for your bucket: 

      gsutil notification list gs://BUCKET_NAME
      Replace BUCKET_NAME with the name of the bucket.
      The output should be similar to the following:

      projects/_/buckets/BUCKET_NAME/notificationConfigs/NOTIFICATION_CONFIG_ID
Cloud Pub/Sub topic: projects/PROJECT_ID/topics/TOPIC_ID
Filters:
  Event Types: OBJECT_ARCHIVE

      This output includes the following values:

      • TOPIC_ID: the ID of the existing Pub/Sub topic.
      • NOTIFICATION_CONFIG_ID: the ID of the notification configuration.

  4. If events are being delivered but no workflow executions are being triggered, this is likely due to an unauthenticated invocation. Ensure that the trigger is associated with a service account that has permission to create workflow executions. For more information, follow the instructions to create a user-managed service account in the "Prepare to create a trigger" section when creating a trigger for a specific provider, event type, and Workflows destination .

  5. If messages are published to a Pub/Sub topic but no workflow executions are being triggered, make sure that the Eventarc payload is not more than 512 KB. For more information on resource limits, see Quotas and Limits.

    1. In the Cloud console, go to the Subscriptions page.

      Go to Subscriptions

    2. Monitor unacknowledged messages in the subscription. For more information, see Monitoring forwarded undeliverable messages.

    3. If there are unacknowledged messages, monitor the workflow logs for the TriggerPubsubExecution method. For more information on viewing workflow logs, see View workflow logs.

      • Use the filter Permission 'workflows.executions.create' denied to verify if the trigger is associated with a service account that has permission to trigger workflow executions. For more information on how to grant the appropriate roles to the service account, follow the instructions in the "Prepare to create a trigger" section when creating a trigger for a specific provider, event type, and Workflows destination.
      • Use the key word event size exceeded to verify if the event size is over 512 KB.

    4. If logs are recorded but events are not delivered, contact support.