Automate builds in response to Pub/Sub events

Cloud Build Pub/Sub triggers enable you to execute builds in response to Google Cloud events published over Pub/Sub. You can use information from a Pub/Sub event to parameterize your build and to decide if a build should execute in response to the event. Pub/Sub triggers can be configured to listen to any Pub/Sub topic.

This page explains how you can create a Pub/Sub trigger to automate builds in response to events on Artifact Registry, Container Registry (Deprecated), and Cloud Storage.

Limitations

Cloud Build Pub/Sub triggers are not supported when VPC Service Controls is used.

Before you begin

  • Enable the Cloud Build API.

    Enable the API

  • Make sure your source code contains a build config file or a Dockerfile in the repository.

  • To use gcloud commands on this page, install the Google Cloud CLI.

Creating a build trigger that responds to Artifact Registry events

You can create a Pub/Sub trigger that responds to Artifact Registry events such as when images are pushed, tagged, or deleted. This section discusses how you can create a Pub/Sub trigger that invokes a build when a new tag is pushed to an existing image. If you are unfamiliar with Artifact Registry, see the Artifact Registry overview.

Console

To create a trigger that listens for a new tag pushed to an existing image in Artifact Registry using the Google Cloud console:

  1. Open the Triggers page:

    Open the Triggers page

  2. Select your project from the top of the page and click Open.

  3. Click Create trigger.

  4. Enter the following trigger settings:

    • Name: Enter a name for your trigger.

    • Region: Select the region for your trigger.

      • If the build config file associated with the trigger specifies a private pool, Cloud Build uses the private pool to run your build. In this case, the region you specify in your trigger must match the region where you created your private pool.
      • If the build config file associated with the trigger does not specify a private pool, Cloud Build uses the default pool to run your build in the same region as your trigger.

    • Description (Optional): Enter a description for your trigger.

    • Event: Select Pub/Sub message as the event to invoke your trigger.

    • Subscription: Select the Pub/Sub topic you would like to subscribe to as the trigger event. You see all existing topics in your project in the drop-down menu.

    • Source: Select the source to build when the Pub/Sub trigger runs. You can specify 1st generation or 2nd generation as your source.

      • Repository: From the list of available repositories, select the desired repository.

      • Branch or Tag: Specify a regular expression with the branch or tag value to match. For information on acceptable regular expression syntax, see RE2 syntax.

      • Comment control: If you selected Pull request (GitHub App only) as your Event, choose one of the following options to control whether a build will automatically be executed by the trigger:

        • Required except for owners and collaborators: When a pull request is created or updated by a repository owner or collaborator, builds will automatically be executed by the trigger. If an external contributor initiates the action, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request.

        • Required: When a pull request is created or updated by any contributor, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request. Builds are executed each time a change to a pull request is made.

        • Not required: When a pull request is created or updated by any contributor, builds will automatically be executed by triggers.

    • Configuration: Select the build config file (located in your remote repository) or create an inline build config file to use for your build.

      • Type: Select the type of configuration to use for your build.
        • Cloud Build configuration file (yaml or json): Use a build config file for your configuration.
        • Dockerfile: Use a Dockerfile for your configuration.
        • Buildpacks: Use buildpacks for your configuration.

      • Location: Specify the location for your configuration.

        • Repository: If your config file is located in your remote repository, provide the location of your build config file, the Dockerfile directory, or the buildpacks directory. If your build config type is a Dockerfile or a buildpack, you will need to provide a name for the resulting image and optionally, a timeout for your build. When you've provided the Dockerfile or buildpack image name, you'll see a preview of the docker build or pack command that your build will execute.
        • Buildpack environment variables (optional): If you selected buildpacks as your configuration type, click Add pack environment variable to specify your buildpack environment variables and values. To learn more about buildpack environment variables, see Environment variables.

        • Inline: If you selected Cloud Build configuration file (yaml or json) as your configuration option, you can specify your build config inline. Click Open Editor to write your build config file in the Google Cloud console using YAML or JSON syntax. Click Done to save your build config.

    • Substitutions (optional): If you selected the build config file as your build config option, you can choose to define trigger-specific substitution variables using this field.

      In the following example, we want to obtain the name of the image tag from the payload and the action associated with the gcr event. To do so, create substitution variables using payload bindings.

      Specify the following variables and values below:

      Variable Name Variable Value
      _IMAGE_TAG $(body.message.data.tag)
      _ACTION $(body.message.data.action)

      body.message references the PubSubMessage published by publishers and consumed by subscribers. To see more examples of the Pub/Sub notification payload, see Notification examples.

    • Filters (optional): You can create filters within a trigger that determine whether or not your trigger will execute a build in response to the incoming payload by specifying filters on substitution variables. The filter expression must evaluate to true in order for a build to execute.

      We recommend using filtering when setting up Pub/Sub triggers on topics with several messages. Filters can be used to precisely control builds that are executed in response to incoming Pub/Sub messages. To learn about risks associated with setting up a trigger without filters, see Risks associated with an unfiltered trigger.

      In the following example, we want the trigger to execute a build if a new tag is pushed to an existing image. We use filter condition operators to check if the _IMAGE_TAG variable matches an existing tag name and if the _ACTION variable matches INSERT to look for newly added data.

      Specify the following as your filters:

      • _IMAGE_TAG != ""
      • _ACTION == INSERT

      Filtering syntax in Pub/Sub triggers use Common Expression Language (CEL) for expression evaluation. To learn more about CEL, see the cel-spec repository.

  1. Click Create to create your build trigger.

gcloud

To create a trigger that listens for a new tag pushed to an existing image in Artifact Registry using the gcloud commands:

  1. Open a terminal window.

  2. Run the gcloud command to create a build trigger in your project. In the example below, the trigger is configured to respond to builds with a tag matching prod and an action matching INSERT based on the specified payload as defined by the substitution variable, _IMAGE_TAG.

     gcloud builds triggers create pubsub \
   --region=REGION \
   --name=TRIGGER_NAME \
   --repository=projects/PROJECT_ID/locations/REGION/connections/CONNECTION_NAME/repositories/REPO_NAME \
   --topic=projects/PROJECT_ID/topics/TOPIC_NAME \
   --build-config=BUILD_CONFIG \ # or --inline-config=INLINE_BUILD_CONFIG
   --substitutions=\
     '_IMAGE_TAG_="$(body.message.data.tag)",' \
     '_ACTION="$(body.message.data.action)"' \
   --subscription-filter='_IMAGE_TAG != "" && _ACTION == "INSERT"' \
   --tag=TAG_NAME  # or --branch=BRANCH_NAME

Where:

  • REGION is the region for your trigger.
  • TRIGGER_NAME is the name of your trigger.
  • PROJECT_ID is the ID of your Cloud project.
  • CONNECTION_NAME is the name of your host connection.
  • REPO_NAME is the name of your repository.
  • TOPIC_NAME is the name of Pub/Sub topic you've subscribed to.
  • BUILD_CONFIG is the path to your build config file.
  • INLINE_BUILD_CONFIG is the path to your inline build config file.
  • TAG_NAME is the name of your tag if you want to set your trigger to build on a tag.
  • BRANCH_NAME is the name of your branch if you want to set your trigger to build on a branch.

Creating a build trigger that responds to Container Registry events

You can create a Pub/Sub trigger that responds to Container Registry events such as when images are pushed, tagged, or deleted. This section discusses how you can create a Pub/Sub trigger that invokes a build when an image matches a tag set up by a custom filter. If you are unfamiliar with Container Registry, see the Quickstart for Container Registry to learn how to push and pull images with tags.

Console

To create a trigger that listens for an image push in Container Registry and matches based on a tag name using the Google Cloud console:

  1. Open the Triggers page:

    Open the Triggers page

  2. Select your project from the top of the page and click Open.

  3. Click Create trigger.

  4. Enter the following trigger settings:

    • Name: Enter a name for your trigger.

    • Region: Select the region for your trigger.

      • If the build config file associated with the trigger specifies a private pool, Cloud Build uses the private pool to run your build. In this case, the region you specify in your trigger must match the region where you created your private pool.
      • If the build config file associated with the trigger does not specify a private pool, Cloud Build uses the default pool to run your build in the same region as your trigger.

    • Description (Optional): Enter a description for your trigger.

    • Event: Select Pub/Sub message as the event to invoke your trigger.

    • Subscription: Select the Pub/Sub topic you would like to subscribe to as the trigger event. You see all existing topics in your project in the drop-down menu.

    • Source: Select the source to build when the Pub/Sub trigger runs. You can specify 1st generation or 2nd generation as your source.

      • Repository: From the list of available repositories, select the desired repository.

      • Branch or Tag: Specify a regular expression with the branch or tag value to match. For information on acceptable regular expression syntax, see RE2 syntax.

      • Comment control: If you selected Pull request (GitHub App only) as your Event, choose one of the following options to control whether a build will automatically be executed by the trigger:

        • Required except for owners and collaborators: When a pull request is created or updated by a repository owner or collaborator, builds will automatically be executed by the trigger. If an external contributor initiates the action, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request.

        • Required: When a pull request is created or updated by any contributor, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request. Builds are executed each time a change to a pull request is made.

        • Not required: When a pull request is created or updated by any contributor, builds will automatically be executed by triggers.

    • Configuration: Select the build config file (located in your remote repository) or create an inline build config file to use for your build.

      • Type: Select the type of configuration to use for your build.
        • Cloud Build configuration file (yaml or json): Use a build config file for your configuration.
        • Dockerfile: Use a Dockerfile for your configuration.
        • Buildpacks: Use buildpacks for your configuration.

      • Location: Specify the location for your configuration.

        • Repository: If your config file is located in your remote repository, provide the location of your build config file, the Dockerfile directory, or the buildpacks directory. If your build config type is a Dockerfile or a buildpack, you will need to provide a name for the resulting image and optionally, a timeout for your build. When you've provided the Dockerfile or buildpack image name, you'll see a preview of the docker build or pack command that your build will execute.
        • Buildpack environment variables (optional): If you selected buildpacks as your configuration type, click Add pack environment variable to specify your buildpack environment variables and values. To learn more about buildpack environment variables, see Environment variables.

        • Inline: If you selected Cloud Build configuration file (yaml or json) as your configuration option, you can specify your build config inline. Click Open Editor to write your build config file in the Google Cloud console using YAML or JSON syntax. Click Done to save your build config.

    • Substitutions (optional): If you selected the build config file as your build config option, you can choose to define trigger-specific substitution variables using this field.

      In the following example, we want to obtain the name of the image tag from the payload and the action associated with the gcr event. To do so, create substitution variables using payload bindings.

      Specify the following variables and values below:

      Variable Name Variable Value
      _IMAGE_TAG $(body.message.data.tag)
      _ACTION $(body.message.data.action)

      body.message references the PubSubMessage published by publishers and consumed by subscribers. To see more examples of the Pub/Sub notification payload, see Notification examples.

    • Filters (optional): You can create filters within a trigger that determine whether or not your trigger will execute a build in response to the incoming payload by specifying filters on substitution variables. The filter expression must evaluate to true in order for a build to execute.

      We recommend using filtering when setting up Pub/Sub triggers on topics with several messages. Filters can be used to precisely control builds that are executed in response to incoming Pub/Sub messages. To learn about risks associated with setting up a trigger without filters, see Risks associated with an unfiltered trigger.

      In the following example, we want the trigger to execute a build if the name of the _IMAGE_TAG tag variable matches a specific tag name, such as prod. You can specify your filter condition operator as "==" for exact matching. You can also check for the action associated with your gcr event as well. For example, you may want to specify _ACTION is INSERT to look for newly added data.

      Specify the following as your filters:

      • _IMAGE_TAG.matches(prod)
      • _ACTION.matches(INSERT)

      Filtering syntax in Pub/Sub triggers use Common Expression Language (CEL) for expression evaluation. To learn more about CEL, see the cel-spec repository. To see more example syntax for filtering you could apply to your Pub/Sub triggers, see Filtering builds.

  1. Click Create to create your build trigger.

gcloud

Creating a build trigger that responds to Cloud Storage events

You can create a Pub/Sub trigger that responds to Cloud Storage events such as when a new binary is pushed to an existing storage bucket. This section discusses how you can create a Pub/Sub trigger that responds with a build when deploying a new binary to an uploaded bucket. If you are unfamiliar with Cloud Storage, see the Quickstarts.

Console

To create a trigger that listens to Cloud Storage events using the Google Cloud console:

  1. Open the Triggers page:

    Open the Triggers page

  2. Select your project from the top of the page and click Open.

  3. Click Create trigger.

  4. Enter the following trigger settings:

    • Name: Enter a name for your trigger.

    • Region: Select the region for your trigger.

      • If the build config file associated with the trigger specifies a private pool, Cloud Build uses the private pool to run your build. In this case, the region you specify in your trigger must match the region where you created your private pool.
      • If the build config file associated with the trigger does not specify a private pool, Cloud Build uses the default pool to run your build in the same region as your trigger.

    • Description (Optional): Enter a description for your trigger.

    • Event: Select Pub/Sub message as the event to invoke your trigger.

    • Subscription: Select the Pub/Sub topic you would like to subscribe to as the trigger event. You see all existing topics in your project in the drop-down menu.

    • Source: Select the source to build when the Pub/Sub trigger runs. You can specify 1st generation or 2nd generation as your source.

      • Repository: From the list of available repositories, select the desired repository.

      • Branch or Tag: Specify a regular expression with the branch or tag value to match. For information on acceptable regular expression syntax, see RE2 syntax.

      • Comment control: If you selected Pull request (GitHub App only) as your Event, choose one of the following options to control whether a build will automatically be executed by the trigger:

        • Required except for owners and collaborators: When a pull request is created or updated by a repository owner or collaborator, builds will automatically be executed by the trigger. If an external contributor initiates the action, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request.

        • Required: When a pull request is created or updated by any contributor, builds will only be executed after an owner or collaborator comments /gcbrun on the pull request. Builds are executed each time a change to a pull request is made.

        • Not required: When a pull request is created or updated by any contributor, builds will automatically be executed by triggers.

    • Configuration: Select the build config file (located in your remote repository) or create an inline build config file to use for your build.

      • Type: Select the type of configuration to use for your build.
        • Cloud Build configuration file (yaml or json): Use a build config file for your configuration.
        • Dockerfile: Use a Dockerfile for your configuration.
        • Buildpacks: Use buildpacks for your configuration.

      • Location: Specify the location for your configuration.

        • Repository: If your config file is located in your remote repository, provide the location of your build config file, the Dockerfile directory, or the buildpacks directory. If your build config type is a Dockerfile or a buildpack, you will need to provide a name for the resulting image and optionally, a timeout for your build. When you've provided the Dockerfile or buildpack image name, you'll see a preview of the docker build or pack command that your build will execute.
        • Buildpack environment variables (optional): If you selected buildpacks as your configuration type, click Add pack environment variable to specify your buildpack environment variables and values. To learn more about buildpack environment variables, see Environment variables.

        • Inline: If you selected Cloud Build configuration file (yaml or json) as your configuration option, you can specify your build config inline. Click Open Editor to write your build config file in the Google Cloud console using YAML or JSON syntax. Click Done to save your build config.

    • Substitutions (optional): If you selected the build config file as your build config option, you can choose to define trigger-specific substitution variables using this field.

      In this example, we want to watch for the deployment of a new binary when it is uploaded to a bucket. To obtain this data, we can create substitution variables using payload bindings.

      Specify the following variables and values below:

      Variable Name Variable Value
      _EVENT_TYPE $(body.message.attributes.eventType)
      _BUCKET_ID $(body.message.attributes.bucketId)
      _OBJECT_ID $(body.message.attributes.objectId)

      body.message references the PubSubMessage published by publishers and consumed by subscribers. To see more examples of the Pub/Sub notification payload, see Notification examples.

    • Filters (optional): You can create filters within a trigger that determine whether or not your trigger will execute a build in response to the incoming payload by specifying filters on substitution variables. The filter expression must evaluate to true in order for a build to execute.

      We recommend using filtering when setting up Pub/Sub triggers on topics with several messages. Filters can be used to precisely control builds that are executed in response to incoming Pub/Sub messages. To learn about risks associated with setting up a trigger without filters, see Risks associated with an unfiltered trigger.

      Since we want the trigger to execute a build if the new binary has been deployed to a specific bucket, we can use "==" operator to check for exact matches. You can also use the "matches" keyword if you want to match by regular expression.

      Specify the following as your filters:

      • _EVENT_TYPE == OBJECT_FINALIZE
      • _OBJECT_ID matches ^<object-id>$
      • _BUCKET_ID matches ^<bucket-id>$
  1. Click Create to create your build trigger
    2. .

gcloud

Risks associated with an unfiltered trigger

If you have not configured filters on your Pub/Sub trigger, your trigger may end up invoking an infinite number of builds if your trigger modifies an artifact or object which unintentionally publishes a new message to the topic it's listening to. For example, your trigger may invoke an infinite number of builds if your trigger:

  • Points to a gcr topic.
  • Creates any image or tag in gcr.
  • Points to a gcs topic for a specific object in your bucket and modifies that object.

If you encounter an infinite loop, you can delete your trigger or update it to point to a separate topic to avoid incurring additional charges for each build you invoke.

What's next