Deploy a workflow from a Git repository using Cloud Build

You can use a Cloud Build trigger to automatically start a build and deploy a workflow from a Git repository. You can configure the trigger to deploy your workflow on any change to the source repository, or deploy the workflow only when the change matches specific criteria.

This approach can help you manage your deployment lifecycle. For example, you can deploy changes to a workflow in a staging environment, run tests against that environment, and then incrementally roll out these changes to the production environment.

Before you begin

These instructions assume you have the Cloud Build Editor role (roles/cloudbuild.builds.editor) in your Google Cloud project so that you can create triggers. You also need a workflow in a source repository such as GitHub or Bitbucket.

Console

Enable the Cloud Build and Workflows APIs.

Enable the APIs
Grant the Workflows Admin role (roles/workflows.admin) to the Cloud Build service account:
1. In the Google Cloud console, go to the IAM page.
  
  Go to IAM
2. Select your project.
3. In the row for the Cloud Build service account (PROJECT_NUMBER@cloudbuild.gserviceaccount.com), click Edit principal.
4. Click Add another role.
5. In the Role list, select the Workflows Admin role.
6. Click Save.
Grant the Service Account User role (roles/iam.serviceAccountUser) on the Compute Engine default service account to the Cloud Build service account. When you have enabled the Compute Engine API, the Compute Engine default service account is PROJECT_NUMBER-compute@developer.gserviceaccount.com.

Caution: Assigning the Service Account User role indirectly grants the role associated with the default service account to the user. For example, if the default service account has the Editor role, the user can then "act as" an Editor. To minimize the impact of these role assignments, we recommend configuring the default service account according to the principle of least privilege. For more information, see Disable automatic role grants to default service accounts.
1. In the Google Cloud console, go to the Service Accounts page.
  
  Go to Service Accounts
2. Select your project.
3. Click the email address of the Compute Engine default service account (PROJECT_NUMBER-compute@developer.gserviceaccount.com).
4. Click the Permissions tab.
5. Click the Grant access button.
6. To add a new principal, enter the email address of your service account (SERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com).
7. In the Select a role list, select the Service Accounts > Service Account User role.
8. Click Save.

gcloud

Enable the Cloud Build and Workflows APIs.

gcloud services enable cloudbuild.googleapis.com \
  workflows.googleapis.com

Grant the Workflows Admin role (roles/workflows.admin) to the Cloud Build service account:

PROJECT_NUMBER=$(gcloud projects describe PROJECT_ID --format='value(projectNumber)')
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=serviceAccount:$PROJECT_NUMBER@cloudbuild.gserviceaccount.com \
  --role=roles/workflows.admin

Replace PROJECT_ID with the ID of your Google Cloud project.

Grant the Service Account User role (roles/iam.serviceAccountUser) on the Compute Engine default service account to the Cloud Build service account. When you have enabled the Compute Engine API, the Compute Engine default service account is PROJECT_NUMBER-compute@developer.gserviceaccount.com.

Caution: Assigning the Service Account User role indirectly grants the role associated with the default service account to the user. For example, if the default service account has the Editor role, the user can then "act as" an Editor. To minimize the impact of these role assignments, we recommend configuring the default service account according to the principle of least privilege. For more information, see Disable automatic role grants to default service accounts.
```
gcloud iam service-accounts add-iam-policy-binding \
  $PROJECT_NUMBER-compute@developer.gserviceaccount.com \
  --member=serviceAccount:$PROJECT_NUMBER@cloudbuild.gserviceaccount.com \
  --role=roles/iam.serviceAccountUser
```

Connect to your source repository

You must connect Cloud Build to your source repository so that Cloud Build can automate builds in response to events that happen in the repository.

Complete the following steps to connect to GitHub or Bitbucket:

In the Google Cloud console, go to the Cloud Build Triggers page:

Go to Triggers
If necessary, select your project and click Open.
From the Region list, select the region where you would like to create your trigger.

Note: If you select global as your region, default pools are used to run your build; otherwise, a private pool in the region of your trigger is used. You must specify the private pool in your build config file or through build arguments.
Click Connect repository.
Select the source repository where you've stored your source code.

For example: GitHub (Cloud Build GitHub App)
Click Continue.
Authenticate to your source repository with your username and password.

If you are signing into GitHub, you are asked to authorise the Google Cloud Build GitHub App to access your GitHub account to proceed.
From the list of available repositories, select the desired repository, and then click OK.

For external repositories such as GitHub and Bitbucket, you must have owner-level permissions for the Google Cloud project in which you're working.
Read the disclaimer and select the checkbox next to it to indicate that you consent to the terms.
Click Connect.
To continue creating a build trigger to automate builds for the source code in the repository, click Create a trigger. Otherwise, click Done.

Create a Cloud Build configuration file

A build configuration file defines the fields that are needed when using a build trigger to start a build. Create the config file in your project root directory and write it using either YAML or JSON.

For example, the following config file deploys and runs a test workflow, and then uses a script to check the output. If the test passes, the workflow is deployed:

steps:
# Deploy the test workflow with the commit sha
- id: 'deploy-test-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  args: ['workflows', 'deploy', '$_WORKFLOW_NAME-$BRANCH_NAME-$SHORT_SHA', '--source', 'gitops/workflow.yaml']

# Run the test workflow and capture the output
- id: 'run-test-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: 'bash'
  args: ['-c', 'gcloud workflows run $_WORKFLOW_NAME-$BRANCH_NAME-$SHORT_SHA > /workspace/testoutput.log']

# Delete the test workflow
- id: 'delete-test-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  args: ['workflows', 'delete', '$_WORKFLOW_NAME-$BRANCH_NAME-$SHORT_SHA', '--quiet']

# Check the test output
- id: 'check-test-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: 'bash'
  args: ['gitops/test-$BRANCH_NAME.sh']

# Deploy the workflow
- id: 'deploy-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  args: ['workflows', 'deploy', '$_WORKFLOW_NAME-$BRANCH_NAME', '--source', 'gitops/workflow.yaml']

The $BRANCH_NAME and $SHORT_SHA substitution variables are populated by Cloud Build when a build is triggered from a Git repository. They represent the name of your branch, and the first seven characters of the commit ID associated with your build, respectively.

The $_WORKFLOW_NAME substitution variable allows you to re-use a config file with different variable values. You can specify its value when you create the build trigger.

For more information, see Create a build configuration file.

Create a build trigger

You can automate the deployment of your workflow by creating a Cloud Build trigger.

To create a build trigger for the config file in the previous section:

In the Google Cloud console, go to the Cloud Build Triggers page:

Go to Triggers
Click Create trigger.
In the Name field, enter a name for your trigger.
For Event, select the event to invoke your trigger.

For example: Push to a branch
For Source, select your repository, and the branch or tag name that will start your trigger. You can use a regular expression to specify a match to a branch or tag.

For example: GoogleCloudPlatform/workflows-demos (repository) and ^main$|^staging$ (matches the main and staging branches)
Expand the Show included and ignored files filters section and specify your workflow as an included file so that when it is changed, a build is invoked.

For example: gitops/workflow.yaml
For Configuration, select Cloud Build configuration file (YAML or JSON) as the type, and Repository as the location.
In the Cloud Build configuration file location field, specify the location of your file.

For example: gitops/cloudbuild.yaml
Optionally, to add a substitution variable, click Add variable and specify a key and value combination.

For example: _WORKFLOW_NAME (variable) and workflows-gitops (value)
To save your build trigger, click Create.

When any changes are pushed to a workflow in the specified branch of the Git repository, it will automatically trigger Cloud Build to deploy the workflow.

For more information, see Create and manage build triggers.

Test the build trigger

You can test the build trigger and config file from the previous sections.

In the staging branch of the Git repository, edit workflow.yaml, and change Hello World to Bye World:

main:
  steps:
    - init:
        assign:
          - message: "Hello World"
    - returnResult:
        return: ${message}

Commit and push the change to the staging branch.
```
git add workflow.yaml
git commit -m "Update workflow.yaml in staging"
git push
```
The Cloud Build trigger runs and initiates a build.
To confirm the success of the build, in the Google Cloud console, go to the Build history page:

Go to Build history

After a build completes, Cloud Build provides an overall status for the build and for each individual build step. For more information, see View build results.
To confirm that a staging workflow is deployed, in the Google Cloud console, go to the Workflows page:

Go to Workflows

You should see a workflow named workflows-gitops-staging listed.

To deploy the staging workflow to production, merge the staging branch to the main branch:

git checkout main
git merge staging
git push

Note that because test-main.sh is expecting Hello World in the output of the workflow, the build will fail:

RESULT_EXPECTED="result: '\"Hello World\"'"
RESULT_ACTUAL=$(grep "result: " $FILE)
if [[ $RESULT_EXPECTED == $RESULT_ACTUAL ]]; then
  echo "Result test passed"
else
  echo "Result test failed. Expected: $RESULT_EXPECTED Actual: $RESULT_ACTUAL"; exit 1;
fi

To successfully deploy a production workflow, in the staging branch, edit workflow.yaml again and change the string back to Hello World.
Commit and push the change to the staging branch and then merge the staging branch to the main branch.
To confirm that a production workflow is deployed, in the Google Cloud console, go to the Workflows page:

Go to Workflows

You should see a workflow named workflows-gitops-main listed.

What's next

Learn more about Cloud Build.
Learn how to troubleshoot build errors.
Learn how to troubleshoot Workflows.