This document shows you how to implement a deployment pipeline for Cloud Run that implements a progression of code from developer branches to production with automated canary testing and percentage-based traffic management. It is intended for platform administrators who are responsible for creating and managing CI/CD pipelines. This document assumes that you have a basic understanding of Git, Cloud Run, and CI/CD pipeline concepts.
Cloud Run lets you deploy and run your applications with little overhead or effort. Many organizations use robust release pipelines to move code into production. Cloud Run provides unique traffic management capabilities that let you implement advanced release management techniques with little effort.
Objectives
- Create your Cloud Run service
- Enable a developer branch
- Implement canary testing
- Roll out safely to production
Costs
In this document, you use the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage,
use the pricing calculator.
Before you begin
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
In the Google Cloud console, activate Cloud Shell.
At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
Preparing your environment
In Cloud Shell, create environment variables to use throughout this tutorial:
export PROJECT_ID=$(gcloud config get-value project) export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
Enable the following APIs:
- Resource Manager
- GKE
- Cloud Build
- Container Registry
- Cloud Run
gcloud services enable \ cloudresourcemanager.googleapis.com \ container.googleapis.com \ secretmanager.googleapis.com \ cloudbuild.googleapis.com \ containerregistry.googleapis.com \ run.googleapis.com
Grant the Cloud Run Admin role (
roles/run.admin
) to the Cloud Build service accountgcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$PROJECT_NUMBER@cloudbuild.gserviceaccount.com \ --role=roles/run.admin
Grant the IAM Service Account User role (
roles/iam.serviceAccountUser
) to the Cloud Build service account for the Cloud Run runtime service accountgcloud iam service-accounts add-iam-policy-binding \ $PROJECT_NUMBER-compute@developer.gserviceaccount.com \ --member=serviceAccount:$PROJECT_NUMBER@cloudbuild.gserviceaccount.com \ --role=roles/iam.serviceAccountUser
Set Git values
If you haven't used Git in Cloud Shell previously, set the user.name
and
user.email
values that you want to use:
git config --global user.email YOUR_EMAIL_ADDRESS
git config --global user.name YOUR_USERNAME
git config --global credential.helper store
YOUR_EMAIL_ADDRESS
: The email used with your GitHub accountYOUR_USERNAME
: Your GitHub user ID
If you're using MFA with GitHub, create a personal access token and use it as your password when interacting with GitHub through the command-line.
- Follow this link to create an access token.
- Leave the GitHub page open.
Store your GitHub user ID in an environment variable for easier access:
export GH_USER=YOUR_GITHUB_ID
Fork the project repository
To create your own writeable version of the lab repository, fork the sample repository into your GitHub account through the GitHub UI.
Clone the sample repository
Clone and prepare the sample repository:
git clone https://github.com/$GH_USER/software-delivery-workshop.git cloudrun-progression
cd cloudrun-progression/labs/cloudrun-progression
Connect your Git repository.
Cloud Build lets you create and manage connections to source code repositories using the Google Cloud console. You can create and manage connections using either the first generation or second generation of Cloud Build repositories. This tutorial uses second generation Cloud Build repositories.
Grant required permissions
To connect your GitHub host, grant the Cloud Build Connection Admin (roles/cloudbuild.connectionAdmin
) role to your user account:
PN=$(gcloud projects describe ${PROJECT_ID} --format="value(projectNumber)")
CLOUD_BUILD_SERVICE_AGENT="service-${PN}@gcp-sa-cloudbuild.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
--member="serviceAccount:${CLOUD_BUILD_SERVICE_AGENT}" \
--role="roles/secretmanager.admin"
Create the host connection
Configure the Cloud Build repository connection:
gcloud alpha builds connections create github $GH_USER --region=us-central1
Click the link provided in the output and follow the onscreen instructions to complete the connection.
Verify the installation of your GitHub connection:
gcloud alpha builds connections describe $GH_USER --region=us-central1
Link the sample repository
Using the host connection you just configured, link in the sample repository you forked:
gcloud alpha builds repositories create cloudrun-progression \
--remote-uri=https://github.com/$GH_USER/software-delivery-workshop.git \
--connection=$GH_USER \
--region=us-central1
Set repository name variable
Store the repository name for later use:
export REPO_NAME=projects/$PROJECT_ID/locations/us-central1/connections/$GH_USER/repositories/cloudrun-progression
Deploy your Cloud Run service
In this section, you build and deploy the initial production application that you use throughout this tutorial.
Deploy the service
In Cloud Shell, build and deploy the application, including a service that requires authentication. To make a public service use the
--allow-unauthenticated
flag.gcloud builds submit --tag gcr.io/$PROJECT_ID/hello-cloudrun gcloud run deploy hello-cloudrun \ --image gcr.io/$PROJECT_ID/hello-cloudrun \ --platform managed \ --region us-central1 \ --tag=prod -q
The output looks like the following:
Deploying container to Cloud Run service [hello-cloudrun] in project [sdw-mvp6] region [us-central1] ✓ Deploying new service... Done. ✓ Creating Revision... ✓ Routing traffic... Done. Service [hello-cloudrun] revision [hello-cloudrun-00001-tar] has been deployed and is serving 100 percent of traffic. Service URL: https://hello-cloudrun-apwaaxltma-uc.a.run.app The revision can be reached directly at https://prod---hello-cloudrun-apwaaxltma-uc.a.run.app
The output includes the service URL and a unique URL for the revision. Your values will differ slightly from what's indicated here.
Validate the deployment
After the deployment is complete, view the newly deployed service on the Revisions page in the Cloud Console.
In Cloud Shell, view the authenticated service response:
PROD_URL=$(gcloud run services describe hello-cloudrun --platform managed --region us-central1 --format=json | jq --raw-output ".status.url") echo $PROD_URL curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $PROD_URL
Enabling Branch Deployments
In this section, you enable a unique URL for development branches in Git.
Setup Branch Trigger
Each branch is represented by a URL identified by the branch name. Commits to the branch trigger a deployment, and the updates are accessible at that same URL.
In Cloud Shell, set up the trigger:
gcloud alpha builds triggers create github \ --name=branchtrigger \ --repository=$REPO_NAME \ --branch-pattern='[^(?!.*main)].*' \ --build-config=labs/cloudrun-progression/branch-cloudbuild.yaml \ --region=us-central1
To review the trigger, go to the Cloud Build Triggers page in the Cloud Console.
Create changes on a branch
In Cloud Shell, create a new branch:
git checkout -b new-feature-1
Open the sample application using your favorite editor or using the Cloud Shell IDE:
edit app.py
In the sample application, modify line 24 to indicate v1.1 instead of v1.0:
@app.route('/') def hello_world(): return 'Hello World v1.1'
To return to your terminal, click Open Terminal.
Execute the branch trigger
In Cloud Shell, commit the change and push to the remote repository:
git add . && git commit -m "updated" && git push origin new-feature-1
To review the build in progress, go to the Cloud Build Build history screen.
To review the new revision, after the build completes go to the Cloud Run Revisions page in the Cloud Console:
In Cloud Shell, get the unique URL for this branch:
BRANCH_URL=$(gcloud run services describe hello-cloudrun --platform managed --region us-central1 --format=json | jq --raw-output ".status.traffic[] | select (.tag==\"new-feature-1\")|.url") echo $BRANCH_URL
Access the authenticated URL:
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $BRANCH_URL
The updated response output looks like the following:
Hello World v1.1
Automate canary testing
When code is released to production, it's common to release code to a small subset of live traffic before migrating all traffic to the new code base.
In this section, you implement a trigger that is activated when code is committed to the main branch. The trigger deploys the code to a unique canary URL and it routes 10% of all live traffic to the new revision.
In Cloud Shell, set up the branch trigger:
gcloud alpha builds triggers create github \ --name=maintrigger \ --repository=$REPO_NAME \ --branch-pattern=main \ --build-config=labs/cloudrun-progression/main-cloudbuild.yaml \ --region=us-central1
To review the new trigger, go to the Cloud Build Triggers page in the Cloud Console.
In Cloud Shell, merge the branch to the main line and push to the remote repository:
git checkout main git merge new-feature-1 git push origin main
To review the build in progress, go to the Cloud Build Builds page.
After the build is complete, to review the new revision, go to the Cloud Run Revisions page in the Cloud Console. Note that 90% of the traffic is routed to prod, 10% to canary, and 0% to the branch revisions.
Review the key lines of main-cloudbuild.yaml
that implement the logic for the canary deploy.
Lines 39-45 deploy the new revision and use the tag flag to route traffic from the unique canary URL:
gcloud run deploy ${_SERVICE_NAME} \
--platform managed \
--region ${_REGION} \
--image gcr.io/${PROJECT_ID}/${_SERVICE_NAME} \
--tag=canary \
--no-traffic
Line 61 adds a static tag to the revision that notes the Git short SHA of the deployment:
gcloud beta run services update-traffic ${_SERVICE_NAME} --update-tags=sha-$SHORT_SHA=$${CANARY} --platform managed --region ${_REGION}
Line 62 updates the traffic to route 90% to production and 10% to canary:
gcloud run services update-traffic ${_SERVICE_NAME} --to-revisions=$${PROD}=90,$${CANARY}=10 --platform managed --region ${_REGION}
In Cloud Shell, get the unique URL for the canary revision:
CANARY_URL=$(gcloud run services describe hello-cloudrun --platform managed --region us-central1 --format=json | jq --raw-output ".status.traffic[] | select (.tag==\"canary\")|.url") echo $CANARY_URL
Review the canary endpoint directly:
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $CANARY_URL
To see percentage-based responses, make a series of requests:
LIVE_URL=$(gcloud run services describe hello-cloudrun --platform managed --region us-central1 --format=json | jq --raw-output ".status.url") for i in {0..20};do curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $LIVE_URL; echo \n done
Releasing to Production
After the canary deployment is validated with a small subset of traffic, you release the deployment to the remainder of the live traffic.
In this section, you set up a trigger that is activated when you create a tag in the repository. The trigger migrates 100% of traffic to the already deployed revision based on the commit SHA of the tag. Using the commit sha ensures the revision validated with canary traffic is the revision utilized for the remainder of production traffic.
In Cloud Shell, set up the tag trigger:
gcloud alpha builds triggers create github \ --name=tagtrigger \ --repository=$REPO_NAME \ --tag-pattern=. \ --build-config=labs/cloudrun-progression/tag-cloudbuild.yaml \ --region=us-central1
To review the new trigger, go to the Cloud Build Triggers page in the Cloud Console.
In Cloud Shell, create a new tag and push to the remote repository:
git tag 1.1 git push origin 1.1
To review the build in progress, go to the Cloud Build Build history screen in the Cloud Console.
After the build is complete, to review the new revision, go to the Cloud Run Revisions page in the Cloud Console. Note that the revision is updated to indicate the prod tag and it is serving 100% of live traffic.
In Cloud Shell, to see percentage-based responses, make a series of requests:
LIVE_URL=$(gCloud Run services describe hello-cloudrun --platform managed --region us-central1 --format=json | jq --raw-output ".status.url") for i in {0..20};do curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $LIVE_URL; echo \n done
Review the key lines of
tag-cloudbuild.yaml
that implement the production deployment logic.Line 37 updates the canary revision adding the prod tag. The deployed revision is now tagged for both prod and canary:
gcloud beta run services update-traffic ${_SERVICE_NAME} --update-tags=prod=$${CANARY} --platform managed --region ${_REGION}
Line 39 updates the traffic for the base service URL to route 100% of traffic to the revision tagged as prod:
gcloud run services update-traffic ${_SERVICE_NAME} --to-revisions=$${NEW_PROD}=100 --platform managed --region ${_REGION}
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
What's next
- Review Managing revisions with Cloud Run.
- Review Cloud Run Rollbacks, gradual rollouts, and traffic migration.
- Review Using tags for accessing revisions.
- Review Creating and managing build triggers in Cloud Build.
- For more reference architectures, diagrams, and best practices, explore the Cloud Architecture Center.