Integrate the Connect gateway with Cloud Build

This is a basic tutorial on how to integrate Cloud Build with the Connect gateway, letting you create a CI/CD pipeline for GKE clusters running across many different environments.

This tutorial assumes that you are familiar with the previous sections in the Connect gateway guide, and are also familiar with Cloud Build. These instructions leverage the cloud-sdk builder image which requires some minor scripting (as you'll see below).

Before you begin

  • Ensure that you have the following command line tools installed:

    • The latest version of the Google Cloud CLI, which includes gcloud, the command line tool for interacting with Google Cloud.
    • kubectl

    If you are using Cloud Shell as your shell environment for interacting with Google Cloud, these tools are installed for you.

  • Ensure that you have initialized the gcloud CLI for use with your project.

  • Ensure the Connect gateway and other required APIs are enabled for your project, as described in the setup guide.

1. Grant IAM roles to the Cloud Build service account

By default, Cloud Build uses a Google Cloud service account to run all required work, with an address in the format MY_PROJECT_NUMBER @cloudbuild.gserviceaccount.com. You can find this service account email address for your project under Cloud Build - Settings in the Google Cloud console.

Screenshot of Cloud Build settings page

Follow the instructions in Grant IAM permissions in the gateway setup guide to grant this account the required roles in your project.

2. Specify RBAC policies for the Cloud Build service account

Follow the instructions in Configure RBAC policies in the gateway setup guide to give the Cloud Build service account appropriate permissions on all the clusters you want to use.

We strongly recommend using Policy Controller to deploy and maintain RBAC policies on multiple clusters.

3. Create a Cloud Build pipeline

The Cloud Build workflow needs a cloudbuild.yaml file to configure the pipeline. The following is a simple example that deploys a static manifest to two different clusters (one GKE cluster on Google Cloud, and one on VMware). You can find out more about how to configure a Cloud Build pipeline in the Cloud Build documentation.

steps:
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: bash
  id: Deploy to cluster on Google Cloud
  args:
  - '-c'
  - |
    set -x && \
    export KUBECONFIG="$(pwd)/gateway-kubeconfig" && \
    gcloud container fleet memberships get-credentials my-gke-cluster && \
    kubectl --kubeconfig gateway-kubeconfig apply -f myapp.yaml
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: bash
  id: Deploy to cluster on VMware
  args:
  - '-c'
  - |
    set -x && \
    export KUBECONFIG="$(pwd)/gateway-kubeconfig" && \
    gcloud container fleet memberships get-credentials my-vmware-cluster && \
    kubectl --kubeconfig gateway-kubeconfig apply -f myapp.yaml

You can put any desired workflow in myapp.yaml to configure clusters. Here is an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Once you push your configuration to your Git repository, the Cloud Build workflow deploys the required application to the specified clusters. You can also set up Cloud Build to detect changes in the linked Git repository to trigger automated application update or installation.

Advanced usage

Since it uses standard Cloud Build concepts, you can adapt and customize our example further to suit your particular CI/CD needs. In particular, if you want to build an image from scratch and deploy it in your pipeline, you can use the gke-deploy builder's prepare mode. For example, the following Cloud Build configuration:

  1. Builds a docker image from the Dockerfile in the root of the Git repo and tags it with the Git SHA.
  2. Pushes the tagged image to the project’s Container Registry.
  3. Prepares the Kubernetes manifests in the manifest directory by setting the correct image tag(s), placing the output manifests in output/expanded.
  4. Deploys to an GKE cluster on premises using the Connect gateway.
steps:
- name: 'gcr.io/cloud-builders/docker'
  id: "Build Container"
  args: ['build', '--tag=gcr.io/$PROJECT_ID/demo-app:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/docker'
  id: "Push to GCR"
  args: ['push', 'gcr.io/$PROJECT_ID/demo-app:$SHORT_SHA']
- name: "gcr.io/cloud-builders/gke-deploy"
  id: "Prepare Manifests"
  args:
  - prepare
  - --filename=manifests/
  - --image=gcr.io/$PROJECT_ID/demo-app:$SHORT_SHA
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: bash
  id: "Deploy to cluster on VMware
  args:
  - '-c'
  - |
    set -x && \
    export KUBECONFIG="$(pwd)/gateway-kubeconfig" && \
    gcloud container fleet memberships get-credentials my-vmware-cluster && \
    kubectl --kubeconfig=gateway-kubeconfig apply -f output/expanded

Note that in this example we had to create an image pull secret to authorize the on-premises GKE cluster to pull images from Container Registry.

For more ideas for Cloud Build usage, see the Cloud Build documentation.