Building, testing, and deploying artifacts

This page explains how to use Cloud Build's official builder images to write build configs to fetch source code and dependencies, and to build, test, and deploy artifacts.

If you already have a build config file and want to run your builds on Cloud Build, read Starting Builds Manually and Automating Builds.

Before you begin

• Read Build Configuration Overview to understand the structure of the Cloud Build build configs.

• Learn about the available cloud builders.

Fetching dependencies

Fetch source code or install dependencies by using a build step that runs tools such as docker, git, npm, and gsutil:

docker

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['pull', 'gcr.io/$PROJECT_ID/latest-image']

git

steps:
- name: 'gcr.io/cloud-builders/git'
  args: ['clone', 'https://github.com/GoogleCloudPlatform/cloud-builders']

npm

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']

gsutil

steps:
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://mybucket/remotefile.zip', 'localfile.zip']

Building container images

To build Docker container images, use the docker build step in which you can invoke docker commands. Arguments passed to this build step will be passed to docker directly, allowing you to run any docker command in this build step.

In your build config file, add instructions to:

• Call docker builder and pass the arguments to invoke the Docker build command. In the following build config, the Docker build step is called twice to build two images, for which the source code files are found in the current working directory at build time, as indicated by .. The source code for gcr.io/my-project/image2 is found in a directory inside the current working directory at build time, subdirectory.

• Add the images field to push the resulting images to Container Registry.

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image2', '.']
  dir: 'subdirectory'
images: ['gcr.io/my-project/image1', 'gcr.io/my-project/image2']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image1", "."]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image2", "."],
      "dir": "subdirectory"
    }
  ],
  "images": ["gcr.io/my-project/image1", "gcr.io/my-project/image2"]
}

Building non-container artifacts

Cloud Build provides supported build steps for common languages and tools that you can use to execute your builds.

To build non-container artifacts, add a build steps section that runs builders such as maven, gradle, go, or bazel:

maven

steps:
- name: 'gcr.io/cloud-builders/mvn'
  args: ['install']
- name: 'gcr.io/cloud-builders/mvn'
  args: ['package']

gradle

steps:
- name: 'gcr.io/cloud-builders/gradle'
  args: ['build']

go

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']

bazel

steps:
- name: 'gcr.io/cloud-builders/bazel'
  args: ['build', '//path/to:target']

Note: Cloud Build build steps are just Docker containers; therefore you can use Cloud Build with any build tool that runs in Docker. For instructions on creating a custom build step, see Creating Custom Build Steps.

Running unit tests and integration tests

If you have the source code available, you can run unit tests and integration tests as build steps.

Assume that you have a JavaScript application with unit tests, a Dockerfile that builds a Docker image, and integration tests that execute against that running image.

To run unit tests and integration tests for this application, add instructions for the following in your build config file:

• Install dependencies to execute the build.
• Run the unit test.
• Build the Docker image of the application.
• Run the application and dependencies in the background using Docker compose.
• Run the integration tests against the running Docker compose stack.

• Push the newly-built image to Container Registry.

  steps:
  - name: 'gcr.io/cloud-builders/npm'
    args: ['install']
  - name: 'gcr.io/cloud-builders/npm'
    args: ['run', 'test:unit']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/gcb-docker-compose:latest', '.']
  - name: 'docker/compose:1.15.0'
    args: ['up', '-d']
    env:
    - 'PROJECT_ID=$PROJECT_ID'
  - name: 'gcr.io/cloud-builders/npm'
    args: ['run', 'test:integration']
    env:
    - 'HOST=counter' # name of the running container
    - 'PORT=50051'
  images: ['gcr.io/$PROJECT_ID/gcb-docker-compose:latest']
  

Deploying artifacts

As part of your continuous deployment pipeline, Cloud Build can perform deployments using command-line tools.

Use the following steps to deploy to GKE, App Engine, Cloud Functions, and Firebase.

Kubernetes Engine

To deploy to GKE with kubectl, call the kubectl build step to update a Deployment resource:

1. Enable the GKE API.
2. Add GKE IAM role:
   1. In GCP Console, visit the IAM menu.
   2. From the list of service accounts, click the Roles drop-down menu beside the Cloud Build [YOUR-PROJECT-NUMBER]@cloudbuild.gserviceaccount.com service account.
   3. Click Kubernetes Engine, then click Kubernetes Engine Admin.
   4. Click Save.

3. Add the kubectl build step in your build config:

   steps:
   - name: 'gcr.io/cloud-builders/docker'
     args: ["build", "-t", "gcr.io/[PROJECT-ID]/[IMAGE]", "."]
   - name: 'gcr.io/cloud-builders/docker'
     args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
   - name: 'gcr.io/cloud-builders/kubectl'
     args:
     - set
     - image
     - deployment
     - [DEPLOYMENT-NAME]
     - [CONTAINER]=gcr.io/[PROJECT-ID]/[IMAGE]:[TAG]
     env:
     - 'CLOUDSDK_COMPUTE_ZONE=[COMPUTE-ZONE]'
     - 'CLOUDSDK_CONTAINER_CLUSTER=[CLUSTER]'
   

This build calls the docker build step to create a Docker image and push it to Container Registry. Then, the build calls the kubectl build step to update a Deployment resource.

To use this example build config, provide the following information:

• [PROJECT-ID], the GCP project project ID
• [IMAGE], the name of the image, and its [TAG] as the desired version or the commit SHA
• [CLUSTER], the name of the cluster for which authentication credentials are requested
• [COMPUTE-ZONE], the project's compute zone
• [DEPLOYMENT-NAME], the name of the Kubernetes Deployment resource to be updated by kubectl set

App Engine

To deploy an application from a container image to App Engine, call the gcloud build step with arguments for gcloud app deploy command:

1. Enable the App Engine API.

2. Grant App Engine IAM to the Cloud Build service account:

   1. In GCP Console, visit the IAM menu.
   2. In the list of members, look for [YOUR-PROJECT-NUMBER]@cloudbuild.gserviceaccount.com.
   3. Click on the pencil icon in that row to grant a new role to that account.
   4. Click Add another role.
   5. Select App Engine, then click App Engine Admin.
   6. Click Save.

   7. Create a build config file that uses gcloud app deploy:

      steps:
      - name: 'gcr.io/cloud-builders/gcloud'
        args: ['app', 'deploy']
      timeout: '1600s'
      

The gcloud build step calls gcloud app deploy command to build a container image with your source code and then deploys the image on App Engine.

Note: Cloud Build build steps and builds have a default timeout of 10 minutes and App Engine deployments could take longer than that to complete. Add a timeout field to make sure that the build doesn't timeout if gcloud app deploy takes longer than 10 minutes to complete.

Cloud Functions

To deploy an application to Cloud Functions call the gcloud build step to invoke gcloud functions deploy:

1. Enable the Cloud Functions API.
2. Grant the Cloud Functions Developer role to the Cloud Build service account:
   1. In GCP Console, visit the IAM menu.
   2. In the list of members, look for [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com.
   3. Click on the pencil icon in that row to grant a new role to that account.
   4. Click Add another role.
   5. Select Cloud Functions and then Cloud Functions Developer.
   6. Click Save.
3. Grant the IAM Service Account User role to the Cloud Build service account on the Cloud Functions runtime service account:
   1. In GCP Console, visit the Service Accounts menu.
   2. In the list of members, look for [PROJECT_ID]@appspot.gserviceaccount.com.
   3. Select the above service account and click SHOW INFO PANEL in the top right corner.
   4. In the Permissions panel, click the Add Member button.
   5. Enter the Cloud Build service account ([PROJECT_NUMBER]@cloudbuild.gserviceaccount.com) in the New Member input field.
   6. In the Role dropdown, select Service Accounts and then Service Account User.
   7. Click Save.

4. Create a build config file that uses gcloud functions deploy:

   steps:
   - name: 'gcr.io/cloud-builders/gcloud'
     args:
     - functions
     - deploy
     - [FUNCTION_NAME]
     - --source=.
     - --trigger-http
   

This builds the application source code and calls gcloud functions deploy to deploy the application Cloud Functions.

Cloud Run

To deploy an application to Cloud Run call the gcloud build step to invoke gcloud beta run deploy:

1. Enable the Cloud Run API.
2. Grant the Cloud Run Admin role to the Cloud Build service account:
   1. In GCP Console, access the IAM menu.
   2. In the list of members, locate and select [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com.
   3. Click on EDIT (pencil) button in that row to grant a new role to that account.
   4. Click Add another role.
   5. Select Cloud Run, and then Cloud Run Admin.
   6. Click Save.
3. Grant the IAM Service Account User role to the Cloud Build service account on the Cloud Run runtime service account:
   1. In GCP Console, access the Service Accounts menu.
   2. In the list of members, locate and select [PROJECT_NUMBER][e-mail].
   3. Click SHOW INFO PANEL in the top right corner.
   4. In the Permissions panel, click the Add Member button.
   5. Enter the Cloud Build service account ([PROJECT_NUMBER]@cloudbuild.gserviceaccount.com) in the New Member input field.
   6. In the Role dropdown, select Service Accounts, and then Service Account User.
   7. Click Save.

4. Create a build config file that uses gcloud beta run deploy:

   steps:
     # build the container image
   - name: 'gcr.io/cloud-builders/docker'
     args: ['build', '-t', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]', '.']
     # push the container image to Container Registry
   - name: 'gcr.io/cloud-builders/docker'
     args: ['push', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]']
     # Deploy container image to Cloud Run
   - name: 'gcr.io/cloud-builders/gcloud'
     args: ['beta', 'run', 'deploy', '[SERVICE-NAME]', '--image', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]', '--region', '[REGION]']
   images:
   - gcr.io/$PROJECT_ID/[SERVICE-NAME]
   

Replace [SERVICE-NAME] and [REGION] with the name and region of the Cloud Run service you are deploying to.

If you are using Cloud Run on GKE, use --cluster and --cluster-location instead of the --region parameter.

This builds the application source code and calls gcloud run deploy to deploy the application Cloud Run.

Note: To make the deployed service public, make a one-time change to the service IAM settings.

Firebase

You can use Firebase in your builds by creating a Firebase custom build step.

To create a Firebase build step, create a Dockerfile and a build config file from the following examples.

Create a Dockerfile with the following contents. This installs the Firebase command-line tool, firebase, when called by the build:

# use latest Node LTS (Boron)
FROM node:boron
# install Firebase CLI
RUN npm install -g firebase-tools

ENTRYPOINT ["/usr/local/bin/firebase"]

Create a build config named cloudbuild.yaml with the following contents. This build config uses the Dockerfile to containerize firebase-tools as firebase. Then, the build pushes the containerized image to Container Registry for use in later builds:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/[PROJECT_ID]/firebase', '.' ]
images:
- 'gcr.io/[PROJECT_ID]/firebase'

With the Dockerfile and cloudbuild.yaml in the same directory, run the following command from your shell or terminal window:

gcloud builds submit --config=cloudbuild.yaml .

The gcloud builds command submits a build to Cloud Build which uses the cloudbuild.yaml build config and the source in the current directory (denoted by .). The build installs the firebase tool in the environment. Then, the tool is containerized and pushed to your Container Registry.

To call firebase in your builds, add the gcr.io/[PROJECT_ID]/firebase build step to your build config:

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'test' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run', 'build.prod' ]
- name: 'gcr.io/[PROJECT_ID]/firebase'
  args: [ 'deploy', '-P', 'js-demo-fe-staging', '--token', '[FIREBASE_TOKEN]']

To use this build config, provide the [FIREBASE_TOKEN] value, which is the Firebase authentication token you would have generated.

Examples of build config files

Build triggered from GitHub

The following config shows a simple build that is triggered from GitHub. This type of configuration is typically used in a CI/CD pipeline.

In this example:

• The npm build step is called to install the dependencies and run unit tests.
• The docker build step is called to build a Docker image of the application and push the image to Container Registry.

• The kubectl build step is called to deploy the built image to the Kubernetes cluster.

YAML

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
- name: 'gcr.io/cloud-builders/docker'
  args: ["build", "-t", "gcr.io/my-project/my-image:$REVISION_ID", "."]
- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/my-project/my-image:$REVISION_ID"]
- name: 'gcr.io/cloud-builders/kubectl'
  args:
  - 'set'
  - 'image'
  - 'deployment/my-deployment'
  - 'my-container=gcr.io/my-project/my-image:$REVISION_ID'
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-east4-b'
  - 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "install"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project/my-image:$REVISION_ID",
            "."
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "push",
            "gcr.io/my-project/my-image:$REVISION_ID"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/kubectl",
        "args": [
            "set",
            "image",
            "deployment/my-deployment",
            "frontend=gcr.io/my-project/my-image:$REVISION_ID"
        ],
        "env": [
            "CLOUDSDK_COMPUTE_ZONE=us-east4-b",
            "CLOUDSDK_CONTAINER_CLUSTER=my-cluster"
        ]
    }
    ]
}

Writing build requests without pushing images

You can use Cloud Build to perform arbitrary tasks without producing Docker images.

The following example:

• Uses docker to build an analysis tool
• Pulls in some data from the data-to-analyze directory

• Pushes the analysis results to a Cloud Storage bucket

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'analyzer', '.']

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://my-data-warehouse/data-to-analyze.tgz', '.']

- name: 'debian'
  args: ['tar', '-xzf', 'data-to-analyze.tgz']

- name: 'analyzer'
  args: ['--output=results.csv']
  dir: 'data-to-analyze'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'data-to-analyze/results.csv', 'gs://my-data-warehouse/results.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "analyzer",
        "."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://my-data-warehouse/data-to-analyze.tgz",
        "."
      ],
    },
    {
      "name": "debian",
      "args": [
        "tar",
        "-xzf",
        "data-to-analyze.tgz"
      ],
    },
    {
      "name": "analyzer",
      "args": [
        "--output=results.csv"
      ],
      "dir": "data-to-analyze"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "data-to-analyze/results.csv",
        "gs://my-data-warehouse/results.tgz"
      ]
    }
  ]
}

What's next

• Learn how to store your built images and artifacts.
• Learn how to run builds manually or automate using build triggers.