Building, testing, and deploying artifacts

This page explains how to use Cloud Build's official builder images to write build configs to fetch source code and dependencies, and to build, test, and deploy artifacts.

If you already have a build config file and want to run your builds on Cloud Build, read Starting Builds Manually and Automating Builds.

Before you begin

• Read Build Configuration Overview to understand the structure of the Cloud Build build configs.

• Learn about the available cloud builders.

Fetching dependencies

Fetch source code or install dependencies by using a build step that runs tools such as docker, git, npm, and gsutil:

docker

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['pull', 'gcr.io/$PROJECT_ID/latest-image']

git

steps:
- name: 'gcr.io/cloud-builders/git'
  args: ['clone', 'https://github.com/GoogleCloudPlatform/cloud-builders']

npm

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']

gsutil

steps:
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://mybucket/remotefile.zip', 'localfile.zip']

Building container images

To build Docker container images, use the docker build step in which you can invoke docker commands. Arguments passed to this build step will be passed to docker directly, allowing you to run any docker command in this build step.

In your build config file, add instructions to:

• Call docker builder and pass the arguments to invoke the Docker build command. In the following build config, the Docker build step is called twice to build two images, for which the source code files are found in the current working directory at build time, as indicated by .. The source code for gcr.io/my-project/image2 is found in a directory inside the current working directory at build time, subdirectory.

• Add the images field to push the resulting images to Container Registry.

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image2', '.']
  dir: 'subdirectory'
images: ['gcr.io/my-project/image1', 'gcr.io/my-project/image2']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image1", "."]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image2", "."],
      "dir": "subdirectory"
    }
  ],
  "images": ["gcr.io/my-project/image1", "gcr.io/my-project/image2"]
}

Building non-container artifacts

Cloud Build provides supported build steps for common languages and tools that you can use to execute your builds.

To build non-container artifacts, add a build steps section that runs builders such as maven, gradle, go, or bazel:

maven

steps:
- name: 'gcr.io/cloud-builders/mvn'
  args: ['install']
- name: 'gcr.io/cloud-builders/mvn'
  args: ['package']

gradle

steps:
- name: 'gcr.io/cloud-builders/gradle'
  args: ['build']

go

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']

bazel

steps:
- name: 'gcr.io/cloud-builders/bazel'
  args: ['build', '//path/to:target']

Note: Cloud Build build steps are just Docker containers; therefore you can use Cloud Build with any build tool that runs in Docker. For instructions on creating a custom build step, see Creating Custom Build Steps.

Running unit tests and integration tests

If you have the source code available, you can run unit tests and integration tests as build steps.

Assume that you have a JavaScript application with unit tests, a Dockerfile that builds a Docker image, and integration tests that execute against that running image.

To run unit tests and integration tests for this application, add instructions for the following in your build config file:

• Install dependencies to execute the build.
• Run the unit test.
• Build the Docker image of the application.
• Run the application and dependencies in the background using Docker compose.
  • For integration tests to communicate with other containers, connect the containers created by the Docker compose stack to the cloudbuild network by adding the --network=cloudbuild flag to your Docker build step.
• Run the integration tests against the running Docker compose stack.

• Push the newly-built image to Container Registry.

  steps:
  - name: 'gcr.io/cloud-builders/npm'
    args: ['install']
  - name: 'gcr.io/cloud-builders/npm'
    args: ['run', 'test:unit']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '--network=cloudbuild', '-t', 'gcr.io/$PROJECT_ID/gcb-docker-compose:latest', '.']
  - name: 'docker/compose:1.15.0'
    args: ['up', '-d']
    env:
    - 'PROJECT_ID=$PROJECT_ID'
  - name: 'gcr.io/cloud-builders/npm'
    args: ['run', 'test:integration']
    env:
    - 'HOST=counter' # name of the running container
    - 'PORT=50051'
  images: ['gcr.io/$PROJECT_ID/gcb-docker-compose:latest']
  

Deploying artifacts

As part of your continuous deployment pipeline, Cloud Build can perform deployments using command-line tools.

Use the following steps to deploy to Google Kubernetes Engine (GKE), App Engine, Cloud Functions, and Firebase.

GKE

To deploy to GKE, call the gke-deploy build step:

1. Enable the GKE API.

2. Create a cluster.

3. Add the GKE role to the service account:

   1. Open the Cloud Build Settings page:

      Open the Cloud Build Settings page

      You'll see the Service account permissions page:

      

   2. Set the status of the Kubernetes Engine Developer role to Enabled.

4. Add one of the following series of build steps in your build config file:

   • Add these build steps if you are supplying your own Kubernetes configuration files that reference your image:

     steps:
     - name: 'gcr.io/cloud-builders/docker'
       args: ["build", "-t", "gcr.io/[PROJECT_ID]/[IMAGE]", "."]
     - name: 'gcr.io/cloud-builders/docker'
       args: ["push", "gcr.io/[PROJECT_ID]/[IMAGE]"]
     - name: 'gcr.io/cloud-builders/gke-deploy'
       args:
       - run
       - --filename=[KUBERNETES_CONFIG_FILE]
       - --image=gcr.io/[PROJECT_ID]/[IMAGE]:[TAG]
       - --location=[LOCATION]
       - --cluster=[CLUSTER]
     

   • Add these build steps if you do not have any Kubernetes configuration files for your image:

     steps:
     - name: 'gcr.io/cloud-builders/docker'
       args: ["build", "-t", "gcr.io/[PROJECT_ID]/[IMAGE]", "."]
     - name: 'gcr.io/cloud-builders/docker'
       args: ["push", "gcr.io/[PROJECT_ID]/[IMAGE]"]
     - name: 'gcr.io/cloud-builders/gke-deploy'
       args:
       - run
       - --image=gcr.io/[PROJECT_ID]/[IMAGE]:[TAG]
       - --location=[LOCATION]
       - --cluster=[CLUSTER]
     

     Note: It is recommended that you have the Kubernetes configuration files for your application in your repository. To help you get started, gke-deploy can provide Kubernetes configuration files for your image. To view the files gke-deploy creates for you, see an example YAML file in the cloud-builders repository.

This build calls the docker build step to create a Docker image and push it to Container Registry. Then, the build calls the gke-deploy build step to deploy to your GKE cluster.

To use this example build config, provide the following information:

• [PROJECT_ID], the ID for your project.
• [IMAGE], the desired name of the container image, usually the application name.
• [TAG], the tag of the container image.
  • If you are building a new container image with each commit, a good practice is to use the commit short SHA as a tag. Cloud Build makes this available as a default substitution, $SHORT_SHA.
• [KUBERNETES_CONFIG_FILE], the file path of your Kuberentes configuration file or the directory path containing your Kubernetes configuration files.
• [CLUSTER], the name of the GKE cluster that the application.
• [LOCATION], the region/zone of the cluster will be deployed to.

For more information on deploying images to GKE, see Deploying to GKE.

App Engine

To deploy an application from a container image to App Engine, call the gcloud build step with arguments for gcloud app deploy command:

1. Enable the App Engine API.

2. Add the App Engine role to the service account:

   1. Open the Cloud Build Settings page:

      Open the Cloud Build Settings page

      You'll see the Service account permissions page:

      

   2. Set the status of the App Engine Admin role to Enable.

3. Create a build config file that uses gcloud app deploy:

   steps:
   - name: 'gcr.io/cloud-builders/gcloud'
     args: ['app', 'deploy']
     timeout: '1600s'
   

The gcloud build step calls gcloud app deploy command to build a container image with your source code and then deploys the image on App Engine.

Note: Cloud Build build steps and builds have a default timeout of 10 minutes and App Engine deployments could take longer than that to complete. Add a timeout field to make sure that the build doesn't timeout if gcloud app deploy takes longer than 10 minutes to complete.

Cloud Functions

To deploy an application to Cloud Functions call the gcloud build step to invoke gcloud functions deploy:

1. Enable the Cloud Functions API.

2. Add the Cloud Functions role to the service account:

   1. Open the Cloud Build Settings page:

      Open the Cloud Build Settings page

      You'll see the Service account permissions page:

      

   2. Set the status of the Cloud Functions Developer role to Enable.

   3. Select GRANT ACCESS TO ALL SERVICE ACCOUNTS to grant the Service Account User role on all service accounts in the project on your page.

3. Create a build config file that uses gcloud functions deploy:

   steps:
   - name: 'gcr.io/cloud-builders/gcloud'
     args:
     - functions
     - deploy
     - [FUNCTION_NAME]
     - --source=.
     - --trigger-http
   

This builds the application source code and calls gcloud functions deploy to deploy the application Cloud Functions.

Cloud Run

To deploy an application to Cloud Run call the gcloud build step to invoke gcloud beta run deploy:

1. Enable the Cloud Run API.

2. Add the Cloud Run role to the service account:

   1. Open the Cloud Build Settings page:

      Open the Cloud Build Settings page

      You'll see the Service account permissions page:

      

   2. Set the status of the Cloud Run Admin role to Enable.

   3. Select GRANT ACCESS TO ALL SERVICE ACCOUNTS to grant the Service Account User role on all service accounts in the project on your page.

3. Create a build config file that uses gcloud beta run deploy:

   steps:
     # build the container image
   - name: 'gcr.io/cloud-builders/docker'
     args: ['build', '-t', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]', '.']
     # push the container image to Container Registry
   - name: 'gcr.io/cloud-builders/docker'
     args: ['push', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]']
     # Deploy container image to Cloud Run
   - name: 'gcr.io/cloud-builders/gcloud'
     args: ['beta', 'run', 'deploy', '[SERVICE-NAME]', '--image', 'gcr.io/$PROJECT_ID/[SERVICE-NAME]', '--region', '[REGION]', '--platform', 'managed']
   images:
   - gcr.io/$PROJECT_ID/[SERVICE-NAME]
   

Replace [SERVICE-NAME] and [REGION] with the name and region of the Cloud Run service you are deploying to.

If you are using Cloud Run on GKE, use gke instead of managed and --cluster and --cluster-location instead of the --region parameter.

This builds the application source code and calls gcloud run deploy to deploy the application Cloud Run.

Note: To make the deployed service public, make a one-time change to the service IAM settings.

For more information on deploying images to Cloud Run, see Deploying to Cloud Run.

Firebase

You can use Firebase in your builds by creating a Firebase custom build step.

To create a Firebase build step, create a Dockerfile and a build config file from the following examples.

Create a Dockerfile with the following contents. This installs the Firebase command-line tool, firebase, when called by the build:

# use latest Node LTS (Boron)
FROM node:boron
# install Firebase CLI
RUN npm install -g firebase-tools

ENTRYPOINT ["/usr/local/bin/firebase"]

Create a build config named cloudbuild.yaml with the following contents. This build config uses the Dockerfile to containerize firebase-tools as firebase. Then, the build pushes the containerized image to Container Registry for use in later builds:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/[PROJECT_ID]/firebase', '.' ]
images:
- 'gcr.io/[PROJECT_ID]/firebase'

With the Dockerfile and cloudbuild.yaml in the same directory, run the following command from your shell or terminal window:

gcloud builds submit --config=cloudbuild.yaml .

The gcloud builds command submits a build to Cloud Build which uses the cloudbuild.yaml build config and the source in the current directory (denoted by .). The build installs the firebase tool in the environment. Then, the tool is containerized and pushed to your Container Registry.

To call firebase in your builds, add the gcr.io/[PROJECT_ID]/firebase build step to your build config:

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'test' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run', 'build.prod' ]
- name: 'gcr.io/[PROJECT_ID]/firebase'
  args: [ 'deploy', '-P', 'js-demo-fe-staging', '--token', '[FIREBASE_TOKEN]']

To use this build config, provide the [FIREBASE_TOKEN] value, which is the Firebase authentication token you would have generated.

Examples of build config files

Build triggered from GitHub

The following config shows a simple build that is triggered from GitHub. This type of configuration is typically used in a CI/CD pipeline.

In this example:

• The npm build step is called to install the dependencies and run unit tests.
• The docker build step is called to build a Docker image of the application and push the image to Container Registry.

• The gke-deploy build step is called to deploy the built image to the Kubernetes cluster.

YAML

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
- name: 'gcr.io/cloud-builders/docker'
  args: ["build", "-t", "gcr.io/my-project/my-image:$REVISION_ID", "."]
- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/my-project/my-image:$REVISION_ID"]
- name: 'gcr.io/cloud-builders/gke-deploy'
  args:
  - 'run'
  - '--image=gcr.io/my-project/my-image:$REVISION_ID'
  - '--location=us-east4-b'
  - '--cluster=my-cluster'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "install"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project/my-image:$REVISION_ID",
            "."
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "push",
            "gcr.io/my-project/my-image:$REVISION_ID"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/gke-deploy",
        "args": [
            "run",
            "--image=gcr.io/my-project/my-image:$REVISION_ID",
            "--location=us-east4-b",
            "--cluster=my-cluster"
        ]
    }
    ]
}

Writing build requests without pushing images

You can use Cloud Build to perform arbitrary tasks without producing Docker images.

The following example:

• Uses docker to build an analysis tool
• Pulls in some data from the data-to-analyze directory

• Pushes the analysis results to a Cloud Storage bucket

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'analyzer', '.']

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://my-data-warehouse/data-to-analyze.tgz', '.']

- name: 'debian'
  args: ['tar', '-xzf', 'data-to-analyze.tgz']

- name: 'analyzer'
  args: ['--output=results.csv']
  dir: 'data-to-analyze'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'data-to-analyze/results.csv', 'gs://my-data-warehouse/results.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "analyzer",
        "."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://my-data-warehouse/data-to-analyze.tgz",
        "."
      ],
    },
    {
      "name": "debian",
      "args": [
        "tar",
        "-xzf",
        "data-to-analyze.tgz"
      ],
    },
    {
      "name": "analyzer",
      "args": [
        "--output=results.csv"
      ],
      "dir": "data-to-analyze"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "data-to-analyze/results.csv",
        "gs://my-data-warehouse/results.tgz"
      ]
    }
  ]
}

What's next

• Learn how to store your built images and artifacts.
• Learn how to run builds manually or automate using build triggers.
• Learn how gke-deploy can expose your application to the Internet with the --expose flag.