Building, Testing, and Deploying Artifacts

This page explains how to use supported build steps provided by Container Builder to write build configs to fetch source code and dependencies, and to build, test, and deploy artifacts.

Before you begin

Calling the build steps

Pre-built images of the supported build steps are hosted at the following path:

gcr.io/cloud-builders/[BUILD-STEP]

Use the steps field in your build config file to call the build steps. For example, to build a Docker image, use the docker build step:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/my-image', '.']

The args field in a build step contains the list of arguments that are passed to the corresponding tool when the build step starts. Therefore in the above example, the args field contains the list of arguments for running the Docker build command.

Fetching dependencies

Fetch source code or install dependencies by using a build step that runs tools such as docker, git, npm, and gsutil:

docker

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['pull', 'gcr.io/$PROJECT_ID/latest-image']

git

steps:
- name: gcr.io/cloud-builders/git
  args: ['clone', 'https://github.com/GoogleCloudPlatform/cloud-builders']

npm

steps:
- name: 'gcr.io/$PROJECT_ID/npm'
  args: ['install']

gsutil

steps:
- name: gcr.io/cloud-builders/gsutil
  args: ['cp', 'gs://mybucket/remotefile.zip', 'localfile.zip']

Building container images

To build Docker container images, use the docker build step in which you can invoke docker commands. Arguments passed to this build step will be passed to docker directly, allowing you to run any docker command in this build step.

The following example demonstrates how to write a basic build config file to build a Docker image. In this example:

  • docker build step is called to build two images, for which the source code files are found in the current working directory at build time, as indicated by .
  • The source code for gcr.io/my-project/image2 is found in a directory inside the current working directory at build time, subdirectory
  • The resulting images are pushed to Container Registry.

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image2', '.']
  dir: 'subdirectory'
images: ['gcr.io/my-project/image1', 'gcr.io/my-project/image2']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image1", "."]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image2", "."],
      "dir": "subdirectory"
    }
  ],
  "images": ["gcr.io/my-project/image1", "gcr.io/my-project/image2"]
}

Building non-container artifacts

Container Builder provides supported build steps for common languages and tools that you can use to execute your builds.

The following examples show builds that use maven, gradle, go, and bazel builders.

maven

steps:
- name: 'gcr.io/cloud-builders/mvn'
  args: ['install']
- name: 'gcr.io/cloud-builders/mvn'
  args: ['package']

gradle

steps:
- name: 'gcr.io/cloud-builders/gradle'
  args: ['build']

go

- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']

bazel

steps:
- name: gcr.io/cloud-builders/bazel
  args: ['build', '//path/to:target']

Container Builder build steps are just Docker containers; therefore you can use Container Builder with any build tool that runs in Docker. For instructions on creating a custom build step, see Creating Custom Build Steps.

Running unit tests and integration tests

If you have the source code available, you can run unit tests and integration tests as build steps.

Assume that you have a JavaScript application with unit tests, a Dockerfile that builds a Docker image, and integration tests that execute against that running image. The following code shows the build config for that application.

This build config:

  • Installs dependencies to execute the build.
  • Runs the unit test.
  • Builds the Docker image of the application.
  • Runs the application and dependencies in the background using Docker compose.
  • Runs the integration tests against the running Docker compose stack.
  • Pushes the newly-built image to Container Registry.

    steps:
    - name: 'gcr.io/cloud-builders/npm'
      args: ['install']
    - name: 'gcr.io/cloud-builders/npm'
      args: ['run', 'test:unit']
    - name: 'gcr.io/cloud-builders/docker'
      args: ['build', '-t', 'gcr.io/$PROJECT_ID/gcb-docker-compose:latest', '.']
    - name: 'docker/compose:1.15.0'
      args: ['up', '-d']
      env:
      - 'PROJECT_ID=$PROJECT_ID'
    - name: 'gcr.io/cloud-builders/npm'
      args: ['run', 'test:integration']
      env:
      - 'HOST=counter' # name of the running container
      - 'PORT=50051'
    images: ['gcr.io/$PROJECT_ID/gcb-docker-compose:latest']
    

Pushing artifacts

When you're building a Docker image, you can push the image using either the images field or the Docker push argument.

The following example pushes the Docker image using the images field. The built image is saved in Container Registry. This method pushes the built images to Container Registry after the successful completion of all build steps.

steps:
- name: gcr.io/cloud-builders/docker
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/[IMAGE]', '.']
images: ['gcr.io/$PROJECT_ID/[IMAGE]']

If you want to push the image as part of your build flow, you can use the Docker push command:

- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]

The difference between using the images field and the Docker push command is that if you use the images field, the pushed image will show up in the build results. This includes the Build description page for a build in the GCP Console, the results of Build.get(), and the results of gcloud container images list. However, if you use the Docker push command to push the built image, the image will not show up in the build results.

If you want to push an image as part of your build flow and still show up in the build results, use both the Docker push command and the images field in your build config:

- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
images: ['gcr.io/$PROJECT_ID/[IMAGE']

If your built artifact is not a container image, you must manually export it. The following example pushes a Go binary to Google Cloud Storage:

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'my-package', 'gs://my-bucket/']

You can also push your artifacts to third-party repositories such as GitHub, DockerHub, and Artifactory. For instructions on pushing artifacts to GitHub, see Accessing Private GitHub Repositories.

Deploying artifacts

As part of your continuous deployment pipeline, Container Builder can perform "fire and forget" deployments using command-line tools.

The following example deployments use popular command-line tools.

kubernetes

To use the Kubernetes kubectl command-line tool in your builds, you must first allow the Container Builder service account to use the Kubernetes Engine API:

  1. Enable the Kubernetes Engine API.
  2. Add Kubernetes Engine IAM role:
  3. In GCP Console, visit the IAM menu.
  4. From the list of service accounts, click the Roles drop-down menu beside the Container Builder [YOUR-PROJECT-ID]@cloudbuild.gserviceaccount.com service account.
  5. Click Kubernetes Engine, then click Kubernetes Engine Admin.
  6. Click Save.

Now, you can use the kubectl build step in your builds. The following is an example build config that uses kubectl:

    steps:
    - name: 'gcr.io/cloud-builders/docker'
      args: ["build", "-t", "gcr.io/[PROJECT-ID]/[IMAGE]", "."]
    - name: 'gcr.io/cloud-builders/docker'
      args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
    - name: 'gcr.io/cloud-builders/kubectl'
      args:
      - set
      - image
      - deployment
      - [DEPLOYMENT-NAME]
      - [CONTAINER]=gcr.io/[PROJECT-ID]/[IMAGE]:[TAG]

This build calls the docker build step to create a Docker image and push it to Container Registry. Then, the build calls the kubectl build step to update a Deployment resource.

To use this example build config, provide the following information:

  • [PROJECT-ID], the GCP project project ID
  • [IMAGE], the name of the image, and its [TAG] as the desired version or the commit SHA
  • [CLUSTER], the name of the cluster for which authentication credentials are requested
  • [COMPUTE-ZONE], the project's compute zone
  • [DEPLOYMENT-NAME], the name of the Kubernetes Deployment resource to be updated by kubectl set

Google App Engine

You can use App Engine's gcloud app deploy command to to deploy an application from a container image.

To use gcloud app deploy in your builds, you must allow the Container Builder service account to use the App Engine API.

Perform the following steps:

  1. Enable the App Engine API.
  2. Add App Engine IAM role:
  3. In GCP Console, visit the IAM menu.
  4. From the list of service accounts, click the Roles drop-down menu beside the Container Builder [YOUR-PROJECT-ID]@cloudbuild.gserviceaccount.com service account.
  5. Click App Engine, then click App Engine Deployer.
  6. Click Save.

You must also have an active App Engine application.

Then, create a build config file that uses gcloud app deploy. The following is an example build config that uses gcloud app deploy:

    steps:
    - name: 'gcr.io/cloud-builders/docker'
      args: ["build", "-t", "gcr.io/[PROJECT-ID]/[IMAGE]", "."]
    - name: 'gcr.io/cloud-builders/docker'
      args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
    - name: 'gcr.io/cloud-builders/gcloud'
      args:
      - app
      - deploy
      - --image-url=gcr.io/[PROJECT-ID]/[IMAGE]:[TAG]

This build calls the docker build step to create a Docker image and push it to Container Registry. Then, the gcloud build step calls gcloud app deploy to deploy the image on App Engine.

To use this example build request, provide the following information:

  • [PROJECT-ID], the Container Builder project ID
  • [IMAGE], the name of the image, and its [TAG] as the desired version or the commit SHA

Firebase

You can use Firebase in your builds by creating a Firebase custom build step.

To create a Firebase build step, create a Dockerfile and a build request file from the following examples:

The following example Dockerfile installs the Firebase command-line tool, firebase, when called by the build:

# use latest Node LTS (Boron)
FROM node:boron
# install Firebase CLI
RUN npm install -g firebase-tools

ENTRYPOINT ["/usr/local/bin/firebase"]

The following example build request, cloudbuild.yaml, uses the Dockerfile to containerize firebase-tools as firebase. Then, the build pushes the containerized image to Container Registry for use in later builds:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/[PROJECT_ID]/firebase', '.' ]
images:
- 'gcr.io/[PROJECT_ID]/firebase'

With the Dockerfile and cloudbuild.yaml in the same directory, run the following command from your shell or terminal window:

gcloud container builds submit --config=cloudbuild.yaml .

The gcloud container builds command submits a build to Container Builder which uses the cloudbuild.yaml build request and the source in the current directory (denoted by .). The build installs the firebase tool in the environment. Then, the tool is containerized and pushed to your Container Registry repository.

Now, firebase can be called in your builds. The following is an example build request that uses the supported npm build step and the firebase build step:

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'test' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run', 'build.prod' ]
- name: 'gcr.io/[PROJECT_ID]/firebase'
  args: [ 'deploy', '-P', 'js-demo-fe-staging', '--token', '[FIREBASE_TOKEN]']

To use this build request, you would have to provide the [FIREBASE_TOKEN] value, which is the Firebase authentication token you would have generated.

Examples of build config files

Creating a Go binary and pushing it to Google Cloud Storage

When you build a Docker image using Container Builder, the built image is automatically pushed to Container Registry. When you build any other binaries using Container Builder, you must manually export the build artifacts.

The following example shows a build config to create a Go binary and push it to Google Cloud Storage.

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['test']
- name: 'gcr.io/cloud-builders/go'
  env:
  - GOOS=linux
  - GOARCH=amd64
  args: ['build', 'my-package']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'my-package', 'gs://my-bucket/']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/go",
        "env": [
            "GOOS=linux",
            "GOARCH=amd64"
        ],
        "args": [
            "build",
            "my-package"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/gsutil",
        "args": [
            "cp",
            "my-package",
            "gs://my-bucket/"
        ]
    }
    ]
}

Build triggered from GitHub

The following config shows a simple build that is triggered from GitHub. This type of configuration is typically used in a CI/CD pipeline.

In this example:

  • The npm build step is called to install the dependencies and run unit tests.
  • The docker build step is called to build a Docker image of the application and push the image to Container Registry.
  • The kubectl build step is called to deploy the built image to the Kubernetes cluster.

YAML

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
- name: 'gcr.io/cloud-builders/docker'
  args: ["build", "-t", "gcr.io/my-project/my-image:$REVISION_ID", "."]
- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/my-project/my-image:$REVISION_ID"]
- name: 'gcr.io/cloud-builders/kubectl'
  args:
  - 'set'
  - 'image'
  - 'deployment/my-deployment'
  - 'my-container=gcr.io/my-project/my-image:$REVISION_ID'
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-east4-b'
  - 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "install"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project/my-image:$REVISION_ID",
            "."
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "push",
            "gcr.io/my-project/my-image:$REVISION_ID"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/kubectl",
        "args": [
            "set",
            "image",
            "deployment/my-deployment",
            "frontend=gcr.io/my-project/my-image:$REVISION_ID"
        ],
        "env": [
            "CLOUDSDK_COMPUTE_ZONE=us-east4-b",
            "CLOUDSDK_CONTAINER_CLUSTER=my-cluster"
        ]
    }
    ]
}

Writing build requests without pushing images

You can use Container Builder to perform arbitrary tasks without producing Docker images.

The example below:

  • Uses docker to build an analysis tool
  • Pulls in some data from the data-to-analyze directory
  • Pushes the analysis results to a Cloud Storage bucket

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'analyzer', '.']

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://my-data-warehouse/data-to-analyze.tgz', '.']

- name: 'debian'
  args: ['tar', '-xzf', 'data-to-analyze.tgz']

- name: 'analyzer'
  args: ['--output=results.csv']
  dir: 'data-to-analyze'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'data-to-analyze/results.csv', 'gs://my-data-warehouse/results.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "analyzer",
        "."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://my-data-warehouse/data-to-analyze.tgz",
        "."
      ],
    },
    {
      "name": "debian",
      "args": [
        "tar",
        "-xzf",
        "data-to-analyze.tgz"
      ],
    },
    {
      "name": "analyzer",
      "args": [
        "--output=results.csv"
      ],
      "dir": "data-to-analyze"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "data-to-analyze/results.csv",
        "gs://my-data-warehouse/results.tgz"
      ]
    }
  ]
}

What's next

Send feedback about...

Cloud Container Builder