Building, Testing, and Deploying Artifacts

This page explains how to use Google Cloud Container Builder's official builder images to write build configs to fetch source code and dependencies, and to build, test, and deploy artifacts.

If you already have a build config file and want to run your builds on Container Builder, read Starting Builds Manually and Automating Builds.

Before you begin

Calling the build steps

The official builder images are hosted at the following path:

gcr.io/cloud-builders/[BUILD-STEP]

Use the steps field in your build config file to call a build step and the args field to pass a list of arguments to the corresponding tool.

Therefore, to run the Docker build command, add the following to your build config:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/my-image', '.']

Fetching dependencies

Fetch source code or install dependencies by using a build step that runs tools such as docker, git, npm, and gsutil:

docker

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['pull', 'gcr.io/$PROJECT_ID/latest-image']

git

steps:
- name: gcr.io/cloud-builders/git
  args: ['clone', 'https://github.com/GoogleCloudPlatform/cloud-builders']

npm

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']

gsutil

steps:
- name: gcr.io/cloud-builders/gsutil
  args: ['cp', 'gs://mybucket/remotefile.zip', 'localfile.zip']

Building container images

To build Docker container images, use the docker build step in which you can invoke docker commands. Arguments passed to this build step will be passed to docker directly, allowing you to run any docker command in this build step.

In your build config file, add instructions to:

  • Call docker builder and pass the arguments to invoke the Docker build command. In the following build config, the Docker build step is called twice to build two images, for which the source code files are found in the current working directory at build time, as indicated by .. The source code for gcr.io/my-project/image2 is found in a directory inside the current working directory at build time, subdirectory.
  • Add the images field to push the resulting images to Container Registry.

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image2', '.']
  dir: 'subdirectory'
images: ['gcr.io/my-project/image1', 'gcr.io/my-project/image2']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image1", "."]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image2", "."],
      "dir": "subdirectory"
    }
  ],
  "images": ["gcr.io/my-project/image1", "gcr.io/my-project/image2"]
}

Building non-container artifacts

Container Builder provides supported build steps for common languages and tools that you can use to execute your builds.

To build non-container artifacts, add a build steps section that runs builders such as maven, gradle, go, or bazel:

maven

steps:
- name: 'gcr.io/cloud-builders/mvn'
  args: ['install']
- name: 'gcr.io/cloud-builders/mvn'
  args: ['package']

gradle

steps:
- name: 'gcr.io/cloud-builders/gradle'
  args: ['build']

go

- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']

bazel

steps:
- name: gcr.io/cloud-builders/bazel
  args: ['build', '//path/to:target']

Running unit tests and integration tests

If you have the source code available, you can run unit tests and integration tests as build steps.

Assume that you have a JavaScript application with unit tests, a Dockerfile that builds a Docker image, and integration tests that execute against that running image.

To run unit tests and integration tests for this application, add instructions for the following in your build config file:

  • Install dependencies to execute the build.
  • Run the unit test.
  • Build the Docker image of the application.
  • Run the application and dependencies in the background using Docker compose.
  • Run the integration tests against the running Docker compose stack.
  • Push the newly-built image to Container Registry.

    steps:
    - name: 'gcr.io/cloud-builders/npm'
      args: ['install']
    - name: 'gcr.io/cloud-builders/npm'
      args: ['run', 'test:unit']
    - name: 'gcr.io/cloud-builders/docker'
      args: ['build', '-t', 'gcr.io/$PROJECT_ID/gcb-docker-compose:latest', '.']
    - name: 'docker/compose:1.15.0'
      args: ['up', '-d']
      env:
      - 'PROJECT_ID=$PROJECT_ID'
    - name: 'gcr.io/cloud-builders/npm'
      args: ['run', 'test:integration']
      env:
      - 'HOST=counter' # name of the running container
      - 'PORT=50051'
    images: ['gcr.io/$PROJECT_ID/gcb-docker-compose:latest']
    

Pushing artifacts

You can push a Docker image using either the images field or the Docker push argument.

To push the Docker image using the images field, add the following snippet to your build config file. This pushes the built image to Container Registry after the successful completion of all build steps:

steps:
- name: gcr.io/cloud-builders/docker
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/[IMAGE]', '.']
images: ['gcr.io/$PROJECT_ID/[IMAGE]']

To push the image as part of your build flow, pass arguments to invoke the Docker push command in the docker build step:

- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]

The difference between using the images field and the Docker push command is that if you use the images field, the pushed image will show up in the build results. This includes the Build description page for a build in the GCP Console, the results of Build.get(), and the results of gcloud container images list. However, if you use the Docker push command to push the built image, the image will not show up in the build results.

To push an image as part of your build flow and still show up in the build results, use both the Docker push command and the images field in your build config:

- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
images: ['gcr.io/$PROJECT_ID/[IMAGE']

If your built artifact is not a container image, you must manually export it. To push a Go binary to Google Cloud Storage, use the gsutil build step to copy the image to the Google Cloud Storage bucket:

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'my-package', 'gs://my-bucket/']

You can also push your artifacts to third-party repositories such as GitHub, DockerHub, and Artifactory. For instructions on pushing artifacts to GitHub, see Accessing Private GitHub Repositories.

Deploying artifacts

As part of your continuous deployment pipeline, Container Builder can perform deployments using command-line tools.

Use the following steps to deploy to Kubernetes Engine, App Engine, Cloud Functions, and Firebase.

Kubernetes Engine

To deploy to Kubernetes Engine with kubectl, call the kubectl build step to update a Deployment resource:

  1. Enable the Kubernetes Engine API.
  2. Add Kubernetes Engine IAM role:
    1. In GCP Console, visit the IAM menu.
    2. From the list of service accounts, click the Roles drop-down menu beside the Container Builder [YOUR-PROJECT-ID]@cloudbuild.gserviceaccount.com service account.
    3. Click Kubernetes Engine, then click Kubernetes Engine Admin.
    4. Click Save.
  3. Add the kubectl build step in your build config:

    steps:
    - name: 'gcr.io/cloud-builders/docker'
      args: ["build", "-t", "gcr.io/[PROJECT-ID]/[IMAGE]", "."]
    - name: 'gcr.io/cloud-builders/docker'
      args: ["push", "gcr.io/[PROJECT-ID]/[IMAGE]"]
    - name: 'gcr.io/cloud-builders/kubectl'
      args:
      - set
      - image
      - deployment
      - [DEPLOYMENT-NAME]
      - [CONTAINER]=gcr.io/[PROJECT-ID]/[IMAGE]:[TAG]
      env:
      - 'CLOUDSDK_COMPUTE_ZONE=[COMPUTE-ZONE]'
      - 'CLOUDSDK_CONTAINER_CLUSTER=[CLUSTER]'
    

This build calls the docker build step to create a Docker image and push it to Container Registry. Then, the build calls the kubectl build step to update a Deployment resource.

To use this example build config, provide the following information:

  • [PROJECT-ID], the GCP project project ID
  • [IMAGE], the name of the image, and its [TAG] as the desired version or the commit SHA
  • [CLUSTER], the name of the cluster for which authentication credentials are requested
  • [COMPUTE-ZONE], the project's compute zone
  • [DEPLOYMENT-NAME], the name of the Kubernetes Deployment resource to be updated by kubectl set

App Engine

To deploy an application from a container image to App Engine, call the gcloud build step with arguments for gcloud app deploy command:

  1. Enable the App Engine API.
  2. Grant App Engine IAM to the Container Builder service account:
    1. In GCP Console, visit the IAM menu.
    2. In the list of members, look for [YOUR-PROJECT-NUMBER]@cloudbuild.gserviceaccount.com.
    3. Click on the pencil icon in that row to grant a new role to that account.
    4. Click Add another role.
    5. Select App Engine, then click App Engine Admin.
    6. Click Save.
  3. Create a build config file that uses gcloud app deploy:

    steps:
    - name: 'gcr.io/cloud-builders/gcloud'
      args: ['app', 'deploy']
    timeout: '1600s'
    

The gcloud build step calls gcloud app deploy command to build a container image with your source code and then deploys the image on App Engine.

Cloud Functions

To deploy an application to Cloud Function call the gcloud build step to invoke gcloud beta functions deploy:

  1. Enable the Cloud Functions API.
  2. Add App Engine IAM role:
    1. In GCP Console, visit the IAM menu.
    2. From the list of service accounts, click the Roles drop-down menu beside the Container Builder [YOUR-PROJECT-ID]@cloudbuild.gserviceaccount.com service account.
    3. Click Editor and click Save.
  3. Create a build config file that uses gcloud beta functions deploy:

    - name: 'gcr.io/cloud-builders/gcloud'
      args:
      - beta
      - functions
      - deploy
      - [FUNCTION-NAME]
      - --source=.
      - --trigger-http
    

This builds the application source code and calls gcloud beta functions deploy to deploy the application on Cloud Functions.

Firebase

You can use Firebase in your builds by creating a Firebase custom build step.

To create a Firebase build step, create a Dockerfile and a build config file from the following examples.

Create a Dockerfile with the following contents. This installs the Firebase command-line tool, firebase, when called by the build:

# use latest Node LTS (Boron)
FROM node:boron
# install Firebase CLI
RUN npm install -g firebase-tools

ENTRYPOINT ["/usr/local/bin/firebase"]

Create a build config named cloudbuild.yaml with the following contents. This build config uses the Dockerfile to containerize firebase-tools as firebase. Then, the build pushes the containerized image to Container Registry for use in later builds:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/[PROJECT_ID]/firebase', '.' ]
images:
- 'gcr.io/[PROJECT_ID]/firebase'

With the Dockerfile and cloudbuild.yaml in the same directory, run the following command from your shell or terminal window:

gcloud container builds submit --config=cloudbuild.yaml .

The gcloud container builds command submits a build to Container Builder which uses the cloudbuild.yaml build config and the source in the current directory (denoted by .). The build installs the firebase tool in the environment. Then, the tool is containerized and pushed to your Container Registry repository.

To call firebase in your builds, add the gcr.io/[PROJECT_ID]/firebase build step to your build config:

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'test' ]
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run', 'build.prod' ]
- name: 'gcr.io/[PROJECT_ID]/firebase'
  args: [ 'deploy', '-P', 'js-demo-fe-staging', '--token', '[FIREBASE_TOKEN]']

To use this build config, provide the [FIREBASE_TOKEN] value, which is the Firebase authentication token you would have generated.

Examples of build config files

Creating a Go binary and pushing it to Google Cloud Storage

When you build a Docker image using Container Builder, the built image is automatically pushed to Container Registry. When you build any other binaries using Container Builder, you must manually export the build artifacts.

The following example shows a build config to create a Go binary and push it to Google Cloud Storage.

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['test']
- name: 'gcr.io/cloud-builders/go'
  env:
  - GOOS=linux
  - GOARCH=amd64
  args: ['build', 'my-package']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'my-package', 'gs://my-bucket/']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/go",
        "env": [
            "GOOS=linux",
            "GOARCH=amd64"
        ],
        "args": [
            "build",
            "my-package"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/gsutil",
        "args": [
            "cp",
            "my-package",
            "gs://my-bucket/"
        ]
    }
    ]
}

Build triggered from GitHub

The following config shows a simple build that is triggered from GitHub. This type of configuration is typically used in a CI/CD pipeline.

In this example:

  • The npm build step is called to install the dependencies and run unit tests.
  • The docker build step is called to build a Docker image of the application and push the image to Container Registry.
  • The kubectl build step is called to deploy the built image to the Kubernetes cluster.

YAML

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
- name: 'gcr.io/cloud-builders/docker'
  args: ["build", "-t", "gcr.io/my-project/my-image:$REVISION_ID", "."]
- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "gcr.io/my-project/my-image:$REVISION_ID"]
- name: 'gcr.io/cloud-builders/kubectl'
  args:
  - 'set'
  - 'image'
  - 'deployment/my-deployment'
  - 'my-container=gcr.io/my-project/my-image:$REVISION_ID'
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-east4-b'
  - 'CLOUDSDK_CONTAINER_CLUSTER=my-cluster'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "install"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/npm",
        "args": [
            "test"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project/my-image:$REVISION_ID",
            "."
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "push",
            "gcr.io/my-project/my-image:$REVISION_ID"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/kubectl",
        "args": [
            "set",
            "image",
            "deployment/my-deployment",
            "frontend=gcr.io/my-project/my-image:$REVISION_ID"
        ],
        "env": [
            "CLOUDSDK_COMPUTE_ZONE=us-east4-b",
            "CLOUDSDK_CONTAINER_CLUSTER=my-cluster"
        ]
    }
    ]
}

Writing build requests without pushing images

You can use Container Builder to perform arbitrary tasks without producing Docker images.

The following example:

  • Uses docker to build an analysis tool
  • Pulls in some data from the data-to-analyze directory
  • Pushes the analysis results to a Cloud Storage bucket

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'analyzer', '.']

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://my-data-warehouse/data-to-analyze.tgz', '.']

- name: 'debian'
  args: ['tar', '-xzf', 'data-to-analyze.tgz']

- name: 'analyzer'
  args: ['--output=results.csv']
  dir: 'data-to-analyze'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'data-to-analyze/results.csv', 'gs://my-data-warehouse/results.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "analyzer",
        "."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://my-data-warehouse/data-to-analyze.tgz",
        "."
      ],
    },
    {
      "name": "debian",
      "args": [
        "tar",
        "-xzf",
        "data-to-analyze.tgz"
      ],
    },
    {
      "name": "analyzer",
      "args": [
        "--output=results.csv"
      ],
      "dir": "data-to-analyze"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "data-to-analyze/results.csv",
        "gs://my-data-warehouse/results.tgz"
      ]
    }
  ]
}

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Container Builder