Writing Build Requests

This page explains how to write and configure build requests.

Build requests are JSON or YAML documents which instruct Container Builder to perform tasks, and to package, build, and push Docker images, based on your specifications.

Build requests are composed of build steps, which are Docker containers that execute code on your behalf. Calling build steps is analogous to calling scripts or shell commands. You can use the supported, open-source build steps provided by Container Builder or any other pullable image in your build steps.

A build request defines a Build resource, the Container Builder API object which locates source code, builds the code, then pushes the resulting images, if any, to Container Registry.

To learn about build requests in detail, see Build Requests and the Build resource documentation.

Basic build requests

A build request is a file in YAML or JSON format that you create. The file must:

  • contain a steps field;
  • contain, within the steps field, a name field for each build step; and
  • contain an args field for each build step.

Build requests do not need to produce Docker images to run successfully. To produce Docker images, add an images field that contains an array of the images that the build should produce.

The example below demonstrates how to write a basic build request. In this example:

  • docker build step is called to build two images, for which the source code files are found in the present working directory at build time, as indicated by .
  • The source code for gcr.io/my-project/image2 is found in a directory inside the present working directory at build time, subdirectory
  • The resulting images are pushed to Container Registry.

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image2', '.']
  dir: 'subdirectory'
images: ['gcr.io/my-project/image1', 'gcr.io/my-project/image2']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image1", "."]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/my-project/image2", "."],
      "dir": "subdirectory"
    }
  ],
  "images": ["gcr.io/my-project/image1", "gcr.io/my-project/image2"]
}

Tagging your builds

You can use tags in your build requests, which allows you to organize your builds into groups and to filter your builds. You can specify strings in tags, such as "prod" or "test".

The following is an example of using tags in a build:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/cb-demo-img', '.' ]
images:
- 'gcr.io/$PROJECT_ID/cb-demo-img'
tags:
- "test"
- "tag2"

To see tagged builds in your cluster, use the --filter option in gcloud container builds list. For example, the following command lists all builds tagged with 'test':

gcloud container builds list --filter "tags='test'"

You can also query builds from the Cloud Platform Console Build History menu.

For more information about tags, see Tags.

Submitting a build

To submit a build, run the following command:

gcloud container builds submit --config [BUILD-REQUEST] [SOURCE-CODE]

For example, the following command submits the cloudbuild.yaml build request using archived source code stored in a Cloud Storage bucket.

gcloud container builds submit --config cloudbuild.yaml gs://container-builder-examples/node-docker-example.tar.gz

You can use . to specify that the source code is in the present working directory:

gcloud container builds submit --config cloudbuild.yaml .

If your build does not require source code, you can specify the --no-source option:

gcloud container builds submit --config cloudbuild.yaml --no-source

Unsupported build request fields

Some build request fields are only used in responses sent by the server. Container Builder ignores response-only fields in build requests. For example, the project ID is generated by the server, so use of the projectId field is unnecessary and ignored.

See the Build resource documentation for the full list of build request fields. Fields used only for responses should not be included in your build requests.

Using the source field

The source field describes where to find the source files to build. Depending on your needs, you may need to include the source field in your build request file.

You should include the source field if you are:

  • Configuring build triggers using YAML files
  • Submitting build requests using a third-party tool, such as curl

When you submit build requests using the gcloud command-line tool, the source field may not be necessary if you specify the source as a command-line argument. You may also specify a source in your build request file. See the gcloud documentation for more information.

Substitutions

The steps and images fields support substitutions for built-in and user-defined variables. See Build Requests for the list of built-in variables and the substitution syntax.

Supplying substitutions in build requests instructs Container Builder to resolve their values at build time.

The example below uses the built-in variables $BUILD_ID, $PROJECT_ID, and $REVISION_ID.

YAML

steps:
# Uses the ubuntu build step:
# to run a shell script; and
# set env variables for its execution
- name: 'ubuntu'
  args: ['bash', './myscript.sh']
  env:
  - 'BUILD=$BUILD_ID'
  - 'PROJECT=$PROJECT_ID'
  - 'REV=$REVISION_ID'

# Uses the docker build step to build an image called my-image
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-image', '.']

# my-image is pushed to Container Registry
images:
- 'gcr.io/$PROJECT_ID/my-image'

JSON

{
  "steps": [
    {
      "name": "ubuntu",
      "args": [
        "bash",
        "./myscript.sh"
      ],
      "env": [
        "BUILD=$BUILD_ID",
        "PROJECT=$PROJECT_ID",
        "REV=$REVISION_ID"
      ]
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "gcr.io/$PROJECT_ID/my-image",
        "."
      ]
    }
  ],
  "images": [
    "gcr.io/$PROJECT_ID/my-image"
  ]
}

Configuring build step order

You can specify the order in which your build steps are called using the waitFor field. Build steps run sequentially by default, but they can be configured to run serially or concurrently. You can use the id field to "name" build steps, then use the name as a value for waitFor.

Use the waitFor field in a build step to specify which steps must run before the build step is run. If no values are provided for waitFor, the build step waits for all prior build steps in the build request to complete successfully before running.

To run a build step immediately at build time, use - in the waitFor field.

The example below calls the gsutil and wget steps concurrently. After these steps have completed, the ubuntu step is called.

YAML

steps:
# Download the binary and the data in parallel.
- name: 'gcr.io/cloud-builders/wget'
  args: ['https://example.com/binary']
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://$PROJECT_ID-data/rawdata.tgz', '.']
  waitFor: ['-']  # The '-' indicates that this step begins immediately.

# Run the binary on the data, once both are downloaded.
- name: 'ubuntu'
  args: ['./binary', 'rawdata.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/wget",
      "args": [
        "https://example.com/binary"
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://$PROJECT_ID-data/rawdata.tgz",
        "."
      ],
      "waitFor": [
        "-"
      ]
    },
    {
      "name": "ubuntu",
      "args": [
        "./binary",
        "rawdata.tgz"
      ]
    }
  ]
}

The example below uses the id field to name certain build steps. The values from id are used in waitFor to define build step order:

  • First, fetch-resources step uses gsutil to copy the local resources to Cloud Storage. Concurrently, go generates, tests, and installs the source code.
  • Then, the docker build step builds the image after all other steps are complete.

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['generate']
- name: 'gcr.io/cloud-builders/go'
  args: ['test', './...']
- name: 'gcr.io/cloud-builders/go'
  args: ['install', 'mytarget']
  id: 'go-install'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', '-r', './somefiles', 'gs://my-resource-bucket/somefiles']
  waitFor: ['-']  # The '-' indicates that this step begins immediately.
  id: 'fetch-resources'

- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/mytarget', '.']
  waitFor: ['go-install', 'fetch-resources']

images: ['gcr.io/$PROJECT_ID/mytarget']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "generate"
      ]
    },
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "test",
        "./..."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/go",
      "args": [
        "install",
        "mytarget"
      ],
      "id": "go-install"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "-r",
        "./somefiles",
        "gs://my-resource-bucket/somefiles"
      ],
      "waitFor": [
        "-"
      ],
      "id": "fetch-resources"
    },
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "gcr.io/$PROJECT_ID/mytarget",
        "."
      ],
      "waitFor": [
        "go-install",
        "fetch-resources"
      ]
    }
  ],
  "images": [
    "gcr.io/$PROJECT_ID/mytarget"
  ]
}

Writing build requests without pushing images

You can use Container Builder to perform arbitrary tasks without producing Docker images.

The example below:

  • Uses docker to build an analysis tool
  • Pulls in some data from the data-to-analyze directory
  • Pushes the analysis results to a Cloud Storage bucket

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'analyzer', '.']

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://my-data-warehouse/data-to-analyze.tgz', '.']
  waitFor: ['-']  # Don't wait for image build before starting the download.
  id: 'download'

- name: 'debian'
  args: ['tar', '-xzf', 'data-to-analyze.tgz']
  waitFor: ['download']

- name: 'analyzer'
  args: ['--output=results.csv']
  dir: 'data-to-analyze'

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'data-to-analyze/results.csv', 'gs://my-data-warehouse/results.tgz']

JSON

{
  "steps": [
    {
      "name": "gcr.io/cloud-builders/docker",
      "args": [
        "build",
        "-t",
        "analyzer",
        "."
      ]
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "gs://my-data-warehouse/data-to-analyze.tgz",
        "."
      ],
      "waitFor": [
        "-"
      ],
      "id": "download"
    },
    {
      "name": "debian",
      "args": [
        "tar",
        "-xzf",
        "data-to-analyze.tgz"
      ],
      "waitFor": [
        "download"
      ]
    },
    {
      "name": "analyzer",
      "args": [
        "--output=results.csv"
      ],
      "dir": "data-to-analyze"
    },
    {
      "name": "gcr.io/cloud-builders/gsutil",
      "args": [
        "cp",
        "data-to-analyze/results.csv",
        "gs://my-data-warehouse/results.tgz"
      ]
    }
  ]
}

Submitting a build request

You can use the gcloud command-line tool to submit your build.

To submit a build, run the following command:

gcloud container builds submit --config [BUILD-REQUEST] [SOURCE-CODE]

where:

  • [BUILD-REQUEST] is the path to the build request file
  • [SOURCE-CODE] is the path or URL source code

If you do not have source code to pass in to your build, use the --no-source flag.

Running builds locally

You can run builds locally using the container-builder-local tool.

What's next

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Container Builder