Build Steps

A build step is a Docker container that Container Builder executes as part of your build request. Build steps are analogous to commands in a script and provide you with the flexibility of executing arbitrary instructions in your build. If you can package a command into a container, Container Builder can execute it as part of your build.

Use the steps field in the Build resource to specify one or more build steps to execute with your input source.

If you specify more than one build step, the assets produced by one step can be passed to the next one via the persistence of the /workspace directory, which allows you to set up a pipeline of build steps.

Container Builder contains a number of official build steps that you can use in your builds. You can also create your own custom build steps.

A build step is exectuted as an instance of docker run.

Anatomy of a build step

Build steps can contain the following fields: name, args, env, dir, id, waitFor, and entrypoint.

name

Name or tag of the Docker image to run.

This field is required.

args

The args field specifies a list of arguments that will be presented to the step when it is started.

If you do not specify the entrypoint field in your build step's build request file and the build step's image did not have an ENTRYPOINT specified in its Dockerfile, the first item in args is used as the entry point with the remaining items in args as the arguments.

This field is optional.

env

The env field specifies a list of environment variable definitions, in the form of key-value strings, to be used when running a step.

For example, you can specify version numbers, $PATH variables, and other variables specific to the containerized application.

The key-value strings take the form of [KEY]=[VALUE]:

  • KEY is the name of the environment variable
  • VALUE is the variable's value

This field is optional.

dir

The dir field specifies the working directory (relative to project source root) to use when running the step.

If you don't specify a value for dir, Container Builder uses the source root as the working directory.

This field is optional.

id

Unique identifier of a step within a build. Used by the waitFor field to determine a step's dependencies

This field is optional.

waitFor

List of step id identifiers that must have successfully completed before this step can start to execute. If this step should run immediately when the build starts, then it should depend on - only. If no waitFor is specified for a step, then it implicitly depends on all previously listed steps

This field is optional.

entrypoint

Defines how the build step should be run when it is called. To learn more, see About the entrypoint field.

This field is optional.

For more information about the build step fields, see BuildStep in the Container Builder API documentation.

Official build steps

The Container Builder team has published a set of supported, open-source build steps on GitHub.

These build steps are hosted at gcr.io/cloud-builders/[BUILD-STEP].

Build step order and dependencies

The order of the build steps in the steps field relates to the order in which the steps are executed. Steps will run serially or concurrently based on the dependencies defined in their waitFor fields.

A step is dependent on every id in its waitFor and will not launch until each dependency has completed successfully.

Steps without the optional waitFor field (or with an empty waitFor) will wait for all prior steps to complete successfully before executing. Thus, if no step contains an id in its waitFor field, then all steps will execute serially in the order they are defined.

Steps can depend on the start of the build by having waitFor contain only -. By declaring that a step depends only on -, the step runs immediately when the build starts. The first step defined depends implicitly on start.

Examples

Below is a build request with two steps which will run serially:

YAML

steps:
- name: foo
- name: bar

JSON

{
"steps": [
    {
      "name": "foo"
    },
    {
      "name": "bar"
    }
  ]
}

The example below shows a build request with two concurrent steps that both depend on start; the third step waits for the first two to complete successfully before launching:

YAML

steps:
- name: foo
  id: A
- name: bar
  id: B
  waitFor:
  - "-"
- name: baz

JSON

{
  "steps": [
    {
      "name": "foo",
      "id": "A"
    },
    {
      "name": "bar",
      "id": "B",
      "waitFor": [
        "-"
      ]
    },
    {
      "name": "baz"
    }
  ]
}

This concurrent build runs steps A and B at the start of the build. The third step will wait implicitly until both previous steps are finished before starting. Note that this example could be simplified by omitting the id fields, which are not referenced in a subsequent waitFor.

Below is a build request with concurrent steps that depend on a previous step:

YAML

steps:
- name: foo
  id: A
- name: bar
  id: B
  waitFor:
  - A
- name: baz
  id: C
  waitFor:
  - A

JSON

{
  "steps": [
    {
      "name": "foo",
      "id": "A"
    },
    {
      "name": "bar",
      "id": "B",
      "waitFor": [
        "A"
      ]
    },
    {
      "name": "baz",
      "id": "C",
      "waitFor": [
        "A"
      ]
    }
  ]
}

Step A will run immediately when the build starts. Steps B and C will run concurrently after A has completed successfully. Note that the id and waitFor fields in step B, and the id field in step C, could be omitted without changing the order of execution.

Build process

When Container Builder runs a build step, it mounts a workspace volume and sets the working directory to /workspace (or /workspace/<dir>, if the dir field of the build step was set). Container Builder then passes any arguments from the build step args field, and any environment variables from the env field, to the image specified in the step.

If you want to emulate this locally for testing purposes, here is a similar docker run command:

docker run \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /root/.docker:/root/.docker \
  --volume /workspace:/workspace \
  --workdir /workspace/[STEP-DIRECTORY] \
  --env [KEY1]=[val1] \
  --env [KEY2]=[val2] \
  build-step-name arg1 arg2 arg3

where:

  • the --workdir directory is /workspace or, optionally, a subdirectory specified in the dir field for the build step;
  • the env flags specify values specified using the env field in the build step; and
  • the command line arguments arg1, arg2, and arg3 are specified using the arg field in the build step.

For example, this build step:

YAML

steps:
- name: foo
  dir: mydir
  env:
  - THIS=THAT
  - THESE=THOSE
  args:
  - "--flag=value"
  - "--another-flag"
  - someArg

JSON

{
  "steps": [
    {
      "name": "foo",
      "dir": "mydir",
      "env": [ "THIS=THAT", "THESE=THOSE" ],
      "args": [ "--flag=value", "--another-flag", "someArg" ]
    }
  ]
}

would result in this Docker command:

docker run \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /root/.docker:/root/.docker \
  --volume /workspace:/workspace \
  --workdir /workspace/mydir \
  --env THIS=THAT \
  --env THESE=THOSE \
  foo --flag=value --another-flag someArg

Note that when executing your build step locally for testing purposes, the build step executes with whatever permissions are available at execution time. In Container Builder, your build step executes with your project's Container Builder Service Account. If you are running locally to debug a permissions issue, be sure to set up your permissions to match those of the Container Builder Service Account to run in an environment as close as possible to the actual Container Builder environment.

Additional Examples

Our public docker build step contains several examples of different ways it might be used as a build step.

Send feedback about...

Cloud Container Builder Documentation