Build steps

A Build Step is a Docker container that Container Builder executes as part of your build. Build steps in your build are analogous to commands in a script and provide you with the flexibility of executing arbitrary instructions in your build. If you can package a command into a container, Container Builder can execute it as part of your build.

Use the steps field in the Build resource to specify one or more build steps to execute against your input source. If you specify more than one build step, the assets produced by one step can be passed to the next one via the persistence of the /workspace directory. This allows you to set up a pipeline of build steps that result in the final built asset(s).

Build step configuration

A build step exectuted as an instance of docker run. The build step configuration specifies the following:

  • name: Name or tag of the Docker image to run (required)
  • args: Arguments passed to docker run with the image name (optional)
  • env: Environment variables to set in the environment when the image is run (optional)
  • dir: Working directory for the build step relative to /workspace, which is the default (optional)
  • id: unique identifier of a step within a build. Used by the waitFor field to determine a step's dependencies (optional)
  • waitFor: List of step id identifiers that must have successfully completed before this step can start to execute. If this step should run immediately when the build starts then it should depend on - only. If no waitFor is specified for a step then it implicitly depends on all previously listed steps (optional)
  • entrypoint: New entrypoint to use instead of the image's default.

Supported build steps

The Google Cloud Container Builder team has published a set of supported open-source build steps on GitHub.

Build step order and dependencies

The order of the build steps in the steps field relates to their order of execution. Steps will run serially or concurrently based on the dependencies defined in their waitFor fields.

A step is dependent on every id in its waitFor and will not launch until each dependency has completed successfully. (Note that a step with the given id must be defined prior to the use of the id as a dependency.)

Steps without the optional waitFor field (or with an empty waitFor) will wait for all prior steps to complete successfully before starting. Thus if no step contains an id in its waitFor field, then all steps will be run serially in the order they are defined.

Steps can depend on the start of the build by having waitFor contain only "-". By declaring that a step depends only on "-", the step runs immediately when the build starts. The first step defined depends implicitly on start.

Example build request

Here is a json build request with two steps which will run serially:

{
"steps": [
    {
      "name": "foo"
    },
    {
      "name": "bar"
    }
  ]
}

This example shows a json build request with two concurrent steps that both depend on start; the third step waits for the first two to complete successfully before launching:

{
  "steps": [
    {
      "name": "foo",
      "id": "A"
    },
    {
      "name": "bar",
      "id": "B",
      "waitFor": [
        "-"
      ]
    },
    {
      "name": "baz"
    }
  ]
}

This concurrent build runs steps A and B at the start of the build. The third step will wait implicitly until both previous steps are finished before starting. Note that this example could be simplified by omitting the id fields which are not referenced in a subsequent waitFor.

Here is a json build request with concurrent steps that depend on a previous step:

{
  "steps": [
    {
      "name": "foo",
      "id": "A"
    },
    {
      "name": "bar",
      "id": "B",
      "waitFor": [
        "A"
      ]
    },
    {
      "name": "baz",
      "id": "C",
      "waitFor": [
        "A"
      ]
    }
  ]
}

Step A will run immediately when the build starts. Steps B and C will run concurrently after A is complete. Note that the id and waitFor fields in step B and the id field in step C could be omitted without changing the order of execution.

Build process

When Container Builder runs a build step, it mounts a workspace volume and sets the workspace directory to /workspace (or /workspace/<dir> if the dir field of the build step was set). It then passes any arguments from the build step args field and any environment variables from the env field to the image specified in the step.

If you want to emulate this locally for testing purposes, here is a similar docker run command:

docker run \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /root/.docker:/root/.docker \
  --volume /workspace:/workspace \
  --workdir /workspace/<step-dir> \
  --env <KEY1=val1> \
  --env <KEY2=val2> \
  build-step-name arg1 arg2 arg3

where:

  • The --workdir directory is /workspace or, optionally, a subdirectory specified in the dir field for the build step
  • The env flags specify values specified using the env field in the build step
  • The command line arguments arg1, arg2 and arg3 are specified using the arg field in the build step

For example, this build step:

{
  "steps": [
    {
      "name": "foo",
      "dir": "mydir",
      "env": [ "THIS=THAT", "THESE=THOSE" ],
      "args": [ "--flag=value", "--another-flag", "someArg" ],
    }
  ]
}

would result in this Docker command:

docker run \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /root/.docker:/root/.docker \
  --volume /workspace:/workspace \
  --workdir /workspace/mydir \
  --env THIS=THAT \
  --env THESE=THOSE \
  foo --flag=value --another-flag someArg

Note that when executing your build step locally for testing purposes, the build step executes with whatever permissions are available at execution time. In Container Builder, your build step executes with your project's Container Builder Service Account. If you are running locally to debug a permissions issue, be sure to set up your permissions to match those of the Container Builder Service Account in order to run with a setup as close as possible to the actual Container Builder setup.

Additional Examples

Our public docker build step contains several examples of different ways it might be used as a build step.

Send feedback about...

Cloud Container Builder Documentation