Creating custom build steps

When writing your build configs, you can use the supported, open-source build steps provided by Cloud Build, or you define your own custom build steps.

A custom build step is a container image that the Cloud Build worker VM pulls and runs with your source volume-mounted to /workspace. Your custom build step can execute any script or binary inside the container; as such, it can do anything a container can do.

Custom build steps are useful for:

  • Downloading source code or packages from external locations
  • Using an external tool chain
  • Caching any necessary libraries
  • Pre-building source (with Cloud Build responsible only for potentially packaging the build into a container image)

A custom build step runs with the source mounted under /workspace, and is run with a working directory somewhere in /workspace. Any files left in /workspace by a given build step are available to other build steps, whether those steps are run concurrently or subsequently.

Your custom build step can push to or pull from Google Container Registry repository (hosted at$PROJECT-NAME/) to which your builder service account has access. The docker command-line tool credentials are not sufficient to provide authenticated access to Docker Hub.

This guide explains how to create a custom build step with an example that walks you through how to create a build step that executes a shell script. You can create a "shell script executor" as a custom build step to execute a shell script from somewhere in your build source.

Creating a custom build step

To create a custom build step, you can create a build config file that builds and pushes the build step image to an image registry, such as Container Registry, that the builder service account can access. Alternatively, you can use another tool to build the custom build step image, then store it in an image registry. Once this is complete, you can invoke your custom build step in future builds.

About the entrypoint field

Your image's Dockerfile may have an ENTRYPOINT and/or CMD field. ENTRYPOINT specifies the entry point to use for your custom build step if the container is to be run as an executable. CMD provides defaults for execution and, if ENTRYPOINT is omitted, should include an executable.

In a build config, the optional entrypoint field defines how the build step should be run when it is invoked. For example, you can specify the main command that should be called when the build step is run: the docker build step's entry point is "/usr/bin/docker". When you use your build step in a later build, you can override the Dockerfile ENTRYPOINT by specifying an entrypoint in that build.

If you do not specify the entrypoint field in your build step's build request file and the build step's image did not have an ENTRYPOINT specified in its Dockerfile, the first item in args is used as the entry point with the remaining items in args as the arguments.

Example: executing a shell script from your source

For a custom build step to execute a shell script from your source, the step's container image must contain a tool capable of running the script. Standard base images such as ubuntu, debian, alpine, and busybox container images can all run scripts; however, the alpine and busybox images do not come with bash pre-installed (and thus cannot run bash scripts), while ubuntu and debian do.

If an image contains all the tools (including the shell) that you need to run your script, you can use that image as a build step directly.

This example uses an ubuntu image to run scripts, since it comes prepackaged with bash and has support for a lot of developer tools. The build itself creates images based on alpine; such images are much smaller and only contain what's needed for the runtime environment.

Here's the example ./cloudbuild.yaml configuration file for the custom build step:

- name: 'ubuntu'
  args: ['bash', './myscript.bash']
- name: ''
  args: ['build', '-t', '$PROJECT_ID/custom-script-test', '.']
images: ['$PROJECT_ID/custom-script-test']

The step runs the following script, called ./myscript.bash:

echo "Hello, world!" > file.txt

Here is the example Dockerfile:

FROM alpine
COPY file.txt /file.txt
ENTRYPOINT ["cat", "/file.txt"]

To submit the build that uses this custom build step, run the following command in your shell or terminal window:

$ gcloud builds submit --config cloudbuild.yaml .
$ gcloud docker -- pull<your-project-id>/custom-script-test
$ docker run<your-project-id>/custom-script-test
Hello, world!

The script that your custom build step runs might require more resources than those provided in the alpine base image used in this example. If that's the case, you can pre-build an image with the necessary resources by using COPY directives in your Dockerfile to bring resources from the workspace into your container image.

For example, suppose you want to run a script that uses curl to pull down a file to be included in the built image. Since the ubuntu image doesn't come prepackaged with the curl command-line tool, we'll make a new image with an ubuntu base image and curl layered on top.

Here is an example ./cloudbuild.yaml configuration file for a build that uses a custom ubuntu-curl build step:

- name: '$PROJECT_ID/ubuntu-curl'
  args: ['bash', './curl.bash']
- name: ''
  args: ['build', '-t', '$PROJECT_ID/custom-script-test2', '.']
images: ['$PROJECT_ID/custom-script-test2']

The Dockerfile.ubuntu-curl that installs the curl tool:

FROM ubuntu
RUN apt-get -q update && apt-get install -qqy curl

The ./curl.bash script might resemble the following:

curl > example.html

The example Dockerfile:

FROM alpine
COPY example.html /example.html
ENTRYPOINT ["cat", "/example.html"]

To run the custom build step for ubuntu-curl and then build the image, run the following command in your shell or terminal window:

# First, build and push the `ubuntu-curl` custom build step.
$ docker build -f Dockerfile.ubuntu-curl -t .
$ gcloud docker -- push

Once you push the image you created using Dockerfile.ubuntu-curl to a Docker Registry, you can use the image directly as a build step.

# Then, use the custom `ubuntu-curl` build step in a new build.
$ gcloud builds submit --config cloudbuild.yaml .
$ gcloud docker -- pull
$ docker run
`<`contents of source`>`

You can accomplish the same thing in one build by adding a preliminary step to build your script executor image. Once you've built that image, you can use it as the next step in your build. This has the advantage of keeping curl.bash clean and performing the bulk of the work in the Cloud Build service.

Here is an example ./cloudbuild.yaml configuration file with the preliminary step added:

- name: ''
  args: ['build', '-f', 'Dockerfile.ubuntu-curl', '-t', 'script-runner', '.']
- name: 'script-runner'
  args: ['bash', './curl.bash']
- name: ''
  args: ['build', '-t', '$PROJECT_ID/custom-script-test2', '.']
images: ['$PROJECT_ID/custom-script-test2']

As long as you leave script-runner out of the images field in your cloudbuild.yaml, the Cloud Build service will not attempt to push it (which would fail in this example). In the context of the build, however, the script-runner image will exist in the image cache and can be used as a build step.

What's next