Build configuration overview

This page explains the structure of the Cloud Build build configuration file.

Dockerfile and build config file

If you're executing non-Docker builds in Cloud Build using the gcloud command-line tool or a build trigger, you must provide a build config file. A build config file contains instructions for Cloud Build to perform tasks based on your specifications. For example, your build config file can contain instructions to build, package, and push Docker images.

If you're executing Docker builds using the gcloud tool or build triggers, you can use either a Dockerfile or a build config file to build the image. For instructions on how to build a Docker image using a Dockerfile, see Quickstarts using Docker.

Structure of a build config file

Build config files are modeled using the Cloud Build API's Build resource.

You can write the build config file using the YAML or the JSON syntax. If you submit build requests using third-party http tools such as curl, use the JSON syntax. You must create the build config file in your project's root directory and Cloud Build will read it each time it starts a build.

A build config file has the following structure:

YAML

steps:
- name: string
  args: [string, string, ...]
  env: [string, string, ...]
  dir: string
  id: string
  waitFor: string
  entrypoint: string
  secretEnv: string
  volumes: object(Volume)
  timeout: string (Duration format)
- name: string
  ...
- name: string
  ...
timeout: string (Duration format)
logsBucket: string
options:
 sourceProvenanceHash: enum(HashType)
 machineType: enum(MachineType)
 diskSizeGb: string (int64 format)
 substitutionOption: enum(SubstitutionOption)
 logStreamingOption: enum(LogStreamingOption)
 logging: enum(LoggingMode)
substitutions: map (key: string, value: string)
tags: [string, string, ...]
secrets: object(Secret)
images:
- [string, string, ...]
artifacts: object (Artifacts)

JSON

{
    "steps": [
    {
        "name": "string",
        "args": [
        [
            "string",
            "string",
            "..."
        ]
        ],
        "env": [
        [
            "string",
            "string",
            "..."
        ]
        ],
        "dir": "string",
        "id": "string",
        "waitFor": "string",
        "entrypoint": "string",
        "secretEnv": "string",
        "volumes": "object(Volume)",
        "timeout": "string (Duration format)"
    },
    {
        "name": "string"
        ...
    },
    {
        "name": "string"
        ...
    }
    ],
    "timeout": "string (Duration format)",
    "logsBucket": "string",
    "options": {
        "sourceProvenanceHash": "enum(HashType)",
        "machineType": "enum(MachineType)",
        "diskSizeGb": "string (int64 format)",
        "substitutionOption": "enum(SubstitutionOption)",
        "logStreamingOption": "enum(LogStreamingOption)",
        "logging": "enum(LoggingMode)"
    },
    "substitutions": "map (key: string, value: string)",
    "tags": [
    [
        "string",
        "string",
        "..."
    ]
    ],
    "secrets": "object(Secret)",
    "images": [
    [
        "string",
        "string",
        "..."
    ]
    ],
    "artifacts": "object(Artifacts)"
}

Each of the sections of the build config file defines a part of the task you want Cloud Build to execute:

Build steps

A build step specifies an action that you want Cloud Build to perform. For each build step, Cloud Build executes a docker container as an instance of docker run. Build steps are analogous to commands in a script and provide you with the flexibility of executing arbitrary instructions in your build. If you can package a build tool into a container, Cloud Build can execute it as part of your build.

You can include one or more build steps in your config file.

Use the steps field in the build config file to specify a build step. Here's a snippet of the kind of configuration you might set in the steps field:

YAML

steps:
- name: 'gcr.io/cloud-builders/git'
  args: ['clone', 'https://github.com/GoogleCloudPlatform/cloud-builders']
  env: ['PROJECT_ROOT=hello']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project-id/myimage', '.']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/git",
        "args": [
            "clone",
            "https://github.com/GoogleCloudPlatform/cloud-builders"
        ],
        "env": [
            "PROJECT_ROOT=hello"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project-id/myimage",
            "."
        ]
    }
    ]
}

name

Use the name field of a build step to specify a cloud builder, which is a container image running common tools. You use a builder in a build step to execute your tasks.

The following snippet shows build steps calling the bazel, gcloud, and docker builders:

YAML

steps:
- name: 'gcr.io/cloud-builders/bazel'
...

- name: 'gcr.io/cloud-builders/gcloud'
...

- name: 'gcr.io/cloud-builders/docker'
...

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/bazel"
        ...
    },
    {
        "name": "gcr.io/cloud-builders/gcloud"
        ...
    },
    {
        "name": "gcr.io/cloud-builders/docker"
        ...
    }
    ]
}

args

The args field of a build step takes a list of arguments and passes them to the builder referenced by the name field. Arguments passed to the builder are passed to the tool thats running in the builder, which allows you to invoke any command supported by the tool. If the builder used in the build step has an entrypoint, args will be used as arguments to that entrypoint. If the builder does not define an entrypoint, the first element in args will be used as the entrypoint, and the remainder will be used as arguments.

The following snippet invokes the docker build command and installs Maven dependencies:

YAML

steps:
- name: 'gcr.io/cloud-builders/mvn'
  args: ['install']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project-id/myimage', '.']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/mvn",
        "args": [
            "install"
        ]
    },
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/my-project-id/myimage",
            "."
        ]
    }
    ]
}

env

The env field of a build step takes a list of environment variables to be used when running the step. The variables are of the form KEY=VALUE.

In the following build config the env field of the build step sets the Compute Engine zone and the GKE cluster prior to executing kubectl:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
- name: 'gcr.io/cloud-builders/kubectl'
  args: ['set', 'image', 'deployment/myimage', 'frontend=gcr.io/myproject/myimage']
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=us-east1-b'
  - 'CLOUDSDK_CONTAINER_CLUSTER=node-example-cluster'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    },
    {
        "name": "gcr.io/cloud-builders/kubectl",
        "args": [
            "set",
            "image",
            "deployment/myimage",
            "frontend=gcr.io/myproject/myimage"
        ],
        "env": [
            "CLOUDSDK_COMPUTE_ZONE=us-east1-b",
            "CLOUDSDK_CONTAINER_CLUSTER=node-example-cluster"
        ]
    }
    ]
}

dir

Use the dir field in a build step to set a working directory to use when running the step. By default, Cloud Build uses a directory named /workspace as a working directory. If your config file has more than one build step, the assets produced by one step can be passed to the next one via the persistence of the /workspace directory, which allows you to set up a pipeline of build steps that share assets. If you set the dir field in the build step, the working directory is set to /workspace/<dir>. If this value is a relative path, it is relative to the build's working directory. If this value is absolute, it may be outside the build's working directory, in which case the contents of the path may not be persisted across build step executions (unless a volume for that path is specified).

The following snippet sets the working directory as examples/hello_world:

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['install', '.']
  env: ['PROJECT_ROOT=hello']
  dir: 'examples/hello_world'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "install",
            "."
        ],
        "env": [
            "PROJECT_ROOT=hello"
        ],
        "dir": "examples/hello_world"
    }
    ]
}

timeout

Use the timeout field in a build step to set a time limit for executing the step. If you don't set this field, the step has no time limit and will be allowed to run until either it completes or the build itself times out. timeout must be specified in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s"

In the following build config, the ubuntu step is timed out after 500 seconds:

YAML

steps:
- name: 'ubuntu'
  args: ['sleep', '600']
  timeout: 500s
- name: 'ubuntu'
  args: ['echo', 'hello world, after 600s']

JSON

{
    "steps": [
    {
        "name": "ubuntu",
        "args": [
            "sleep",
            "600"
        ],
        "timeout": "500s"
    },
    {
        "name": "ubuntu",
        "args": [
            "echo",
            "hello world, after 600s"
        ]
    }
    ]
}

Note: The timeout field exists for both the build step and the build: the timeout field of a build step specifies the amount of time the step is allowed to run, and the timeout field of a build specifies the amount of time the build is allowed to run.

id

Use the id field to set a unique identifier for a build step. id is used with the waitFor field to configure the order in which build steps should be run. For instructions on using waitFor and id, see Configuring build step order.

waitFor

Use the waitFor field in a build step to specify which steps must run before the build step is run. If no values are provided for waitFor, the build step waits for all prior build steps in the build request to complete successfully before running. For instructions on using waitFor and id, see Configuring build step order.

entrypoint

Use the entrypoint in a build step to specify an entrypoint if you don't want to use the default entrypoint of the builder. If you don't set this field, Cloud Build will use the builder's entrypoint. The following snippet sets the entrypoints for the npm build step:

YAML

steps:
- name: 'gcr.io/cloud-builders/npm'
  entrypoint: 'node'
  args: ['--version']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/npm",
        "entrypoint": "node",
        "args": [
            "--version"
        ]
    }
    ]
}

secretEnv

A list of environment variables which are encrypted using a Cloud KMS crypto key. These values must be specified in the build's secrets. For information on using this field see Using the encrypted variable in build requests.

volumes

A Volume is a Docker container that is mounted into build steps to persist files across build steps. When Cloud Build runs a build step, it automatically mounts a workspace volume into /workspace. You can specify additional volumes to be mounted into your build steps' containers using the volumes field for your steps.

For example, the following build config file writes a file into a volume in the first step and reads it in the second step. If these steps did not specify /persistent_volume path as a persistent volume, the first step would write the file at that path, then that file would be discarded before the second step executes. By specifying the volume with the same name in both steps, the contents of /persistent_volume in the first step are persisted to the second step.

YAML

steps:
- name: 'ubuntu'
  volumes:
  - name: 'vol1'
    path: '/persistent_volume'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
        echo "Hello, world!" > /persistent_volume/file
- name: 'ubuntu'
  volumes:
  - name: 'vol1'
    path: '/persistent_volume'
  args: ['cat', '/persistent_volume/file']

JSON

  {
    "steps": [
      {
        "name": "ubuntu",
        "volumes": [
          {
            "name": "vol1",
            "path": "/persistent_volume"
          }
        ],
        "entrypoint": "bash",
        "args": [
          "-c",
          "echo \"Hello, world!\" > /persistent_volume/file\n"
        ]
      },
      {
        "name": "ubuntu",
        "volumes": [
          {
            "name": "vol1",
            "path": "/persistent_volume"
          }
        ],
        "args": [
          "cat",
          "/persistent_volume/file"
        ]
     }
    ]
  }

timeout

Use the timeout field for a build to specify the amount of time that the build must be allowed to run, to second granularity. If this time elapses, work on the build will cease and the build status will be TIMEOUT. If timeout is not set, a default timeout of 10 minutes will apply to the build. timeout must be specified in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s"

In the following snippet, timeout is set to 660 seconds to avoid the build from timing out because of the sleep:

YAML

steps:
- name: 'ubuntu'
  args: ['sleep', '600']
timeout: 660s

JSON

{
    "steps": [
    {
        "name": "ubuntu",
        "args": [
            "sleep",
            "600"
        ]
    }
    ],
    "timeout": "660s"
}

Note: The timeout field exists for both the build step and the build: the timeout field of a build step specifies the amount of time the step is allowed to run, and the timeout field of a build specifies the amount of time the build is allowed to run.

logsBucket

Set the logsBucket field for a build to specify a Cloud Storage bucket where logs must be written. If you don't set this field, Cloud Build will use a default bucket to store your build logs.

The following snippet sets a logs bucket to store the build logs:

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['install', '.']
logsBucket: 'gs://mybucket'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "install",
            "."
        ]
    }
    ],
    "logsBucket": "gs://mybucket"
}

options

Use the options field to specify the following optional arguments for your build:

sourceProvenanceHash: Set the sourceProvenanceHash option to specify the hash algorithm for source provenance. The following snippet specifies that the hash algorithm is SHA256:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
options:
    sourceProvenanceHash: ['SHA256']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    }
    ],
    "options": {
        "sourceProvenanceHash": ["SHA256"]
    }
}

machineType: Cloud Build provides two high-CPU virtual machine types to run your builds: 8 CPUs and 32 CPUs. The default machine type is 1 CPU. Requesting a high-CPU virtual machine may increase the startup time of your build. Add the machineType option to request a virtual machine with a higher CPU:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
options:
 machineType: 'N1_HIGHCPU_8'

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    },
    ],
    "options": {
        "machineType": "N1_HIGHCPU_8"
    }
}

For more information on using the machineType option see Speeding up builds.

diskSizeGb: Use the diskSizeGb option to request a custom disk size for your build. The maximum size you can request is 1000 GB.

The following snippet requests a disk size of 200 GB:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
options:
 diskSizeGb: 200

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    }
    ],
    "options": {
        "diskSizeGb": 200
    }
}

logStreamingOption: Use this option to specify if you want to stream build logs to Cloud Storage. By default, Cloud Build collects build logs on build completion; this option specifies if you want to stream build logs in real time through the build process. The following snippet specifies that build logs are streamed to Cloud Storage:

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['install', '.']
options:
 logStreamingOption: STREAM_ON

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "install",
            "."
        ]
    }
    ],
    "options": {
        "logStreamingOption": "STREAM_ON"
    }
}

logging: Use this option to specify if you want to store logs in Stackdriver Logging or Cloud Storage. If you do not set this option, Cloud Build stores the logs in both Stackdriver Logging and Cloud Storage. You can set the logging option to GCS_ONLY to store the logs only in Cloud Storage. The following snippet specifies that the logs are stored in Cloud Storage:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
options:
 logging: GCS_ONLY

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    }
    ],
    "options": {
        "logging": "GCS_ONLY"
    }
}

substitutionOption: You'll set this option along with the substitutions field below to specify the behavior when there is an error in the substitution checks.

substitutions

Use substitutions in your build config file to substitute specific variables at build time. Substitutions are helpful for variables whose value isn't known until build time, or to re-use an existing build request with different variable values. By default, the build returns an error if there's a missing substitution variable or a missing substitution. However, you can use the ALLOW_LOOSE option to skip this check.

The following snippet uses substitutions to print "hello world." The ALLOW_LOOSE substitution option is set, which means the build will not return an error if there's a missing substitution variable or a missing substitution.

YAML

steps:
- name: 'ubuntu'
  args: ['echo', 'hello ${_SUB_VALUE}']
substitutions:
    _SUB_VALUE: world
options:
    substitution_option: 'ALLOW_LOOSE'

JSON

{
    "steps": [
    {
        "name": "ubuntu",
        "args": [
            "echo",
            "hello ${_SUB_VALUE}"
        ]
    }
    ],
    "substitutions": {
        "_SUB_VALUE": "world"
},
    "options": {
        "substitution_option": "ALLOW_LOOSE"
    }
}

For additional instructions on using substitutions, see Substituting variable values.

tags

Use the tags field to organize your builds into groups and to filter your builds. The following config sets two tags named mytag1 and mytag2:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  ...
- name: 'ubuntu'
  ...
tags: ['mytag1', 'mytag2']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker"
    },
    {
        "name": "ubuntu"
    }
    ],
    "tags": [
        "mytag1",
        "mytag2"
    ]
}

secrets

Secret pairs a set of secret environment variables containing encrypted values with the Cloud KMS key to use to decrypt the value. Use the secrets field to define secrets to decrypt using Cloud KMS. For an example of how to use this field, see Using encrypted secrets and credentials.

images

The images field in the build config file specifies one or more Docker images to be pushed by Cloud Build to Container Registry. You may have a build that performs tasks without producing any Docker images, but if you build images and don't push them to Container Registry, the images are discarded on build completion. If a specified image is not produced during the build, the build will fail. For more information on storing images see Storing Images and Artifacts.

The following build config sets the images field to store the built image:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/myproject/myimage', '.']
images: ['gcr.io/myproject/myimage']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "-t",
            "gcr.io/myproject/myimage",
            "."
        ]
    }
    ],
    "images": [
        "gcr.io/myproject/myimage"
    ]
}

artifacts

The artifacts field in the build config file specifies one or more non-container artifacts to be stored in Cloud Storage. For more information on storing non-container artifacts see Storing Images and Artifacts.

The following build config sets the artifacts field to store the built Go package to gs://mybucket/:

YAML

steps:
- name: 'gcr.io/cloud-builders/go'
  args: ['build', 'my-package']
artifacts:
  objects:
    location: 'gs://mybucket/'
    paths: ['my-package']

JSON

{
    "steps": [
    {
        "name": "gcr.io/cloud-builders/go",
        "args": [
            "build",
            "my-package"
        ]
    }
    ],
    "artifacts": {
      "objects": {
        "location": "gs://mybucket/",
        "paths": [
            "my-package"
        ]
      } 
    }
}

What's next

• Learn how to create a basic build config file to configure builds for Cloud Build.
• Read Starting a Build Manually for instructions on how to run builds in Cloud Build.