Substituting variable values

Use substitutions in your build config file to substitute specific variables at build time.

Substitutions are helpful for variables whose value isn't known until build time, or to re-use an existing build request with different variable values.

Cloud Build provides built-in substitutions or you can define your own substitutions. Use substitutions in your build's steps and images to resolve their values at build time.

This page explains how to use default substitutions or define your own substitutions.

Using default substitutions

Cloud Build provides the following default substitutions for all builds:

• $PROJECT_ID: ID of your Cloud project
• $BUILD_ID: ID of your build
• $PROJECT_NUMBER: your project number
• $LOCATION: the region associated with your build

Cloud Build provides the following default substitutions for builds invoked by triggers:

• $TRIGGER_NAME: the name associated with your trigger
• $COMMIT_SHA: the commit ID associated with your build
• $REVISION_ID: the commit ID associated with your build
• $SHORT_SHA : the first seven characters of COMMIT_SHA
• $REPO_NAME: the name of your repository
• $REPO_FULL_NAME: the full name of your repository, including either the user or organization
• $BRANCH_NAME: the name of your branch
• $TAG_NAME: the name of your tag
• $REF_NAME: the name of your branch or tag
• $TRIGGER_BUILD_CONFIG_PATH: the path to your build configuration file used during your build execution; otherwise, an empty string if your build is configured inline on the trigger or uses a Dockerfile or Buildpack.
• $SERVICE_ACCOUNT_EMAIL: email of the service account you are using for the build. This is either a default service account or a user-specified service account.
• $SERVICE_ACCOUNT: the resource name of the service account, in the format projects/PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT_EMAIL

Note: Both $COMMIT_SHA and $REVISION_ID represent the commit ID associated with your build. Support for both substitutions is available.

Cloud Build provides the following GitHub-specific default substitutions available for pull request triggers:

• $_HEAD_BRANCH : head branch of the pull request
• $_BASE_BRANCH : base branch of the pull request
• $_HEAD_REPO_URL : url of the head repo of the pull request
• $_PR_NUMBER : number of the pull request

If a default substitution is not available (such as with sourceless builds, or with builds that use storage source), then occurrences of the missing variable are replaced with an empty string.

When starting a build using gcloud builds submit, you can specify variables that would normally come from triggered builds with the --substitutions argument. Specifically, you can manually provide values for:

• $TRIGGER_NAME
• $COMMIT_SHA
• $REVISION_ID
• $SHORT_SHA
• $REPO_NAME
• $REPO_FULL_NAME
• $BRANCH_NAME
• $TAG_NAME
• $REF_NAME
• $TRIGGER_BUILD_CONFIG_PATH
• $SERVICE_ACCOUNT_EMAIL
• $SERVICE_ACCOUNT

For example, the following command uses the TAG_NAME substitution:

gcloud builds submit --config=cloudbuild.yaml \
    --substitutions=TAG_NAME="test"

The following example uses the default substitutions $BUILD_ID, $PROJECT_ID, $PROJECT_NUMBER, and $REVISION_ID.

YAML

steps:
# Uses the ubuntu build step:
# to run a shell script; and
# set env variables for its execution
- name: 'ubuntu'
  args: ['bash', './myscript.sh']
  env:
  - 'BUILD=$BUILD_ID'
  - 'PROJECT_ID=$PROJECT_ID'
  - 'PROJECT_NUMBER=$PROJECT_NUMBER'
  - 'REV=$REVISION_ID'

# Uses the docker build step to build an image called my-image
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-image', '.']

# my-image is pushed to Container Registry
images:
- 'gcr.io/$PROJECT_ID/my-image'

JSON

{
  "steps": [{
      "name": "ubuntu",
      "args": [
        "bash",
        "./myscript.sh"
      ],
      "env": [
        "BUILD=$BUILD_ID",
        "PROJECT_ID=$PROJECT_ID",
        "PROJECT_NUMBER=$PROJECT_NUMBER",
        "REV=$REVISION_ID"
      ]
    }, {
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/$PROJECT_ID/my-image", "."]
    }],
  "images": [
    "gcr.io/$PROJECT_ID/my-image"
  ]
}

The example below shows a build request using the docker build step to build an image, then pushes the image to Container Registry using the default $PROJECT_ID substitution:

In this example:

• The build request has one build step, which uses the docker build step in gcr.io/cloud-builders to build the Docker image.
  • The args field in the step specifies the arguments to pass to the docker command, in this case build -t gcr.io/my-project/cb-demo-img ., will be invoked (after $PROJECT_ID is substituted with your project ID).

• The images field contains the image's name. If the build is successful, the resulting image is pushed to Container Registry. If the image is not created successfully by the build, the build will fail.

YAML

steps:
- name: gcr.io/cloud-builders/docker
  args: ["build", "-t", "gcr.io/$PROJECT_ID/cb-demo-img", "."]
images:
- gcr.io/$PROJECT_ID/cb-demo-img

JSON

{
  "steps": [{
      "name": "gcr.io/cloud-builders/docker",
      "args": ["build", "-t", "gcr.io/$PROJECT_ID/cb-demo-img", "."]
    }],
  "images": [
    "gcr.io/$PROJECT_ID/cb-demo-img"
  ]
}

Using user-defined substitutions

You can also define your own substitutions. User-defined substitutions must conform to the following rules:

• Substitutions must begin with an underscore (_) and use only uppercase-letters and numbers (respecting the regular expression _[A-Z0-9_]+). This prevents conflicts with built-in substitutions. To use an expression starting with $ you must use $$. Thus:
  • $FOO is invalid since it is not a built-in substitution.
  • $$FOO evaluates to the literal string $FOO.
• The number of parameters is limited to 200 parameters. The length of a parameter key is limited to 100 bytes and the length of a parameter value is limited to 4000 bytes.

You can specify variables in one of two ways: $_FOO or ${_FOO}:

• Both $_FOO and ${_FOO} evaluate to the value of _FOO. However, ${} lets the substitution work without surrounding spaces, which allows for substitutions like ${_FOO}BAR.
• $$ allows you to include a literal $ in the template. Thus:
  • $_FOO evaluates to the value of _FOO.
  • $$_FOO evaluates to the literal string $_FOO.
  • $$$_FOO evaluates to the literal string $ followed by the value of _FOO.

To use the substitutions, use the --substitutions argument in the gcloud command or specify them in the config file.

The following example shows a build config with two user-defined substitutions called _NODE_VERSION_1 and _NODE_VERSION_2:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build',
         '--build-arg',
         'node_version=${_NODE_VERSION_1}',
         '-t',
         'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}',
         '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build',
         '--build-arg',
         'node_version=${_NODE_VERSION_2}',
         '-t',
         'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}',
         '.']
substitutions:
    _NODE_VERSION_1: v6.9.1 # default value
    _NODE_VERSION_2: v6.9.2 # default value
images: [
    'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}',
    'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}'
]

JSON

{
    "steps": [{
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "--build-arg",
            "node_version=${_NODE_VERSION_1}",
            "-t",
            "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}",
            "."
        ]
    }, {
        "name": "gcr.io/cloud-builders/docker",
        "args": [
            "build",
            "--build-arg",
            "node_version=${_NODE_VERSION_2}",
            "-t",
            "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}",
            "."
        ]
    }],
    "substitutions": {
        "_NODE_VERSION_1": "v6.9.1"
        "_NODE_VERSION_1": "v6.9.2"
    },
    "images": [
        "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}",
        "gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_2}"
    ]
}

To override the substitution value you specified in the build config file, use the --substitutions flag in the gcloud builds submit command. Note that substitutions are a mapping of variables to values rather than arrays or sequences. You can override default substitution variable values except for $PROJECT_ID and $BUILD_ID. The following command overrides the default value for _NODE_VERSION_1 specified in the build config file above:

gcloud builds submit --config=cloudbuild.yaml \
  --substitutions=_NODE_VERSION_1="v6.9.4",_NODE_VERSION_2="v6.9.5" .

By default, the build returns an error if there's a missing substitution variable or a missing substitution. However, you can set the ALLOW_LOOSE option to skip this check.

The following snippet prints "hello world" and defines an unused substitution. Because the ALLOW_LOOSE substitution option is set, the build will be successful despite the missing substitution.

YAML

steps:
- name: 'ubuntu'
  args: ['echo', 'hello world']
substitutions:
    _SUB_VALUE: unused
options:
    substitutionOption: 'ALLOW_LOOSE'

JSON

{
    "steps": [
    {
        "name": "ubuntu",
        "args": [
            "echo",
            "hello world"
        ]
    }
    ],
    "substitutions": {
        "_SUB_VALUE": "unused"
},
    "options": {
        "substitution_option": "ALLOW_LOOSE"
    }
}

If your build is invoked by a trigger, the ALLOW_LOOSE option is set by default. In this case, your build will not return an error if there is a missing substitution variable or a missing substitution. You cannot override the ALLOW_LOOSE option for builds invoked by triggers.

If the ALLOW_LOOSE option is not specified, unmatched keys in your substitutions mapping or build request will result in error. For example, if your build request includes $_FOO and the substitutions mapping doesn't define _FOO, you will receive an error after running your build or invoking a trigger if your trigger includes substitution variables.

The following substitution variables always contain a default empty-string value even if you don't set the ALLOW_LOOSE option:

• $REPO_NAME
• $REPO_FULL_NAME
• $BRANCH_NAME
• $TAG_NAME
• $COMMIT_SHA
• $SHORT_SHA

When defining a substitution variable, you aren't limited to static strings. You also have access to the event payload that invoked your trigger. These are available as payload bindings. You can also apply bash parameter expansions on substitution variables and store the resulting string as a new substitution variable. To learn more, see Using payload bindings and bash parameter expansions in substitutions.

Dynamic substitutions

You can reference the value of another variable within a user-defined substitution by setting the dynamicSubstitutions option to true in your build config file. If your build is invoked by a trigger, the dynamicSubstitutions field is always set to true and does not need to be specified in your build config file. If your build is invoked manually, you must set the dynamicSubstitutions field to true for bash parameter expansions to be interpreted when running your build.

The following build config file shows the substitution variable ${_IMAGE_NAME} referencing the variable, ${PROJECT_ID}. The dynamicSubstitutions field is set to true so the reference is applied when invoking a build manually:

YAML

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', '${_IMAGE_NAME}', '.']
substitutions:
    _IMAGE_NAME: 'gcr.io/${PROJECT_ID}/test-image'
options:
    dynamicSubstitutions: true

JSON

{
   "steps": [
      {
         "name": "gcr.io/cloud-builders/docker",
         "args": [
            "build",
            "-t",
            "${_IMAGE_NAME}",
            "."
         ]
      }
   ],
   "substitutions": {
      "_IMAGE_NAME": "gcr.io/${PROJECT_ID}/test-image"
   },
   "options": {
      "dynamic_substitutions": true
   }
}

For more information, see Applying bash parameter expansions.

Mapping substitutions to environment variables

Scripts don't directly support substitutions, but they support environment variables. You can map substitutions to environment variables, either automatically all at once, or manually by defining every environment variable yourself.

Map substitutions automatically

• At the build level. To automatically map all the substitutions to environment variables, which will be available throughout the entire build, set automapSubstitutions to true as an option at the build level. For example, the following build config file shows the user-defined substitution $_USER and the default substitution $PROJECT_ID mapped to environment variables:

  YAML

  steps:
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Hello $_USER"
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Your project ID is $PROJECT_ID"
  options:
    automapSubstitutions: true
  substitutions:
    _USER: "Google Cloud"
  

  JSON

  {
    "steps": [
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Hello $_USER'"
      },
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'"
      }
    ],
    "options": {
      "automap_substitutions": true
    },
    "substitutions": {
      "_USER": "Google Cloud"
    }
  }
  

• At the step level. To automatically map all the substitutions and make them available as environment variables in a single step, set the automapSubstitutions field to true in that step. In the following example, only the second step will show the substitutions correctly, because it's the only one with automatic substitutions mapping enabled:

  YAML

  steps:
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Hello $_USER"
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Your project ID is $PROJECT_ID"
    automapSubstitutions: true
  substitutions:
    _USER: "Google Cloud"
  

  JSON

  {
    "steps": [
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Hello $_USER'"
      },
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'",
        "automap_substitutions": true
      }
    ],
    },
    "substitutions": {
      "_USER": "Google Cloud"
    }
  

  Additionally, you can make the substitutions available as environment variables in the entire build, then ignore them in one step. Set automapSubstitutions to true at the build level, then set the same field to false in the step where you want to ignore the substitutions. In the following example, even though mapping substitutions is enabled at the build level, the project ID will not be printed in the second step, because automapSubstitutions is set to false in that step:

  YAML

  steps:
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Hello $_USER"
  - name: 'ubuntu'
    script: |
      #!/usr/bin/env bash
      echo "Your project ID is $PROJECT_ID"
    automapSubstitutions: false
  options:
    automapSubstitutions: true
  substitutions:
    _USER: "Google Cloud"
  

  JSON

  {
    "steps": [
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Hello $_USER'"
      },
      {
        "name": "ubuntu",
        "script": "#!/usr/bin/env bash echo 'Your project ID is $PROJECT_ID'",
        "automap_substitutions": false
      }
    ],
    "options": {
      "automap_substitutions": true
    },
    },
    "substitutions": {
      "_USER": "Google Cloud"
    }
  

Map substitutions manually

You can manually map the substitutions to environment variables. Every environment variable is defined at the step level using the env field, and the scope of the variables is restricted to the step where they are defined. This field takes a list of keys and values.

The following example shows how to map the substitution $PROJECT_ID to the environment variable BAR:

YAML

steps:
- name: 'ubuntu'
  env:
  - 'BAR=$PROJECT_ID'
  script: 'echo $BAR'

JSON

{
  "steps": [
    {
      "name": "ubuntu",
      "env": [
        "BAR=$PROJECT_ID"
      ],
      "script": "echo $BAR"
    }
  ]
}

What's next

• Learn how to use payload bindings and bash parameter expansions in substitutions.
• Learn how to create a basic build configuration file.
• Learn how to create and manage build triggers.
• Learn how to run builds manually in Cloud Build.