Set and remove custom metadata


Each metadata entry is stored on the metadata server as key-value pairs. Metadata keys are case sensitive. Your keys can be either predefined or custom metadata keys.

Custom metadata enables you to create and use your own metadata key-value pairs on an individual VM or a project. You can add new custom metadata keys, update the values of your existing keys, and remove any custom metadata entries when you don't need them. Setting custom metadata is useful for passing in arbitrary values to VMs in a project. It is also useful for creating startup and shutdown scripts.

This documents provides information about how to do the following:

Before you begin

  • Review the basics of how VM metadata for Compute Engine is defined, categorized, and arranged. For more information, see About VM metadata.
  • If you haven't already, set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine as follows.

    Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    1. Install the Google Cloud CLI, then initialize it by running the following command:

      gcloud init
    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI, then initialize it by running the following command:

      gcloud init

Required roles

To get the permissions that you need to set or remove custom metadata on VMs, ask your administrator to grant you the following IAM roles:

For more information about granting roles, see Manage access.

These predefined roles contain the permissions required to set or remove custom metadata on VMs. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to set or remove custom metadata on VMs:

  • If your VMs use service accounts: iam.serviceAccounts.actAs on the service accounts or project
  • To add, update, or remove custom project-wide metadata:
    • compute.projects.get on the project
    • compute.projects.setCommonInstanceMetadata on the project
  • To add, update, or remove custom project zonal metadata:
    • compute.instanceSettings.get on the instance settings in the required zone in the project
    • compute.instanceSettings.update on the instance settings in the required zone in the project
  • To add, update, or remove custom instance metadata:
    • compute.instances.get on the VM
    • compute.instances.setMetadata on the VM

You might also be able to get these permissions with custom roles or other predefined roles.

Limitations

Compute Engine enforces a combined total limit of 512 KB for all metadata entries. Maximum size limits are also applied to each key and value in the following way:

  • Each metadata key has a maximum limit of 128 bytes.
  • Each metadata key is case sensitive.

    For project zonal metadata, Compute Engine doesn't allow you to create two separate metadata keys with the same string, even if they are written in different cases. For example, if you have an existing project zonal metadata key called project-zonal-metadata-key, then you can't create new project zonal metadata keys such as Project-Zonal-Metadata-Key, PROJECT-ZONAL-METADATA-KEY, or any other variations of the same string.

  • Each metadata value has a maximum limit of 256 KB.

  • Each metadata value is case sensitive except for boolean values.

For example, SSH keys are stored as custom metadata under the ssh-keys key. If your metadata content or value for this key exceeds the 256 KB limit, you won't be able to add more SSH keys. If you run into this limit, consider removing unused keys to free up metadata space for new keys.

Also, if you provide the startup or shutdown script contents directly, the contents of these startup and shutdown script contents might also be stored as custom metadata and count toward these size limitations. To avoid this, store your startup or shutdown script as a file hosted at an external location, such as Cloud Storage, and provide the startup script URL when creating a VM. This way, these files are downloaded onto the VM, rather than stored in the metadata server.

Boolean values

For fields that accept boolean values, TRUE or FALSE, the following values can also be used:

Status Alternative values
TRUE Y, Yes, 1
FALSE N, No, 0

Boolean values are not case-sensitive. For example, you can use False, false, or FALSE to disable a feature.

Set custom metadata on VMs

This section provides information about how to add new custom metadata, or update existing custom metadata values, for your Compute Engine VMs in one of the following ways:

Set custom project-wide metadata

You can add or update the custom metadata for all instances in a project by using the Google Cloud console, the Google Cloud CLI, or REST.

Use these instructions to apply metadata settings to all VMs in the project. For example, if you define a project-wide metadata pair of baz=bat, that metadata pair is automatically applied to all VMs in the project.

Console

  1. In the Google Cloud console, go to the Metadata page.

    Go to the Metadata page

  2. Click Edit.
  3. Add or edit a metadata entry.
  4. Save your changes.

gcloud

Use the project-info add-metadata command.

gcloud compute project-info add-metadata \
    --metadata=KEY=VALUE

Replace the following:

  • KEY: the name of your metadata key
  • VALUE: the value stored for this key

Example

For example to set two new entries foo=bar and baz=bat on a project, run the following command:

gcloud compute project-info add-metadata \
    --metadata=foo=bar,baz=bat

You can optionally specify one or more files from which to read metadata by using the --metadata-from-file flag.

REST

  1. Optional. To perform optimistic locking, you can optionally provide a fingerprint.

    A fingerprint is a random string of characters generated by Compute Engine. The fingerprint changes after each request, and if you provide a mismatched fingerprint, your request is rejected.

    If you do not provide a fingerprint, no check for consistency is performed, and the projects().setCommonInstanceMetadata request succeeds. This behaviour is different from instances().setMetadata and instanceSettings().patch methods, where a fingerprint is always required.

    To get the current fingerprint of a project, call the project().get method.

    GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
    

    The output is similar to the following:

    {
      "name": "myproject",
      "commonInstanceMetadata": {
        "kind": "compute#metadata",
        "fingerprint": "FikclA7UBC0=",
        ...
      }
    }
    
  2. Make a request to the projects().setCommonInstanceMetadata method and set your custom metadata key-value pairs:

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/setCommonInstanceMetadata
    
    {
      "fingerprint": "FikclA7UBC0=",
      "items": [
        {
        "key": "foo",
        "value": "bar"
        }
      ]
    }
    

Replace PROJECT_ID with your your project ID.

Set custom project zonal metadata

You can add or update the project zonal metadata entries for your VMs in one of the following ways:

  • You can add new custom metadata keys at a project zonal level and set the metadata values for all VMs that are in a specific zone in a project.
  • You can update the values of existing custom project zonal metadata keys for all VMs that are in a specific zone in a project.
  • For project metadata keys that already have project-wide values, you can override those project-wide values for all VMs in a specific zone and specify those VMs to instead use project zonal values.

Things to note before you set custom project zonal metadata

Project and project zonal metadata entries are stored in the same project/ directory. If you set different values for the same custom metadata keys for VMs on a project level and on a project zonal level, then the project zonal metadata values for those keys take precedence over the project-wide metadata values in the respective zones.

  • If you add a project zonal metadata value for a metadata key that already has a project-wide value, then Compute Engine overrides the project-wide value for the VMs in this specified zone and updates the /project directory with the project zonal value.
  • If you add a new project-wide metadata value for a metadata key that already has a project zonal value, then nothing changes. Compute Engine retains the project zonal value in the /project directory in the specific zone.
  • If you don't specify a project zonal value for a custom metadata key in a specific zone, but the key has a project-wide value, then your VMs continue to have the project-wide values in those zones.

Procedure

You can add or update the custom metadata for all VM instances in a specific zone in a project by using the Google Cloud CLI or REST.

gcloud

  • To add or update custom project zonal metadata, use the gcloud beta compute project-zonal-metadata add command.

    gcloud beta compute project-zonal-metadata add \
      --zone=ZONE \
      --project=PROJECT_ID  \
      --metadata=KEY1=VALUE1,KEY2=VALUE2,...

    Replace the following:

    • PROJECT_ID: your project ID
    • ZONE: the zone where you want to add or update project zonal metadata.
    • KEY1, KEY2...: the custom project zonal metadata keys for which you want to add or update values.
    • VALUE1, VALUE2...: the project zonal metadata values that you want to set for your existing and new project zonal metadata keys. Depending on your custom metadata key and value, one of the following happens:

    • If the corresponding custom metadata key is an existing project-wide key, then Compute Engine overrides the project-wide metadata value and uses the project zonal value for all VMs in the specified zone. All VMs in the specified zone inherit the new project zonal value. If you make any future changes to the project-wide value for the same key, then those updates don't affect this project zonal value and VMs in this zone continue to have the project zonal value as their project metadata.

    • If the corresponding custom metadata key is an existing project zonal metadata key and the specified metadata value is a new value, then Compute Engine updates the project zonal value of the existing key.

    • If the corresponding custom metadata key is a new key that is not part of the existing project-wide or project zonal metadata, then Compute Engine creates the project zonal metadata key and adds this metadata value.

    • If the corresponding custom metadata key is an existing project zonal metadata key and the specified metadata value is the same as the existing value, then the project zonal metadata entry remains unchanged.

Example: Add a new custom project zonal metadata entry

For example, consider a project called my-project with the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"value-1" and "key-2":"value-2"

To add "key-4":"value-4" as a new custom project zonal metadata pair in the us-central1-a zone, run the following command

gcloud beta compute project-zonal-metadata add \
    --metadata=key-4=value-4 \
    --project=my-project \
    --zone=us-central1-a

Example: Update the values of an existing custom project zonal metadata entry

Consider the same example project my-project, which now has the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"value-1", "key-2":"value-2", and "key-4":"value-4"

To update the project zonal metadata values of key-1 and key-4 in us-cerntral1-a zone with new values, run the following command.

gcloud beta compute project-zonal-metadata add \
    --metadata=key-1=new-value-1,key-4=new-value-4 \
    --project=my-project \
    --zone=us-central1-a

Example: Override the project-wide value for a metadata key and use a project zonal value

Consider the same example project my-project, which now has the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"new-value-1", "key-2":"value-2", and "key-4":"new-value-4"

In this example project, consider key-3, which has a project-wide metadata value of value-c. Suppose you want to set a project zonal metadata value value-3 for this key for all VMs in the us-central1-a zone. When you perform the operation, for all the VMs in the us-central1-a zone, Compute Engine overrides the project-wide values and uses the project zonal values. VMs in all other zones of the project retain their prevailing project-wide or project zonal metadata values for key-3.

To override the project-wide value for key-3 and set a project zonal value, run the following command:

gcloud beta compute project-zonal-metadata add \
    --metadata=key-3=value-3 \
    --project=my-project \
    --zone=us-central1-a

REST

  1. Get the current fingerprint and view any existing key-value pairs for the project in that zone.

    To perform optimistic locking, you must provide a fingerprint. A fingerprint is a random string of characters generated by Compute Engine. The fingerprint changes every time you make a request to add, update, or remove project zonal metadata, and if you provide a mismatched fingerprint, Compute Engine rejects your request.

    If you do not provide a fingerprint, a check for consistency is performed and your update request doesn't succeed. This works so that only one request can be made at a time, preventing collisions. This behavior matches instances().setMetadata, where a fingerprint is always required.

    To get the current fingerprint of the project zonal metadata, make a GET request to the instanceSettings().get method.

    GET https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instanceSettings
    

    Replace the following:

    • PROJECT_ID: your project ID
    • ZONE: the zone where you want to set the project zonal metadata.

    The following is an example output for this request:

        {
          "fingerprint": "VlRIl8dx9vk=",
          "metadata": {
            ...
          }
        }
    
  2. To add or update the project zonal metadata, make a PATCH request to the instanceSettings().patch method. You must provide the following with your request:

    • An update mask. Use the update_mask query parameter. The update mask must contain the metadata keys for the following:

      • The new custom project zonal metadata that you want to add
      • The existing custom project zonal metadata for which you want to update values

      You must add the string metadata.items. as a prefix for each key—for example, metadata.items.key1,metadata.items.key3.

    • In the request body, provide the following:

      • The metadata keys and values for the new custom project zonal metadata that you want to add
      • The metadata keys and values for the existing custom project zonal metadata that you want to update
      • The current fingerprint value

    PATCH https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instanceSettings?update_mask=PREFIXED_METADATA_KEYS
    {
      "fingerprint": "FINGER_PRINT",
      "metadata": {
        "items": {
          "KEY1": "VALUE1",
          "KEY2": "VALUE2",
          ...
        }
      }
    }
    

    Replace the following:

    • PROJECT_ID: the ID of the project.
    • ZONE: the zone where you want to set the project zonal metadata.
    • PREFIXED_METADATA_KEYS: the list of metadata keys for the following, where each key is prefixed with the string metadata.items. :

      • The new custom project zonal metadata that you want to add
      • The existing custom project zonal metadata for which you want to update values

      For example, suppose your current project zonal metadata keys are key-1 and key-2. If you want to add a new metadata key key-3 and also update one of the current metadata keys, key-1, then your update mask must have the following string:

      metadata.items.key1,metadata.items.key3

    • FINGER_PRINT: the current fingerprint value.

    • KEY1, KEY2...: the custom project zonal metadata keys for which you want to add or update values. Specify all the custom project zonal metadata keys that you specified in the update mask.

    • VALUE1, VALUE2...: the project zonal metadata values that you want to set for your existing and new project zonal metadata keys. Depending on your custom metadata key and value, one of the following happens:

      • If the corresponding custom metadata key is an existing project-wide key, then Compute Engine overrides the project-wide metadata value and uses the project zonal value for all VMs in the specified zone. All VMs in the specified zone inherit the new project zonal value. If you make any future changes to the project-wide value for the same key, then those updates don't affect this project zonal value and VMs in this zone continue to have the project zonal value as their project metadata.
      • If the corresponding custom metadata key is an existing project zonal metadata key and the specified metadata value is a new value, then Compute Engine updates the project zonal value of the existing key.
      • If the corresponding custom metadata key is a new key that is not part of the existing project-wide or project zonal metadata, then Compute Engine creates the project zonal metadata key and adds this metadata value.
      • If the corresponding custom metadata key is an existing project zonal metadata key and the specified metadata value is the same as the existing value, then the project zonal metadata entry remains unchanged.

Example: Add a new custom project zonal metadata entry

For example, consider a project called my-project with the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"value-1" and "key-2":"value-2"

To add "key-4":"value-4" as a new custom project zonal metadata pair in the us-central1-a zone, make the following PATCH request:

PATCH https://compute.googleapis.com/compute/beta/projects/my-project/zones/us-central1-a/instanceSettings?update_mask=metadata.items.key-4
{
  "fingerprint": "VlRIl8dx9vk=",
  "metadata": {
  "items": {
    "key-4": "value-4"
    }
  }
}

Example: Update the values of an existing custom project zonal metadata entry

Consider the same example project my-project, which now has the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"value-1", "key-2":"value-2", and "key-4":"value-4"

To update the project zonal metadata values of key-1 and key-4 in us-cerntral1-a zone with new values, make the following PATCH request:

PATCH https://compute.googleapis.com/compute/beta/projects/my-project/zones/us-central1-a/instanceSettings?update_mask=metadata.items.key-1,metadata.items.key-4
{
  "fingerprint": "VlRIl8dx9vk=",
  "metadata": {
  "items": {
    "key-1": "new-value-1",
    "key-4": "new-value-4"
    }
  }
}

Example: Override the project-wide value for a metadata key and use a project zonal value

Consider the same example project my-project, which now has the following custom metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"new-value-1", "key-2":"value-2", and "key-4":"new-value-4"

In this example project, consider key-3, which has a project-wide metadata value of value-c. Suppose you want to set a project zonal metadata value value-3 for this key for all VMs in the us-central1-a zone. When you perform the operation, for all the VMs in the us-central1-a zone, Compute Engine overrides the project-wide values and uses the project zonal values. VMs in all other zones of the project retain their prevailing project-wide or project zonal metadata values for key-3.

To override the project-wide metadata value for key-3 and use the custom project zonal value value-3 instead, make the following PATCH request:

PATCH https://compute.googleapis.com/compute/beta/projects/my-project/zones/us-central1-a/instanceSettings?update_mask=metadata.items.key-3
{
  "fingerprint": "VlRIl8dx9vk=",
  "metadata": {
  "items": {
    "key-3": "value-3"
    }
  }
}

Set custom instance metadata

You can add or update the custom metadata for a single VM instance by using the Google Cloud console, the Google Cloud CLI, or REST.

You can set custom instance metadata in one of the following ways:

Add custom instance metadata during VM creation

Use these instructions to add metadata on a specific VM instance at the time of its creation.

Console

  1. In the Google Cloud console, go to the Create an instance page.

    Go to Create an instance

  2. Specify the VM details.

  3. Expand the Advanced options section, and do the following:

    1. Expand the Management section.
    2. To add multiple key-value pairs for your custom metadata, in the Metadata section, click Add item.
  4. To create the VM, click Create.

gcloud

To set custom metadata, use the gcloud compute instances create command with the --metadata flag.

gcloud compute instances create VM_NAME \
    --metadata=KEY=VALUE

Replace the following:

  • VM_NAME: the name of your VM
  • KEY: the name of your metadata key
  • VALUE: the value stored for this key

Example

For example to set a new key foo that has a value bar on a VM named example-instance, run the following command:

gcloud compute instances create example-instance \
    --metadata=foo=bar

REST

Use the instances.insert method and provide the custom metadata as part of the metadata property in your request:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

{
  "machineType": "zones/MACHINE_TYPE_ZONE/machineTypes/MACHINE_TYPE",
  "name": "VM_NAME",
  "...": [
    {
    }
    ],
  "metadata": {
    "items": [
      {
        "key": "KEY",
        "value": "VALUE"
      }
    ]
  },
  ..
}

Replace the following:

  • PROJECT_ID: your project ID
  • ZONE: zone to create the VM in
  • MACHINE_TYPE: machine type, predefined or custom, for the new VM
  • VM_NAME: name of the new VM
  • KEY: the name of your metadata key
  • VALUE: the value stored for this key

Add or update custom instance metadata on an existing VM

Use these instructions to update metadata on a specific VM instance that already exists.

Console

  1. In the Google Cloud console, go to the VM instances page.

    Go to the VM instances page

  2. Click the instance for which you want to update metadata.
  3. Click the Edit button at the top of the page.
  4. Under Custom metadata, click Add item or edit the existing metadata entries.
  5. Save your changes.

gcloud

Updating VM metadata with the gcloud CLI is an additive action. Specify only the metadata keys that you want to add or change. If a key that you provided already exists, the value for that key is updated with the new value.

Use the instances add-metadata command:

gcloud compute instances add-metadata VM_NAME \
    --metadata=KEY=VALUE,KEY=VALUE

Replace the following:

  • VM_NAME: the name of your VM
  • KEY: the name of your metadata key
  • VALUE: the value stored for this key

Examples

If you want to add the foo=bar entry, use:

gcloud compute instances add-metadata VM_NAME \
    --metadata=foo=bar

If you want to change the foo=bar entry to foo=bat, use:

gcloud compute instances add-metadata VM_NAME \
    --metadata=foo=bat

REST

  1. Get the current fingerprint and view any existing key-value pairs for the VM. To do this, call the instances().get method.

    A fingerprint is a random string of characters generated by Compute Engine and is used to perform optimistic locking. To update the VM, you need to provide the matching fingerprint value. The fingerprint changes after each request, and if you provide a mismatched fingerprint, your request is rejected. This works so that only one update can be made at a time, preventing collisions.

    GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME
    

    Replace the following:

    • PROJECT_ID: your project ID
    • ZONE: the zone where your VM is located
    • VM_NAME: name of your VM

    The output is similar to the following:

    {
      ...
      "name": "example-instance",
      "metadata": {
        "kind": "compute#metadata",
        "fingerprint": "zhma6O1w2l8="
        "items": [
          {
            "key": "foo",
            "value": "bar"
          }
        ]
        },
      ...
    }
    
  2. Make a request to the instances().setMetadata method. Provide a list of the new metadata values and the current fingerprint value.

    If the VM has existing key-value pairs that you want to keep, you must include them in this request with the new key-value pairs.

    Example

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/setMetadata
    
    {
    "fingerprint": "zhma6O1w2l8=",
    "items": [
      {
        "key": "foo",
        "value": "bar"
      },
      {
        "key": "baz",
        "value": "bat"
      }
    ]
    }
    

    Replace the following:

    • PROJECT_ID: your project ID
    • ZONE: the zone where your VM is located
    • VM_NAME: name of your VM

Remove custom metadata from VMs

You can remove custom metadata from your Compute Engine VMs in one of the following ways:

Remove custom project-wide metadata

You can remove custom project-wide metadata by using the Google Cloud console or the Google Cloud CLI.

Console

  1. In the Google Cloud console, go to the Metadata page.

    Go to the Metadata page

  2. Click edit Edit at the top of the page.
  3. In the Metadata section, click delete Delete item for each of the metadata entries that you want to remove.
  4. To confirm your changes and remove the custom project-wide metadata, click Save.

gcloud

To remove custom project-wide metadata, use the gcloud compute project-info remove-metadata command.

  • If you want to remove the custom metadata entries for specific keys, specify those keys by using the --keys flag, and exclude the values of those keys.
gcloud compute project-info remove-metadata \
    --keys=KEY1,KEY2,...
  • If you want to remove all custom metadata for the VM, specify the --all flag.
gcloud compute project-info remove-metadata \
    --all

Replace the following:

  • KEY1, KEY2...: the custom instance metadata keys that you want to remove.

Remove custom project zonal metadata

You can remove custom project zonal metadata by using the Google Cloud CLI or the Compute Engine API.

gcloud

To remove custom project zonal metadata, use the gcloud beta compute project-zonal-metadata remove command and specify all the metadata keys that you want to remove by using the --keys flag.

gcloud beta compute project-zonal-metadata remove \
    --project=PROJECT_ID  \
    --zone=ZONE \
    --keys=KEY1,KEY2,...

Replace the following:

  • PROJECT_ID: your project ID
  • ZONE: the zone where you want to set the project zonal metadata.
  • KEY1, KEY2...: the custom project zonal metadata keys that you want to remove.

After you run the command, if any of the specified keys have project-wide metadata values available, then the VMs in the specified zone inherit those project-wide values. If the metadata key was set only at a project zonal level and there isn't a corresponding project-wide key, then the VMs lose that metadata information.

Example:

Consider an example project my-project, which has the following custom project metadata:

  • Project-wide metadata: "key-1":"value-a", "key-2":"value-b", and "key-3":"value-c"
  • Project zonal metadata in us-central1-a zone: "key-1":"new-value-1", "key-2":"value-2", "key-3":"value-3", and "key-4":"new-value-4"

To remove all the project zonal metadata for the us-central1-a zone, run the following command.

gcloud beta compute project-zonal-metadata remove \
    --metadata=key-1,key-2,key-3,key-4 \
    --project=my-project \
    --zone=us-central1-a

After you run the command, VMs in the us-central1-a zone possess the following custom project metadata:

  • "key-1":"value-a"
  • "key-2":"value-b"
  • "key-3":"value-c"

REST

  1. Get the current fingerprint and view any existing key-value pairs for the project in that zone.

    To perform optimistic locking, you must provide a fingerprint. A fingerprint is a random string of characters generated by Compute Engine. The fingerprint changes every time you make a request to add, update, or remove project zonal metadata, and if you provide a mismatched fingerprint, Compute Engine rejects your request.

    If you do not provide a f