About VM metadata

Every virtual machine (VM) instance stores its metadata on a metadata server. Your VM automatically has access to the metadata server API without any additional authorization. Compute Engine maintains the metadata keys and values for your VMs and projects in directories. Each directory stores metadata entries in the form of key-value pairs. Some directories contain subdirectories.

This document provides an overview about VM metadata and explains about the types and properties of VM metadata.

VM metadata uses

The following sections describe a few scenarios where you can use metadata entries to manage your VMs.

Startup and shutdown scripts

The metadata server is particularly useful when used in combination with startup and shutdown scripts because you can use the metadata server to programmatically get unique information about a VM, without additional authorization.

For example, you can write a startup script that gets the metadata key-value pair for a VM's external IP and use that IP in your script to set up a database. Because the Compute Engine predefined metadata keys are the same on every VM, you can reuse your script without having to update it for each VM. This helps you create less brittle code for your applications.

Host maintenance

The metadata server provides information about a VM's scheduling option in the scheduling/ metadata directory by using the maintenance-event key. You can use these metadata values to notify you when a maintenance event is about to happen so that you can prepare your environment for the event. For more information, see Get live migration notices.

Guest attributes

Guest attributes are a specific type of custom metadata that your applications can write to while running on your VMs. Use guest attributes only for use cases that require small amounts of data that don't change frequently. For more information about guest attributes, see Set and query guest attributes.

Partner attributes

Partner attributes are a specific type of instance metadata. Google Cloud services can use partner attributes to create a namespace within which they can define instance metadata entries. You can set, update, delete, and view the values of the instance metadata entries to configure that service.

For example, when you use managed workload identities for Compute Engine, you can specify the configuration details in the metadata entries of that service's namespace.

Metadata security considerations

When you make a request to get information from the metadata server, your request and the subsequent metadata response never leave the physical host that is running the VM.

However, any process that can query the metadata URL, has access to all values in the metadata server. This includes any custom metadata values, client certificates, and private keys that you write to the server. Google recommends that you exercise caution when writing sensitive values to the metadata server or when running third-party processes. You must sandbox any process that shouldn't be able to access the metadata server.

Metadata server endpoints

The metadata server is accessible from the following endpoints:

  • An http endpoint: http://metadata.google.internal/computeMetadata/v1. This is accessible from all VMs including Shielded VMs.
  • An https endpoint: https://metadata.google.internal/computeMetadata/v1. This is accessible from Shielded VMs only.

HTTPS metadata server endpoint

The HTTPS metadata server endpoint (https://metadata.google.internal/computeMetadata/v1) provides added security for transmission of information between the metadata server and the VM. This endpoint is only available for Shielded VMs.

Benefits of using the HTTPS metadata server endpoint

Using the https endpoint to query the metadata server provides the following benefits:

  • Improves security: helps to prevent unauthorized access to your sensitive metadata. It prevents an attacker from being able to perform any of the following actions:

    • Spoofing or impersonating the metadata server to gain access to a VM
    • Viewing or tampering with sensitive metadata before it reaches the VM

  • Reduces costs: helps you avoid the costs associated with security breaches

How the process works

For shielded VMs that have the guest environment installed, the following processes take place on your VM:

  1. When your VM boots, Compute Engine completes the following:

    1. Compute Engine creates three certificates as follows:

      • A self-signed root certificate: a unique certificate generated for the VM.
      • A server identity certificate: a certificate for the metadata server.

      • A client identity certificate: a certificate for the client. This certificate is not cached in the metadata server and is recreated on each call to the client certificate endpoint from the guest environment.

        For storage locations of client identity and root certificates, see Where are certificates stored.

    2. Compute Engine transfers the public portion of the root certificate to the guest environment of the VM by using a Google-generated UEFI variable. This root certificate is then stored on the VM.

  2. Periodically, the guest environment requests a client identity certificate. When this happens, the guest agent downloads this certificate from the metadata server and validates it by using the root certificate for that VM.

When you make a query to the HTTPS metadata server endpoint, you specify the client identity certificates which are then used by the metadata server and the VM to verify that this query is authorized.

Where are certificates stored

The following sections list the storage location for root and client identity certificates that are generated by Compute Engine.

Root certificates

CentOS/RHEL/Rocky

Root certificates for CentOS, Red Hat Enterprise Linux (RHEL), and Rocky Linux VMs are stored in the following locations:

  • /run/google-mds-mtls/root.crt
  • /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Debian/Ubuntu

Root certificates for Debian and Ubuntu VMs are stored in the following locations:

  • /run/google-mds-mtls/root.crt
  • /etc/ssl/certs/ca-certificates.crt

Fedora

Root certificates for Fedora VMs are stored in the following locations:

  • /run/google-mds-mtls/root.crt
  • /etc/pki/tls/certs/ca-bundle.crt

SLES

Root certificates for SUSE Linux Enterprise Server (SLES) VMs are stored in the following locations:

  • /run/google-mds-mtls/root.crt
  • /etc/ssl/ca-bundle.pem

Windows

Root certificates for Windows VMs are stored in the following locations:

  • C:\ProgramData\Google\ComputeEngine\mds-mtls-root.crt
  • Cert:\LocalMachine\Root
Client identity certificates

Client identity certificates are accessible to all processes that are running on the VM. This is needed so that all processes have access to the metadata server by using the https endpoint, similar to the http endpoint. For more information, see Metadata security considerations.

Linux

Client identity certificates for Linux VMs are stored in the following location:

/run/google-mds-mtls/client.key

Windows

Client identity certificates for Windows VMs are stored in the following locations:

  • C:\ProgramData\Google\ComputeEngine\mds-mtls-client.key
  • Cert:\LocalMachine\My

Predefined and custom metadata keys

Each metadata entry is stored on the metadata server as key-value pairs. Metadata keys are case sensitive. Your keys can be either predefined or custom metadata keys.

Predefined metadata keys

Predefined metadata keys are metadata keys that are created by Compute Engine. When you create a VM, Compute Engine automatically sets the metadata values for some of these keys on that VM—for example, the VM instance ID or the project ID. For predefined keys where Compute Engine doesn't automatically set a value, you can choose from a set of values that are available depending on the system configuration. For example, to enable OS login for a VM, you can set the value of the enable-oslogin predefined key to TRUE for that VM. To disable OS login for that VM, you can update the value of the key to FALSE. You can only update the values for these keys but not the keys themselves.

For more information about predefined metadata keys and a list of these keys, see Predefined metadata keys.

Custom metadata keys

Custom metadata enables you to create and use your own metadata key-value pairs on an individual VM or a project. You can add new custom metadata keys, update the values of your existing keys, and remove any custom metadata entries when you don't need them. Setting custom metadata is useful for passing in arbitrary values to VMs in a project. It is also useful for creating startup and shutdown scripts.

To learn how you can add, update, or remove custom metadata for your VMs, see Configure custom metadata.

Types of metadata

VM metadata entries can provide information specific to an individual VM or a project. Your metadata is divided into project, zonal, and instance metadata, based on the scope at which you set the metadata.

Project metadata

Project metadata is defined at project scope and provides information that applies to all VMs in a project. When you set this metadata, the metadata entries propagate to all VMs in that project.

You can use both predefined and custom metadata keys to set project metadata. Learn more about predefined project metadata keys and how to set custom project metadata.

Zonal metadata

Zonal metadata is defined at a zonal scope within a project and provides information about VMs in that specific zone in that project. When you set zonal metadata, the metadata entries propagate to all VMs in that configured zone in that project. When compared to project metadata, zonal metadata helps you with fault isolation and provides greater reliability.

Compute Engine doesn't provide any predefined keys for zonal metadata. You must create your own custom metadata keys to set zonal metadata. Learn more about how to set custom zonal metadata.

Instance metadata

Instance metadata provides information about a specific VM instance. You set instance metadata separately for each individual VM instance.

You can use both predefined and custom metadata keys to set instance metadata. Learn more about predefined instance metadata keys and how to Set custom instance metadata.

How metadata is arranged

Compute Engine stores and maintains the metadata keys and values for your VMs and projects in directory listings. Depending on the type of metadata, Compute Engine stores metadata entries in one of the following directories:

Type of metadata Directory
Project-wide and project-zonal metadata
  • http endpoint: 
    http://metadata.google.internal/computeMetadata/v1/project
  • https endpoint (Preview): 
    https://metadata.google.internal/computeMetadata/v1/project
Instance metadata
  • http endpoint: 
    http://metadata.google.internal/computeMetadata/v1/instance
  • https endpoint: 
    https://metadata.google.internal/computeMetadata/v1/instance

Each directory stores metadata entries in the form of key-value pairs. Some metadata entries are also directories that contain other metadata keys. The metadata entries that function as directories are marked by a trailing slash (/) in the metadata key name. For example, /project/attributes/ is a directory under the project/ directory that contains other metadata keys. To create your own metadata directory listing, you must use a trailing slash (/) in the metadata key name when you create your custom metadata entry.

Project and zonal metadata entries are stored in the same project/ directory. If you set different values for the same custom metadata keys for VMs on a project level and on a zonal level, then the zonal metadata values for those keys take precedence over the project metadata values in the respective zones.

  • If you add a zonal metadata value for a metadata key that already has a project metadata value, then Compute Engine overrides the project metadata value for the VMs in this specified zone and updates the /project directory with the zonal value.
  • If you add a new project-wide metadata value for a metadata key that already has a zonal metadata value, then nothing changes. Compute Engine retains the zonal metadata value in the /project directory in the specific zone.
  • If you don't specify a zonal metadata value for a custom metadata key in a specific zone, but the key has a project metadata value, then your VMs continue to have the project metadata values in those zones.

For example, suppose you define a project-wide metadata pair of key-1=value-1. Suppose you also define a zonal metadata pair of key-1=zonal-value-1 for only the us-central1-a zone. All the VMs in the us-central1-a zone for your project inherit key-1=zonal-value1 as the metadata pair. The metadata pair remains key-1=value-1 for all VMs in other zones where you haven't set any zonal metadata for key-1.

What's next?