Use VM instance placement policies

---

Each virtual machine (VM) instance runs on a physical server, or host, in a rack that is placed in a cluster in a data center. You can use the following placement policies to control where your VMs are physically located relative to each other within a zone:

• Spread placement policies. Use spread policies when you want VMs spread out from each other. This can help reduce the impact of host system failures or optimize a live migration of your VMs.
• Compact placement policies. Use compact policies when you want VMs to be located close to each other for low network latency between the VMs.

Before you begin

• If you want to use the command-line examples in this guide, do the following:
  1. Install or update to the latest version of the Google Cloud CLI.
  2. Set a default region and zone.
• If you want to use the API examples in this guide, set up API access.

Restrictions

Placement policies have the following restrictions:

• Spread placement policies:
  • Support up to 8 VMs per policy.
  • Support only C2, G2, N1, N2, and N2D machine types.
  • Can't be used with the VM placement topology API.
  • Can't be used with reservations of VMs.
• Compact placement policies:
  • Support up to 150 VMs in each policy.
  • Support only A2, C2, G2, C2D, N2, and N2D machine types.
  • Don't support live migration.
  • For host maintenance events, can be used only with VMs configured to TERMINATE.
  • Can be used only for reservations of VMs for a single project. Reservations shared across projects or reservations attached to committed-use discounts aren't supported.
  • Can't be used with sole-tenant nodes.
  • Can't be applied to existing VMs.

Create a placement policy

To control where your VMs are located relative to one another, use the following process:

1. Create either a spread or compact placement policy with the placement configuration that your VMs need.
   • Spread placement policies strictly place your VMs across the underlying datacenter infrastructure so that the VMs don't share the same host or power system. This approach reduces the impact of host or power failures.
   • Compact placement policies put your VMs close together for low network latency between the VMs.
2. Apply the placement policy to one or more VMs. VMs that share the same policy are placed relative to each other based on how you defined the policy. You can apply the policy to new and existing VMs or specify the policy in an instance template, which you can use to create to standalone VMs, managed instance groups (MIGs), and VM reservations.

Create a spread placement policy

To create a spread placement policy where VMs are located across several distinct availability domains, specify the number of availability domains that this policy should use to separate VMs from each other.

gcloud

Use the gcloud CLI to create the policy.

gcloud compute resource-policies create group-placement POLICY_NAME \
    --availability-domain-count DOMAIN_COUNT \
    --region REGION \
    --project PROJECT_ID

Replace the following:

• POLICY_NAME: the name for the new policy
• DOMAIN_COUNT: the number of distinct sets of host hardware and physical networks that this policy will use to separate VMs
• REGION: the region where you plan to create VMs that use this policy
• PROJECT_ID: your project ID

API

Create a spread placement policy by using the resourcePolicies.insert method in the Google Cloud console APIs & services.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies

In the body of the request, provide details of the placement policy:

{
  "name": "POLICY_NAME",
  "groupPlacementPolicy": {
    "availabilityDomainCount": DOMAIN_COUNT
  }
}

Replace the following:

• PROJECT_ID: your project ID
• REGION: the region where you plan to create VMs that use this policy
• POLICY_NAME: the name for the new policy
• DOMAIN_COUNT: the number of distinct sets of host hardware and physical networks that this policy will use to separate VMs

Create a compact placement policy

To create a compact placement policy where VMs are located closer to each other and on the same network infrastructure, specify a COLLOCATED policy.

Note: If you specify a fixed number of VMs in a compact placement policy (for example, by using the --vm-count flag), then Compute Engine creates those VMs only when it's possible to create all of them at once. To avoid potential delays, Google recommends that you do not specify a fixed number of VMs in your placement policy. If you need to deploy a specific number of VMs at once, after creating your placement policy, use the bulk API to create those VMs.

gcloud

Use the gcloud CLI to create the policy.

gcloud compute resource-policies create group-placement POLICY_NAME \
    --collocation COLLOCATED \
    --region REGION \
    --project PROJECT_ID

Replace the following:

• POLICY_NAME: the name for the new policy
• REGION: the region where you plan to create VMs that use this policy
• PROJECT_ID: your project ID

API

Create a compact placement policy using the resourcePolicies.insert method in the Google Cloud console APIs & services.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies

In the body of the request, provide details of the placement policy:

{
  "name": "POLICY_NAME",
  "groupPlacementPolicy": {
    "collocation": "COLLOCATED"
  }
}

Replace the following:

• PROJECT_ID: your project ID
• REGION: the region where you plan to create VMs that use this policy
• POLICY_NAME: the name for the new policy

Apply placement policies to VMs

After you create a placement policy, you can apply it to new or existing VMs. Support for placement policies depends on your scenario.

Scenario	Support for compact placement	Support for spread placement
Apply a placement policy to a new VM
Apply a placement policy to an existing VM
Specify a placement policy in an instance template that you can use to create VMs
Specify a placement policy in an instance template that you can use to create or update a MIG
Specify a placement policy in an instance template that you can use to create VM reservations

Apply a placement policy to a new VM

After you create a placement policy, apply it to one or more VMs. VMs that share the same policy are placed relative to each other based on how you defined the policy.

gcloud

Apply a placement policy to a VM by including the --resource-policies flag when you create VMs individually or in bulk.

Note: If you run multiple commands to create instances, use the --async flag to prevent the commands from timing out while Compute Engine creates the VMs under the placement policy.

For compact placement policies, you must include the --maintenance-policy=TERMINATE and --no-restart-on-failure flags.

For example, to create a VM that uses a compact placement policy, use the following command:

gcloud compute instances create VM_NAME \
    --zone=ZONE \
    --resource-policies=POLICY_NAME \
    --image-family=IMAGE_FAMILY \
    --image-project=IMAGE_PROJECT \
    --project=PROJECT_ID \
    --maintenance-policy=TERMINATE \
    --no-restart-on-failure \
    --async

Replace the following:

• VM_NAME: the name for the new VM.

• ZONE: the zone where you want to create the new VM.

  This zone must be in the same region where the placement policy is located.

• POLICY_NAME: the name of the placement policy that you want to apply to this VM.

  You can apply more than one placement policy to a VM.

• IMAGE_FAMILY: one of the available image families.

• IMAGE_PROJECT: the image project to which the image belongs.

• PROJECT_ID: your project ID.

API

Apply a placement policy to a VM by including the --resource-policies flag when you create VMs individually or in bulk.

For example, to create a single VM, use the following command:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances

In the body of the request, provide the resource policy. For compact placement policies, you must include the "onHostMaintenance": "TERMINATE" and "automaticRestart": false arguments:

{
  "name": "VM_NAME",
  "machineType": "machineTypes/MACHINE_TYPE"
  "networkInterfaces": [{
    "accessConfigs": [{
      "type": "ONE_TO_ONE_NAT",
      "name": "External NAT"
    }],
    "network": "global/networks/default"
  }],
  "scheduling": {
    "onHostMaintenance": "TERMINATE",
    "automaticRestart": false
  },
  "disks": [{
     "autoDelete": "true",
     "boot": "true",
     "type": "PERSISTENT",
     "initializeParams": {
       "sourceImage": "projects/IMAGE_PROJECT/global/images/family/IMAGE_FAMILY"
     }
  }],
  "resourcePolicies": [
    "projects/PROJECT_ID/regions/REGION/resourcePolicies/POLICY_NAME"
  ]
}

Replace the following:

• PROJECT_ID: your project ID.

• ZONE: the zone where you want to create the new VM.

  This zone must be in the same region where the placement policy is located.

• VM_NAME: the name for the new VM.

• MACHINE_TYPE: the machine type of the VM.

• IMAGE_PROJECT: the image project to which the image belongs.

• IMAGE_FAMILY: one of the available image families.

• REGION: the region where you created the placement policy.

• POLICY_NAME: the name of the placement policy that you want to apply to this VM.

  You can apply more than one placement policy to a VM.

Apply a spread placement policy to an existing VM

If you create a spread placement policy, you can apply it to one or more existing VMs without restarting the VMs.

Note: You can't apply a compact placement policy to existing VMs.

gcloud

Apply a spread placement policy to an existing instance by using the add-resource-policies command.

gcloud compute instances add-resource-policies VM_NAME \
    --zone=ZONE \
    --resource-policies=SPREAD_PLACEMENT_POLICY_NAME \
    --project=PROJECT_ID

Replace the following:

• VM_NAME: the name of the VM.

• ZONE: the zone of the VM.

  This zone must be in the same region where the placement policy is located.

• SPREAD_PLACEMENT_POLICY_NAME: the name of an existing spread placement policy that you want to apply to this VM.

• PROJECT_ID: your project ID.

API

Apply a spread placement policy to an existing instance by using the addResourcePolicies method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/addResourcePolicies

In the body of the request, provide the spread placement policy.

{
  "resourcePolicies": [
    "projects/PROJECT_ID/regions/REGION/resourcePolicies/SPREAD_PLACEMENT_POLICY_NAME"
  ]
}

Replace the following:

• PROJECT_ID: your project ID.

• ZONE: the zone where you want to create the new VM.

  This zone must be in the same region where the placement policy is located.

• VM_NAME: the name for the new VM.

• REGION: the region where you plan to create VM instances that use this policy.

• SPREAD_PLACEMENT_POLICY_NAME: the name of an existing spread placement policy that you want to apply to this VM.

Create an instance template that specifies a placement policy

You can specify a placement policy when creating an instance template. Specifically:

• If you specify a compact placement policy in an instance template, you can use the template to create MIGs, VMs, and VM reservations.

• If you specify a spread placement policy in an instance template, you can only use the template to create MIGs and VMs.

gcloud

To create an instance template that specifies a resource policy, use the gcloud compute instance-templates create command with the --resource-policies flag. To specify a compact placement policy, you must also specify the --maintenance-policy=TERMINATE and --no-restart-on-failure flags.

For example, to create an instance template that has default VM properties and includes a compact placement policy, use the following command:

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --resource-policies=COMPACT_PLACEMENT_POLICY_NAME \
    --maintenance-policy=TERMINATE \
    --no-restart-on-failure

Replace the following:

• INSTANCE_TEMPLATE_NAME: the name of the instance template.

• COMPACT_PLACEMENT_POLICY_NAME: the name of an existing compact placement policy.

Important: Placement policies are regional resources. If you specify a placement policy when creating an instance template, this policy restricts the use of the template to the region where it exists. For example, if you create a compact placement policy in us-central1 and include the policy in an instance template, you can only use the template to create MIGs, VMs, or VM reservations within the same region.

API

To create an instance template that specifies a resource policy, make a POST request to the instanceTemplates.insert method. In the request body, specify the placement policies in the resourcePolicies field. To specify a compact placement policy, you must also specify the "onHostMaintenance": "TERMINATE" and "automaticRestart": false fields.

For example, to create an instance template that has default VM properties and includes a compact placement policy, use the following command:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "resourcePolicies": {
      "COMPACT_PLACEMENT_POLICY_NAME"
    },
    "scheduling": {
      "onHostMaintenance": "TERMINATE",
      "automaticRestart": false
    },
    ...
  }
}

Replace the following:

• PROJECT_ID: the project ID of your current project.

• INSTANCE_TEMPLATE_NAME: the name of the instance template.

• COMPACT_PLACEMENT_POLICY_NAME: the name of an existing compact placement policy.

Important: Placement policies are regional resources. If you specify a placement policy when creating an instance template, this policy restricts the use of the template to the region where it exists. For example, if you create a compact placement policy in us-central1 and include the policy in an instance template, you can only use the template to create MIGs, VMs, or VM reservations within the same region.

If you want to use the template to create a MIG, VM, or VM reservation, see the following docs:

• Apply a placement policy to VMs in a managed instance group.
• Create a VM from an instance template. Review the restrictions.
• If your instance template specifies a compact placement policy, you can use it to Create a reservation for a single project.

Apply a placement policy to VMs in a managed instance group

After you create a placement policy, create an instance template that specifies the policy. Then create a MIG or update an existing MIG with that instance template. The placement policy will apply to VMs in the group that use that instance template.

If you have a workload that requires extensive communication between VMs, we recommend that you create a compact placement policy and use a regional MIG with the any single zone distribution shape. With the any-single-zone shape, whenever a regional MIG has no VMs and needs to scale out, the group picks the optimal zone based on your reservations, quotas, and hardware requirements.

Note: Placement policies are individual policies that can apply only to one set of instances at a time. If you create multiple managed instance groups using the same template, the instances in those groups will share the same placement policy including its VM limits. To overcome this limitation, create one placement policy for each template and use that template only to create one managed instance group.

Apply a placement policy to a new MIG

You can use the gcloud CLI or the Compute Engine API.

gcloud

To create a MIG with VMs that are based on the template that you created earlier, use the instance-groups managed create command. For example, to create a regional MIG with the any-single-zone distribution shape, use the following command:

gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \
    --template TEMPLATE_NAME \
    --size SIZE \
    --region REGION \
    --zones ZONES \
    --target-distribution-shape=any-single-zone

Replace the following:

• INSTANCE_GROUP_NAME: the name for this instance group
• TEMPLATE_NAME: the name of the instance template to use for this group
• SIZE: the size of the instance group

• REGION: the region for this instance group

• ZONES (optional): a list of zones in the region where the MIG can deploy VM instances. By default, Compute Engine selects three zones for you.

  • If you want your MIG to be able to use all zones in the region, specify all the available zones. You can get a list of zones in region with the following command:

    gcloud compute zones list --filter=region:REGION --format='list(NAME)'

  • Note that you cannot update a regional MIG to use different zones after it is created.

API

To create a MIG with VMs that are based on the template that you created earlier, call the instanceGroupManagers.insert or regionInstanceGroupManagers.insert method and specify that template. For example, to create a regional MIG with the any-single-zone distribution shape, use the following method:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers

{
  "name": "INSTANCE_GROUP_NAME",
  "instanceTemplate": "global/instanceTemplates/TEMPLATE_NAME",
  "targetSize": SIZE,
  "distributionPolicy": {
      "zones": [
          {"zone": "zones/ZONE1"},
          {"zone": "zones/ZONE2"},
          {"zone": "zones/ZONE3"},
      ],
      "targetShape": "ANY_SINGLE_ZONE"
  }
}

Replace the following:

• PROJECT_ID: the project ID for this request
• REGION: the region for the group
• INSTANCE_GROUP_NAME: the name of the MIG
• TEMPLATE_NAME: the instance template to use
• SIZE: the target number of VMs for the group
• ZONES: the name of a zone in the region where the MIG can deploy VM instances.
  • If you want your MIG to be able to use all zones in the region, specify all the available zones. You can get a list of zones in region by calling the regions.get method.
  • Note that you cannot update a regional MIG to use different zones after it is created.

To learn more about creating MIGs, see Basic scenarios for creating managed instance groups (MIGs).

Apply a placement policy to an existing MIG

After you create a placement policy, create an instance template that specifies the policy. Then apply the template to an existing MIG. You can use the gcloud CLI or the Compute Engine API.

gcloud

To update a MIG to use the new template, and to automatically roll out the new template to existing VMs in the MIG, you can use the instance-groups managed rolling-action start-update command–for example:

gcloud compute instance-groups managed rolling-action start-update INSTANCE_GROUP_NAME \
    --template TEMPLATE_NAME \
    --type=proactive

API

To update a MIG to use the new template, call the patch method on a regional or zonal MIG.

For example, for a regional MIG, the following request shows the minimal configuration necessary for automatically updating 100% of the instances to the new instance template.

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/INSTANCE_GROUP_NAME

{
  "instanceTemplate": "global/instanceTemplates/TEMPLATE_NAME",
  "updatePolicy": {
    "type": "PROACTIVE"
   }
}

To learn about other ways to update VMs in MIGs, see Update and apply new configurations to VMs in a MIG.

View placement policies

You can view placement policies applied to a VM and details of a specific placement policy by using the gcloud CLI and the Compute Engine API.

View the placement policy of a VM

gcloud

To view a VM's resource placement policy, use the gcloud compute instances describe command:

gcloud compute instances describe VM_NAME

Replace VM_NAME with the name of your VM.

If a placement policy is available, the output contains the resourcePolicies field:

resourcePolicies:
https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGIONS/resourcePolicies/POLICY_NAME

API

To view a VM's resource placement policy, use the instances.get method:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME

Replace the following:

• PROJECT_ID: your project ID
• ZONE: the zone containing the VM
• VM_NAME: the name for your VM

If a placement policy is available, the resourcePolicies field returns resource policies of your VM.

"resourcePolicies": [
"https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies/VM_NAME"
],

View details of a placement policy

gcloud

To view details of a placement policy, use the gcloud compute resource-policies describe command:

gcloud compute resource-policies describe POLICY_NAME

Replace POLICY_NAME with the name of the placement policy.

If the placement policy is available, the output contains details of the placement policy:

...
groupPlacementPolicy:
  availabilityDomainCount: 2
kind: compute#resourcePolicy
name: POLICY_NAME
region: https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION
...

API

To view details of a placement policy, use the resourcePolicies.get method:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies/POLICY_NAME

Replace the following:

• PROJECT_ID: your project ID
• REGION: the region containing the VM
• POLICY_NAME: the name for the placement policy

If the placement policy is available, the response body contains details of the placement policy:

...
"region": "https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION",
"name": "POLICY_NAME",
"groupPlacementPolicy": {
  "availabilityDomainCount": 2
},
"kind": "compute#resourcePolicy"
...

View VM placement topology

You can view information about where a VM is located in relation to another VM. This information helps you build a topology for your VMs, helping you determine which VMs are closest to each other, and which VMs share the least amount of hardware.

You can compare the VM placement topology information only for VMs that use the same placement policy.

gcloud

View the resource properties of a VM that was created with a placement policy :

gcloud compute instances describe VM_NAME \
    --format="table[box,title=VM-Topology](resourcePolicies.scope():sort=1,resourceStatus.physicalHost:label=location)"

Replace VM_NAME with the name of the VM instance that uses a placement policy.

The output should be similar to the following:

VM-Topology

RESOURCE_POLICIES: us-central1/resourcePolicies/policy_name']
PHYSICAL_HOST: /xxxxxxxx/xxxxxx/xxxxx

The value for PHYSICAL_HOST is composed of three fields. These fields contain hashed values that represent the cluster, server rack, and host machine where the VM is located. When comparing this value with other VMs, the more fields that have the same string, the closer the VMs are located to each other. For example, two VMs that belong to the same project, cluster, and run on the same rack have the same value for the first two parts of the PHYSICAL_HOST field.

API

View the details of a VM that was created with a compact placement policy. Make a GET request to the instances.get method:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME

Replace the following:

• PROJECT_ID: The project ID for this query.
• ZONE: The zone for the instance that you want to query.
• VM_NAME: The name of the VM that uses a placement policy.

Your response body contains a snippet similar to the following:

{
  ...
  resourcePolicies:
  - https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/ZONE/resourcePolicies/POLICY_NAME
  resourceStatus:
      physical_host: /xxxxxxxx/xxxxxx/xxxxx
  ...
}

The value for physical_host is composed of three fields, which contain hashed values that represent the cluster, server rack, and host machine where the VM is located. When comparing this value with other VMs, the more fields that have the same string, the closer the VMs are located to each other. For example, two VMs that belong to the same project, cluster, and run on the same rack have the same value for the first two parts of the physicalHost field.

If the VM doesn't use a compact placement policy, the value of the property appears in the output as:

resourceStatus: {}

Delete a placement policy

You can delete a placement policy that you don't need.

gcloud

Use the gcloud CLI to delete the policy:

gcloud compute resource-policies delete POLICY_NAME \
    --region REGION \
    --project PROJECT_ID

Replace the following:

• POLICY_NAME: the name of the new policy to remove
• REGION: the region where you created the VM instances that use this policy
• PROJECT_ID: your project ID

API

To create a spread placement policy, use the resourcePolicies.insert method in the Google Cloud console APIs & services:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/resourcePolicies

In the body of the request, provide details of the placement policy:

{
  "name": "POLICY_NAME",
  "groupPlacementPolicy": {
    "availabilityDomainCount": DOMAIN_COUNT
  }
}

Replace the following:

• PROJECT_ID: your project ID
• REGION: the region where you plan to create VM instances that use this policy
• POLICY_NAME: the name for the new policy
• DOMAIN_COUNT: the number of distinct sets of host hardware and physical networks that this policy will use to separate instances

What's next?

• Check the instance's status to see when it's ready to use.
• Connect to your instance.
• Read tips for designing a robust system that can handle service disruptions.