Creating managed instance groups

A managed instance group (MIG) contains identical virtual machine (VM) instances that are based on an instance template. MIGs maintain high availability of your applications by proactively keeping your VMs available, that is, in the RUNNING state.

MIGs support autohealing, load balancing, autoscaling, and auto-updating.

You can create both zonal (single-zone) MIGs and regional (multi-zone) MIGs. Regional MIGs provide higher availability because they spread VMs across multiple zones within the same region.

For more information about instance groups, see the Instance groups overview.

Before you begin

Limitations

  • Each MIG can contain up to 1,000 VMs.
  • When updating a MIG, no more than 1,000 instances can be specified in a single request.

Using MIGs for stateless applications

You can use MIGs for stateless applications, which don't depend on the specific state of the underlying virtual machine (VM) instance to run. Your application should not depend on VM properties that don't persist, such as the contents of attached disks or in-memory data. If your application requires the preservation of VM properties, see stateful MIGs.

The default behavior for all persistent disks in stateless MIGs is to delete or recreate them when the corresponding VM is deleted or recreated, so you should not rely on attached disks as persistent data in a stateless MIG. To retain your data, Google recommends that you regularly maintain up-to-date custom images with updated software and configurations. To retain your data, use startup scripts, and configure your applications to back up your data in another centralized location, such as Cloud Storage.

In your instance templates, you can specify a container image or a custom image with relevant startup scripts, so that when a VM is recreated, it has the necessary apps installed and has access to required data. For more information about creating instance templates, see Deterministic instance templates.

Using MIGs for stateful applications (beta)

Managed instance groups also support stateful applictions. A stateful MIG preserves each VM's unique state (VM instance name, attached persistent disks, and/or metadata) on machine restart, recreation, autohealing, or update.

Use stateful MIGs for applications that have stateful data or configuration, such as databases, data processing applications, legacy applications, and long-running batch workloads with checkpointing.

Consider using stateful MIGs whenever you deploy a stateful application or cluster to Compute Engine and would like to improve its availability with autohealing and multi-zone deployment, or would like to simplify and accelerate updates with automated rolling updates.

For information about how to create a stateful MIG or add stateful configuration to an existing MIG, see Configuring stateful MIGs.

Creating a managed instance group

Before you create a managed instance group, you must create an instance template to specify the operating system image or container image and the settings for every VM in the group.

After you create the template, create the managed instance group through the Google Cloud Console, the gcloud compute tool, or the API.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click Create an instance group.
  3. Enter a name for the MIG, and select the zone where you want to locate the group.
  4. Under Group type, select Managed instance group.
  5. Under Instance template, select an instance template. If no templates are available, create an instance template.
  6. Specify the number of instances that you want to create in the group. Optionally, enable Autoscaling so that the group automatically adds or removes instances based on its utilization, or enable autohealing to perform application-based health checking on VMs within the group.
  7. Click Create to create the new group.

gcloud

Create an instance group with the instance-groups managed create subcommand:

gcloud compute instance-groups managed create instance-group-name \
    --base-instance-name base-name \
    --size size \
    --template instance-template \
    --zone zone

Replace the following:

  • instance-group-name: The name for this instance group.
  • base-name: The name to use for VMs created in this instance group. Because these VMs are based on a common template, each VM is assigned a random string as part of its VM name. The base name is prepended to this random string. For example, if you set the base name to example, VMs will have names like example-yahs and example-qtyz. If you need specific names, see Creating instances with specific names in MIGs
  • size: The size of the instance group.
  • instance-template: The name of the instance template to use for this group.

  • zone: One of the zones available for Compute Engine.

    For example, the following command creates an instance group named example-group, whose base VM name is test. The group contains three instances:

    gcloud compute instance-groups managed create example-group \
  --base-instance-name test \
  --size 3 \
  --template an-instance-template

API

In the API, construct a POST request to the instanceGroupManagers.insert method or the regionInstanceGroupManagers.insert method. In the request body, include the group name, group size, base name for VMs in the group, and the URL to the instance template.

POST https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers

{
  "baseInstanceName": "base-name",
  "versions": [
    {
      "instanceTemplate": "global/instanceTemplates/instance-template"
    }
  ],
  "name": "instance-group-name",
  "targetSize": size
}

Replace the following:

  • project-id: The project ID for the request.
  • zone: The zone for the request.
    • For a regional MIG, replace zones/zone with regions/region and specify a region.
  • instance-group-name: The name for this instance group.
  • base-name: The name to use for VMs created in this instance group. Because these VMs are based on a common template, each VM is assigned a random string as part of its VM name. The base name is prepended to this random string. For example, if you set the base name to example, VMs will have names like example-yahs and example-qtyz. If you need specific names, see Creating instances with specific names in MIGs
  • size: The size of the instance group.
  • instance-template: The instance template to use for this group.

Depending on how you configure and act on a MIG, various policies and actions can affect the instances in the group. To determine which managed instances are up and running, see Checking the status of managed instances.

Changing the instance template for a managed instance group

You can change the instance template for a managed instance group without applying any changes to existing instances. The managed instance group uses the new template when you make a request to add or recreate instances, but the template does not automatically update existing VMs in the group. This lets you control exactly which VMs are updated, but it causes your instance group to contain dissimilar VMs.

After you create a new instance template, change the instance template for an existing instance group.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Under the Name column of the list, click the name of the instance group where you want to change the instance template.
  3. Click Edit group to modify this managed instance group.
  4. Under Instance template, select the new instance template that you want to use on this group.
  5. Click Save to apply the new template.

gcloud

To use the set-instance-template method to update a template, pass the new template to the instance-groups managed set-instance-template subcommand:

gcloud compute instance-groups managed set-instance-template instance-group-name \
    --template instance-template \
    --zone zone

API

Construct a request to the instanceGroupManagers service with the name of the target managed instance group. Include the URL of the new instance template in the request body:

POST https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/setInstanceTemplate

{
 "instanceTemplate": "global/instanceTemplates/instance-template
}

After you change the instance template, recreate individual instances or update all of the VMs in the group with a rolling update.

Resizing a managed instance group

To add or remove VMs in a MIG, you can:

Automatically resizing a MIG

You can configure managed instance groups to automatically add or remove VMs based on their workloads. Your applications can gracefully handle increases in traffic, and you can reduce your costs when the need for compute resources is lower. To start scaling your managed instance groups, see Autoscaling groups of instances.

Manually setting the size of a MIG

If a managed instance group is not already set to automatically scale, you can resize the group manually to change the number of instances. If you increase the size, the managed instance group uses the current instance template to add new instances. If you decrease the size, the managed instance group deletes VMs from the group. The group deletes instances with a currentAction of DELETING, CREATING, and RECREATING before it deletes instances that are running with no scheduled actions.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

You can resize a managed instance group using the Google Cloud Console, the gcloud compute tool, or the API.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Under the Name column of the list, click the name of the instance group where you want to change the group size.
  3. Click Edit group to modify this managed instance group.
  4. Under Number of instances, specify the number of instances that you want to include in this managed instance group. If Autoscaling is enabled, the group is automatically adding or removing instances as necessary. However, you can change the Minimum number of instances and the Maximum number of instances values to indirectly adjust the group size through Autoscaler.
  5. Click Save to apply the new template.

gcloud

Use the resize subcommand.

gcloud compute instance-groups managed resize instance-group-name \
    --size new-size \
    --zone zone

API

Construct a request to the resize method. Specify the new instance group size as a parameter.

POST https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/resize?size=new-size

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

After you make a request to resize a managed instance group, the VMs start or stop as soon as the system can provision or delete them. This process can take a significant amount of time depending on the number of instances in the group. Verify the status of instances in your managed instance group.

Disabling creation retries mode during resize

By default, if the initial creation of a VM instance fails, the managed instance group continuously retries to create the VM, until the VM is successfully created. However, if you do not want automatic creation retries, you can disable the creation retry mode by setting the --nocreation-retries flag when you resize the instance group. In this mode, the managed instance group attempts to create all instances only once. If there is an error during instance creation, the managed instance group gives up on this instance, removes it from the group's list of managed instances, and decreases the target size of the managed instance group.

This mode applies only during the first creation attempt of a VM. If a VM is created successfully while this mode is enabled, the VM behaves the same way as all other VMs created with a regular resize request. In particular, if a running VM dies unexpectedly at a later time and needs to be recreated, this mode does not affect the recreation behavior in that scenario.

The disable creation retry mode is especially useful in scenarios where you have systems automatically creating groups of VMs where an exact number of VMs is not required. You might prefer to stabilize quickly on the size of the managed instance group and be flexible in the number of VMs in the group, rather than to wait indefinitely until all the requested instances are created, which might be delayed temporarily or permanently because of quota errors or other unrelated issues.

To resize a managed instance group in the disabled creation retries mode, use the gcloud tool or the API.

gcloud

Use the resize subcommand with the --no-creation-retries flag:

gcloud beta compute instance-groups managed resize instance-group-name --size new-size \
    --no-creation-retries \
    --zone zone

API

Construct a request to resizeAdvanced method. Specify the new instance group size and the noCreationRetries field in the request body.

POST https://compute.googleapis.com/compute/beta/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/resizeAdvanced

{
 "targetSize": size
 "noCreationRetries": true
}

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

You can see which instances are being created and in which mode by using the listManagedInstances method. Instances that are being created in the disabled creation retries mode have a currentAction of CREATING_WITHOUT_RETRIES.

Creating instances with specific names in MIGs

If you have a system that depends on specific names, you can create VMs with those names. The names that you assign to these managed instances persist if the VM is recreated. See Adding instances with specific names.

Deleting specific instances from a group

You can delete individual VM instances in a managed instance group. Deleting instances reduces the specified targetSize of the instance group and removes the VMs from any target pools of which they are a member.

Deleting instances from a managed instance group does not change any specified autoscaler settings. If you delete instances from a managed instance group, the autoscaler might detect an increase in the workload on the other instances in the group and increase the group size back to its previous level. To prevent this, turn off or delete the autoscaler before attempting to delete the instances.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Delete instances from a managed instance group using the Google Cloud Console, the gcloud compute tool, or the API.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Under the Name column of the list, click the name of the instance group where you want to delete individual instances. A page opens with the instance group properties and a list of instances that are included in the group.
  3. On the list of instances, select one or more instances that you want to delete.
  4. Click Delete. The selected instances are deleted.

gcloud

Use the instance-groups managed delete-instances subcommand:

gcloud compute instance-groups managed delete-instances instance-group-name \
    --instances example-i3n2,example-z2x9 \
    --zone zone

API

Construct a request to the instanceGroupManagers.deleteInstances method. In the request body, include the URLs to one or more instances that you want to delete.

POST https://www.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/deleteInstances

{
 "instances": [
  "zones/zone/instances/example-instance-i3n2",
  "zones/zone/instances/example-instance-l6n1"
 ]
}

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

After you make a request to delete instances in a managed instance group, the instances stop as soon as the system can delete them. This process can take a significant amount of time depending on the number of instances that you delete from the group. Verify the status of instances in your managed instance group.

Abandoning instances from a group

You can separate a VM instance from a managed instance group to more easily debug issues with individual VMs without affecting the group as a whole. Abandoning an instance from a group also removes that VM from load balancers that were assigned to the managed instance group. Target pools that were manually assigned to specific individual instances are not removed.

Abandoning instances reduces the specified targetSize of the instance group, but does not change any specified autoscaler settings. Managed instance groups with an autoscaler continue to add or remove VMs automatically as necessary.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Abandon instances from a managed instance group using the Google Cloud Console, the gcloud compute tool, or the API.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Under the Name column of the list, click the name of the instance group from which you want to remove instances. A page opens with the instance group properties and a list of instances that are included in the group.
  3. On the list of instances, select one or more instances that you want to remove from the group.
  4. Click Remove from group. The selected instances leave the group, but continue to run outside of the group.

gcloud

To remove an instance from the instance group without deleting the instance, use the abandon-instances subcommand.

gcloud compute instance-groups managed abandon-instances instance-group-name \
    --instances example-i3n2,example-z2x9 \
    --zone zone

API

Construct a request to the instanceGroupManagers.abandonInstances method. In the request body, include the URLs to one or more instances that you want to abandon.

POST https://www.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/abandonInstances

{
 "instances": [
  "zones/zone/instances/example-instance-i3n2",
  "zones/zone/instances/example-instance-l6n1"
 ]
}

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

After you make a request to abandon instances from a managed instance group, the group removes the instances as soon as possible. Verify the status of instances in your managed instance group.

Recreating instances in the group

Recreating a managed instance deletes the specified VM and creates a new VM using the instance template that is assigned to the managed instance group.

Use this method to update selected VMs so that they use the latest instance template. If you need to recreate all of the VMs in a managed instance group, start a rolling update instead.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Recreate selected VMs in managed instance group using the gcloud compute tool, or the API.

gcloud

Use the instance-groups managed recreate-instances subcommand.

gcloud compute instance-groups managed recreate-instances instance-group-name \
    --instances example-i3n2,example-z2x9 \
    --zone zone

API

Construct a request to the instanceGroupManagers.recreateInstances method. In the request body, include the URLs to one or more instances that you want to recreate.

POST https://www.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/recreateInstances

{
 "instances": [
  "zones/zone/instances/example-instance-i3n2",
  "zones/zone/instances/example-instance-l6n1"
 ]
}

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

After you make a request to recreate instances in a managed instance group, the new VMs start as soon as the system can provision them. This process can take a significant amount of time depending on the number of instances that you recreate. Verify the status of instances in your managed instance group.

Deleting a managed instance group

When you delete a managed instance group, all VMs in the group are deleted. If you must keep any of the VMs in this managed instance group, abandon the instances first to remove the VMs from the group. Then, delete the managed instance group.

When you delete a managed instance group and its instances using the Google Cloud Console or the gcloud compute tool, any attached autoscalers are automatically deleted. However, if using the API, you must first issue separate requests to delete any attached autoscalers with autoscalers.delete or regionAutoscalers.delete.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Select one or more groups on the list that you want to delete.
  3. Click Delete to delete the group and all of the VMs in the managed instance group.

gcloud

Use the instance-groups managed delete subcommand.

gcloud compute instance-groups managed delete instance-group-name \
    --zone zone

API

Construct a request to the instanceGroupManagers.delete method.

DELETE https://www.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name

If the instance group is a regional managed instance group, replace zones/zone with regions/region.

Creating groups of preemptible instances

You can use managed instance groups to quickly create multiple preemptible instances, which can reduce the costs of the VMs in your managed instance groups. For example, you can create a group of preemptible instances, use them to run a batch processing task, and then delete the group when the task is complete.

To create a group of preemptible instances, set the premptible option in an instance template, and then use the template to create the managed instance group.

Console

  1. In the Cloud Console, go to the Instance templates page.

    Go to the Instance templates page

  2. Click New instance template.
  3. Fill in the properties that you want for your instance template.
  4. Click Show advanced options to expand the Availability policies section.
  5. Set Preemptibility to On.
  6. Click Create to create the template.
  7. Use this template to create a managed instance group

gcloud

In gcloud compute, create an instance template using the instance-templates create command. Include the --preemptible flag.

gcloud compute instance-templates create instance-template \
    --preemptible

After you create the instance template, use it to create a managed instance group.

API

Use the instanceTemplates().insert method to create a new instance template. Include the preemptible property under scheduling and set it to true.

{
"name": "instance-template",
"properties": {
  "machineType": "zones/zone/machineTypes/machine-type",
  "networkInterfaces": [
    {
      "network": "global/networks/default",
      "accessConfigs":
      [
        {
          "name": "external-IP",
          "type": "ONE_TO_ONE_NAT"
        }
      ]
    }
  ],
  "scheduling":
  {
    "preemptible": true
  },
  "disks":
  [
    {
      "type": "PERSISTENT",
      "boot": true,
      "mode": "READ_WRITE",
      "initializeParams":
      {
        "sourceImage": "projects/debian-cloud/global/images/family/debian-9"
      }
    }
  ]
  }
}

After you create the instance template, use it to create a managed instance group.

Understanding the instanceTemplate and versions fields

When you create a managed instance group, you supply an instance template that the managed instance group uses to create individual VM instances. By default, Compute Engine describes the instance template used in two separate API properties: the top-level instanceTemplate property and the versions property. For example, in the following managed instance group, notice that both the instanceTemplate and versions fields are populated:

{

 "name": "example-group",
 "zone": "zones/us-central1-a",
 "instanceTemplate": "global/instanceTemplates/example-it",
 "versions": [
  {
   "name": "v3",
   "instanceTemplate": "global/instanceTemplates/example-it",
   "targetSize": {
    "calculated": 3
   }
  }
 ]...
}

Compute Engine automatically populates both the top-level instanceTemplate field and the versions field for backward compatibility. We recommend specifying the versions field and omitting the top-level instanceTemplates field whenever possible. However, if your application code currently sets the top-level instanceTemplate field, it is still a valid request.

To learn more about the Managed Instance Group Updater, read Rolling out updates to MIGs.

(Advanced) Canarying instance templates with a managed instance group

It is possible to create a managed instance group where there are two groups of VMs that use different instance templates. For example, you might want to create a managed instance group that has 20 VM instances and 10 VMs should run on a specific operating image while the rest run on a different operating system image. This feature allows you to compare the two different instance template versions before deciding on one.

In the API, make a POST request to the following URL:

POST https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers

Your request body should omit the top-level instanceTemplates field but contain the versions property with two instance templates set. In each versions object, you specify an instance template. For one of the versions objects but not both, you must also specify the targetSize. For example, the following request creates an instance group where 50% of the VM instances use the example-template instance template and the rest of the VM instances use the small-machine-type template:

{
  "baseInstanceName": "example-instances",
  "name": "example-group",
  "targetSize": 5,
  "versions":
  [
    {
      "instanceTemplate": "global/instanceTemplates/example-template",
      "targetSize":
      {
        "percent": 50
      }
    },
   {
     "instanceTemplate": "global/instanceTemplates/small-machine-type"
   }
  ]
}

Managed instance groups and IAM

All the operations performed by Compute Engine as part of the managed instance group are performed using the Google APIs service account for your project. This per-project service account has an email address like the following, where project-id is the numerical ID of the corresponding project:

project-id@cloudservices.gserviceaccount.com

The Google APIs service account is different from the default Compute Engine service account.

It is up to you to ensure that the service account used by the managed instance group has enough privileges to create resources based on the instance template. In particular, this means that the service account needs to be granted the compute.instanceAdmin.v1 and, optionally, the serviceAccountUser role, to create and manage VMs in the instance group. The serviceAccountUser role is required only if the managed instance group will be creating VMs that can run as a service account. Keep in mind that this account is also used by other processes, including Deployment Manager.

When you create a managed instance group or update an instance template, Compute Engine validates that the Google APIs service account has the following:

  • serviceAccountUser role. Important if you plan to create instances that can run as a service account.
  • Permissions to all the resources referenced from instance templates, such as images, disks, VPC networks and subnets.

To learn more about service accounts, read the Service accounts overview.

Updating all instances in a managed instance group

See Rolling out updates to MIGs.

Troubleshooting

My managed instance group keeps failing to create a VM. What's going on?

There are several issues that can prevent the instance group from successfully creating or recreating a VM instance. Some common issues include:

  • The managed instance group is attempting to create or recreate both the instance and the boot persistent disk, but the persistent disk already exists. By default, new boot persistent disks are created when new instances are created. These disks are named after the VM. If an VM has the name my-awesome-instance, the disk is also named my-awesome-instance. If a persistent disk already exists with that name, the request fails. Delete the existing persistent disk to resolve this issue.

  • Your instance template has set the disks.autoDelete option to false for boot persistent disks so that when a VM has been deleted (for example, because of autohealing), the persistent disk was not deleted. When the managed instance group attempted to recreate the VM with the same name, it ran into the same issue where a persistent disk already exists with the same name. Delete the existing persistent disk to resolve the immediate problem and update the instance template to set the disks.autoDelete to true if you would like boot persistent disks to be deleted alongside the instance.

  • Your instance template might be invalid. If you updated your instance template recently, there could be an invalid property that causes the managed instance group to fail VM creation. Some invalid properties could be:

    • You specified a resource that doesn't exist, such as a source image.
    • You misspelled a resource name.
    • You tried to attach additional non-boot persistent disks in read/write mode. Because instance groups contain multiple VMs, any additional disks you want to share between all of the VMs in the group can be attached only in read-only mode.

  • If you configured an autohealing policy but you did not configure (or misconfigured) the firewall rule that lets the health check probes reach your application, then your VMs appear unhealthy, and the MIG continuously tries to recreate them. For information about how to configure a health check firewall rule, see this example health check set up.

What's next