Add and remove VMs from a MIG

---

After you've created a managed instance group (MIG), you might want to add or remove VMs from that group to meet your workload's needs. If you have reached the default limit for the maximum number of VMs that you can add to a MIG, and you need more VMs, you can increase the size limit of the MIG.

To add or remove VMs in a MIG, you can do the following:

• Automatically add or remove VMs by setting the target size of the MIG:
  • Use autoscaling for stateless applications
  • Manually set the size of the MIG
• Manually add, remove or abandon specific instances:
  • Create instances with specific names
  • Delete specific instances
  • Abandon specific instances

Before you begin

• If you haven't already, set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine as follows.
  

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI, then initialize it by running the following command:

     gcloud init

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by running gcloud components update.
  2. Set a default region and zone.

  Terraform

  To use the Terraform samples on this page from a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.

  1. Install the Google Cloud CLI.

  2. To initialize the gcloud CLI, run the following command:

     gcloud init

  3. Create local authentication credentials for your Google Account:

     gcloud auth application-default login

  For more information, see Set up authentication for a local development environment.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI, then initialize it by running the following command:

  gcloud init

Set the group's target size

You can set a MIG's target size automatically by using an autoscaler or set the size manually.

Automatically adding and removing VMs in a MIG

You can configure managed instance groups to automatically add or remove VMs based on their workloads. Your applications can gracefully handle increases in traffic, and you can reduce your costs when the need for compute resources is lower. To start scaling your managed instance groups, see Create a MIG with autoscaling.

For more information, see Autoscaling groups of instances.

Manually setting the size of a MIG

If a managed instance group is not already set to automatically scale, you can resize the group manually to change the number of instances. If you increase the size, the managed instance group uses the current instance template to add new instances. If you decrease the size, the managed instance group deletes VMs from the group. The group deletes instances with a currentAction of DELETING, CREATING, and RECREATING before it deletes instances that are running with no scheduled actions.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Permissions required for this task

To perform this task, you must have the following permissions:

• compute.instanceGroupManagers.update on the managed instance group

Console

1. In the Google Cloud console, go to the Instance groups page.

   Go to Instance groups

2. Under the Name column of the list, click the name of the instance group where you want to change the group size.

3. Click Edit to modify this managed instance group.

4. Under Number of instances, specify the number of instances that you want to include in this managed instance group. If Autoscaling is enabled, the group is automatically adding or removing instances as necessary. However, you can change the Minimum number of instances and the Maximum number of instances values to indirectly adjust the group size through Autoscaler.

5. Click Save to apply the new template.

gcloud

Use the resize command.

gcloud compute instance-groups managed resize INSTANCE_GROUP_NAME \
    --size NEW_SIZE \
    --zone ZONE

REST

Call the resize method for a zonal or regional MIG resource. Specify the new instance group size as a parameter.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resize?size=NEW_SIZE

If the instance group is a regional managed instance group, replace zones/ZONE with regions/REGION.

After you make a request to resize a managed instance group, the VMs start or stop as soon as the system can provision or delete them. This process can take a significant amount of time depending on the number of instances in the group. Verify the status of instances in your managed instance group.

Note: Because new VMs are created with the current instance template, your group can contain dissimilar VMs if you add instances to the group after you change the instance template. Existing VMs use their previous instance templates until you recreate them, and new VMs use the currently specified instance template for the MIG.

Disabling creation retries mode during resize

Beta

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see the launch stage descriptions.

By default, if the initial creation of a VM instance fails, the managed instance group continuously retries to create the VM, until the VM is successfully created. However, if you do not want automatic creation retries, you can disable the creation retries mode when you resize the instance group. In this mode, the managed instance group attempts to create all instances only once. If there is an error during instance creation, the managed instance group gives up on this instance, removes it from the group's list of managed instances, and decreases the target size of the managed instance group.

This mode applies only during the first creation attempt of a VM. If a VM is created successfully while this mode is enabled, the VM behaves the same way as all other VMs created with a regular resize request. In particular, if a running VM dies unexpectedly at a later time and needs to be recreated, this mode does not affect the recreation behavior in that scenario.

Disabling the creation retries mode is especially useful in scenarios where you have systems automatically creating groups of VMs where an exact number of VMs is not required. You might prefer to stabilize quickly on the size of the managed instance group and be flexible in the number of VMs in the group, rather than to wait indefinitely until all the requested instances are created, which might be delayed temporarily or permanently because of quota errors or other unrelated issues.

Limitations

• You cannot disable creation retries for a regional MIG with an EVEN target distribution shape.
• For regional MIGs with a BALANCED or ANY_SINGLE_ZONE target distribution shape, you must disable proactive instance redistribution before you can disable creation retries mode.
• You cannot disable creation retries for autoscaled MIGs.

To resize a managed instance group when creation retries mode is disabled, use the Google Cloud CLI or REST.

Important: Google recommends that you do not concurrently run two resize operations with different creation retries modes as it might result in an unexpected target size. For example, if you issue a resize command without creation retries and, before it completes, you issue a second resize command with creation retries enabled, the second command does not invalidate the first one. Some instances are created without retries and if any of them fail, the group's target size decreases accordingly.

Permissions required for this task

To perform this task, you must have the following permissions:

• compute.instanceGroupManagers.update on the managed instance group

gcloud

Use the beta resize command with the --no-creation-retries flag.

gcloud beta compute instance-groups managed resize INSTANCE_GROUP_NAME \
    --size NEW_SIZE \
    --no-creation-retries \
    [--region REGION | --zone ZONE]

Replace the following:

• INSTANCE_GROUP_NAME: The name of the MIG that you want to resize.
• NEW_SIZE: The target number of running instances in managed instance group.
• ZONE: For zonal MIGs, the zone where the MIG is located.
• REGION: For regional MIGs, the region where the MIG is located.

REST

For a zonal MIG, use the beta instanceGroupManagers.resizeAdvanced method or, for a regional MIG, use the beta regionInstanceGroupManagers.resizeAdvanced method. Specify the new group size and the noCreationRetries field in the request body.

For example, make the following call to resize a zonal MIG with creation retries mode disabled.

POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeAdvanced

{
 "targetSize": NEW_SIZE,
 "noCreationRetries": true
}

Replace the following:

• PROJECT_ID: Your project ID.
• ZONE: The zone where the MIG is located. For a regional MIG, replace zones/ZONE with regions/REGION and specify the region. For more information, see About regional MIGs.
• INSTANCE_GROUP_NAME: The name of the MIG that you want to resize.

• NEW_SIZE: The target number of running instances in the managed instance group.

  You can see which instances are being created and in which mode by using the listManagedInstances method. When the creation retries mode is disabled, instances that you create have a currentAction of CREATING_WITHOUT_RETRIES.

Add, remove or abandon specific instances

Creating instances with specific names in MIGs

If you have a system that depends on specific names, you can add VMs with specific names to an existing MIG.

The names that you assign to these managed instances persist if the MIG recreates the VM. For more information about preserving the state of MIG instances, see stateful MIGs.

Limitations

If you create instances with specific names:

• You must turn off autoscaling during the instance creation process.
• You must set the MIG's update policy to be opportunistic.
• If you are updating a regional MIG:
  • You must disable proactive instance redistribution.
  • You can't select which zone to create the VM in.
• You can't use duplicate VM instance names within a project.

You can use the gcloud CLI or REST to add VMs with specific names to an existing MIG.

Permissions required for this task

To perform this task, you must have the following permissions:

• compute.instanceGroupManagers.update on the managed instance group

gcloud

Use the instance-groups managed create-instance command.

gcloud compute instance-groups managed create-instance INSTANCE_GROUP_NAME \
    --instance INSTANCE_NAME \
    [--region REGION | --zone ZONE]

Replace the following:

• INSTANCE_GROUP_NAME is the name of the regional or zonal MIG to add a new instance to.
• INSTANCE_NAME is the name you want to assign to the new instance.

REST

Send a POST request to the regionInstanceGroupManagers.createInstances method. For a zonal managed instance group, use the instanceGroupManagers.createInstances method. In the request body, include the instances field with names for one or more instances.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/INSTANCE_GROUP_NAME/createInstances
{
  "instances": [
    {
      "name": "INSTANCE_NAME_1"
    },
    {
      "name": "INSTANCE_NAME_2"
    }
  ]
}

Deleting specific instances from a group

You can delete individual VM instances in a managed instance group. Deleting instances reduces the specified targetSize of the instance group and removes the VMs from any target pools of which they are a member.

Deleting specific instances from a managed instance group doesn't change any specified autoscaler settings. If you delete instances from a managed instance group, the autoscaler might detect an increase in the workload on the other instances in the group and increase the group size back to its previous level. To prevent this, stop the autoscaler before attempting to delete the instances.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Permissions required for this task

To perform this task, you must have the following permissions:

• compute.instanceGroupManagers.update on the managed instance group

Console

1. In the Google Cloud console, go to the Instance groups page.

   Go to Instance groups

2. Under the Name column of the list, click the name of the instance group where you want to delete individual instances. A page opens with the instance group properties and a list of instances that are included in the group.

3. In the Members section, from the list of instances select one or more instances that you want to delete.

4. Click Delete instance. The selected instances are deleted.

gcloud

To delete an instance with the gcloud CLI, use the instance-groups managed delete-instances command.

gcloud compute instance-groups managed delete-instances INSTANCE_GROUP_NAME \
    --instances INSTANCE_NAME_1,INSTANCE_NAME_2 \ 
    [--region REGION | --zone ZONE]

The command returns the operation status per instance:

• SUCCESS is returned if the gcloud CLI deleted the instance.
• FAIL is returned for all instances if any instance that you specify in the request is not a member of the group, if any member is already being deleted or abandoned, or if the request fails for any other reason.
• SKIPPED is returned only if you use the --skip-instances-on-validation-error flag and the instance is not a member of the group or is already being deleted or abandoned.
• MEMBER_NOT_FOUND is returned only for regional groups when the gcloud CLI wasn't able to resolve the zone from the instance name.

If you anticipate that your deletion request might conflict with concurrent operations, use the --skip-instances-on-validation-error flag. This flag specifies whether the request should proceed even if the request includes instances that are not members of the group or that are already being deleted or abandoned—for example, instances that are being deleted by an autoscaler. By default, if you omit this flag and such an instance is specified in the request, the operation fails. The operation always fails if the request contains a badly formatted instance name or a reference to an instance that exists in a zone or region other than the group's zone or region.

For example, the following command requests deletion of four instances, but three of them are no longer in the group:

gcloud compute instance-groups managed delete-instances example-mig \
    --instances=example-instance-1,example-instance-2,example-instance-3,example-instance-4 \
    --region=us-central1 \
    --skip-instances-on-validation-error

In this example, three instances didn't pass validation, so they were skipped, while example-instance-3 was deleted:

Updated [https://www.googleapis.com/compute/v1/projects/example-project/regions/us-central1/instanceGroupManagers/example-mig].
PROJECT          ZONE           INSTANCE               STATUS   VALIDATION_ERROR
example-project  us-central1-a  example-instance-1     SKIPPED  Cannot delete instance https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances/example-instance-1 that was already deleted.
example-project  us-central1-a  example-instance-2     SKIPPED  Cannot delete instance https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances/example-instance-2 that was already abandoned.
example-project  us-central1-a  example-instance-3     SUCCESS
example-project  us-central1-a  example-instance-4     SKIPPED  Cannot find instance https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances/example-instance-4.

If you omit the --skip-instances-on-validation-error flag and if any instance does not pass validation, then no instance is deleted:

gcloud compute instance-groups managed delete-instances example-mig \
    --instances=example-instance-1,example-instance-2,example-instance-3,example-instance-4 \
    --region=us-central1

Updated [https://www.googleapis.com/compute/v1/projects/example-project/regions/us-central1/instanceGroupManagers/example-mig].
PROJECT          ZONE             INSTANCE              STATUS
example-project  us-central1-a    example-instance-1    FAIL
example-project  us-central1-a    example-instance-2    FAIL
example-project  us-central1-a    example-instance-3    FAIL
example-project  us-central1-a    example-instance-4    FAIL

REST

Send a POST request to the regionInstanceGroupManagers.deleteInstances method. For a zonal managed instance group, use the instanceGroupManagers.deleteInstances method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/INSTANCE_GROUP_NAME/deleteInstances

{
  "instances": [
    "zones/ZONE/instances/INSTANCE_NAME_1",
    "zones/ZONE/instances/INSTANCE_NAME_2"
   ]
}

If you anticipate that your deletion request might conflict with concurrent operations, use the skipInstancesOnValidationError field. This field specifies whether the request should proceed even if the request includes instances that are not members of the group or that are already being deleted or abandoned—for example, instances that are being deleted by an autoscaler. By default, if you omit this flag and such an instance is specified in the request, the operation fails. The operation always fails if the request contains a badly formatted instance name or a reference to an instance that exists in a zone or region other than the group's zone or region.

For example, the following request to delete 2 instances will succeed even if one of the instances is already deleted or being deleted:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/INSTANCE_GROUP_NAME/deleteInstances

{
 "instances": [
  "zones/ZONE/instances/INSTANCE_NAME_1",
  "zones/ZONE/instances/INSTANCE_NAME_2"
 ],
 "skipInstancesOnValidationError": true
}

You can view instances that were skipped in the operation by viewing the operation. For regional MIGs, use the regionOperations.get method. For zonal MIGs, use the zoneOperations.get method. Warnings are listed in the operation's warning field.

After you make a request to delete instances in a managed instance group, the instances stop as soon as the system can delete them. This process can take a significant amount of time depending on the number of instances that you delete from the group. Verify the status of the group or verify the status of the managed instances.

Abandoning instances from a MIG

You can separate a VM instance from a managed instance group to more easily debug issues with individual instances without affecting the group as a whole. Abandoning an instance from a group also removes the instance from load balancers that were assigned to the managed instance group. Target pools that were manually assigned to specific individual instances are not removed.

You cannot add an abandoned VM instance back to the group. Abandoning instances reduces the specified targetSize of the instance group, but doesn't change any specified autoscaler settings. Managed instance groups with an autoscaler continue to add or remove instances automatically as necessary. For a regional MIG with autoscaling set to ONLY_SCALE_OUT mode, if you abandon a VM instance to reduce the MIG target size to the autoscaler's recommended size, then the autoscaler might create a new VM instance in the same zone of the abandoned VM. In such cases, to avoid creation of a new VM instance, try abandoning VM instance from a different zone.

If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Permissions required for this task

To perform this task, you must have the following permissions:

• compute.instanceGroupManagers.update on the managed instance group

Console

1. In the Google Cloud console, go to the Instance groups page .

   Go to Instance groups

2. Under the Name column of the list, click the name of the instance group from which you want to remove instances. A page opens with the instance group properties and a list of instances that are included in the group.

3. In the Members section, from the list of instances select one or more instances that you want to remove from the group.

4. Click Remove from group. The selected instances leave the group, but continue to run outside of the group.

gcloud

To remove an instance from the instance group without deleting the VM, use the instance-groups managed abandon-instances command.

gcloud compute instance-groups managed abandon-instances INSTANCE_GROUP_NAME \
    --instances INSTANCE_NAME_1,INSTANCE_NAME_2 \
    [--region REGION | --zone ZONE]

REST

Send a POST request to the regionInstanceGroupManagers.abandonInstances method. For a zonal managed instance group, use the instanceGroupManagers.abandonInstances method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instanceGroupManagers/INSTANCE_GROUP_NAME/abandonInstances

{
 "instances": [
  "zones/ZONE/instances/INSTANCE_NAME_1",
  "zones/ZONE/instances/INSTANCE_NAME_2"
 ]
}

After you make a request to abandon instances from a managed instance group, the group removes the instances as soon as possible. Verify the status of the group or verify the status of the managed instances.

Increase the group's size limit

By default, the size limit for a zonal MIG is 1,000 VMs, and the size limit for a regional MIG is 2,000 VMs. You can further increase these limits to 2,000 VMs for a zonal MIG and 4,000 VMs for a regional MIG.

To increase the limits, set the group's listManagedInstancesResults field to PAGINATED. If you still need more VMs in the group, create multiple MIGs or contact support.

Caution: When you set the group's listManagedInstancesResults field to PAGINATED, the listManagedInstances method changes its behavior. With the default PAGELESS setting, the list method ignores the maxResults query parameter and always returns all the instances. With the PAGINATED setting, the list method returns paginated result with maxResults capped at 500 like other lists in Compute Engine. You need to make sure that all calls to these methods are aware of pagination and use the pageToken query parameter to page through large list of instances.

There is no impact on Google Cloud CLI or the Google Cloud console which already fully support pagination.

Console

1. In the Google Cloud console, go to the Instance groups page.

   Go to Instance groups

2. Under the Name column of the list, click the name of the instance group of which you want to increase the size limit.

3. Click Edit to modify the instance group.

4. Expand the advanced settings section by clicking Show Advanced Configuration.

5. Under Managed instance list API call results, select the Paginated option.

6. Click Save to apply the changes.

gcloud

For an existing MIG, use the update command:

gcloud compute instance-groups managed update INSTANCE_GROUP_NAME \
    --list-managed-instances-results=PAGINATED

For a new MIG, use the create command:

gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \
    --template INSTANCE_TEMPLATE \
    --size SIZE \
    --list-managed-instances-results=PAGINATED

Terraform

To increase a MIG's size limit, set the list_managed_instances_results argument to PAGINATED.

The following sample increases the size limit of a zonal MIG. For more information about the resource used in the sample, see the google_compute_instance_group_manager resource. For a regional MIG, use the google_compute_region_instance_group_manager resource.

resource "google_compute_instance_group_manager" "default" {
  name                           = "my-igm"
  base_instance_name             = "test"
  target_size                    = 7
  zone                           = "us-central1-f"
  list_managed_instances_results = "PAGINATED"
  version {
    instance_template = google_compute_instance_template.default.id
    name              = "primary"
  }
}

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

REST

For an existing zonal MIG, use the instanceGroupManagers.patch method or, for a regional MIG, use the regionInstanceGroupManagers.patch method–for example:

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/region/REGION/regionInstanceGroupManagers/INSTANCE_GROUP_NAME

{
  "listManagedInstancesResults": "PAGINATED",
}

For a new zonal MIG, use the instanceGroupManagers.insert method or, for a regional MIG, use the regionInstanceGroupManagers.insert method.

If you want to switch back to pageless list results, you must also revert to the group's default size limit. To switch back, first ensure that the group does not exceed its default size limit, then set the listManagedInstancesResults field to PAGELESS.

What's next

• Learn about working with managed instances, for example, to delete, abandon, and recreate managed instances.
• View info about managed instance groups and managed instances.
• Troubleshoot managed instance groups.