This document describes how to create a managed instance group (MIG). A MIG is a group of virtual machine (VM) instances that you control as a single entity. MIGs support features such as autohealing, load balancing, autoscaling, auto-updating, and stateful workloads.
You can create regional MIGs or zonal MIGs. Regional MIGs provide higher availability compared to zonal MIGs because the instances in a regional MIG are spread across multiple zones in a single region. This document provides information about creating either zonal or regional MIGs. However, regional MIGs have additional options and considerations. For more information about regional MIGs, see the Regional MIGs overview.
For conceptual information about instance groups and their features, see the Instance groups overview.
Before you begin
- If you want to use the API examples in this guide, set up API access.
- If you want to use the command-line examples in this guide, install the
gcloud
command-line tool. - Create an instance template, which is a building block for a managed instance group.
Limitations
- With a regional MIG, you can create up to 2,000 VMs in a region, with a maximum of 1,000 VMs per zone. With a zonal MIG, you can create up to 1,000 VMs. If you need more, contact support.
- When updating a MIG, you can specify up to 1,000 VMs in a single request.
If you want a stateful MIG, review the stateful MIG limitations.
If you want a regional MIG, review the regional MIG limitations.
Shared VPC on interfaces other than
nic0
for managed instance groups is supported in Preview. This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Platform Terms of Service. Pre-GA features may have limited support, and changes to pre-GA features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.
Stateless or stateful MIGs
You can use MIGs for stateless serving or batch workloads, such as a website frontend or image processing from a queue, or for stateful applications, such as databases or legacy applications.
Using MIGs for stateless applications
Stateless applications don't depend on the specific state of the underlying virtual machine (VM) instance to run. If you use a stateless MIG, your application should not depend on VM properties that don't persist, such as the contents of attached disks or in-memory data. If your application requires the preservation of VM properties, see Configuring stateful MIGs.
The default behavior for all persistent disks in a MIG is to delete or recreate them when the corresponding VM is deleted or recreated, so you should not rely on attached disks as persistent data. To retain your data in a stateless MIG, we recommend that you regularly maintain up-to-date custom images with updated software and configurations or use startup scripts and configure your applications to back up necessary data in another centralized location, such as Cloud Storage.
In your instance templates, you can specify a container image or a custom image with relevant startup scripts, so that when a VM is recreated, it has the necessary apps installed and has access to required data. For more information about creating instance templates, see Deterministic instance templates.
Using MIGs for stateful applications
Managed instance groups also support stateful applications. A stateful MIG preserves each VM's unique state (VM instance name, attached persistent disks, and/or metadata) on machine restart, recreation, autohealing, or update.
Use stateful MIGs for applications that have stateful data or configuration, such as databases, data processing applications, legacy applications, and long-running batch workloads with checkpointing.
Consider using stateful MIGs whenever you deploy a stateful application or cluster to Compute Engine and would like to improve its availability with autohealing and multi-zone deployment, or would like to simplify and accelerate updates with automated rolling updates.
For information about how to create a stateful MIG or add stateful configuration to an existing MIG, see Configuring stateful MIGs.
Creating a managed instance group
A MIG creates each of its managed instances based on the
instance templates and optional
stateful configuration
that you specify. Each managed instance is a data entity within the MIG that
contains the current status and intended state for an actual VM instance. MIGs
maintain high availability of your applications by proactively keeping actual
VMs available, that is, in the RUNNING
state.
You can create a MIG through the Google Cloud Console, the
gcloud
tool, or the
Compute Engine API. If you'd rather
create a regional MIG,
see Creating regional MIGs.
Console
In the Cloud Console, go to the Instance groups page.
Click Create an instance group.
If you want to create a stateful MIG, select the New managed instance group (stateful) option. To help you decide, see When to use stateful MIGs.
Enter a name for the MIG, and select the zone where you want to locate the group.
Under Group type, select Managed instance group.
Under Instance template, select an instance template. If no templates are available, create an instance template.
Specify the number of instances that you want to create in the group.
Optionally, for stateless MIGs, enable Autoscaling so that the group automatically adds or removes instances based on its utilization
Optionally, enable autohealing to perform application-based health checking on VMs within the group.
Click Create to create the new group.
gcloud
Create an instance group with the
instance-groups managed create
command.
gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \ --size SIZE \ --template INSTANCE_TEMPLATE \ --zone ZONE
Replace the following:
INSTANCE_GROUP_NAME
: the name for this instance group.SIZE
: the size of the instance group.INSTANCE_TEMPLATE
: the name of the instance template to use for this group.ZONE
: one of the zones available for Compute Engine. If you want to create a regional MIG, see Creating and managing regional MIGs.
You can optionally supply the --base-instance-name
flag. Because these
VMs are based on a common template, each VM is assigned a random string as
part of its VM name. The base name is prepended to this random string. For
example, if you set the base name to test
, VMs will have names like
test-yahs
and test-qtyz
. If you need specific names, see
Creating instances with specific names in MIGs.
For example, the following command creates an instance group named
example-group, with base VM name test
. The group contains three
instances:
gcloud compute instance-groups managed create example-group \ --base-instance-name test \ --size 3 \ --template an-instance-template
API
Call the
instanceGroupManagers.insert
method
or the
regionInstanceGroupManagers.insert
method.
In the request body, include the group name, group size,
and the URL to the instance template.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers { "versions": [ { "instanceTemplate": "global/instanceTemplates/INSTANCE_TEMPLATE" } ], "name": "INSTANCE_GROUP_NAME", "targetSize": SIZE }
Replace the following:
PROJECT_ID
: the project ID for the request.ZONE
: the zone for the request. For a regional MIG, replacezones/ZONE
withregions/REGION
and specify a region. For more information, see Creating and managing regional MIGs.INSTANCE_GROUP_NAME
: the name for this instance group.SIZE
: the size of the instance group.INSTANCE_TEMPLATE
: the instance template to use for this group.
You can optionally supply the base-name
field. Because these
VMs are based on a common template, each VM is assigned a random string as
part of its VM name. The base name is prepended to this random string. For
example, if you set the base name to test
, VMs will have names like
test-yahs
and test-qtyz
. If you need specific names, see
Creating instances with specific names in MIGs.
Depending on how you configure and act on a MIG, various policies and actions can affect the instances in the group. To determine which managed instances are up and running, see Checking the status of managed instances.
Changing the instance template for a managed instance group
To learn how to apply a new instance template to a MIG, see Updating instances in a MIG.
Resizing a managed instance group
To add or remove VMs in a MIG, you can:
- Use autoscaling for stateless applications.
- Manually set the size of the MIG.
- Create instances with specific names or delete or abandon specific instances.
Automatically resizing a MIG
You can configure managed instance groups to automatically add or remove VMs based on their workloads. Your applications can gracefully handle increases in traffic, and you can reduce your costs when the need for compute resources is lower. To start scaling your managed instance groups, see Autoscaling groups of instances.
Manually setting the size of a MIG
If a managed instance group is not already set to
automatically scale, you can resize the group
manually to change the number of instances. If you increase the
size, the managed instance group uses the current instance template to add new
instances. If you decrease the size, the managed instance group deletes
VMs from the group. The group deletes instances with a
currentAction
of DELETING
, CREATING
, and RECREATING
before it deletes instances that are
running with no scheduled actions.
If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.
You can resize a managed instance group by using the
Google Cloud Console, the
gcloud
tool, or the
Compute Engine API.
Console
In the Cloud Console, go to the Instance groups page.
Under the Name column of the list, click the name of the instance group where you want to change the group size.
Click Edit group to modify this managed instance group.
Under Number of instances, specify the number of instances that you want to include in this managed instance group. If Autoscaling is enabled, the group is automatically adding or removing instances as necessary. However, you can change the Minimum number of instances and the Maximum number of instances values to indirectly adjust the group size through Autoscaler.
Click Save to apply the new template.
gcloud
Use the resize
command.
gcloud compute instance-groups managed resize INSTANCE_GROUP_NAME \ --size NEW_SIZE \ --zone ZONE
API
Call the resize
method for a
zonal
or
regional
MIG resource. Specify the new instance group size as a parameter.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resize?size=NEW_SIZE
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
After you make a request to resize a managed instance group, the VMs start or stop as soon as the system can provision or delete them. This process can take a significant amount of time depending on the number of instances in the group. Verify the status of instances in your managed instance group.
Disabling creation retries mode during resize
By default, if the initial creation of a VM instance fails,
the managed instance group continuously retries to create the VM,
until the VM is successfully created. However, if you do not want
automatic creation retries, you can disable the creation retry mode by
setting the --nocreation-retries
flag when you resize the instance group.
In this mode, the managed instance group attempts to create all instances only
once. If there is an
error
during instance creation, the managed instance group gives up on this instance,
removes it from the
group's list of managed instances,
and decreases the target size of the managed instance group.
This mode applies only during the first creation attempt of a VM. If a VM is created successfully while this mode is enabled, the VM behaves the same way as all other VMs created with a regular resize request. In particular, if a running VM dies unexpectedly at a later time and needs to be recreated, this mode does not affect the recreation behavior in that scenario.
The disable creation retry mode is especially useful in scenarios where you have systems automatically creating groups of VMs where an exact number of VMs is not required. You might prefer to stabilize quickly on the size of the managed instance group and be flexible in the number of VMs in the group, rather than to wait indefinitely until all the requested instances are created, which might be delayed temporarily or permanently because of quota errors or other unrelated issues.
To resize a managed instance group in the disabled creation retries mode, use
the gcloud
tool or the
Compute Engine API.
gcloud
Use the beta resize
command
with the --no-creation-retries
flag.
gcloud beta compute instance-groups managed resize INSTANCE_GROUP_NAME --size NEW_SIZE \ --no-creation-retries \ --zone ZONE
API
Call the beta resizeAdvanced
method for a
zonal
or
regional
MIG resource. Specify the new group size and the noCreationRetries
field
in the request body.
POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeAdvanced { "targetSize": SIZE "noCreationRetries": true }
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
You can see which instances are being created and in which mode by using
the listManagedInstances
method. Instances that are being created in the
disabled creation retries mode have a
currentAction
of CREATING_WITHOUT_RETRIES
.
Creating instances with specific names in MIGs
If you have a system that depends on specific names, you can create VMs with those names. The names that you assign to these managed instances persist if the VM is recreated. See Adding instances with specific names.
Deleting specific instances from a group
You can delete individual VM instances in a managed instance group. Deleting
instances reduces the specified targetSize
of the instance group and removes
the VMs from any
target pools of which they
are a member.
Deleting instances from a managed instance group does not change any specified autoscaler settings. If you delete instances from a managed instance group, the autoscaler might detect an increase in the workload on the other instances in the group and increase the group size back to its previous level. To prevent this, turn off or delete the autoscaler before attempting to delete the instances.
If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.
Delete instances from a managed instance group by using the
Google Cloud Console, the
gcloud
tool, or the
Compute Engine API.
Console
In the Cloud Console, go to the Instance groups page.
Under the Name column of the list, click the name of the instance group where you want to delete individual instances. A page opens with the instance group properties and a list of instances that are included in the group.
On the list of instances, select one or more instances that you want to delete.
Click Delete instance. The selected instances are deleted.
gcloud
Use the delete-instances
command.
gcloud compute instance-groups managed delete-instances INSTANCE_GROUP_NAME \ --instances example-i3n2,example-z2x9 \ --zone ZONE
API
Call the deleteInstances
method for a
zonal
or
regional
MIG resource. In the request body, include the URLs to one or more
instances that you want to delete.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/deleteInstances { "instances": [ "zones/ZONE/instances/example-instance-i3n2", "zones/ZONE/instances/example-instance-l6n1" ] }
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
After you make a request to delete instances in a managed instance group, the instances stop as soon as the system can delete them. This process can take a significant amount of time depending on the number of instances that you delete from the group. Verify the status of instances in your managed instance group.
Abandoning instances from a group
You can separate a VM instance from a managed instance group to more easily debug issues with individual VMs without affecting the group as a whole. Abandoning an instance from a group also removes that VM from load balancers that were assigned to the managed instance group. Target pools that were manually assigned to specific individual instances are not removed.
Abandoning instances reduces the specified targetSize
of the instance group,
but does not change any specified autoscaler settings. Managed instance groups
with an autoscaler continue to add or remove VMs automatically as
necessary.
If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.
Abandon instances from a managed instance group by using the
Google Cloud Console, the
gcloud
tool, or the
Compute Engine API.
Console
In the Cloud Console, go to the Instance groups page.
Under the Name column of the list, click the name of the instance group from which you want to remove instances. A page opens with the instance group properties and a list of instances that are included in the group.
On the list of instances, select one or more instances that you want to remove from the group.
Click Remove from group. The selected instances leave the group, but continue to run outside of the group.
gcloud
To remove an instance from the instance group
without deleting the instance, use the
abandon-instances
command.
gcloud compute instance-groups managed abandon-instances INSTANCE_GROUP_NAME \ --instances example-i3n2,example-z2x9 \ --zone ZONE
API
Call the abandonInstances
method for a
zonal
or
regional
MIG resource. In the request body, include the URLs to one or more
instances that you want to abandon.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/abandonInstances { "instances": [ "zones/ZONE/instances/example-instance-i3n2", "zones/ZONE/instances/example-instance-l6n1" ] }
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
After you make a request to abandon instances from a managed instance group, the group removes the instances as soon as possible. Verify the status of instances in your managed instance group.
Recreating instances in the group
Recreating a managed instance deletes the specified VM and creates a new VM using the instance template that is assigned to the managed instance group.
Use this method to update selected VMs so that they use the latest instance template. If you need to recreate all of the VMs in a managed instance group, start a rolling update instead.
If the group is part of a backend service that has enabled connection draining, it can take up to an additional 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.
Recreate selected VMs in managed instance group by using the
gcloud
tool, or the
Compute Engine API.
gcloud
Use the
recreate-instances
command.
gcloud compute instance-groups managed recreate-instances INSTANCE_GROUP_NAME \ --instances example-i3n2,example-z2x9 \ --zone ZONE
API
Call the recreateInstances
method for a
zonal
or
regional
MIG resource. In the request body, include the URLs to one or more instances
that you want to recreate.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/recreateInstances { "instances": [ "zones/ZONE/instances/example-instance-i3n2", "zones/ZONE/instances/example-instance-l6n1" ] }
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
After you make a request to recreate instances in a managed instance group, the new VMs start as soon as the system can provision them. This process can take a significant amount of time depending on the number of instances that you recreate. Verify the status of instances in your managed instance group.
Deleting a managed instance group
When you delete a managed instance group, all VMs in the group are deleted. If you must keep any of the VMs in this managed instance group, abandon the instances first to remove the VMs from the group. Then, delete the managed instance group.
When you delete a managed instance group and its instances by using the
Google Cloud Console or the gcloud
tool,
any attached autoscalers are automatically deleted. However, if you use the
Compute Engine API, you must first
issue separate requests to delete any attached autoscalers with
autoscalers.delete
or regionAutoscalers.delete
.
Console
In the Cloud Console, go to the Instance groups page.
Select one or more groups on the list that you want to delete.
Click Delete to delete the group and all of the VMs in the managed instance group.
gcloud
Use the
delete
command.
gcloud compute instance-groups managed delete INSTANCE_GROUP_NAME \ --zone ZONE
API
Call the delete
method for a
zonal
or
regional
MIG resource.
DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME
If the instance group is a regional managed instance group, replace
zones/ZONE
with regions/REGION
.
Creating groups of preemptible instances
You can use managed instance groups to quickly create multiple preemptible instances, which can reduce the costs of the VMs in your managed instance groups. For example, you can create a group of preemptible instances, use them to run a batch processing task, and then delete the group when the task is complete.
To create a group of preemptible instances, set the premptible option in an instance template, and then use the template to create the managed instance group.
Console
In the Cloud Console, go to the Instance templates page.
Click New instance template.
Fill in the properties that you want for your instance template.
Click Show advanced options to expand the Availability policies section.
Set Preemptibility to On.
Click Create to create the template.
Use this template to create a managed instance group
gcloud
Create an instance template by using the
instance-templates create
command.
Include the --preemptible
flag.
gcloud compute instance-templates create INSTANCE_TEMPLATE \ --preemptible
After you create the instance template, use it to create a managed instance group.
API
Call the
instanceTemplates.insert
method
to create a new instance template. Include the scheduling.preemptible
property and set it to true
.
{ "name": "INSTANCE_TEMPLATE", "properties": { "machineType": "zones/ZONE/machineTypes/MACHINE_TYPE", "networkInterfaces": [ { "network": "global/networks/default", "accessConfigs": [ { "name": "external-IP", "type": "ONE_TO_ONE_NAT" } ] } ], "scheduling": { "preemptible": true }, "disks": [ { "type": "PERSISTENT", "boot": true, "mode": "READ_WRITE", "initializeParams": { "sourceImage": "projects/debian-cloud/global/images/family/debian-9" } } ] } }
After you create the instance template, use it to create a managed instance group.
Understanding the instanceTemplate
and versions
fields
When you create a managed instance group, you supply an instance template
that the managed instance group uses to create individual VM instances.
By default, Compute Engine describes the instance template used
in two separate Compute Engine API properties: the top-level instanceTemplate
property
and the versions
property. For example, in the following managed instance
group, notice that both the instanceTemplate
and versions
fields are
populated:
{ "name": "example-group", "zone": "zones/us-central1-a", "instanceTemplate": "global/instanceTemplates/example-it", "versions": [ { "name": "v3", "instanceTemplate": "global/instanceTemplates/example-it", "targetSize": { "calculated": 3 } } ]... }
Compute Engine automatically populates both the top-level
instanceTemplate
field and the versions
field for backward compatibility.
We recommend specifying the versions
field and omitting the top-level
instanceTemplates
field whenever you can. However, if your application code
currently sets the top-level instanceTemplate
field, it is still a valid
request.
To learn more, read about the Relationship between versions and instanceTemplate fields.
(Advanced) Canarying instance templates with a managed instance group
You can create a managed instance group where there are two groups of VMs that use different instance templates. For example, you might want to create a managed instance group that has 20 VM instances and 10 VMs should run on a specific operating image while the rest run on a different operating system image. This feature lets you compare the two different instance template versions before deciding on one.
For more information, read about Canary updates.
Managed instance groups and IAM
All the operations performed by Compute Engine as part of the managed instance
group are performed by the Google APIs service account for your project.
This per-project service account has an email address like the following, where
PROJECT_ID
is the numerical ID of the corresponding
project:
PROJECT_ID@cloudservices.gserviceaccount.com
The Google APIs service account is different from the default Compute Engine service account.
It is up to you to ensure that the service account used by the managed instance
group has enough privileges to create resources based on the
instance template. In particular, this means that the service account needs to
be granted the compute.instanceAdmin.v1
and, optionally, the
serviceAccountUser
role, to create and manage VMs in the instance
group. The serviceAccountUser
role is required only if the managed
instance group will be creating VMs that can run as a service account.
Keep in mind that this account is also used by other processes,
including Deployment Manager.
When you create a managed instance group or update an instance template, Compute Engine validates that the Google APIs service account has the following:
serviceAccountUser
role. Important if you plan to create instances that can run as a service account.- Permissions to all the resources referenced from instance templates, such as images, disks, VPC networks and subnets.
To learn more about service accounts, read the Service accounts overview.
Updating all instances in a managed instance group
See Updating instances in a MIG.
What's next
- Learn about working with managed instances, for example, to delete, abandon, and recreate managed instances.
- Get info about managed instance groups and managed instances.
- Choose a load balancer or add an instance group to a load balancer.
- Enable autohealing for your managed instance group.
- Enable autoscaling for your managed instance group.
- Try a tutorial:
- Troubleshoot managed instance groups.