Distributing instances by using regional MIGs

This page describes how to create regional managed instance groups (MIGs). You can also create zonal (single-zone) MIGs. Both zonal MIGs and regional MIGs support autohealing, load balancing, autoscaling, auto-updating, and stateful workloads.

Regional MIGs provide higher availability compared to zonal MIGs because a regional MIG's managed instances are evenly spread across multiple zones in a single region. But regional MIGs have different limitations compared to zonal MIGs.

Whether zonal or regional, the MIG bases each of its managed instances on a common instance template and optional stateful configuration. Each managed instance is a data entity within the MIG that contains the current status and intended state for an actual VM instance. MIGs maintain high availability of your applications by proactively keeping actual VMs available, that is, in the RUNNING state.

For more information about instance groups and their features, see the Instance groups overview.

Before you begin

Limitations

  • Each regional MIG can contain up to 2,000 VMs.
  • When updating a MIG, no more than 1,000 VMs can be specified in a single request.
  • You can't use regional MIGs with a load balancer that uses the maxRate balancing option.
  • When used with autoscaling based on Cloud Monitoring metrics:
    • Unlike single-zone MIGs, regional MIGs don't support filtering for per-instance metrics.
    • Unlike single-zone MIGs, regional MIGs don't support autoscaling using per-group metrics.

Choosing regional managed instance groups

Google recommends regional MIGs over zonal MIGs because they allow you to spread your application load across multiple zones, rather than confining your application to a single zone or managing multiple instance groups across different zones. Using multiple zones protects against zonal failures and unforeseen scenarios where an entire group of instances in a single zone malfunctions. If that happens, your application can continue serving traffic from instances running in another zone in the same region.

In the case of a zone failure, or if a group of VMs in a zone stops responding, a regional MIG continues supporting your VMs as follows:

  • The number of VMs that are part of the regional MIG in the remaining zones continue to serve traffic. No new VMs are added and no VMs are redistributed (unless you set up autoscaling).

  • After the failed zone has recovered, the MIG starts serving traffic again from that zone.

When designing for robust and scalable apps, use regional MIGs.

Proactive instance redistribution

By default, a regional MIG attempts to maintain an even distribution of VMs across zones in the region to maximize the availability of your application in the event of a zone-level failure.

If you delete or abandon instances from your group, causing uneven distribution across zones, the group proactively redistributes instances to reestablish an even distribution.

To reestablish an even distribution across zones, the group deletes VMs in zones with more VMs and adds VMs to zones with fewer VMs. The group automatically picks which VMs to delete.

Proactive redistribution reestablishes even distribution across zones.
Example of proactive redistribution

For example, suppose you have a regional MIG with 4 instances spread across 3 zones: a, b, and c. If you delete 3 managed instances in c, the group attempts to rebalance so that the instances are again evenly distributed across the zones. In this case, the group deletes 2 instances (one from a and one from b) and creates 2 instances in zone c, so that each zone has 3 instances and even distribution is achieved. There is no way to selectively determine which instances are deleted. The group temporarily loses capacity while the new instances start up.

To prevent automatic redistribution of your VMs, you can turn off proactive instance redistribution when creating or updating a regional MIG.

This is useful when you need to:

  • Delete or abandon VMs from the group without affecting other running VM instances. For example, you can delete a batch worker VM after job completion without affecting other workers.
  • Protect VMs with stateful apps from undesirable automatic deletion by proactive redistribution.
Disabling proactive redistribution can affect capacity during a
            zonal failure.
Uneven distribution after disabling proactive redistribution

If you turn off proactive instance redistribution, a MIG does not proactively add or remove VMs to achieve balance but still opportunistically converges toward balance during resize operations, treating each resize operation as an opportunity to balance the group. For example, when scaling in, the group automatically uses the rescaling as an opportunity to remove VMs from bigger zones; when scaling out, the group uses the opportunity to add VMs to smaller zones.

Provisioning the correct MIG size to ensure availability

A variety of events may cause one or more VMs to become unavailable, and you can help mitigate this issue using multiple Google Cloud services:

  • Use a regional MIG to distribute your application across multiple zones.
  • Use autohealing to recreate failed VMs.
  • Use load balancing to automatically direct user traffic away from unavailable VMs.

However, even if you use these services, your users may still experience issues if too many of your VMs are simultaneously unavailable.

To be prepared for the extreme case where one zone fails or an entire group of VMs stops responding, Google strongly recommend overprovisioning your MIG. Depending on your application needs, overprovisioning your group prevents your system from failing entirely if a zone or group of VMs becomes unresponsive.

Google makes recommendations for overprovisioning with the priority of keeping your application available for your users. These recommendations include provisioning and paying for more VMs than your application might need on a day-to-day basis. Base your overprovisioning decisions on application needs and cost limitations.

Estimating the recommended group size

Compute Engine recommends that you provision enough VMs such that, if all of the VMs in any one zone are unavailable, your remaining VMs still meet the minimum number of VMs that you require.

Use the following table to determine the minimum recommended size for your group:

Number of Zones Additional VMs Recommended Total VMs
2 +100% 200%
3 +50% 150%
4 +33% 133%

The recommended number of additional VMs is inversely proportional to the number of zones where your MIG is located. Thus, you can reduce the number of additional VMs by evenly distributing your application across a higher number of zones.

Provisioning a regional MIG in three or more zones

When you create a regional MIG in a region with at least three zones, Google recommends overprovisioning your group by at least 50%. By default, a regional MIG creates VMs in three zones. Having VM in three zones already helps you preserve at least 2/3 of your serving capacity, and if a single zone fails, the other two zones in the region can continue to serve traffic without interruption. By overprovisioning to 150%, you can ensure that if 1/3 of the capacity is lost, 100% of traffic is supported by the remaining zones.

For example, if you need 20 VMs in your MIG across three zones, we recommend, at a minimum, an additional 50% of VMs. In this case, 50% of 20 is 10 more VMs, for a total of 30 VMs in the group. If you create a regional MIG with a size of 30, the group distributes your VMs as evenly as possible across the three zones, like so:

Zone Number of VMs
example-zone-1 10
example-zone-2 10
example-zone-3 10

If any single zone fails, you still have 20 VMs serving traffic.

Provisioning a regional MIG in two zones

To provision your VMs in two zones instead of three, Google recommends doubling the number of VMs. For example, if you need 20 VMs for your service, distributed across two zones, we recommend that you configure a regional MIG with 40 VMs, so that each zone has 20 VMs. If a single zone fails, you still have 20 VMs serving traffic.

Zone Number of VMs
example-zone-1 20
example-zone-2 20

If the number of VMs in your group is not easily divisible across two zones, Compute Engine divides the VMs as evenly as possible and randomly puts the remaining VMs in one of the zones.

Provisioning a regional MIG in one zone

It is possible to create a regional MIG with just one zone. This is similar to creating a zonal MIG.

Creating a single zone regional MIG is not recommended because it offers the minimum guarantee for highly available applications. If the zone fails, your entire MIG is unavailable, potentially disrupting your users.

Selecting zones for your group

The default configuration for a regional MIG is to distribute VMs as evenly as possible across three zones. For various reasons, you might want to select specific zones for your application. For example, if you require GPUs for your VMs, you might select only zones that support GPUs. You might have persistent disks that are only available in certain zones, or you might want to start with VMs in just a few zones, rather than in three random zones within a region.

If you want to choose the number of zones or choose the specific zones the group should run in, you must do that when you first create the group. After you choose specific zones during creation, you cannot change or update the zones later.

  • To select more than three zones within a region, you must explicitly specify the individual zones. For example, to select all four zones within a region, you must provide all four zones explicitly in your request. If you do not, Compute Engine selects three zones by default.

  • To select two or fewer zones in a region, you must explicitly specify the individual zones. Even if the region only contains two zones, you must still explicitly specify the zones in your request.

Regardless whether you choose specific zones or whether you just want to select the region and allow Compute Engine to create VMs in all zones within the region, by default, the new VMs are distributed evenly across all zones. As a best practice, make sure you provision enough VMs to support your apps in the specified number of zones.

Creating a regional MIG

Create a regional MIG in the gcloud command-line tool, the console, or the API.

If there is not enough capacity in each zone to support VMs for the group, Compute Engine creates as many VMs as possible and continues attempting to create the remaining VMs when additional quota becomes available.

Because you are creating a regional MIG, keep in mind that certain resources are zonal, such as persistent disks. If you are specifying zonal resources in your instance template, like additional persistent disks, the disk must be present in all zones so it can be attached to the VMs created by this regional MIG.

By default, if you do not explicitly specify individual zones in your request, Compute Engine automatically chooses three zones to create VMs in. If you need to create VMs in more than or fewer than three zones, or you want to pick which zones are used, provide a list of zones in your request. For more information, see Selecting zones for your group.

By default, proactive instance redistribution is enabled. If you need to manually manage the number of VMs in each zone, you can disable proactive instance redistribution and you cannot configure autoscaling. For more information, see proactive instance redistribution.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click Create Instance Group to create a new instance group.
  3. Under Location, select Multiple zones.
  4. Choose a desired region.
  5. If you want to choose specific zones, click Configure zones to select the zones you want to use.
  6. If you want to disable proactive instance redistribution
    1. Ensure that Autoscaling mode is set to Off.
    2. Set Instance redistribution to Off.
  7. Choose an instance template for the instance group or create a new one.
  8. Specify the number of VMs for this group. Remember to provision enough VMs to support your application if a zone failure happens.
  9. Continue with the rest of the MIG creation process.

gcloud

All MIGs require an instance template. If you don't have one, create an instance template. For example, the following command creates a basic instance template with default properties:

gcloud compute instance-templates create example-template

Next, use the instance-groups managed create subcommand with the --region flag. For example, this command creates a regional managed instance group in three zones within the us-east1 region:

gcloud compute instance-groups managed create example-rmig \
    --template example-template --base-instance-name example-instances \
    --size 30 --region us-east1

If you want to select the specific zones the group should use, provide the --zones flag:

gcloud compute instance-groups managed create example-rmig \
    --template example-template --base-instance-name example-instances \
    --size 30 --zones us-east1-b,us-east1-c

If you want to disable proactive instance redistribution, set the --instance-redistribution-type flag to NONE. You cannot disable proactive instance redistribution if autoscaling is enabled.

gcloud compute instance-groups managed create example-rmig \
    --template example-template --base-instance-name example-instances \
    --size 30 --instance-redistribution-type NONE

API

All MIGs require an instance template. If you don't have one, create an instance template.

In the API, construct a POST request to the regionInstanceGroupManagers.insert method. In the request body, include the desired group name, group size, base name for instances in the group, and the URL to the instance template.

POST https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers

{
  "baseInstanceName": "base-instance-name",
  "instanceTemplate": "global/instanceTemplates/instance-template-name",
  "name": "instance-group-name",
  "targetSize": "target-size",
  "distributionPolicy": {
     "zones": [
       {"zone": "zones/zone"},
       {"zone": "zones/zone"}
      ]
   }
}

Replace the following:

  • project-id: The project ID for this request.
  • region: The region for the group.
  • base-instance-name: The instance name for each VM instance that is created as part of the group. For example, a base instance name of example-instance would create instances that have names like example-instance-[RANDOM_STRING] where [RANDOM_STRING] is generated by the server.
  • instance-template-name: The instance template to use.
  • instance-group-name: The name of the MIG.
  • target-size: The target number of VMs for the group.

If you want to select specific zones or if you are creating VMs in a region with less than or more than three zones, include the distributionPolicy property in your request and supply a list of zones. Replace [ZONE]with the name of the zone to create VMs in.

POST https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers

{
  "baseInstanceName": "base-instance-name",
  "instanceTemplate": "global/instanceTemplates/instance-template-name",
  "name": "instance-group-size",
  "targetSize": "target-size",
  "distributionPolicy": {
     "zones": [
       {"zone": "zones/zone"},
       {"zone": "zones/zone"}
      ]
   }
}

For example, the following creates a regional MIG named example-rmig with 10 managed instances distributed across us-east1-b and us-east1-c zones:

POST https://compute.googleapis.com/compute/v1/projects/myproject/regions/us-east1/instanceGroupManagers

{

  "baseInstanceName": "example-instance",
  "instanceTemplate": "global/instanceTemplates/example-instance",
  "name": "example-rmig",
  "targetSize": 10,
  "distributionPolicy": {
      "zones": [
        {"zone": "zones/us-east1-b"},
        {"zone": "zones/us-east1-c"}
      ]
   }
}

If you want to disable proactive instance redistribution, include the updatePolicy property in your request and set its instanceRedistributionType to NONE.

You cannot disable proactive instance redistribution if autoscaling is enabled.

POST https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers

{
  "baseInstanceName": "base-instance-name",
  "instanceTemplate": "global/instanceTemplates/[INSTANCE_TEMPLATE_NAME]",
  "name": "instance-group-name",
  "targetSize": "target-size",
  "updatePolicy": {
     "instanceRedistributionType": "NONE"
   },
}

Listing instances in a regional managed instance group

To get a list of managed instances for your regional MIG, use the Cloud Console, the instance-groups managed list-instances command in the gcloud command-line tool, or make a request to the regionInstanceGroupManagers.listManagedInstances method.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click the name of the regional MIG you want to view the instances of.

The instance group details page loads with a list of instances in the instance group.

gcloud

Run the instance-groups managed list-instances command:

gcloud compute instance-groups managed list-instances instance-group-name --region region

Replace the following:

  • instance-group-name: The name of the MIG.
  • region: The region of the MIG.

For example, the following command lists instances that part of a MIG named example-rmig in the region us-east1:

gcloud compute instance-groups managed list-instances example-rmig --region us-east1

API

In the API, construct an empty GET request to the regionInstanceGroupManagers.listManagedInstances method.

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/instance-group-name

For example:

GET https://compute.googleapis.com/compute/v1/projects/myproject/regions/us-east1/instanceGroupManagers/example-rmig

Updating a regional managed instance group

You can update a regional MIG using the Updater feature. The Updater allows you to update a subset of your VMs or all of your VMs within a group to a new instance template. You can also use the Updater to perform canary updates and control the speed of your update.

Relatedly, you can also change the instance template of a MIG without updating existing VMs by using the set-instance-template command in gcloud or the setInstanceTemplate method in the API. Note that changing the instance template does not automatically update existing VMs to the new instance template. You must recreate individual instances or run the Updater to apply the changes. However, new VMs to the group will use the new instance template.

Disabling and reenabling proactive instance redistribution

Proactive instance redistribution maintains an even number of VMs across the selected zones in the region. This configuration maximizes the availability of your application in the event of a zone-level failure.

Proactive instance redistribution is turned on by default for regional MIGs, but you can turn it off for non-autoscaled MIGs. When proactive instance redistribution is turned off, the group does not attempt to proactively redistribute VMs across zones. This is useful if you need to:

  • Manually delete or abandon managed instances from the group without affecting other running VMs.
  • Automatically delete a batch worker VM upon job completion without affecting other workers.
  • Protect VMs with stateful applications from unintended automatic deletion by proactive instance redistribution.

If you delete or abandon managed instances from the group and it causes an imbalance of VMs across zones, then you must manually rebalance the group before you can reenable proactive redistribution. To manually rebalance the group, resize it or delete managed instances from zones with more VMs.

When you resize a MIG that has proactive instance redistribution turned off, the group still opportunistically converges toward balance, treating each resize operation as an opportunity to balance the group: when the group grows, the group always tries to add VMs to the zones with the smallest number of VMs; when the group shrinks, the group always removes VMs from the zones with the largest number of VMs.

Example of manually resizing a group to achieve even redistribution

Use the console, the gcloud tool, or the API to create a regional MIG with proactive instance redistribution disabled, or to turn proactive instance redistribution off or on for an existing group.

Creating a group with proactive redistribution disabled

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click Create Instance Group to create a new instance group.
  3. Under Location, select Multiple zones.
  4. Choose a desired region.
  5. If you want to choose specific zones, click Configure zones to select the zones you want to use.
  6. To disable proactive instance redistribution:
    1. Ensure that Autoscaling mode is set to Off.
    2. Set Instance redistributiond to Off.
  7. Choose an instance template for the group or create a new one.
  8. Specify the number of VMs for this group. Remember to provision enough VMs to support your application if a zone failure happens.
  9. Continue with the rest of the MIG creation process.

gcloud

To create a new regional MIG without proactive instance redistribution, use the gcloud compute instance-groups managed create command with the --instance-redistribution-type flag set to NONE.

gcloud compute instance-groups managed create instance-group-name \
    --template template \
    --size target-size \
    --zones zones \
    --instance-redistribution-type NONE

Replace the following:

  • instance-group-name: The name for the MIG.
  • template: The name of the instance template to use for the group.
  • target-size: The target size of the group.
  • zones: The list of zones in a single region where you need to deploy VMs.

For example:

gcloud compute instance-groups managed create example-rmig \
    --template example-template \
    --size 30 \
    --zones us-east1-b,us-east1-c \
    --instance-redistribution-type NONE

API

To create a non-autoscaled regional MIG without proactive instance redistribution, construct a POST request to invoke the regionInstanceGroupManagers.insert method. In the request body, include the updatePolicy property and set its instanceRedistributionType field to NONE.

POST https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/instance-group-name
{
    "name": "instance-group-name",
    "baseInstanceName": "base-instance-name",
    "instanceTemplate": "global/instanceTemplates/template",
    "targetSize": "target-size",
    "distributionPolicy": {
        "zones": [
            {"zone": "zones/zone"},
            {"zone": "zones/zone"}
        ]
    },
    "updatePolicy": {
        "instanceRedistributionType": "NONE"
    }
}

Replace the following:

  • project-id. The project ID for this request.
  • region. The region for the instance group.
  • instance-group-name. The name for the MIG.
  • base-instance-name. The name prefix for each instance. The instance name suffix is auto generated.
  • template. The name of the instance template to use for the group.
  • target-size. The target size of the instance group.
  • zone. The name of a zone in the single region where you need to deploy VMs.

Turning off proactive instance redistribution

Before you can turn off proactive instance redistribution, you must turn off autoscaling.

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Select the MIG to update and click Edit group.
  3. Ensure that Autoscaling mode is set to Off.
  4. Set Instance redistribution to Off to disable automatic redistribution.
  5. Click Save.

gcloud

To turn off proactive instance redistribution for a non-autoscaled regional MIG, use the compute instance-groups managed update command with the --instance-redistribution-type flag set to NONE.

gcloud compute instance-groups managed update instance-group-name
    --instance-redistribution-type NONE 
--region region

Replace the following:

  • instance-group-name. The name of the MIG.
  • region. The region of the instance group.

API

In the API, construct a PATCH request to the regionInstanceGroupManagers.patch method. In the request body, include the updatePolicy property and set its instanceRedistributionType field to NONE

 PATCH https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/[instance-group-name

{
    "updatePolicy": {
         "instanceRedistributionType": "NONE"
    }
}

Replace the following:

  • project-id. The project ID for this request.
  • region. The region for the instance group.
  • instance-group-name. The name of a non-autoscaled MIG.

Turning on proactive instance distribution

To turn on proactive instance distribution, use a similar command as for turning off proactive instance redistribution, but set the instance redistribution type to PROACTIVE.

If you manually deleted or abandoned some managed instances resulting in an uneven distribution of VMs across the region, then, before you can reenable proactive instance redistribution, you must manually rebalance the group. The difference in the number of VMs between any two zones should not exceed 1 VM.

You can achieve an even distribution of instances across zones manually by deleting VMs from zones with more instances or by resizing the group up to fill up the zones with fewer VMs until the distribution is even.

A regional MIG does not allow turning on proactive instance redistribution when VMs are distributed unevenly across zones (the difference in the number of VMs between two zones is 2 or more VMs). This is to prevent an unintended automatic deletion of VMs from zones with more VMs, which would be triggered to achieve even distribution.

Autoscaling a regional managed instance group

Compute Engine offers autoscaling for MIGs, which allows your groups to automatically add VMs (scale out) or remove VMs (scale in) based on increases or decreases in load.

If you enable autoscaling for a regional MIG, the feature behaves as follows:

  • An autoscaling policy is applied to the group as a whole (not to individual zones). For example, if you enable autoscaler to target 66% CPU utilization, the autoscaler tracks all VMs in the group to maintain an average 66% utilization across all VMs in all zones.

  • Autoscaling attempts to evenly distribute VMs across available zones when possible. In general, the autoscaler keeps zones balanced in size by growing smaller zones and expecting that load will get redirected from bigger zones, for example, through a load balancer. We do not recommend configuring a custom load balancer that prefers one zone as this could cause unexpected behavior.

  • If your workflow uses VMs evenly in 3 zones and a zone experiences a failure, or a group of VMs within a zone fails, 1/3 of capacity might be lost but 2/3 of the capacity remains in the other zones. We recommend that you overprovision your autoscaled regional MIG to avoid overloading surviving servers during the time a zone is lost.

  • If resources (for example, preemptible VMs) are temporarily unavailable in a zone, the group continues to try to create those VMs in that zone. After the resources become available again, the group acquires the desired number of running VMs.

  • If load balanacing is enabled and if resources are unavailable in a zone causing higher utilization of existing resources in that zone, new VMs might be created in zones with lower utilization rates, which can result in a temporary uneven distribution.

The autoscaler only adds VMs to a zone up to 1/n of the specified maximum for the group, where n is the number of provisioned zones. For example, if you are using the default of 3 zones, and if 15 is the maxNumReplicas configured for autoscaling, the autoscaler can only add up to 1/3 * 15 = 5 VMs per zone for the group. If one zone fails, the autoscaler only scales up to 2/3 of the maxNumReplicas in the remaining two zones combined.

Provisioning your autoscaler configuration

Similar to the advice on overprovisioning a MIG, you should overprovision your autoscaler configuration so that:

  • The autoscaling utilization target is 2/3 of your desired utilization target.
  • To accommodate for the lowered utilization target, autoscaler will add more VMs, so you should increase the maxNumReplicas to be 50% more than the number you would have set without taking into account overprovisioning.

For example, if you expect that 20 VMs can handle your peak loads and the target utilization is 80%, set the autoscaler to:

  • 2/3 * 0.8 = 0.53 or 53% for target utilization instead of 80%
  • 3/2 * 20 = 30 for max number of VMs instead of 20

This setup ensures that in the case of a single-zone failure, your MIG should not run out of capacity because the remaining 2/3 of VMs should be able to handle the increased load from the offline zone (since you lowered the target utilization well below its capacity). The autoscaler will also add new VMs up to the maximum number of VMs you specified to maintain the 2/3 utilization target.

However, you shouldn't rely solely on overprovisioning your MIG to handle increased load. As a best practice, Google recommends that you regularly load test your applications to make sure it can handle the increased utilization that might be caused by a zonal outage removing 1/3 of VMs.

Enabling autoscaling

Console

  1. In the Cloud Console, go to the Instance groups page.

    Go to the Instance groups page

  2. If you do not have an instance group, create one. Otherwise, click the name of an existing regional MIG from the list.
  3. On the instance group details page, click the Edit Group button.
  4. Under Autoscaling, check On.
  5. Fill out the properties for the autoscaling configuration.
  6. Save your changes.

gcloud

Using the gcloud command-line tool, using the set-autoscaling subcommand to enable regional autoscaling, followed by the --region flag. For more information on creating an autoscaler, read the autoscaling documentation.

For example, the following snippets sets up an autoscaler for an example instance group. Replace us-east1 with the region of your managed instance group and example-rmig with the name of the regional managed instance group:

gcloud compute instance-groups managed set-autoscaling example-rmig \
    --target-cpu-utilization 0.8 --max-num-replicas 5 --region us-east1

API

To set up regional autoscaling in the API, call the [regionAutoscalers.insert method[(/compute/docs/reference/rest/v1/regionAutoscalers/insert)

POST https://compute.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/[REGION]/regionAutoscalers/

Your request body must contain the name, target, and autoscalingPolicy fields. autoscalingPolicy must define cpuUtilization and maxNumReplicas.

To make it easy to identify, we recommend you name the autoscaler resource after the MIG resource that uses it.

{
 "name": "<var>autoscaler-name</var>",
 "target": "regions/us-east1/instanceGroupManagers/<var>instance-group-name</var>",
 "autoscalingPolicy": {
    "maxNumReplicas": <var>max-number-of-instances</var>,
    "cpuUtilization": {
       "utilizationTarget": <var>target-utilization</var>
     },
    "coolDownPeriodSec": <var>seconds</var>
  }
}

For example:

{
 "name": "example-rmig",
 "target": "regions/us-east1/instanceGroupManagers/example-rmig",
 "autoscalingPolicy": {
    "maxNumReplicas": 10,
    "cpuUtilization": {
       "utilizationTarget": 0.8
     },
    "coolDownPeriodSec": 30
  }
}

Updating an autoscaler

You can update a regional autoscaler as you would a zonal autoscaler. Read the documentation on updating an autoscaler.

Adding a regional managed instance group to a load balancer

Google Cloud Platform load balancing uses instance groups to serve traffic. Depending on the type of load balancer you are using, you can add instance groups to a target pool or backend service. To read more about managed instance groups and load balancing, see the Instance groups overview.

You can assign a regional MIG as a target of a backend service for external load balancing and internal load balancing or as part of a target pool for Network load balancing.

For HTTP(S) load balancing, only maxRatePerInstance and maxUtilization are supported for regional MIGs.

Adding a regional managed instance group to a backend service

A backend service is necessary for creating the following types of load balancing services:

  • External HTTP(S) Load Balancing
  • Internal HTTP(S) Load Balancing
  • SSL Proxy Load Balancing
  • TCP Proxy Load Balancing
  • Internal TCP/UDP Load Balancing

A backend service can contain multiple backends. An instance group is a type of backend. The instances in the instance group respond to traffic from the load balancer. The backend service in turn knows which instances it can use, how much traffic they can handle, and how much traffic they are currently handling. In addition, the backend service monitors health checking and does not send new connections to unhealthy instances.

Use these instructions to add a managed instance group to a backend service.

Console

  1. Go to the Load balancing page in the Cloud Console.

    Go to the Load balancing page

  2. Click the name of the backend service to which you are adding the managed instance group.
  3. Click Edit.
  4. Click +Add backend.
  5. Select the instance group you want to add.
  6. Edit any optional settings you want to change.
  7. Save your changes.

gcloud

With the gcloud command-line tool, use the add-backend command:

    gcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
        --instance-group=INSTANCE_GROUP \
        [--instance-group-region=INSTANCE_GROUP_REGION | --instance-group-zone=INSTANCE_GROUP_ZONE] \
        --balancing-mode=BALANCING_MODE

Additional parameters are required depending on the balancing mode of the managed instance group. For more information, see the add-backend command in the SDK.

API

To add a backend service using the REST API, see backendServices.

Adding a regional managed instance group to a target pool

A target pool is an object that contains one or more virtual machine instances. A target pool is used in Network Load Balancing, where a network load balancer forwards user requests to the attached target pool. The instances that are part of that target pool serve these requests and return a response. You can add a managed instance group to a target pool so that when instances are added or removed from the instance group, the target pool is also automatically updated with the changes.

Before you can add a managed instance group to a target pool, the target pool must exist. For more information, see the documentation for Adding a target pool.

To add an existing managed instance group to a target pool, follow these instructions. This causes all VM instances that are part of the managed instance group to be added to the target pool.

Console

  1. Go to the Target Pools page in the Cloud Console.

    Go to the Target Pools page

  2. Click the target pool you want to add the instance group to.
  3. Click the Edit button.
  4. Scroll down to the VM instances section and click on Select instance groups.
  5. Select an instance group from the drop-down menu.
  6. Save your changes.

gcloud

With the gcloud command-line tool, use the set-target-pools command:

gcloud beta compute instance-groups managed set-target-pools [INSTANCE_GROUP] \
    --target-pools [TARGET_POOL,..] [--region REGION]

where:

  • [INSTANCE_GROUP] is the name of the instance group.
  • [TARGET_POOL] is the name of one or more target pools to add this instance group to.
  • [REGION] is the region of the instance group.

API

In the API, make a POST request to the following URI:

POST https://compute.googleapis.com/compute/beta/projects/[PROJECT_ID]/regions/[REGION]/regionInstanceGroupManagers/[INSTANCE_GROUP]/setTargetPools

where:

  • [PROJECT_ID] is the project ID for this request.
  • [REGION] is the region for the instance group.
  • [INSTANCE_GROUP] is the name of the instance group.

The request body should contain a list of URIs to the target pools you want to add this group. For example:

{
  "targetPools": [
    "regions/us-central1/targetPools/example-targetpool-1",
    "regions/us-central1/targetPools/example-targetpool-2"
  ]
}

Simulating a zone outage for a regional managed instance group

To test that your regional MIG is overprovisioned enough and can survive a zone outage, you can use the following example to simulate a single zone failure.

This script stops and starts Apache as the default scenario. If this doesn't apply to your application, replace the commands that stop and start Apache with your own failure and recovery scenario.

  1. Deploy and run this script continuously in every VM in the group. You can do this by adding the script to the instance template or by including the script in a custom image and using the image in the instance template.

    #!/usr/bin/env bash
    
    # Copyright 2016 Google Inc. All Rights Reserved.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    set -o nounset
    set -o errexit
    set -o pipefail
    
    function GetMetadata() {
      curl -s "$1" -H "Metadata-Flavor: Google"
    }
    
    PROJECT_METADATA_URL="http://metadata.google.internal/computeMetadata/v1/project/attributes"
    INSTANCE_METADATA_URL="http://metadata.google.internal/computeMetadata/v1/instance"
    ZONE=$(GetMetadata "$INSTANCE_METADATA_URL/zone" | cut -d '/' -f 4)
    INSTANCE_NAME=$(hostname)
    
    # We keep track of the state to make sure failure and recovery is triggered only once.
    STATE="healthy"
    while true; do
      if [[ "$ZONE" = "$(GetMetadata $PROJECT_METADATA_URL/failed_zone)" ]] && \
         [[ "$INSTANCE_NAME" = *"$(GetMetadata $PROJECT_METADATA_URL/failed_instance_names)"* ]]; then
        if [[ "$STATE" = "healthy" ]]; then
          STATE="failure"
          # Do something to simulate failure here.
          echo "STARTING A FAILURE"
          /etc/init.d/apache2 stop
        fi
      else
        if [[ "$STATE" = "failure" ]] ; then
          STATE="healthy"
          # Do something to recover here.
          echo "RECOVERING FROM FAILURE"
          /etc/init.d/apache2 start
        fi
      fi
      sleep 5
    done
    
    
  2. Simulate a zone failure by setting these two project metadata fields:

    • failed_zone: Sets the zone where you want to simulate the outage (limit the failure to just one zone).
    • failed_instance_names: Choose the VMs to take offline by name (to limit the failure to only VM names containing this string).

    You can set this metadata using the gcloud command-line tool. For example, the following command sets the zone outage to the europe-west1-b zone and affects VMs that have names starting with instance-base-name:

    gcloud compute project-info add-metadata --metadata failed_zone='europe-west1-b',failed_instance_names='instance-base-name-'
  3. After you are done simulating the outage, recover from the failure by removing the metadata keys:

    gcloud compute project-info remove-metadata --keys failed_zone,failed_instance_names

Here are some ideas for failure scenarios you can run using this script:

  • Stop your application completely to see how the MIG responds.
  • Make your VMs return as "unhealthy" on load balancing health checks.
  • Modify iptables to block some of the traffic to and from the VM.
  • Shutdown the VMs. By default, it will be recreated by the regional MIG shortly after but the new incarnation will immediately shutdown itself as soon as the script runs and as long as the metadata values are set. This will result in a crash loop.

What's next