Autoscaling groups of instances

Managed instance groups (MIGs) offer autoscaling capabilities that let you automatically add or delete virtual machine (VM) instances from a MIG based on increases or decreases in load. Autoscaling helps your apps gracefully handle increases in traffic and reduce costs when the need for resources is lower. You define the autoscaling policy and the autoscaler performs automatic scaling based on the measured load.

Autoscaling works by adding more VMs to your MIG when there is more load (scaling out, sometimes referred to as scaling up), and deleting VMs when the need for VMs is lowered (scaling in or down).

Specifications

  • Autoscaling only works with zonal and regional managed instance groups (MIGs). Unmanaged instance groups are not supported.
  • Autoscaling does not work with regional MIGs if proactive instance redistribution is disabled.
  • Autoscaling does not work with stateful MIGs.
  • You cannot create instances with specific names while autoscaling is turned on. However, you can turn on autoscaler after VM instances with specific names are created.
  • You cannot use autoscaling if your MIG has stateful configuration.
  • Do not use Compute Engine autoscaling with MIGs that are owned by Google Kubernetes Engine. For Google Kubernetes Engine groups, use cluster autoscaling instead.

    If you are not sure whether your group is part of a GKE cluster, look for the gke prefix in the MIG name. For example, gke-test-1-3-default-pool-eadji9ah.

  • An autoscaler can make scaling decisions based on multiple metrics, but it can handle only one policy per metric type except in the case of Cloud Monitoring metrics; an autoscaler can handle up to five policies based on Monitoring metrics. The autoscaler calculates the recommended number of virtual machines for each policy and then scales based on the policy that provides the largest number of virtual machines in the group.

  • Autoscaling works independently from autohealing. If you configure autohealing for your group and an instance fails the health check, the autohealer attempts to recreate the instance. Recreating an instance can cause the number of instances in the group to fall below the autoscaling threshold (minNumReplicas) that you specify.

  • If you autoscale a regional MIG, an instance can be added then immediately deleted from one of the zones. This happens when the utilization in the zone triggers a scale out but the overall utilization in the regional MIG does not require the additional instance or the additional instance is required in a different zone.

Fundamentals

Autoscaling uses the following fundamental concepts and services.

Managed instance groups

Autoscaling is a feature of managed instance groups (MIGs). A managed instance group is a collection of virtual machine (VM) instances that is created from a common instance template. An autoscaler adds or deletes instances from a managed instance group. Although Compute Engine has both managed and unmanaged instance groups, only managed instance groups can be used with autoscaler.

To understand the difference between a managed instance group and an unmanaged instance group, see Instance groups.

Autoscaling policy and target utilization

To create an autoscaler, specify the autoscaling policy and a target utilization level that the autoscaler uses to determine when to scale the group. You can choose to scale using the following policies:

  • Average CPU utilization.
  • HTTP load balancing serving capacity, which can be based on either utilization or requests per second.
  • Cloud Monitoring metrics.

The autoscaler continuously collects usage information based on the policy, compares actual utilization to your desired target utilization, and uses this information to determine whether the group needs to remove instances (scale in) or add instances (scale out).

The target utilization level is the level at which you want to maintain your virtual machine (VM) instances. For example, if you scale based on CPU utilization, you can set your target utilization level at 75% and the autoscaler will maintain the CPU utilization of the specified group of instances at or close to 75%. The utilization level for each metric is interpreted differently based on the autoscaling policy.

For a brief summary of each policy, see policies. For a detailed discussion of each policy, see:

Cool down period

While an instance is initializing, information about its usage might not reflect normal circumstances, so that usage information might not be reliable for autoscaler decisions and you might want to omit that data. Specify a cool down period to allow your instances to finish initializing before the autoscaler begins collecting usage information from them. By default, the cool down period is 60 seconds.

Actual initialization times vary because of numerous factors. We recommend that you test how long your application takes to initialize. To do this, create an instance and time the startup process from when the instance becomes RUNNING until the application is ready.

If you set a cool down period value that is significantly longer than the time it takes for an instance to initialize, then your autoscaler might ignore legitimate utilization data, and it might underestimate the required size of your group, causing a delay in scaling out.

Stabilization period

For the purposes of scaling in, the autoscaler calculates the group's recommended target size based on peak load over the last 10 minutes. These last 10 minutes are referred to as the stabilization period.

Using the stabilization period, the autoscaler ensures that the recommended size for your managed instance group is always sufficient to serve the peak load observed during the previous 10 minutes.

This 10-minute stabilization period might appear as a delay in scaling in, but it is actually a built-in feature of autoscaling. The delay ensures that the smaller group size is enough to support peak load from the last 10 minutes.

Autoscaling mode

If you need to investigate or configure your group without interference from autoscaler operations, you can temporarily turn off or restrict autoscaling activities. The autoscaler's configuration persists while it is turned off or restricted, and all autoscaling activities resume when you turn it on again or lift the restriction.

Scale-in controls

If your workloads take many minutes to initialize (for example, due to lengthy installation tasks), you can reduce the risk of response latency caused by abrupt scale-in events by configuring scale-in controls. Specifically, if you expect load spikes to follow soon after declines, you can limit the scale-in rate to prevent autoscaling from reducing a MIG's size by more VM instances than your workload can tolerate.

You don't have to configure scale-in controls if your application initializes quickly enough to pick up load spikes on scale out.

To configure scale-in controls, set the following properties in your autoscaling policy.

  • Maximum allowed reduction. The number of VM instances that your workload can afford to lose (from its peak size) within the specified trailing time window. Use this parameter to limit how much your group can be scaled in so that you can still serve a likely load spike until more instances start serving. The smaller you set the maximum allowed reduction, the longer it takes for your group to scale in.

  • Trailing time window. The history within which the autoscaler monitors the peak size required by your workload. The autoscaler will not resize below the maximum allowed reduction subtracted from the peak size observed in this period. You can use this parameter to define how long the autoscaler should wait before removing instances, as defined by the maximum allowed reduction. With a longer trailing time window, the autoscaler considers more historical peaks, making scale-in more conservative and stable.

For more information, see Configuring scale-in controls and Understanding autoscaler decisions.

The recommended group size is the autoscaler's recommended number of VMs that the managed instance group should maintain, based on peak load observed during the last 10 minutes. These last 10 minutes are referred to as the stabilization period. The recommended target size is recalculated constantly. If you set an autoscaling policy with scale-in controls, then the recommendedSize is constrained by your scale-in controls.

What's next

  1. If you don't have an existing MIG, review how to create a managed instance group.
  2. Create an autoscaler that scales on:

  3. Manage your autoscaler, for example, to get information about it, to configure scale-in controls, or to temporarily restrict it.