Hide
Compute Engine

Creating Groups of Instances

You can create and manage groups of virtual machine instances so that you don't have to individually control each instance in your project. Google Compute Engine offers two different types of instance groups:

  • Managed instance groups
  • Unmanaged instance groups

Use managed instance groups for most situations. Managed instance groups provide the following benefits:

  • When your applications require additional compute resources, managed instance groups can automatically scale the number of instances in the group.
  • Managed instance groups can work with load balancing services to distribute network traffic to all of the instances in the group.
  • If an instance in the group stops, crashes, or is deleted by an action other than the instance groups commands, the managed instance group automatically recreates the instance so it can resume its processing tasks. The recreated instance uses the same name and the same instance template as the previous instance, even if the group references a different instance template.
  • Managed instance groups use an instance template to define the properties for every instance in the group. You can easily update all of the instances in the group by specifying a new template in a rolling update.

If you must create a group of dissimilar instances that do not follow an instance template, see Unmanaged Instance Groups. Use unmanaged instance groups only if you need to apply load balancing to your pre-existing configurations or to groups of dissimilar instances.

Contents

Creating a managed instance group

Before you create a managed instance group, you must create an instance template to specify the operating system image and the settings for every instance in the group.

Because managed instance groups are designed to add and remove instances frequently, it is useful to create a private image and specify it in the instance template. Prepare your private image with the applications and settings that your instances need so you don't have to manually configure those items on individual instances in the managed instance group.

Alternatively, you can create an instance template that uses a public image and a startup script to prepare the instance after it starts running. Private images are more deterministic and start more quickly than instances with startup scripts. However, startup scripts are more flexible and allow you to update the applications and settings in your instances more easily.

After you create the template, create the managed instance group through the Developers Console, the gcloud compute tool, or the API.

Developers Console


  1. In the Developers Console, go to the Instance groups page.
  2. Click Create an instance group or New instance group.
  3. Enter a name for the managed instance group, and select the zone where you want to locate the group.
  4. Click Use instance template.
  5. Select an instance template. If no templates are available, create an instance template first.
  6. Specify the number of instances that you want to create in the group. Optionally, you can enable Autoscaling so that the group will automatically add or remove instances based on instance CPU usage.
  7. Click Create to create the new group.

gcloud


Create an instance group with the instance-groups managed create command:

$ gcloud compute instance-groups managed create example-group \
  --base-instance-name example \
  --size 3 \
  --template example-template

API


In the API, construct a POST request to the instanceGroupManagers service. In the request body, include the desired group name, group size, base name for instances in the group, and the URL to the instance template.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers

{
  "baseInstanceName": "example",
  "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/myproject/global/instanceTemplates/example-template",
  "name": "example-group",
  "targetSize": 3
}

After you create a managed instance group, the new instances start in the group as soon as the system can provision them. This process can take a significant amount of time depending on the number of instances in the group. Verify the status of instances in your managed instance group.

Retrieving existing groups and group descriptions

Get information about your existing managed instance groups.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups. Note that this page also lists unmanaged instance groups.
  2. Under the Name column of the list, click the name of the instance group that you want to examine. A page opens with the instance group properties and a list of instances that are included in the group.

gcloud


List all managed instance groups within a project:

$ gcloud compute instance-groups managed list

To get information about a specific group:

$ gcloud compute instance-groups managed describe example-group

API


List all managed instance groups within a project. Construct a GET request to the instanceGroupManagers service:

GET https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers

Get information about a specific group. Construct a GET request to the instanceGroupManagers service and include the name of a specific managed instance group:

GET https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group

Changing the instance template for a managed instance group

You can change the instance template for a managed instance group without applying any changes to existing instances. The managed instance group uses the new template when it adds or recreates instances, but does not change its existing instances. This allows you to control exactly which instances are updated, but it causes your instance group to contain dissimilar instances.

After you create a new instance template, change the instance template for an existing instance group.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups.
  2. Under the Name column of the list, click the name of the instance group where you want to change the instance template.
  3. Click Edit group to modify this managed instance group.
  4. Under Instance template select the new instance template that you want to use on this group.
  5. Click Save to apply the new template.

gcloud


To use the set-instance-template method to update a template, pass the new template to the instance-groups managed set-instance-template subcommand:

$ gcloud compute instance-groups managed set-instance-template example-group \
  --template example-template

API


Construct a request to the instanceGroupManagers service with the name of the target managed instance group. Include the URL of the new instance template in the request body:

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/setInstanceTemplate

{
 "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/myproject/global/instanceTemplates/example-new-template"
}

After you change the instance template, recreate individual instances or update all of the instances in the group with a rolling update.

Automatically resizing a managed instance group

You can configure managed instance groups to automatically add or remove instances based on their workloads. Your applications can gracefully handle increases in traffic, and you can reduce your costs when the need for compute resources is lower. To start scaling your managed instance groups, see Scaling Groups of Instances.

Manually resizing a managed instance group

If a managed instance group is not already set to automatically scale, you can resize a group manually to change the number of instances in the group. If you increase the size, the managed instance group uses the current instance template to add new instances. If you decrease the size, the managed instance group deletes instances from the group. The group deletes instances with a currentAction of DELETING, CREATING, and RECREATING before it deletes instances that are running with no scheduled actions.

Resize a managed instance group using the Developers Console, the gcloud compute tool, or the API.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups.
  2. Under the Name column of the list, click the name of the instance group where you want to change the group size.
  3. Click Edit group to modify this managed instance group.
  4. Under Number of instances specify the number of instances that you want to include in this managed instance group. If Autoscaling is enabled, the group is automatically adding or removing instances as necessary. However, you can change the Minimum number of instances and the Maximum number of instances values to indirectly adjust the group size through Autoscaler.
  5. Click Save to apply the new template.

gcloud


The command usage is:

$ gcloud compute instance-groups managed resize example-group --size 7

API


Construct a request to the instanceGroupManagers service with the name of the target managed instance group. Specify the new instance size as a parameter.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/resize?size=10

After you make a request to resize a managed instance group, the instances start or stop as soon as the system can provision or delete them. This process can take a significant amount of time depending on the number of instances in the group. Verify the status of instances in your managed instance group.

Deleting individual instances from a group

You can delete individual instances in a managed instance group. Deleting instances reduces the specified targetSize of the instance group and removes the instances from any target pools of which they are a member.

Deleting instances from a managed instance group does not change any specified autoscaler settings. If you delete instances from a managed instance group, the autoscaler might detect an increase in the workload on the other instances in the group and increase the group size back to its previous level. To prevent this, stop the autoscaler before attempting to delete the instances.

Delete instances from a managed instance group using the Developers Console, the gcloud compute tool, or the API.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups.
  2. Under the Name column of the list, click the name of the instance group where you want to delete individual instances. A page opens with the instance group properties and a list of instances that are included in the group.
  3. On the list of instances, select one or more instances that you want to delete.
  4. Click Delete. The selected instances are deleted.

gcloud


To delete an instance with gcloud, use the instance-groups managed delete-instances subcommand:

$ gcloud compute instance-groups managed delete-instances example-group \
  --instance example-i3n2,example-z2x9

API


Construct a request to the instanceGroupManagers service with the name of the target managed instance group. In the request body, include the URLs to one or more instances that you want to delete.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/deleteInstances

{
 "instances": [
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-i3n2",
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-l6n1"
 ]
}

After you make a request to delete instances in a managed instance group, the instances stop as soon as the system can delete them. This process can take a significant amount of time depending on the number of instances that you delete from the group. Verify the status of instances in your managed instance group.

Abandoning instances from a group

You can separate an instance from a managed instance group to more easily debug issues with individual instances without affecting the group as a whole. Abandoning an instance from a group also removes the instance from any target pools that were assigned by the managed instance group. Target pools that are assigned to specific individual instances are not removed.

Abandoning instances reduces the specified targetSize of the instance group, but does not change any specified autoscaler settings. Managed instance groups with an autoscaler continue to add or remove instances automatically as necessary.

Abandon instances from a managed instance group using the Developers Console, the gcloud compute tool, or the API.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups.
  2. Under the Name column of the list, click the name of the instance group from which you want to remove instances. A page opens with the instance group properties and a list of instances that are included in the group.
  3. On the list of instances, select one or more instances that you want to remove from the group.
  4. Click Remove from group. The selected instances leave the group, but continue to run outside of the group.

gcloud


To remove an instance from the instance group without deleting the instance, use the abandon-instances subcommand.

$ gcloud compute instance-groups managed abandon-instances example-group \
  --instance example-i3n2,example-z2x9

API


Construct a request to the instanceGroupManagers service with the name of the target managed instance group. In the request body, include the URLs to one or more instances that you want to abandon.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/abandonInstances

{
 "instances": [
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-i3n2",
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-l6n1"
 ]
}

After you make a request to abandon instances from a managed instance group, the group removes the instances as soon as possible. This process can take a significant amount of time depending on the number of instances that you abandon. Verify the status of instances in your managed instance group.

Recreating instances in the group

Update specific instances in a managed instance group. Recreating the instance deletes the specified instances and creates new instances using the instance template that is assigned to the managed instance group.

Recreate instances in managed instance group using the gcloud compute tool, or the API.

gcloud


Use the instance-groups managed recreate-instances subcommand.

$ gcloud compute instance-groups managed recreate-instances example-group \
  --instance example-i3n2,example-z2x9

API


Construct a request to the instanceGroupManagers service with the name of the target managed instance group. In the request body, include the URLs to one or more instances that you want to recreate.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/recreateInstances

{
 "instances": [
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-i3n2",
  "https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-instance-l6n1"
 ]
}

After you make a request to recreate instances in a managed instance group, the new instances start as soon as the system can provision them. This process can take a significant amount of time depending on the number of instances that you recreate. Verify the status of instances in your managed instance group.

If you need to recreate all of the instances in a managed instance group, start a rolling update.

Verify the status of instances in a managed instance group

Several commands and requests create, delete, and modify instances in a managed instance group. Those operations are are returned as DONE once the group has scheduled actions to create, delete, or update those instances. However, this does not mean that instances in the group have been created, deleted, or updated until those scheduled actions are complete. You must verify the status of those instances with the gcloud compute tool or the API.

gcloud


Use the instance-groups managed list-instances command to list the instances in the group and the current actions for those instances.

$ gcloud compute instance-groups managed list-instances example-group

NAME               STATUS  ACTION   LAST_ERROR
example-group-0gnk RUNNING NONE
example-group-15xy         CREATING Error QUOTA_EXCEEDED: Instance 'example-group-15xy' creation failed: Quota 'IN_USE_ADDRESSES' exceeded.  Limit: 23.0
example-group-18ep         CREATING Error QUOTA_EXCEEDED: Instance 'example-group-18ep' creation failed: Quota 'CPUS' exceeded.  Limit: 24.0, Error QUOTA_EXCEEDED: Instance 'example-group-18ep' creation failed: Quota 'IN_USE_ADDRESSES' exceeded.  Limit: 23.0
example-group-1u1y         CREATING

In this example, example-group contains four instances. One instances is running, two instances are being created but cannot be created due to address and CPU quotas, and one instance that is being created but has not yet encountered any errors.

Alternatively, you can use the instance-groups managed wait-until-stable command to automatically check an instance group and make your script wait until all of the instances in the group are stable.

$ gcloud compute instance-groups managed wait-until-stable example-group

API


Construct a request to the instanceGroupManagers service and specify the name of the managed instance group that contains the instances you want to verify.

POST https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group/listManagedInstances

The request gets the following response:

{
 "managedInstances": [
  {
   "instance": "https://content.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-group-0gnk",
   "id": "16960422116594945029",
   "instanceStatus": "RUNNING",
   "currentAction": "NONE"
  },
  {
   "instance": "https://content.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-group-15xy",
   "currentAction": "CREATING",
   "lastAttempt": {
    "errors": {
     "errors": [
      {
       "code": "QUOTA_EXCEEDED",
       "message": "Instance 'example-group-15xy' creation failed: Quota 'IN_USE_ADDRESSES' exceeded.  Limit: 23.0"
      }
     ]
    }
   }
  },
  {
   "instance": "https://content.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-group-18ep",
   "currentAction": "CREATING",
   "lastAttempt": {
    "errors": {
     "errors": [
      {
       "code": "QUOTA_EXCEEDED",
       "message": "Instance 'example-group-18ep' creation failed: Quota 'CPUS' exceeded.  Limit: 24.0"
      },
      {
       "code": "QUOTA_EXCEEDED",
       "message": "Instance 'example-group-18ep' creation failed: Quota 'IN_USE_ADDRESSES' exceeded.  Limit: 23.0"
      }
     ]
    }
   }
  },
  {
   "instance": "https://content.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances/example-group-1u1y",
   "id": "7239700230861444556",
   "instanceStatus": "RUNNING",
   "currentAction": "CREATING"
  }
 ]
}

Deleting a managed instance group

When you delete a managed instance group, all instances in the group and any attached autoscalers are also deleted. If you must keep any of the instances in this managed instance group, abandon the instances first to remove the instances from the group. Then, delete the managed instance group.

Delete an entire managed instance group and its instances using the Developers Console, the gcloud compute tool, or the API.

Developers Console


  1. In the Developers Console, go to the Instance groups page. If you have existing instance groups, the page lists those groups.
  2. Select one or more groups on the list that you want to delete.
  3. Click Delete to delete the group and all of the instances in the managed instance group.

gcloud


Use the instance-groups managed delete subcommand.

$ gcloud compute instance-groups managed delete example-group

API


Construct a DELETE request to the instanceGroupManagers service and specify the name of the managed instance group that you want to delete.

DELETE https://www.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instanceGroupManagers/example-group

Updating all instances in a managed instance group

You can update all of the instances in a managed instance group at once rather than updating the instances individually. Use this process to easily apply a new instance template to all of the existing instances in a managed instance group.

Sign up

  1. The rolling update feature is currently in Alpha and requires that you request access to use it.

    Request access

  2. After you have been given access to the feature, you can enable the API in the Google Developers Console.

    Enable the Updater API

Starting a rolling update

First, create a new instance template with the desired instance properties and disk image. Then, start a rolling update to apply the new template to the instances in the managed instance group. If you set a new template for the group without updating your existing instances, only the instances that you recreate or add to the group later will receive the new template.

Start a rolling update using the gcloud compute tool.

$ gcloud alpha compute rolling-updates start \
  --group example-group \
  --template example-template \
  --zone us-central1-f

You can change the default settings for rolling out your update by using the following optional flags:

  • --max-num-concurrent-instances, which defines how many instances are updated at the same time.
  • --instance-startup-timeout, which defines the maximum number of seconds that the update waits for an instance to start after the updates have been applied. If the instance does not start before the time limit, the updater records the update as a failure.
  • --min-instance-update-time, which defines the minimum number of seconds that the updater spends to update each instance. The updater starts the next update only when the current update is complete and the minimum update time is spent.
  • --max-num-failed-instances, which defines the maximum number of instance updates that can fail before the updater records the entire group update as a failure.
  • --auto-pause-after-instances, which tells the update to automatically pause after updating a specific number of instances. After the pause, you can decide whether to cancel or continue the update.

Monitoring rolling updates

View a list of rolling updates with the updates list command. The list includes rolling updates that completed up to 30 days ago.

To view a list your rolling updates:

$ gcloud alpha compute rolling-updates list \
  --group example-group \
  --zone us-central1-f

To view information about a single update, use the describe subcommand:

$ gcloud alpha compute rolling-updates describe UPDATE \
  --zone us-central1-f

The value of UPDATE must be the id of the update, returned by the initial start subcommand. You can also get the identifier by performing a rolling-updates describe or rolling-updates list request.

For example, an update id looks like the following:

$ gcloud alpha compute rolling-updates list --group example-group --zone us-central1-f
ID                                    GROUP_NAME    TEMPLATE_NAME   STATUS           STATUS_MESSAGE
a9c939b4-fcd7-a5ec-bde9-e0f92277f718  example-group template-update ROLLING_FORWARD  0/5 instances

Pausing a rolling update

You can pause a rolling update with the rolling-updates pause subcommand:

$ gcloud alpha compute rolling-updates pause UPDATE --zone us-central1-f

The value of UPDATE must be the id of the update, returned by the initial start subcommand. You can also get the identifier by performing a rolling-updates describe or rolling-updates list request.

If the group is in the middle of an update for a single instance, it will pause after it finishes that specific instance update. After you pause a rolling update, you can cancel the update, roll it back, or restart it.

Rolling back an update

You can roll back an update that is paused by using the rolling-updates rollback subcommand. This stops the update from being applied to more instances, and instances already created with the new template will be recreated with the last template applied before the current update.

To roll back an update:

$ gcloud alpha compute rolling-updates rollback UPDATE --zone us-central1-f

The value of UPDATE must be the id of the update, returned by the initial start subcommand. You can also get the identifier by performing a rolling-updates describe or rolling-updates list request.

Continuing a rolling update

If you paused a rolling update, you can continue the update using the rolling-updates resume subcommand:

$ gcloud alpha compute rolling-updates resume UPDATE --zone us-central1-f

The value of UPDATE must be the id of the update, returned by the initial start subcommand. You can also get the identifier by performing a rolling-updates describe or rolling-updates list request.

Canceling a rolling update

You can cancel a rolling update altogether using the rolling-updates cancel subcommand. This permanently stops the update and the service does not apply the template to additional instances. Instances that have already been recreated in the new template will remain in that state. You can only cancel an update that has already been paused.

For example, to cancel an update in the us-central1-a zone:

$ gcloud alpha compute rolling-updates cancel UPDATE --zone us-central1-f

The value of UPDATE must be the id of the update, returned by the initial start subcommand. You can also get the identifier by performing a rolling-updates describe or rolling-updates list request.

Rolling updates and autoscaling

A rolling update can be applied to all instance groups, whether or not they have autoscaling enabled. If the group is using autoscaling when you start a rolling update, the interaction between the Autoscaler and a rolling update is as follows:

  • If you are not using the auto-pause feature for your rolling update, the Autoscaler immediately starts to use the new template for all instances it adds.
  • If you are using the auto-pause feature, the Autoscaler continues using the previous template until the update gets automatically paused after updating the subset of instances. After you choose to update the rest of the instances, Autoscaler will start using the new template for any instances it adds. Instances that may have been added by Autoscaler using the old template will get updated to the new template.