View info about MIGs and managed instances

Use the methods on this page to view information about your managed instance groups (MIGs) and to view information about the virtual machine (VM) instances within each MIG.

Before you begin

  • Review the limitations then create a group.
  • If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    1. Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init
    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI, then initialize it by running the following command: 

      gcloud init

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

View info about MIGs

You can retrieve information about groups themselves, for example, to inspect the policies that are attached to the group and to check if the group is stable or undergoing changes. To get info about the VM instances within a MIG, see View info about managed instances.

List MIGs

To get a list of your managed instance groups, use the console, the gcloud CLI, or REST.

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups.

    If you have existing instance groups, the page lists those groups, including both managed instance groups and unmanaged instance groups.

    The Group type column specifies whether the group is managed, stateful, or unmanaged. The stateful group types are also managed instance groups.

gcloud

List all managed instance groups within a project:

gcloud compute instance-groups managed list

REST

For zonal MIGs, call the instanceGroupManagers.list method method:

GET https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers

For regional MIGs, call the regionInstanceGroupManagers.list method.

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers

View a MIG's properties

To view information about a specific MIG, including its policies (for example, the group's autohealing and update policies), use the console, the gcloud CLI, or REST.

To view the group's id value, you must use the gcloud CLI or the Compute Engine API.

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups.

    If you have existing instance groups, the page lists those groups. The page also lists unmanaged instance groups.

  2. Under the Name column of the list, click the name of the instance group that you want to examine. A page opens with the instance group properties and a list of instances that are included in the group.

  3. Click Details to see the group's configuration.

gcloud

To get information about a specific group:

gcloud compute instance-groups managed describe instance-group \
    [--zone zone | --region region]

The command returns details about the group, including attached policies.

Use the --format flag to limit the output. For example, the following command only returns the group's autoscaler configuration if it exists.

gcloud compute instance-groups managed describe instance-group --format="(autoscaler)"

REST

For zonal MIG, call the instanceGroupManagers.get method. For regional MIGs, call the regionInstanceGroupManagers.get method.

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/instance-group

The response includes details about the group, including attached policies except for the group's autoscaling policy. Although autoscaling is a feature of MIGs, it is a separate REST resource.

Autoscaling policy

If an autoscaler is attached to the group, the autoscaler resource is listed in the group's status.autoscaler field.

To get information about that autoscaler, call the autoscalers.get method for a zonal MIG or the regionAutoscalers.get method for a regional MIG. The URL of the autoscaler resource includes its name, which, by default, is the name of the group that the autoscaler is attached to. For example, you might make the following request for a regional MIG's autoscaler:

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/regionAutoscalers/mig_name

Check whether a MIG is stable

Several commands and requests create, delete, and modify instances in a managed instance group. Those operations are returned as DONE after the group has scheduled actions to create, delete, or update those instances. However, this doesn't mean that instances in the group have been created, deleted, or updated until those underlying actions are complete. You must verify the status of the group, or alternatively verify the status of those instances.

A managed instance group is stable when all instances in the managed instance group are not undergoing any change, no future changes are scheduled for the instances, and the group itself is not being modified.

You might need to know whether the group is stable when you have dependencies that rely on all instances in the group being healthy and serving. For example, you might want to start scheduling jobs, sending traffic, or assigning key ranges after you have confirmation that all the VMs are running and healthy.

Verify that all instances in a managed instance group are running and healthy by checking the value of the group's status.isStable field.

gcloud

Use the describe command:

gcloud compute instance-groups managed describe instance-group-name \
    [--zone zone | --region region]

The gcloud CLI returns detailed information about the MIG including its status.isStable field.

To pause a script until the MIG is stable, use the wait-until command with the --stable flag. For example:

gcloud compute instance-groups managed wait-until instance-group-name \
    --stable \
    [--zone zone | --region region]
Waiting for group to become stable, current operations: deleting: 4
Waiting for group to become stable, current operations: deleting: 4
...
Group is stable

The command returns after status.isStable is set to true for the MIG.

REST

For a zonal MIG, make a GET request to the instanceGroupManagers.get method:

GET https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/get

For a regional managed instance group, replace zones/zone with regions/region:

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/instance-group-name/get

These requests returns detailed information about the MIG including its status.isStable field.

status.isStable set to false indicates that changes are active, pending, or that the MIG itself is being modified.

status.isStable set to true indicates the following:

  • None of the instances in the MIG are undergoing any type of change and the currentAction for all instances is NONE.
  • No changes are pending for instances in the MIG.
  • The MIG itself is not being modified.

Remember that the stability of a MIG depends on numerous factors because a MIG can be modified in numerous ways. For example:

  • You make a request to roll out a new instance template.
  • You make a request to create, delete, resize or update instances in the MIG.
  • An autoscaler requests to resize the MIG.
  • An autohealer resource is replacing one or more unhealthy instances in the MIG.
  • In a regional MIG, some of the instances are being redistributed.

As soon as all actions are finished, status.isStable is set to true again for that MIG.

For individual VM instances, see Checking the status of managed instances.

Check whether VM repairs are turned off in a MIG

To check whether VM repairs are turned off in a MIG, use the console, the gcloud CLI, or the API.

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups

  2. Click the name of the instance group that you want to view. A page opens with the instance group properties and a list of instances that are included in the group.
  3. Click the Details tab.
  4. In the VM instance lifecycle section, check the Default action on failure field. If the value is No action, then the repairs are turned off in the MIG.

gcloud

Use the describe method as follows:

gcloud compute instance-groups managed describe instance-group \
    --format="(instanceLifecyclePolicy)"

In the response body, check for the defaultActionOnFailure field. The following are the possible values:

  • REPAIR: MIG recreates a failed or an unhealthy VM in the same zone.
  • DO_NOTHING: Repairs are turned off in the MIG. The MIG does not take any action on a failed or an unhealthy VM in the group.

The following is a sample output which indicates that the MIG repairs a VM by recreating it.

instanceLifecyclePolicy:
  defaultActionOnFailure: REPAIR
  forceUpdateOnRepair: NO

API

For a zonal MIG, use the instanceGroupManagers.get method, or, for a regional MIG, use the regionInstanceGroupManagers.get method.

To view the MIG action on VM failures in a zonal MIG, make the following call:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP

In the response body, check the instanceLifecyclePolicy.defaultActionOnFailure field. The following are the possible values:

  • REPAIR: MIG recreates a failed or an unhealthy VM in the same zone.
  • DO_NOTHING: Repairs are turned off in the MIG. The MIG does not take any action on a failed or an unhealthy VM in the group.

The following is a sample response which indicates that the MIG repairs a VM by recreating it.

"instanceLifecyclePolicy": {
  "forceUpdateOnRepair": "NO",
  "defaultActionOnFailure": "REPAIR"
}

If you don't want a MIG to repair a failed or an unhealthy VM in the group, then you can turn off repairs in the MIG.

View historical autohealing operations

If you have configured autohealing, you can view past autohealing events.

Monitor rolling updates

If you initiated a rolling update, and want to know when the update is complete, see Monitoring updates.

View stateful configuration

If you configured a stateful MIG and want to review your configuration, see Viewing stateful configuration.

View info about the managed VM instances within a MIG

List a MIG's managed instances

To get a list of the managed instances in a group, including their states and templates, use the console, the gcloud CLI, or REST.

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups.

    If you have existing instance groups, the page lists those groups. The page also lists unmanaged instance groups.

  2. Under the Name column of the list, click the name of the instance group that you want to examine. A page opens with the instance group properties and a list of instances that are included in the group.

gcloud

Use the gcloud CLI list-instances sub-command.

gcloud compute instance-groups managed list-instances instance-group \
    [--zone zone | --region region]

The command returns a list of instances and their details, including their current status, instance template, and last error.

REST

For a zonal MIG, construct a GET request to the instanceGroupManagers.listManagedInstances method and include the name of a specific managed instance group:

GET https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group

For regional (multiple-zone) managed instance groups, use the regionInstanceGroupManagers resource instead: regionInstanceGroupManagers.list.

GET https://compute.googleapis.com/compute/v1/projects/project-id/regions/region/instanceGroupManagers/instance-group

The response includes a list of instances and their details, including their current status and instance template.

Check the status of managed instances

Several commands and requests create, delete, and modify instances in a managed instance group. Those operations are returned as DONE after the group has scheduled actions to create, delete, or update those instances. However, this doesn't mean that instances in the group have been created, deleted, or updated until those underlying actions are complete. You must verify the status of those instances, or alternatively verify the status of the group itself.

Use the Google Cloud CLI or REST to see details about the instances in a managed instance group. Details include instance status and current actions that the group is performing on its instances.

gcloud

All managed instances

To check the status and current actions on all instances in the group, use the list-instances command.

gcloud compute instance-groups managed list-instances INSTANCE_GROUP_NAME \
    [--zone=ZONE | --region=REGION]

The command returns a list of instances in the group, including their status, current actions, and other details:

NAME               ZONE           STATUS   HEALTH_STATE  ACTION  INSTANCE_TEMPLATE  VERSION_NAME  LAST_ERROR
vm-instances-9pk4  us-central1-f                          CREATING  my-new-template
vm-instances-h2r1  us-central1-f  STOPPING                DELETING  my-old-template
vm-instances-j1h8  us-central1-f  RUNNING                 NONE      my-old-template
vm-instances-ngod  us-central1-f  RUNNING                 NONE      my-old-template

The HEALTH_STATE column appears empty unless you have set up health checking.

A specific managed instance

To check the status and current action for a specific instance in the group, use the describe-instance command.

gcloud compute instance-groups managed describe-instance INSTANCE_GROUP_NAME \
    --instance INSTANCE_NAME \
    [--zone=ZONE | --region=REGION]

The command returns details about the instance, including instance status, current action, and, for stateful MIGs, preserved state:

currentAction: NONE
id: '6789072894767812345'
instance: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instances/example-mig-hz41
instanceStatus: RUNNING
name: example-mig-hz41
preservedStateFromConfig:
  metadata:
    example-key: example-value
preservedStateFromPolicy:
  disks:
    persistent-disk-0:
      autoDelete: NEVER
      mode: READ_WRITE
      source: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/disks/example-mig-hz41
version:
  instanceTemplate: https://www.googleapis.com/compute/v1/projects/example-project/global/instanceTemplates/example-template

REST

Call the listManagedInstances method on a regional or zonal MIG resource. For example, to see details about the instances in a zonal MIG resource, you can make the following request:

GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/listManagedInstances

The call returns a list of instances for the MIG including each instance's instanceStatus and currentAction.

{
 "managedInstances": [
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-f/instances/vm-instances-prvp",
   "id": "5317605642920955957",
   "instanceStatus": "RUNNING",
   "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/example-project/global/instanceTemplates/example-template",
   "currentAction": "REFRESHING"
  },
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-f/instances/vm-instances-pz5j",
   "currentAction": "DELETING"
  },
  {
   "instance": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-f/instances/vm-instances-w2t5",
   "id": "2800161036826218547",
   "instanceStatus": "RUNNING",
   "instanceTemplate": "https://www.googleapis.com/compute/v1/projects/example-project/global/instanceTemplates/example-template",
   "currentAction": "REFRESHING"
  }
 ]
}

To see a list of valid instanceStatus field values, see VM instance lifecycle.

If an instance is undergoing some type of change, the managed instance group sets the instance's currentAction field to one of the following actions to help you track the progress of the change. Otherwise, the currentAction field is set to NONE.

Possible currentAction values are:

  • ABANDONING. The instance is being removed from the MIG.
  • CREATING. The instance is in the process of being created.
  • CREATING_WITHOUT_RETRIES. The instance is being created without retries; if the instance isn't created on the first try, the MIG doesn't try to replace the instance again.
  • DELETING. The instance is in the process of being deleted.
  • RECREATING. The instance is being replaced.
  • REFRESHING. The instance is being removed from its current target pools and being readded to the list of current target pools (this list might be the same or different from existing target pools).
  • RESTARTING. The instance is in the process of being restarted using the stop and start methods.
  • RESUMING. The instance is in the process of being resumed after being suspended.
  • STARTING. The instance is in the process of being started after being stopped.
  • STOPPING. The instance is being stopped.
  • SUSPENDING. The instance is being suspended.
  • VERIFYING. The instance has been created and is in the process of being verified.
  • NONE. No actions are being performed on the instance.

To check if the group as a whole is stable or is undergoing changes, see Checking the status of the group.

Check health states

If you have configured autohealing for your managed instance group, you can review the health state of each VM. See Checking health status.

List instance errors

If your managed instances encountered errors, you can view those errors to diagnose and mitigate the cause.

Use the Google Cloud console, the gcloud CLI, or REST to list errors.

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups

  2. Under the Name column of the list, click the name of the instance group that you want to examine. A page opens with the instance group properties and a list of instances that are included in the group.

  3. Click Errors.

gcloud

Use the instance-groups managed list-errors command to list the most recent errors in the group

gcloud compute instance-groups managed list-errors instance-group-name \
    [--zone zone | --region region]

For example:

gcloud compute instance-groups managed list-errors example-group \
    --region us-west1
 
INSTANCE_URL                                                                                        ACTION    ERROR_CODE      ERROR_MESSAGE                                                                                     TIMESTAMP                      INSTANCE_TEMPLATE                                                                                                            VERSION_NAME

https://www.googleapis.com/compute/v1/projects/my-project/zones/us-central1-a/instances/igm-1-v0hl  CREATING  QUOTA_EXCEEDED  Instance 'igm-1-v0hl' creation failed: Quota 'CPUS' exceeded.  Limit: 1.0 in zone us-central1-a.  2019-07-11T07:44:40.156-07:00  https://www.googleapis.com/compute/v1/projects/my-project/global/instanceTemplates/instance-template-1
https://www.googleapis.com/compute/v1/projects/my-project/zones/us-central1-a/instances/igm-1-qp3d  CREATING  QUOTA_EXCEEDED  Instance 'igm-1-qp3d' creation failed: Quota 'CPUS' exceeded.  Limit: 1.0 in zone us-central1-a.  2019-07-11T07:44:40.081-07:00  https://www.googleapis.com/compute/v1/projects/my-project/global/instanceTemplates/instance-template-1
...

REST

Construct a GET request to the listErrors method for a zonal or regional managed instance group.

For example:

GET https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/listErrors&maxResults=100

The request gets a response similar to:


{
 "items": [
  {
   "error": {
    "code": "QUOTA_EXCEEDED",
    "message": "Instance 'example-mig-m1sz' creation failed: Quota 'CPUS' exceeded.  Limit: 50.0 in zone us-central1-c."
   },
   "timestamp": "2019-01-03T18:47:28.457-08:00",
   "instanceDetails" : {
    "instance": "zones/us-central1-c/instances/example-mig-m1sz",
    "action": "CREATING",
    "version": {
     "name": "example-version-name",
     "instanceTemplate": "global/instanceTemplates/example-template",
    }
   }
  }
 ],
 ...
 "nextPageToken": "Ch0yMDE5LTA3LTI5VDExOjA4OjA2LjYxMi0wNzowMBJ5aHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vY29tcHV0ZS9zdGFnaW5nX2FscGhhL3Byb2plY3RzL215cnpha2VyZXltcy1zdGFnaW5nL3pvbmVzL3VzLWNlbnRyYWwxLWlyMS9pbnN0YW5jZXMvaWdtLTEtcXAzZA=="
}

The listErrors API supports list pagination. To get the next page of results, use the nextPageToken field from one response in a subsequent request. For example: GET https://compute.googleapis.com/compute/v1/projects/project-id/zones/zone/instanceGroupManagers/instance-group-name/listErrors&maxResults=100&nextPageToken=next-page-token, replacing next-page-token with the token that was obtained from the previous response.

The list of possible errors includes, but is not limited to, the following examples:

  • A new instance cannot be created due to:
    • Unavailability of preemptible capacity.
    • Lack of quota, for example, for CPUs in a given region.
    • The limit on the number of VM instances was reached for the network associated with the group.
    • The disk image specified in the instance template's properties.disks[].initializeParams.sourceImage field was deleted, or its permissions have changed.
    • The credit card on the billing account associated with the project refused a payment.
  • An existing instance cannot be deleted because:
    • The service account that group uses doesn't have compute.instances.delete permission.
    • Too many concurrent deletions were requested, exceeding rate limits.
  • An instance cannot be added to a target pool because the target pool was deleted.

After inspecting such errors, you can mitigate them. For example:

  • Move your workload to a different zone that has preemptible capacity.
  • Request a quota increase in a given region.
  • Add the compute.instances.delete permission to the service account used by the group.

If an action that is triggered by a managed instance group fails (for example, if an autoscaler encounters an error when attempting to increase the group's size), the group might immediately retry that action. If the last immediate retry attempt fails, the MIG records the error, and you can view the error by listing instance errors. After some backoff time, the group again retries the failed action. If the error persists, it might be recorded multiple times.

Compute Engine retains the most recent 500 errors for the last 8 days even if the underlying instances no longer exist or could not be created.

Check if a VM instance is part of a MIG

To check if a specific VM is a member of a group, you can use the console or REST.

Console

  1. In the Google Cloud console, go to the VM instances page.

    Go to VM instances

  2. Click an instance to access its VM instance details.
  3. If the VM instance is a member of a managed instance group, the name of that managed instance group appears under the heading In use by. If the VM instance is not a member of a group, the heading In use by doesn't appear on the instance details page.

REST

See Viewing referrers to VM instances.

Alternatively, to identify whether a VM is or was a member of a managed instance group, look for these two metadata keys in the instance's metadata:

  • instance-template indicates the template the VM was created from.
  • created-by indicates the managed instance group that created the VM.

Even if you abandon the instance, the VM will still have these metadata entries, unless you manually remove them.

For example, if there was an VM named random-instance-biy and you wanted to know whether the VM was created by a managed instance group, you can describe the instance and look for the metadata keys. For example:

gcloud compute instances describe random-instance-biy \
    --zone us-central1-f

gcloud CLI returns a response similar to:

canIpForward: false
cpuPlatform: Intel Ivy Bridge
creationTimestamp: '2016-08-24T14:11:38.012-07:00'
...
metadata:
  items:
  - key: instance-template
    value: projects/123456789012/global/instanceTemplates/example-it
  - key: created-by
    value: projects/123456789012/zones/us-central1-f/instanceGroupManagers/igm-metadata

View the preserved state of a managed instance

If you configured a stateful MIG and want to view the preserved state of each managed instance, see Viewing stateful configuration and preserved state.

What's next