View, cancel, or delete resize requests in a MIG


This document describes how to do the following after creating a resize request in a managed instance group (MIG):

After you create resize requests in a MIG, you can do one or more of the following:

  • View resize requests to monitor their states or troubleshoot them.

  • Cancel resize requests to stop the MIG from creating the requested number of virtual machine (VM) instances.

  • Delete resize requests when you no longer need them.

Before you begin

  • If you haven't already, then see how resize requests work.
  • If you haven't already, then set up authentication. Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

    Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    1. Install the Google Cloud CLI, then initialize it by running the following command:

      gcloud init
    2. Set a default region and zone.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI, then initialize it by running the following command:

      gcloud init

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Required roles

To get the permissions that you need to view, cancel, or delete resize requests in a MIG, ask your administrator to grant you the Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1) IAM role on the project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to view, cancel, or delete resize requests in a MIG. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to view, cancel, or delete resize requests in a MIG:

  • To cancel or delete resize requests in a MIG: compute.instanceGroupManagers.update
  • To view a list of resize requests in a MIG: compute.instanceGroupManagers.list
  • To view the details of a resize request: compute.instanceGroupManagers.get

You might also be able to get these permissions with custom roles or other predefined roles.

View your resize requests

To view information about the resize requests in a MIG, use one of the following methods:

View a list of resize requests in a MIG

To view a list of all resize requests in a MIG, select one of the following options:

To view a list of all resize requests in a regional MIG, use the gcloud CLI or REST API. Otherwise, to view a list of all resize requests in a zonal MIG, select any of the following options:

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups

  2. In the Name column, click the name of the MIG that contains resize requests.

    The overview page of the MIG opens.

  3. In the Resize requests row, click Edit resize requests.

    The Resize requests pane appears. If a resize request's Details column shows the Quota exceeded or ETA: indefinite value, then your request has encountered an error. You can click these values to learn more.

    For more information about troubleshooting errors, see View the details of a resize request in this document.

gcloud

Replace the following:

  • INSTANCE_GROUP_NAME: the name of a MIG with accepted, successful, canceled, or failed resize requests.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

The output is similar to the following for a zonal MIG:

NAME: rr-01
LOCATION: us-central1-a
SCOPE: zone
RESIZE_BY: 5
STATE: SUCCEEDED
REQUESTED_RUN_DURATION: P1D

NAME: rr-02
LOCATION: us-central1-a
SCOPE: zone
RESIZE_BY: 10
STATE: ACCEPTED
REQUESTED_RUN_DURATION: P3D

REST

  • To view a list of resize requests in a zonal MIG, send a GET request using the instanceGroupManagerResizeRequests.list method.

    GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests
    
  • To view a list of resize requests in a regional MIG, send a GET request using the beta.regionInstanceGroupManagerResizeRequests.list method.

    GET https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/regions/REGION/regionInstanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests
    

Replace the following:

  • PROJECT_ID: the ID of the project where a MIG with accepted, successful, canceled, or failed resize requests is located.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

  • INSTANCE_GROUP_NAME: the name of an existing MIG with accepted, successful, canceled, or failed resize requests.

The output from a zonal MIG is similar to the following:

{
  "kind": "compute#instanceGroupManagerResizeRequestList",
  "id": "projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests",
  "items": [
    {
      "kind": "compute#instanceGroupManagerResizeRequest",
      "id": "4247139565532196982",
      "creationTimestamp": "2024-01-08T07:51:53.034-08:00",
      "name": "rr-01",
      "zone": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a",
      "count": 5,
      "resizeBy": 5,
      "requestedRunDuration": {
        "seconds": "86400",
        "nanos": 0
      },
      "state": "SUCCEEDED",
      "status": {},
      "selfLink": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/rr-01",
      "selfLinkWithId": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/4247139565532196982"
    },
    {
      "kind": "compute#instanceGroupManagerResizeRequest",
      "id": "8095866098849525652",
      "creationTimestamp": "2024-01-11T08:04:11.851-08:00",
      "name": "rr-02",
      "zone": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a",
      "count": 10,
      "resizeBy": 10,
      "requestedRunDuration": {
        "seconds": "259200",
        "nanos": 0
      },
      "state": "ACCEPTED",
      "status": {},
      "selfLink": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/rr-02",
      "selfLinkWithId": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/8095866098849525652"
    }
  ],
  "selfLink": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests"
}

View the details of a resize request

You can view the details of a resize request to review its configuration and, optionally, troubleshoot it if the request hasn't succeeded yet.

If an accepted resize request isn't succeeding, then you can troubleshoot it by checking the status.lastAttempt.error.errors.code field in the request's details. The possible error codes are as follows:

  • QUOTA_EXCEEDED: your project lacks quota for the requested resources. To increase your project's quota, see Request a higher quota.

  • ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: the requested resources are temporarily unavailable. Compute Engine has scheduled the creation of the requested resources and plans to create them when they become available. If the message field contains Expected time is indefinite, then Google recommends canceling the request and trying one or more of the following:

    • Create a new resize request with a lower number of requested VMs.

    • Use a different machine type in the MIG, and then create a new resize request. To use a different machine type, create a new instance template, and then use that template to create or update a MIG.

    • Create a resize request in a MIG located in a different region or zone.

To view the details of a resize request, select one of the following options:

gcloud

Replace the following:

  • INSTANCE_GROUP_NAME: the name of a MIG.

  • RESIZE_REQUEST_NAME: the name of a resize request that you want to view the details of.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

The output is similar to the following for a zonal MIG:

creationTimestamp: '2024-09-23T02:27:09.575-07:00'
id: '6386622402379156098'
kind: compute#instanceGroupManagerResizeRequest
name: example-request
requestedRunDuration:
  nanos: 0
  seconds: '86400'
resizeBy: 10
selfLink: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/example-request
selfLinkWithId: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/6386622402379156098
state: ACCEPTED
status:
  lastAttempt:
    error:
      errors:
      - code: QUOTA_EXCEEDED
        message: Quota 'NVIDIA_A100_GPUS' exceeded. Limit: 1500 in region us-central1.
      - code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
        message: There are currently not enough resources available to fulfill the request. Expected time is indefinite.
        - errorDetails:
          - errorInfo:
            - metadatas:
              - estimatedAvailabilityTime: '9999-12-31T23:59:59.999999999Z'
zone: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a

REST

  • To view the details of a resize request in a zonal MIG, send a GET request using the instanceGroupManagerResizeRequests.get method.

    GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME
    
  • To view the details of a resize request in a regional MIG, send a GET request using the beta.regionInstanceGroupManagerResizeRequests.get method.

    GET https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/regions/REGION/regionInstanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME
    

Replace the following:

  • PROJECT_ID: the ID of the project where the zonal MIG is located.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

  • INSTANCE_GROUP_NAME: the name of a MIG where the resize request is located.

  • RESIZE_REQUEST_NAME: the name of an existing resize request that you want to view the details of.

The output is similar to the following:

{
  "kind": "compute#instanceGroupManagerResizeRequest",
  "id": "6386622402379156098",
  "creationTimestamp": "2024-09-23T02:27:09.575-07:00",
  "name": "example-request",
  "zone": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a",
  "resizeBy": 10,
  "requestedRunDuration": {
    "seconds": "86400",
    "nanos": 0
  },
  "state": "SUCCEEDED",
  "status": {
    "lastAttempt": {
      "error": {
        "errors": [
          {
            "code": "QUOTA_EXCEEDED",
            "message": "Quota 'NVIDIA_A100_GPUS' exceeded. Limit: 1500 in region us-central1."
          },
          {
            "code": "ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS",
            "message": "There are currently not enough resources available to fulfill the request. Expected time is indefinite.",
            "errorDetails": [
              {
                "errorInfo":{
                  "metadatas":{
                    "estimatedAvailabilityTime": "9999-12-31T23:59:59.999999999Z"
                  }
                }
              }
            ]
          }
        ]
      }
    }
  },
  "selfLink": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/example-request",
  "selfLinkWithId": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-central1-a/instanceGroupManagers/example-mig/resizeRequests/6386622402379156098"
}

Cancel resize requests in a MIG

You can cancel resize requests in a MIG to stop the MIG from attempting to create the requested number of VMs. You can cancel accepted (ACCEPTED) resize requests only. After canceling a resize request, you can either delete it or let Compute Engine automatically delete it after 14 days.

To cancel multiple resize requests simultaneously, use the Google Cloud console or Google Cloud CLI. Otherwise, to cancel a single resize request, select any of the following options:

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups

  2. In the Name column, click the name of the MIG that contains resize requests.

    The overview page of the MIG opens.

  3. In the Resize requests row, click Edit resize requests.

    The Resize requests pane appears.

  4. Select the resize requests to cancel.

  5. Click Cancel, and then click Confirm.

gcloud

Replace the following:

  • INSTANCE_GROUP_NAME: the name of a MIG with accepted resize requests.

  • RESIZE_REQUEST_NAMES: a comma-separated list of names of accepted resize requests in the specified zonal MIG. For example, specify request-1,request-2.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

REST

  • To cancel a resize request in a zonal MIG, send a POST request using the instanceGroupManagerResizeRequests.cancel method.

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME/cancel
    
  • To cancel a resize request in a regional MIG, send a POST request using the beta.regionInstanceGroupManagerResizeRequests.cancel method.

    POST https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/regions/REGION/regionInstanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME/cancel
    

Replace the following:

  • PROJECT_ID: the ID of the project where an existing zonal MIG with an accepted resize request is located.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

  • INSTANCE_GROUP_NAME: the name of the MIG.

  • RESIZE_REQUEST_NAME: the name of the resize request to cancel.

Delete resize requests in a MIG

By default, Compute Engine automatically deletes a resize request 14 days after setting the request's state to one of the following:

  • Successful (SUCCEEDED)

  • Failed (FAILED)

  • Canceled (CANCELLED)

However, you can immediately delete a resize request before then, as described in this section.

Deleting a successful resize request doesn't delete the VMs created through the request. The MIG automatically deletes those VMs at the end of their requested run duration. However, if your job has finished running and you no longer need the VMs, then delete the VMs.

To delete multiple resize requests simultaneously, use the Google Cloud console or gcloud CLI. Otherwise, to delete a single resize request, select any of the following options:

Console

  1. In the Google Cloud console, go to the Instance groups page.

    Go to Instance groups

  2. In the Name column, click the name of the MIG that contains resize requests.

    The overview page of the MIG opens.

  3. In the Resize requests row, click Edit resize requests.

    The Resize requests pane appears.

  4. Select the resize requests to delete.

  5. Click Delete, and then click Confirm.

gcloud

Replace the following:

  • INSTANCE_GROUP_NAME: the name of a MIG with successful, failed, or canceled resize requests.

  • RESIZE_REQUEST_NAMES: a comma-separated list of names of resize requests to delete in the MIG. For example, specify request-1,request-2.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

REST

  • To delete a resize request in a zonal MIG, send a DELETE request using the instanceGroupManagerResizeRequests.delete method.

    DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME
    
  • To delete a resize request in a regional MIG, send a DELETE request using the beta.regionInstanceGroupManagerResizeRequests.delete method.

    DELETE https://compute.googleapis.com/compute/beta/projects/PROJECT_ID/regions/REGION/regionInstanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests/RESIZE_REQUEST_NAME
    

Replace the following:

  • PROJECT_ID: the ID of the project where an existing zonal MIG with successful, failed, or canceled resize requests is located.

  • ZONE: the zone where the MIG is located.

  • REGION: the region where the MIG is located.

  • INSTANCE_GROUP_NAME: the name of the MIG.

  • RESIZE_REQUEST_NAME: the name of the resize request to delete.

What's next