Autoscaling node groups

This page describes how to use the autoscaler for sole-tenant node groups.

If you use sole-tenant nodes for your workloads, you can use the node group autoscaler to automatically manage the sizes of your node groups. Configure autoscaling on a node group while creating the node group or after creating the node group.

Before using the node group autoscaler, read the overview of the node group autoscaler.

Before you begin

Enabling the node group autoscaler

Configure autoscaling on a new or existing node group.

gcloud

The example below shows how to use the node-groups create command to enable the autoscaler when you are creating a node group. To add an autoscaler to an existing node group, use the node-groups update command.

gcloud beta compute sole-tenancy node-groups create group-name --node-template template-name --target-size size --maintenance-policy maintenance-policy --zone zone --autoscaler-mode mode --max-nodes max-nodes --min-nodes min-nodes

Replace the following:

  • group-name: Name of the node group to create.
  • template-name: Name of the node template from which to create the node group.
  • size: Target initial number of nodes in the node group.
  • maintenance-policy: Specifies if VMs migrate and if they are restarted during host maintenance events. Set it to one of the following values:
    • default: VMs live migrate to a new node.
    • migrate-within-node-group: VMs live migrate to another node in the node group.
    • restart-in-place: VMs restart on the same node after they are terminated due to a maintenance event.
  • zone: Zone in which to create the node group.
  • mode: Mode for the autoscaler on this node group. Set to one of the following values:
    • off: Disables the autoscaler.
    • on: Enables scaling in and scaling out.
    • only-scale-out: Enables only scaling out. You must use this mode if your node groups are configured to restart their hosted VMs on minimal servers.
  • max-nodes: Maximum size of the node group. Set to a value less than or equal to 100 and greater than or equal to min-nodes.
  • min-nodes: Minimum size of the node group, and must be an integer value less than or equal to max-nodes. The default value is 0.

API

The example below shows how to use the nodeGroups.insert command to enable the autoscaler when you are creating a node group. To add an autoscaler to an existing node group, use the nodeGroups.patch command.

POST https://www.googleapis.com/compute/beta/projects/project-id/zones/zone/nodeGroups?initialNodeCount=initial-node-count

  {
    "name": "group-name",
    "nodeTemplate": "template-name",
    "autoscalingPolicy": {
      "mode": "mode",
      "min_nodes": min-nodes,
      "max_nodes": max-nodes
    }
  }

Replace the following:

  • project-id: ID of the project for which to add a node group with an autoscaler.
  • zone: Zone in which to create the new node group.
  • initial-node-count: Required when creating the node group. This specifies the initial number of nodes in the node group. If the value for min-nodes is greater than the initial node count, the size of the node group is scaled out to the value of min-nodes.
  • group-name: Name of the new node group.
  • template-name: Name of the node template from which to create the node group.
  • mode: Mode for the autoscaler on this node group. Set to one of the following:
    • OFF: Disables the autoscaler.
    • ON: Enables scaling in and scaling out.
    • ONLY_SCALE_OUT: Enables only scaling out. You must use this mode if your node groups are configured to restart their hosted VMs on minimal servers.
  • max-nodes: Maximum size of the node group. Set to a value less than or equal to 100 and greater than or equal to min-nodes.
  • min-nodes: Minimum size of the node group, and must be an integer value less than or equal to max-nodes. The default value is 0.

Updating autoscaler settings

Change the autoscaler settings on a node group by updating the autoscaler mode or by updating the minimum and maximum size of the node group.

gcloud

The example below shows how to use the node-groups update command to change the mode of the autoscaler on a node group.

gcloud beta compute sole-tenancy node-groups update name --autoscaler-mode mode --max-nodes max-nodes --min-nodes min-nodes

Replace the following:

  • name: Name of the node group on which to change the autoscaler mode.
  • mode: Mode for the autoscaler on this node group. Set to one of the following:
    • off: Disables the autoscaler.
    • on: Enables scaling in and scaling out.
    • only-scale-out: Enables only scaling out. You must use this mode if your node groups are configured to restart their hosted VMs on minimal servers.
  • max-nodes: Maximum size of the node group. Set to a value less than or equal to 100 and greater than or equal to min-nodes.
  • min-nodes: Minimum size of the node group, and must be an integer value less than or equal to max-nodes. The default value is 0.

API

The example below shows how to use the nodeGroups.patch command to change the mode of an autoscaler on a node group.

PATCH https://www.googleapis.com/compute/beta/projects/project-id/zones/group-zone/nodeGroups/group-name

{
  "nodeTemplate": "template-name",
  "autoscalingPolicy": {
    "mode": "mode",
    "minSize": min-nodes,
    "maxSize": max-nodes
  }
}

Replace the following:

  • project-id: ID of the project containing the node group for which to change the autoscaler mode.
  • group-zone: Zone containing the node group for which to change the autoscaler mode.
  • group-name: Name of the node group for which to change the autoscaler mode.
  • template-name: Name of the node template from which the node group was created.
  • mode: Mode for the autoscaler on this node group. Set to one of the following:
    • OFF: Disables the autoscaler.
    • ON: Enables scaling in and scaling out.
    • ONLY_SCALE_OUT: Enables only scaling out. You must use this mode if your node groups are configured to restart their hosted VMs on minimal servers.
  • max-nodes: Maximum size of the node group. Set to a value less than or equal to 100 and greater than or equal to min-nodes.
  • min-nodes: Minimum size of the node group, and must be an integer value less than or equal to max-nodes. The default value is 0.

Manually updating the size of autoscaled node groups

When the autoscaler is enabled, node group size is managed automatically, but you can schedule or remove VMs on that node group to effectively manually update the group size.

To manually decrease the size of an autoscaled node group, delete VMs from the node until the node is empty. When the node is empty, the autoscaler removes the empty node, which decreases the size of the node group.

To manually increase the size of an autoscaled node group, set the minimum size of the group to a value greater than the current size. When the minimum size of a group is set to a value greater than the current size, the autoscaler scales out the group size to the newly specified minimum size.

When node groups are set to only scale out, the autoscaler automatically manages increases in group size and disables manual group size increases. With this setting, you can decrease the size of a group by removing VMs from a node until that node is empty, and then you can remove the empty node.

Disabling the autoscaler

When you no longer need to use the autoscaler to automatically manage the sizes of your node groups, or if you need to manually manage the sizes of your node groups, disable the autoscaler.

gcloud

gcloud beta compute sole-tenancy node-groups update name --autoscaler-mode OFF

Replace name with the name of the node group from which to remove the autoscaling policy.

API

The example below shows how to use the nodeGroups.patch command to turn off an autoscaler on a node group.

PATCH https://www.googleapis.com/compute/beta/projects/project-id/zones/group-zone/nodeGroups/group-name

{
  "nodeTemplate": "template-name",
  "autoscalingPolicy": {
    "mode": "mode"
  }
}

Replace the following:

  • project-id: ID of the project containing the node group for which to change the autoscaler mode.
  • group-zone: Zone containing the node group for which to change the autoscaler mode.
  • group-name: Name of the node group for which to change the autoscaler mode.
  • template-name: Name of the node template from which the node group was created.
  • mode: Mode for the autoscaler on this node group. Set to OFF to disable the autoscaler on this node group.

Viewing autoscaler activity

In the Google Cloud Console, view the autoscaler adjusting the sizes of your node groups. The Cloud Console shows the current size of the node group, and if the autoscaler is currently adjusting the size of group, you can also see the target size of the node group.

Console

  1. In the Cloud Console, go to the Sole-tenant nodes page.

    Go to the Sole-tenant nodes page

  2. Click on Node groups.

  3. View the number of nodes in each node group, and if Compute Engine is scaling the node group, you can also view the target number of nodes.

Resolving autoscaler errors

Autoscaling might fail if you have no remaining CPU quota in the region, the number of nodes in a group is at the maximum number allowed, or there was a billing issue.

Depending on the error, you might need to request an increase in CPU quota or create a new sole-tenant node group.

What's next