Automatic node resizing

In Google Distributed Cloud, you can enable automatic node resizing for a cluster. Resizing means that the CPU and memory resources assigned to a node are adjusted.

If automatic node resizing is enabled for an admin cluster, the add-on nodes in the admin cluster are resized according to the total number of nodes in the admin cluster. For each user cluster, there are one or more control-plane nodes in the admin cluster. So as you create more user clusters, the total number of nodes in the admin cluster increases, and the add-on nodes in the admin cluster are automatically increased in size.

If automatic node resizing is enabled for a user cluster the control-plane nodes for the user cluster are resized according to the number of nodes in the user cluster. So as you add more worker nodes to the user cluster, the control-plane nodes are increased in size. Recall that the control-plane nodes for the user cluster are in the admin cluster.

A highly available (HA) user cluster, user cluster has three control-plane nodes in the admin cluster, and a non-HA user cluster has one control-plane node in the admin cluster. Resizing a node requires that the node be re-created. So when the control-plane node for a non-HA user cluster is resized, there might be a small amount of downtime.

Enable node resizing for the add-on nodes of an admin cluster

In your admin cluster configuration file, set addonNode.autoResize.enabled to true:

addonNode:
  autoResize:
    enabled: true

For a new admin cluster, complete the instructions in Create an admin cluster (quickstart) or Creating an admin cluster.

For an existing admin cluster:

gkectl update admin --config ADMIN_CLUSTER_CONFIG --kubeconfig ADMIN_CLUSTER_KUBECONFIG

Replace the following:

  • ADMIN_CLUSTER_CONFIG: the path of your admin cluster configuration file

  • ADMIN_CLUSTER_KUBECONFIG: the path of your admin cluster kubeconfig file

Enable node resizing for the control-plane nodes of a user cluster

In your user cluster configuration file, set masterNode.autoResize.enabled to true:

masterNode:
  autoResize:
    enabled: true

For a new user cluster, complete the instructions in Create a user cluster (quickstart) or Creating a user cluster.

For an existing cluster:

gkectl update cluster --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config USER_CLUSTER_CONFIG

Replace the following:

  • ADMIN_CLUSTER_KUBECONFIG: the path of your admin cluster kubeconfig file

  • USER_CLUSTER_CONFIG: the path of your user cluster configuration file

After automatic node resizing is enabled for a user cluster, any attempt to change the CPU or memory of a control-plane node by running gkectl update is rejected. If you want to use gkectl update to change the CPU or memory of a user cluster control-plane node, you must disable automatic node resizing.

Disable node resizing for the add-on nodes of an admin cluster

In your admin cluster configuration file, set addonNode.autoResize.enabled to false:

addonNode:
  autoResize:
    enabled: false

Update the cluster:

gkectl update admin --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config ADMIN_CLUSTER_CONFIG

Disable node resizing for the control-plane nodes of a user cluster

In your user cluster configuration file, set masterNode.autoResize.enabled to false:

masterNode:
  autoResize:
    enabled: false

Update the cluster:

gkectl update cluster --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config USER_CLUSTER_CONFIG

When you disable automatic node resizing for a user cluster, the CPU and memory for the control-plane nodes are reconciled to match the CPU and memory values specified in the user cluster configuration file.

Logs and events

To find the name of the auto resizing Pod:

kubectl --kubeconfig ADMIN_KUBECONFIG describe deployment auto-resize-controller

The name of the Pod has the form auto-resize-controller-xxxxx.

To see the logs from the auto resizing Pod:

kubectl --kubeconfig ADMIN_KUBECONFIG logs AUTO_RESIZE_POD_NAME

The auto resizing controller generates events on the MachineDeployment objects to give visibility into decisions and actions relevant to resizing.

To find the names of the MachineDeployments:

kubectl --kubeconfig ADMIN_CLUSTER_KUBECONFIG get machinedeployments --all-namespaces

To see the events:

kubectl --kubeconfig ADMIN_CLUSTER_KUBECONFIG describe machinedeployment USER_CONTROL_PLANE_NODE_MACHINE_DEPLOYMENT
kubectl --kubeconfig ADMIN_CLUSTER_KUBECONFIG describe machinedeployment ADMIN_ADDON_NODE_MACHINE_DEPLOYMENT