Configure node pool update policy

Version 1.13 

• Latest version
• Version 1.16
• Earlier versions (unsupported)

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Node pools are updated in parallel by default. This can potentially causing two issues:

• Quota issue

  For each node pool, it will create one surge node, so that N node pools will create N surge nodes when they are updated in parallel. This may result in resource exhaustion if there is limited extra capacity for creating those surge nodes.

• PDB deadlock issue

  Draining more than one node at a time could potentially cause PDB deadlocks.

This doc shows how to specify a node pool update policy to configure the maximum number of nodes to be updated concurrently during node pool update, which can help avoid the two issues mentioned. In Anthos 1.13, we restrict the number to be either 0 or 1. 0 means preserving the current parallel behavior , which is the default value. 1 means updating the node pools sequentially.

Note that while sequential update can avoid the two issues, it could make the node pool update process take longer time than updating in parallel.

Configure node pool update policy

In user cluster seed config user-cluster.yaml, you can configure the node pool update to be done sequentially as follows:

nodePoolUpdatePolicy:
  maximumConcurrentNodePoolUpdate: 1

Note that this value for maximumConcurrentNodePoolUpdate is only allowed to be either 0 or 1. The sequential behavior applies to both node pool creation and update. Also in case any node pool update/create encountered issues and got stuck, the current behavior is that we will block there and won't move on to update next node pool.