Define compact placement for GKE nodes


You can control whether your Google Kubernetes Engine (GKE) nodes are physically located relative to each other within a zone by using a compact placement policy.

Overview

When you create node pools in a GKE cluster, you can define a compact placement policy, which specifies that nodes within the node pool should be placed in closer physical proximity to each other within a zone. Having nodes closer to each other can reduce network latency between nodes, which can be especially useful for tightly-coupled batch workloads.

Limitations

Compact placement in GKE node pools has the following limitations:

Best practices

Keep node pool size limited

In some locations, it might not be possible to create a large node pool using a compact placement policy. To limit the size of such node pools to what's necessary, you should consider creating a node pool per workload requiring compact placement.

Creating a compact placement policy

To create compact placement policies, in the Google Cloud CLI, you specify the placement-type=COMPACT option during node pool or cluster creation. With this setting, GKE attempts to place nodes within a node pool in closer physical proximity to each other.

If the placement-type flag is not specified, then by default there are no requirements on node placement.

Create a compact placement policy in a new cluster

When you create a new cluster, you can specify a compact placement policy that will be applied to the default node pool. Any subsequent node pools that you create for the cluster, you will need to specify whether to apply compact placement.

To create a new cluster where the default node pool has a compact placement policy applied, use the following command:

gcloud beta container clusters create CLUSTER_NAME \
    --machine-type MACHINE_TYPE \
    –-placement-type COMPACT

Replace the following:

  • CLUSTER_NAME: The name of your new cluster.
  • MACHINE_TYPE: The type of machine to use for nodes, which must be a C2 machine type (for example, c2-standard-4).
  • –-placement-type COMPACT: Applies compact placement for the nodes in the default node pool.

Create a compact placement policy on an existing cluster

On an existing cluster, you can create a node pool that has a compact placement policy applied.

To create a node pool that has a compact placement policy applied, use the following command:

gcloud beta container node-pools create NODEPOOL_NAME \
    --machine-type MACHINE_TYPE \
    --cluster CLUSTER_NAME \
    --placement-type COMPACT

Replace the following:

  • NODEPOOL_NAME: The name of your new node pool.
  • MACHINE_TYPE: The type of machine to use for nodes, which must be a C2 machine type (for example, c2-standard-4).
  • CLUSTER_NAME: The name of your existing cluster.
  • –-placement-type COMPACT: Indicates to apply compact placement for the nodes in the new node pool.

Create a workload on nodes that use compact placement

To run workloads on dedicated nodes that use compact placement, you can use several Kubernetes mechanisms, such as assigning pods to nodes and preventing scheduling unwanted pods on a group of nodes to achieve this.

In the following example, we add a taint to the dedicated nodes and add a corresponding toleration and affinity to the Pods.

  1. Add a taint to nodes in the node pool that has a compact placement policy:

    kubectl taint nodes -l cloud.google.com/gke-nodepool=NODEPOOL_NAME dedicated-pool=NODEPOOL_NAME:NoSchedule
    
  2. In the workload definition, specify the necessary toleration and a node affinity. Here's an example with a single Pod:

    apiVersion: v1
    kind: Pod
    metadata:
      <...>
    spec:
      <...>
      tolerations:
      - key: dedicated-pool
        operator: "Equal"
        value: "NODEPOOL_NAME"
        effect: "NoSchedule"
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: dedicated-pool
                operator: In
                values:
                - NODEPOOL_NAME
    

What's next