This page explains how to prepare a workload to be scheduled on Arm nodes in a GKE Standard cluster. To learn more about scheduling Arm workloads with Autopilot, see Deploy Autopilot workloads on Arm architecture.
To successfully schedule a workload to an Arm node, you must have the following:
- An Arm-compatible container image. For guidance on how to check this, see Is my workload ready for Arm?
- Arm nodes where your Arm-compatible workloads can be scheduled. To create the necessary resources, see Create clusters and node pools with Arm nodes.
- A cluster in a Google Cloud region or zone that has available Arm virtual machines (VMs). For a filterable table of machine types and platforms, see Available regions and zones.
Overview
By default, GKE schedules workloads only to x86-based
nodes—Compute Engine machine series with Intel
or AMD processors—by placing a
taint (kubernetes.io/arch=arm64:NoSchedule
)
on all Arm nodes. This taint prevents x86-compatible workloads from being
inadvertently scheduled to your Arm nodes. If you want to deploy a workload to
an Arm node, use the fields described on this page to guide the scheduler to
send the workload to the desired type of node.
Use one of the following fields:
- A node selector.
- A node affinity rule.
When you use a node selector or node affinity rule, GKE will only schedule your Arm-compatible workloads when you've declared that the workload's container image can run on the node's architecture.
If you schedule an Arm-compatible workload with a node selector or with a node affinity rule as described in the following sections, GKE automatically adds a toleration to the workload configuration so that the Pods can run on the Arm nodes.
This toleration added to the workload matches the taint (kubernetes.io/arch=arm64:NoSchedule
)
added to all Arm nodes to allow your workload to be scheduled on Arm nodes.
In some situations, such as when you have multi-architecture images that can run on any node, you might want to manually add this toleration to the workload configuration. For instructions, refer to Use toleration for scheduling multi-arch workloads to any architectures.
Use a node selector to schedule an Arm workload
Add the following node selector to the specification:
nodeSelector:
kubernetes.io/arch: arm64
The node selector specifies that this workload should only be scheduled to
nodes with the arm64
label, which all Arm nodes on GKE clusters
have.
When this node selector is included in the workload configuration, GKE adds the toleration to match the taint to permit the workload to be scheduled on Arm nodes.
Use a node affinity rule to schedule an Arm workload
You can also use node affinity to schedule your workload.
Schedule workload to a single architecture
Add the following node affinity to the specification:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- arm64
The node affinity rule specifies that the workload should only be scheduled to
nodes with the arm64
label, which all Arm nodes on GKE clusters
have.
When this node affinity rule is included in the workload configuration, GKE adds the toleration to match the taint to permit the workload to be scheduled on Arm nodes.
Schedule workload to x86 and Arm architectures
If you want to schedule a workload across both x86 (Intel and AMD processors) and Arm architectures, you can specify this in different ways.
Use toleration for scheduling multi-arch workloads to any architectures
If you have a multi-arch image that you want to be scheduled to any available architecture type in a Standard cluster, you only need to add the toleration to the workload specification. You do not need the node selector or node affinity rules described on this page as the workload can be scheduled to all architecture types.
Add the toleration:
tolerations:
- key: kubernetes.io/arch
operator: Equal
value: arm64
effect: NoSchedule
Using this toleration, GKE could schedule a workload to nodes with any architecture type.
For example, if you have a cluster with the following node pools:
- my-t2a-node-pool, using t2a-standard-16 VMs (
arm64
). - my-c2-node-pool, using c2-standard-8 VMs (
amd64
). - my-t2d-node-pool, using t2-standard-48 VMs (
amd64
).
If you deploy to this cluster a workload which uses a multi-arch image and the
arm64
toleration in the workload configuration, GKE may
schedule the workload across all of the node pools.
Use node affinity rule for scheduling multi-architecture workloads to any architectures
If you want a workload to be scheduled on nodes across architecture types including x86 and Arm, you can also use a node affinity rule. With node affinity rules, you can specify exactly which architecture types you want to schedule the workload on. This approach is recommended for scheduling workloads on Autopilot clusters. To learn more, see Deploy Autopilot workloads on Arm architecture.
With x86-based workloads, you do not need these node selectors, node affinity rules, or tolerations for the workload to be scheduled. If you have an image that you only want to schedule to x86-based nodes, you do not need to use these fields.
To schedule workloads to any architecture type, list both arm64
and amd64
the values
section of the node affinity field. amd64
includes any nodes
using x86 processors.
The following example specifies that this workload could be scheduled on nodes with Arm processors or x86 processors:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- arm64
- amd64
The labels for each architecture type are:
arm64
for nodes using Arm processors (for example, Tau T2A).amd64
for nodes using AMD processors (for example, Tau T2D) or nodes using Intel processors (for example, C2).
For example, if you have a cluster with the following node pools and the given node affinity rule:
- my-t2a-node-pool, using t2a-standard-16 VMs (
arm64
). - my-c2-node-pool, using c2-standard-8 VMs (
amd64
). - my-t2d-node-pool, using t2-standard-48 VMs (
amd64
).
If you deploy to this cluster a workload which uses a multi-arch image and the
node affinity with arm64
included in the values
list, GKE
adds the toleration in the workload configuration and may
schedule the workload across all of the node pools.
Deploy the workload
Now that you have specified where your Arm-compatible workloads should be scheduled, you can deploy your workload.
When you deploy a workload to a GKE cluster, the instructions are the same across architecture types. You can deploy an Arm-compatible workload as you would deploy any other workload, as long as you've completed the prerequisite steps. To see examples of deploying workloads, view the following pages:
Troubleshooting
For common errors and troubleshooting information, refer to Troubleshooting Arm workloads.