GKE on AWS lets you run Arm workloads built for Arm-based AWS Graviton processors.
Limitations
Arm node pools running Kubernetes versions earlier than 1.24.8-gke.1300 automatically add a taint during node pool creation to prevent Arm workloads from being scheduled on non-Arm nodes. Arm node pools in clusters at version 1.24.8-gke.1300 or higher no longer add this taint. If you're upgrading from a cluster earlier than 1.24.8-gke.1300, you must create this taint yourself or otherwise take this into account when upgrading.
Arm node pools on GKE on AWS don't support Cloud Service Mesh, Config Sync, or Policy Controller. You must run these products on an x86 node pool.
Clusters running Kubernetes version 1.24 need an x86 node pool to run the Connect Agent. If your cluster runs Kubernetes version 1.25 or later, you don't need an x86 node pool.
This page explains how to create an Arm node pool, why multi-architecture images are the recommended way to deploy Arm workloads, and how to schedule Arm workloads.
Before you begin
Before you create node pools for your Arm workloads, you need the following resources:
- An existing AWS cluster to create the node pool in. This cluster must run Kubernetes version 1.24 or later.
- An IAM instance profile for the node pool VMs.
- A subnet where the node pool VMs will run.
If your cluster is running Kubernetes version 1.24, an x86 node pool to run the Connect Agent.
For details on how to create a node pool in GKE on AWS, see Create a node pool.
Create an Arm node pool
GKE on AWS supports node pools built on the Canonical Ubuntu
arm64 minimal node image and containerd
runtime.
To create an Arm node pool and add it to an existing cluster, run the following command:
gcloud container aws node-pools create NODE_POOL_NAME \
--cluster CLUSTER_NAME \
--instance-type INSTANCE_TYPE \
--root-volume-size ROOT_VOLUME_SIZE \
--iam-instance-profile NODEPOOL_PROFILE \
--node-version NODE_VERSION \
--min-nodes MIN_NODES \
--max-nodes MAX_NODES \
--max-pods-per-node MAX_PODS_PER_NODE \
--location GOOGLE_CLOUD_LOCATION \
--subnet-id NODEPOOL_SUBNET \
--ssh-ec2-key-pair SSH_KEY_PAIR_NAME \
--config-encryption-kms-key-arn CONFIG_KMS_KEY_ARN
Replace the following:
NODE_POOL_NAME
: the name you choose for your node poolCLUSTER_NAME
: the name of the cluster to attach the node pool toINSTANCE_TYPE
: one of the following instance types:m6g
m6gd
t4g
r6g
r6gd
c6g
c6gd
c6gn
x2gd
c7g
im4gn
g5g
These instance types are powered by Arm-based AWS Graviton processors. You also need to specify the instance size that you want. For example,
m6g.medium
. For a complete list, see Supported AWS instance types.
ROOT_VOLUME_SIZE
: the desired size for each node's root volume, in GbNODEPOOL_PROFILE
: the IAM instance profile for node pool VMsNODE_VERSION
: the Kubernetes version to install on each node in the node pool, which must be version 1.24 or later. For example,1.24.3-gke.200
.MIN_NODES
: the minimum number of nodes the node pool can containMAX_NODES
: the maximum number of nodes the node pool can containMAX_PODS_PER_NODE
: the maximum number of Pods that can be created on any single node in the poolGOOGLE_CLOUD_LOCATION
: the name of the Google Cloudlocation from which this node pool will be managed
NODEPOOL_SUBNET
: the ID of the subnet the node pool will run on. If this subnet is outside of the VPC's primary CIDR block, you need to take additional steps. For more information, see security groups.SSH_KEY_PAIR_NAME
: the name of the AWS SSH key pair created for SSH access (optional)CONFIG_KMS_KEY_ARN
: the Amazon Resource Name (ARN) of the AWS KMS key that encrypts user data
Understand multi-architecture images
Container images must be compatible with the architecture of the node where you intend to run the Arm workloads. To ensure that your container image is Arm-compatible, we recommend that you use multi-architecture ("multi-arch") images.
A multi-arch image is an image that can support multiple architectures. It looks like a single image with a single tag, but contains a set of images to run on different machine architectures. Multi-arch images are compatible with the Docker Image Manifest V2 Scheme 2 or OCI Image Index Specifications.
When you deploy a multi-arch image to a cluster, the container runtime automatically chooses the image that is compatible with the architecture of the node you're deploying to. Once you have a multi-arch image for a workload, you can deploy this workload across multiple architectures. Scheduling a single-architecture image onto an incompatible node causes an error at load time.
To learn more about how to use multi-arch images with Arm workloads, see Build multi-architecture images for Arm workloads in the Google Kubernetes Engine (GKE) documentation.