Each GKE on AWS release comes with Kubernetes version notes. These are similar to release notes but are specific to a Kubernetes version and may offer more technical detail.
GKE on AWS supports the following Kubernetes versions:
Kubernetes 1.28
1.28.7-gke.1700
- Bug Fix: Fixed an issue where the Instance Metadata Service (IMDS) emulator sometimes failed to bind to an IP address on the node. The IMDS emulator enables nodes to securely access AWS EC2 instance metadata.
1.28.5-gke.1200
- Bug Fixes
- Fixed a bug for file descriptor leak in runc (CVE-2024-21626).
- Security Fixes
- Fixed CVE-2023-38039.
- Fixed CVE-2023-46219.
- Fixed CVE-2023-39326.
- Fixed CVE-2023-44487.
- Fixed CVE-2023-45142.
- Fixed CVE-2023-45285.
- Fixed CVE-2023-48795.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6932.
- Fixed CVE-2023-6931.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6817.
1.28.5-gke.100
- Security Fixes
- Fixed CVE-2022-28948.
- Fixed CVE-2023-29491.
- Fixed CVE-2023-36054.
- Fixed CVE-2023-5363.
- Fixed CVE-2023-47038.
- Fixed CVE-2023-5981.
- Fixed CVE-2023-4806.
- Fixed CVE-2023-4016.
- Fixed CVE-2023-4813.
- Fixed CVE-2022-48522.
- Fixed CVE-2023-46218.
- Fixed CVE-2023-5156.
- Fixed CVE-2023-39804.
- Fixed CVE-2023-5869.
- Fixed CVE-2023-39417.
- Fixed CVE-2023-5868.
- Fixed CVE-2023-5870.
- Fixed GHSA-6xv5-86q9-7xr8.
1.28.3-gke.700
- Breaking Change: Starting from Kubernetes 1.28, clusters require outbound HTTPS connectivity to
{GCP_LOCATION}-gkemulticloud.googleapis.com
. Ensure your proxy server and/or firewall allows for this traffic. Breaking Change: Starting from Kubernetes 1.28, the Multi-Cloud API service agent role requires a new
Iam:getinstanceprofile
permission on your AWS project. This permission is used by the Multi-Cloud Service to inspect the instance profiles attached to in-cluster Virtual Machine Instances.Feature: Added rollback support for AWS node pools that have failed update operations. This allows customers to revert node pools back to their original state.
Feature: Added support for pulling images from private Google Artifact Registry and private Google Container Registry without exported Google Service Account key. The image pull credentials are managed and automatically rotated by Google.
Feature: Removed the need to explicitly add Google IAM bindings for most features.
- No longer need to add any bindings for
gke-system/gke-telemetry-agent
when creating a cluster. - No longer need to add any bindings for
gmp-system/collector
orgmp-system/rule-evaluator
when enabling managed data collection for Google Managed Service for Prometheus. - No longer need to add any bindings for
gke-system/binauthz-agent
when enabling binary authorization.
- No longer need to add any bindings for
Feature: AWS Surge update is now Generally Available. Surge updates allow you to configure the speed and disruption of node pool updates. For more details about how to enable and configure Surge settings on your AWS node pools, see Configure Surge updates of node pools.
Feature: Upgraded the kernel for Ubuntu 22.04 to linux-aws 6.2.
Feature: Added support for creating node pools using the following AWS EC2 instances: G5, I4g, M7a, M7g, M7i, R7g, R7i, and R7iz.
Bug Fix: Improved launch template creation. Tags provided by customers are propagated to instances.
- This change primarily enhances support for IAM policy rules. It specifically addresses rules that prohibit the use of launch templates which don't support tag propagation, even in cases where the associated Auto Scaling Group (ASG) does propagate tags.
- This can be a breaking change, depending on the specifics of the customer's IAM policy regarding tag checks. Therefore, it's important to exercise caution during the upgrade process, as improper handling may leave a cluster in a degraded state.
- Action
ec2:CreateTags
on resourcearn:aws:ec2:*:*:instance/*
is required for the Anthos Multi-Cloud API service agent role. Please check https://cloud.google.com/anthos/clusters/docs/multi-cloud/aws/how-to/create-aws-iam-roles#create_service_agent_role for latest info. - We suggest customers try creating a throw-away 1.28 cluster and confirm that IAM policies work correctly before attempting to upgrade to 1.28.
Bug Fix: Upgrading a cluster to version 1.28 will clean up obsolete resources that may have been created in older versions (up to 1.25) but are no longer relevant. The following resources in the namespace
gke-system
are deleted if they exist:- daemonsets
fluentbit-gke-windows
andgke-metrics-agent-windows
- configmaps
fluentbit-gke-windows-config
andgke-metrics-agent-windows-conf
- daemonsets
Bug Fix: Enhanced Cloud Logging's ingestion of logs from Anthos clusters on AWS:
- Fixed an issue in timestamp parsing.
- Assigned the correct severity level to the
anthos-metadata-agent
's error logs.
Security Fixes
- Fixed CVE-2023-3610
- Fixed CVE-2023-3776
- Fixed CVE-2023-3611
- Fixed CVE-2023-5197
- Fixed CVE-2023-44487
- Fixed CVE-2023-39325
- Fixed CVE-2023-4147
- Fixed CVE-2022-1996
- Fixed CVE-2023-47108
- Fixed CVE-2023-45142
- Fixed CVE-2023-29409
- Fixed CVE-2023-3978
- Fixed CVE-2023-39323
Kubernetes 1.27
1.27.11-gke.1600
- Bug Fix: Fixed an issue where the Instance Metadata Service (IMDS) emulator sometimes failed to bind to an IP address on the node. The IMDS emulator enables nodes to securely access AWS EC2 instance metadata.
1.27.10-gke.500
- Bug Fixes:
- Fixed a bug for file descriptor leak in runc (CVE-2024-21626).
- Security Fixes:
- Fixed CVE-2023-39323.
- Fixed CVE-2023-39325.
- Fixed CVE-2023-39326.
- Fixed CVE-2023-3978.
- Fixed CVE-2023-44487.
- Fixed CVE-2023-45142.
- Fixed CVE-2023-45285.
- Fixed CVE-2023-48795.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6932.
- Fixed CVE-2023-6931.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6817.
1.27.9-gke.100
- Security Fixes
- Fixed CVE-2023-5363.
- Fixed CVE-2023-47038.
- Fixed CVE-2023-5981.
- Fixed CVE-2023-2975.
- Fixed CVE-2023-40217.
- Fixed CVE-2023-29002.
- Fixed CVE-2023-38545.
- Fixed CVE-2023-28321.
- Fixed CVE-2023-0464.
- Fixed CVE-2023-1255.
- Fixed CVE-2023-41332.
- Fixed CVE-2023-0465.
- Fixed CVE-2023-4016.
- Fixed CVE-2022-29458.
- Fixed CVE-2022-3996.
- Fixed CVE-2023-2602.
- Fixed CVE-2023-38546.
- Fixed CVE-2023-34242.
- Fixed CVE-2023-0466.
- Fixed CVE-2022-48522.
- Fixed CVE-2023-28322.
- Fixed CVE-2023-30851.
- Fixed CVE-2023-2283.
- Fixed CVE-2023-27594.
- Fixed CVE-2023-2603.
- Fixed CVE-2023-27593.
- Fixed CVE-2023-5156.
- Fixed CVE-2023-39347.
- Fixed CVE-2023-1667.
- Fixed CVE-2023-2650.
- Fixed CVE-2023-31484.
- Fixed CVE-2023-27595.
- Fixed CVE-2023-41333.
- Fixed CVE-2023-5869.
- Fixed CVE-2023-39417.
- Fixed CVE-2023-5868.
- Fixed CVE-2023-5870.
- Fixed GHSA-6xv5-86q9-7xr8.
1.27.7-gke.600
Feature: Added support for creating node pools using the 'G5' AWS EC2 instance.
Bug Fix: Enhanced Cloud Logging's ingestion of logs from Anthos clusters on AWS:
- Fixed an issue in timestamp parsing.
- Assigned the correct severity level to the
anthos-metadata-agent
's error logs.
Security Fixes
- Fixed CVE-2023-5197
- Fixed CVE-2023-44487
- Fixed CVE-2023-39325
- Fixed CVE-2023-4147
- Fixed CVE-2022-1996
1.27.6-gke.700
- Security Fixes
- Fixed CVE-2015-3276
- Fixed CVE-2022-29155
1.27.5-gke.200
Feature: Ubuntu 22.04 now uses linux-aws 6.2 kernel version.
Security Fixes
- Fixed CVE-2023-3610
- Fixed CVE-2023-3776
- Fixed CVE-2023-3611
1.27.4-gke.1600
Kubernetes OSS release notes * Deprecation: Disabled the unauthenticated kubelet read-only port 10255. Once a node pool is upgraded to version 1.27, workloads running on it will no longer be able to connect to port 10255.
- Feature: AWS Surge update feature is available in preview mode. Surge updates allow you to configure the speed and disruption of node pool updates. Please contact your account team to opt into the preview.
- Feature: Upgraded the EBS CSI Driver to v1.20.0.
- Feature: Upgraded the EFS CSI Driver to v1.5.7.
Feature: Upgraded the
snapshot-controller
andcsi-snapshot-validation-webhook
to v6.2.2. This new version introduces an important change to the API. Specifically, theVolumeSnapshot
,VolumeSnapshotContents
, andVolumeSnapshotClass
v1beta1 APIs are no longer available.Feature: Added support for a new
admin-groups
flag in the create and update APIs. This flag allows customers to quickly and easily authenticate listed groups as cluster administrators, eliminating the need to manually create and apply RBAC policies.Feature: Added Binary Authorization support which is a deploy-time security control that ensures only trusted container images are deployed. With Binary Authorization, you can require images to be signed by trusted authorities during the development process and then enforce signature validation when deploying. By enforcing validation, you can gain tighter control over your container environment by ensuring only verified images are integrated into the build-and-release process. For details about how to enable Binary Authorization on your clusters, see How to enable Binary Authorization.
Feature: Enabled gzip compression for
fluent-bit
(a log processor and forwarder),gke-metrics-agent
(a metrics collector), andaudit-proxy
(an audit log proxy).fluent-bit
compresses log data from both control plane and workloads before sending it to Cloud Logging,gke-metrics-agent
compresses metrics data from both control plane and workloads before sending it to Cloud Monitoring, andaudit-proxy
compresses audit log data before sending it to Audit Logging. This reduces network bandwidth and costs.Feature: Creating AWS SPOT node pools is now GA.
Feature: Node Auto Repair is now GA.
Feature: Improved security by adding file-integrity checks and fingerprint validation for binary artifacts downloaded from Cloud Storage.
Feature: Added an
ignore_errors
option to the delete API to handle cases where accidentally deleted IAM roles or manual removal of resources prevent the deletion of clusters or node pools. By appending?ignore_errors=true
to theDELETE
request URL, users can now forcibly remove clusters or node pools. However, this approach might result in orphaned resources in AWS or Azure, requiring manual cleanup.Feature: Added support for automatic periodic defragmentation of
etcd
andetcd-events
on the control plane. This feature reduces unnecessary disk storage and helps to preventetcd
and the control plane from becoming unavailable due to disk storage issues.Feature: Changed the metrics names for Kubernetes resource metrics to use a metrics prefix of
kubernetes.io/anthos/
rather thankubernetes.io/
. For details refer to the metrics reference documentation.Feature: Changed default etcd version to v3.4.21 on new clusters for improved stability. Existing clusters upgraded to this version will use etcd v3.5.6.
Feature: Improved node resource management by reserving resources for the kubelet. While this feature is crucial for preventing Out of Memory (OOM) errors by ensuring system and Kubernetes processes have the resources they need, it may lead to workload disruptions. The reservation of resources for the kubelet may affect the available resources for Pods, potentially affecting the capacity of smaller nodes to handle existing workloads. Customers should verify that smaller nodes can still support their workloads with this new feature activated.
- The reserved memory percentages are as follows:
- 255 MiB for machines with less than 1GB of memory
- 25% of the first 4GB of memory
- 20% of the next 4GB
- 10% of the next 8GB
- 6% of the next 112GB
- 2% of any memory above 128GB
- The reserved CPU percentages are as follows:
- 6% of the first core
- 1% of the next core
- 0.5% of the next 2 cores
- 0.25% of any cores above 4 cores
Bug Fixes
- Enabled the cluster autoscaler to balance nodes across different
availability zones. This is achieved using the
--balance-similar-node-groups
flag.
- Enabled the cluster autoscaler to balance nodes across different
availability zones. This is achieved using the
Security Fixes
- Fixed CVE-2021-43565
- Fixed CVE-2022-3821
- Fixed CVE-2022-4415
- Fixed CVE-2022-21698
- Fixed CVE-2022-41723.
- Fixed CVE-2022-41725.
- Fixed CVE-2023-24539
- Fixed CVE-2023-24540
- Fixed CVE-2023-29400
Kubernetes 1.26
1.26.14-gke.1500
- Bug Fix: Fixed an issue where the Instance Metadata Service (IMDS) emulator sometimes failed to bind to an IP address on the node. The IMDS emulator enables nodes to securely access AWS EC2 instance metadata.
1.26.13-gke.400
- Bug Fixes:
- Fixed a bug for file descriptor leak in runc (CVE-2024-21626).
- Security Fixes:
- Fixed CVE-2021-43565.
- Fixed CVE-2022-21698.
- Fixed CVE-2022-27191.
- Fixed CVE-2022-28948.
- Fixed CVE-2023-39318.
- Fixed CVE-2023-39319.
- Fixed CVE-2023-39323.
- Fixed CVE-2023-39325.
- Fixed CVE-2023-39326.
- Fixed CVE-2023-3978.
- Fixed CVE-2023-44487.
- Fixed CVE-2023-45142.
- Fixed CVE-2023-45285.
- Fixed CVE-2023-47108.
- Fixed CVE-2023-48795.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6932.
- Fixed CVE-2023-6931.
- Fixed CVE-2024-0193.
- Fixed CVE-2023-6817.
1.26.12-gke.100
- Security Fixes
- Fixed CVE-2023-5363.
- Fixed CVE-2023-47038.
- Fixed CVE-2023-5981.
- Fixed CVE-2023-2975.
- Fixed CVE-2023-4527.
- Fixed CVE-2023-29002.
- Fixed CVE-2023-38545.
- Fixed CVE-2023-28321.
- Fixed CVE-2023-0464.
- Fixed CVE-2023-1255.
- Fixed CVE-2023-41332.
- Fixed CVE-2023-0465.
- Fixed CVE-2023-4016.
- Fixed CVE-2022-29458.
- Fixed CVE-2022-3996.
- Fixed CVE-2023-2602.
- Fixed CVE-2023-38546.
- Fixed CVE-2023-34242.
- Fixed CVE-2023-0466.
- Fixed CVE-2022-48522.
- Fixed CVE-2023-28322.
- Fixed CVE-2023-30851.
- Fixed CVE-2023-2283.
- Fixed CVE-2023-27594.
- Fixed CVE-2023-2603.
- Fixed CVE-2023-27593.
- Fixed CVE-2023-5156.
- Fixed CVE-2023-39347.
- Fixed CVE-2023-1667.
- Fixed CVE-2023-2650.
- Fixed CVE-2023-31484.
- Fixed CVE-2023-27595.
- Fixed CVE-2023-41333.
- Fixed CVE-2023-5869.
- Fixed CVE-2023-39417.
- Fixed CVE-2023-5868.
- Fixed CVE-2023-5870.
1.26.10-gke.600
Feature: Added support for creating node pools using the 'G5' AWS EC2 instance.
Bug Fix: Upgraded the Elastic File System (EFS) Container Storage Interface (CSI) driver
aws-efs-csi-driver
to version v1.3.8-gke.21.Bug Fix: Enhanced Cloud Logging's ingestion of logs from Anthos clusters on AWS:
- Fixed an issue in timestamp parsing.
- Assigned the correct severity level to the
anthos-metadata-agent
's error logs.
Security Fixes
- Fixed CVE-2023-5197
- Fixed CVE-2023-44487
- Fixed CVE-2023-39325
- Fixed CVE-2023-4147
- Fixed CVE-2022-1996
1.26.9-gke.700
- Security Fixes
- Fixed CVE-2015-3276
- Fixed CVE-2022-29155
1.26.8-gke.200
Feature: Ubuntu 22.04 now uses linux-aws 6.2 kernel version.
Security Fixes
- Fixed CVE-2023-3610
- Fixed CVE-2023-3776
- Fixed CVE-2023-3611
1.26.7-gke.500
- Security Fixes
- Fixed CVE-2022-3821
- Fixed CVE-2022-4415
1.26.5-gke.1400
- Security Fixes
- Fixed CVE-2022-27664
- Fixed CVE-2022-32149
- Fixed CVE-2022-41723
- Fixed CVE-2023-24534
- Fixed CVE-2023-24536
- Fixed CVE-2023-24537
- Fixed CVE-2023-24538
1.26.5-gke.1200
- Bug Fixes
- Configures the cluster autoscaler to balance the number of nodes across availability zones using --balance-similar-node-groups.
1.26.4-gke.2200
Kubernetes OSS release notes * Feature: Ubuntu 22.04 is using linux-aws 5.19 kernel.
Bug Fixes
- Fixed an issue where Kubernetes would incorrectly apply the default StorageClass to PersistentVolumeClaims which have the deprecated annotation volume.beta.kubernetes.io/storage-class.
- Fixed an issue in which the logging agent consumed increasingly high amounts of memory.
Security Fixes
- Fixed an issue affecting netfilter connection tracking (conntrack), which is responsible for monitoring network connections. The fix ensures proper insertion of new connections into the conntrack table and overcomes the limitations caused by changes made to Linux kernel versions 5.15 and higher.
1.26.2-gke.1001
Known Issue: Kubernetes 1.26.2 will incorrectly apply the default StorageClass to PersistentVolumeClaims which have the deprecated annotation
volume.beta.kubernetes.io/storage-class
.Feature: Updated OS image to Ubuntu 22.04.
cgroupv2
is now used as the default control group configuration.- Ubuntu 22.04 uses
cgroupv2
by default. We recommend that you check if any of your applications access thecgroup
filesystem. If they do, they must be updated to usecgroupv2
. Some example applications that might require updates to ensure compatibility withcgroupv2
are: - Third-party monitoring and security agents that depend on the
cgroup
filesystem. - If
cAdvisor
is being used as a stand-alone DaemonSet for monitoring Pods and containers, it should be updated to version v0.43.0 or later. - If you are using JDK, we recommend that you use version 11.0.16 and later, or version 15 and later. These versions fully support
cgroupv2
. - If you are using the uber-go/automaxprocs package, make sure to use version v1.5.1 or higher.
- Ubuntu 22.04 removes the
timesyncd
package. Instead,chrony
is now used for the Amazon Time Sync Service. - For more information, see the Ubuntu release notes
- Ubuntu 22.04 uses
Feature: Sends metrics for control plane components to Cloud Monitoring. This includes a subset of the Prometheus metrics from kube-apiserver, etcd, kube-scheduler, kube-controller-manager. Metrics names use the prefix
kubernetes.io/anthos/
.Feature: Enabled sending Kubernetes resource metadata to Google Cloud Platform, improving both the user interface and cluster metrics. For the metadata to be ingested properly, customers need to enable the
Config Monitoring for Ops
API. This API can be enabled either in the Google Cloud Console , or by manually enabling theopsconfigmonitoring.googleapis.com
API in the gcloud CLI. Additionally, customers must follow the steps outlined in the Authorize Cloud Logging/Monitoring documentation to add the necessary IAM bindings. If applicable, addopsconfigmonitoring.googleapis.com
to your Proxy Allowlist.Feature: Added preview feature for creating Spot AWS node pool.
Feature: Creating node pools using ARM-based (Graviton) instance types is now GA.
Feature: Enabled kubelet graceful node shutdown. Non-system Pods are given 15 seconds to terminate, after which system Pods (with the
system-cluster-critical
orsystem-node-critical
priority classes) have 15 seconds to gracefully terminate.Feature: Enabled Node auto repair feature in preview mode. Please contact your account team to opt into the preview.
Feature: Added tags to dynamically created EFS Access Point resource.
Feature: Clusters now have per-node-pool subnet security group rules instead of VPC-wide rules
- Previously, the control plane allowed inbound traffic from the entire primary IP range of the VPC on ports TCP/443 and TCP/8123, which are used by node pools.
- Now, the control plane narrows the allowed inbound traffic to each IP range of the node pool subnets on ports TCP/443 and TCP/8123; multiple node pools can share one subnet.
- This change supports node pools running outside of the VPC's primary IP range and improves the security of the control plane.
- If you relied on the VPC-wide security group rule for allowing traffic from outside of the cluster (e.g. from a bastion host for kubectl), then as part of the upgrade you should create a security group, add a VPC-wide rule to it, and attach the security group to the control plane (via the AwsCluster.controlPlane.securityGroupIds field).
Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.
Security Fix: Set the hop limit of the IMDS emulator response to 1. This secures the communication of IMDS data between the emulator and a workload.
Version support windows
Release dates and end of support dates for supported Kubernetes versions are listed on the GKE on AWS version lifespans page.