Unsupported Kubernetes cluster versions

The following versions of GKE on AWS are unsupported.

Kubernetes 1.25

1.25.14-gke.700

Kubernetes OSS release notes.

1.25.13-gke.200

Kubernetes OSS release notes.

1.25.12-gke.500

Kubernetes OSS release notes. * Feature: Expanded the list of metrics collected from node pools to include gke-metrics-agent, cilium-agent, cilium-operator, coredns, fluentbit-gke, kubelet, and konnectivity-agent.

1.25.10-gke.1400

Kubernetes OSS release notes.

1.25.10-gke.1200

Kubernetes OSS release notes.

  • Bug Fixes
    • Configures the cluster autoscaler to balance the number of nodes across availability zones using --balance-similar-node-groups.
  • Security Fixes
    • Migrated node pool metrics agent and metrics server to authenticated kubelet port.

1.25.8-gke.500

Kubernetes OSS release notes.

  • Bug Fixes

    • Fixed an issue in which the logging agent consumed increasingly high amounts of memory.

  • Security Fixes

1.25.7-gke.1000

Kubernetes OSS release notes.

  • Feature: Added tags to dynamically created EFS Access Point resource.

  • Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.

1.25.6-gke.1600

Kubernetes OSS release notes.

1.25.5-gke.2000

Kubernetes OSS release notes. * Feature: Updated Anthos Identity Service to better handle concurrent authentication webhook requests.

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.
  • Bug Fix: Fixed an issue with AWS EFS CSI driver where EFS hostnames can't be resolved when AWS VPC is configured to use a custom DNS server.

  • Bug Fix: Fixed an issue where authentication through the Anthos Service Mesh dashboard failed due to inability to impersonate end user.

  • Security Fixes

1.25.5-gke.1500

Kubernetes OSS release notes.

  • Known Issue: Some UI surfaces in Google Cloud console can't authorize to the cluster and might display the cluster as unreachable. A workaround is to manually apply RBAC permitting user impersonation. For details, see Troubleshooting.

  • Security Fixes

1.25.4-gke.1300

Kubernetes OSS release notes.

  • Known Issue: Some UI surfaces in Google Cloud console can't authorize to the cluster and might display the cluster as unreachable. A workaround is to manually apply RBAC permitting user impersonation. For details, see Troubleshooting.

  • Deprecation: removed deprecated in-tree volume plugins flocker, quobyte and storageos.

  • Feature: Enhanced security by restricting static pods running on the cluster's control plane VMs to run as non-root Linux users.

  • Feature: Added support for dynamically updating AWS node pool security groups. To update security groups, you must have the following permissions in your API role -

    • ec2:ModifyInstanceAttribute
    • ec2:DescribeInstances

  • Feature: Added support for dynamically updating AWS node pool tags. To update node pool tags, you must have the following permissions in your API role -

    • autoscaling:CreateOrUpdateTags
    • autoscaling:DeleteTags
    • ec2:CreateTags
    • ec2:DeleteTags
    • ec2:DescribeLaunchTemplates

  • Feature: EFS dynamic provisioning is now available in GA for clusters at version 1.25 or later. To use this feature, you must add the following permissions to the control plane role:

    • ec2:DescribeAvailabilityZones
    • elasticfilesystem:DescribeAccessPoints
    • elasticfilesystem:DescribeFileSystems
    • elasticfilesystem:DescribeMountTargets
    • elasticfilesystem:CreateAccessPoint
    • elasticfilesystem:DeleteAccessPoint

  • Feature: Uploading of workload metrics using Google Managed Service for Prometheus with managed collection to Cloud Monarch is now available in GA.

  • Feature: Added support to enable and update CloudWatch metrics collection on AWS node pool's auto scaling group. To enable or update metrics collection via create or update API, you must add the following permissions to your API role:

    • autoscaling:EnableMetricsCollection
    • autoscaling:DisableMetricsCollection

  • Feature: Azure AD GA. This feature allows cluster admins to configure RBAC policies based on Azure AD groups for authorization in clusters. This supports retrieval of groups information for users belonging to more than 200 groups, thus overcoming a limitation of regular OIDC configured with Azure AD as the identity provider.

  • Feature: Added a new token manager (gke-token-manager) to generate tokens for control plane components, using the service account signing key. Benefits:

    1. Eliminate the dependency on kube-apiserver for control plane components to authenticate to Google services. Previously, control plane components would use the TokenRequest API and were reliant on a healthy kube-apiserver. Whereas now the gke-token-manager component mints the tokens directly using the service account signing key.
    2. Eliminate the RBAC for generating token for controlplane components.
    3. Uncouple the logging and kube-apiserver. So that the logging can be ingested before the kube-apiserver is up.
    4. Make the controlplane more resilience. When the kube-apiserver is out of service the controlplane components can still get the tokens and keep working.

  • Feature: As a preview feature, ingest a variety of metrics from the control plane components to Cloud Monitoring, including kube-apiserver, etcd, kube-scheduler and kube-controller-manager.

  • Feature: Users in a Google Group can access AWS clusters using Connect Gateway by granting necessary RBAC permission to the Group. More details at Set up the Connect gateway with Google Groups.

  • Bug Fix: Fixed an issue which could result in outdated versions of gke-connect-agent not being removed after cluster upgrades.

  • Security Fixes

Kubernetes 1.24

1.24.14-gke.2700

Kubernetes OSS release notes.

1.24.14-gke.1400

Kubernetes OSS release notes.

  • Bug Fixes
    • Configures the cluster autoscaler to balance the number of nodes across availability zones using --balance-similar-node-groups.

1.24.13-gke.500

Kubernetes OSS release notes.

  • Bug Fixes

    • Fixed an issue in which the logging agent consumed increasingly high amounts of memory.

  • Security Fixes

1.24.11-gke.1000

Kubernetes OSS release notes.

  • Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.

1.24.10-gke.1200

Kubernetes OSS release notes.

1.24.9-gke.2000

Kubernetes OSS release notes.

  • Feature: Updated Anthos Identity Service to better handle concurrent authentication webhook requests.

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.

  • Security Fixes

1.24.9-gke.1500

Kubernetes OSS release notes.

1.24.8-gke.1300

Kubernetes OSS release notes.

1.24.5-gke.200

Kubernetes OSS release notes.

1.24.3-gke.2200

Kubernetes OSS release notes.

  • Bug Fix: Fix a bug where creating a Kubernetes Service resource with type LoadBalancer and annotation service.beta.kubernetes.io/aws-load-balancer-type: nlb, would remain with an empty target group. See https://github.com/kubernetes/cloud-provider-aws/issues/301

1.24.3-gke.2100

Kubernetes OSS release notes.

  • Feature: Upload Kubernetes resource metrics to Google Cloud Monitoring for Windows node pools.
  • Feature: Provided a webhook for easy IMDS emulator injection.
  • Feature: go1.18 stops accepting certificates signed with the SHA-1 hash algorithm by default. Admission/conversion webhooks or aggregated server endpoints using these insecure certificates will break by default in 1.24. The environment variable GODEBUG=x509sha1=1 is set in Anthos on-AWS clusters as a temporary workaround to let these insecure certificates continue to work. However, the go team is anticipated to remove support on this workaround in the near coming releases. Customers should check and ensure there aren't any admission/conversion webhooks or aggregated server endpoints that are using such insecure certificates before upgrading to the upcoming breaking version.
  • Feature: GKE on AWS now supports EFS dynamic provisioning in preview mode, for Kubernetes clusters at version 1.24 or later. To use this feature, you must add the following permissions to the control plane role: ec2:DescribeAvailabilityZones elasticfilesystem:DescribeAccessPoints elasticfilesystem:DescribeFileSystems elasticfilesystem:DescribeMountTargets elasticfilesystem:CreateAccessPoint elasticfilesystem:DeleteAccessPoint

  • Feature: Improve network connectivity checks during cluster and node pool creation to help troubleshooting.

  • Feature: Support updates to AWS control plane tags. To update tags, you need to add the following permissions to the API role - autoscaling:CreateOrUpdateTags autoscaling:DeleteTags ec2:CreateTags ec2:DescribeLaunchTemplates ec2:DescribeSecurityGroupRules ec2:DeleteTags elasticloadbalancing:AddTags elasticloadbalancing:RemoveTags

  • Feature: Upload workload metrics using Google Managed Service for Prometheus to Cloud Monarch is available as invite only private preview.

  • Security Fixes

Kubernetes 1.23

1.23.16-gke.2800

Kubernetes OSS release notes

1.23.16-gke.200

Kubernetes OSS release notes

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.

  • Bug Fix: Fixed cpp-httplib issues with kubeapi server unable to reach AIS.

  • Security Fixes

1.23.14-gke.1800

Kubernetes OSS release notes.

1.23.14-gke.1100

Kubernetes OSS release notes

1.23.11-gke.300

Kubernetes OSS release notes

1.23.9-gke.2200

Kubernetes OSS release notes

  • Bug Fix: Fix a bug where creating a Kubernetes Service resource with type LoadBalancer and annotation service.beta.kubernetes.io/aws-load-balancer-type: nlb, would remain with an empty target group. See https://github.com/kubernetes/cloud-provider-aws/issues/301

1.23.9-gke.2100

Kubernetes OSS release notes

1.23.9-gke.800

Kubernetes OSS release notes

1.23.8-gke.1700

Kubernetes OSS release notes

1.23.7-gke.1300

Kubernetes OSS release notes.

  • Feature: Disable profiling endpoint (/debug/pprof) by default in kube-scheduler and kube-controller-manager.

  • Feature: Update kube-apiserver and kubelet to only use Strong Cryptographic Ciphers. Supported Ciphers used by Kubelet:

    TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256

    Supported Ciphers used by kube api-server:

    TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384

  • Feature: Add an instance metadata server (IMDS) emulator.

  • Security Fixes

Kubernetes 1.22

1.22.15-gke.100

Kubernetes OSS release notes

1.22.12-gke.2300

Kubernetes OSS release notes

1.22.12-gke.1100

Kubernetes OSS release notes

1.22.12-gke.200

Kubernetes OSS release notes

1.22.10-gke.1500

Kubernetes OSS release notes.

1.22.8-gke.2100

Kubernetes OSS release notes.

  • Feature: Windows nodes now use pigz to improve image layer extraction performance.

1.22.8-gke.1300

  • Bug Fixes
    • Fixed an issue where addons cannot be applied when Windows nodepools are enabled.
    • Fixed an issue where logging agent could fill up attached disk space.
  • Security Fixes
    • Fixed CVE-2022-1055.
    • Fixed CVE-2022-0886.
    • Fixed CVE-2022-0492.
    • Fixed CVE-2022-24769.
    • This release includes the following Role-based access control (RBAC) changes:
    • Scoped down anet-operator permissions for Lease update.
    • Scoped down anetd Daemonset permissions for Nodes and pods.
    • Scoped down fluentbit-gke permissions for service account tokens.
    • Scoped down gke-metrics-agent for service account tokens.
    • Scoped down coredns-autoscaler permissions for Nodes, ConfigMaps and Deployments.

1.22.8-gke.200

Kubernetes OSS release notes.

  • Feature: The default instance type for clusters and node pools created under Kubernetes v1.22 is now m5.large instead of t3.medium.
  • Feature: When you create a new cluster using Kubernetes version 1.22, you can now configure custom logging parameters.
  • Feature: As a preview feature, you can now choose Windows as your node pool image type when you create node pools with Kubernetes version 1.22.
  • Feature: As a preview feature, you can now configure host machines as dedicated hosts.
  • Feature: You can now view most common asynchronous cluster and nodepool boot errors in the long running operation error field. For more information, see the gcloud container aws operations list reference documentation.
  • Security Fixes

Kubernetes 1.21

1.21.14-gke.2900

Kubernetes OSS release notes

1.21.14-gke.2100

Kubernetes OSS release notes

1.21.11-gke.1900

Kubernetes OSS release notes.

1.21.11-gke.1800

Kubernetes OSS release notes.

1.21.11-gke.1100

  • Security Fixes
    • Fixed CVE-2022-1055.
    • Fixed CVE-2022-0886.
    • Fixed CVE-2022-0492.
    • Fixed CVE-2022-24769.
    • RBAC fixes:
    • Scoped down anet-operator permissions for Lease update.
    • Scoped down anetd Daemonset permissions for Nodes and pods.
    • Scoped down fluentbit-gke permissions for service account tokens.
    • Scoped down gke-metrics-agent for service account tokens.
    • Scoped down coredns-autoscaler permissions for Nodes, ConfigMaps and Deployments.

1.21.11-gke.100

Kubernetes OSS release notes.

1.21.6-gke.1500

Kubernetes OSS release notes.

1.21.5-gke.2800

Kubernetes OSS release notes.