Kubernetes version notes archive

This page contains a historical archive of Kubernetes version notes for unsupported versions. To view more recent version notes, see Kubernetes version notes.

Kubernetes 1.28

1.28.14-gke.200

Kubernetes OSS release notes

1.28.13-gke.600

Kubernetes OSS release notes

1.28.12-gke.100

Kubernetes OSS release notes

1.28.11-gke.600

Kubernetes OSS release notes

1.28.10-gke.1300

Kubernetes OSS release notes

1.28.10-gke.800

Kubernetes OSS release notes

1.28.9-gke.400

Kubernetes OSS release notes

1.28.8-gke.800

Kubernetes OSS release notes

1.28.7-gke.1700

Kubernetes OSS release notes

1.28.5-gke.1200

Kubernetes OSS release notes

1.28.5-gke.100

Kubernetes OSS release notes

1.28.3-gke.700

Kubernetes OSS release notes

  • Breaking Change: Starting from 1.28, clusters require outbound HTTPS connectivity to {GCP_LOCATION}-gkemulticloud.googleapis.com. Ensure your proxy server and/or firewall allows for this traffic.

  • Feature: Removed the need to explicitly add Google IAM bindings for most features.

    1. No longer need to add any bindings for gke-system/gke-telemetry-agent when creating a cluster.
    2. No longer need to add any bindings for gmp-system/collector or gmp-system/rule-evaluator when enabling managed data collection for Google Managed Service for Prometheus.

  • Feature: Ubuntu 22.04 now uses linux-azure 6.2 kernel version.

  • Bug Fix: Monitoring metrics for the gke-azure-encryption-provider control plane Pod are now reported on the kube-system namespace. Previously, they were mistakenly being reported on the default namespace.

  • Bug Fix: Upgrading a cluster to version 1.28 will clean up obsolete resources that may have been created in older versions (up to 1.25) but are no longer relevant. The following resources in the namespace gke-system are deleted if exist:

    • daemonsets fluentbit-gke-windows and gke-metrics-agent-windows
    • configmaps fluentbit-gke-windows-config and gke-metrics-agent-windows-conf

  • Bug Fix: Enhanced Cloud Logging's ingestion of logs from Anthos clusters on Azure:

    • Fixed an issue in timestamp parsing.
    • Assigned the correct severity level to the anthos-metadata-agent's error logs.

  • Security Fixes

Kubernetes 1.27

1.27.14-gke.1600

Kubernetes OSS release notes

1.27.14-gke.1200

Kubernetes OSS release notes

1.27.14-gke.700

Kubernetes OSS release notes

1.27.13-gke.500

Kubernetes OSS release notes

1.27.12-gke.800

Kubernetes OSS release notes

1.27.11-gke.1600

Kubernetes OSS release notes

1.27.10-gke.500

Kubernetes OSS release notes

1.27.9-gke.100

Kubernetes OSS release notes

1.27.7-gke.600

Kubernetes OSS release notes

1.27.6-gke.700

Kubernetes OSS release notes

1.27.5-gke.200

Kubernetes OSS release notes

1.27.4-gke.1600

Kubernetes OSS release notes

  • Deprecation: Disabled the unauthenticated kubelet read-only port 10255. Once a node pool is upgraded to version 1.27, workloads running on it will no longer be able to connect to port 10255.

  • Feature: Upgraded the Azuredisk CSI Driver to v1.28.1.

  • Feature: Upgraded the Azurefile CSI Driver to v1.28.1.

  • Feature: Upgraded the snapshot-controller and csi-snapshot-validation-webhook to v6.2.2. This new version introduces an important change to the API. Specifically, the VolumeSnapshot, VolumeSnapshotContents, and VolumeSnapshotClass v1beta1 APIs are no longer available.

  • Feature: Added support for a new admin-groups flag in the create and update APIs. This flag allows customers to quickly and easily authenticate listed groups as cluster administrators, eliminating the need to manually create and apply RBAC policies.

  • Feature: Enabled gzip compression for fluent-bit (a log processor and forwarder), gke-metrics-agent (a metrics collector), and audit-proxy (an audit log proxy). fluent-bit compresses log data from both control plane and workloads before sending it to Cloud Logging, gke-metrics-agent compresses metrics data from both control plane and workloads before sending it to Cloud Monitoring, and audit-proxy compresses audit log data before sending it to Audit Logging. This reduces network bandwidth and costs.

  • Feature: Node Auto Repair is now GA.

  • Feature: Improved security by adding file-integrity checks and fingerprint validation for Google-managed binary artifacts downloaded from Cloud Storage.

  • Feature: Added support for automatic periodic defragmentation of etcd and etcd-events on the control plane. This feature reduces unnecessary disk storage and helps to prevent etcd and the control plane from becoming unavailable due to disk storage issues.

  • Feature: Changed the metrics names for Kubernetes resource metrics to use a metrics prefix of kubernetes.io/anthos/ rather than kubernetes.io/. For details refer to the metrics reference documentation.

  • Feature: Changed default etcd version to v3.4.21 on new clusters for improved stability. Existing clusters upgraded to this version will use etcd v3.5.6.

  • Feature: Improved node resource management by reserving resources for the kubelet. While this feature is crucial for preventing Out of Memory (OOM) errors by ensuring system and Kubernetes processes have the resources they need, it may lead to workload disruptions. The reservation of resources for the kubelet may affect the available resources for Pods, potentially affecting the capacity of smaller nodes to handle existing workloads. Customers should verify that smaller nodes can still support their workloads with this new feature activated.

    • The reserved memory percentages are as follows:
    • 255 MiB for machines with less than 1GB of memory
    • 25% of the first 4GB of memory
    • 20% of the next 4GB
    • 10% of the next 8GB
    • 6% of the next 112GB
    • 2% of any memory above 128GB
    • The reserved CPU percentages are as follows:
    • 6% of the first core
    • 1% of the next core
    • 0.5% of the next 2 cores
    • 0.25% of any cores above 4 cores

  • Security Fixes

Kubernetes 1.26

1.26.14-gke.1500

Kubernetes OSS release notes

1.26.13-gke.400

Kubernetes OSS release notes

1.26.12-gke.100

Kubernetes OSS release notes

1.26.10-gke.600

Kubernetes OSS release notes

1.26.9-gke.700

Kubernetes OSS release notes

1.26.8-gke.200

Kubernetes OSS release notes

1.26.7-gke.500

Kubernetes OSS release notes

1.26.5-gke.1400

Kubernetes OSS release notes

1.26.5-gke.1200

Kubernetes OSS release notes

1.26.4-gke.2200

Kubernetes OSS release notes

  • Bug Fixes

    • Fixed an issue where Kubernetes would incorrectly apply the default StorageClass to PersistentVolumeClaims which have the deprecated annotation volume.beta.kubernetes.io/storage-class.
    • Fixed an issue in which the logging agent consumed increasingly high amounts of memory.

  • Security Fixes

    • Fixed CVE-2023-1872.
    • Fixed an issue affecting netfilter connection tracking (conntrack), which is responsible for monitoring network connections. The fix ensures proper insertion of new connections into the conntrack table and overcomes the limitations caused by changes made to Linux kernel versions 5.15 and higher.

1.26.2-gke.1001

Kubernetes OSS release notes

  • Known Issue: Kubernetes 1.26.2 will incorrectly apply the default StorageClass to PersistentVolumeClaims which have the deprecated annotation volume.beta.kubernetes.io/storage-class.

  • Feature: Updated OS image to Ubuntu 22.04. cgroupv2 is now used as the default control group configuration.

    • Ubuntu 22.04 uses cgroupv2 by default. We recommend that you check if any of your applications access the cgroup filesystem. If they do, they must be updated to use cgroupv2. Some example applications that might require updates to ensure compatibility with cgroupv2 are:
    • Third-party monitoring and security agents that depend on the cgroup filesystem.
    • If cAdvisor is being used as a stand-alone DaemonSet for monitoring Pods and containers, it should be updated to version v0.43.0 or later.
    • If you are using JDK, we recommend that you use version 11.0.16 and later, or version 15 and later. These versions fully support cgroupv2.
    • If you are using the uber-go/automaxprocs package, make sure to use version v1.5.1 or higher.
    • For more information, see the Ubuntu release notes

  • Feature: Sends metrics for control plane components to Cloud Monitoring. This includes a subset of the Prometheus metrics from kube-apiserver, etcd, kube-scheduler, kube-controller-manager. Metrics names use the prefix kubernetes.io/anthos/.

  • Feature: Enabled sending Kubernetes resource metadata to Google Cloud Platform, improving both the user interface and cluster metrics. For the metadata to be ingested properly, customers need to enable the Config Monitoring for Ops API. This API can be enabled either in the Google Cloud Console , or by manually enabling the opsconfigmonitoring.googleapis.com API in the gcloud CLI. Additionally, customers must follow the steps outlined in the Authorize Cloud Logging/Monitoring documentation to add the necessary IAM bindings. If applicable, add opsconfigmonitoring.googleapis.com to your Proxy Allowlist.

  • Feature: Enabled kubelet graceful node shutdown. Non-system Pods are given 15 seconds to terminate, after which system Pods (with the system-cluster-critical or system-node-critical priority classes) have 15 seconds to gracefully terminate.

  • Feature: Enabled Node auto repair feature in preview mode. Please contact your account team to opt into the preview.

  • Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.

  • Security Fixes:

Kubernetes 1.25

1.25.14-gke.700

Kubernetes OSS release notes

1.25.13-gke.200

Kubernetes OSS release notes

1.25.12-gke.500

Kubernetes OSS release notes * Feature: Expanded the list of metrics collected from node pools to include gke-metrics-agent, cilium-agent, cilium-operator, coredns, fluentbit-gke, kubelet, and konnectivity-agent.

1.25.10-gke.1400

Kubernetes OSS release notes

1.25.10-gke.1200

Kubernetes OSS release notes

  • Security Fixes
    • Migrated node pool metrics agent and metrics server to authenticated kubelet port.

1.25.8-gke.500

Kubernetes OSS release notes

  • Bug Fixes

    • Fixed an issue in which the logging agent consumed increasingly high amounts of memory.

  • Security Fixes

1.25.7-gke.1000

Kubernetes OSS release notes

  • Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.

  • Security Fixes

1.25.6-gke.1600

Kubernetes OSS release notes

1.25.5-gke.2000

Kubernetes OSS release notes

  • Feature: Updated Anthos Identity Service to better handle concurrent authentication webhook requests.

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.

  • Bug Fix: Fixed an issue where authentication through the Anthos Service Mesh dashboard failed due to inability to impersonate end user.

  • Security Fixes

1.25.5-gke.1500

Kubernetes OSS release notes

  • Known Issue: Some UI surfaces in Google Cloud console can't authorize to the cluster and might display the cluster as unreachable. A workaround is to manually apply RBAC permitting user impersonation. For details, see Troubleshooting.

  • Security Fixes

1.25.4-gke.1300

Kubernetes OSS release notes

Kubernetes 1.24

1.24.14-gke.2700

Kubernetes OSS release notes

1.24.14-gke.1400

Kubernetes OSS release notes

1.24.13-gke.500

Kubernetes OSS release notes

  • Bug Fixes

    • Fixed an issue in which the logging agent consumed increasingly high amounts of memory.

  • Security Fixes

1.24.11-gke.1000

Kubernetes OSS release notes

  • Bug Fixes: Newly-created clusters now use etcd v3.4.21 for improved stability. Existing clusters of previous versions were already using etcd v3.5.x and will not be downgraded to v3.4.21 during cluster upgrade; these clusters will instead use v3.5.6.

  • Security Fixes

1.24.10-gke.1200

Kubernetes OSS release notes

  • Bug Fix: Fixed an issue that could cause cluster upgrades to fail if certain types of validating admission webhooks are registered.
  • Bug Fix: Fixed Cilium security ID propagation so that IDs are properly passed in the tunnel header when requests are forwarded to Services of type NodePort and LoadBalancer.
  • Security Fixes

1.24.9-gke.2000

Kubernetes OSS release notes

  • Feature: Updated Anthos Identity Service to better handle concurrent authentication webhook requests.

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.

  • Security Fixes

1.24.9-gke.1500

Kubernetes OSS release notes

1.24.8-gke.1300

Kubernetes OSS release notes

1.24.5-gke.200

Kubernetes OSS release notes

1.24.3-gke.2100

Kubernetes OSS release notes

  • Feature: Disable profiling endpoint (/debug/pprof) by default in kube-scheduler and kube-controller-manager.
  • Feature: Update kube-apiserver and kubelet to only use Strong Cryptographic Ciphers.
  • Feature: go1.18 stops accepting certificates signed with the SHA-1 hash algorithm by default. Admission/conversion webhooks or aggregated server endpoints using these insecure certificates will break by default in 1.24. The environment variable GODEBUG=x509sha1=1 is set in Anthos on-Azure clusters as a temporary workaround to let these insecure certificates continue to work. However, the go team is anticipated to remove support on this workaround in the near coming releases. Customers should check and ensure there aren't any admission/conversion webhooks or aggregated server endpoints that are using such insecure certificates before upgrading to the upcoming breaking version.
  • Feature: Improve network connectivity checks during cluster and node pool creation to help troubleshooting.
  • Feature: Upload Kubernetes resource metrics to Google Cloud Monitoring for Windows node pools.
  • Feature: Deploy Daemonset azure-cloud-node-manager with kubelet credentials to complete node initialization.
  • Feature: Update kubelet to apply external Azure cloud provider.

  • Feature: Upload workload metrics using Google Managed Service for Prometheus to Cloud Monarch is available as invite only private preview.

  • Security Fixes

Kubernetes 1.23

1.23.16-gke.2800

Kubernetes OSS release notes

1.23.16-gke.200

Kubernetes OSS release notes

  • Bug Fix: Fixed an issue where certain errors were not propagated and reported during cluster create/update operations.

  • Bug Fix: Fixed cpp-httplib issues with kubeapi server unable to reach AIS.

  • Security Fixes

1.23.14-gke.1800

Kubernetes OSS release notes

1.23.14-gke.1100

Kubernetes OSS release notes

1.23.11-gke.300

Kubernetes OSS release notes

1.23.9-gke.2100

Kubernetes OSS release notes

1.23.9-gke.800

Kubernetes OSS release notes

1.23.8-gke.1700

Kubernetes OSS release notes

1.23.7-gke.1300

Kubernetes OSS release notes

  • Feature: Source code of Azuredisk available at https://console.cloud.google.com/storage/browser/gke-multi-cloud-api-release/azuredisk-csi-driver
  • Feature: Source code of Azurefile available at https://console.cloud.google.com/storage/browser/gke-multi-cloud-api-release/azurefile-csi-driver
  • Feature: Disable profiling endpoint (/debug/pprof) by default in kube-scheduler and kube-controller-manager.

  • Feature: Update kube-apiserver and kubelet to only use Strong Cryptographic Ciphers. Supported Ciphers used by Kubelet:

    TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256

    Supported Ciphers used by kube api-server:

    TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384

  • Security Fixes

Kubernetes 1.22

1.22.15-gke.100

Kubernetes OSS release notes

1.22.12-gke.2300

Kubernetes OSS release notes

1.22.12-gke.1100

Kubernetes OSS release notes

1.22.12-gke.200

Kubernetes OSS release notes

1.22.10-gke.1500

Kubernetes OSS release notes

1.22.8-gke.2100

Kubernetes OSS release notes

  • Feature: Windows nodes now use pigz to improve image layer extraction performance.

1.22.8-gke.1300

  • Feature: You cannot create new clusters with this version, or upgrade existing clusters to this version. However existing clusters or node pools at this version will continue working, and can be upgraded to a later version.

  • Bug Fixes

    • Fixed an issue where addons cannot be applied when Windows nodepools are enabled.
    • Fixed an issue where logging agent could fill up attached disk space.

  • Security Fixes

    • Fixed CVE-2022-1055.
    • Fixed CVE-2022-0886.
    • Fixed CVE-2022-0492.
    • Fixed CVE-2022-24769.
    • This release includes the following Role-based access control (RBAC) changes:
    • Scoped down anet-operator permissions for Lease update.
    • Scoped down anetd Daemonset permissions for Nodes and pods.
    • Scoped down fluentbit-gke permissions for service account tokens.
    • Scoped down gke-metrics-agent for service account tokens.
    • Scoped down coredns-autoscaler permissions for Nodes, ConfigMaps and Deployments.

1.22.8-gke.200

Kubernetes OSS release notes

  • Feature: You cannot create new clusters with this version, or upgrade existing clusters to this version. However existing clusters or node pools at this version will continue working, and can be upgraded to a later version.

  • Feature: When you create a new cluster using Kubernetes version 1.22, you can now configure custom logging parameters.

  • Feature: As a preview feature, you can now choose Windows as your node pool image type when you create node pools with Kubernetes version 1.22.

  • Feature: You can now view most common asynchronous cluster and nodepool boot errors in the long running operation error field. For more information, see the gcloud container azure operations list reference documentation.

  • Bug Fixes

    • GKE Connect Agent now correctly reads and applies the cluster's proxy settings.

  • Security Fixes

Kubernetes 1.21

1.21.14-gke.2900

Kubernetes OSS release notes

1.21.14-gke.2100

Kubernetes OSS release notes

1.21.11-gke.1900

Kubernetes OSS release notes

1.21.11-gke.1800

Kubernetes OSS release notes

1.21.11-gke.1100

You cannot create new clusters with this version, or upgrade existing clusters to this version. However existing clusters or node pools at this version will continue working, and can be upgraded to a later version.

  • Security Fixes
    • Fixed CVE-2022-1055.
    • Fixed CVE-2022-0886.
    • Fixed CVE-2022-0492.
    • Fixed CVE-2022-24769.
    • RBAC fixes:
    • Scoped down anet-operator permissions for Lease update.
    • Scoped down anetd Daemonset permissions for Nodes and pods.
    • Scoped down fluentbit-gke permissions for service account tokens.
    • Scoped down gke-metrics-agent for service account tokens.
    • Scoped down coredns-autoscaler permissions for Nodes, ConfigMaps and Deployments.

1.21.11-gke.100

Kubernetes OSS release notes * Feature: You cannot create new clusters with this version, or upgrade existing clusters to this version. However existing clusters or node pools at this version will continue working, and can be upgraded to a later version. * Bug Fixes * GKE Connect Agent now correctly reads and applies the cluster's proxy settings.

1.21.6-gke.1500

Kubernetes OSS release notes

Security Fixes - Fixed CVE-2021-4154, see GCP-2022-002 for more details. - Fixed CVE-2022-0185, see GCP-2022-002 for more details. - Fixed CVE-2021-4034, see GCP-2022-004 for more details. - Fixed CVE-2021-43527, see GCP-2022-005 for more details.

1.21.5-gke.2800

Kubernetes OSS release notes