This documentation is for the most recent version of GKE on Azure, released in November 2021. See the Release notes for more information.

Known issues for GKE on Azure

This page lists selected known issues for GKE on Azure, and steps you can take to reduce their impact.

If you need additional assistance, reach out to Cloud Customer Care.

Operations

Cluster autoscaler might incorrectly scale up from zero nodes

Versions affected by this issue are the following:

All versions

Cluster autoscaler doesn't correctly scale up from zero nodes for node pools with custom labels or taints.

This issue occurs because the GKE on Azure cluster autoscaler didn't configure the node pool labels and taint tags on the corresponding node pool Auto Scaling Group during node pool provisioning. For node pools with zero nodes, the cluster autoscaler can't create the node templates correctly because of these missing tags. This could lead to incorrect scaling decisions, such as Pods not being scheduled to the applicable nodes, or nodes being provisioned that aren't really needed. For more information, see Auto-Discovery Setup.

Networking

Application timeouts caused by conntrack table insertion failures

Versions affected by this issue are the following:

All versions of 1.23 starting from 1.23.8-gke.1700.
All versions of 1.24 starting from 1.24.0-gke.0.
Versions of 1.25 ranging from 1.25.0-gke.0 up to, but not including, 1.25.10-gke.1200.
Versions from 1.26.0-gke.0 up to, but not including, 1.26.4-gke.2200.

Clusters running on an Ubuntu OS that uses kernel 5.15 or higher are susceptible to netfilter connection tracking (conntrack) table insertion failures. Insertion failures can occur even when the conntrack table has room for new entries. The failures are caused by changes in kernel 5.15 and higher that restrict table insertions based on chain length.

To see if you are affected by this issue, check the in-kernel connection tracking system statistics with the following command:

sudo conntrack -S

The response looks like this:

cpu=0       found=0 invalid=4 insert=0 insert_failed=0 drop=0 early_drop=0
error=0 search_restart=0 clash_resolve=0 chaintoolong=0
cpu=1       found=0 invalid=0 insert=0 insert_failed=0 drop=0 early_drop=0
error=0 search_restart=0 clash_resolve=0 chaintoolong=0
cpu=2       found=0 invalid=16 insert=0 insert_failed=0 drop=0 early_drop=0
error=0 search_restart=0 clash_resolve=0 chaintoolong=0
cpu=3       found=0 invalid=13 insert=0 insert_failed=0 drop=0 early_drop=0
error=0 search_restart=0 clash_resolve=0 chaintoolong=0
cpu=4       found=0 invalid=9 insert=0 insert_failed=0 drop=0 early_drop=0
error=0 search_restart=0 clash_resolve=0 chaintoolong=0
cpu=5       found=0 invalid=1 insert=0 insert_failed=0 drop=0 early_drop=0
error=519 search_restart=0 clash_resolve=126 chaintoolong=0

If a chaintoolong value in the response is a non-zero number, you are affected by this issue.

Solution

If you are running version 1.26.2-gke.1001, upgrade to version 1.26.4-gke.2200 or later.

Usability

Unreachable clusters detected error in UI

Versions affected by this issue are 1.25.5-gke.1500 and 1.25.4-gke.1300.

Some UI surfaces in Google Cloud console can't authorize to the cluster and might display the cluster as unreachable.

Solution

Upgrade your cluster to the latest available patch of version 1.25. This issue was fixed in version 1.25.5-gke.2000.

API errors

Kubernetes 1.22 deprecates and replaces several APIs. If you've upgraded your cluster to version 1.22 or later, any calls your application makes to one of the deprecated APIs fail.

Solution

Upgrade your application to replace the deprecated API calls with their newer counterparts.