Cannot create Google Kubernetes Engine private cluster

Problem

You tried to create a new private Google Kubernetes Engine cluster, but it failed the creation as node was unable to register with primary due to the error described below:

Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady.

Environment

  • Private Google Kubernetes Engine Cluster with Private Google Access enabled at subnet level

Solution

  1. Add the CNAME record (*.gcr.io = gcr.io) or remove the gcr.io (A record to 4 IPs used by private.googleapi.com).

Cause

You added an A record that points gcr.io to the 4 IPs used by private.googleapi.com (199.36.153.8, 199.36.153.9, 199.36.153.10, 199.36.153.11), but the Google Container Registry requests can be *.gcr.io (for example, us.gcr.io). This causes the image pull to fail for kube-system workloads, such as the Init container used to install the CNI plugin preventing a cluster from being created.

Error Reference:

[DateTime] gke-cluster-1-default-pool-271a0bbd-5gvj kubelet[1419]: E1120 06:37:48.161556 1419 kubelet.go:2195] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is ...