Using Cloud DNS for GKE

Standard

---

This page explains how to use Cloud DNS as a Kubernetes DNS provider for Google Kubernetes Engine (GKE) Standard clusters.

Using Cloud DNS as a DNS provider does not enable clients outside of a cluster to resolve and reach Kubernetes Services directly. You still need to expose Services externally using a Load Balancer and register their cluster external IP addresses on your DNS infrastructure.

For more information about using kube-dns as a DNS provider, see Service discovery and DNS. To learn how to use a custom version of kube-dns or a custom DNS provider, see Setting up a custom kube-dns Deployment.

How Cloud DNS for GKE works

Cloud DNS can be used as the DNS provider for GKE, providing Pod and Service DNS resolution with managed DNS that does not require a cluster-hosted DNS provider. DNS records for Pods and Services are automatically provisioned in Cloud DNS for cluster IP, headless and external name Services.

Cloud DNS supports the full Kubernetes DNS specification and provides resolution for A, AAAA, SRV and PTR records for Services within a GKE cluster. PTR records are implemented using response policy rules.

Using Cloud DNS as the DNS provider for GKE offers many benefits over cluster-hosted DNS:

• Removes overhead of managing the cluster-hosted DNS server. Cloud DNS requires no scaling, monitoring, or managing of DNS instances because it is a fully managed service hosted in the highly scalable Google infrastructure.
• Local resolution of DNS queries on each GKE node. Similar to NodeLocal DNSCache, Cloud DNS caches DNS responses locally, providing low latency and high scalability DNS resolution.
• Integration with Google Cloud's operations suite for DNS monitoring and logging. For more information, see Enabling and disabling logging for private managed zones.

Architecture

When Cloud DNS is the DNS provider for GKE, a controller runs as a GKE-managed Pod. This Pod runs on the control plane nodes of your cluster and syncs the cluster DNS records into a managed private DNS zone.

The following diagram shows how the Cloud DNS control plane and data plane resolve cluster names:

Diagram: Resolving cluster names

In the diagram, the Service backend selects the running backend Pods. The clouddns-controller creates a DNS record for Service backend.

The Pod frontend sends a DNS request to resolve the IP address of the Service named backend to the Compute Engine local metadata server at 169.254.169.254. The metadata server runs locally on the node, sending cache misses to Cloud DNS.

The Cloud DNS data plane runs locally within each GKE node or Compute Engine virtual machine (VM) instance. Depending on the type of Kubernetes Service, Cloud DNS resolves the Service name to its virtual IP address, for Cluster IP Services, or the list of endpoint IP addresses, for Headless Services.

After the Pod frontend resolves the IP address, the Pod can send traffic to the Service backend and any Pods behind the Service.

Note: kube-dns continues to run on new and existing GKE clusters that have Cloud DNS enabled on versions 1.19 and later. You can disable kube-dns by scaling the kube-dns Deployment and autoscaler to zero.

DNS scopes

Cloud DNS has two kinds of DNS scopes, GKE cluster scope and Virtual Private Cloud (VPC) scope. A cluster cannot operate in both modes simultaneously.

• GKE cluster scope: DNS records are only resolvable within the cluster, which is the same behavior as kube-dns. Only nodes running in the GKE cluster can resolve Service names. By default, clusters have DNS names that end in *.cluster.local. These DNS names are only visible within the cluster and do not overlap or conflict with *.cluster.local DNS names for other GKE clusters in the same project. This is the default mode.
• VPC scope: DNS records are resolvable within the entire VPC. Compute Engine VMs and on-premise clients can connect using Cloud Interconnect or Cloud VPN and directly resolve GKE Service names. You must set a unique custom domain for each cluster, which means that all Service and Pod DNS records are unique within the VPC. This mode reduces communication friction between GKE and non-GKE resources.

Cloud DNS resources

When you use Cloud DNS as your DNS provider for your GKE cluster, the Cloud DNS controller creates resources in Cloud DNS for your project. The resources that GKE creates depends on the Cloud DNS scope.

Scope	Forward lookup zone	Reverse lookup zone
Cluster scope	1 private zone per cluster per Compute Engine zone (in the region)	1 response policy zone per cluster per Compute Engine zone (in the region)
VPC scope	1 private zone per cluster (global zone)	1 response policy zone per cluster (global zone)

The naming convention used for these Cloud DNS resources is the following:

Scope	Forward lookup zone	Reverse lookup zone
Cluster scope	gke-CLUSTER_NAME-CLUSTER_HASH-dns	gke-CLUSTER_NAME-CLUSTER_HASH-rp
VPC scope	gke-CLUSTER_NAME-CLUSTER_HASH-dns	gke-NETWORK_NAME_HASH-rp

In addition to the zones mentioned in the previous table, the Cloud DNS controller creates the following zones in your project, depending on your configuration:

Custom DNS configuration	Zone type	Zone naming convention
Stub domain	Forwarding (global zone)	gke-CLUSTER_NAME-CLUSTER_HASH-DOMAIN_NAME_HASH
Custom upstream nameserver(s)	Forwarding (global zone)	gke-CLUSTER_NAME-CLUSTER_HASH-upstream

To learn more about how to create custom stub domains or custom upstream nameservers, see Adding custom resolvers for stub domains.

Managed zones and forwarding zones

To serve internal DNS traffic, the Cloud DNS controller creates a managed DNS zone in each Compute Engine zone of the region the cluster belongs to. For example, if you deploy a cluster in the us-central1-c zone, the Cloud DNS controller creates a managed zone in us-central1-a, us-central1-b, us-central1-c, and us-central1-f.

For each DNS stubDomain, the Cloud DNS controller creates one forwarding zone.

The Cloud DNS processes each DNS upstream using one managed zone with the . DNS name.

Pricing

When Cloud DNS is the DNS provider for GKE, DNS queries from Pods inside the GKE cluster are billed according to Cloud DNS pricing.

Queries to a VPC scoped DNS zone managed by GKE are billed using the standard Cloud DNS pricing.

Requirements

Cloud DNS for GKE has the following requirements for cluster scope:

• For Standard, GKE version 1.24.7-gke.800, 1.25.3-gke.700 or later.
• Google Cloud CLI version 411.0.0 or later.
• The Cloud DNS API must be enabled in your project.

Cloud DNS for GKE has the following requirements for VPC scope:

• For Standard, GKE version 1.19 or later.
• Google Cloud CLI version 364.0.0 or later.
• The Cloud DNS API must be enabled in your project.

Restrictions and limitations

The following limitations apply:

• Cloud DNS for GKE is not supported with Autopilot. You must not change the DNS provider for an Autopilot cluster from kube-dns to Cloud DNS. The gcloud container clusters update command does not enforce this restriction.
• Manual changes to the managed private DNS zones are not supported and are overwritten by the Cloud DNS controller. Modifications to the DNS records in those zones do not persist when the controller restarts.
• After you enable Cloud DNS for GKE in a cluster, kube-dns continues to run in the cluster. You can disable kube-dns by scaling the kube-dns Deployment and autoscaler to zero.
• You cannot change the DNS scope in a cluster after you have set the scope with the --cluster-dns-scope flag. If you need to change the DNS scope, recreate the cluster with a different DNS scope.
• Custom stub domains and upstream DNS server configurations apply to the DNS configurations of Pods and nodes. Pods using host networking or processes that run directly on the host also use the stub domain and upstream nameserver configurations.
• Custom stub domains and upstream nameservers configured through the kube-dns Configmap are automatically applied to Cloud DNS for cluster scope DNS. VPC scope DNS ignores the kube-dns ConfigMap and you must apply these configurations directly on Cloud DNS.
• For VPC scope, the secondary IP address range for Services must not be shared with any other clusters in that subnetwork.
• For VPC scope, the response policy associated with a PTR record is attached to the VPC network. If there are any other response policies bound to the cluster network, PTR record resolution fails for Kubernetes service IP addresses.
• If you try to create a headless Service with a number of Pods exceeding the allowed quota, Cloud DNS does not create record sets or records for the Service.

Quotas

Cloud DNS uses quotas to limit number of resources that GKE can create for DNS entries. Quotas and limits for Cloud DNS might be different from the limitations of kube-dns for your project.

The following default quotas are applied to each managed zone in your project when using Cloud DNS for GKE:

Kubernetes DNS resource	Corresponding Cloud DNS resource	Quota
Number of DNS records	Max bytes per managed zone	2,000,000 (50MB max for a managed zone)
Number of Pods per headless Service (IPv4/IPv6)	Number of records per resource record set	1,000
Number of GKE clusters in a project	Number of response policies per project	100
Number of PTR records per cluster	Number of rules per response policy	100,000

Note: For VPC scope, the same limits apply to the managed zone used for the entire VPC. These quotas apply to the set of clusters using Cloud DNS VPC scope and running in the same VPC.

Resource limits

The Kubernetes resources that you create per cluster contribute to Cloud DNS resource limits, as described in the following table:

Limit	Contribution to limit
Resource record sets per managed zone	Number of services plus number of headless service endpoints with valid hostnames, per cluster.
Records per resource record set	Number of endpoints per headless service. Does not impact other service types.
Number of rules per response policy	For cluster scope, number of services plus number of headless service endpoints with valid hostnames per cluster. For VPC scope, number of services plus number of headless endpoints with hostnames from all clusters in the VPC.

To learn more about how DNS records are created for Kubernetes, see Kubernetes DNS-Based Service Discovery.

Before you begin

Before you start, make sure you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI.
  Note: For existing gcloud CLI installations, make sure to set the compute/region and compute/zone properties. By setting default locations, you can avoid errors in gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location.

• Enable the Cloud DNS API in your project:

  Enable Cloud DNS API

Enable cluster scope DNS

In cluster scope DNS, only nodes running in the GKE cluster can resolve service names, and service names do not conflict between clusters. This behavior is the same as kube-dns in GKE clusters, which means that you can migrate clusters from kube-dns to Cloud DNS cluster scope without downtime or changes to your applications.

The following diagram shows how Cloud DNS creates a private DNS zone for a GKE cluster. Only processes and Pods running on the nodes in the cluster can resolve the cluster's DNS records, because only the nodes are in the DNS scope.

Diagram: Cluster scope DNS

Enable cluster scope DNS in a new cluster

You can create a cluster with cluster scope Cloud DNS enabled using the gcloud CLI or the Google Cloud console:

gcloud

Create a cluster using the --cluster-dns flag:

gcloud container clusters create CLUSTER_NAME \
    --cluster-dns=clouddns \
    --cluster-dns-scope=cluster \
    --region=COMPUTE_REGION

Note: For Google Cloud CLI versions earlier than 411.0.0, use gcloud beta container instead of gcloud container.

Replace the following:

• CLUSTER_NAME: the name of the cluster.
• COMPUTE_REGION: the Compute Engine region for the new cluster. For zonal clusters, use --zone=COMPUTE_ZONE.

The --cluster-dns-scope=cluster flag is optional in the command because cluster is the default value.

Console

1. Go to the Google Kubernetes Engine page in the Google Cloud console.

   Go to Google Kubernetes Engine

2. Click Create.

3. From the navigation pane, under Cluster, click Networking.

4. In the DNS provider section, click Cloud DNS.

5. Select Cluster scope.

6. Configure your cluster as needed.

7. Click Create.

Enable cluster scope DNS in an existing cluster

You can migrate an existing GKE cluster from kube-dns to Cloud DNS cluster scope using the gcloud CLI or the Google Cloud console.

When you migrate an existing cluster, the nodes in the cluster do not use Cloud DNS as a DNS provider until you recreate the nodes.

After you enable Cloud DNS for a cluster, the settings only apply if you upgrade existing node pools or you add new node pools to the cluster. When you upgrade a node pool, the nodes are recreated.

You can also migrate clusters that have running applications without interrupting cluster communication by enabling Cloud DNS as a DNS provider in each node pool separately. A subset of the nodes are operational at all times because some node pools use kube-dns and some node pools use Cloud DNS.

In the following steps, you enable Cloud DNS for a cluster and then upgrade your node pools. Upgrading your node pools recreates the nodes. The nodes then use Cloud DNS for DNS resolution instead of kube-dns.

gcloud

1. Update the existing cluster:

   gcloud container clusters update CLUSTER_NAME \
       --cluster-dns=clouddns \
       --cluster-dns-scope=cluster \
       --region=COMPUTE_REGION
   

   Replace the following:

   • CLUSTER_NAME: the name of the cluster.
   • COMPUTE_REGION: the Compute Engine region for your cluster. For zonal clusters, use --zone=COMPUTE_ZONE.

   The response is similar to the following:

   All the node-pools in the cluster need to be re-created by the user to start using Cloud DNS for DNS lookups. It is highly recommended to complete this step
   shortly after enabling Cloud DNS.
   Do you want to continue (Y/n)?
   

   After you confirm, the Cloud DNS controller runs on the GKE control plane, but your Pods do not use Cloud DNS for DNS resolution until you upgrade your node pool or you add new node pools to the cluster.

2. Upgrade the node pools in the cluster to use Cloud DNS:

   gcloud container clusters upgrade CLUSTER_NAME \
       --node-pool=POOL_NAME \
       --region=COMPUTE_REGION
   

   Note: For Google Cloud CLI versions earlier than 411.0.0, use gcloud beta container instead of gcloud container.

   Replace the following:

   • CLUSTER_NAME: the name of the cluster.
   • POOL_NAME: the name of the node pool to upgrade.

   If the node pool and control plane are running the same version, upgrade the control plane first, as described in Manually upgrading the control plane and then perform the node pool upgrade.

   Note: Upgrading to the same version that the node pool currently runs on, has no effect. You must upgrade to a new version. Alternatively, you can create a new node pool in a new version.

   Confirm the response and repeat this command for each node pool in the cluster. If your cluster has one node pool, omit the --node-pool flag.

Console

1. Go to the Google Kubernetes Engine page in the Google Cloud console.

   Go to Google Kubernetes Engine

2. Click the name of the cluster you want to modify.

3. Under Networking, in the DNS provider field, click Edit DNS provider.

4. Click Cloud DNS.

5. Click Cluster scope.

6. Click Save changes.

Enable VPC scope DNS

In VPC scope DNS, a cluster's DNS names are resolvable within the entire VPC. Any client in the VPC can resolve cluster DNS records.

VPC scope DNS enables the following use cases:

• Headless service discovery for non-GKE clients within the same VPC.
• GKE Service resolution from on-premise or 3rd party cloud clients. For more information, see Inbound server policy.
• Service resolution where a client can decide which cluster they want to communicate with using the custom cluster DNS domain.

In the following diagram, two GKE clusters use VPC scope DNS in the same VPC. Both clusters have a custom DNS domain, .cluster1 and .cluster2, instead of the default .cluster.local domain. A VM communicates with the headless backend Service by resolving backend.default.svc.cluster1. Cloud DNS resolves the headless Service to the individual Pod IPs in the Service and the VM communicates directly with the Pod IPs.

Note: Only headless Services resolve to IP addresses that can be contacted outside of the GKE cluster. ClusterIP Services use a virtual IP address that only exists locally in the cluster.

Diagram: VPC scope DNS

You can also perform this type of resolution from other networks when connected to the VPC through Cloud Interconnect or Cloud VPN. DNS server policies enable clients from networks connected to the VPC to resolve names in Cloud DNS, which includes GKE Services if the cluster is using VPC scope DNS.

Enable VPC scope DNS in a new cluster

You can enable VPC scope DNS in a new cluster using the gcloud CLI or the Google Cloud console.

Warning: If you specify a cluster-dns-domain other than cluster.local, all *.cluster.local queries for Kubernetes services fail. Ensure that you do not perform DNS lookups for names that use *.cluster.local.

gcloud

You can create a new cluster with VPC scope Cloud DNS enabled using the following command:

gcloud container clusters create CLUSTER_NAME \
    --cluster-dns=clouddns \
    --cluster-dns-scope=vpc \
    --cluster-dns-domain=CUSTOM_DOMAIN \
    --region=COMPUTE_REGION

Note: For Google Cloud CLI versions earlier than 364.0.0, use gcloud beta container instead of gcloud container.

Replace the following:

• CLUSTER_NAME: the name of the cluster.
• COMPUTE_REGION: the Compute Engine region for the new cluster. For zonal clusters, use --zone=COMPUTE_ZONE.
• CUSTOM_DOMAIN: the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.

Console

1. Go to the Google Kubernetes Engine page in the Google Cloud console.

   Go to Google Kubernetes Engine

2. Click Create.

3. From the navigation pane, under Cluster, click Networking.

4. In the DNS provider section, click Cloud DNS.

5. Select VPC scope.

6. Enter the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.

7. Configure your cluster as needed.

8. Click Create.

Enable VPC scope DNS in an existing cluster

You can migrate an existing GKE cluster from kube-dns to Cloud DNS VPC scope using the gcloud CLI or the Google Cloud console.

When you migrate an existing cluster, the nodes in the cluster do not use Cloud DNS as a DNS provider until you recreate the nodes.

After you enable Cloud DNS for a cluster, the settings only apply if you upgrade existing node pools or you add new node pools to the cluster. When you upgrade a node pool, the nodes are recreated.

You can also migrate clusters that have running applications without interrupting cluster communication by enabling Cloud DNS as a DNS provider in each node pool separately. A subset of the nodes are operational at all times because some node pools use kube-dns and some node pools use Cloud DNS.

In the following steps, you enable Cloud DNS for a cluster and then upgrade your node pools. Upgrading your node pools recreates the nodes. The nodes then use Cloud DNS for DNS resolution instead of kube-dns.

gcloud

1. Update the existing cluster:

   gcloud container clusters update CLUSTER_NAME \
       --cluster-dns=clouddns \
       --cluster-dns-scope=vpc \
       --cluster-dns-domain=CUSTOM_DOMAIN \
       --region=COMPUTE_REGION
   

   Replace the following:

   • CLUSTER_NAME: the name of the cluster.
   • COMPUTE_REGION: the Compute Engine region for the new cluster. For zonal clusters, use --zone=COMPUTE_ZONE.
   • CUSTOM_DOMAIN: the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.

   The response is similar to the following:

   All the node-pools in the cluster need to be re-created by the user to start using Cloud DNS for DNS lookups. It is highly recommended to complete this step
   shortly after enabling Cloud DNS.
   Do you want to continue (Y/n)?
   

   After you confirm, the Cloud DNS controller runs on the GKE control plane. Your Pods do not use Cloud DNS for DNS resolution until you upgrade your node pool or you add new node pools to the cluster.

2. Upgrade the node pools in the cluster to use Cloud DNS:

   gcloud container clusters upgrade CLUSTER_NAME \
       --node-pool=POOL_NAME
   

   Replace the following:

   • CLUSTER_NAME: the name of the cluster.
   • POOL_NAME: the name of the node pool to upgrade.

   If the node pool and control plane are running the same version, upgrade the control plane first, as described in Manually upgrading the control plane and then perform the node pool upgrade.

   Note: Upgrading to the same version that the node pool currently runs on, has no effect. You must upgrade to a new version. Alternatively, you can create a new node pool in a new version.

   Confirm the response and repeat this command for each node pool in the cluster. If your cluster has one node pool, omit the --node-pool flag.

Console

1. Go to the Google Kubernetes Engine page in the Google Cloud console.

   Go to Google Kubernetes Engine

2. Click the name of the cluster you want to modify.

3. Under Networking, in the DNS provider field, click Edit DNS provider.

4. Click Cloud DNS.

5. Click VPC scope.

6. Click Save changes.

Verify Cloud DNS

Verify that Cloud DNS for GKE is working correctly for your cluster:

1. Verify that your nodes are using Cloud DNS by connecting to a Pod on a node and running the command cat /etc/resolv.conf:

   kubectl exec -it POD_NAME -- cat /etc/resolv.conf | grep nameserver
   

   Replace POD_NAME with the name of the Pod.

   The output is similar to the following:

   nameserver 169.254.169.254
   

   If the output is an IP address similar to 10.x.y.10, then the Pod is using kube-dns. If the output is 169.254.20.10, then the Pod is using NodeLocal DNSCache. 169.254.169.254 is the IP address of the metadata server where the Cloud DNS data plane listens for requests on port 53. The nodes no longer use the kube-dns Service address for DNS resolution and all DNS resolution occurs on the local node.

2. Deploy a sample application to your cluster:

   kubectl run dns-test --image us-docker.pkg.dev/google-samples/containers/gke/hello-app:2.0
   

3. Expose the sample application with a Service:

   kubectl expose pod dns-test --name dns-test-svc --port 8080
   

4. Verify that the Service deployed successfully:

   kubectl get svc dns-test-svc
   

   The output is similar to the following:

   NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
   dns-test-svc   ClusterIP   10.47.255.11    <none>        8080/TCP   6m10s
   

   The value of CLUSTER-IP is the virtual IP address for your cluster. In this example, the virtual IP address is 10.47.255.11.

5. Verify that your Service name was created as a record in the private DNS zone for your cluster:

   gcloud dns record-sets list \
       --zone=PRIVATE_DNS_ZONE \
       --name=dns-test-svc.default.svc.cluster.local.
   

   Replace PRIVATE_DNS_ZONE with the name of the managed DNS zone.

   The output is similar to the following:

   NAME: dns-test-svc.default.svc.cluster.local.
   TYPE: A
   TTL: 30
   DATA: 10.47.255.11
   

Disable Cloud DNS

You can disable Cloud DNS using the gcloud CLI or the Google Cloud console:

gcloud

Update the cluster to use kube-dns:

gcloud container clusters update CLUSTER_NAME \
    --cluster-dns=default

Console

1. Go to the Google Kubernetes Engine page in the Google Cloud console.

   Go to Google Kubernetes Engine

2. Click the name of the cluster you want to modify.

3. Under Networking, in the DNS provider field, click Edit DNS provider.

4. Click Kube-dns.

5. Click Save changes.

Clean up

After completing the exercises on this page, follow these steps to remove resources and prevent unwanted charges incurring on your account:

1. Delete the service:

   kubectl delete service dns-test-svc
   

2. Delete the Pod:

   kubectl delete Pod dns-test
   

3. You can also delete the cluster.

Use Cloud DNS with Shared VPC

Cloud DNS for GKE supports Shared VPC for both VPC and cluster scope.

The GKE controller creates a managed private zone in the same project as the GKE cluster.

The GKE service account for the GKE cluster does not require Identity and Access Management (IAM) for DNS outside of its own project because the managed zone and GKE cluster reside within the same project.

More than one cluster per service project

Starting in GKE versions 1.22.3-gke.700, 1.21.6-gke.1500, you can create clusters in multiple service projects referencing a VPC in the same host project. If you already have clusters using Shared VPC and Cloud DNS VPC scope, you must manually migrate them with the following steps:

• Upgrade your existing clusters that have Shared VPC enabled to GKE version 1.22.3-gke.700+ or 1.21.6-gke.1500+.
• Migrate the response policy from the service project to the host project. You only need to perform this migration once per Shared VPC network.

You can migrate your response policy using the Google Cloud console.

Perform the following steps in your service project:

1. Go to the Cloud DNS zones page.

   Go to Cloud DNS zones

2. Click the Response policy zones tab.

3. Click the response policy for your VPC network. You can identify the response policy by the description, which is similar to "Response policy for GKE clusters on network NETWORK_NAME."

4. Click the In use by tab.

5. Next to the name of your host project, click to remove the network binding.

6. Click the Response policy rules tab.

7. Select all of the entries in the table.

8. Click Remove response policy rules.

9. Click Delete response policy.

After you delete the response policy, the DNS controller creates the response policy in the host project automatically. Clusters from other service projects share this response policy.

Support custom stub domains and upstream nameservers

Cloud DNS for GKE supports custom stub domains and upstream nameservers configured using kube-dns ConfigMap. This support is only available for GKE cluster scope.

Cloud DNS translates stubDomains and upstreamNameservers values into Cloud DNS forwarding zones.

Note: The custom DNS configuration applies to both nodes and Pods in the cluster. This behavior is different from kube-dns, where the custom DNS configuration applies only to Pods.

Troubleshooting

To learn how to enable DNS logging, see Enabling and disabling logging for private managed zones.

For more information about troubleshooting DNS issues, see Troubleshooting DNS in GKE.

Unable to update existing cluster or create cluster with Cloud DNS enabled

Ensure you are using the correct version. Cloud DNS for GKE requires GKE version 1.19 or later for clusters using VPC scope, or GKE version 1.24.7-gke.800, 1.25.3-gke.700 or later for clusters using cluster scope.

Pods use kube-dns even after Cloud DNS is enabled on an existing cluster

Ensure you have upgraded or recreated your node pools after you enable Cloud DNS on the cluster. Until this step is complete, Pods continue to use kube-dns.

Pod is unable to resolve DNS lookups

1. Verify the Pod is using Cloud DNS by connecting to a Pod and running the command cat /etc/resolv.conf:

   kubectl exec -it POD_NAME -- cat /etc/resolv.conf | grep nameserver
   

   Replace POD_NAME with the name of the Pod.

   The output is similar to the following:

   nameserver 169.254.169.254
   

   If the output is an IP address similar to 10.x.y.10, then the Pod is using kube-dns. If the output is 169.254.20.10, then the Pod is using NodeLocalDNS.

2. Confirm the managed zone exists and contains the necessary DNS entry:

   gcloud dns managed-zones list --format list
   

   The output is similar to the following:

    - creationTime: 2021-02-12T19:24:37.045Z
      description: Private zone for GKE cluster "cluster1" with cluster suffix "cluster.local" in project "project-243723"
      dnsName: cluster.local.
      id: 5887499284756055830
      kind: dns#managedZone
      name: gke-cluster1-aa94c1f9-dns
      nameServers: ['ns-gcp-private.googledomains.com.']
      privateVisibilityConfig: {'kind': 'dns#managedZonePrivateVisibilityConfig'}
      visibility: private
   

   The value of name in the response shows that Google Cloud created a private zone named gke-cluster1-aa94c1f9-dns.

3. Verify that Cloud DNS contains entries for your cluster:

   gcloud dns record-sets list --zone ZONE_NAME | grep SERVICE_NAME
   

   Replace the following:

   • ZONE_NAME: the name of the private zone.
   • SERVICE_NAME: the name of the service.

   The output shows that Cloud DNS contains an A record for the domain dns-test.default.svc.cluster.local. and the IP address of your cluster, 10.47.255.11:

   dns-test.default.svc.cluster.local.                A     30     10.47.255.11
   

4. Enable Cloud DNS logging to track queries. Every log entry contains information about the query, including DNS latency.

DNS lookups on nodes fail after enabling Cloud DNS on a cluster

If you enable cluster scope Cloud DNS in a GKE cluster that has custom stub domains or upstream nameservers, the custom config applies to both nodes and Pods in the cluster because Cloud DNS cannot distinguish between Pod and node DNS requests. DNS lookups on nodes might fail if the custom upstream server cannot resolve the queries.

What's next

• Read an overview of how GKE provides managed DNS.
• Read DNS for services and Pods for a general overview of how DNS is used in Kubernetes clusters.
• Learn about Scopes and hierarchies in Cloud DNS.
• Learn about Enabling and disabling logging for private managed zones.