This page explains how to use Cloud DNS as a Kubernetes DNS provider for Google Kubernetes Engine (GKE).
Using Cloud DNS as a DNS provider does not enable clients outside of a cluster to resolve and reach Kubernetes Services directly. You still need to expose Services externally using a Load Balancer and register their cluster external IP addresses on your DNS infrastructure.
For more information about using kube-dns as a DNS provider, see Service discovery and DNS. To learn how to use a custom version of kube-dns or a custom DNS provider, see Setting up a custom kube-dns Deployment.
How Cloud DNS for GKE works
Cloud DNS can be used as the DNS provider for GKE, providing Pod and Service DNS resolution with managed DNS that does not require a cluster-hosted DNS provider. DNS records for Pods and Services are automatically provisioned in Cloud DNS for cluster IP, headless and external name Services.
Cloud DNS supports the full Kubernetes DNS specification and provides resolution for A, AAAA, SRV and PTR records for Services within a GKE cluster. PTR records are implemented using response policy rules.
Using Cloud DNS as the DNS provider for GKE offers many benefits over cluster-hosted DNS:
- Removes overhead of managing the cluster-hosted DNS server. Cloud DNS requires no scaling, monitoring, or managing of DNS instances because it is a fully managed service hosted in the highly scalable Google infrastructure.
- Local resolution of DNS queries on each GKE node. Similar to NodeLocal DNSCache, Cloud DNS caches DNS responses locally, providing low latency and high scalability DNS resolution.
- Integration with Google Cloud's operations suite for DNS monitoring and logging. For more information, see Enabling and disabling logging for private managed zones.
Architecture
When Cloud DNS is the DNS provider for GKE, a controller runs as a GKE-managed Pod. This Pod runs on the control plane nodes of your cluster and syncs the cluster DNS records into a managed private DNS zone.
The following diagram shows how the Cloud DNS control plane and data plane resolve cluster names:
In the diagram, the Service backend
selects the running backend
Pods.
The clouddns-controller
creates a DNS record for Service backend
.
The Pod frontend
sends a DNS request to resolve the IP address of the Service
named backend
to the
Compute Engine local metadata server at
169.254.169.254
. The metadata server runs locally on the node, sending cache
misses to Cloud DNS.
The Cloud DNS data plane runs locally within each GKE node or Compute Engine virtual machine (VM) instance. Depending on the type of Kubernetes Service, Cloud DNS resolves the Service name to its virtual IP address, for Cluster IP Services, or the list of endpoint IP addresses, for Headless Services.
After the Pod frontend
resolves the IP address, the Pod can send traffic to
the Service backend
and any Pods behind the Service.
DNS scopes
Cloud DNS has two kinds of DNS scopes, GKE cluster scope and Virtual Private Cloud (VPC) scope. A cluster cannot operate in both modes simultaneously.
- GKE cluster scope: DNS records are
only resolvable within the cluster, which is the same behavior as kube-dns.
Only nodes running in the GKE cluster can resolve Service names.
By default, clusters have DNS names that end in
*.cluster.local
. These DNS names are only visible within the cluster and do not overlap or conflict with*.cluster.local
DNS names for other GKE clusters in the same project. This is the default mode. - VPC scope: DNS records are resolvable within the entire VPC. Compute Engine VMs and on-premise clients can connect using Cloud Interconnect or Cloud VPN and directly resolve GKE Service names. You must set a unique custom domain for each cluster, which means that all Service and Pod DNS records are unique within the VPC. This mode reduces communication friction between GKE and non-GKE resources.
Cloud DNS resources
When you use Cloud DNS as your DNS provider for your GKE cluster, the Cloud DNS controller creates resources in Cloud DNS for your project. The resources that GKE creates depends on the Cloud DNS scope.
Scope | Forward lookup zone | Reverse lookup zone |
---|---|---|
Cluster scope | 1 private zone per cluster per Compute Engine zone (in the region) | 1 response policy zone per cluster per Compute Engine zone (in the region) |
VPC scope | 1 private zone per cluster (global zone) | 1 response policy zone per cluster (global zone) |
The naming convention used for these Cloud DNS resources is the following:
Scope | Forward lookup zone | Reverse lookup zone |
---|---|---|
Cluster scope | gke-CLUSTER_NAME-CLUSTER_HASH-dns |
gke-CLUSTER_NAME-CLUSTER_HASH-rp |
VPC scope | gke-CLUSTER_NAME-CLUSTER_HASH-dns |
gke-NETWORK_NAME_HASH-rp |
In addition to the zones mentioned in the previous table, the Cloud DNS controller creates the following zones in your project, depending on your configuration:
Custom DNS configuration | Zone type | Zone naming convention |
---|---|---|
Stub domain | Forwarding (global zone) | gke-CLUSTER_NAME-CLUSTER_HASH-DOMAIN_NAME_HASH |
Custom upstream nameserver(s) | Forwarding (global zone) | gke-CLUSTER_NAME-CLUSTER_HASH-upstream |
To learn more about how to create custom stub domains or custom upstream nameservers, see Adding custom resolvers for stub domains.
Managed zones and forwarding zones
To serve internal DNS traffic, the Cloud DNS controller creates a
managed DNS zone in each
Compute Engine zone of the region the cluster belongs to. For example,
if you deploy a cluster in the us-central1-c
zone, the Cloud DNS
controller creates a managed zone in us-central1-a
, us-central1-b
,
us-central1-c
, and us-central1-f
.
For each DNS stubDomain
, the Cloud DNS controller creates one
forwarding zone.
The Cloud DNS processes each DNS upstream using one managed zone
with the .
DNS name.
Pricing
When Cloud DNS is the DNS provider for GKE Standard clusters, DNS queries from Pods inside the GKE cluster are billed according to Cloud DNS pricing.
Queries to a VPC scoped DNS zone managed by GKE are billed using the standard Cloud DNS pricing.
Requirements
Cloud DNS for GKE has the following requirements for cluster scope:
- For Standard, GKE version 1.24.7-gke.800, 1.25.3-gke.700 or later.
- For Autopilot, GKE version 1.25.9-gke.400, 1.26.4-gke.500 or later.
- Google Cloud CLI version 411.0.0 or later.
- The Cloud DNS API must be enabled in your project.
Cloud DNS for GKE has the following requirements for VPC scope:
- For Standard, GKE version 1.19 or later.
- Google Cloud CLI version 364.0.0 or later.
- The Cloud DNS API must be enabled in your project.
Restrictions and limitations
The following limitations apply:
- VPC scope is not supported on Autopilot clusters, only cluster scope is supported.
- Cloud DNS is not compliant with an Impact Level 4 (IL4) compliance regime. Cloud DNS for GKE cannot be used in an Assured Workload environment with an IL4 compliance regime. You need to use kube-dns in such regulated environment. For GKE Autopilot clusters, the selection of kube-dns or Cloud DNS is automatically done based on your compliance regime.
- Manual changes to the managed private DNS zones are not supported and are overwritten by the Cloud DNS controller. Modifications to the DNS records in those zones do not persist when the controller restarts.
- After you enable Cloud DNS for GKE in a cluster, kube-dns continues to run in the cluster. You can disable kube-dns by scaling the kube-dns Deployment and autoscaler to zero.
- You cannot change the DNS scope in a cluster after you have set
the scope with the
--cluster-dns-scope
flag. If you need to change the DNS scope, recreate the cluster with a different DNS scope. - Custom stub domains and upstream DNS server configurations apply to the DNS configurations of Pods and nodes. Pods using host networking or processes that run directly on the host also use the stub domain and upstream nameserver configurations. This is only supported in Standard.
- Custom stub domains and upstream nameservers configured through the kube-dns Configmap are automatically applied to Cloud DNS for cluster scope DNS. VPC scope DNS ignores the kube-dns ConfigMap and you must apply these configurations directly on Cloud DNS. This is only supported in Standard.
- There is no migration path from kube-dns to VPC scope, the operation is disruptive. Recreate the cluster when switching from kube-dns to VPC scope or vice versa.
- For VPC scope, the secondary IP address range for Services must not be shared with any other clusters in that subnetwork.
- For VPC scope, the response policy associated with a PTR record is attached to the VPC network. If there are any other response policies bound to the cluster network, PTR record resolution fails for Kubernetes service IP addresses.
- If you try to create a headless Service with a number of Pods exceeding the allowed quota, Cloud DNS does not create record sets or records for the Service.
- Switching from kube-dns to Cloud DNS VPC scope is a disruptive operation. We recommend that you perform this operation during a maintenance window.
Quotas
Cloud DNS uses quotas to limit number of resources that GKE can create for DNS entries. Quotas and limits for Cloud DNS might be different from the limitations of kube-dns for your project.
The following default quotas are applied to each managed zone in your project when using Cloud DNS for GKE:
Kubernetes DNS resource | Corresponding Cloud DNS resource | Quota |
---|---|---|
Number of DNS records | Max bytes per managed zone | 2,000,000 (50MB max for a managed zone) |
Number of Pods per headless Service (IPv4/IPv6) | Number of records per resource record set | GKE 1.24 to 1.25: 1,000 (IPv4/IPv6) GKE 1.26 and later: 3,500/2,000 (IPv4/IPv6) |
Number of GKE clusters in a project | Number of response policies per project | 100 |
Number of PTR records per cluster | Number of rules per response policy | 100,000 |
Resource limits
The Kubernetes resources that you create per cluster contribute to Cloud DNS resource limits, as described in the following table:
Limit | Contribution to limit |
---|---|
Resource record sets per managed zone | Number of services plus number of headless service endpoints with valid hostnames, per cluster. |
Records per resource record set | Number of endpoints per headless service. Does not impact other service types. |
Number of rules per response policy | For cluster scope, number of services plus number of headless service endpoints with valid hostnames per cluster. For VPC scope, number of services plus number of headless endpoints with hostnames from all clusters in the VPC. |
To learn more about how DNS records are created for Kubernetes, see Kubernetes DNS-Based Service Discovery.
Before you begin
Before you start, make sure you have performed the following tasks:
- Enable the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- If you want to use the Google Cloud CLI for this task,
install and then
initialize the
gcloud CLI. If you previously installed the gcloud CLI, get the latest
version by running
gcloud components update
.
Enable the Cloud DNS API in your project:
Enable cluster scope DNS
In cluster scope DNS, only nodes running in the GKE cluster can resolve service names, and service names do not conflict between clusters. This behavior is the same as kube-dns in GKE clusters, which means that you can migrate clusters from kube-dns to Cloud DNS cluster scope without downtime or changes to your applications.
The following diagram shows how Cloud DNS creates a private DNS zone for a GKE cluster. Only processes and Pods running on the nodes in the cluster can resolve the cluster's DNS records, because only the nodes are in the DNS scope.
Enable cluster scope DNS in a new cluster
GKE Autopilot cluster
New Autopilot clusters in version 1.25.9-gke.400, 1.26.4-gke.500 or later default to Cloud DNS cluster scope.
GKE Standard cluster
You can create a GKE Standard cluster with Cloud DNS cluster scope enabled using the gcloud CLI or the Google Cloud console:
gcloud
Create a cluster using the --cluster-dns
flag:
gcloud container clusters create CLUSTER_NAME \
--cluster-dns=clouddns \
--cluster-dns-scope=cluster \
--location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the cluster.COMPUTE_LOCATION
: the Compute Engine location for the cluster.
The --cluster-dns-scope=cluster
flag is optional in the command because
cluster
is the default value.
Console
Go to the Google Kubernetes Engine page in the Google Cloud console.
Click add_box Create.
From the navigation pane, under Cluster, click Networking.
In the DNS provider section, click Cloud DNS.
Select Cluster scope.
Configure your cluster as needed.
Click Create.
Enable cluster scope DNS in an existing cluster
GKE Autopilot cluster
You cannot migrate an existing GKE Autopilot cluster from kube-dns to Cloud DNS cluster scope. To enable Cloud DNS cluster scope, recreate the Autopilot clusters in version 1.25.9-gke.400, 1.26.4-gke.500 or later.
GKE Standard cluster
You can migrate an existing GKE Standard cluster from kube-dns to Cloud DNS cluster scope using the gcloud CLI or the Google Cloud console in a GKE Standard cluster.
When you migrate an existing cluster, the nodes in the cluster do not use Cloud DNS as a DNS provider until you recreate the nodes.
After you enable Cloud DNS for a cluster, the settings only apply if you upgrade existing node pools or you add new node pools to the cluster. When you upgrade a node pool, the nodes are recreated.
You can also migrate clusters that have running applications without interrupting cluster communication by enabling Cloud DNS as a DNS provider in each node pool separately. A subset of the nodes are operational at all times because some node pools use kube-dns and some node pools use Cloud DNS.
In the following steps, you enable Cloud DNS for a cluster and then upgrade your node pools. Upgrading your node pools recreates the nodes. The nodes then use Cloud DNS for DNS resolution instead of kube-dns.
gcloud
Update the existing cluster:
gcloud container clusters update CLUSTER_NAME \ --cluster-dns=clouddns \ --cluster-dns-scope=cluster \ --location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the cluster.COMPUTE_LOCATION
: the Compute Engine location for your cluster.
The response is similar to the following:
All the node-pools in the cluster need to be re-created by the user to start using Cloud DNS for DNS lookups. It is highly recommended to complete this step shortly after enabling Cloud DNS. Do you want to continue (Y/n)?
After you confirm, the Cloud DNS controller runs on the GKE control plane, but your Pods do not use Cloud DNS for DNS resolution until you upgrade your node pool or you add new node pools to the cluster.
Upgrade the node pools in the cluster to use Cloud DNS:
gcloud container clusters upgrade CLUSTER_NAME \ --node-pool=POOL_NAME \ --location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the cluster.POOL_NAME
: the name of the node pool to upgrade.
If the node pool and control plane are running the same version, upgrade the control plane first, as described in Manually upgrading the control plane and then perform the node pool upgrade.
Confirm the response and repeat this command for each node pool in the cluster. If your cluster has one node pool, omit the
--node-pool
flag.
Console
Go to the Google Kubernetes Engine page in the Google Cloud console.
Click the name of the cluster you want to modify.
Under Networking, in the DNS provider field, click edit Edit DNS provider.
Click Cloud DNS.
Click Cluster scope.
Click Save changes.
Enable VPC scope DNS
In VPC scope DNS, a cluster's DNS names are resolvable within the entire VPC. Any client in the VPC can resolve cluster DNS records.
VPC scope DNS enables the following use cases:
- Headless service discovery for non-GKE clients within the same VPC.
- GKE Service resolution from on-premise or 3rd party cloud clients. For more information, see Inbound server policy.
- Service resolution where a client can decide which cluster they want to communicate with using the custom cluster DNS domain.
In the following diagram, two GKE clusters use VPC
scope DNS in the same VPC. Both clusters have a custom DNS domain,
.cluster1
and .cluster2
, instead of the default .cluster.local
domain. A
VM communicates with the headless backend Service by resolving
backend.default.svc.cluster1
. Cloud DNS resolves the headless Service to
the individual Pod IPs in the Service and the VM communicates directly with the
Pod IPs.
You can also perform this type of resolution from other networks when connected to the VPC through Cloud Interconnect or Cloud VPN. DNS server policies enable clients from networks connected to the VPC to resolve names in Cloud DNS, which includes GKE Services if the cluster is using VPC scope DNS.
Enable VPC scope DNS in a new cluster
GKE Autopilot cluster
You cannot create a GKE Autopilot cluster with VPC scope DNS, this feature is not supported on GKE Autopilot.
GKE Standard cluster
You can enable VPC scope DNS in a new cluster using the gcloud CLI or the Google Cloud console.
gcloud
You can create a new cluster with VPC scope Cloud DNS enabled using the following command:
gcloud container clusters create CLUSTER_NAME \
--cluster-dns=clouddns \
--cluster-dns-scope=vpc \
--cluster-dns-domain=CUSTOM_DOMAIN \
--location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the cluster.COMPUTE_LOCATION
: the Compute Engine location for the cluster.CUSTOM_DOMAIN
: the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.
Console
Go to the Google Kubernetes Engine page in the Google Cloud console.
Click add_box Create.
From the navigation pane, under Cluster, click Networking.
In the DNS provider section, click Cloud DNS.
Select VPC scope.
Enter the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.
Configure your cluster as needed.
Click Create.
Enable VPC scope DNS in an existing cluster
GKE Autopilot cluster
You cannot migrate a GKE Autopilot cluster from kube-dns to Cloud DNS VPC scope, this feature is not supported on GKE Autopilot.
GKE Standard cluster
You can migrate an existing GKE cluster from kube-dns to Cloud DNS VPC scope using the gcloud CLI or the Google Cloud console. The migration is supported in GKE Standard only.
After you enable Cloud DNS for a cluster, the settings only apply if you upgrade existing node pools or you add new node pools to the cluster. When you upgrade a node pool, the nodes are recreated.
In the following steps, you enable Cloud DNS for a cluster and then upgrade your node pools. Upgrading your node pools recreates the nodes. The nodes then use Cloud DNS for DNS resolution instead of kube-dns.
gcloud
Update the existing cluster:
gcloud container clusters update CLUSTER_NAME \ --cluster-dns=clouddns \ --cluster-dns-scope=vpc \ --cluster-dns-domain=CUSTOM_DOMAIN \ --location=COMPUTE_LOCATION
Replace the following:
CLUSTER_NAME
: the name of the cluster.COMPUTE_LOCATION
: the Compute Engine location for the cluster.CUSTOM_DOMAIN
: the name of a domain. You must ensure this name is unique within the VPC because GKE does not confirm this value. You cannot change this value after it is set. You must not use a domain that ends in ".local", or you might experience DNS resolution failures.
The response is similar to the following:
All the node-pools in the cluster need to be re-created by the user to start using Cloud DNS for DNS lookups. It is highly recommended to complete this step shortly after enabling Cloud DNS. Do you want to continue (Y/n)?
After you confirm, the Cloud DNS controller runs on the GKE control plane. Your Pods do not use Cloud DNS for DNS resolution until you upgrade your node pool or you add new node pools to the cluster.
Upgrade the node pools in the cluster to use Cloud DNS:
gcloud container clusters upgrade CLUSTER_NAME \ --node-pool=POOL_NAME
Replace the following:
CLUSTER_NAME
: the name of the cluster.POOL_NAME
: the name of the node pool to upgrade.
If the node pool and control plane are running the same version, upgrade the control plane first, as described in Manually upgrading the control plane and then perform the node pool upgrade.
Confirm the response and repeat this command for each node pool in the cluster. If your cluster has one node pool, omit the
--node-pool
flag.
Console
Go to the Google Kubernetes Engine page in the Google Cloud console.
Click the name of the cluster you want to modify.
Under Networking, in the DNS provider field, click edit Edit DNS provider.
Click Cloud DNS.
Click VPC scope.
Click Save changes.
Verify Cloud DNS
Verify that Cloud DNS for GKE is working correctly for your cluster:
Verify that your nodes are using Cloud DNS by connecting to a Pod on a node and running the command
cat /etc/resolv.conf
:kubectl exec -it POD_NAME -- cat /etc/resolv.conf | grep nameserver
Replace
POD_NAME
with the name of the Pod.Based on the cluster mode, the output is similar to the following:
GKE Autopilot cluster
nameserver 169.254.20.10
Because the NodeLocal DNSCache is enabled by default in GKE Autopilot, the Pod is using NodeLocal DNSCache.
Only when the local cache does not have an entry for the name being looked up, NodeLocal DNSCache forwards the request to Cloud DNS.
GKE Standard cluster
nameserver 169.254.169.254
The Pod is using
169.254.169.254
as thenameserver
, which is the IP address of the metadata server where the Cloud DNS data plane listens for requests on port 53. The nodes no longer use the kube-dns Service address for DNS resolution and all DNS resolution occurs on the local node.If the output is an IP address similar to
10.x.y.10
, then the Pod is using kube-dns. See the Troubleshooting section to understand why your pod is still using kube-dns.If the output is
169.254.20.10
, it means that you have enabled NodeLocal DNSCache in your cluster, then the Pod is using NodeLocal DNSCache.Deploy a sample application to your cluster:
kubectl run dns-test --image us-docker.pkg.dev/google-samples/containers/gke/hello-app:2.0
Expose the sample application with a Service:
kubectl expose pod dns-test --name dns-test-svc --port 8080
Verify that the Service deployed successfully:
kubectl get svc dns-test-svc
The output is similar to the following:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dns-test-svc ClusterIP 10.47.255.11 <none> 8080/TCP 6m10s
The value of
CLUSTER-IP
is the virtual IP address for your cluster. In this example, the virtual IP address is10.47.255.11
.Verify that your Service name was created as a record in the private DNS zone for your cluster:
gcloud dns record-sets list \ --zone=PRIVATE_DNS_ZONE \ --name=dns-test-svc.default.svc.cluster.local.
Replace
PRIVATE_DNS_ZONE
with the name of the managed DNS zone.The output is similar to the following:
NAME: dns-test-svc.default.svc.cluster.local. TYPE: A TTL: 30 DATA: 10.47.255.11
Disable Cloud DNS for GKE
GKE Autopilot cluster
You cannot disable Cloud DNS in a GKE Autopilot cluster that was created with Cloud DNS by default. See the requirements for more information about GKE Autopilot clusters using Cloud DNS by default.
GKE Standard cluster
You can disable Cloud DNS cluster scope using the gcloud CLI or the Google Cloud console in a GKE Standard cluster.
gcloud
Update the cluster to use kube-dns:
gcloud container clusters update CLUSTER_NAME \
--cluster-dns=default
Console
Go to the Google Kubernetes Engine page in the Google Cloud console.
Click the name of the cluster you want to modify.
Under Networking, in the DNS provider field, click edit Edit DNS provider.
Click Kube-dns.
Click Save changes.
Clean up
After completing the exercises on this page, follow these steps to remove resources and prevent unwanted charges incurring on your account:
Delete the service:
kubectl delete service dns-test-svc
Delete the Pod:
kubectl delete Pod dns-test
You can also delete the cluster.
Use Cloud DNS with Shared VPC
Cloud DNS for GKE supports Shared VPC for both VPC and cluster scope.
The GKE controller creates a managed private zone in the same project as the GKE cluster.
The GKE service account for the GKE cluster does not require Identity and Access Management (IAM) for DNS outside of its own project because the managed zone and GKE cluster reside within the same project.
More than one cluster per service project
Starting in GKE versions 1.22.3-gke.700, 1.21.6-gke.1500, you can create clusters in multiple service projects referencing a VPC in the same host project. If you already have clusters using Shared VPC and Cloud DNS VPC scope, you must manually migrate them with the following steps:
- Upgrade your existing clusters that have Shared VPC enabled to GKE version 1.22.3-gke.700+ or 1.21.6-gke.1500+.
- Migrate the response policy from the service project to the host project. You only need to perform this migration once per Shared VPC network.
You can migrate your response policy using the Google Cloud console.
Perform the following steps in your service project:
Go to the Cloud DNS zones page.
Click the Response policy zones tab.
Click the response policy for your VPC network. You can identify the response policy by the description, which is similar to "Response policy for GKE clusters on network NETWORK_NAME."
Click the In use by tab.
Next to the name of your host project, click delete to remove the network binding.
Click the Response policy rules tab.
Select all of the entries in the table.
Click Remove response policy rules.
Click delete Delete response policy.
After you delete the response policy, the DNS controller creates the response policy in the host project automatically. Clusters from other service projects share this response policy.
Support custom stub domains and upstream nameservers
Cloud DNS for GKE supports custom stub domains and upstream nameservers configured using kube-dns ConfigMap. This support is only available for GKE cluster scope.
Cloud DNS translates stubDomains
and upstreamNameservers
values
into Cloud DNS forwarding zones.
Known issues
Terraform plans to recreate Autopilot cluster due to dns_config
change
If you use terraform-provider-google
or terraform-provider-google-beta
, you
might experience an issue where Terraform tries tries to recreate an
Autopilot cluster. This error occurs because newly created
Autopilot clusters running version 1.25.9-gke.400, 1.26.4-gke.500,
1.27.1-gke.400 or later use Cloud DNS as a DNS provider instead of kube-dns.
This issue is resolved in version 4.80.0 of the Terraform provider of Google Cloud.
If you cannot update the version of terraform-provider-google
or
terraform-provider-google-beta
, you can add lifecycle.ignore_changes
to
the resource to ensure that google_container_cluster
ignores changes to
dns_config
:
lifecycle {
ignore_changes = [
dns_config,
]
}
Troubleshooting
To learn how to enable DNS logging, see Enabling and disabling logging for private managed zones.
For more information about troubleshooting DNS issues, see Troubleshooting DNS in GKE.
Unable to update existing cluster or create cluster with Cloud DNS enabled
Ensure you are using the correct version. Cloud DNS for GKE requires GKE version 1.19 or later for clusters using VPC scope, or GKE version 1.24.7-gke.800, 1.25.3-gke.700 or later for clusters using cluster scope.
Pods use kube-dns even after Cloud DNS is enabled on an existing cluster
Ensure you have upgraded or recreated your node pools after you enable Cloud DNS on the cluster. Until this step is complete, Pods continue to use kube-dns.
Pod is unable to resolve DNS lookups
Verify the Pod is using Cloud DNS by connecting to a Pod and running the command
cat /etc/resolv.conf
:kubectl exec -it POD_NAME -- cat /etc/resolv.conf | grep nameserver
Replace
POD_NAME
with the name of the Pod.The output is similar to the following:
nameserver 169.254.169.254
If the output is an IP address similar to
10.x.y.10
or34.118.224.10
(only in GKE Autopilot clusters with versions 1.27 and later), then the Pod is using kube-dns. If the output is169.254.20.10
, then the Pod is using NodeLocal DNSCache.Confirm the managed zone exists and contains the necessary DNS entry:
gcloud dns managed-zones list --format list
The output is similar to the following:
- creationTime: 2021-02-12T19:24:37.045Z description: Private zone for GKE cluster "cluster1" with cluster suffix "cluster.local" in project "project-243723" dnsName: cluster.local. id: 5887499284756055830 kind: dns#managedZone name: gke-cluster1-aa94c1f9-dns nameServers: ['ns-gcp-private.googledomains.com.'] privateVisibilityConfig: {'kind': 'dns#managedZonePrivateVisibilityConfig'} visibility: private
The value of
name
in the response shows that Google Cloud created a private zone namedgke-cluster1-aa94c1f9-dns
.Verify that Cloud DNS contains entries for your cluster:
gcloud dns record-sets list --zone ZONE_NAME | grep SERVICE_NAME
Replace the following:
ZONE_NAME
: the name of the private zone.SERVICE_NAME
: the name of the service.
The output shows that Cloud DNS contains an A record for the domain
dns-test.default.svc.cluster.local.
and the IP address of your cluster,10.47.255.11
:dns-test.default.svc.cluster.local. A 30 10.47.255.11
Enable Cloud DNS logging to track queries. Every log entry contains information about the query, including DNS latency.
DNS lookups on nodes fail after enabling Cloud DNS on a cluster
If you enable cluster scope Cloud DNS in a GKE cluster that has custom stub domains or upstream nameservers, the custom config applies to both nodes and Pods in the cluster because Cloud DNS cannot distinguish between Pod and node DNS requests. DNS lookups on nodes might fail if the custom upstream server cannot resolve the queries.
What's next
- Read an overview of how GKE provides managed DNS.
- Read DNS for services and Pods for a general overview of how DNS is used in Kubernetes clusters.
- Learn about Scopes and hierarchies in Cloud DNS.
- Learn about Enabling and disabling logging for private managed zones.