A new flexible DNS-based approach for accessing the GKE control plane

If you run Google Kubernetes Engine (GKE), you know it’s important to secure access to the cluster control plane that handles Kubernetes API requests, so you can prevent unauthorized access while still being able to control the cluster. 

Previously, GKE provided two primary methods to secure the control plane: authorized networks and disabling public endpoints. But when using these methods, it can be difficult to access the cluster. You need creative solutions such as bastion hosts to gain access via the cluster’s private network, and the list of authorized networks must be kept up to date across all clusters.

Today, we are excited to announce a new DNS-based endpoint for GKE clusters, allowing enhanced flexibility in access methods and security controls. The DNS-based endpoint is available today on every cluster, regardless of version or cluster configuration.

The new DNS-based endpoint addresses several of the current challenges associated with Kubernetes control plane access, including:

1. Complex IP-based firewall / allowlist configurations: IP address-based authorized network configuration / ACL is prone to human configuration error.  

2. Static configurations based on IP addresses: As network configuration and IP ranges change, you need to change the authorized network IP Firewall configuration accordingly.

3. Proxy / bastion hosts: When accessing the GKE control plane from a remote network, different cloud location, or from a VPC that is different from the VPC where the cluster resides, you need to set up a proxy or bastion host. 

These challenges have resulted in a complex configuration and a confusing user experience for GKE customers.

Introducing a new DNS-based endpoint

The DNS name resolves to a frontend that is accessible from any network that can reach Google Cloud APIs, including VPC networks, on-premises networks, or other cloud networks. This frontend The new DNS-based endpoint for GKE provides a unique DNS or fully qualified domain name (FQDN) for each cluster control plane. applies security policies to reject unauthorized traffic, and then forwards traffic to your cluster.

This approach provides a number of benefits:

1. Simple flexible access from anywhere

Using the DNS-based endpoint eliminates the need for a bastion host or proxy nodes. Authorized users can access your control plane from different clouds, on-prem deployments or from home without jumping through proxies. With DNS-based endpoints, there are no restrictions for transiting multiple VPCs, as the only requirement is access to Google APIs. If desired, you can still limit access to specific networks using VPC Service Controls.

2. Dynamic security

Access to your control plane over the DNS-based endpoint is protected via the same IAM policies used to protect all GCP API access. Using IAM policies, you can ensure that only authorized users can access the control plane, irrespective of which IP or network they use. If needed, you can simply revoke a particular identity’s access without worrying about network IP address configuration and boundaries. IAM roles can be customized to suit your organization’s needs.

For more details on the exact permissions required to configure IAM roles policies and authentication tokens, see Customize your network isolation.

3. Two layers of security

In addition to IAM policies, you can also configure network-based controls with VPC Service Controls, providing a multi-layer security model for your cluster control plane. VPC Service Controls adds context-aware access controls based on attributes such as network origin. You can achieve the same level of security as a private cluster that can only be accessed from a VPC network. VPC Service Controls are used by all Google Cloud APIs, aligning the security configuration of your clusters with your services and data hosted in all other Google Cloud APIs. You can make strong guarantees about preventing unauthorized access to services and data for all Google Cloud resources in a project. VPC Service Controls integrate with Cloud Audit Logs to monitor access to the control plane.

In our customer’s voice:

“When using private IP based control plane access we had to configure and manage a complex networking solution to be able to access the GKE control plane from our various VPC and data center sites. DNS-based GKE control plane access, in partnership with VPC Service Controls, will greatly simplify our access to the GKE control plane and help us implement a dynamic security based access policy based on Identity and Authentication.”  - Keith Ferguson - ANZx Cloud & Engineering Platforms Technology Lead, ANZ Bank

How to configure DNS-based access

Configuring DNS-based access for the GKE cluster control plane is a straightforward process. Check out the following steps.

1. Enable the DNS-based endpoint

Enable DNS-based access for a new cluster with the following command:

Alternatively, enable DNS-based access for an existing cluster with the following command:

2. Configure IAM

Access to the control plane requires requests to be authenticated with a role with the new IAM permission container.clusters.connect. Assign one of the following IAM roles to your user:

• roles/container.developer

• roles/container.viewer

Here is an example of configuring a user to access cluster with pre-configured roles:

Alternatively, you can configure a custom IAM role with the container.clusters.connect permission.

Here’s an example of configuring a role to use container.clusters.connect:

3. Ensure your client can access Google APIs

If your client is connecting from a Google VPC, you will need to ensure that it has connectivity to Google APIs. One way to do this is to activate Private Google Access, which allows clients to connect to Google APIs without going over the public internet. Private Google Access is configured on individual subnets.

Tip: Private Google Access is already enabled for node subnetworks.

Below, you can see an example of turning on Private Google Access in a subnet:

4. [Optional] Configuring access via Private Service Connect for Google APIs

Cluster’s DNS endpoint can be accessible via Private Service Connect for Google APIs endpoint used for accessing the rest of the Google APIs. The Access Google APIs through endpoints page has all required steps to configure Private Service Connect for Google APIs endpoint. Accessing cluster’s DNS via custom endpoint is not supported, therefore as explained in use an endpoint section, creating an A record between “* gke.goog” and private IP assigned to Private Service Connect for Google APIs and creating a CNAME to “*.gke.goog” is required to make it work.

Try out DNS-based access

Now you’re ready to try out DNS-based access. Use the following command to generate a kubeconfig file using the cluster’s DNS address:

Then, use kubectl to access your cluster. You can use this directly from Cloud Shell to access clusters without a public IP endpoint, something that required setting up a proxy before.

Additional security with VPC Service Controls

Optionally, you can also use VPC Service Controls as an additional layer of security for your control plane access.

Here’s an example of an ingress policy that only allows requests from VMs in a specific GCP project:

What about the IP-based endpoint?

Of course, you can still use the existing IP-based endpoint to access the control plane, allowing you to try out DNS-based access without affecting your existing clients. Then, once you’re happy with DNS-based access, you can disable IP-based access, which provides additional security and simplified cluster management:

What’s next

With DNS-based endpoints, you gain increased flexibility in managing the security of your cluster control planes, while also reducing the complexity of accessing clusters from private networks.

To learn more about how to use DNS-based endpoints, check out these references:

• Cluster isolation concept page

• Cluster isolation how-to page