This document describes how Google Kubernetes Engine (GKE) secures your cluster control plane components.
Under the Shared Responsibility Model, Google manages the GKE control plane components for you. The control plane includes the Kubernetes API server, etcd storage, and other controllers. Google is responsible for securing the control plane, though you might be able to configure certain options based on your requirements. You are responsible for securing your nodes, containers, and Pods.
Hardened operating system
GKE control plane components run on Container-Optimized OS, which is a security-hardened operating system designed by Google. For a detailed description of the security features built into Container-Optimized OS, see the Container-Optimized OS security overview.
Architecture and isolation
In a GKE cluster, the control plane components run on Compute Engine instances owned by Google, in a Google-managed project. Each instance runs these components for only one cluster.
For details about how cluster components authenticate to each other, see Cluster trust.
Control plane access to your project
GKE uses a
Google-managed service account
named the Kubernetes Engine Service Agent to actuate cluster resources on your
behalf such as nodes, disks, and load balancers. The service account is
automatically granted the
Kubernetes Engine Service Agent role
(roles/container.serviceAgent) on your project.
The Kubernetes Engine Service Agent has the following email address:
service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com
In this email address, PROJECT_NUMBER is your
project number.
Administrative access to the cluster
SSH sessions by Google Site Reliability Engineers are audit logged through Google's internal audit infrastructure, which is available for forensics and security response. For more information, see Administrative access in the Google Security Whitepaper.
etcd security
In Google Cloud, customer content is encrypted at the filesystem layer by default. Therefore, disks that host etcd storage for GKE clusters are encrypted at the filesystem layer. For more information, see Encryption at rest.
etcd listens on two TCP ports. Port 2379 is for etcd clients, like the Kubernetes API server. Port 2379 is bound to the local loopback network interface, so it is only accessible from the VM that is running the Kubernetes API server. Port 2380 is for server-to-server communication. Traffic on port 2380 is encrypted by mutual TLS. That is, each server must prove its identity to the other. In a regional cluster, communication between etcd servers to establish a quorum is encrypted by mutual TLS.
Certificate authority and cluster trust
Each cluster has its own root certificate authority (CA). An internal Google service manages root keys for this CA. Each cluster also has its own CA for etcd. Root keys for the etcd CA are distributed to the metadata of the VMs that run the Kubernetes API server. Communication between nodes and the Kubernetes API server is protected by TLS. For more information, see Cluster trust.
Vulnerability and patch management
GKE adheres to Google standards for testing, qualifying, and gradually rolling out changes to the control plane. This minimizes the risk of a control plane component becoming unavailable. GKE adheres to a service level agreement that defines many aspects of availability.
GKE control plane components are managed by a team of Google site reliability engineers, and are kept up to date with the latest security patches. This includes patches to the host operating system, Kubernetes components, and containers running on the control plane VMs.
GKE applies new kernel, OS, and Kubernetes-level fixes promptly to control plane VMs. When these contain fixes for known vulnerabilities, additional information is available in the GKE security bulletins. GKE scans all Kubernetes system and GKE-specific containers for vulnerabilities using Artifact Analysis, and keeps the containers patched, benefitting the whole Kubernetes ecosystem.
Google engineers participate in finding, fixing, and disclosing Kubernetes security bugs. Google also pays external security researchers, through the Google-wide vulnerability reward program, to look for security bugs. In some cases, such as the dnsmasq vulnerability in October 2017, GKE was able to patch all running clusters before the vulnerability became public.
What you can see
The security features discussed previously in this topic are managed by Google. This section and the following section discuss security features that you can monitor and configure.
Audit logging is enabled by default for clusters created since GKE version 1.8.3. This provides a detailed record, available in Google Cloud Observability, of calls made to the Kubernetes API server. You can view the log entries in the Logs Explorer in the Google Cloud console. You can also use BigQuery to view and analyze these logs.
What you can configure
By default, the Kubernetes API server uses a public IP address. You can protect the Kubernetes API server by using authorized networks and private clusters, which allow you to assign a private IP address to the Kubernetes API server and disable access on the public IP address.
You can handle cluster authentication in GKE by using IAM as the identity provider. For enhanced authentication security, basic authentication and client certificate issuance are disabled by default for clusters created with GKE version 1.12 and later.
You can enhance the security of your control plane by doing credential rotation on a regular basis. When credential rotation is initiated, the TLS certificates and cluster certificate authority are rotated automatically. GKE also rotates the IP address of your Kubernetes API server. For more information, see Role-Based Access Control (RBAC) and Credential rotation.