With the speed of development in Kubernetes, there are often new security features for you to use. This page guides you through implementing our current guidance for hardening your Kubernetes Engine cluster. For a general overview of security topics, read the Security Overview.
Disable the Kubernetes web UI (Dashboard)
You should disable the Kubernetes Web UI (Dashboard) when running on Kubernetes Engine.
The Kubernetes Web UI (Dashboard) is backed by a highly privileged Kubernetes Service Account. The Cloud Console provides much of the same functionality, so you don't need these permissions.
To disable the Kubernetes Web UI:
gcloud container clusters update [CLUSTER_NAME] \ --update-addons=KubernetesDashboard=DISABLED
You should disable Attribute-Based Access Control (ABAC), and instead use Role-Based Access Control (RBAC) in Kubernetes Engine.
In Kubernetes, RBAC is used to grant permissions to resources at the cluster and namespace level. RBAC allows you to define roles with rules containing a set of permissions. RBAC has significant security advantages and is now stable in Kubernetes, so it’s time to disable ABAC.
If you're still relying on ABAC, first review the Prerequisites for using RBAC. If you upgraded your cluster from an older version and are using ABAC, you should update your access controls configuration:
gcloud container clusters update [CLUSTER_NAME] \ --no-enable-legacy-authorization
To create a new cluster with the above recommendation:
gcloud container clusters create [CLUSTER_NAME] \ --no-enable-legacy-authorization
Restrict pod to pod traffic with a Network Policy
By default, all pods in a cluster can communicate with each other. You should control pod to pod communication as needed for your workload.
You can use Kubernetes' Network Policies to make it much more difficult for attackers to move laterally within your cluster. You can also use the Kubernetes Network Policy API to create pod-level firewall rules. These firewall rules determine which pods and services can access one another inside your cluster.
To enable network policy enforcement when creating a new cluster, specify the
gcloud container clusters create [CLUSTER_NAME] \ --zone=[ZONE] \ --enable-network-policy
After you enable Network Policy, you have to actually define a policy. Since this is specific to your exact topology, we can’t provide a specific recommendation. The Kubernetes documentation, however, has an excellent walkthrough for a simple nginx deployment.
To learn more about Network Policies in Kubernetes Engine, see Setting a Cluster Network Policy.
Use least privilege service accounts for your nodes
Each Kubernetes Engine node has an IAM Service Account associated with it. By default, nodes are given the Compute Engine default service account, which you can find by navigating to the IAM section of the Cloud Console. This account has broad access by default, making it useful to wide variety of applications, but it has more permissions than are required to run your Kubernetes Engine cluster. You should create and use a minimally privileged service account to run your Kubernetes Engine cluster instead of using the Compute Engine default service account.
The following commands create an IAM service account with the minimum permissions required to operate Kubernetes Engine:
gcloud iam service-accounts create [SA_NAME] \ --display-name=[SA_NAME] gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/logging.logWriter gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/monitoring.metricWriter gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/monitoring.viewer
If you use private images in Google Container Registry, you also need to grant access to those:
gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/storage.objectViewer
If your cluster already exists, you can now create a new node pool with this new service account:
gcloud container node-pools create [NODE_POOL] \ --service-account=[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --cluster=[CLUSTER_NAME]
If you need your Kubernetes Engine cluster to have access to other Google Cloud services, we recommend that you create an additional role and provision it to workloads via Kubernetes secrets, rather than re-use this one.
Reduce your node service account scopes
If you don't want to use a custom service account, an alternative recommendation is to reduce the permissions of the default node service account.
By default, your node service account has access scopes. Access scopes are the legacy method of specifying permissions for your instance. Before the existence of IAM roles, access scopes were the only mechanism for granting permissions to service accounts.
If you are not creating a separate service account for your nodes, you should limit the scopes of the node service account to reduce the possibility of a privilege escalation in an attack. This ensures that your default service account does not have permissions beyond those necessary to run your cluster. While the default scopes are limited, they may include scopes beyond the minimally required scopes needed to run your cluster.
The default scopes for the nodes in Kubernetes Engine are
trace.append. When setting scopes, these are specified as
If you are accessing private images in Google Container Registry, the minimally
required scopes are only
To create a cluster with the custom scopes, use the
gcloud container clusters create [CLUSTER_NAME] \ --scopes=[CUSTOM_SCOPES]
If you do not specify scopes or a custom service account,
gke-default is used.
If you specify a custom service account,
gcloud configuration includes
this behavior is already enabled for all Kubernetes versions. To set the default
for your environment:
gcloud config set container/new_scopes_behavior true
To learn more about permissions for all scopes, see Google scopes.
Conceal node metadata (Beta)
Some practical attacks against Kubernetes rely on access to the VM's metadata server to extract the node’s credentials.
Using a minimally privileged service account as advised above is the first step to mitigate these attacks. The next step is to prevent workloads from impersonating nodes. You should use metadata concealment to protect potentially sensitive system metadata from workloads running on your cluster.
To learn more about metadata concealment, see Protecting Cluster Metadata.
Restrict pod permissions with a Pod Security Policy (Beta)
By default, pods in Kubernetes can operate with capabilities beyond what they require. You should constrain the pod's capabilities to only those required for that workload.
Kubernetes offers controls for restricting your pods to execute with only their minimum required capabilities. Pod Security Policy allows you to set smart defaults for your pods, and enforce controls you want to enable across your fleet. The policies you define should be specific to the needs of your application. The restricted-psp.yaml example policy is a good starting point.
To learn more about Pod Security Policy, see Using PodSecurityPolicies.