Hardening your cluster's security

With the speed of development in Kubernetes, there are often new security features for you to use. This page guides you through implementing our current guidance for hardening your Kubernetes Engine cluster. For a general overview of security topics, read the Security Overview.

Disable the Kubernetes web UI (Dashboard)

You should disable the Kubernetes Web UI (Dashboard) when running on Kubernetes Engine.

The Kubernetes Web UI (Dashboard) is backed by a highly privileged Kubernetes Service Account. The Cloud Console provides much of the same functionality, so you don't need these permissions.

To disable the Kubernetes Web UI:

gcloud container clusters update [CLUSTER_NAME] \
    --update-addons=KubernetesDashboard=DISABLED

Disable ABAC

You should disable Attribute-Based Access Control (ABAC), and instead use Role-Based Access Control (RBAC) in Kubernetes Engine.

In Kubernetes, RBAC is used to grant permissions to resources at the cluster and namespace level. RBAC allows you to define roles with rules containing a set of permissions. RBAC has significant security advantages and is now stable in Kubernetes, so it’s time to disable ABAC.

If you're still relying on ABAC, first review the Prerequisites for using RBAC. If you upgraded your cluster from an older version and are using ABAC, you should update your access controls configuration:

gcloud container clusters update [CLUSTER_NAME] \
    --no-enable-legacy-authorization

To create a new cluster with the above recommendation:

gcloud container clusters create [CLUSTER_NAME] \
    --no-enable-legacy-authorization

Restrict pod to pod traffic with a Network Policy

By default, all pods in a cluster can communicate with each other. You should control pod to pod communication as needed for your workload.

You can use Kubernetes' Network Policies to make it much more difficult for attackers to move laterally within your cluster. You can also use the Kubernetes Network Policy API to create pod-level firewall rules. These firewall rules determine which pods and services can access one another inside your cluster.

To enable network policy enforcement when creating a new cluster, specify the --enable-network-policy flag:

gcloud container clusters create [CLUSTER_NAME] \
    --zone=[ZONE] \
    --enable-network-policy

After you enable Network Policy, you have to actually define a policy. Since this is specific to your exact topology, we can’t provide a specific recommendation. The Kubernetes documentation, however, has an excellent walkthrough for a simple nginx deployment.

To learn more about Network Policies in Kubernetes Engine, see Setting a Cluster Network Policy.

Use least privilege service accounts for your nodes

Each Kubernetes Engine node has an IAM Service Account associated with it. By default, nodes are given the Compute Engine default service account, which you can find by navigating to the IAM section of the Cloud Console. This account has broad access by default, making it useful to wide variety of applications, but it has more permissions than are required to run your Kubernetes Engine cluster. You should create and use a minimally privileged service account to run your Kubernetes Engine cluster instead of using the Compute Engine default service account.

Kubernetes Engine requires, at a minimum, the service account to have the monitoring.viewer, monitoring.metricWriter, and logging.logWriter roles. Read more about monitoring roles and logging roles.

The following commands create an IAM service account with the minimum permissions required to operate Kubernetes Engine:

gcloud iam service-accounts create [SA_NAME] \
    --display-name=[SA_NAME]

gcloud projects add-iam-policy-binding [PROJECT_ID] \
    --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
    --role roles/logging.logWriter

gcloud projects add-iam-policy-binding [PROJECT_ID] \
    --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
    --role roles/monitoring.metricWriter

gcloud projects add-iam-policy-binding [PROJECT_ID] \
    --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
    --role roles/monitoring.viewer

If you use private images in Google Container Registry, you also need to grant access to those:

gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
  --role roles/storage.objectViewer

If your cluster already exists, you can now create a new node pool with this new service account:

gcloud container node-pools create [NODE_POOL] \
  --service-account=[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
  --cluster=[CLUSTER_NAME]

If you need your Kubernetes Engine cluster to have access to other Google Cloud services, we recommend that you create an additional role and provision it to workloads via Kubernetes secrets, rather than re-use this one.

Reduce your node service account scopes

If you don't want to use a custom service account, an alternative recommendation is to reduce the permissions of the default node service account.

By default, your node service account has access scopes. Access scopes are the legacy method of specifying permissions for your instance. Before the existence of IAM roles, access scopes were the only mechanism for granting permissions to service accounts.

If you are not creating a separate service account for your nodes, you should limit the scopes of the node service account to reduce the possibility of a privilege escalation in an attack. This ensures that your default service account does not have permissions beyond those necessary to run your cluster. While the default scopes are limited, they may include scopes beyond the minimally required scopes needed to run your cluster.

The default scopes for the nodes in Kubernetes Engine are devstorage.read_only, logging.write, monitoring, service.management.readonly, servicecontrol, and trace.append. When setting scopes, these are specified as gke-default. If you are accessing private images in Google Container Registry, the minimally required scopes are only logging.write, monitoring, and devstorage.read_only.

To create a cluster with the custom scopes, use the --scopes option:

gcloud container clusters create [CLUSTER_NAME] \
    --scopes=[CUSTOM_SCOPES]

If you do not specify scopes or a custom service account, gke-default is used. If you specify a custom service account, cloud-platform and userinfo.email are used.

If your gcloud configuration includes container/new_scopes_behavior true, this behavior is already enabled for all Kubernetes versions. To set the default for your environment:

gcloud config set container/new_scopes_behavior true

To learn more about permissions for all scopes, see Google scopes.

Conceal node metadata (Beta)

Some practical attacks against Kubernetes rely on access to the VM's metadata server to extract the node’s credentials.

Using a minimally privileged service account as advised above is the first step to mitigate these attacks. The next step is to prevent workloads from impersonating nodes. You should use metadata concealment to protect potentially sensitive system metadata from workloads running on your cluster.

To learn more about metadata concealment, see Protecting Cluster Metadata.

Restrict pod permissions with a Pod Security Policy (Beta)

By default, pods in Kubernetes can operate with capabilities beyond what they require. You should constrain the pod's capabilities to only those required for that workload.

Kubernetes offers controls for restricting your pods to execute with only their minimum required capabilities. Pod Security Policy allows you to set smart defaults for your pods, and enforce controls you want to enable across your fleet. The policies you define should be specific to the needs of your application. The restricted-psp.yaml example policy is a good starting point.

To learn more about Pod Security Policy, see Using PodSecurityPolicies.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Kubernetes Engine