With the speed of development in Kubernetes, there are often new security features for you to use. This page guides you through implementing our current guidance for hardening your Kubernetes Engine cluster. For a general overview of security topics, read the Security Overview.
Disable the Kubernetes web UI (Dashboard)
You should disable the Kubernetes Web UI (Dashboard) when running on Kubernetes Engine.
The Kubernetes Web UI (Dashboard) is backed by a highly privileged Kubernetes Service Account. The Cloud Console provides much of the same functionality, so you don't need these permissions.
To disable the Kubernetes Web UI:
gcloud container clusters update [CLUSTER_NAME] \ --update-addons=KubernetesDashboard=DISABLED
You should disable Attribute-Based Access Control (ABAC), and instead use Role-Based Access Control (RBAC) in Kubernetes Engine.
In Kubernetes, RBAC is used to grant permissions to resources at the cluster and namespace level. RBAC allows you to define roles with rules containing a set of permissions. RBAC has significant security advantages and is now stable in Kubernetes, so it’s time to disable ABAC.
If you're still relying on ABAC, first review the Prerequisites for using RBAC. If you upgraded your cluster from an older version and are using ABAC, you should update your access controls configuration:
gcloud container clusters update [CLUSTER_NAME] \ --no-enable-legacy-authorization
To create a new cluster with the above recommendation:
gcloud container clusters create [CLUSTER_NAME] \ --no-enable-legacy-authorization
Restrict Pod to Pod traffic with a Network Policy
By default, all Pods in a cluster can communicate with each other. You should control Pod to Pod communication as needed for your workload.
You can use Kubernetes' Network Policies to make it much more difficult for attackers to move laterally within your cluster. You can also use the Kubernetes Network Policy API to create Pod-level firewall rules. These firewall rules determine which Pods and services can access one another inside your cluster.
To enable network policy enforcement when creating a new cluster, specify the
gcloud container clusters create [CLUSTER_NAME] \ --zone=[ZONE] \ --enable-network-policy
After you enable Network Policy, you have to actually define a policy. Since this is specific to your exact topology, we can’t provide a specific recommendation. The Kubernetes documentation, however, has an excellent walkthrough for a simple nginx deployment.
To learn more about Network Policies in Kubernetes Engine, see Setting a Cluster Network Policy.
Use least privilege service accounts for your nodes
Each Kubernetes Engine node has an IAM Service Account associated with it. By default, nodes are given the Compute Engine default service account, which you can find by navigating to the IAM section of the Cloud Console. This account has broad access by default, making it useful to wide variety of applications, but it has more permissions than are required to run your Kubernetes Engine cluster. You should create and use a minimally privileged service account to run your Kubernetes Engine cluster instead of using the Compute Engine default service account.
The following commands create an IAM service account with the minimum permissions required to operate Kubernetes Engine:
gcloud iam service-accounts create [SA_NAME] \ --display-name=[SA_NAME] gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/logging.logWriter gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/monitoring.metricWriter gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/monitoring.viewer
If you use private images in Google Container Registry, you also need to grant access to those:
gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "serviceAccount:[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --role roles/storage.objectViewer
If your cluster already exists, you can now create a new node pool with this new service account:
gcloud container node-pools create [NODE_POOL] \ --service-account=[SA_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \ --cluster=[CLUSTER_NAME]
If you need your Kubernetes Engine cluster to have access to other Google Cloud services, you should create an additional service account and grant your workloads access to the service account by storing its private key in a Kubernetes Secret. To learn more, refer to Authenticating to Google Cloud Platform with Service Accounts.
Reduce your node service account scopes
If you don't want to use a custom service account, an alternative recommendation is to reduce the permissions of the default node service account.
By default, your node service account has access scopes. Access scopes are the legacy method of specifying permissions for your instance. Before the existence of IAM roles, access scopes were the only mechanism for granting permissions to service accounts.
If you are not creating a separate service account for your nodes, you should limit the scopes of the node service account to reduce the possibility of a privilege escalation in an attack. This ensures that your default service account does not have permissions beyond those necessary to run your cluster. While the default scopes are limited, they may include scopes beyond the minimally required scopes needed to run your cluster.
The default scopes for the nodes in Kubernetes Engine are
trace.append. When setting scopes, these are specified as
If you are accessing private images in Google Container Registry, the minimally
required scopes are only
To create a cluster with the custom scopes, use the
gcloud container clusters create [CLUSTER_NAME] \ --scopes=[CUSTOM_SCOPES]
If you do not specify scopes or a custom service account,
gke-default is used.
If you specify a custom service account,
gcloud configuration includes
this behavior is already enabled for all Kubernetes versions. To set the default
for your environment:
gcloud config set container/new_scopes_behavior true
To learn more about permissions for all scopes, see Google scopes.
Restrict client authentication methods
There are several methods of authenticating to the Kubernetes API
server. In Kubernetes Engine, the supported methods are OpenID connect tokens,
x509 client certificates, and static passwords. Kubernetes Engine manages
gcloud for you using the OpenID connect token method,
setting up the Kubernetes configuration, getting an access token, and keeping
it up to date.
The other authentication methods, x509 certificates and static passwords, present a wider surface of attack for cluster compromise. With these other methods, credentials are generated for you when you create a new cluster. Unless your application is using these methods of authentication, you should disable them. You should use the OpenID authentication method, and disable your cluster's client certificate and static password authentication methods.
The client certificate and static password can only be retrieved by a user
container.clusters.getCredentials permission. Note that the
roles/editor roles all have this
permission, so use those roles wisely. Read more about Kubernetes Engine
Disable authentication with a client certificate
With certificate authentication, a client presents a certificate that the API server verifies with the specified certificate authority. In Kubernetes Engine, client certificates are signed by the cluster root Certificate Authority.
With ABAC, client certificates can authenticate to the API server by default. But with RBAC enabled, client certificates must be granted permissions. A cluster with RBAC enabled and ABAC disabled still has these certificates, but the certificates are effectively useless.
To create a cluster without generating a client certificate, use the
gcloud container clusters create [CLUSTER_NAME] \ --no-issue-client-certificate
Currently, there is no way to remove a client certificate from an existing cluster.
Disable authentication with a static password
A static password is a username and password combination that the API server validates. In Kubernetes Engine, these are generated for a username "admin" by default.
To create a cluster without generating a static password, use the
gcloud container clusters create [CLUSTER_NAME] \ --no-enable-basic-auth
To update an existing cluster and remove the static password:
gcloud container clusters update [CLUSTER_NAME] \ --no-enable-basic-auth
Conceal node metadata (Beta)
Some practical attacks against Kubernetes rely on access to the VM's metadata server to extract the node’s credentials.
Using a minimally privileged service account as advised above is the first step to mitigate these attacks. The next step is to prevent workloads from impersonating nodes. You should use metadata concealment to protect potentially sensitive system metadata from workloads running on your cluster.
To learn more about metadata concealment, see Protecting Cluster Metadata.
Restrict Pod permissions with a Pod Security Policy (Beta)
By default, Pods in Kubernetes can operate with capabilities beyond what they require. You should constrain the Pod's capabilities to only those required for that workload.
Kubernetes offers controls for restricting your Pods to execute with only their minimum required capabilities. Pod Security Policy allows you to set smart defaults for your Pods, and enforce controls you want to enable across your fleet. The policies you define should be specific to the needs of your application. The restricted-psp.yaml example policy is a good starting point.
To learn more about Pod Security Policy, see Using PodSecurityPolicies.