CIS Kubernetes Benchmark

This document introduces the CIS Kubernetes Benchmark, explains how to audit your compliance with the benchmark, and explains what Google Distributed Cloud configures when you cannot implement a recommendation yourself.

About the CIS Benchmarks

The Center for Internet Security (CIS) releases benchmarks for best practice security recommendations. The CIS Kubernetes Benchmark provides a set of recommendations for configuring Kubernetes to support a strong security posture. The Benchmark is tied to a specific Kubernetes release. The CIS Kubernetes Benchmark is written for the open source Kubernetes distribution and intended to be as universally applicable across distributions as possible.

Access the benchmark

The CIS Kubernetes Benchmark is available on the CIS website.

Recommendation levels

The following table describes the recommendation levels in the CIS Kubernetes Benchmark.

Level Description
Level 1

Recommendations intend to:

  • be practical and prudent;
  • provide a clear security benefit; and
  • not inhibit the utility of the technology beyond acceptable means.

    		•
    Level 2

    Extends the Level 1 profile.

    Recommendations exhibit one or more of the following characteristics:

  • are intended for environments or use cases where security is paramount;
  • acts as defense in depth measure; or
  • may negatively inhibit the utility or performance of the technology.

    		•

    Assessment status

    An assessment status is included for every recommendation. The assessment status indicates whether the given recommendation can be automated or requires manual steps to implement. Both statuses are equally important and are determined and supported as defined in the following tables:

    1.30 and higher

    To get the most up-to-date benchmark information, we recommend that you generate your own benchmarks using the open-source tool, kube-bench.

    1.29

    Versions

    This section refers to the following versions:

    Anthos version Kubernetes version CIS Kubernetes Benchmark version
    1.29 1.29.4 1.9

    Status of Google Distributed Cloud admin cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.17 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.19 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.22 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.23 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.24 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.29 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud admin cluster

    # Recommendation Level Status Value Justification
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's API server pod specification is stored in etcd.
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's controller manager pod specification is stored in etcd.
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's scheduler pod specification is stored in etcd.
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's etcd specification is stored in admin cluster's etcd.
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control 2001:2001 The etcd container runs as 2001 and the etcd data directory is owned by 2001:2001.
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2000:2000 The kube-scheduler container runs as 2000 and this file is owned by 2000:2000.
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2002:2002 The controller-manager container runs as 2002 and this file is owned by 2002:2002.
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment not set The AlwaysPullImages admission controller provides some protection for private registry images in noncooperative multitenant clusters, at the cost of making container registries a single-point-of-failure for creating new Pods across the entire cluster. Google Distributed Cloud does not enable the AlwaysPullImages admission controller, which leaves it up to cluster admins to implement admission policy to make this tradeoff for themselves.
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control not set PodSecurityPolicy will be removed from Kubernetes in 1.25. As a replacement, Pod Security Admission is enabled by default as of 1.23.
    1.2.18 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    1.2.19 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.

    Status of Google Distributed Cloud user cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.17 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.19 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.22 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.23 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.24 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.29 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Warn
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Warn
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Automated) L1 Fail
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Automated) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud user cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.17 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.19 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.22 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.23 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.24 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.29 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Fail
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Warn
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Warn
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Automated) L1 Fail
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Automated) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    1.28

    Versions

    This section refers to the following versions:

    Anthos version Kubernetes version CIS Kubernetes Benchmark version
    1.28 1.29.7 1.7

    Status of Google Distributed Cloud admin cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --secure-port argument is not set to 0 - NoteThis recommendation is obsolete and will be deleted per the consensus process (Manual) L1 Pass
    1.2.17 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.22 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.23 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.24 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.29 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.30 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Pass
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Manual) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud admin cluster

    # Recommendation Level Status Value Justification
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's API server pod specification is stored in etcd.
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's controller manager pod specification is stored in etcd.
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's scheduler pod specification is stored in etcd.
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's etcd specification is stored in admin cluster's etcd.
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control 2001:2001 The etcd container runs as 2001 and the etcd data directory is owned by 2001:2001.
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2000:2000 The kube-scheduler container runs as 2000 and this file is owned by 2000:2000.
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2002:2002 The controller-manager container runs as 2002 and this file is owned by 2002:2002.
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment not set The AlwaysPullImages admission controller provides some protection for private registry images in noncooperative multitenant clusters, at the cost of making container registries a single-point-of-failure for creating new Pods across the entire cluster. Google Distributed Cloud does not enable the AlwaysPullImages admission controller, which leaves it up to cluster admins to implement admission policy to make this tradeoff for themselves.
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control not set PodSecurityPolicy will be removed from Kubernetes in 1.25. As a replacement, Pod Security Admission is enabled by default as of 1.23.
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control not set Google Distributed Cloud manages kubelet server TLS using the --rotate-server-certificates flag.

    Status of Google Distributed Cloud user cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --secure-port argument is not set to 0 - NoteThis recommendation is obsolete and will be deleted per the consensus process (Manual) L1 Pass
    1.2.17 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.22 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.23 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.24 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.29 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.30 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Pass
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Manual) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud user cluster

    # Recommendation Level Status Value Justification
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's API server pod specification is stored in etcd.
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's controller manager pod specification is stored in etcd.
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's scheduler pod specification is stored in etcd.
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's etcd specification is stored in admin cluster's etcd.
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control 2001:2001 The etcd container runs as 2001 and the etcd data directory is owned by 2001:2001.
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2000:2000 The kube-scheduler container runs as 2000 and this file is owned by 2000:2000.
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2002:2002 The controller-manager container runs as 2002 and this file is owned by 2002:2002.
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment not set The AlwaysPullImages admission controller provides some protection for private registry images in noncooperative multitenant clusters, at the cost of making container registries a single-point-of-failure for creating new Pods across the entire cluster. Google Distributed Cloud does not enable the AlwaysPullImages admission controller, which leaves it up to cluster admins to implement admission policy to make this tradeoff for themselves.
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control not set PodSecurityPolicy will be removed from Kubernetes in 1.25. As a replacement, Pod Security Admission is enabled by default as of 1.23.
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control not set Google Distributed Cloud manages kubelet server TLS using the --rotate-server-certificates flag.

    1.16

    Versions

    This section refers to the following versions:

    Anthos version Kubernetes version CIS Kubernetes Benchmark version
    1.16.0 1.27.1 1.7

    Status of Google Distributed Cloud admin cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --secure-port argument is not set to 0 - NoteThis recommendation is obsolete and will be deleted per the consensus process (Manual) L1 Pass
    1.2.17 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.22 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.23 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.24 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.29 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.30 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Pass
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Manual) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud admin cluster

    # Recommendation Level Status Value Justification
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's API server pod specification is stored in etcd.
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's controller manager pod specification is stored in etcd.
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's scheduler pod specification is stored in etcd.
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's etcd specification is stored in admin cluster's etcd.
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control 2001:2001 The etcd container runs as 2001 and the etcd data directory is owned by 2001:2001.
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2000:2000 The kube-scheduler container runs as 2000 and this file is owned by 2000:2000.
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2002:2002 The controller-manager container runs as 2002 and this file is owned by 2002:2002.
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment not set The AlwaysPullImages admission controller provides some protection for private registry images in noncooperative multitenant clusters, at the cost of making container registries a single-point-of-failure for creating new Pods across the entire cluster. Google Distributed Cloud does not enable the AlwaysPullImages admission controller, which leaves it up to cluster admins to implement admission policy to make this tradeoff for themselves.
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control not set PodSecurityPolicy will be removed from Kubernetes in 1.25. As a replacement, Pod Security Admission is enabled by default as of 1.23.
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control not set Google Distributed Cloud manages kubelet server TLS using the --rotate-server-certificates flag.

    Status of Google Distributed Cloud user cluster

    # Recommendation Level Status
    1 Control Plane Security Configuration
    1.1 Control Plane Node Configuration Files
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control
    1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated) L1 Pass
    1.1.9 Ensure that the Container Network Interface file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual) L1 Pass
    1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated) L1 Pass
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control
    1.1.13 Ensure that the admin.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated) L1 Pass
    1.1.15 Ensure that the scheduler.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.17 Ensure that the controller-manager.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control
    1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Automated) L1 Pass
    1.1.20 Ensure that the Kubernetes PKI certificate file permissions are set to 600 or more restrictive (Manual) L1 Pass
    1.1.21 Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual) L1 Pass
    1.2 API Server
    1.2.1 Ensure that the --anonymous-auth argument is set to false (Manual) L1 Pass
    1.2.2 Ensure that the --token-auth-file parameter is not set (Automated) L1 Pass
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn
    1.2.4 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Automated) L1 Pass
    1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated) L1 Pass
    1.2.6 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    1.2.7 Ensure that the --authorization-mode argument includes Node (Automated) L1 Pass
    1.2.8 Ensure that the --authorization-mode argument includes RBAC (Automated) L1 Pass
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn
    1.2.10 Ensure that the admission control plugin AlwaysAdmit is not set (Automated) L1 Pass
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control
    1.2.13 Ensure that the admission control plugin ServiceAccount is set (Automated) L1 Pass
    1.2.14 Ensure that the admission control plugin NamespaceLifecycle is set (Automated) L1 Pass
    1.2.15 Ensure that the admission control plugin NodeRestriction is set (Automated) L1 Pass
    1.2.16 Ensure that the --secure-port argument is not set to 0 - NoteThis recommendation is obsolete and will be deleted per the consensus process (Manual) L1 Pass
    1.2.17 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.2.18 Ensure that the --audit-log-path argument is set (Automated) L1 Pass
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control
    1.2.20 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated) L1 Pass
    1.2.21 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated) L1 Pass
    1.2.22 Ensure that the --request-timeout argument is set as appropriate (Manual) L1 Pass
    1.2.23 Ensure that the --service-account-lookup argument is set to true (Automated) L1 Pass
    1.2.24 Ensure that the --service-account-key-file argument is set as appropriate (Automated) L1 Pass
    1.2.25 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Automated) L1 Pass
    1.2.26 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Automated) L1 Pass
    1.2.27 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    1.2.28 Ensure that the --etcd-cafile argument is set as appropriate (Automated) L1 Pass
    1.2.29 Ensure that the --encryption-provider-config argument is set as appropriate (Manual) L1 Pass
    1.2.30 Ensure that encryption providers are appropriately configured (Manual) L1 Pass
    1.2.31 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    1.3 Controller Manager
    1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Manual) L1 Pass
    1.3.2 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated) L1 Pass
    1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated) L1 Pass
    1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated) L1 Pass
    1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated) L2 Pass
    1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    1.4 Scheduler
    1.4.1 Ensure that the --profiling argument is set to false (Automated) L1 Pass
    1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) L1 Pass
    2 Etcd Node Configuration
    2 Etcd Node Configuration
    2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated) L1 Pass
    2.2 Ensure that the --client-cert-auth argument is set to true (Automated) L1 Pass
    2.3 Ensure that the --auto-tls argument is not set to true (Automated) L1 Pass
    2.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Automated) L1 Pass
    2.5 Ensure that the --peer-client-cert-auth argument is set to true (Automated) L1 Pass
    2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated) L1 Pass
    2.7 Ensure that a unique Certificate Authority is used for etcd (Manual) L2 Pass
    3 Control Plane Configuration
    3.1 Authentication and Authorization
    3.1.1 Client certificate authentication should not be used for users (Manual) L2 Pass
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn
    3.2 Logging
    3.2.1 Ensure that a minimal audit policy is created (Manual) L1 Pass
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control
    4 Worker Node Security Configuration
    4.1 Worker Node Configuration Files
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail
    4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated) L1 Pass
    4.1.3 If proxy kubeconfig file exists ensure permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.4 If proxy kubeconfig file exists ensure ownership is set to root:root (Manual) L1 Pass
    4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 600 or more restrictive (Automated) L1 Pass
    4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Automated) L1 Pass
    4.1.7 Ensure that the certificate authorities file permissions are set to 600 or more restrictive (Manual) L1 Pass
    4.1.8 Ensure that the client certificate authorities file ownership is set to root:root (Manual) L1 Pass
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn
    4.1.10 If the kubelet config.yaml configuration file is being used validate file ownership is set to root:root (Manual) L1 Pass
    4.2 Kubelet
    4.2.1 Ensure that the --anonymous-auth argument is set to false (Automated) L1 Pass
    4.2.2 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Automated) L1 Pass
    4.2.3 Ensure that the --client-ca-file argument is set as appropriate (Automated) L1 Pass
    4.2.4 Verify that the --read-only-port argument is set to 0 (Manual) L1 Pass
    4.2.5 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Manual) L1 Pass
    4.2.6 Ensure that the --make-iptables-util-chains argument is set to true (Automated) L1 Pass
    4.2.7 Ensure that the --hostname-override argument is not set (Manual) L1 Pass
    4.2.8 Ensure that the eventRecordQPS argument is set to a level which ensures appropriate event capture (Manual) L1 Pass
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control
    4.2.10 Ensure that the --rotate-certificates argument is not set to false (Automated) L1 Pass
    4.2.11 Verify that the RotateKubeletServerCertificate argument is set to true (Manual) L1 Pass
    4.2.12 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual) L1 Pass
    4.2.13 Ensure that a limit is set on pod PIDs (Manual) L1 Pass

    Descriptions of Failures and Equivalent Controls for Google Distributed Cloud user cluster

    # Recommendation Level Status Value Justification
    1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's API server pod specification is stored in etcd.
    1.1.3 Ensure that the controller manager pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's controller manager pod specification is stored in etcd.
    1.1.5 Ensure that the scheduler pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's scheduler pod specification is stored in etcd.
    1.1.7 Ensure that the etcd pod specification file permissions are set to 600 or more restrictive (Automated) L1 Equivalent Control N/A In kubeception mode, user cluster's etcd specification is stored in admin cluster's etcd.
    1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated) L1 Equivalent Control 2001:2001 The etcd container runs as 2001 and the etcd data directory is owned by 2001:2001.
    1.1.16 Ensure that the scheduler.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2000:2000 The kube-scheduler container runs as 2000 and this file is owned by 2000:2000.
    1.1.18 Ensure that the controller-manager.conf file ownership is set to root:root (Automated) L1 Equivalent Control 2002:2002 The controller-manager container runs as 2002 and this file is owned by 2002:2002.
    1.2.3 Ensure that the --DenyServiceExternalIPs is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.9 Ensure that the admission control plugin EventRateLimit is set (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    1.2.11 Ensure that the admission control plugin AlwaysPullImages is set (Manual) L1 Depends On Environment not set The AlwaysPullImages admission controller provides some protection for private registry images in noncooperative multitenant clusters, at the cost of making container registries a single-point-of-failure for creating new Pods across the entire cluster. Google Distributed Cloud does not enable the AlwaysPullImages admission controller, which leaves it up to cluster admins to implement admission policy to make this tradeoff for themselves.
    1.2.12 Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used (Manual) L1 Equivalent Control not set PodSecurityPolicy will be removed from Kubernetes in 1.25. As a replacement, Pod Security Admission is enabled by default as of 1.23.
    1.2.19 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated) L1 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    3.1.2 Service account token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.1.3 Bootstrap token authentication should not be used for users (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    3.2.2 Ensure that the audit policy covers key security concerns (Manual) L2 Equivalent Control not set Google Distributed Cloud captures audit logs but does not use these flags for auditing. See Google Distributed Cloud Audit policy for more details.
    4.1.1 Ensure that the kubelet service file permissions are set to 600 or more restrictive (Automated) L1 Fail not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.1.9 If the kubelet config.yaml configuration file is being used validate permissions set to 600 or more restrictive (Manual) L1 Warn not set Anthos clusters on VMware does not support the Event Rate Limit admission controller as it is a Kubernetes Alpha feature.
    4.2.9 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual) L2 Equivalent Control not set Google Distributed Cloud manages kubelet server TLS using the --rotate-server-certificates flag.

    Audit benchmarks

    Specific instructions for auditing each recommendation is available as part of the relevant CIS Benchmark. However, you may wish to automate some of these checks to simplify the verification of these controls in your environment. The following tool can help with this.

    Automated auditing of the CIS Kubernetes Benchmark

    You can use an open-source tool kube-bench to test your cluster configuration against the CIS Kubernetes Benchmark.

    Make sure to specify the appropriate version. For example:

    kube-bench node --benchmark cis-1.7