使用 Pod 安全政策限制条件

政策控制器附带默认限制条件模板库,可用于 Pod 安全政策软件包以实现许多与 Kubernetes Pod 安全政策 (PSP) 相同的保护功能,它具有在强制执行政策前测试政策并排除特定资源的覆盖范围的额外功能。

该软件包包含这些限制条件,它们提供了映射到以下 Kubernetes Pod 安全政策 (PSP) 字段名称的参数:

限制 PSP 字段名称 PSP 控制项
psp-v2022-psp-allow-privilege-escalation allowPrivilegeEscalation 限制升级到 root 特权
psp-v2022-psp-apparmor annotations 容器使用的 AppArmor 配置文件
psp-v2022-psp-capabilities allowedCapabilities、requiredDropCapabilities Linux 功能
psp-v2022-flexvolume-drivers allowedFlexVolumes 允许特定的 FlexVolume 驱动程序
psp-v2022-psp-forbidden-sysctls forbiddenSysctls 容器使用的 sysctl 配置文件
psp-v2022-psp-fsgroup fsGroup 分配拥有 Pod 卷的 FSGroup
psp-v2022-psp-host-filesystem allowedHostPaths 使用主机文件系统
psp-v2022-psp-host-namespace hostPID、hostIPC 使用主机命名空间
psp-v2022-psp-host-network-ports hostNetwork、hostPorts 使用主机网络和端口
psp-v2022-psp-privileged-container 特权 | 具有特权的 运行特权容器
psp-v2022-psp-proc-mount allowedProcMountTypes 容器允许的 Proc 装载类型
psp-v2022-psp-readonlyrootfilesystem readOnlyRootFilesystem 要求使用只读根文件系统
psp-v2022-psp-seccomp annotations 容器使用的 seccomp 配置文件
psp-v2022-psp-selinux-v2 seLinux 容器的 SELinux 上下文
psp-v2022-psp-pods-allowed-user-ranges runAsUser、runAsGroup、supplementalGroups、fsGroup 容器的用户 ID 和群组 ID
psp-v2022-psp-volume-types volumes 使用卷类型

准备工作

  1. 安装并初始化 Google Cloud CLI,它提供了以下说明中使用的 gcloudkubectl 命令。如果您使用 Cloud Shell,则系统会预安装 Google Cloud CLI。
  2. 在集群上安装政策控制器以及默认的限制条件模板库。

使用 Policy Controller 进行审核

政策控制器允许您为 Kubernetes 集群强制执行政策。为了帮助测试您的工作负载及其对上表中列出的 Google 推荐最佳实践的合规性,您可以在“审核”模式下部署这些限制条件以发现违规行为,更重要的是让您有机会在 Kubernetes 集群上强制执行之前处理这些违规行为。

您可以在 spec.enforcementAction 设置为 dryrun 的情况下,使用 kubectl、kptConfig Sync 应用这些政策。

kubectl

  1. (可选)使用 kubectl 预览政策限制条件:

    kubectl kustomize https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022
    
  2. 使用 kubectl 应用政策限制条件:

    kubectl apply -k https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022
    

    输出如下所示:

    k8spspallowprivilegeescalationcontainer.constraints.gatekeeper.sh/psp-v2022-psp-allow-privilege-escalation created
    k8spspallowedusers.constraints.gatekeeper.sh/psp-v2022-psp-pods-allowed-user-ranges created
    k8spspapparmor.constraints.gatekeeper.sh/psp-v2022-psp-apparmor created
    k8spspcapabilities.constraints.gatekeeper.sh/psp-v2022-psp-capabilities created
    k8spspfsgroup.constraints.gatekeeper.sh/psp-v2022-psp-fsgroup created
    k8spspflexvolumes.constraints.gatekeeper.sh/psp-v2022-psp-flexvolume-drivers created
    k8spspforbiddensysctls.constraints.gatekeeper.sh/psp-v2022-psp-forbidden-sysctls created
    k8spsphostfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-host-filesystem created
    k8spsphostnamespace.constraints.gatekeeper.sh/psp-v2022-psp-host-namespace created
    k8spsphostnetworkingports.constraints.gatekeeper.sh/psp-v2022-psp-host-network-ports created
    k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-v2022-psp-privileged-container created
    k8spspprocmount.constraints.gatekeeper.sh/psp-v2022-psp-proc-mount created
    k8spspreadonlyrootfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-readonlyrootfilesystem created
    k8spspselinuxv2.constraints.gatekeeper.sh/psp-v2022-psp-selinux-v2 created
    k8spspseccomp.constraints.gatekeeper.sh/psp-v2022-psp-seccomp created
    k8spspvolumetypes.constraints.gatekeeper.sh/psp-v2022-psp-volume-types created
    
  3. 验证政策限制条件是否已安装,并检查集群中是否存在违规行为:

    kubectl get -k https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022
    

    输出类似于以下内容:

    NAME                                                                                                         ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspallowprivilegeescalationcontainer.constraints.gatekeeper.sh/psp-v2022-psp-allow-privilege-escalation   dryrun               0
    
    NAME                                                                                  ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspallowedusers.constraints.gatekeeper.sh/psp-v2022-psp-pods-allowed-user-ranges   dryrun               0
    
    NAME                                                              ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspapparmor.constraints.gatekeeper.sh/psp-v2022-psp-apparmor                        0
    
    NAME                                                                      ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspcapabilities.constraints.gatekeeper.sh/psp-v2022-psp-capabilities   dryrun               0
    
    NAME                                                            ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspfsgroup.constraints.gatekeeper.sh/psp-v2022-psp-fsgroup                        0
    
    NAME                                                                           ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspflexvolumes.constraints.gatekeeper.sh/psp-v2022-psp-flexvolume-drivers                        0
    
    NAME                                                                               ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspforbiddensysctls.constraints.gatekeeper.sh/psp-v2022-psp-forbidden-sysctls                        0
    
    NAME                                                                           ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spsphostfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-host-filesystem                        0
    
    NAME                                                                         ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spsphostnamespace.constraints.gatekeeper.sh/psp-v2022-psp-host-namespace   dryrun               0
    
    NAME                                                                                   ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spsphostnetworkingports.constraints.gatekeeper.sh/psp-v2022-psp-host-network-ports   dryrun               0
    
    NAME                                                                                     ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-v2022-psp-privileged-container   dryrun               0
    
    NAME                                                                 ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspprocmount.constraints.gatekeeper.sh/psp-v2022-psp-proc-mount                        0
    
    NAME                                                                                          ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspreadonlyrootfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-readonlyrootfilesystem                        0
    
    NAME                                                                 ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspselinuxv2.constraints.gatekeeper.sh/psp-v2022-psp-selinux-v2                        0
    
    NAME                                                            ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspseccomp.constraints.gatekeeper.sh/psp-v2022-psp-seccomp   dryrun               0
    
    NAME                                                                     ENFORCEMENT-ACTION   TOTAL-VIOLATIONS
    k8spspvolumetypes.constraints.gatekeeper.sh/psp-v2022-psp-volume-types                        0
    
  4. (可选)根据集群环境的要求,调整限制条件文件中的 PSP 字段名称 parameters。如需了解详情,请查看上表中特定 PSP 字段名称的链接。例如,在 psp-host-network-ports 中:

    parameters:
      hostNetwork: true
      min: 80
      max: 9000
    

kpt

  1. 安装并设置 kpt。在这些说明中,kpt 用于自定义和部署 Kubernetes 资源。

  2. 使用 kpt 从 GitHub 下载 Pod 安全政策 (PSP) v2022 政策捆绑包:

    kpt pkg get https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022
    
  3. 运行 set-enforcement-action kpt 函数将政策的强制执行操作设置为 dryrun

    kpt fn eval psp-v2022 -i gcr.io/kpt-fn/set-enforcement-action:v0.1 \
      -- enforcementAction=dryrun
    
  4. 使用 kpt 初始化工作目录,此操作会创建一个资源来跟踪更改:

    cd psp-v2022
    kpt live init
    
  5. (可选)根据集群环境的要求,调整限制条件文件中的 PSP 字段名称 parameters。如需了解详情,请查看上表中特定 PSP 字段名称的链接。例如,在 psp-host-network-ports 中:

    parameters:
      hostNetwork: true
      min: 80
      max: 9000
    
  6. 使用 kpt 应用政策限制条件:

    kpt live apply
    
  7. 验证政策限制条件是否已安装,并检查集群中是否存在违规行为:

    kpt live status --output table --poll-until current
    

    状态为 CURRENT,即确认限制条件成功安装。

Config Sync

  1. 安装并设置 kpt。在这些说明中,kpt 用于自定义和部署 Kubernetes 资源。

  2. 如果您是第一次使用 Anthos Config Management,请启用 Anthos Config Management

使用 Config Sync 将政策部署到其集群的运维人员可以按照以下说明操作:

  1. 更改为 Config Sync 的同步目录:

    cd SYNC_ROOT_DIR
    

    如需使用 resourcegroup.yaml 创建或附加 .gitignore,请执行以下操作:

    echo resourcegroup.yaml >> .gitignore
    

  2. 创建一个专用的 policies 目录:

    mkdir -p policies
    
  3. 使用 kpt 从 GitHub 下载 Pod 安全政策 (PSP) v2022 政策捆绑包:

    kpt pkg get https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022 policies/psp-v2022
    
  4. 运行 set-enforcement-action kpt 函数将政策的强制执行操作设置为 dryrun

    kpt fn eval policies/psp-v2022 -i gcr.io/kpt-fn/set-enforcement-action:v0.1 -- enforcementAction=dryrun
    
  5. (可选)根据集群环境的要求,调整限制条件文件中的 PSP 字段名称 parameters。如需了解详情,请查看上表中特定 PSP 字段名称的链接。例如,在 psp-host-network-ports 中:

    parameters:
      hostNetwork: true
      min: 80
      max: 9000
    
  6. (可选)预览将创建的政策限制条件:

    kpt live init policies/psp-v2022
    kpt live apply --dry-run policies/psp-v2022
    

    输出如下所示:

    Dry-run strategy: client
    inventory update started
    inventory update finished
    apply phase started
    k8spspallowprivilegeescalationcontainer.constraints.gatekeeper.sh/psp-v2022-psp-allow-privilege-escalation apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspallowedusers.constraints.gatekeeper.sh/psp-v2022-psp-pods-allowed-user-ranges apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspapparmor.constraints.gatekeeper.sh/psp-v2022-psp-apparmor apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspcapabilities.constraints.gatekeeper.sh/psp-v2022-psp-capabilities apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspfsgroup.constraints.gatekeeper.sh/psp-v2022-psp-fsgroup apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspflexvolumes.constraints.gatekeeper.sh/psp-v2022-psp-flexvolume-drivers apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspforbiddensysctls.constraints.gatekeeper.sh/psp-v2022-psp-forbidden-sysctls apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spsphostfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-host-filesystem apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spsphostnamespace.constraints.gatekeeper.sh/psp-v2022-psp-host-namespace apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spsphostnetworkingports.constraints.gatekeeper.sh/psp-v2022-psp-host-network-ports apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-v2022-psp-privileged-container apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspprocmount.constraints.gatekeeper.sh/psp-v2022-psp-proc-mount apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspreadonlyrootfilesystem.constraints.gatekeeper.sh/psp-v2022-psp-readonlyrootfilesystem apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspselinuxv2.constraints.gatekeeper.sh/psp-v2022-psp-selinux-v2 apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspseccomp.constraints.gatekeeper.sh/psp-v2022-psp-seccomp apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    k8spspvolumetypes.constraints.gatekeeper.sh/psp-v2022-psp-volume-types apply skipped: inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
    apply phase finished
    inventory update started
    inventory update finished
    apply result: 16 attempted, 0 successful, 16 skipped, 0 failed
    
  7. 如果 Config Sync 的同步目录使用 Kustomize,请将 policies/psp-v2022 添加到根 kustomization.yaml。否则,请移除 policies/psp-v2022/kustomization.yaml 文件:

    rm SYNC_ROOT_DIR/policies/psp-v2022/kustomization.yaml
    
  8. 将更改推送到 Config Sync 代码库:

    git add SYNC_ROOT_DIR/policies/psp-v2022
    git commit -m 'Adding Pod Security Policy audit enforcement'
    git push
    
  9. 验证安装状态:

    watch gcloud beta container fleet config-management status --project PROJECT_ID
    

    状态 SYNCED 用于确认政策的安装情况。

查看违规问题

在审核模式下安装政策限制条件后,您可以使用政策控制器信息中心在界面中查看集群的违规问题。

您还可以使用 kubectl 来通过以下命令查看针对集群的违规行为:

kubectl get constraint -l policycontroller.gke.io/bundleName=psp-v2022 -o json | jq -cC '.items[]| [.metadata.name,.status.totalViolations]'

如果存在违规行为,您可以使用以下命令查看每个限制条件的违规消息列表:

kubectl get constraint -l policycontroller.gke.io/bundleName=psp-v2022 -o json | jq -C '.items[]| select(.status.totalViolations>0)| [.metadata.name,.status.violations[]?]'

强制执行安全政策

查看集群中的政策违规并根据需要调整任何 PSP 字段名称参数后,下一步就是强制执行这些政策,以使准入控制器阻止任何不合规的资源应用于集群。

kubectl

  1. 使用 kubectl 将政策的强制执行操作设置为 deny

    kubectl get constraint -l policycontroller.gke.io/bundleName=psp-v2022 -o json | jq '.items[].spec.enforcementAction="deny"' | kubectl apply -f -
    
  2. 验证政策限制条件强制执行措施是否已更新:

    kubectl get -k https://github.com/GoogleCloudPlatform/acm-policy-controller-library.git/bundles/psp-v2022
    

kpt

  1. 运行 set-enforcement-action kpt 函数将政策的强制执行操作设置为 deny

    kpt fn eval -i gcr.io/kpt-fn/set-enforcement-action:v0.1 -- enforcementAction=deny
    
  2. 应用政策限制条件:

    kpt live apply
    

Config Sync

使用 Config Sync 将政策部署到其集群的运维人员可以按照以下说明操作:

  1. 更改为 Config Sync 的同步目录:

    cd SYNC_ROOT_DIR
    
  2. 运行 set-enforcement-action kpt 函数将政策的强制执行操作设置为 deny

    kpt fn eval policies/psp-v2022 -i gcr.io/kpt-fn/set-enforcement-action:v0.1 -- enforcementAction=deny
    
  3. 将更改推送到 Config Sync 代码库:

    git add SYNC_ROOT_DIR/policies/psp-v2022
    git commit -m 'Enforcing Pod Security Policy policies for GKE'
    git push
    
  4. 验证安装状态:

    gcloud alpha anthos config sync repo list --project PROJECT_ID
    

    SYNCED 列中显示的代码库将确认政策的安装情况。

测试政策执行情况

执行快速测试以确认政策执行和对集群上不合规资源的阻止。

使用以下命令在集群上创建不合规的资源:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: wp-non-compliant
  labels:
    app: wordpress
spec:
  containers:
    - image: wordpress
      name: wordpress
      ports:
      - containerPort: 80
        name: wordpress
EOF

准入控制器应生成一个错误,其中列出了此资源违反政策的行为。 例如:

Error from server (Forbidden): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request:
[psp-v2022-psp-pods-allowed-user-ranges] Container wordpress is attempting to run without a required securityContext/fsGroup. Allowed fsGroup: {"ranges": [{"max": 200, "min": 100}], "rule": "MustRunAs"}
[psp-v2022-psp-pods-allowed-user-ranges] Container wordpress is attempting to run without a required securityContext/runAsGroup. Allowed runAsGroup: {"ranges": [{"max": 200, "min": 100}], "rule": "MustRunAs"}
[psp-v2022-psp-pods-allowed-user-ranges] Container wordpress is attempting to run without a required securityContext/runAsUser
[psp-v2022-psp-pods-allowed-user-ranges] Container wordpress is attempting to run without a required securityContext/supplementalGroups. Allowed supplementalGroups: {"ranges": [{"max": 200, "min": 100}], "rule": "MustRunAs"}
[psp-v2022-psp-allow-privilege-escalation] Privilege escalation container is not allowed: wordpress
[psp-v2022-psp-seccomp] Seccomp profile 'not configured' is not allowed for container 'wordpress'. Found at: no explicit profile found. Allowed profiles: {"RuntimeDefault", "docker/default", "runtime/default"}
[psp-v2022-psp-capabilities] container  is not dropping all required capabilities. Container must drop all of ["must_drop"] or "ALL"
[psp-v2022-psp-readonlyrootfilesystem] only read-only root filesystem container is allowed: wordpress