This page explains the recommended way to apply Pod-level security controls to your Google Kubernetes Engine (GKE) clusters.
Overview
Gatekeeper is an admission controller that validates requests to create and update Pods on Kubernetes clusters, using the Open Policy Agent (OPA).
Using Gatekeeper allows administrators to define policies with a constraint,
which is a set of conditions that permit or deny deployment behaviors in
Kubernetes. You can then enforce these policies on a cluster using a
ConstraintTemplate
. This document provides examples for restricting the
security capabilities of workloads to ensure enforce, test, and audit security
policies using Gatekeeper.
In addition to enforcing the same capabilities as Kubernetes PodSecurityPolicies, Gatekeeper can also:
- Roll out policies: Enforce policy in a gradual, scoped manner to limit the risk of disrupting workloads.
- Dry-run policy changes: Provide mechanisms for testing policy impact and range prior to enforcement.
- Audit existing policies: Ensure the application of security controls to new and existing workloads (audit controls).
Kubernetes Open Source Software (OSS) is in the process of deprecating Kubernetes PodSecurityPolicies and Google no longer recommends their use.
Concepts
Gatekeeper introduces two concepts in order to provide administrators with a powerful and flexible means of controlling their cluster: constraints, and constraint templates, both of which are concepts inherited from the Open Policy Agent Constraint Framework.
Constraints are the representation of your security policy—they define the requirements and range of enforcement. Constraint templates are reusable statements (written in Rego) that apply logic to evaluate specific fields in Kubernetes objects, based on requirements defined in constraints.
For example, you might have a constraint that declares allowable seccomp profiles that can be applied to Pods in a specific namespace, and a comparable constraint template that provides the logic for extracting these values and handling enforcement.
The following constraint template, from the Gatekeeper repository,
checks for the existence of securityContext.privileged
in a Pod specification:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8spspprivilegedcontainer
spec:
crd:
spec:
names:
kind: K8sPSPPrivilegedContainer
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8spspprivileged
violation[{"msg": msg, "details": {}}] {
c := input_containers[_]
c.securityContext.privileged
msg := sprintf("Privileged container is not allowed: %v, securityContext: %v", [c.name, c.securityContext])
}
input_containers[c] {
c := input.review.object.spec.containers[_]
}
input_containers[c] {
c := input.review.object.spec.initContainers[_]
}
To extend the above constraint template, the following constraint defines the
scope (kinds
) for the specific enforcement of this constraint template in a
dryrun
mode:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sPSPPrivilegedContainer
metadata:
name: psp-privileged-container
spec:
enforcementAction: dryrun
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
With Gatekeeper, you can create your own constraints and constraint templates to meet your specific needs. You can also use a standard set of constraints and constraint templates in the Gatekeeper repository that have been defined to enable quick adoption and security enforcement. Each constraint is also accompanied with example Pod configurations.
Before you begin
Before you start, make sure you have performed the following tasks:
- Ensure that you have enabled the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- Ensure that you have installed the Cloud SDK.
Set up default gcloud
settings using one of the following methods:
- Using
gcloud init
, if you want to be walked through setting defaults. - Using
gcloud config
, to individually set your project ID, zone, and region.
Using gcloud init
If you receive the error One of [--zone, --region] must be supplied: Please specify
location
, complete this section.
-
Run
gcloud init
and follow the directions:gcloud init
If you are using SSH on a remote server, use the
--console-only
flag to prevent the command from launching a browser:gcloud init --console-only
-
Follow the instructions to authorize
gcloud
to use your Google Cloud account. - Create a new configuration or select an existing one.
- Choose a Google Cloud project.
- Choose a default Compute Engine zone for zonal clusters or a region for regional or Autopilot clusters.
Using gcloud config
- Set your default project ID:
gcloud config set project PROJECT_ID
- If you are working with zonal clusters, set your default compute zone:
gcloud config set compute/zone COMPUTE_ZONE
- If you are working with Autopilot or regional clusters, set your default compute region:
gcloud config set compute/region COMPUTE_REGION
- Update
gcloud
to the latest version:gcloud components update
Enabling Gatekeeper on a cluster
You can enable Gatekeeper on a GKE cluster with one of the following methods. These methods cannot coexist on the same cluster, so be sure to pick only one.
Using Anthos Config Management (recommended)
Anthos Config Management offers Policy Controller, which is a policy engine built on the Gatekeeper open source project. Google recommends the use of Anthos Config Management because it solves common problems associated with enforcing policy at scale, including policy-as-code, multi-cluster support, integration with Cloud Logging, and ability to synchronize configuration.
To enable Policy Controller on a cluster, follow the Anthos Config Management installation guide.
Using Google Cloud Marketplace
Gatekeeper is also available as a Kubernetes application in the Google Cloud Marketplace. Gatekeeper can be installed either on an existing cluster, or on a newly-created cluster.
To deploy Gatekeeper from Cloud Marketplace, see Deploying an application from Google Cloud Marketplace.
Enabling constraints and constraint templates
Unlike PodSecurityPolicy, Gatekeeper and its constraint templates can be installed and enabled without adversely impacting existing or new workloads. For this reason, it's recommended that all applicable Pod security constraint templates be applied to the cluster.
Additionally, Gatekeeper constraints can be implemented to enforce controls for specific objects, such as namespaces and Pods.
Observe the example below that limits the scope to Pods located in the production namespace by defining them in the constraint match statement:
...
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
namespaces:
- "production"
Testing policies
Introducing new policies to existing clusters can have adverse behavior, for example by restricting existing workloads. One of the benefits of using Gatekeeper for Pod security is the ability to test the effectiveness and impact a policy will have without making actual changes, using a dry-run mode. This allows for policy configuration to be tested against running clusters without enforcement. Policy violations are logged and identified without interference.
The following steps demonstrate how a developer, operator, or administrator can apply constraint templates and constraints to determine their effectiveness or potential impact:
Apply the Gatekeeper config for replicating data for audit and dry-run functionality:
kubectl create -f- <<EOF apiVersion: config.gatekeeper.sh/v1alpha1 kind: Config metadata: name: config namespace: "gatekeeper-system" spec: sync: syncOnly: - group: "" version: "v1" kind: "Namespace" - group: "" version: "v1" kind: "Pod" EOF
With no constraints applied, let's run a workload with elevated privileges:
kubectl create -f- <<EOF apiVersion: v1 kind: Pod metadata: name: nginx labels: app: nginx spec: containers: - name: nginx image: nginx securityContext: privileged: true EOF
Load the
k8spspprivilegedcontainer
constraint template mentioned above:kubectl create -f- <<EOF apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8spspprivilegedcontainer spec: crd: spec: names: kind: K8sPSPPrivilegedContainer targets: - target: admission.k8s.gatekeeper.sh rego: | package k8spspprivileged violation[{"msg": msg, "details": {}}] { c := input_containers[_] c.securityContext.privileged msg := sprintf("Privileged container is not allowed: %v, securityContext: %v", [c.name, c.securityContext]) } input_containers[c] { c := input.review.object.spec.containers[_] } input_containers[c] { c := input.review.object.spec.initContainers[_] } EOF
Now let's create a new constraint to extend this constraint template. This time, we will set the
enforcementAction
todryrun
:kubectl create -f- <<EOF apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sPSPPrivilegedContainer metadata: name: psp-privileged-container spec: enforcementAction: dryrun match: kinds: - apiGroups: [""] kinds: ["Pod"] EOF
With Gatekeeper synchronizing running object data, and passively checking for violations, we can confirm if any violations were found by checking the
status
of the constraint:kubectl get k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-privileged-container -o yaml
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sPSPPrivilegedContainer metadata: ... name: psp-privileged-container ... spec: enforcementAction: dryrun match: kinds: - apiGroups: - "" kinds: - Pod status: auditTimestamp: "2019-12-15T22:19:54Z" byPod: - enforced: true id: gatekeeper-controller-manager-0 violations: - enforcementAction: dryrun kind: Pod message: 'Privileged container is not allowed: nginx, securityContext: {"privileged": true}' name: nginx namespace: default
Let's run another privileged Pod, to confirm that the policy does not interfere with deployments:
kubectl create -f- <<EOF apiVersion: v1 kind: Pod metadata: name: privpod labels: app: privpod spec: containers: - name: nginx image: nginx securityContext: privileged: true EOF
This new Pod will be successfully deployed.
To clean up the resources created in this section, run the following commands:
kubectl delete k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-privileged-container kubectl delete constrainttemplate k8spspprivilegedcontainer kubectl delete pod/nginx kubectl delete pod/privpod
Enforcing policies
Now that we can confirm the validity and impact of a policy without impacting existing or new workloads, let's implement a policy with full enforcement.
Building on the examples used to validate the policy above, the following steps demonstrate how a developer, operator, or administrator can apply constraint templates and constraints to enforce a policy:
Load the
k8spspprivilegedcontainer
constraint template mentioned earlier:kubectl create -f- <<EOF apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8spspprivilegedcontainer spec: crd: spec: names: kind: K8sPSPPrivilegedContainer targets: - target: admission.k8s.gatekeeper.sh rego: | package k8spspprivileged violation[{"msg": msg, "details": {}}] { c := input_containers[_] c.securityContext.privileged msg := sprintf("Privileged container is not allowed: %v, securityContext: %v", [c.name, c.securityContext]) } input_containers[c] { c := input.review.object.spec.containers[_] } input_containers[c] { c := input.review.object.spec.initContainers[_] } EOF
Now let's create a new constraint to extend this constraint template. This time, we won't set the
enforcementAction
key. By default, theenforcementAction
key is set todeny
:kubectl create -f- <<EOF apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sPSPPrivilegedContainer metadata: name: psp-privileged-container spec: match: kinds: - apiGroups: [""] kinds: ["Pod"] EOF
Attempt to deploy a container that declares privileged permissions:
kubectl create -f- <<EOF apiVersion: v1 kind: Pod metadata: name: nginx labels: app: nginx spec: containers: - name: nginx image: nginx securityContext: privileged: true EOF
The following error message should be received:
Error from server ([denied by psp-privileged-container] Privileged container is not allowed: nginx, securityContext: {"privileged": true}): error when creating "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: [denied by psp-privileged-container] Privileged container is not allowed: nginx, securityContext: {"privileged": true}
To clean up, run the following commands:
kubectl delete k8spspprivilegedcontainer.constraints.gatekeeper.sh/psp-privileged-container kubectl delete constrainttemplate k8spspprivilegedcontainer
What's next
Gatekeeper provides an incredibly powerful means to enforce and validate security on GKE clusters using declarative policies. Gatekeeper's use extends beyond security however, and can be used in other aspects of administration and operations.