Restrict actions on GKE resources using custom organization policies

This page shows you how to restrict specific operations on Google Kubernetes Engine (GKE) resources in your organization using custom constraints in the Google Cloud Organization Policy Service. To learn more about Organization Policy, refer to Custom organization policies.

About organization policies and constraints

The Google Cloud Organization Policy gives you centralized, programmatic control over your organization's resources. As the organization policy administrator, you can define an organization policy, which is a set of restrictions called constraints that apply to Google Cloud resources and descendants of those resources in the Google Cloud resource hierarchy. You can enforce organization policies at at the organization, folder, or project level.

Organization Policy provides predefined constraints for various Google Cloud services. However, if you want more granular, customizable control over the specific fields that are restricted in your organization policies, you can also create custom constraints and use those custom constraints in a custom organization policy.

Supported resources in GKE

For GKE, you can create custom constraints for the CREATE or UPDATE methods on any field in the Cluster or NodePool resource of the Google Kubernetes Engine API v1, except for output-only fields and the following fields:

  • projects.locations.clusters.masterAuth.clientKey
  • projects.locations.clusters.masterAuth.password

Policy inheritance

By default, policies are inherited by the descendants of the resources on which you enforce the policy. For example, if you enforce a policy on a folder, Google Cloud enforces the policy on all projects in the folder. To learn more about this behavior and how to change it, refer to Hierarchy evaluation rules.

Pricing

Organization policies and constraints are offered at no charge.

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
    • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.

  • To get the permissions that you need to create constraints and enforce organization policies, ask your administrator to grant you the Organization Policy Administrator (roles/orgpolicy.policyAdmin) IAM role on your Google Cloud organization. For more information about granting roles, see Manage access.

    You might also be able to get the required permissions through custom roles or other predefined roles.

  • Ensure that you know your organization ID.

Create a custom constraint

To create a new custom constraint, you define the constraint in a YAML file and apply the custom constraint in your organization using the Google Cloud CLI.

  1. Create a YAML file for the custom constraint:

    name: organizations/ORGANIZATION_ID/customConstraints/custom.CONSTRAINT_NAME
resourceTypes:
- container.googleapis.com/RESOURCE_NAME
methodTypes:
- METHOD1
- METHOD2
condition: "resource.OBJECT_NAME.FIELD_NAME == VALUE"
actionType: ACTION
displayName: DISPLAY_NAME
description: DESCRIPTION

    Replace the following:

    • ORGANIZATION_ID: your organization ID, such as 123456789.
    • CONSTRAINT_NAME: the name you want for your new custom constraint. A custom constraint must start with custom., and can only include uppercase letters, lowercase letters, or numbers, for example, custom.enableGkeAutopilot. The maximum length of this field is 70 characters, not counting the prefix, for example, organizations/123456789/customConstraints/custom..
    • RESOURCE_NAME: the name (not the URI) of the GKE API REST resource containing the object and field you want to restrict. For example, Cluster or NodePool.
    • METHOD1,METHOD2,...: a list of RESTful methods for which to enforce the constraint. Can be CREATE or CREATE and UPDATE.

    • condition: the condition to validate the request against, written in Common Expression Language (CEL). This field has a maximum length of 1000 characters. The expression must contain the following fields, and supports logical operators such as && and ||:

      • OBJECT_NAME: the name of the GKE API object that you want to restrict, in pascalCase formatting. For example, privateClusterConfig.
      • FIELD_NAME: the name of the GKE API field that you want to restrict, in pascalCase formatting. For example, enablePrivateNodes.
      • VALUE: the value of the field. For boolean fields, use true or false. For string fields, use "STRING".

    • ACTION: the action to take if the condition is met. This can be either ALLOW or DENY.

    • DISPLAY_NAME: a human-friendly name for the constraint. This field has a maximum length of 200 characters.

    • DESCRIPTION: a human-friendly description of the constraint to display as an error message when the policy is violated. This field has a maximum length of 2000 characters.

  2. Apply the custom constraint:

    gcloud org-policies set-custom-constraint PATH_TO_FILE

    Replace PATH_TO_FILE with the file path of your custom constraint definition.

  3. Verify that the custom constraint exists:

    gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID

    The output is similar to the following:

    CONSTRAINT                     LIST_POLICY    BOOLEAN_POLICY    ETAG
custom.enableGkeAutopilot      -              SET               COCsm5QGENiXi2E=
...

Enforce the custom constraint

To enforce the new custom constraint, create an organization policy that references the constraint, and then apply the organization policy.

  1. Create a YAML file for the organization policy:

    name: RESOURCE_HIERARCHY/policies/POLICY_NAME
spec:
  rules:
  - enforce: true

    Replace the following:

    • RESOURCE_HIERARCHY: the location of the new policy, which affects the scope of enforcement. Use the Google Cloud resource hierarchy as a guide. For example, if you wanted to enforce the policy in a specific project, use projects/PROJECT_ID. To enforce the policy in a specific organization, use organizations/ORGANIZATION_ID.
    • POLICY_NAME: the name of the new policy.

  2. Enforce the policy:

    gcloud org-policies set-policy PATH_TO_POLICY

    Replace PATH_TO_POLICY with the path to your policy definition file.

  3. Verify that the policy exists:

    gcloud org-policies list \
    --RESOURCE_FLAG=RESOURCE_ID

    Replace the following:

    • RESOURCE_FLAG: the Google Cloud resource where you enforced the policy. For example, project or folder.
    • RESOURCE_ID: the ID of the resource where you enforced the policy. For example, your Google Cloud folder ID.

    For a list of arguments, refer to gcloud org-policies list.

    The output is similar to the following:

    CONSTRAINT                                    LIST_POLICY    BOOLEAN_POLICY    ETAG
iam.disableWorkloadIdentityClusterCreation    -              SET               CO3UkJAGEOj1qsQB
custom.enableGkeAutopilot                     -              SET               COCsm5QGENiXi2E=
custom.enableBinAuth                          -              SET               CJfKiZUGEJju7LUD

Example: Create a custom constraint and enforce a policy

The following example creates a custom constraint and policy that requires all new clusters in a specific project to be Autopilot clusters.

Before you begin, you should know the following:

  • Your organization ID
  • A project ID

Create the constraint

  1. Save the following file as constraint-enable-autopilot.yaml:

    name: organizations/ORGANIZATION_ID/customConstraints/custom.enableGkeAutopilot
resourceTypes:
- container.googleapis.com/Cluster
methodTypes:
- CREATE
condition: "resource.autopilot.enabled == false"
actionType: DENY
displayName: Enable GKE Autopilot
description: All new clusters must be Autopilot clusters.

    This defines a constraint where for every new cluster, if the cluster mode is not Autopilot, the operation is denied.

  2. Apply the constraint:

    gcloud org-policies set-custom-constraint ~/constraint-enable-autopilot.yaml

  3. Verify that the constraint exists:

    gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID

    The output is similar to the following:

    CUSTOM_CONSTRAINT                       ACTION_TYPE  METHOD_TYPES   RESOURCE_TYPES                     DISPLAY_NAME
custom.enableGkeAutopilot               DENY         CREATE         container.googleapis.com/Cluster   Enable GKE Autopilot
...

Create the policy

  1. Save the following file as policy-enable-autopilot.yaml:

    name: projects/PROJECT_ID/policies/custom.enableGkeAutopilot
spec:
  rules:
  - enforce: true

    Replace PROJECT_ID with your project ID.

  2. Apply the policy:

    gcloud org-policies set-policy ~/policy-enable-autopilot.yaml

  3. Verify that the policy exists:

    gcloud org-policies list --project=PROJECT_ID

    The output is similar to the following:

    CONSTRAINT                  LIST_POLICY    BOOLEAN_POLICY    ETAG
custom.enableGkeAutopilot   -              SET               COCsm5QGENiXi2E=

After you apply the policy, wait for about two minutes for Google Cloud to start enforcing the policy.

Test the policy

Try to create a GKE Standard cluster in the project:

gcloud container clusters create org-policy-test \
    --project=PROJECT_ID \
    --zone=COMPUTE_ZONE \
    --num-nodes=1

The output is the following:

Operation denied by custom org policies: ["customConstraints/custom.enableGkeAutopilot": "All new clusters must be Autopilot clusters."]

Sample custom constraints for common use cases

The following sections provide the syntax of some custom constraints that you might find useful:

Description Constraint syntax
Do not disable node auto-upgrade for new node pools 

    name: organizations/ORGANIZATION_ID/customConstraints/custom.enableAutoUpgrade
    resourceTypes:
    - container.googleapis.com/NodePool
    methodTypes:
    - CREATE
    condition: "resource.management.autoUpgrade == true"
    actionType: ALLOW
    displayName: Enable node auto-upgrade
    description: All node pools must have node auto-upgrade enabled.
Enable Workload Identity Federation for GKE for new clusters 

    name: organizations/ORGANIZATION_ID/customConstraints/custom.enableWorkloadIdentity
    resourceTypes:
    - container.googleapis.com/Cluster
    methodTypes:
    - CREATE
    condition: "has(resource.workloadIdentityConfig.workloadPool) || resource.workloadIdentityConfig.workloadPool.size() > 0"
    actionType: ALLOW
    displayName: Enable Workload Identity on new clusters
    description: All new clusters must use Workload Identity.
Do not disable Cloud Logging on existing clusters 

    name: organizations/ORGANIZATION_ID/customConstraints/custom.enableLogging
    resourceTypes:
    - container.googleapis.com/Cluster
    methodTypes:
    - UPDATE
    condition: "resource.loggingService == 'none'"
    actionType: DENY
    displayName: Do not disable Cloud Logging
    description: You cannot disable Cloud Logging on existing GKE cluster.
Only allow Standard node pool creation or update when legacy metadata endpoints are disabled 

    name: organizations/ORGANIZATION_ID/customConstraints/custom.nodeConfigMetadata
    resourceTypes:
    - container.googleapis.com/NodePool
    methodTypes:
    - CREATE
    - UPDATE
    condition: "'disable-legacy-endpoints' in resource.config.metadata && resource.config.metadata['disable-legacy-endpoints'] == 'true'"
    actionType: ALLOW
    displayName: Disable legacy metadata endpoints
    description: You can only create or update node pools if you disable legacy
    metadata endpoints.

This constraint sample shows you how to set a custom constraint on a map value. The condition field uses the index operator on the map key disable-legacy-endpoints. If you use regular field selection syntax instead, like in the preceding samples, you see a INVALID_CUSTOM_CONSTRAINT_CONDITION error.

What's next