Use custom organization policies

    This page shows you how to use Organization Policy Service custom constraints to restrict specific operations on the following Google Cloud resources:

    • pubsub.googleapis.com/Schema
    • pubsub.googleapis.com/Snapshot
    • pubsub.googleapis.com/Subscription
    • pubsub.googleapis.com/Topic

    To learn more about Organization Policy, see Custom organization policies.

    About organization policies and constraints

    The Google Cloud Organization Policy Service gives you centralized, programmatic control over your organization's resources. As the organization policy administrator, you can define an organization policy, which is a set of restrictions called constraints that apply to Google Cloud resources and descendants of those resources in the Google Cloud resource hierarchy. You can enforce organization policies at the organization, folder, or project level.

    Organization Policy provides predefined constraints for various Google Cloud services. However, if you want more granular, customizable control over the specific fields that are restricted in your organization policies, you can also create custom constraints and use those custom constraints in an organization policy.

    Policy inheritance

    By default, organization policies are inherited by the descendants of the resources on which you enforce the policy. For example, if you enforce a policy on a folder, Google Cloud enforces the policy on all projects in the folder. To learn more about this behavior and how to change it, refer to Hierarchy evaluation rules.

    Before you begin

    1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

    2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

      Go to project selector

    3. Make sure that billing is enabled for your Google Cloud project.

    4. Install the Google Cloud CLI.

    5. To initialize the gcloud CLI, run the following command: 

      gcloud init

    6. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

      Go to project selector

    7. Make sure that billing is enabled for your Google Cloud project.

    8. Install the Google Cloud CLI.

    9. To initialize the gcloud CLI, run the following command: 

      gcloud init
    10. Ensure that you know your organization ID.

    Required roles

    To get the permissions that you need to manage custom organization policies, ask your administrator to grant you the Organization Policy Administrator (roles/orgpolicy.policyAdmin) IAM role on the organization resource. For more information about granting roles, see Manage access to projects, folders, and organizations.

    You might also be able to get the required permissions through custom roles or other predefined roles.

    Create a custom constraint

    A custom constraint is defined in a YAML file by the resources, methods, conditions, and actions that are supported by the service on which you are enforcing the organization policy. Conditions for your custom constraints are defined using Common Expression Language (CEL). For more information about how to build conditions in custom constraints using CEL, see the CEL section of Creating and managing custom constraints.

    To create a custom constraint, create a YAML file using the following format:

    name: organizations/ORGANIZATION_ID/customConstraints/CONSTRAINT_NAME
resourceTypes:
- RESOURCE_NAME
methodTypes:
- CREATE
- UPDATE
condition: "CONDITION"
actionType: ACTION
displayName: DISPLAY_NAME
description: DESCRIPTION

    Replace the following:

    • ORGANIZATION_ID: your organization ID, such as 123456789.

    • CONSTRAINT_NAME: the name you want for your new custom constraint. A custom constraint must start with custom., and can only include uppercase letters, lowercase letters, or numbers. For example, custom.messageRetentionSet. The maximum length of this field is 70 characters.

    • RESOURCE_NAME: the fully qualified name of the Google Cloud resource containing the object and field you want to restrict. For example, pubsub.googleapis.com/Topic.

    • CONDITION: a CEL condition that is written against a representation of a supported service resource. This field has a maximum length of 1000 characters. See Supported resources for more information about the resources available to write conditions against. For example, "has(resource.messageRetentionDuration)".

    • ACTION: the action to take if the condition is met. Possible values are ALLOW and DENY.

    • DISPLAY_NAME: a human-friendly name for the constraint. This field has a maximum length of 200 characters.

    • DESCRIPTION: a human-friendly description of the constraint to display as an error message when the policy is violated. This field has a maximum length of 2000 characters.

    For more information about how to create a custom constraint, see Defining custom constraints.

    Set up a custom constraint

    After you have created the YAML file for a new custom constraint, you must set it up to make it available for organization policies in your organization. To set up a custom constraint, use the gcloud org-policies set-custom-constraint command: 
    gcloud org-policies set-custom-constraint CONSTRAINT_PATH
     Replace CONSTRAINT_PATH with the full path to your custom constraint file. For example, /home/user/customconstraint.yaml. Once completed, your custom constraints are available as organization policies in your list of Google Cloud organization policies. To verify that the custom constraint exists, use the gcloud org-policies list-custom-constraints command: 
    gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID
     Replace ORGANIZATION_ID with the ID of your organization resource. For more information, see Viewing organization policies.

    Enforce a custom organization policy

    You can enforce a boolean constraint by creating an organization policy that references it, and then applying that organization policy to a Google Cloud resource.

    Console

    1. In the Google Cloud console, go to the Organization policies page.

      Go to Organization policies

    2. From the project picker, select the project for which you want to set the organization policy.
    3. From the list on the Organization policies page, select your constraint to view the Policy details page for that constraint.
    4. To configure the organization policy for this resource, click Manage policy.
    5. On the Edit policy page, select Override parent's policy.
    6. Click Add a rule.
    7. In the Enforcement section, select whether enforcement of this organization policy is on or off.
    8. Optional: To make the organization policy conditional on a tag, click Add condition. Note that if you add a conditional rule to an organization policy, you must add at least one unconditional rule or the policy cannot be saved. For more information, see Setting an organization policy with tags.
    9. If this is a custom constraint, you can click Test changes to simulate the effect of this organization policy. For more information, see Test organization policy changes with Policy Simulator.
    10. To finish and apply the organization policy, click Set policy. The policy requires up to 15 minutes to take effect.

    gcloud

    To create an organization policy that enforces a boolean constraint, create a policy YAML file that references the constraint: 

          name: projects/PROJECT_ID/policies/CONSTRAINT_NAME
      spec:
        rules:
        - enforce: true

    Replace the following:

    • PROJECT_ID: the project on which you want to enforce your constraint.
    • CONSTRAINT_NAME: the name you defined for your custom constraint. For example, custom.messageRetentionSet.

    To enforce the organization policy containing the constraint, run the following command: 

        gcloud org-policies set-policy POLICY_PATH

    Replace POLICY_PATH with the full path to your organization policy YAML file. The policy requires up to 15 minutes to take effect.

    Test the custom organization policy

    The following example creates a custom constraint and policy that require all new topics in a specific project to enforce in-transit guarantees for messages.

    Before you begin, you must know the following:

    • Your organization ID
    • A project ID

    Create the constraint

    1. Save the following file as constraint-require-message-retention.yaml:

      name: organizations/ORGANIZATION_ID/customConstraints/custom.messageRetentionSet
resourceTypes:
- Topic
methodTypes:
- CREATE
- UPDATE
condition: "has(resource.messageRetentionDuration)"
actionType: ALLOW
displayName: Require message retention
description: All topics must set a message retention duration.

      This defines a constraint where every topic must set a message retention duration.

      1. Apply the constraint:

        gcloud org-policies set-custom-constraint constraint-require-message-retention.yaml

      2. Verify that the constraint exists:

        gcloud org-policies list-custom-constraints --organization=ORGANIZATION_ID

        The output is similar to the following:

        CUSTOM_CONSTRAINT                       ACTION_TYPE   METHOD_TYPES     RESOURCE_TYPES                     DISPLAY_NAME
custom.messageRetentionSet              ALLOW         CREATE,UPDATE    pubsub.googleapis.com/Topic        Require message retention
...

      Create the policy

      1. Save the following file as policy-require-message-retention.yaml:

        name: projects/PROJECT_ID/policies/custom.messageRetentionSet
spec:
  rules:
  - enforce: true

        Replace PROJECT_ID with your project ID.

      2. Apply the policy:

        gcloud org-policies set-policy policy-`policy-require-message-retention.yaml`:

      3. Verify that the policy exists:

        gcloud org-policies list --project=PROJECT_ID

        The output is similar to the following:

        CONSTRAINT                         LIST_POLICY    BOOLEAN_POLICY    ETAG
custom.messageRetentionSet         -              SET               COCsm5QGENiXi2E=

      After you apply the policy, wait for about two minutes for Google Cloud to start enforcing the policy.

      Test the policy

      Try creating a topic without message retention enabled:

      gcloud pubsub topics create org-policy-test \
  --project=PROJECT_ID \

      The output is the following:

      Operation denied by custom org policies: ["customConstraints/custom.messageRetentionSet": "All topics must set a message retention duration."].

ERROR: Failed to create topic [projects/PROJECT_ID/topics/org-policy-test]: Operation denied by org policy: ["customConstraints/custom.messageRetentionSet": "All topics must set a message retention duration."].
ERROR: (gcloud.pubsub.topics.create) Failed to create the following: [org-policy-test].

      Example custom organization policies for common use cases

      The following table provides the syntax of some custom constraints for common use cases:

      Description Constraint syntax
      Require topics to enforce in-transit guarantees for messages and to provide at least one allowed persistence region 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.enforceInTransitWithRegions
  resourceTypes:
  - pubsub.googleapis.com/Topic
  methodTypes:
  - CREATE
  - UPDATE
  condition: "resource.messageStoragePolicy.enforceInTransit == true && resource.messageStoragePolicy.allowedPersistenceRegions.size() >= 1"
  actionType: ALLOW
  displayName: Require in-transit enforcement and regions
  description: All topics must enable in-transit message storage policy enforcement and contain a non-empty list of allowed persistence regions.
      Require topics to enforce in-transit guarantees for messages and exclude a particular region 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.enforceInTransitWithoutUsEast1
  resourceTypes:
  - pubsub.googleapis.com/Topic
  methodTypes:
  - CREATE
  - UPDATE
  condition: "resource.messageStoragePolicy.enforceInTransit && !resource.messageStoragePolicy.allowedPersistenceRegions.exists(r, r == 'us-east1')"
  actionType: ALLOW
  displayName: Require in-transit enforcement and restrict regions
  description: All topics must enable in-transit message storage policy enforcement, and exclude us-east1 from the allowed persistence regions.
      Require a minimum message retention duration for subscriptions 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.minMessageRetentionDuration
  resourceTypes:
  - pubsub.googleapis.com/Subscription
  methodTypes:
  - CREATE
  - UPDATE
  condition: "resource.messageRetentionDuration < duration('30m')"
  actionType: DENY
  displayName: Minimum message retention duration
  description: All subscriptions must retain unacked messages for at least 30 minutes.
      Require subscriptions to enable dead lettering 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.requireDeadLetterPolicy
  resourceTypes:
  - pubsub.googleapis.com/Subscription
  methodTypes:
  - CREATE
  - UPDATE
  condition: "has(resource.deadLetterPolicy)"
  actionType: ALLOW
  displayName: Require dead letter policy
  description: All subscriptions must set a dead letter policy.
      Require schemas to be of type Avro 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.schemaIsAvro
  resourceTypes:
  - pubsub.googleapis.com/Schema
  methodTypes:
  - CREATE
  - UPDATE
  condition: "resource.type == 'AVRO'"
  actionType: ALLOW
  displayName: Schema is Avro
  description: All schemas must be of type Avro.
      Require authenticated push for subscriptions 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.authenticatedPush
  resourceTypes:
  - pubsub.googleapis.com/Subscription
  methodTypes:
  - CREATE
  - UPDATE
  condition: "has(resource.pushConfig.pushEndpoint) && has(resource.pushConfig.oidcToken.serviceAccountEmail) && has(resource.pushConfig.oidcToken.audience)"
  actionType: ALLOW
  displayName: Subscription uses authenticated push
  description: All subscriptions must enable push authentication.
      Require topics to enable CMEK 
        name: organizations/ORGANIZATION_ID/customConstraints/custom.topicHasCmek
  resourceTypes:
  - pubsub.googleapis.com/Topic
  methodTypes:
  - CREATE
  - UPDATE
  condition: "has(resource.kmsKeyName)"
  actionType: ALLOW
  displayName: Topic has CMEK
  description: All topics must use a Cloud KMS CryptoKey.

      Pub/Sub supported resources

      The following table lists the Pub/Sub resources that you can reference in custom constraints.

      Resource Field
      pubsub.googleapis.com/Schema resource.definition
      resource.name
      resource.type
      pubsub.googleapis.com/Snapshot resource.expireTime
      resource.name
      resource.topic
      pubsub.googleapis.com/Subscription resource.ackDeadlineSeconds
      resource.bigqueryConfig.dropUnknownFields
      resource.bigqueryConfig.serviceAccountEmail
      resource.bigqueryConfig.table
      resource.bigqueryConfig.useTableSchema
      resource.bigqueryConfig.useTopicSchema
      resource.bigqueryConfig.writeMetadata
      resource.cloudStorageConfig.bucket
      resource.cloudStorageConfig.filenameDatetimeFormat
      resource.cloudStorageConfig.filenamePrefix
      resource.cloudStorageConfig.filenameSuffix
      resource.cloudStorageConfig.maxBytes
      resource.cloudStorageConfig.maxDuration
      resource.cloudStorageConfig.maxMessages
      resource.cloudStorageConfig.serviceAccountEmail
      resource.deadLetterPolicy.deadLetterTopic
      resource.deadLetterPolicy.maxDeliveryAttempts
      resource.enableExactlyOnceDelivery
      resource.enableMessageOrdering
      resource.expirationPolicy.ttl
      resource.filter
      resource.messageRetentionDuration
      resource.name
      resource.pushConfig.attributes
      resource.pushConfig.noWrapper.writeMetadata
      resource.pushConfig.oidcToken.audience
      resource.pushConfig.oidcToken.serviceAccountEmail
      resource.pushConfig.pushEndpoint
      resource.retainAckedMessages
      resource.retryPolicy.maximumBackoff
      resource.retryPolicy.minimumBackoff
      resource.topic
      pubsub.googleapis.com/Topic resource.ingestionDataSourceSettings.awsKinesis.awsRoleArn
      resource.ingestionDataSourceSettings.awsKinesis.consumerArn
      resource.ingestionDataSourceSettings.awsKinesis.gcpServiceAccount
      resource.ingestionDataSourceSettings.awsKinesis.streamArn
      resource.kmsKeyName
      resource.messageRetentionDuration
      resource.messageStoragePolicy.allowedPersistenceRegions
      resource.messageStoragePolicy.enforceInTransit
      resource.name
      resource.schemaSettings.encoding
      resource.schemaSettings.firstRevisionId
      resource.schemaSettings.lastRevisionId
      resource.schemaSettings.schema

      What's next