Selectively enforce firewall policies in GKE

This page shows you how to selectively enforce Cloud Next Generation Firewall network firewall policies in Google Kubernetes Engine (GKE) by using Tags. To learn how to use Tags in GKE for other purposes, like billing management or conditional IAM policies, see Manage GKE resources using Tags.

About Tags

Tags are key-value pairs that let you annotate and manage your Google Cloud resources at the organization or project level. You can use tags to organize your resources and to conditionally apply policies like firewalls or IAM policies. You can use IAM access control to define who can attach, create, update, or delete tags.

To learn more about Tags, see the Tags overview in the Resource Manager documentation.

Use Tags to enforce network firewall policies

You can use Tags to conditionally apply global or regional network firewall policies to your GKE nodes. You must designate the GCE_FIREWALL purpose for tags that you want to use with network firewall policies. When you apply firewall purpose tags to GKE clusters or node pools, GKE automatically attaches those tags to the corresponding Compute Engine virtual machines (VMs).

Tags for network firewall policies replace the need to use network tags, which are metadata that anyone can attach to the underlying Compute Engine VMs for Virtual Private Cloud firewall rule enforcement, and which don't support IAM access control. If you currently use network tags with VPC firewall rules, we recommend that you migrate to network firewall policies and use secure firewall tags. For a detailed comparison, see Compare network tags with Tags in this document.

Tags for network firewall policies workflow

To use tags with network firewall policies in GKE, you do the following:

  1. Create a tag:

    1. Define a tag key at the organization or project level, such as env.
    2. Define the possible tag values for the key, such as dev, staging, and prod.

    3. Designate the tag for network firewall policy use.

  2. Grant users access to interact with the firewall tag.

  3. Apply tag key-value pairs to specific GKE clusters or node pools. GKE automatically attaches the tags to the underlying Compute Engine VMs for firewall policy enforcement.

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
    • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.

Requirements and limitations

  • Tags for network firewall policies are supported in GKE version 1.28 and later. If you use a GKE version earlier than 1.28, use network tags with VPC firewall rules instead.
  • The GKE cluster and the tag must be associated with the same VPC network.
  • In Standard clusters, each node pool supports up to five attached firewall tags.
  • Autopilot clusters support up to five firewall tags.
  • GKE rejects tag keys that use the gke-managed prefix.
  • You must create the tag key-value pairs before you can attach them to clusters or node pools.

IAM roles and permissions

To get the permissions that you need to use Tags for firewall policies in GKE, ask your administrator to grant you the following IAM roles:

  • To grant the required permissions for tags to users and GKE service agents:
  • To create and administer tags: Tag Administrator (roles/resourcemanager.tagAdmin) on the organization or the project
  • To attach tags to resources: Tag User (roles/resourcemanager.tagUser) on the project

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Create tags

Tags must exist for you to attach them to clusters or nodes. To create a tag, see Use Tags for firewalls in the Cloud NGFW documentation.

For example, to create a project-scoped firewall tag, run the following commands:

  1. Create the tag key:

    gcloud resource-manager tags keys create TAG_KEY \
    --parent=projects/PROJECT_ID \
    --purpose=GCE_FIREWALL \
    --purpose-data=network=PROJECT_ID/NETWORK_NAME

    Replace the following:

    • TAG_KEY: the name of the tag key, such as env
    • PROJECT_ID: your Google Cloud project ID
    • NETWORK_NAME: the name of the VPC network that you'll use with the tag

  2. Get the ID of the tag key:

    gcloud resource-manager tags keys describe PROJECT_ID/TAG_KEY \
    --format="value(name)"

    The output is tagKeys/KEY_ID, where KEY_ID is a numerical ID for the key. Make a note of this ID for later.

  3. Add a tag value to the tag key:

    gcloud resource-manager tags values create TAG_VALUE \
    --parent=tagKeys/KEY_ID

    Replace TAG_VALUE with the name of an allowed value for that tag key, such as dev.

Tag syntax in gcloud CLI commands

When you refer to tags using the gcloud CLI, you must format the key-value pairs using one of the following syntaxes:

Tag syntax 
tagKeys/KEY_ID=tagValues/VALUE_ID

Replace the following:

  • KEY_ID: the numerical key ID
  • VALUE_ID: the numerical value ID

For example, tagKeys/123456789=tagValues/987654321.

 
ORGANIZATION_ID/TAG_KEY=TAG_VALUE

Replace the following:

  • ORGANIZATION_ID: your numerical Google Cloud organization ID
  • TAG_KEY: the name of the tag key that you created
  • TAG_VALUE: the name of the tag value that you created

For example, 12345678901/env=dev.

 
PROJECT_ID/TAG_KEY=TAG_VALUE

Replace the following:

  • PROJECT_ID: your Google Cloud project ID
  • TAG_KEY: the name of the tag key that you created
  • TAG_VALUE: the name of the tag value that you created

For example, example-project/env=dev.

 
PROJECT_NUMBER/TAG_KEY=TAG_VALUE

Replace the following:

  • PROJECT_ID: the numerical identifier for your Google Cloud project
  • TAG_KEY: the name of the tag key that you created
  • TAG_VALUE: the name of the tag value that you created

For example, 11223344556/env=dev.

Target tags with firewall policies

After you create tags, you can refer to specific key-value pairs in firewall policy rules. For instructions, see Use Tags for firewalls.

Grant IAM permissions to service agents

For GKE to automatically attach tags to new nodes during scale-up events, you must grant the corresponding IAM roles to Google Cloud-managed service accounts, also called service agents.

  1. Grant the Tag User role (roles/resourcemanager.tagUser) to the Kubernetes Engine Service Agent:

    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com \
    --role=roles/resourcemanager.tagUser \
    --condition=None

    Replace PROJECT_NUMBER with the Google Cloud project number of the cluster. To find the project number, run the following command:

    gcloud projects describe PROJECT_ID --format="value(projectNumber)"

  2. Grant the Tag Holds Administrator role (roles/resourcemanager.tagHoldAdmin) to the Kubernetes Engine Service Agent for the tag key-value pair:

    gcloud resource-manager tags values add-iam-policy-binding PROJECT_ID/TAG_KEY/TAG_VALUE \
    --member=serviceAccount:service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com \
    --role=roles/resourcemanager.tagHoldAdmin

    This role lets the service agent prevent tag deletion if the key-value pair is still in use in GKE.

  3. Grant the Tag User role (roles/resourcemanager.tagUser) to the Google APIs Service Agent:

    gcloud projects add-iam-policy-binding PROJECT_ID \
    --member=serviceAccount:PROJECT_NUMBER@cloudservices.gserviceaccount.com \
    --role=roles/resourcemanager.tagUser \
    --condition=None

Grant additional IAM roles for tags outside the project

To use tags that are owned by an organization or a different project than the cluster project, do the following additional steps:

  1. Grant the Tag User role (roles/resourcemanager.tagUser) to the Kubernetes Engine Service Agent access for the tags in the parent resource:

    gcloud resource-manager tags keys add-iam-policy-binding PARENT_RESOURCE/TAG_KEY \
    --member=serviceAccount:service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com \
    --role=roles/resourcemanager.tagUser \
    --condition=None

    Replace the following:

    • PARENT_RESOURCE: the project ID or the organization ID of the resource that owns that tag
    • PROJECT_NUMBER: the project number of the cluster project

  2. Grant the Tag User role (roles/resourcemanager.tagUser) to the Google APIs Service Agent access for the tags in the parent resource:

    gcloud resource-manager tags keys add-iam-policy-binding PARENT_RESOURCE/TAG_KEY \
    --member=serviceAccount:PROJECT_NUMBER@cloudservices.gserviceaccount.com \
    --role=roles/resourcemanager.tagUser \
    --condition=None

  3. Grant the Tag Holds Administrator role (roles/resourcemanager.tagHoldAdmin) to the Kubernetes Engine Service Agent for the tag key-value pair:

    gcloud resource-manager tags values add-iam-policy-binding PARENT_RESOURCE/TAG_KEY/TAG_VALUE \
    --member=serviceAccount:service-PROJECT_NUMBER@container-engine-robot.iam.gserviceaccount.com \
    --role=roles/resourcemanager.tagHoldAdmin

Attach firewall tags to Autopilot clusters

You attach firewall tags to Autopilot clusters at the cluster level. GKE automatically applies these cluster-level tags to every node.

Attach tags when you create a new Autopilot cluster

Run the following command:

gcloud beta container clusters create-auto CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=TAG1,TAG2,...

Replace the following:

  • CLUSTER_NAME: the name of the new cluster.
  • LOCATION: the Compute Engine region of the cluster.
  • TAG1,TAG2,...: a comma-separated set of key-value pairs to attach. Each key-value pair must use a supported syntax, as described in the Tag syntax in commands section. For example, example-project/env=dev,1234567901/team=sre.

Attach tags to existing Autopilot clusters

Run the following command:

gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=TAG1,TAG2,...

When you update the tags on a cluster, GKE overwrites any existing tags on all of the nodes.

Attach firewall tags to Standard clusters and node pools

The method that you use to attach tags depends on whether you want other node pools in the cluster to inherit the tags, as follows:

Standard cluster firewall tags
--autoprovisioning-resource-manager-tags

Cluster-level setting

GKE applies the tags to all new auto-provisioned node pools in the cluster.

If you use this flag in an existing cluster, GKE doesn't apply the tags to existing node pools in the cluster. Existing node pools retain any tags that were already applied before the update. To update the tags of existing node pools, use the --resource-manager-tags flag.
--resource-manager-tags

Node pool-level setting

GKE applies the tags to the specified node pool. If you use this flag during cluster creation, GKE applies the tags to the default node pool that GKE creates. If you use this flag on an auto-provisioned node pool, GKE overwrites any existing tags on the node pool.

Attach firewall tags to Standard clusters

You can attach tags to new or existing Standard clusters. When you attach tags to an entire cluster, GKE considers these tags to be set at the cluster-level.

Attach tags to a new Standard cluster with node auto-provisioning

GKE uses cluster-level tags for new auto-provisioned nodes by default. The default node pool that GKE creates in the cluster isn't automatically provisioned and doesn't get those tags.

gcloud beta container clusters create CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=TAG1,TAG2,... \
    --enable-autoprovisioning \
    --max-cpu=MAX_CPU \
    --max-memory=MAX_MEMORY

Replace the following:

  • CLUSTER_NAME: the name of the new cluster
  • LOCATION: the Compute Engine region or zone for the cluster
  • TAG1,TAG2,...: a comma-separated set of key-value pairs to attach. Each key-value pair must use a supported syntax, as described in the Tag syntax in commands section. For example, example-project/env=dev,1234567901/team=sre.
  • MAX_CPU: the maximum number of cores for the cluster
  • MAX_MEMORY: the maximum memory capacity for the cluster in gigabytes

Attach tags when you enable node auto-provisioning on an existing cluster

GKE only applies these tags to new auto-provisioned node pools. Existing node pools retain any tags that they had before the update.

  1. Attach tags to the cluster:

    gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=TAG1,TAG2,...

  2. Enable node auto-provisioning on the cluster:

    gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --enable-autoprovisioning \
    --max-cpu=MAX_CPU \
    --max-memory=MAX_MEMORY

Attach firewall tags to node pools

You can attach tags to new or existing node pools regardless of whether they use node auto-provisioning. GKE considers these tags to be a node pool-level setting.

Attach tags to the default node pool

GKE attaches the tags that you specify using the --resource-manager-tags flag when you create a cluster to the default node pool that GKE creates in the cluster.

gcloud beta container clusters create CLUSTER_NAME \
    --location=LOCATION \
    --resource-manager-tags=TAG1,TAG2,...

Replace the following:

  • CLUSTER_NAME: the name of the new cluster
  • LOCATION: the Compute Engine region or zone for the cluster
  • TAG1,TAG2,...: a comma-separated set of key-value pairs to attach. Each key-value pair must use a supported syntax, as described in the Tag syntax in commands section. For example, example-project/env=dev,1234567901/team=sre.

Attach tags to a new node pool

When you use the --resource-manager-tags flag during node pool creation, GKE attaches the tags that you specify to that node pool.

gcloud beta container node-pools create NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=LOCATION \
    --resource-manager-tags=TAG1,TAG2,...

Replace the following:

  • NODE_POOL_NAME: the name of the new node pool
  • CLUSTER_NAME: the name of the cluster
  • LOCATION: the Compute Engine region or zone of the cluster
  • TAG1,TAG2,...:a comma-separated set of key-value pairs to attach. Each key-value pair must use a supported syntax, as described in the Tag syntax in commands section. For example, example-project/env=dev,1234567901/team=sre.

Attach tags to an existing node pool

When you update the tags on an existing node pool using the --resource-manager-tags flag, GKE overwrites any existing tags on that node pool. You can use this command to update the tags on auto-provisioned node pools.

gcloud beta container node-pools update NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=LOCATION \
    --resource-manager-tags=TAG1,TAG2,...

Replace NODE_POOL_NAME with the name of the node pool to update.

Toggle auto-provisioning settings in existing clusters and node pools

When you update tags at the cluster level, GKE applies those new tags to all new node pools in the cluster and retains the tags that were attached to existing node pools.

When you update existing node pools to enable or disable node auto-provisioning, consider the following implications for tags:

  • When you enable or disable node auto-provisioning, the node pool retains any existing tags. GKE doesn't overwrite these tags with cluster-level tags, even during node re-creation.
  • If you manually update the tags on specific node pools, GKE overwrites the existing tags with your specified tags for that node pool.

Verify the firewall tags on cluster

  • List the tags on Autopilot clusters:

    gcloud beta container clusters describe CLUSTER_NAME \
    --location=LOCATION \
    --format="value(nodePoolAutoConfig.resourceManagerTags)"

  • List the tags on specific node pools:

    gcloud beta container node-pools describe NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --location=LOCATION \
  --format="value(config.resourceManagerTags)"

Detach firewall tags from clusters and node pools

To remove firewall tags from clusters and node pools, update the resource with an empty value for the tags.

Detach tags from Autopilot clusters

Run the following command:

gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=

Detach tags from node pools

  1. Detach cluster-level node autoprovisioning tags:

    gcloud beta container clusters update CLUSTER_NAME \
    --location=LOCATION \
    --autoprovisioning-resource-manager-tags=

    GKE won't attach tags to new auto-provisioned node pools.

  2. Detach node pool tags:

    gcloud beta container node-pools update NODE_POOL_NAME \
    --cluster=CLUSTER_NAME \
    --location=LOCATION \
    --resource-manager-tags=

    GKE removes existing tags from that node pool.

Delete tags keys and values

To delete a tag key or value, ensure that the tag is detached from all resources. Then, see Deleting tags in the Resource Manager documentation.

Compare network tags with Tags

Using Tags for firewall policy enforcement has significant security and usability benefits compared to network tags. Similarly, network firewall policies improve on the capabilities of VPC firewall rules by facilitating firewall rule enforcement across entire organizations, folders, projects, or networks.

Using Tags with network firewall policies is a more secure, scalable way to manage access to your GKE environments across your organization. You can use network tags in the same cluster as Tags, although you can't use network tags to enforce network firewall policies.

For a detailed comparison between Tags and network tags, see Comparison of Tags and network tags in the Cloud NGFW documentation.

Functional differences in auto-provisioned node pools

In Autopilot clusters and in Standard node pools that don't use node auto-provisioning, network tags and Tags exhibit similar behavior. The following table shows the functional differences between network tags and Tags in auto-provisioned node pools in Standard clusters:

Action Network tags behavior Tags behavior
GKE auto-provisions a node pool GKE applies the cluster-level network tags GKE applies the cluster-level tags
You update tags or network tags on an auto-provisioned node pool
  • If cluster-level network tags exist, the update operation fails
  • If cluster-level network tags don't exist, GKE overwrites the existing network tags for the node pool
 GKE overwrites existing tags for the node pool regardless of whether cluster-level tags exist
You update tags or network tags for the entire cluster GKE overwrites the network tags for new and existing auto-provisioned node pools in the cluster. GKE applies the new cluster-level tags to new auto-provisioned node pools. Existing auto-provisioned node pools retain the tags that they had prior to the update.

What's next