Tag tables and datasets

This document describes how to use tags to conditionally apply Identity and Access Management (IAM) policies to BigQuery tables and datasets.

A tag is a key-value pair that you can attach directly to a table or dataset, or that a table or dataset can inherit from other Google Cloud resources. You can conditionally apply policies based on whether a resource has a specific tag. For example, you might conditionally grant the BigQuery Data Viewer role to a principal on any dataset with the environment:dev tag.

For more information about using tags across the Google Cloud resource hierarchy, see Tags overview.

To grant permissions to many related BigQuery resources at the same time, including resources that don't exist yet, consider using IAM Conditions.

Required permissions

To use tags in BigQuery, you need the following permissions:

  • To attach a tag to a dataset, you need the bigquery.datasets.createTagBinding IAM permission on the dataset, and the resourcemanager.tagValueBindings.create permission at the project level on the tag value that you want to attach.
  • To attach a tag to a table, you need the bigquery.tables.createTagBinding IAM permission on the table, and the resourcemanager.tagValueBindings.create permission at the project level on the tag value that you want to attach.
  • To remove a tag from a dataset, you need the bigquery.datasets.deleteTagBinding IAM permission on the dataset, and the resourcemanager.tagValueBindings.delete permission at the project level on the tag value that you want to delete.
  • To remove a tag from a table, you need the bigquery.tables.deleteTagBinding IAM permission on the table, and the resourcemanager.tagValueBindings.delete permission at the project level on the tag value that you want to delete.
  • To list the tags attached to a dataset or view the Tags section of the Edit details panel for a dataset, you need the bigquery.datasets.listTagBindings IAM permission.
  • To list the tag keys that are associated with a parent organization or project in the Edit details panel for a dataset or table, you need the resourcemanager.tagKeys.list permission at the tag key's parent level and the resourcemanager.tagKeys.get permission for each tag key.
  • To list the tag values of keys that are associated with a parent organization or project in the Edit details panel for a dataset or table, you need the resourcemanager.tagValues.list permission at the tag value's parent level and the resourcemanager.tagValues.get permission for each tag value.

Both of the following predefined IAM roles include all of the necessary BigQuery permissions:

  • BigQuery Data Owner (roles/bigquery.dataOwner)
  • BigQuery Admin (roles/bigquery.admin)

The Resource Manager permissions are included in the Tag User role (roles/resourcemanager.tagUser).

Create tag keys and values

Before you can attach a tag, you need to create a tag and configure its value. To create tag keys and tag values, refer to Creating a tag and Adding tag values.

Attach tags to a dataset

After you create a tag, you can attach it to a dataset. You can attach only one tag value to a dataset for any given tag key. You can attach a maximum of 50 tags to a dataset.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

  3. In the Dataset info section, click Edit details.

  4. In the Tags section, select the tags that you want to add to the dataset.

  5. Click Save.

gcloud

To attach a tag to a dataset using the command line, create a tag binding resource by using the gcloud resource-manager tags bindings create command:

gcloud resource-manager tags bindings create \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be attached, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value
  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource—for example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset
  • LOCATION: the location of your dataset

List tags attached to a dataset

The following steps provide a list of tag bindings attached directly to a dataset. They do not return any tags that are inherited from parent resources.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

    The tags appear in the Dataset info section.

gcloud

To get a list of tag bindings attached to a resource, use the gcloud resource-manager tags bindings list command:

gcloud resource-manager tags bindings list \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource—for example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset

  • LOCATION: the location of your dataset

The output is similar to the following:

name: tagBindings/%2F%2Fbigquery.googleapis.com%2Fprojects%2Fmy_project%2Fdatasets%2Fmy_dataset/tagValues/4567890123
parent: //bigquery.googleapis.com/projects/my_project/datasets/my_dataset
tagValue: tagValues/4567890123

Detach tags from a dataset

You can detach a tag from a resource by deleting the tag binding resource. If you need to delete a tag, you should first detach the tag.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

  3. In the Dataset info section, click Edit details.

  4. In the Tags section, click Delete item next to the tag you want to delete.

  5. Click Save.

gcloud

To detach a tag from a dataset using the command line, delete the tag binding by using the gcloud alpha resource-manager tags bindings delete command:

gcloud alpha resource-manager tags bindings delete \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be attached, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value
  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource—for example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset
  • LOCATION: the location of your dataset

Attach tags when you create a new table

After you create a tag, you can attach it to a new table. You can only attach one tag value to a dataset for any given tag key. You can attach a maximum of 50 tags to a table.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project, and then select a dataset.

  3. In the Dataset info section, click Create table.

  4. Enter the information for your new table. For more details, see Create and use tables.

  5. In the Tags section, select the tags that you want to add to the new table. You can't manually enter tag IDs at this step. You can only select tags from your project or organization.

  6. Click Create table.

bq

Use the bq mk --table command with the --add_tags flag:

bq mk --table \
    --schema=SCHEMA \
    --add_tags=TAGS \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • SCHEMA: the inline schema definition.
  • TAGS: the tags that you are attaching to the new table, separated by commas—for example, 556741164180/env:prod,myProject/department:sales. Each tag must have the namespaced key name and value short name.
  • PROJECT_ID: the ID of the project where you are creating a table.
  • DATASET_ID: the ID of the dataset where you are creating a table.
  • TABLE_ID: the ID of the new table that you are creating.

API

Call the tables.insert method with a defined table resource, and include the tags in the resource_tags field.

Attach tags to an existing table

After you create a tag, you can attach it to an existing table. You can only attach one tag value to a table for any given tag key. You can attach a maximum of 50 tags to a table.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

  3. In the Details tab, click Edit details.

  4. In the Tags section, select the tags that you want to add to the table. You can't manually enter tag IDs at this step. You can only select tags from your project or organization.

  5. Click Save.

bq

Use the bq update command with the --add_tags flag:

bq update \
    --add_tags=TAGS \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • TAGS: the tags that you are attaching to the table, separated by commas—for example, 556741164180/env:prod,myProject/department:sales. Each tag must have the namespaced key name and value short name.
  • PROJECT_ID: the ID of the project that contains your table.
  • DATASET_ID: the ID of the dataset that contains your table.
  • TABLE_ID: the ID of the table that you are updating.

API

Call the tables.update method with a defined table resource, and include the tags in the resource_tags field.

List tags attached to a table

You can list the tags that are attached directly to a table. This process doesn't list the tags that are inherited from parent resources.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

    The tags are visible in the Details tab.

bq

Use the bq show command and look for the tags column. If there are no tags on the table, the tags column isn't displayed.

bq show \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • PROJECT_ID: the ID of the project that contains your table
  • DATASET_ID: the ID of the dataset that contains your table
  • TABLE_ID: the ID of your table

API

Call the tables.get method with a defined table resource, and look for the resource_tags field.

Detach tags from a table

You can detach a tag from a table by deleting the tag binding resource. If you need to delete a tag, you should detach the tag first.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

  3. In the Details tab, click Edit details.

  4. In the Tags section, remove the tags that you want to detach from the table.

  5. Click Save.

bq

To remove some tags from a table, use the bq update command with the --remove_tags flag:

bq update \
    --remove_tags=TAG_KEYS \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • TAG_KEYS: the tag keys that you are detaching from the table, separated by commas—for example, 556741164180/env,myProject/department. Each tag key must have the namespaced key name.
  • PROJECT_ID: the ID of the project that contains your table.
  • DATASET_ID: the ID of the dataset that contains your table.
  • TABLE_ID: the ID of the table that you are updating.

To remove all tags from a table, use the bq update command with the --clear_all_tags flag:

bq update \
    --clear_all_tags \
    PROJECT_ID:DATASET_ID.TABLE_ID

API

Call the tables.update method with a defined table resource, and remove the tags in the resource_tags field. To remove all tags, remove the resource_tags field.

Delete tags

You can't delete a tag if it's referenced by a dataset or table. You should detach all existing tag binding resources before deleting the tag key or value itself. To delete tag keys and tag values, see Deleting tags.

Example

Suppose you are an administrator of an organization. Your data analysts are all members of the group analysts@example.com, which has the BigQuery Data Viewer IAM role on the project userData. A data analyst intern is hired, and according to the company policy they should only have permission to view the anonymousData dataset in the userData project. You can control their access using tags.

  1. Create a tag with the key employee_type and the value intern:

    Example of creating tag key and values.

  2. In the Google Cloud console, go to the IAM page.

    Go to IAM

  3. Locate the row that contains the intern whose dataset access you want to restrict, and click Edit principal in that row.

  4. From the Role menu, select BigQuery Data Viewer.

  5. Click Add condition.

  6. In the Title and Description fields, enter values that describe the IAM tag condition that you want to create.

  7. On the Condition builder tab, click Add.

  8. In the Condition type menu, select Resource, and then select Tag.

  9. In the Operator menu, select has value.

  10. In the Value path field, enter the tag value path in the form ORGANIZATION/TAG_KEY/TAG_VALUE—for example, example.org/employee_type/intern.

    Example of an IAM condition using tags.

    This IAM tag condition restricts the intern's access to datasets that have the intern tag.

  11. To save the tag condition, click Save.

  12. To save all changes that you made in the Edit permissions pane, click Save.

  13. To attach the intern tag value to the anonymousData dataset, use the command line to run the gcloud alpha resource-manager tags bindings create command:

    gcloud alpha resource-manager tags bindings create \
    --tag-value=tagValues/4567890123 \
    --parent=//bigquery.googleapis.com/projects/userData/datasets/anonymousData \
    --location=US
    

Limitations

  • Table tags aren't supported on BigQuery Omni tables, tables in hidden datasets, or temporary tables. Additionally, cross-region queries in BigQuery Omni don't use tags during access control checks of tables in other regions.

  • You can't use table tags with the Cloud Resource Manager API.

  • You can attach a maximum of 50 tags to a dataset or table.

  • Wildcard queries aren't supported for tables that have tags.

  • Some services outside of BigQuery cannot properly verify IAM tag conditions. If the tag condition is positive, meaning that a user is granted a role on a resource only if that resource has a particular tag, then access is denied to the resource regardless of what tags are attached to it. If the tag condition is negative, meaning that a user is granted a role on a resource only if that resource doesn't have a particular tag, then the tag condition is not checked.

    For example, Data Catalog cannot verify IAM tag conditions on BigQuery datasets. Suppose there is a conditional IAM policy that gives an intern the BigQuery Data Viewer role on datasets with the employee_type=intern tag. Since this is a positive tag condition, the intern cannot view datasets by searching in Data Catalog even if those datasets do have the employee_type=intern tag. If the tag condition was changed to a negative one, so that the intern could only view datasets that did not have the employee_type=intern tag, then the check would be skipped entirely and the intern could view the datasets that they couldn't normally access in BigQuery.

What's next