Tag tables and datasets

This document describes how to use tags to conditionally apply Identity and Access Management (IAM) policies to BigQuery tables and datasets.

A tag is a key-value pair that you can attach directly to a table or dataset, or one that a table or dataset can inherit from other Google Cloud resources. You can conditionally apply policies based on whether a resource has a specific tag. For example, you might conditionally grant the BigQuery Data Viewer role to a principal on any dataset with the environment:dev tag.

For more information about using tags across the Google Cloud resource hierarchy, see Tags overview.

To grant permissions to many related BigQuery resources at the same time, including resources that don't exist yet, consider using IAM Conditions.

Before you begin

You need to grant IAM roles that give users the necessary permissions to perform each task in this document. You also need to create tag keys and values to attach to resources.

Required permissions

To use tags in BigQuery, you need the following permissions:

  • To attach a tag to a dataset, you need the bigquery.datasets.createTagBinding IAM permission on the dataset, and the resourcemanager.tagValueBindings.create permission at the project level on the tag value that you want to attach.
  • To attach a tag to a table, you need the bigquery.tables.createTagBinding IAM permission on the table, and the resourcemanager.tagValueBindings.create permission at the project level on the tag value that you want to attach.
  • To remove a tag from a dataset, you need the bigquery.datasets.deleteTagBinding IAM permission on the dataset, and the resourcemanager.tagValueBindings.delete permission at the project level on the tag value that you want to delete.
  • To remove a tag from a table, you need the bigquery.tables.deleteTagBinding IAM permission on the table, and the resourcemanager.tagValueBindings.delete permission at the project level on the tag value that you want to delete.
  • To list the tag keys that are associated with a parent organization or project in the Edit details panel for a dataset or table, you need the resourcemanager.tagKeys.list permission at the tag key's parent level and the resourcemanager.tagKeys.get permission for each tag key.
  • To list the tag values of keys that are associated with a parent organization or project in the Edit details panel for a dataset or table, you need the resourcemanager.tagValues.list permission at the tag value parent level and the resourcemanager.tagValues.get permission for each tag value.

If you are using tags with the Cloud Resource Manager API or gcloud, you also need the following permissions:

  • To list the tags attached to a dataset with the Cloud Resource Manager API or the gcloud CLI, you need the bigquery.datasets.listTagBindings IAM permission.
  • To list the effective tags for a dataset, you need the bigquery.datasets.listEffectiveTags IAM permission.
  • To list the tags attached to a table with the Cloud Resource Manager API or the gcloud CLI, you need the bigquery.tables.listTagBindings IAM permission.
  • To list the effective tags for a table, you need the bigquery.tables.listEffectiveTags IAM permission.

Both of the following predefined IAM roles include all of the necessary BigQuery permissions:

  • BigQuery Data Owner (roles/bigquery.dataOwner)
  • BigQuery Admin (roles/bigquery.admin)

The Resource Manager permissions are included in the Tag User role (roles/resourcemanager.tagUser).

You can also use tags to conditionally deny access with IAM policies to BigQuery tables and datasets (preview). For more information, see Deny policies.

Create tag keys and values

Before you can attach a tag, you need to create a tag and configure its value. To create tag keys and tag values, see Creating a tag and Adding tag values.

Attach tags to a dataset

You can create a new BigQuery dataset when you attach a tag, or you can attach a tag to an existing dataset. You can attach only one tag value to a dataset for any given tag key. You can attach a maximum of 50 tags to a dataset.

Console

For new datasets:

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, select the project where you want to create your dataset.

  3. Click more_vert View actions > Create dataset.

  4. Under Tags, click Select scope.

  5. Select the scope under which your tags exist.

  6. Select and add the tags that you want to add to the dataset.

  7. Click Create dataset.

For existing datasets:

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

  3. In the Dataset info section, click Edit details.

  4. In the Tags section, select the tags that you want to add to the dataset.

  5. Click Save.

bq

For new datasets, use the bq mk --dataset command with the --add_tags flag:

bq mk --dataset \
    --add_tags=TAG[,...] \
    PROJECT_ID:DATASET_ID

For existing datasets, use the bq update command with the --add_tags flag:

bq update \
    --add_tags=TAG[,...] \
    PROJECT_ID:DATASET_ID

Replace the following:

  • TAG: the tag that you are attaching to the new table. Multiple tags are separated by commas. For example, 556741164180/env:prod,myProject/department:sales. Each tag must have the namespaced key name and value short name.
  • PROJECT_ID: the ID of the project where you are creating a dataset, or where an existing dataset is located.
  • DATASET_ID: the ID of the new dataset that you are creating, or the ID of an existing dataset.

gcloud

To attach a tag to a dataset using the command line, create a tag binding resource by using the gcloud resource-manager tags bindings create command:

gcloud resource-manager tags bindings create \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be attached, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value.
  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset.
  • LOCATION: the location of your dataset.

API

For new datasets, call the datasets.insert method and add your tags to the resource_tags field.

For existing datasets, first call the datasets.get method to get the dataset resource, including the resource_tags field. Add your tags to the resource_tags field and pass the updated dataset resource back using the datasets.update method.

List tags attached to a dataset

The following steps provide a list of tag bindings attached directly to a dataset. They don't return any tags that are inherited from parent resources.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

    The tags appear in the Dataset info section.

bq

To list tags attached to a dataset, use the bq show command.

bq show PROJECT_ID:DATASET_ID

Replace the following:

  • PROJECT_ID: the ID of the project containing your dataset.
  • DATASET_ID: the ID of the dataset for which you want to list the tags.

gcloud

To get a list of tag bindings attached to a resource, use the gcloud resource-manager tags bindings list command:

gcloud resource-manager tags bindings list \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset.

  • LOCATION: the location of your dataset.

The output is similar to the following:

name: tagBindings/%2F%2Fbigquery.googleapis.com%2Fprojects%2Fmy_project%2Fdatasets%2Fmy_dataset/tagValues/4567890123
parent: //bigquery.googleapis.com/projects/my_project/datasets/my_dataset
tagValue: tagValues/4567890123

API

Call the datasets.get method to get the dataset resource. The dataset resource includes tags attached to the dataset in the resource_tags field.

Views

Use the INFORMATION_SCHEMA.SCHEMATA_OPTIONS view.

For example, the following query shows all tags attached to all datasets in a region. This query returns a table with columns including schema_name (the dataset names), option_name (always 'tags'), object_type (always ARRAY<STRUCT<STRING, STRING>>), and option_value, which contains arrays of STRUCT objects representing tags associated with each dataset. For datasets without assigned tags, the option_value column returns an empty array.

SELECT * from region-REGION.INFORMATION_SCHEMA.SCHEMATA_OPTIONS
WHERE option_name='tags'

Replace the following:

  • REGION: the region where your datasets are located.

Detach tags from a dataset

You can detach a tag from a resource by deleting the tag binding resource. If you need to delete a tag, you should first detach the tag.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and select a dataset.

  3. In the Dataset info section, click Edit details.

  4. In the Tags section, click Delete item next to the tag you want to delete.

  5. Click Save.

bq

Use the bq update command with the --remove_tags flag:

bq update \
    --remove_tags=REMOVED_TAG[,...] \
    PROJECT_ID:DATASET_ID

Replace the following:

  • REMOVED_TAG: the tag that you are removing from the table. Only accepts keys without value pairs, and multiple keys are separated by commas.
  • PROJECT_ID: the ID of the project that contains your dataset.
  • DATASET_ID: the ID of the dataset to detach tags from.

Alternatively, if you want to remove all tags from a dataset, use the bq update command with the --clear_all_tags flag:

bq update \
    --clear_all_tags
    PROJECT_ID:DATASET_ID

gcloud

To detach a tag from a dataset using the command line, delete the tag binding by using the gcloud resource-manager tags bindings delete command:

gcloud resource-manager tags bindings delete \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be detached, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value.
  • RESOURCE_ID: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset.
  • LOCATION: the location of your dataset.

API

Call the datasets.get method to get the dataset resource, including the resource_tags field. Remove your tags from the resource_tags field and pass the updated dataset resource back using the datasets.update method.

Attach tags when you create a new table

After you create a tag, you can attach it to a new table. You can only attach one tag value to a dataset for any given tag key. You can attach a maximum of 50 tags to a table.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project, and then select a dataset.

  3. In the Dataset info section, click Create table.

  4. Enter the information for your new table. For more details, see Create and use tables.

  5. In the Tags section, select the tags that you want to add to the new table.

  6. Click Create table.

bq

Use the bq mk --table command with the --add_tags flag:

bq mk --table \
    --schema=SCHEMA \
    --add_tags=TAG[,...] \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • SCHEMA: the inline schema definition.
  • TAG: the tag that you are attaching to the new table. Multiple tags are separated by commas. For example, 556741164180/env:prod,myProject/department:sales. Each tag must have the namespaced key name and value short name.
  • PROJECT_ID: the ID of the project where you are creating a table.
  • DATASET_ID: the ID of the dataset where you are creating a table.
  • TABLE_ID: the ID of the new table that you are creating.

API

Call the tables.insert method with a defined table resource. Include the tags in the resource_tags field.

Attach tags to an existing table

After you create a tag, you can attach it to an existing table. You can only attach one tag value to a table for any given tag key.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

  3. In the Details tab, click Edit details.

  4. In the Tags section, select the tags that you want to add to the table.

  5. Click Save.

bq

Use the bq update command with the --add_tags flag:

bq update \
    --add_tags=TAG[,...] \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • TAG: the tag that you are attaching to the table. Multiple tags are separated by commas. For example, 556741164180/env:prod,myProject/department:sales. Each tag must have the namespaced key name and value short name.
  • PROJECT_ID: the ID of the project that contains your table.
  • DATASET_ID: the ID of the dataset that contains your table.
  • TABLE_ID: the ID of the table that you are updating.

gcloud

To attach a tag to a table using the command line, create a tag binding resource by using the gcloud resource-manager tags bindings create command:

gcloud resource-manager tags bindings create \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be attached, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value.
  • RESOURCE_ID: the full ID of the table, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table
  • LOCATION: the location of your table.

API

Call the tables.update method with a defined table resource. Include the tags in the resource_tags field.

List tags attached to a table

You can list tags that are attached directly to a table. This process doesn't list tags that are inherited from parent resources.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

    The tags are visible in the Details tab.

bq

Use the bq show command and look for the tags column. If there are no tags on the table, the tags column isn't displayed.

bq show \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • PROJECT_ID: the ID of the project that contains your table.
  • DATASET_ID: the ID of the dataset that contains your table.
  • TABLE_ID: the ID of your table.

gcloud

To get a list of tag bindings attached to a resource, use the gcloud resource-manager tags bindings list command:

gcloud resource-manager tags bindings list \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • RESOURCE_ID: the full ID of the table, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table.

  • LOCATION: the location of your dataset.

The output is similar to the following:

name: tagBindings/%2F%2Fbigquery.googleapis.com%2Fprojects%2Fmy_project%2Fdatasets%2Fmy_dataset/tagValues/4567890123
parent: //bigquery.googleapis.com/projects/my_project/datasets/my_dataset
tagValue: tagValues/4567890123

API

Call the tables.get method with a defined table resource, and look for the resource_tags field.

Views

Use the INFORMATION_SCHEMA.TABLE_OPTIONS view.

For example, the following query shows all tags attached to all tables in a dataset. This query returns a table with columns including schema_name (the dataset name), option_name (always 'tags'), object_type (always ARRAY<STRUCT<STRING, STRING>>), and option_value, which contains arrays of STRUCT objects representing tags associated with each dataset. For tables without assigned tags, the option_value column returns an empty array.

SELECT * from DATASET_ID.INFORMATION_SCHEMA.TABLE_OPTIONS
WHERE option_name='tags'

Replace DATASET_ID with the ID of the dataset that contains your table.

Detach tags from a table

You can remove a tag association from a table by deleting the tag binding. If you need to delete a tag, you should detach the tag first.

Console

  1. In the Google Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, expand your project and dataset, and then select a table.

  3. In the Details tab, click Edit details.

  4. In the Tags section, remove the tags that you want to detach from the table.

  5. Click Save.

bq

To remove some tags from a table, use the bq update command with the --remove_tags flag:

bq update \
    --remove_tags=TAG_KEYS \
    PROJECT_ID:DATASET_ID.TABLE_ID

Replace the following:

  • TAG_KEYS: the tag keys that you are detaching from the table, separated by commas. For example, 556741164180/env,myProject/department. Each tag key must have the namespaced key name.
  • PROJECT_ID: the ID of the project that contains your table.
  • DATASET_ID: the ID of the dataset that contains your table.
  • TABLE_ID: the ID of the table that you are updating.

To remove all tags from a table, use the bq update command with the --clear_all_tags flag:

bq update \
    --clear_all_tags \
    PROJECT_ID:DATASET_ID.TABLE_ID

gcloud

To remove a tag association from a table using the command line, delete the tag binding by using the gcloud resource-manager tags bindings delete command:

gcloud resource-manager tags bindings delete \
    --tag-value=TAGVALUE_NAME \
    --parent=RESOURCE_ID \
    --location=LOCATION

Replace the following:

  • TAGVALUE_NAME: the permanent ID or namespaced name of the tag value to be deleted, such as tagValues/4567890123 or 1234567/my_tag_key/my_tag_value.
  • RESOURCE_ID: the full ID of the table, including the API domain name (//bigquery.googleapis.com/) to identify the type of resource. For example, //bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table.
  • LOCATION: the location of your dataset.

API

Call the tables.update method with a defined table resource, and remove the tags in the resource_tags field. To remove all tags, remove the resource_tags field.

Delete tags

You can't delete a tag if it's referenced by a dataset or table. You should detach all existing tag binding resources before deleting the tag key or value itself. To delete tag keys and tag values, see Deleting tags.

Example

Suppose you are an administrator of an organization. Your data analysts are all members of the group analysts@example.com, which has the BigQuery Data Viewer IAM role on the project userData. A data analyst intern is hired, and according to the company policy they should only have permission to view the anonymousData dataset in the userData project. You can control their access using tags.

  1. Create a tag with the key employee_type and the value intern:

    Example of creating tag key and values.

  2. In the Google Cloud console, go to the IAM page.

    Go to IAM

  3. Locate the row that contains the intern whose dataset access you want to restrict, and click Edit principal in that row.

  4. From the Role menu, select BigQuery Data Viewer.

  5. Click Add condition.

  6. In the Title and Description fields, enter values that describe the IAM tag condition that you want to create.

  7. On the Condition builder tab, click Add.

  8. In the Condition type menu, select Resource, then select Tag.

  9. In the Operator menu, select has value.

  10. In the Value path field, enter the tag value path in the form ORGANIZATION/TAG_KEY/TAG_VALUE. For example, example.org/employee_type/intern.

    Example of an IAM condition using tags.

    This IAM tag condition restricts the intern's access to datasets that have the intern tag.

  11. To save the tag condition, click Save.

  12. To save any changes that you made in the Edit permissions pane, click Save.

  13. To attach the intern tag value to the anonymousData dataset, use the command line to run the gcloud resource-manager tags bindings create command:

    gcloud resource-manager tags bindings create \
--tag-value=tagValues/4567890123 \
--parent=//bigquery.googleapis.com/projects/userData/datasets/anonymousData \
--location=US

Limitations

  • Table tags aren't supported on BigQuery Omni tables, tables in hidden datasets, or temporary tables. Additionally, cross-region queries in BigQuery Omni don't use tags during access control checks of tables in other regions.

  • You can attach a maximum of 50 tags to a dataset or table.

  • Access control checks for wildcard queries don't consider conditional access granted for tagged tables.

  • Some services outside of BigQuery cannot properly verify IAM tag conditions. If the tag condition is positive, meaning that a user is granted a role on a resource only if that resource has a particular tag, then access is denied to the resource regardless of what tags are attached to it. If the tag condition is negative, meaning that a user is granted a role on a resource only if that resource doesn't have a particular tag, then the tag condition is not checked.

    For example, Data Catalog cannot verify IAM tag conditions on BigQuery datasets. Suppose there is a conditional IAM policy that gives an intern the BigQuery Data Viewer role on datasets with the employee_type=intern tag. Since this is a positive tag condition, the intern cannot view datasets by searching in Data Catalog even if those datasets do have the employee_type=intern tag. If the tag condition was changed to a negative one, so that the intern could only view datasets that did not have the employee_type=intern tag, then the check would be skipped entirely and the intern could view the datasets that they couldn't normally access in BigQuery.

What's next