Tag tables, views, and datasets
This document describes how to use tags to conditionally apply Identity and Access Management (IAM) policies to BigQuery tables, views, and datasets.
A tag is a key-value pair that you can attach directly to a table, view, or
dataset or a key-value pair that a table, view, or dataset can
inherit from other
Google Cloud resources. You can conditionally apply policies based on
whether a resource has a specific tag. For example, you might conditionally
grant the BigQuery Data Viewer role to a principal on any dataset with the
environment:dev
tag.
For more information about using tags across the Google Cloud resource hierarchy, see Tags overview.
To grant permissions to many related BigQuery resources at the same time, including resources that don't exist yet, consider using IAM Conditions.
Before you begin
You need to grant IAM roles that give users the necessary permissions to perform each task in this document. You also need to create tag keys and values to attach to resources.
Required permissions
To use tags in BigQuery, you need the following permissions:
- To attach a tag to a table or view, you need the
bigquery.tables.createTagBinding
IAM permission on the table or view, and theresourcemanager.tagValueBindings.create
permission at the project level on the tag value that you want to attach. - To attach a tag to a dataset, you need the
bigquery.datasets.createTagBinding
IAM permission on the dataset, and theresourcemanager.tagValueBindings.create
permission at the project level on the tag value that you want to attach. - To remove a tag from a table or view, you need the
bigquery.tables.deleteTagBinding
IAM permission on the table or view, and theresourcemanager.tagValueBindings.delete
permission at the project level on the tag value that you want to delete. - To remove a tag from a dataset, you need the
bigquery.datasets.deleteTagBinding
IAM permission on the dataset, and theresourcemanager.tagValueBindings.delete
permission at the project level on the tag value that you want to delete. - To list the tag keys that are associated with a parent organization or
project in the Edit details panel for a table, view, or dataset, you need
the
resourcemanager.tagKeys.list
permission at the tag key's parent level and theresourcemanager.tagKeys.get
permission for each tag key. - To list the tag values of keys that are associated with a parent organization
or project in the Edit details panel for a table, view, or dataset, you
need the
resourcemanager.tagValues.list
permission at the tag value parent level and theresourcemanager.tagValues.get
permission for each tag value.
If you are using tags with the Cloud Resource Manager API or gcloud, you also need the following permissions:
- To list the tags attached to a
table or view with the Cloud Resource Manager API or the gcloud CLI, you
need the
bigquery.tables.listTagBindings
IAM permission. - To list the effective tags for
a table or view, you need the
bigquery.tables.listEffectiveTags
IAM permission. - To list the tags attached to a
dataset with the Cloud Resource Manager API or the gcloud CLI,
you need the
bigquery.datasets.listTagBindings
IAM permission. - To list the effective tags for
a dataset, you need the
bigquery.datasets.listEffectiveTags
IAM permission.
Both of the following predefined IAM roles include all of the necessary BigQuery permissions:
- BigQuery Data Owner (
roles/bigquery.dataOwner
) - BigQuery Admin (
roles/bigquery.admin
)
The Resource Manager permissions are included in the Tag User role
(roles/resourcemanager.tagUser
).
You can also use tags to conditionally deny access with IAM policies to BigQuery tables, views, and datasets (preview). For more information, see Deny policies.
Create tag keys and values
Before you can attach a tag, you need to create a tag and configure its value. To create tag keys and tag values, see Creating a tag and Adding tag values.
Tag tables
The following sections describe how to attach tags to new and existing tables, list tags attached to a table, and detach tags from a table.
Attach tags when you create a new table
After you create a tag, you can attach it to a new table. You can attach only one tag value to a table for any given tag key. You can attach a maximum of 50 tags to a table.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project, and then select a dataset.
In the Dataset info section, click
Create table.Enter the information for your new table. For more details, see Create and use tables.
In the Tags section, select the tags that you want to add to the new table.
Click Create table.
bq
Use the
bq mk --table
command
with the --add_tags
flag:
bq mk --table \ --schema=SCHEMA \ --add_tags=TAG \ PROJECT_ID:DATASET_ID.TABLE_ID
Replace the following:
SCHEMA
: the inline schema definition.TAG
: the tag that you are attaching to the new table. Multiple tags are separated by commas. For example,556741164180/env:prod,myProject/department:sales
. Each tag must have the namespaced key name and value short name.PROJECT_ID
: the ID of the project where you are creating a table.DATASET_ID
: the ID of the dataset where you are creating a table.TABLE_ID
: the ID of the new table that you are creating.
Terraform
Use the
google_bigquery_table
resource.
To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries.
The following example creates a table named mytable
, then attaches
tags to it by populating the resource_tags
field:
To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.
Prepare Cloud Shell
- Launch Cloud Shell.
-
Set the default Google Cloud project where you want to apply your Terraform configurations.
You only need to run this command once per project, and you can run it in any directory.
export GOOGLE_CLOUD_PROJECT=PROJECT_ID
Environment variables are overridden if you set explicit values in the Terraform configuration file.
Prepare the directory
Each Terraform configuration file must have its own directory (also called a root module).
-
In Cloud Shell, create a directory and a new
file within that directory. The filename must have the
.tf
extension—for examplemain.tf
. In this tutorial, the file is referred to asmain.tf
.mkdir DIRECTORY && cd DIRECTORY && touch main.tf
-
If you are following a tutorial, you can copy the sample code in each section or step.
Copy the sample code into the newly created
main.tf
.Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.
- Review and modify the sample parameters to apply to your environment.
- Save your changes.
-
Initialize Terraform. You only need to do this once per directory.
terraform init
Optionally, to use the latest Google provider version, include the
-upgrade
option:terraform init -upgrade
Apply the changes
-
Review the configuration and verify that the resources that Terraform is going to create or
update match your expectations:
terraform plan
Make corrections to the configuration as necessary.
-
Apply the Terraform configuration by running the following command and entering
yes
at the prompt:terraform apply
Wait until Terraform displays the "Apply complete!" message.
- Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.
API
Call the
tables.insert
method
with a defined table resource.
Include the tags in the resource_tags
field.
Attach tags to an existing table
After you create a tag, you can attach it to an existing table. You can attach only one tag value to a table for any given tag key.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and dataset, and then select a table.
In the Details tab, click
Edit details.In the Tags section, select the tags that you want to add to the table.
Click Save.
bq
Use the
bq update
command
with the --add_tags
flag:
bq update \ --add_tags=TAG \ PROJECT_ID:DATASET_ID.TABLE_ID
Replace the following:
TAG
: the tag that you are attaching to the table. Multiple tags are separated by commas. For example,556741164180/env:prod,myProject/department:sales
. Each tag must have the namespaced key name and value short name.PROJECT_ID
: the ID of the project that contains your table.DATASET_ID
: the ID of the dataset that contains your table.TABLE_ID
: the ID of the table that you are updating.
gcloud
To attach a tag to a table using the command line, create a
tag binding resource by using the
gcloud resource-manager tags bindings create
command:
gcloud resource-manager tags bindings create \ --tag-value=TAGVALUE_NAME \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
TAGVALUE_NAME
: the permanent ID or namespaced name of the tag value to be attached, such astagValues/4567890123
or1234567/my_tag_key/my_tag_value
.RESOURCE_ID
: the full ID of the table, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table
LOCATION
: the location of your table.
Terraform
Add tags to the table's resource_tags
field, and then apply the
updated configuration using the google_bigquery_table
resource. For
more information, see the Terraform example in
Attach tags when you create a new table.
API
Call the
tables.update
method
with a defined table resource.
Include the tags in the resource_tags
field.
List tags attached to a table
You can list tags that are attached directly to a table. This process doesn't list tags that are inherited from parent resources.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and dataset, and then select a table.
The tags are visible in the Details tab.
bq
Use the
bq show
command
and look for the tags
column. If there are no tags on the table, the
tags
column isn't displayed.
bq show \ PROJECT_ID:DATASET_ID.TABLE_ID
Replace the following:
PROJECT_ID
: the ID of the project that contains your table.DATASET_ID
: the ID of the dataset that contains your table.TABLE_ID
: the ID of your table.
gcloud
To get a list of tag bindings attached to a resource, use the
gcloud resource-manager tags bindings list
command:
gcloud resource-manager tags bindings list \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
RESOURCE_ID
: the full ID of the table, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table
.LOCATION
: the location of your dataset.
The output is similar to the following:
name: tagBindings/%2F%2Fbigquery.googleapis.com%2Fprojects%2Fmy_project%2Fdatasets%2Fmy_dataset/tagValues/4567890123 parent: //bigquery.googleapis.com/projects/my_project/datasets/my_dataset tagValue: tagValues/4567890123
Terraform
Use the terraform state show
command to list the attributes of the
table, including the resource_tags
field. Run this command
in the directory where the table's Terraform configuration file has been
run.
terraform state show google_bigquery_table.default
API
Call the
tables.get
method
with a defined table resource,
and look for the resource_tags
field.
Views
Use the
INFORMATION_SCHEMA.TABLE_OPTIONS
view.
For example, the following query shows all tags attached to all tables in a
dataset. This query returns a table with columns including schema_name
(the dataset name), option_name
(always 'tags'
),
object_type
(always ARRAY<STRUCT<STRING, STRING>>
), and option_value
,
which contains arrays of STRUCT
objects representing tags associated with
each dataset. For tables without assigned tags, the option_value
column
returns an empty array.
SELECT * from DATASET_ID.INFORMATION_SCHEMA.TABLE_OPTIONS WHERE option_name='tags'
Replace DATASET_ID
with the ID of the dataset
that contains your table.
Detach tags from a table
You can remove a tag association from a table by deleting the tag binding. If you need to delete a tag, you should detach the tag first.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and dataset, and then select a table.
In the Details tab, click
Edit details.In the Tags section, remove the tags that you want to detach from the table.
Click Save.
bq
To remove some tags from a table, use the
bq update
command
with the --remove_tags
flag:
bq update \ --remove_tags=TAG_KEYS \ PROJECT_ID:DATASET_ID.TABLE_ID
Replace the following:
TAG_KEYS
: the tag keys that you are detaching from the table, separated by commas. For example,556741164180/env,myProject/department
. Each tag key must have the namespaced key name.PROJECT_ID
: the ID of the project that contains your table.DATASET_ID
: the ID of the dataset that contains your table.TABLE_ID
: the ID of the table that you are updating.
To remove all tags from a table, use the
bq update
command
with the --clear_all_tags
flag:
bq update \ --clear_all_tags \ PROJECT_ID:DATASET_ID.TABLE_ID
gcloud
To remove a tag association from a table using the command line, delete the
tag binding by using the
gcloud resource-manager tags bindings delete
command:
gcloud resource-manager tags bindings delete \ --tag-value=TAGVALUE_NAME \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
TAGVALUE_NAME
: the permanent ID or namespaced name of the tag value to be deleted, such astagValues/4567890123
or1234567/my_tag_key/my_tag_value
.RESOURCE_ID
: the full ID of the table, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset/my_table
.LOCATION
: the location of your dataset.
Terraform
Remove your tags from the table's resource_tags
field, and then apply
the updated configuration using the google_bigquery_table
resource.
API
Call the
tables.update
method
with a defined table resource,
and remove the tags in the resource_tags
field. To remove all tags, remove
the resource_tags
field.
Tag datasets
The following sections describe how to attach tags to new and existing datasets, list tags attached to a dataset, and detach tags from a dataset.
Attach tags when you create a new dataset
After you create a tag, you can attach it to a new BigQuery dataset. You can attach only one tag value to a dataset for any given tag key. You can attach a maximum of 50 tags to a dataset.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, select the project where you want to create your dataset.
Click > Create dataset.
View actionsIn the Tags menu, click Select scope.
Select the scope for your tags.
Select and add the tags that you want to add to the dataset.
Click Create dataset.
bq
Use the
bq mk --dataset
command
with the --add_tags
flag:
bq mk --dataset \ --add_tags=TAG \ PROJECT_ID:DATASET_ID
Replace the following:
TAG
: the tag that you are attaching to the new dataset. Multiple tags are separated by commas. For example,556741164180/env:prod,myProject/department:sales
. Each tag must have the namespaced key name and value short name.PROJECT_ID
: the ID of the project where you are creating a dataset.DATASET_ID
: the ID of the new dataset that you are creating.
Terraform
Use the
google_bigquery_dataset
resource.
To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries.
The following example creates a dataset named my_dataset
, then attaches
tags to it by populating the resource_tags
field:
To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.
Prepare Cloud Shell
- Launch Cloud Shell.
-
Set the default Google Cloud project where you want to apply your Terraform configurations.
You only need to run this command once per project, and you can run it in any directory.
export GOOGLE_CLOUD_PROJECT=PROJECT_ID
Environment variables are overridden if you set explicit values in the Terraform configuration file.
Prepare the directory
Each Terraform configuration file must have its own directory (also called a root module).
-
In Cloud Shell, create a directory and a new
file within that directory. The filename must have the
.tf
extension—for examplemain.tf
. In this tutorial, the file is referred to asmain.tf
.mkdir DIRECTORY && cd DIRECTORY && touch main.tf
-
If you are following a tutorial, you can copy the sample code in each section or step.
Copy the sample code into the newly created
main.tf
.Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.
- Review and modify the sample parameters to apply to your environment.
- Save your changes.
-
Initialize Terraform. You only need to do this once per directory.
terraform init
Optionally, to use the latest Google provider version, include the
-upgrade
option:terraform init -upgrade
Apply the changes
-
Review the configuration and verify that the resources that Terraform is going to create or
update match your expectations:
terraform plan
Make corrections to the configuration as necessary.
-
Apply the Terraform configuration by running the following command and entering
yes
at the prompt:terraform apply
Wait until Terraform displays the "Apply complete!" message.
- Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.
API
Call the
datasets.insert
method
and add your tags to the resource_tags
field.
Attach tags to an existing dataset
After you create a tag, you can attach it to an existing dataset. You can attach only one tag value to a dataset for any given tag key.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and select a dataset.
In the Dataset info section, click
Edit details.In the Tags section, select the tags that you want to add to the dataset.
Click Save.
bq
Use the
bq update
command
with the --add_tags
flag:
bq update \ --add_tags=TAG \ PROJECT_ID:DATASET_ID
Replace the following:
TAG
: the tag that you are attaching to the dataset. Multiple tags are separated by commas. For example,556741164180/env:prod,myProject/department:sales
. Each tag must have the namespaced key name and value short name.PROJECT_ID
: the ID of the project where the existing dataset is located.DATASET_ID
: the ID of the existing dataset.
gcloud
To attach a tag to a dataset using the command line, create a
tag binding resource by using the
gcloud resource-manager tags bindings create
command:
gcloud resource-manager tags bindings create \ --tag-value=TAGVALUE_NAME \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
TAGVALUE_NAME
: the permanent ID or namespaced name of the tag value to be attached, such astagValues/4567890123
or1234567/my_tag_key/my_tag_value
.RESOURCE_ID
: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset
.LOCATION
: the location of your dataset.
Terraform
Add tags to the dataset's resource_tags
field, and then apply the
updated configuration using the google_bigquery_dataset
resource. For
more information, see the Terraform example in
Attach tags when you create a new dataset.
API
Call the
datasets.get
method
to get the dataset resource, including the resource_tags
field. Add your
tags to the resource_tags
field and pass the updated dataset resource
back using the
datasets.update
method.
List tags attached to a dataset
The following steps provide a list of tag bindings attached directly to a dataset. They don't return any tags that are inherited from parent resources.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and select a dataset.
The tags appear in the Dataset info section.
bq
To list tags attached to a dataset, use the
bq show
command.
bq show PROJECT_ID:DATASET_ID
Replace the following:
PROJECT_ID
: the ID of the project containing your dataset.DATASET_ID
: the ID of the dataset for which you want to list the tags.
gcloud
To get a list of tag bindings attached to a resource, use the
gcloud resource-manager tags bindings list
command:
gcloud resource-manager tags bindings list \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
RESOURCE_ID
: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset
.LOCATION
: the location of your dataset.
The output is similar to the following:
name: tagBindings/%2F%2Fbigquery.googleapis.com%2Fprojects%2Fmy_project%2Fdatasets%2Fmy_dataset/tagValues/4567890123 parent: //bigquery.googleapis.com/projects/my_project/datasets/my_dataset tagValue: tagValues/4567890123
Terraform
Use the terraform state show
command to list the attributes of the
dataset, including the resource_tags
field. Run this command
in the directory where the dataset's Terraform configuration file has been
run.
terraform state show google_bigquery_dataset.default
API
Call the
datasets.get
method
to get the dataset resource. The dataset resource includes tags attached to
the dataset in the resource_tags
field.
Views
Use the
INFORMATION_SCHEMA.SCHEMATA_OPTIONS
view.
For example, the following query shows all tags attached to all datasets
in a region. This query returns a table with columns including schema_name
(the dataset names), option_name
(always 'tags'
),
object_type
(always ARRAY<STRUCT<STRING, STRING>>
), and option_value
,
which contains arrays of STRUCT
objects representing tags associated with
each dataset. For datasets without assigned tags, the option_value
column
returns an empty array.
SELECT * from region-REGION.INFORMATION_SCHEMA.SCHEMATA_OPTIONS WHERE option_name='tags'
Replace the following:
REGION
: the region where your datasets are located.
Detach tags from a dataset
You can detach a tag from a resource by deleting the tag binding resource. If you need to delete a tag, you should first detach the tag.
Console
In the Google Cloud console, go to the BigQuery page.
In the Explorer pane, expand your project and select a dataset.
In the Dataset info section, click
Edit details.In the Tags section, click
Delete item next to the tag you want to delete.Click Save.
bq
Use the
bq update
command
with the --remove_tags
flag:
bq update \ --remove_tags=REMOVED_TAG \ PROJECT_ID:DATASET_ID
Replace the following:
REMOVED_TAG
: the tag that you are removing from the dataset. Multiple tags are separated by commas. Only accepts keys without value pairs. For example,556741164180/env,myProject/department
. Each tag must have the namespaced key name.PROJECT_ID
: the ID of the project that contains your dataset.DATASET_ID
: the ID of the dataset to detach tags from.
Alternatively, if you want to remove all tags from a dataset, use the
bq update
command
with the --clear_all_tags
flag:
bq update \ --clear_all_tags PROJECT_ID:DATASET_ID
gcloud
To detach a tag from a dataset using the command line, delete the tag
binding by using the
gcloud resource-manager tags bindings delete
command:
gcloud resource-manager tags bindings delete \ --tag-value=TAGVALUE_NAME \ --parent=RESOURCE_ID \ --location=LOCATION
Replace the following:
TAGVALUE_NAME
: the permanent ID or namespaced name of the tag value to be detached, such astagValues/4567890123
or1234567/my_tag_key/my_tag_value
.RESOURCE_ID
: the full ID of the dataset, including the API domain name (//bigquery.googleapis.com/
) to identify the type of resource. For example,//bigquery.googleapis.com/projects/my_project/datasets/my_dataset
.LOCATION
: the location of your dataset.
Terraform
Remove your tags from the dataset's resource_tags
field, and then apply
the updated configuration using the google_bigquery_dataset
resource.
API
Call the
datasets.get
method
to get the dataset resource, including the resource_tags
field. Remove
your tags from the resource_tags
field and pass the updated dataset
resource back using the
datasets.update
method.
Tag other table-like resources
You can similarly tag BigQuery Views, Materialized Views, Clones, and Snapshots.
Delete tags
You can't delete a tag if it's referenced by a table, view, or dataset. You should detach all existing tag binding resources before deleting the tag key or value itself. To delete tag keys and tag values, see Deleting tags.
Example
Suppose you are an administrator of an organization. Your
data analysts are all members of the group analysts@example.com, which has the
BigQuery Data Viewer IAM role on the project userData
. A data
analyst intern is hired, and according to the company policy they should only
have permission to view the anonymousData
dataset in the userData
project.
You can control their access using tags.
Create a tag with the key
employee_type
and the valueintern
:In the Google Cloud console, go to the IAM page.
Locate the row that contains the intern whose dataset access you want to restrict, and click
Edit principal in that row.From the Role menu, select BigQuery Data Viewer.
Click Add condition.
In the Title and Description fields, enter values that describe the IAM tag condition that you want to create.
On the Condition builder tab, click Add.
In the Condition type menu, select Resource, then select Tag.
In the Operator menu, select has value.
In the Value path field, enter the tag value path in the form
ORGANIZATION/TAG_KEY/TAG_VALUE
. For example,example.org/employee_type/intern
.This IAM tag condition restricts the intern's access to datasets that have the
intern
tag.To save the tag condition, click Save.
To save any changes that you made in the Edit permissions pane, click Save.
To attach the
intern
tag value to theanonymousData
dataset, use the command line to run thegcloud resource-manager tags bindings create
command:gcloud resource-manager tags bindings create \ --tag-value=tagValues/4567890123 \ --parent=//bigquery.googleapis.com/projects/userData/datasets/anonymousData \ --location=US
Limitations
Table tags aren't supported on BigQuery Omni tables, tables in hidden datasets, or temporary tables. Additionally, cross-region queries in BigQuery Omni don't use tags during access control checks of tables in other regions.
You can attach a maximum of 50 tags to a table or dataset.
All tables referenced in a wildcard query must have exactly the same set of tag keys and values.
Some services outside of BigQuery cannot properly verify IAM tag conditions. If the tag condition is positive, meaning that a user is granted a role on a resource only if that resource has a particular tag, then access is denied to the resource regardless of what tags are attached to it. If the tag condition is negative, meaning that a user is granted a role on a resource only if that resource doesn't have a particular tag, then the tag condition is not checked.
For example, Data Catalog cannot verify IAM tag conditions on BigQuery datasets. Suppose there is a conditional IAM policy that gives an intern the BigQuery Data Viewer role on datasets with the
employee_type=intern
tag. Since this is a positive tag condition, the intern cannot view datasets by searching in Data Catalog even if those datasets do have theemployee_type=intern
tag. If the tag condition was changed to a negative one, so that the intern could only view datasets that did not have theemployee_type=intern
tag, then the check would be skipped entirely and the intern could view the datasets that they couldn't normally access in BigQuery.
What's next
- For an overview of tags in Google Cloud, see Tags overview.
- For more information about how to use tags, see Creating and managing tags.
- For information about how to control access to BigQuery resources with IAM Conditions, see Control access with IAM Conditions.