Add a policy tag to a BigQuery table using Data Catalog

Learn how to get started with policy tags in Data Catalog to protect columns with sensitive data in BigQuery tables:

  1. Create a BigQuery table from a publicly available dataset called Chicago taxi trips.
  2. Protect the column that holds company names with a new policy tag.
  3. Query the table to verify that a user without the required role can't access the data.

Before you begin

Set up your project:

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Enable the Data Catalog and BigQuery APIs.

    Enable the APIs

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Enable the Data Catalog and BigQuery APIs.

    Enable the APIs

Add a public dataset to your project

  1. In Cloud console, go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer section, click Add data and select Explore public datasets from the drop-down list.

  3. In the Marketplace pane, search for Chicago taxi trips and click the Chicago Taxi Trips result

  4. Click View Dataset.

The dataset is added to your project. The active tab displays public info page for the chicago_taxi_trips dataset. It shows details such as Dataset ID, Data location, or Last modified date.

Create a dataset and a table

  1. Create a dataset.

    1. In the Cloud console, open the BigQuery page.

      Go to BigQuery

    2. In the Explorer pane, select the project where you want to create the dataset.

    3. Click the Actions icon and click Create dataset.

    4. In the Create dataset page, fill in the following details:

      • For Dataset ID, enter: policy_tags_demo
      • For Data location, select us (multiple regions in United States).
      • Leave the Enable table expiration box unchecked.
      • Click Create dataset.
  2. Copy a publicly accessible table to your policy_tags_demo dataset.

    1. In the Cloud console, open the BigQuery page.

      Go to BigQuery

    2. In the Explorer pane, search for the taxi_trips dataset.

    3. From the search results, under chicago_taxi_trips dataset, click the taxi_trips table.

    4. In the table details tab, click Copy.

    5. In the Copy table pane, fill in the following information:

      1. In the Project name field, click Browse and select your project.
      2. In the Dataset name drop-down list, select policy_tags_demo.
      3. For the Table name, enter my_taxi_trips_copy and click Copy.
    6. In the Explorer pane, confirm that the my_taxi_trips_copy table is listed under policy_tags_demo dataset.

Query the table to verify results visibility

Your new table doesn't have any policy tags attached, so you can view all the data it contains. Check which taxi company is the most popular:

  1. In the Cloud console, open the BigQuery page.

    Go to BigQuery

  2. Click Compose new query.

    Compose new query

  3. In the Query editor area, enter the following:

      SELECT company, COUNT(*) AS number_of_trips 
      FROM `PROJECT_ID.policy_tags_demo.my_taxi_trips_copy` GROUP BY company
    

    Replace PROJECT_ID with your project's identifier. For more information on finding your project ID, see Identifying projects.

  4. Click Run.

    A new pane opens with the query results. Looks like Yellow Cab is the winner!

    In the following sections, you create a policy tag and attach it to the company column to restrict column access.

Create a taxonomy and a policy tag

Taxonomies are organizational units used for grouping and managing policy tags. A policy tag can exist only inside a taxonomy. Create a new taxonomy:

  1. Open the Data Catalog Taxonomies page in the Cloud console.

    Open the Taxonomies page

  2. Click Create taxonomy.

  3. On the New taxonomy page:

    1. For Taxonomy name, enter: Taxi trips policy tags
    2. For Description, enter: Demo taxonomy for the policy tags Quickstart
    3. Under Policy Tags, enter:

      1. For the Name field: Sensitive taxi data
      2. For the Description field: Taxi company name
    4. Click Save.

    5. On the Policy tag taxonomy page, toggle on the Enforce access control slider.

Add your new policy tag to the company column

  1. Open the Data Catalog home page.

    Open the Data Catalog home page

  2. In the Search box, enter my_taxi_trips_copy and select your table from the results list.

  3. On the asset page, scroll down to the Schema section.

  4. In the Schema table, find the company row, and under Policy Tags click +.

  5. In the Add a policy tag pane, expand the Taxi trips policy tags taxonomy and select the Sensitive taxi data tag.

  6. At the bottom of the pane, click Select.

The company column in your table is now protected. Users without the Data Catalog Fine-Grained Reader role can't see it in query results.

Even though you created the policy tag, your user account isn't automatically added to the list of Fine-Grained Readers so you can't see this column when you query the table. Follow the next section to verify .

Query the table to verify that results are hidden

  1. In the Cloud console, open the BigQuery page.

    Go to BigQuery

  2. Click Compose new query.

    Compose new query

  3. In the Query editor area, enter the following:

      SELECT company, COUNT(*) AS number_of_trips 
      FROM `PROJECT_ID.policy_tags_demo.my_taxi_trips_copy` GROUP BY company
    

    Replace PROJECT_ID with your project's identifier. For more information on finding your project ID, see Identifying projects.

  4. Click Run.

    The column is protected with a policy tag, so the results tab displays an Access Denied error about insufficient permissions.

Add the Fine-Grained Reader role to view the protected column

Grant your account the Fine-Grained Reader role on the Sensitive taxi data policy tag.

  1. Go to the Data Catalog > Policy tags page.

    Go to Data Catalog policy tags

  2. Click Taxi trips policy tags.

  3. In the Policy tags section, select the Sensitive taxi data policy tag.

  4. In the Sensitive taxi data info pane, click Add principal.

    If you can't see the info pane, click Show info panel.

  5. In the Add principals pane:

    1. In the New principals box, enter your email address.
    2. From the Select a role menu, select Data Catalog > Fine-Grained Reader.
    3. Click Save.

You now have access to the company column tagged with the Sensitive taxi data policy tag. Query the table again to see if it works.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the Cloud console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete the dataset

  1. Go to the BigQuery page.

    Go to BigQuery

  2. In the Explorer pane, search for the policy_tags_demo dataset you created.

  3. Click the Actions option and click Delete dataset.

  4. Confirm your delete action.

Delete the policy tag taxonomy

  1. Go to the Data Catalog > Policy tags page.

    Go to Data Catalog policy tags

  2. Click Taxi trips policy tags.

  3. On the Policy tag taxonomy page, click Delete policy tag taxonomy.

  4. Confirm your delete action.

What's next

See the BigQuery column-level security guide for a detailed overview of policy tags.