Using Labels

To help organize your BigQuery resources, you can add labels to your datasets, tables, and views. After labeling your resources, you can search for them based on label values. For example, you can use labels to group datasets by purpose, environment, department, and so on.

Before you begin

Overview

Labels are key:value pairs that you can attach to a resource. BigQuery labels have the same characteristics as labels in Compute Engine and Cloud Resource Manager:

  • Label keys and values can be no longer than 63 characters.
  • Label keys and values can contain only lowercase letters, numbers, underscores, hyphens, and international characters.
  • Label keys and values cannot exceed 128 bytes in size.
  • Label keys must begin with a letter.
  • Label keys must be unique within a dataset.

Each of your resources can have up to 64 labels. By default, datasets, tables, and views do not have labels when they are created.

Labels must have a key, but can have an empty value. Labels with no value can be used as tags.

Creating or updating a label

Creating or updating labels requires update permissions. Adding a label to a dataset requires bigquery.datasets.update permissions. Adding a label to a table or view requires bigquery.tables.update permissions.

The following predefined IAM roles include bigquery.datasets.update permissions:

The following predefined IAM roles include bigquery.tables.update permissions:

For more information on IAM roles and permissions in BigQuery, see Access Control.

To create or update a label for a resource:

Web UI

  1. In the web UI, select the appropriate resource (a dataset, table, or view).

  2. For datasets, the Dataset Details page is automatically opened. For tables and views, click Details to open the details page.

  3. On the details page, to the right of Labels, click Edit.

    Edit labels

  4. In the Edit Labels dialog:

    • Enter your key and value or update the existing ones. To apply additional labels, click Add Label.
    • Click OK.

      New label

Command-line

Use the bq update command with the set_label flag.

bq update --set_label [KEY1:VALUE1] --set_label [KEY2:VALUE2] --set_label [KEYN:VALUEN] [RESOURCE_ID]

Where [KEY1:VALUE1], [KEY2:VALUE2], and [KEYN:VALUEN] are label key:value pairs that you want to create, and [RESOURCE_ID] is a valid dataset ID, table ID, or view ID.

For example, to add a label to track departments, you can create a label with a department key:

bq update --set_label department:shipping [RESOURCE_ID]

API

  1. Populate the labels field in the resource's configuration object. For example, to populate a label for a dataset configuration object:

          {
            "kind": "bigquery#dataset",
            ...
            "friendlyName": string,
            "description": string,
            "labels": {
              "[KEY1]": "[VALUE1]",
              "[KEY2]": "[VALUE2]"
            }
             ...
          }
    
      Where `[KEY1]` and `[KEY2]` are label keys, and `[VALUE1]` and `[VALUE2]`
      are the corresponding label values that you want to create.
    

  2. To replace all labels on a resource, call the update method for that resource: datasets.update() or tables.update(). tables.update is also used for views.
  3. Alternatively, to replace only the labels specified and otherwise preserve existing labels, call the patch method for the resource: datasets.patch() or tables.patch().

    If there are concurrent calls to update() and patch(), BigQuery executes the last one received.

Java

This sample uses the Google HTTP Client Library for Java to send a request to the BigQuery API.
static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
static final JsonFactory JSON_FACTORY = new JacksonFactory();

public static class Dataset {
  @Key private Map<String, String> labels;

  public Map<String, String> getLabels() {
    return this.labels;
  }

  public Dataset addLabel(String key, String value) {
    if (this.labels == null) {
      this.labels = new HashMap<>();
    }
    this.labels.put(key, value);
    return this;
  }
}

/**
 * Add or modify a label on a dataset.
 *
 * <p>See <a href="https://cloud.google.com/bigquery/docs/labeling-datasets">the BigQuery
 * documentation</a>.
 */
public static void labelDataset(
    String projectId, String datasetId, String labelKey, String labelValue) throws IOException {

  // Authenticate requests using Google Application Default credentials.
  GoogleCredential credential = GoogleCredential.getApplicationDefault();
  credential = credential.createScoped(Arrays.asList("https://www.googleapis.com/auth/bigquery"));

  // Get a new access token.
  // Note that access tokens have an expiration. You can reuse a token rather than requesting a
  // new one if it is not yet expired.
  credential.refreshToken();
  String accessToken = credential.getAccessToken();

  // Set the content of the request.
  Dataset dataset = new Dataset();
  dataset.addLabel(labelKey, labelValue);
  HttpContent content = new JsonHttpContent(JSON_FACTORY, dataset);

  // Send the request to the BigQuery API.
  String urlFormat =
      "https://www.googleapis.com/bigquery/v2/projects/%s/datasets/%s"
          + "?fields=labels&access_token=%s";
  GenericUrl url = new GenericUrl(String.format(urlFormat, projectId, datasetId, accessToken));
  HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
  HttpRequest request = requestFactory.buildPostRequest(url, content);
  request.setParser(JSON_FACTORY.createJsonObjectParser());

  // Workaround for transports which do not support PATCH requests.
  // See: http://stackoverflow.com/a/32503192/101923
  request.setHeaders(new HttpHeaders().set("X-HTTP-Method-Override", "PATCH"));
  HttpResponse response = request.execute();

  // Check for errors.
  if (response.getStatusCode() != 200) {
    throw new RuntimeException(response.getStatusMessage());
  }

  Dataset responseDataset = response.parseAs(Dataset.class);
  System.out.printf(
      "Updated label \"%s\" with value \"%s\"\n",
      labelKey, responseDataset.getLabels().get(labelKey));
}

Python

This sample uses the Google Auth Library for Python to send a request to the BigQuery API with an AuthorizedSession, which is compatible with a Requests session.
def label_dataset(dataset_id, label_key, label_value, project_id=None):
    """Add or modify a label on a dataset."""
    # Authenticate requests using Google Application Default credentials.
    credentials, default_project_id = google.auth.default(
            scopes=['https://www.googleapis.com/auth/bigquery'])
    session = google.auth.transport.requests.AuthorizedSession(credentials)

    if project_id is None:
        project_id = default_project_id

    # Send a PATCH request to add or modify a label.
    url_format = (
        'https://www.googleapis.com/bigquery/v2/'
        'projects/{project_id}/datasets/{dataset_id}')
    response = session.patch(
        url_format.format(project_id=project_id, dataset_id=dataset_id),
        params={'fields': 'labels'},
        json={
            'labels': {
                label_key: label_value,
            }
        })

    # Check the response for errors.
    response.raise_for_status()

    # Print the new label value from the response.
    labels = response.json()['labels']
    print(
        'Updated label "{}" with value "{}"'.format(
            label_key,
            labels[label_key]))

Deleting a label

Deleting labels requires update permissions. Deleting a label from a dataset requires bigquery.datasets.update permissions. Deleting a label from a table or view requires bigquery.tables.update permissions.

The following predefined IAM roles include bigquery.datasets.update permissions:

The following predefined IAM roles include bigquery.tables.update permissions:

For more information about IAM roles and permissions in BigQuery, see access control.

To delete a label from a resource:

Web UI

  1. In the web UI, select the appropriate resource (a dataset, table, or view).

  2. For datasets, the Dataset Details page is automatically opened. For tables and views, click Details to open the details page.

  3. On the details page, to the right of Labels, click Edit.

  4. In the Edit Labels dialog:

    • Click the delete icon (X) for each label you want to remove.
    • Click OK.

      Delete label

Command-line

Use the bq update command with the clear_label flag.

bq update --clear_label [KEY1] --clear_label [KEY2] --clear_label [KEYN] [RESOURCE_ID]

Where [KEY1], [KEY2], and [KEYN] are label keys that you want to delete, and [RESOURCE_ID] is a valid dataset ID, table ID, or view ID.

For example, to delete a label that tracks departments, specify the department key:

bq update --clear_label department [RESOURCE_ID]

API

To remove all labels from a resource:

  1. Remove the labels field from the resource's configuration object. For example, to remove all labels from a dataset:

    {
      "kind": "bigquery#dataset",
      ...
      "friendlyName": string,
      "description": string,
       ...
    }
    
  2. Call datasets.update() or tables.update(). Tables.update is also used for views.

To delete specific labels from a resource and otherwise preserve the other existing labels:

  1. Change the value of the labels you want to delete to null in the resource's configuration object. For example, to delete specific labels from a dataset:

    {
      "kind": "bigquery#dataset",
      ...
      "friendlyName": string,
      "description": string,
      "labels": {
        "[KEY1]": null,
        "[KEY2]": null
      }
       ...
    }
    

    Where [KEY1] and [KEY2] are label keys that you want to delete.

  2. Call datasets.patch() or tables.patch().

Creating a tag

A label that has a key but an empty value is used as a tag. You can create a new label with no value, or you can turn an existing label into a tag.

To create a tag:

Web UI

  1. In the web UI, select the appropriate resource (a dataset, table, or view).

  2. For datasets, the Dataset Details page is automatically opened. For tables and views, click Details to open the details page.

  3. On the details page, to the right of Labels, click Edit.

  4. In the Edit Labels dialog:

    • Enter a new key and leave the value blank or delete the value for an existing label. To apply additional tags, click Add Label.
    • Click OK.

      Add tag

Command-line

Use the bq update command with the set_label flag. Specify the key, followed by a colon, but leave the value unspecified. This can be used to update an existing label to a tag or to add a new tag.

bq update --set_label [KEY1]: --set_label [KEY2]: --set_label [KEYN]: [RESOURCE_ID]

Where [KEY1], [KEY2], and [KEYN] are label keys that you want to use as tags, and [RESOURCE_ID] is a valid dataset ID, table ID, or view ID.

For example, to delete the value from the existing department label, specify the department key with an empty value:

bq update --set_label department: [RESOURCE_ID]

API

  1. Add labels with the value set to the empty string ("") in the configuration object, or replace the value of existing labels with the empty string. For example, to create a tag for a dataset from an existing label:

    {
      "kind": "bigquery#dataset",
      ...
      "friendlyName": string,
      "description": string,
      "labels": {
        "[KEY1]": "",
        "[KEY2]": ""
      }
       ...
    }
    

    Where [KEY1] and [KEY2] are label keys that you want to keep, but with no values attached to them.

  2. Call the patch method for the resource: datasets.patch() or tables.patch().

Viewing labels

Getting information about labels requires get permissions for that resource: bigquery.datasets.get or bigquery.tables.get. All BigQuery IAM roles include bigquery.datasets.get permissions except bigquery.jobUser. All BigQuery IAM roles include bigquery.tables.get permissions except bigquery.user and bigquery.jobUser.

For more information about IAM roles and permissions in BigQuery, see Access control.

To view a resource's labels:

Command-line

Use the bq show command with the resource ID.

bq show [RESOURCE_ID]

Where [RESOURCE_ID] is a valid dataset ID, table ID, or view ID.

The command-line tool returns the labels associated with that dataset.

API

Call the datasets.get() command or the tables.get() command. The response includes all labels associated with that resource.

Alternatively, you can use datasets.list() to view the labels for multiple datasets or tables.list() to view the labels for multiple tables and views.

Filtering using labels

To filter for a set of resources based on labels, create a filter specification for use in the bq command-line tool or the BigQuery API that uses the following syntax:

field[:value][ field[:value]...

Where:

  • field is expressed as labels.[KEY] where [KEY] is a label key.
  • value is an optional label value.

The filter specification has the following characteristics:

  • Only the AND logical operator is supported. Space-separated comparisons are treated as having implicit AND operators.
  • The only field currently eligible for filtering is "labels.key" where "key" is the name of a label.
  • The filter can include up to ten expressions.
  • Filtering is case-sensitive.

Filter examples

To search for all resources that have a department key with a value of shipping, use the following specification:

labels.department:shipping

To search for multiple key:value pairs, separate them with a space. For example, to search for all resources where the value of the department key is shipping and the value of the location key is usa, use the following specification:

labels.department:shipping labels.location:usa

You can filter on the presence of a key alone, rather than matching against a key:value pair. The following filter specification returns all resources with labels named department, without regard to the values that might be associated with the keys:

labels.department

An equivalent specification uses an asterisk to represent all possible values associated with the department key:

labels.department:*

Generating filtered lists

To generate a filtered list of resources:

Command-line

Use the bq ls command with the filter flag where [FILTER_SPECIFICATION] is a valid filter specification:

bq ls --filter "[FILTER_SPECIFICATION]"

The command-line tool returns a list of datasets that meet the filter requirements.

For example, the following command returns a list of resources that have a department key with a value of shipping:

bq ls --filter "labels.department:shipping"

API

  1. Add a filter property that specifies the filter you want to use. For example, to add a filter property to a dataset:

    {
      "kind": "bigquery#dataset",
      ...
      "friendlyName": string,
      "description": string,
      "filter": "[FILTER_SPECIFICATION]",
       ...
    }
    

    Where [FILTER_SPECIFICATION] is a valid filter specification.

  2. Call the datasets.list() command or the tables.list() command. The response includes all labels associated with that resource.

What's next

Send feedback about...