Adding and using labels

To help organize your BigQuery resources, you can add labels to your datasets, tables, and views. Labels are key-value pairs that you can attach to a resource. When you create BigQuery resources, labels are optional.

After labeling your resources, you can search for them based on label values. For example, you can use labels to group datasets by purpose, environment, department, and so on.

This page explains how to label your BigQuery resources.

For information on managing labels, see Managing labels.

What are labels?

A label is a key-value pair that helps you organize your Google Cloud Platform BigQuery resources. You can attach a label to each resource, then filter the resources based on their labels. Information about labels is forwarded to the billing system, so you can break down your billing charges by label.

Common uses of labels

Here are some common use cases for labels:

  • Team or cost center labels: Add labels based on team or cost center to distinguish BigQuery resources owned by different teams (for example, team:research and team:analytics). You can use this type of label for cost accounting or budgeting.

  • Component labels: For example, component:redis, component:frontend, component:ingest, and component:dashboard.

  • Environment or stage labels: For example, environment:production and environment:test.

  • Owner or contact labels: Add labels based on the owner or a primary contact for BigQuery resources.

  • State labels: For example, state:active, state:readytodelete, and state:archive.

Requirements for labels

The labels applied to a resource must meet the following requirements:

  • Each resource can have multiple labels, up to a maximum of 64.
  • Each label must be a key-value pair.
  • Keys have a minimum length of 1 character and a maximum length of 63 characters, and cannot be empty. Values can be empty, and have a maximum length of 63 characters.
  • Keys and values can contain only lowercase letters, numeric characters, underscores, and dashes. All characters must use UTF-8 encoding, and international characters are allowed.
  • The key portion of a label must be unique. However, you can use the same key with multiple resources.
  • Keys must start with a lowercase letter or international character.

Adding dataset labels

A label can be added to a BigQuery dataset when it is created by using the command-line tool's bq mk command or by calling the datasets.insert API method. Currently, you cannot add a label to a dataset when it's created via the BigQuery web UI.

This page discusses how to add a label to a dataset after it is created. For more information on adding a label when you create a dataset, see Creating a dataset.

When you add a label to a dataset, it is included in your storage billing data, but you will not see dataset labels in your job-related billing data.

Required permissions

To add a label to an existing dataset you must have OWNER access at the dataset level, or you must be assigned a project-level IAM role that includes bigquery.datasets.update permissions. The following predefined, project-level IAM roles include bigquery.datasets.update permissions:

In addition, because the bigquery.user role has bigquery.datasets.create permissions, a user assigned to the bigquery.user role can update any dataset that user creates. When a user assigned to the bigquery.user role creates a dataset, that user is given OWNER access to the dataset. OWNER access to a dataset gives the user full control over it.

For more information on IAM roles and permissions in BigQuery, see Access Control. For more information on dataset-level roles, see Primitive roles for datasets.

Adding a label to a dataset

To add a label to a dataset after it is created:

Web UI

  1. In the web UI, select the dataset.

  2. On the Dataset Details page, to the right of Labels, click Edit.

    Edit labels

  3. In the Edit Labels dialog:

    • Enter your key and value to add a label. To apply additional labels, click Add Label. Each key can be used only once per dataset, but you can use the same key in different datasets in the same project.
    • Modify the existing keys or values to update a label.
    • Click OK.

      New label

Command-line

To add a label to an existing dataset, issue the bq update command with the set_label flag. Repeat the flag to add multiple labels.

If the dataset is in a project other than your default project, add the project ID to the dataset in the following format: [PROJECT_ID]:[DATASET].

    bq update --set_label [KEY:VALUE] [PROJECT_ID]:[DATASET]

Where:

  • [KEY:VALUE] corresponds to a key:value pair for a label that you want to add. The key must be unique.
  • [PROJECT_ID] is your project ID.
  • [DATASET] is the dataset you're labeling.

Examples:

To add a label to track departments, enter the bq update command and specify department as the label key. For example, to add a department:shipping label to mydataset in your default project, enter:

    bq update --set_label department:shipping mydataset

To add multiple labels to a dataset, repeat the set_label flag and specify a unique key for each label.For example, to add a department:shipping label and cost_center:logistics label to mydataset in your default project, enter:

    bq update --set_label department:shipping --set_label cost_center:logistics mydataset

API

To add a label to an existing dataset, call the datasets.patch method and populate the labels property for the dataset resource.

Because the datasets.update method replaces the entire dataset resource, the datasets.patch method is preferred.

Go

Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Go API reference documentation .

// To run this sample, you will need to create (or reuse) a context and
// an instance of the bigquery client.  For example:
// import "cloud.google.com/go/bigquery"
// ctx := context.Background()
// client, err := bigquery.NewClient(ctx, "your-project-id")
ds := client.Dataset(datasetID)
meta, err := ds.Metadata(ctx)
if err != nil {
	return err
}

update := bigquery.DatasetMetadataToUpdate{}
update.SetLabel("color", "green")
if _, err := ds.Update(ctx, update, meta.ETag); err != nil {
	return err
}

Java

This sample uses the Google HTTP Client Library for Java to send a request to the BigQuery API.
static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
static final JsonFactory JSON_FACTORY = new JacksonFactory();

public static class Dataset {
  @Key private Map<String, String> labels;

  public Map<String, String> getLabels() {
    return this.labels;
  }

  public Dataset addLabel(String key, String value) {
    if (this.labels == null) {
      this.labels = new HashMap<>();
    }
    this.labels.put(key, value);
    return this;
  }
}

/**
 * Add or modify a label on a dataset.
 *
 * <p>See <a href="https://cloud.google.com/bigquery/docs/labeling-datasets">the BigQuery
 * documentation</a>.
 */
public static void labelDataset(
    String projectId, String datasetId, String labelKey, String labelValue) throws IOException {

  // Authenticate requests using Google Application Default credentials.
  GoogleCredential credential = GoogleCredential.getApplicationDefault();
  credential = credential.createScoped(Arrays.asList("https://www.googleapis.com/auth/bigquery"));

  // Get a new access token.
  // Note that access tokens have an expiration. You can reuse a token rather than requesting a
  // new one if it is not yet expired.
  credential.refreshToken();
  String accessToken = credential.getAccessToken();

  // Set the content of the request.
  Dataset dataset = new Dataset();
  dataset.addLabel(labelKey, labelValue);
  HttpContent content = new JsonHttpContent(JSON_FACTORY, dataset);

  // Send the request to the BigQuery API.
  String urlFormat =
      "https://www.googleapis.com/bigquery/v2/projects/%s/datasets/%s"
          + "?fields=labels&access_token=%s";
  GenericUrl url = new GenericUrl(String.format(urlFormat, projectId, datasetId, accessToken));
  HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
  HttpRequest request = requestFactory.buildPostRequest(url, content);
  request.setParser(JSON_FACTORY.createJsonObjectParser());

  // Workaround for transports which do not support PATCH requests.
  // See: http://stackoverflow.com/a/32503192/101923
  request.setHeaders(new HttpHeaders().set("X-HTTP-Method-Override", "PATCH"));
  HttpResponse response = request.execute();

  // Check for errors.
  if (response.getStatusCode() != 200) {
    throw new RuntimeException(response.getStatusMessage());
  }

  Dataset responseDataset = response.parseAs(Dataset.class);
  System.out.printf(
      "Updated label \"%s\" with value \"%s\"\n",
      labelKey, responseDataset.getLabels().get(labelKey));
}

Python

Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Python API reference documentation .

# from google.cloud import bigquery
# client = bigquery.Client()
# dataset_ref = client.dataset('my_dataset')
# dataset = client.get_dataset(dataset_ref)  # API request

assert dataset.labels == {}
labels = {'color': 'green'}
dataset.labels = labels

dataset = client.update_dataset(dataset, ['labels'])  # API request

assert dataset.labels == labels

Adding table and view labels

A label can be added to a table or view when it is created by using the command-line tool's bq mk command or by calling the tables.insert API method.

This page discusses how to add a label to an existing table or view. For more information on adding a label when you create a table or view, see Creating a table or Creating a view.

A label can be added after a table or view is created by using the BigQuery web UI, the command-line tool's bq update command, or by calling the tables.patch API method. Because views are treated like table resources, you use the tables.patch method to modify both views and tables.

Required permissions

To add a label to an existing table or view, you must have OWNER access at the dataset level, or you must be assigned a project-level IAM role that includes bigquery.tables.update permissions. The following predefined, project-level IAM roles include bigquery.tables.update permissions:

In addition, because the bigquery.user role has bigquery.datasets.create permissions, a user assigned to the bigquery.user role can update tables and views in any dataset that user creates. When a user assigned to the bigquery.user role creates a dataset, that user is given OWNER access to the dataset. OWNER access to a dataset gives the user full control over it and the tables and views in it.

For more information on IAM roles and permissions in BigQuery, see Access Control. For more information on dataset-level roles, see Primitive roles for datasets.

Adding a label to a table or view

To add a label to an existing table or view:

Web UI

  1. In the web UI, select the table, or view.

  2. On the details page, to the right of Labels, click Edit. This example shows the details for a table.

    Edit labels

  3. In the Edit Labels dialog:

    • Enter your key and value to add a label. To apply additional labels, click Add Label. Each key can be used only once per table or view, but you can use the same key in tables or views in different datasets.
    • Click OK.

      New label

Command-line

To add a label to an existing table or view, issue the bq update command with the set_label flag. Repeat the flag to add multiple labels.

If the table or view is in a project other than your default project, add the project ID to the dataset in the following format: [PROJECT_ID]:[DATASET].

    bq update --set_label [KEY:VALUE] [PROJECT_ID]:[DATASET].[TABLE_OR_VIEW]

Where:

  • [KEY:VALUE] corresponds to a key:value pair for a label that you want to add. The key must be unique.
  • [PROJECT_ID] is your project ID.
  • [DATASET] is the dataset that contains the table or view you're labeling.
  • [TABLE_OR_VIEW] is the name of the table or view you're labeling.

Examples:

To add a table label that tracks departments, enter the bq update command and specify department as the label key. For example, to add a department:shipping label to mytable in your default project, enter:

    bq update --set_label department:shipping mydataset.mytable

To add a view label that tracks departments, enter the bq update command and specify department as the label key. For example, to add a department:shipping label to myview in your default project, enter:

    bq update --set_label department:shipping mydataset.myview

To add multiple labels to a table or view, repeat the set_label flag and specify a unique key for each label. For example, to add a department:shipping label and cost_center:logistics label to mytable in your default project, enter:

    bq update --set_label department:shipping --set_label cost_center:logistics mydataset.mytable

API

To add a label to an existing table or view, call the tables.patch method and populate the labels property for the table resource.

Because views are treated like table resources, you use the tables.patch method to modify both views and tables.

Because the tables.update method replaces the entire dataset resource, the tables.patch method is preferred.

Go

Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Go API reference documentation .

// To run this sample, you will need to create (or reuse) a context and
// an instance of the bigquery client.  For example:
// import "cloud.google.com/go/bigquery"
// ctx := context.Background()
// client, err := bigquery.NewClient(ctx, "your-project-id")
tbl := client.Dataset(datasetID).Table(tableID)
meta, err := tbl.Metadata(ctx)
if err != nil {
	return err
}

update := bigquery.TableMetadataToUpdate{}
update.SetLabel("color", "green")
if _, err := tbl.Update(ctx, update, meta.ETag); err != nil {
	return err
}

Java

This sample uses the Google HTTP Client Library for Java to send a request to the BigQuery API.
public static class Table {
  @Key private Map<String, String> labels;

  public Map<String, String> getLabels() {
    return this.labels;
  }

  public Table addLabel(String key, String value) {
    if (this.labels == null) {
      this.labels = new HashMap<>();
    }
    this.labels.put(key, value);
    return this;
  }
}

/**
 * Add or modify a label on a table.
 *
 * <p>See <a href="https://cloud.google.com/bigquery/docs/labeling-datasets">the BigQuery
 * documentation</a>.
 */
public static void labelTable(
    String projectId,
    String datasetId,
    String tableId,
    String labelKey,
    String labelValue)
    throws IOException {

  // Authenticate requests using Google Application Default credentials.
  GoogleCredential credential = GoogleCredential.getApplicationDefault();
  credential = credential.createScoped(Arrays.asList("https://www.googleapis.com/auth/bigquery"));

  // Get a new access token.
  // Note that access tokens have an expiration. You can reuse a token rather than requesting a
  // new one if it is not yet expired.
  credential.refreshToken();
  String accessToken = credential.getAccessToken();

  // Set the content of the request.
  Table table = new Table();
  table.addLabel(labelKey, labelValue);
  HttpContent content = new JsonHttpContent(JSON_FACTORY, table);

  // Send the request to the BigQuery API.
  String urlFormat =
      "https://www.googleapis.com/bigquery/v2/projects/%s/datasets/%s/tables/%s"
          + "?fields=labels&access_token=%s";
  GenericUrl url =
      new GenericUrl(String.format(urlFormat, projectId, datasetId, tableId, accessToken));
  HttpRequestFactory requestFactory = HTTP_TRANSPORT.createRequestFactory();
  HttpRequest request = requestFactory.buildPostRequest(url, content);
  request.setParser(JSON_FACTORY.createJsonObjectParser());

  // Workaround for transports which do not support PATCH requests.
  // See: http://stackoverflow.com/a/32503192/101923
  request.setHeaders(new HttpHeaders().set("X-HTTP-Method-Override", "PATCH"));
  HttpResponse response = request.execute();

  // Check for errors.
  if (response.getStatusCode() != 200) {
    throw new RuntimeException(response.getStatusMessage());
  }

  Table responseTable = response.parseAs(Table.class);
  System.out.printf(
      "Updated label \"%s\" with value \"%s\"\n",
      labelKey, responseTable.getLabels().get(labelKey));
}

Python

Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Python API reference documentation .

# from google.cloud import bigquery
# client = bigquery.Client()
# table_ref = client.dataset('my_dataset').table('my_table')
# table = client.get_table(table_ref)  # API request

assert table.labels == {}
labels = {'color': 'green'}
table.labels = labels

table = client.update_table(table, ['labels'])  # API request

assert table.labels == labels

Adding job labels

Labels can be added to query jobs via the command line by using the command-line tool's --label flag. The command-line tool supports adding labels only to query jobs.

You can also add a label to a job when it's submitted via the API by specifying the labels property in the job resource when you call the jobs.insert method. The API can be used to add labels to any job type.

You cannot add labels to or update labels on pending, running, or completed jobs.

When you add a label to a job, the label is included in your billing data.

Required permissions

No special permissions are required for adding a label to a job. If you have jobs.create permissions, you can add a label to your job when you submit it.

To run a job, you must have bigquery.jobs.create permissions. bigquery.jobs.create permissions are required for jobs that are automatically created by BigQuery, and they are required for jobs that you run programmatically.

To run BigQuery jobs, grant bigquery.jobs.create permissions to your user or service account, or grant your account a project-level predefined IAM role that includes bigquery.jobs.create permissions. The following predefined IAM roles include bigquery.jobs.create permissions:

For more information on IAM roles and permissions in BigQuery, see access control.

Adding a label to a job

To add a label to a job:

Web UI

Adding labels to jobs is not supported by the BigQuery web UI.

Command-line

To add a label to a query job, issue the bq query command with the --label flag. Repeat the flag to add multiple labels. The --nouse_legacy_sql flag indicates that your query is in standard SQL syntax.

bq query --label [KEY:VALUE] --nouse_legacy_sql '[QUERY]'

Where:

  • [KEY:VALUE] corresponds to a key:value pair for a label that you want to add to the query job. The key must be unique. To add multiple labels to a query job, repeat the --label flag and specify a unique key for each label.
  • [QUERY] is a valid standard SQL query.

Examples:

To add a label to a query job, enter:

    bq query --label department:shipping --nouse_legacy_sql 'SELECT column1, column2 FROM `mydataset.mytable`'

To add multiple labels to a query job, repeat the --label flag and specify a unique key for each label. For example, to add a department:shipping label and cost_center:logistics label to a query job, enter:

    bq query --label department:shipping --label cost_center:logistics --nouse_legacy_sql 'SELECT column1, column2 FROM `mydataset.mytable`'

API

To add a label to a job, call the jobs.insert method and populate the labels property for the job resource. You can use the API to add labels to any job type.

Go

Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Go API reference documentation .

// To run this sample, you will need to create (or reuse) a context and
// an instance of the bigquery client.  For example:
// import "cloud.google.com/go/bigquery"
// ctx := context.Background()
// client, err := bigquery.NewClient(ctx, "your-project-id")
tbl := client.Dataset(datasetID).Table(tableID)
meta, err := tbl.Metadata(ctx)
if err != nil {
	return err
}

update := bigquery.TableMetadataToUpdate{}
update.SetLabel("color", "green")
if _, err := tbl.Update(ctx, update, meta.ETag); err != nil {
	return err
}

Creating a tag

A label that has a key with an empty value is used as a tag. You can create a new label with no value, or you can turn an existing label into a tag.

Tags can be useful in situations where you are labeling a resource, but you do not need the key:value format. For example, if you have a table that contains test data that is used by multiple groups (support, development, and so on), you can add a test_data tag to the table to identify it.

To create a tag:

Web UI

  1. In the web UI, select the appropriate resource (a dataset, table, or view).

  2. For datasets, the Dataset Details page is automatically opened. For tables and views, click Details to open the details page.

  3. On the details page, to the right of Labels, click Edit.

  4. In the Edit Labels dialog:

    • Enter a new key and leave the value blank. To apply additional tags, click Add Label.
    • Click OK.

      Add tag

Command-line

To add a tag to an existing resource, use the bq update command with the set_label flag. Specify the key, followed by a colon, but leave the value unspecified.

bq update --set_label [KEY]: [RESOURCE_ID]

Where:

  • [KEY] is the label key that you want to use as a tag.
  • [RESOURCE_ID] is a valid dataset, table, or view name. If the resource is in a project other than your default project, add the project ID in the following format: [PROJECT_ID]:[DATASET].

Examples:

Enter the following command to create a test_data tag for mydataset.mytable. mydataset is in your default project.

bq update --set_label test_data: mydataset

API

Call the datasets.patch method or the tables.patch method and add labels with the value set to the empty string ("") in the dataset resource or the table resource. You can turn existing labels into tags by replacing their values with the empty string.

Because views are treated like table resources, you use the tables.patch method to modify both views and tables. Also, because the tables.update method replaces the entire dataset resource, the tables.patch method is preferred.

Viewing labels

You can view labels by using the BigQuery web UI, the command-line tool's bq show command, or by calling the datasets.get or tables.get API methods. Because views are treated like table resources, you use the tables.get method to get label information for both views and tables.

Required permissions

The permissions required for viewing labels depends on the type of resource you access.

Dataset permissions

To get information about a dataset, you must be assigned the dataset-level READER role, or you must be assigned a project-level IAM role that includes bigquery.datasets.get permissions. All predefined, project-level IAM roles include bigquery.datasets.get permissions except for bigquery.jobUser.

In addition, a user assigned the bigquery.user role has bigquery.datasets.create permissions. This allows a user assigned to the bigquery.user role to get information about any dataset that user creates. When a user assigned to the bigquery.user role creates a dataset, that user is given OWNER access to the dataset. OWNER access to a dataset gives the user full control over it.

For more information on IAM roles and permissions in BigQuery, see Access Control. For more information on dataset-level roles, see Primitive roles for datasets.

Table and view permissions

To get information about tables, you must be assigned the READER role on the dataset, or you must be assigned a predefined, project-level IAM role that includes bigquery.tables.get permissions. If you are granted bigquery.tables.get permissions at the project level, you can get information about all tables in the project. All predefined, project-level IAM roles include bigquery.tables.get permissions except for bigquery.user and bigquery.jobUser.

In addition, a user assigned the bigquery.user role has bigquery.datasets.create permissions. This allows a user assigned to the bigquery.user role to get information about tables and views in any dataset that user creates. When a user assigned to the bigquery.user role creates a dataset, that user is given OWNER access to the dataset. OWNER access to a dataset gives the user full control over it and all the tables and views in it.

For more information on IAM roles and permissions in BigQuery, see Access Control. For more information on dataset-level roles, see Primitive roles for datasets.

Job permissions

In order to get job data and metadata, you must have bigquery.jobs.get permissions. The following project-level, predefined IAM role includes bigquery.jobs.get permissions:

If you grant an account the bigquery.admin role, the user can view all job data in the project no matter who submitted the job.

The following roles are granted bigquery.jobs.get permissions for self-created jobs. These users can only view job data for jobs they submit:

For more information on IAM roles and permissions in BigQuery, see access control.

Viewing dataset, table, and view labels

To view a resource's labels:

Web UI

  1. In the web UI, select the appropriate resource (a dataset, table, or view).

  2. For datasets, the Dataset Details page is automatically opened. For tables and views, click Details to open the details page. Label information appears in the information table for the resource.

Command-line

Use the bq show command with the resource ID. The --format flag can be used to control the output. If the resource is in a project other than your default project, add the project ID in the following format: [PROJECT_ID]:[DATASET]. For readability, the output is controlled by setting the --format flag to pretty.

bq show --format=pretty [RESOURCE_ID]

Where [RESOURCE_ID] is a valid dataset, table, or view name.

Examples:

Enter the following command to display labels for mydataset in your default project.

bq show --format=pretty mydataset

Enter the following command to display labels for mydataset.mytable. mydataset is in myotherproject, not your default project.

bq show --format=pretty myotherproject:mydataset.mytable

API

Call the datasets.get method or the tables.get method. The response includes all labels associated with that resource.

Alternatively, you can use datasets.list to view the labels for multiple datasets or tables.list to view the labels for multiple tables and views.

Because views are treated like table resources, you use the tables.get and tables.list methods to view label information for both views and tables.

Go

This sample uses the Google HTTP Client Library for Java to send a request to the BigQuery API.
// To run this sample, you will need to create (or reuse) a context and
// an instance of the bigquery client.  For example:
// import "cloud.google.com/go/bigquery"
// ctx := context.Background()
// client, err := bigquery.NewClient(ctx, "your-project-id")
meta, err := client.Dataset(datasetID).Metadata(ctx)
if err != nil {
	return err
}
fmt.Fprintf(w, "Dataset %s labels:\n", datasetID)
if len(meta.Labels) == 0 {
	fmt.Fprintln(w, "Dataset has no labels defined.")
	return nil
}
for k, v := range meta.Labels {
	fmt.Fprintf(w, "\t%s:%s\n", k, v)
}

Python

Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Python API reference documentation .

# from google.cloud import bigquery
# client = bigquery.Client()
# dataset_id = 'my_dataset'

dataset_ref = client.dataset(dataset_id)
dataset = client.get_dataset(dataset_ref)  # API request

# View dataset labels
print('Dataset ID: {}'.format(dataset_id))
print('Labels:')
if dataset.labels:
    for label, value in dataset.labels.items():
        print('\t{}: {}'.format(label, value))
else:
    print("\tDataset has no labels defined.")

Viewing job labels

After the query job is submitted, the job labels do not appear in the BigQuery web UI. To see the labels on a job, issue the bq show -j [JOB_ID] command.

Web UI

You cannot view job labels by using the BigQuery web UI.

CLI

To see the labels for a query job using the bq command-line tool, enter the bq show -j command with the query job's job ID. The --format flag can be used to control the output. For example, if your query job has job ID bqjob_r1234d57f78901_000023746d4q12_1, enter the following command:

bq show -j --format=pretty bqjob_r1234d57f78901_000023746d4q12_1

The output should look like the following:

+----------+---------+-----------------+----------+-------------------+-----------------+--------------+----------------------+
| Job Type |  State  |   Start Time    | Duration |    User Email     | Bytes Processed | Bytes Billed |        Labels        |
+----------+---------+-----------------+----------+-------------------+-----------------+--------------+----------------------+
| query    | SUCCESS | 03 Dec 15:00:41 | 0:00:00  | email@example.com | 255             | 10485760     | department:shipping  |
|          |         |                 |          |                   |                 |              | costcenter:logistics |
+----------+---------+-----------------+----------+-------------------+-----------------+--------------+----------------------+

API

Call the jobs.get method. The response includes all labels associated with that resource.

Filtering datasets using labels

To filter datasets based on labels, create a filter specification for use in the bq command-line tool or the BigQuery API that uses the following syntax:

    "field[:value][ field[:value]..."

Where:

  • field is expressed as labels.[KEY] where [KEY] is a label key.
  • value is an optional label value.

Currently, you cannot list tables or views based on a filter specification.

Limitations

The filter specification has the following limitations:

  • Only the AND logical operator is supported. Space-separated comparisons are treated as having implicit AND operators.
  • The only field currently eligible for filtering is "labels.key" where "key" is the name of a label.
  • The filter can include up to ten expressions.
  • Filtering is case-sensitive.
  • You cannot filter tables and views using a filter specification.
  • Currently, you cannot filter datasets using the BigQuery web UI.

Filter specification examples

To list datasets that have a department:shipping label, use the following filter specification:

labels.department:shipping

To list datasets using multiple labels, separate the key:value pairs with a space. The space is treated as a logical AND operator. For example, to list datasets with the department:shipping label and the location:usa label, use the following filter specification:

labels.department:shipping labels.location:usa

You can filter on the presence of a key alone, rather than matching against a key:value pair. The following filter specification lists all datasets labeled department regardless of the value:

labels.department

An equivalent filter specification uses an asterisk to represent all possible values associated with the department key:

labels.department:*

You can also use tags in a filter specification. For example, to list datasets with the department:shipping label and test_data tag, use the following filter specification:

labels.department:shipping labels.test_data

Generating filtered lists

To generate a filtered list of datasets:

Web UI

Currently, you cannot filter datasets using the BigQuery web UI.

Command-line

Issue the bq ls command with the --filter flag. If you are listing datasets in a project other than your default project, specify the --project_id flag.

bq ls --filter "[FILTER_SPECIFICATION]" --project_id [PROJECT_ID]

Where:

  • [FILTER_SPECIFICATION] is a valid filter specification. The command-line tool returns a list of datasets that meet the filter requirements.
  • [PROJECT_ID] is your project ID.

Examples:

Enter the following command to list datasets in your default project that have a department:shipping label:

bq ls --filter "labels.department:shipping"

Enter the following command to list datasets in your default project that have a department:shipping label and a test_data tag.

bq ls --filter "labels.department:shipping labels.test_data"

Enter the following command to list datasets in myotherproject that have a department:shipping label:

bq ls --filter "labels.department:shipping" --project_id myotherproject

API

Call the datasets.list API method and provide the filter specification using the filter property.

Go

Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Go API reference documentation .

// To run this sample, you will need to create (or reuse) a context and
// an instance of the bigquery client.  For example:
// import "cloud.google.com/go/bigquery"
// ctx := context.Background()
// client, err := bigquery.NewClient(ctx, "your-project-id")
it := client.Datasets(ctx)
it.Filter = "labels.color:green"
for {
	dataset, err := it.Next()
	if err == iterator.Done {
		break
	}
	if err != nil {
		return err
	}
	fmt.Fprintf(w, "dataset: %s\n", dataset.DatasetID)
}

Python

Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries . For more information, see the BigQuery Python API reference documentation .

# from google.cloud import bigquery
# client = bigquery.Client()

# The following label filter example will find datasets with an
# arbitrary 'color' label set to 'green'
label_filter = 'labels.color:green'
datasets = list(client.list_datasets(filter=label_filter))

if datasets:
    print('Datasets filtered by {}:'.format(label_filter))
    for dataset in datasets:  # API request(s)
        print('\t{}'.format(dataset.dataset_id))
else:
    print('No datasets found with this filter.')

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Need help? Visit our support page.