Data Catalog client libraries

This page shows how to get started with the Cloud Client Libraries for the Data Catalog API. Client libraries make it easier to access Google Cloud APIs from a supported language. Although you can use Google Cloud APIs directly by making raw requests to the server, client libraries provide simplifications that significantly reduce the amount of code you need to write.

Read more about the Cloud Client Libraries and the older Google API Client Libraries in Client libraries explained.

Install the client library

C#

Install-Package Google.Cloud.DataCatalog.V1

For more information, see Setting Up a C# Development Environment.

Go

go get cloud.google.com/go/datacatalog/apiv1

For more information, see Setting Up a Go Development Environment.

Java

If you are using Maven, add this to your pom.xml file:

Maven:
<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-datacatalog</artifactId>
    <version>DATA_CATALOG_LIBRARY_VERSION</version>
</dependency>

If you are using Gradle, add this to your dependencies:

compile group: 'com.google.cloud', name: 'google-cloud-datacatalog', version: 'DATA_CATALOG_LIBRARY_VERSION'

See Data Catalog Java Cloud Client Library reference for the latest Data Catalog version.

For more information, see Setting Up a Java Development Environment.

Node.js

npm install --save @google-cloud/datacatalog

For more information, see Setting Up a Node.js Development Environment.

PHP

composer require google/cloud-data-catalog

For more information, see Using PHP on Google Cloud.

Python

pip install --upgrade google-cloud-datacatalog

For more information, see Setting Up a Python Development Environment.

Ruby

gem install google-cloud-data_catalog

For more information, see Setting Up a Ruby Development Environment.

Set up authentication

To authenticate calls to Google Cloud APIs, client libraries support Application Default Credentials (ADC); the libraries look for credentials in a set of defined locations and use those credentials to authenticate requests to the API. With ADC, you can make credentials available to your application in a variety of environments, such as local development or production, without needing to modify your application code.

For production environments, the way you set up ADC depends on the service and context. For more information, see Set up Application Default Credentials.

For a local development environment, you can set up ADC with the credentials that are associated with your Google Account:

1. Install the Google Cloud CLI, then initialize it by running the following command:

   gcloud init

2. If you're using a local shell, then create local authentication credentials for your user account:

   gcloud auth application-default login

   You don't need to do this if you're using Cloud Shell.

   A sign-in screen appears. After you sign in, your credentials are stored in the local credential file used by ADC.

Use the client library

The following example shows how to use the client library.

Java

import com.google.cloud.datacatalog.v1.DataCatalogClient;
import com.google.cloud.datacatalog.v1.Entry;
import com.google.cloud.datacatalog.v1.LookupEntryRequest;

public class LookupEntryBigQueryDataset {

  /**
   * Lookup the Data Catalog entry referring to a BigQuery Dataset
   *
   * @param projectId The project ID to which the Dataset belongs, e.g. 'my-project'
   * @param datasetId The dataset ID to which the Catalog Entry refers, e.g. 'my_dataset'
   */
  public static void lookupEntry(String projectId, String datasetId) {
    // String projectId = "my-project"
    // String datasetId = "my_dataset"

    // Get an entry by the resource name from the source Google Cloud Platform service.
    String linkedResource =
        String.format("//bigquery.googleapis.com/projects/%s/datasets/%s", projectId, datasetId);
    LookupEntryRequest request =
        LookupEntryRequest.newBuilder().setLinkedResource(linkedResource).build();

    // Alternatively, lookup by the SQL name of the entry would have the same result:
    // String sqlResource = String.format("bigquery.dataset.`%s`.`%s`", projectId, datasetId);
    // LookupEntryRequest request =
    // LookupEntryRequest.newBuilder().setSqlResource(sqlResource).build();

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DataCatalogClient dataCatalogClient = DataCatalogClient.create()) {
      Entry entry = dataCatalogClient.lookupEntry(request);
      System.out.printf("Entry name: %s\n", entry.getName());
    } catch (Exception e) {
      System.out.print("Error during lookupEntryBigQueryDataset:\n" + e.toString());
    }
  }
}

Node.js

// -------------------------------
// Import required modules.
// -------------------------------
const {DataCatalogClient} = require('@google-cloud/datacatalog').v1;
const datacatalog = new DataCatalogClient();

const lookup = async () => {
  // TODO(developer): Uncomment the following lines before running the sample.
  // const projectId = 'my-project'
  // const datasetId = 'my_dataset'
  const resourceName = `//bigquery.googleapis.com/projects/${projectId}/datasets/${datasetId}`;
  const request = {linkedResource: resourceName};
  const [result] = await datacatalog.lookupEntry(request);
  return result;
};

const response = await lookup();
console.log(response);

Python

from google.cloud import datacatalog_v1

datacatalog = datacatalog_v1.DataCatalogClient()

bigquery_project_id = "my_bigquery_project"
dataset_id = "my_dataset"
table_id = "my_table"
pubsub_project_id = "my_pubsub_project"
topic_id = "my_topic"


# To facilitate testing, we replace values with alternatives
# provided by the testing harness.
bigquery_project_id = override_values.get(
    "bigquery_project_id", bigquery_project_id
)
dataset_id = override_values.get("dataset_id", dataset_id)
table_id = override_values.get("table_id", table_id)
pubsub_project_id = override_values.get("pubsub_project_id", pubsub_project_id)
topic_id = override_values.get("topic_id", topic_id)

# BigQuery Dataset via linked_resource
resource_name = f"//bigquery.googleapis.com/projects/{bigquery_project_id}/datasets/{dataset_id}"

entry = datacatalog.lookup_entry(request={"linked_resource": resource_name})
print(
    f"Retrieved entry {entry.name} for BigQuery Dataset resource {entry.linked_resource}"
)

# BigQuery Dataset via sql_resource
sql_resource = f"bigquery.dataset.`{bigquery_project_id}`.`{dataset_id}`"

entry = datacatalog.lookup_entry(request={"sql_resource": sql_resource})
print(
    f"Retrieved entry {entry.name} for BigQuery Dataset resource {entry.linked_resource}"
)

# BigQuery Table via linked_resource
resource_name = (
    f"//bigquery.googleapis.com/projects/{bigquery_project_id}/datasets/{dataset_id}"
    f"/tables/{table_id}"
)

entry = datacatalog.lookup_entry(request={"linked_resource": resource_name})
print(f"Retrieved entry {entry.name} for BigQuery Table {entry.linked_resource}")

# BigQuery Table via sql_resource
sql_resource = f"bigquery.table.`{bigquery_project_id}`.`{dataset_id}`.`{table_id}`"

entry = datacatalog.lookup_entry(request={"sql_resource": sql_resource})
print(
    f"Retrieved entry {entry.name} for BigQuery Table resource {entry.linked_resource}"
)

# Pub/Sub Topic via linked_resource
resource_name = (
    f"//pubsub.googleapis.com/projects/{pubsub_project_id}/topics/{topic_id}"
)

entry = datacatalog.lookup_entry(request={"linked_resource": resource_name})
print(
    f"Retrieved entry {entry.name} for Pub/Sub Topic resource {entry.linked_resource}"
)

# Pub/Sub Topic via sql_resource
sql_resource = f"pubsub.topic.`{pubsub_project_id}`.`{topic_id}`"

entry = datacatalog.lookup_entry(request={"sql_resource": sql_resource})
print(
    f"Retrieved entry {entry.name} for Pub/Sub Topic resource {entry.linked_resource}"
)

Additional resources

C#

The following list contains links to more resources related to the client library for C#:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

Go

The following list contains links to more resources related to the client library for Go:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

Java

The following list contains links to more resources related to the client library for Java:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

Node.js

The following list contains links to more resources related to the client library for Node.js:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

PHP

The following list contains links to more resources related to the client library for PHP:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

Python

The following list contains links to more resources related to the client library for Python:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code

Ruby

The following list contains links to more resources related to the client library for Ruby:

• API reference
• Client libraries best practices
• Issue tracker
• google-data-catalog on Stack Overflow
• Source code