Package google.cloud.datacatalog.v1

Index

DataCatalog

Data Catalog API service allows you to discover, understand, and manage your data.

CreateEntry

rpc CreateEntry(CreateEntryRequest) returns (Entry)

Creates an entry.

You can create entries only with 'FILESET', 'CLUSTER', 'DATA_STREAM', or custom types. Data Catalog automatically creates entries with other types during metadata ingestion from integrated systems.

You must enable the Data Catalog API in the project identified by the parent parameter. For more information, see Data Catalog resource project.

An entry group can have a maximum of 100,000 entries.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateEntryGroup

rpc CreateEntryGroup(CreateEntryGroupRequest) returns (EntryGroup)

Creates an entry group.

An entry group contains logically related entries together with Cloud Identity and Access Management policies. These policies specify users who can create, edit, and view entries within entry groups.

Data Catalog automatically creates entry groups with names that start with the @ symbol for the following resources:

  • BigQuery entries (@bigquery)
  • Pub/Sub topics (@pubsub)
  • Dataproc Metastore services (@dataproc_metastore_{SERVICE_NAME_HASH})

You can create your own entry groups for Cloud Storage fileset entries and custom entries together with the corresponding IAM policies. User-created entry groups can't contain the @ symbol, it is reserved for automatically created groups.

Entry groups, like entries, can be searched.

A maximum of 10,000 entry groups may be created per organization across all locations.

You must enable the Data Catalog API in the project identified by the parent parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTag

rpc CreateTag(CreateTagRequest) returns (Tag)

Creates a tag and assigns it to:

  • An Entry if the method name is projects.locations.entryGroups.entries.tags.create.
  • Or EntryGroupif the method name is projects.locations.entryGroups.tags.create.

Note: The project identified by the parent parameter for the tag and the tag template used to create the tag must be in the same organization.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTagTemplate

rpc CreateTagTemplate(CreateTagTemplateRequest) returns (TagTemplate)

Creates a tag template.

You must enable the Data Catalog API in the project identified by the parent parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTagTemplateField

rpc CreateTagTemplateField(CreateTagTemplateFieldRequest) returns (TagTemplateField)

Creates a field in a tag template.

You must enable the Data Catalog API in the project identified by the parent parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteEntry

rpc DeleteEntry(DeleteEntryRequest) returns (Empty)

Deletes an existing entry.

You can delete only the entries created by the CreateEntry method.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteEntryGroup

rpc DeleteEntryGroup(DeleteEntryGroupRequest) returns (Empty)

Deletes an entry group.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTag

rpc DeleteTag(DeleteTagRequest) returns (Empty)

Deletes a tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTagTemplate

rpc DeleteTagTemplate(DeleteTagTemplateRequest) returns (Empty)

Deletes a tag template and all tags that use it.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTagTemplateField

rpc DeleteTagTemplateField(DeleteTagTemplateFieldRequest) returns (Empty)

Deletes a field in a tag template and all uses of this field from the tags based on this template.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetEntry

rpc GetEntry(GetEntryRequest) returns (Entry)

Gets an entry.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetEntryGroup

rpc GetEntryGroup(GetEntryGroupRequest) returns (EntryGroup)

Gets an entry group.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetIamPolicy

rpc GetIamPolicy(GetIamPolicyRequest) returns (Policy)

Gets the access control policy for a resource.

May return:

  • ANOT_FOUND error if the resource doesn't exist or you don't have the permission to view it.
  • An empty policy if the resource exists but doesn't have a set policy.

Supported resources are:

  • Tag templates
  • Entry groups

Note: This method doesn't get policies from Google Cloud Platform resources ingested into Data Catalog.

To call this method, you must have the following Google IAM permissions:

  • datacatalog.tagTemplates.getIamPolicy to get policies on tag templates.
  • datacatalog.entryGroups.getIamPolicy to get policies on entry groups.
Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetTagTemplate

rpc GetTagTemplate(GetTagTemplateRequest) returns (TagTemplate)

Gets a tag template.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListEntries

rpc ListEntries(ListEntriesRequest) returns (ListEntriesResponse)

Lists entries.

Note: Currently, this method can list only custom entries. To get a list of both custom and automatically created entries, use SearchCatalog.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListEntryGroups

rpc ListEntryGroups(ListEntryGroupsRequest) returns (ListEntryGroupsResponse)

Lists entry groups.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListTags

rpc ListTags(ListTagsRequest) returns (ListTagsResponse)

Lists tags assigned to an Entry. The columns in the response are lowercased.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
LookupEntry

rpc LookupEntry(LookupEntryRequest) returns (Entry)

Gets an entry by its target resource name.

The resource name comes from the source Google Cloud Platform service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ModifyEntryContacts

rpc ModifyEntryContacts(ModifyEntryContactsRequest) returns (Contacts)

Modifies contacts, part of the business context of an Entry.

To call this method, you must have the datacatalog.entries.updateContacts IAM permission on the corresponding project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ModifyEntryOverview

rpc ModifyEntryOverview(ModifyEntryOverviewRequest) returns (EntryOverview)

Modifies entry overview, part of the business context of an Entry.

To call this method, you must have the datacatalog.entries.updateOverview IAM permission on the corresponding project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
RenameTagTemplateField

rpc RenameTagTemplateField(RenameTagTemplateFieldRequest) returns (TagTemplateField)

Renames a field in a tag template.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
RenameTagTemplateFieldEnumValue

rpc RenameTagTemplateFieldEnumValue(RenameTagTemplateFieldEnumValueRequest) returns (TagTemplateField)

Renames an enum value in a tag template.

Within a single enum field, enum values must be unique.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
SearchCatalog

rpc SearchCatalog(SearchCatalogRequest) returns (SearchCatalogResponse)

Searches Data Catalog for multiple resources like entries and tags that match a query.

This is a Custom Method that doesn't return all information on a resource, only its ID and high level fields. To get more information, you can subsequently call specific get methods.

Note: Data Catalog search queries don't guarantee full recall. Results that match your query might not be returned, even in subsequent result pages. Additionally, returned (and not returned) results can vary if you repeat search queries.

For more information, see Data Catalog search syntax.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
SetIamPolicy

rpc SetIamPolicy(SetIamPolicyRequest) returns (Policy)

Sets an access control policy for a resource. Replaces any existing policy.

Supported resources are:

  • Tag templates
  • Entry groups

Note: This method sets policies only within Data Catalog and can't be used to manage policies in BigQuery, Pub/Sub, Dataproc Metastore, and any external Google Cloud Platform resources synced with the Data Catalog.

To call this method, you must have the following Google IAM permissions:

  • datacatalog.tagTemplates.setIamPolicy to set policies on tag templates.
  • datacatalog.entryGroups.setIamPolicy to set policies on entry groups.
Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
StarEntry

rpc StarEntry(StarEntryRequest) returns (StarEntryResponse)

Marks an Entry as starred by the current user. Starring information is private to each user.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
TestIamPermissions

rpc TestIamPermissions(TestIamPermissionsRequest) returns (TestIamPermissionsResponse)

Gets your permissions on a resource.

Returns an empty set of permissions if the resource doesn't exist.

Supported resources are:

  • Tag templates
  • Entry groups

Note: This method gets policies only within Data Catalog and can't be used to get policies from BigQuery, Pub/Sub, Dataproc Metastore, and any external Google Cloud Platform resources ingested into Data Catalog.

No Google IAM permissions are required to call this method.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UnstarEntry

rpc UnstarEntry(UnstarEntryRequest) returns (UnstarEntryResponse)

Marks an Entry as NOT starred by the current user. Starring information is private to each user.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateEntry

rpc UpdateEntry(UpdateEntryRequest) returns (Entry)

Updates an existing entry.

You must enable the Data Catalog API in the project identified by the entry.name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateEntryGroup

rpc UpdateEntryGroup(UpdateEntryGroupRequest) returns (EntryGroup)

Updates an entry group.

You must enable the Data Catalog API in the project identified by the entry_group.name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTag

rpc UpdateTag(UpdateTagRequest) returns (Tag)

Updates an existing tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTagTemplate

rpc UpdateTagTemplate(UpdateTagTemplateRequest) returns (TagTemplate)

Updates a tag template.

You can't update template fields with this method. These fields are separate resources with their own create, update, and delete methods.

You must enable the Data Catalog API in the project identified by the tag_template.name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTagTemplateField

rpc UpdateTagTemplateField(UpdateTagTemplateFieldRequest) returns (TagTemplateField)

Updates a field in a tag template.

You can't update the field type with this method.

You must enable the Data Catalog API in the project identified by the name parameter. For more information, see Data Catalog resource project.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

PolicyTagManager

Policy Tag Manager API service allows you to manage your policy tags and taxonomies.

Policy tags are used to tag BigQuery columns and apply additional access control policies. A taxonomy is a hierarchical grouping of policy tags that classify data along a common axis.

CreatePolicyTag

rpc CreatePolicyTag(CreatePolicyTagRequest) returns (PolicyTag)

Creates a policy tag in a taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTaxonomy

rpc CreateTaxonomy(CreateTaxonomyRequest) returns (Taxonomy)

Creates a taxonomy in a specified project.

The taxonomy is initially empty, that is, it doesn't contain policy tags.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeletePolicyTag

rpc DeletePolicyTag(DeletePolicyTagRequest) returns (Empty)

Deletes a policy tag together with the following:

  • All of its descendant policy tags, if any
  • Policies associated with the policy tag and its descendants
  • References from BigQuery table schema of the policy tag and its descendants
Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTaxonomy

rpc DeleteTaxonomy(DeleteTaxonomyRequest) returns (Empty)

Deletes a taxonomy, including all policy tags in this taxonomy, their associated policies, and the policy tags references from BigQuery columns.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetIamPolicy

rpc GetIamPolicy(GetIamPolicyRequest) returns (Policy)

Gets the IAM policy for a policy tag or a taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetPolicyTag

rpc GetPolicyTag(GetPolicyTagRequest) returns (PolicyTag)

Gets a policy tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetTaxonomy

rpc GetTaxonomy(GetTaxonomyRequest) returns (Taxonomy)

Gets a taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListPolicyTags

rpc ListPolicyTags(ListPolicyTagsRequest) returns (ListPolicyTagsResponse)

Lists all policy tags in a taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListTaxonomies

rpc ListTaxonomies(ListTaxonomiesRequest) returns (ListTaxonomiesResponse)

Lists all taxonomies in a project in a particular location that you have a permission to view.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
SetIamPolicy

rpc SetIamPolicy(SetIamPolicyRequest) returns (Policy)

Sets the IAM policy for a policy tag or a taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
TestIamPermissions

rpc TestIamPermissions(TestIamPermissionsRequest) returns (TestIamPermissionsResponse)

Returns your permissions on a specified policy tag or taxonomy.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdatePolicyTag

rpc UpdatePolicyTag(UpdatePolicyTagRequest) returns (PolicyTag)

Updates a policy tag, including its display name, description, and parent policy tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTaxonomy

rpc UpdateTaxonomy(UpdateTaxonomyRequest) returns (Taxonomy)

Updates a taxonomy, including its display name, description, and activated policy types.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

PolicyTagManagerSerialization

Policy Tag Manager Serialization API service allows you to manipulate your policy tags and taxonomies in a serialized format.

Taxonomy is a hierarchical group of policy tags.

ExportTaxonomies

rpc ExportTaxonomies(ExportTaxonomiesRequest) returns (ExportTaxonomiesResponse)

Exports taxonomies in the requested type and returns them, including their policy tags. The requested taxonomies must belong to the same project.

This method generates SerializedTaxonomy protocol buffers with nested policy tags that can be used as input for ImportTaxonomies calls.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ImportTaxonomies

rpc ImportTaxonomies(ImportTaxonomiesRequest) returns (ImportTaxonomiesResponse)

Creates new taxonomies (including their policy tags) in a given project by importing from inlined or cross-regional sources.

For a cross-regional source, new taxonomies are created by copying from a source in another region.

For an inlined source, taxonomies and policy tags are created in bulk using nested protocol buffer structures.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ReplaceTaxonomy

rpc ReplaceTaxonomy(ReplaceTaxonomyRequest) returns (Taxonomy)

Replaces (updates) a taxonomy and all its policy tags.

The taxonomy and its entire hierarchy of policy tags must be represented literally by SerializedTaxonomy and the nested SerializedPolicyTag messages.

This operation automatically does the following:

  • Deletes the existing policy tags that are missing from the SerializedPolicyTag.
  • Creates policy tags that don't have resource names. They are considered new.
  • Updates policy tags with valid resources names accordingly.
Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

BigQueryConnectionSpec

Specification for the BigQuery connection.

Fields
connection_type

ConnectionType

The type of the BigQuery connection.
has_credential

bool

True if there are credentials attached to the BigQuery connection; false otherwise.
cloud_sql

CloudSqlBigQueryConnectionSpec

Specification for the BigQuery connection to a Cloud SQL instance.

ConnectionType

The type of the BigQuery connection.

Enums
CONNECTION_TYPE_UNSPECIFIED Unspecified type.
CLOUD_SQL Cloud SQL connection.

BigQueryDateShardedSpec

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.

Fields
dataset

string

Output only. The Data Catalog resource name of the dataset entry the current table belongs to. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.
table_prefix

string

Output only. The table name prefix of the shards.

The name of any given shard is [table_prefix]YYYYMMDD. For example, for the MyTable20180101 shard, the table_prefix is MyTable.
shard_count

int64

Output only. Total number of shards.
latest_shard_resource

string

Output only. BigQuery resource name of the latest shard.

BigQueryRoutineSpec

Fields specific for BigQuery routines.

Fields
imported_libraries[]

string

Paths of the imported libraries.

BigQueryTableSpec

Describes a BigQuery table.

Fields
table_source_type

TableSourceType

Output only. The table source type.
Union field type_spec. Output only. type_spec can be only one of the following:
view_spec

ViewSpec

Table view specification. Populated only if the table_source_type is BIGQUERY_VIEW.
table_spec

TableSpec

Specification of a BigQuery table. Populated only if the table_source_type is BIGQUERY_TABLE.

BusinessContext

Business Context of the entry.

Fields
entry_overview

EntryOverview

Entry overview fields for rich text descriptions of entries.
contacts

Contacts

Contact people for the entry.

CloudSqlBigQueryConnectionSpec

Specification for the BigQuery connection to a Cloud SQL instance.

Fields
instance_id

string

Cloud SQL instance ID in the format of project:location:instance.
database

string

Database name.
type

DatabaseType

Type of the Cloud SQL database.

DatabaseType

Supported Cloud SQL database types.

Enums
DATABASE_TYPE_UNSPECIFIED Unspecified database type.
POSTGRES Cloud SQL for PostgreSQL.
MYSQL Cloud SQL for MySQL.

ColumnSchema

A column within a schema. Columns can be nested inside other columns.

Fields
column

string

Required. Name of the column.

Must be a UTF-8 string without dots (.). The maximum size is 64 bytes.
type

string

Required. Type of the column.

Must be a UTF-8 string with the maximum size of 128 bytes.
description

string

Optional. Description of the column. Default value is an empty string.

The description must be a UTF-8 string with the maximum size of 2000 bytes.
mode

string

Optional. A column's mode indicates whether values in this column are required, nullable, or repeated.

Only NULLABLE, REQUIRED, and REPEATED values are supported. Default mode is NULLABLE.
subcolumns[]

ColumnSchema

Optional. Schema of sub-columns. A column can have zero or more sub-columns.

Contacts

Contact people for the entry.

Fields
people[]

Person

The list of contact people for the entry.

Person

A contact person for the entry.

Fields
designation

string

Designation of the person, for example, Data Steward.
email

string

Email of the person in the format of john.doe@xyz, <john.doe@xyz>, or John Doe<john.doe@xyz>.

CreateEntryGroupRequest

Request message for CreateEntryGroup.

Fields
parent

string

Required. The names of the project and location that the new entry group belongs to.

Note: The entry group itself and its child resources might not be stored in the location specified in its name.
entry_group_id

string

Required. The ID of the entry group to create.

The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and must start with a letter or underscore. The maximum size is 64 bytes when encoded in UTF-8.
entry_group

EntryGroup

The entry group to create. Defaults to empty.

CreateEntryRequest

Request message for CreateEntry.

Fields
parent

string

Required. The name of the entry group this entry belongs to.

Note: The entry itself and its child resources might not be stored in the location specified in its name.
entry_id

string

Required. The ID of the entry to create.

The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores (_). The maximum size is 64 bytes when encoded in UTF-8.
entry

Entry

Required. The entry to create.

CreatePolicyTagRequest

Request message for CreatePolicyTag.

Fields
parent

string

Required. Resource name of the taxonomy that the policy tag will belong to.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.update
policy_tag

PolicyTag

The policy tag to create.

CreateTagRequest

Request message for CreateTag.

Fields
parent

string

Required. The name of the resource to attach this tag to.

Tags can be attached to entries or entry groups. An entry can have up to 1000 attached tags.

Note: The tag and its child resources might not be stored in the location specified in its name.
tag

Tag

Required. The tag to create.

CreateTagTemplateFieldRequest

Request message for CreateTagTemplateField.

Fields
parent

string

Required. The name of the project and the template location region.
tag_template_field_id

string

Required. The ID of the tag template field to create.

Note: Adding a required field to an existing template is not allowed.

Field IDs can contain letters (both uppercase and lowercase), numbers (0-9), underscores (_) and dashes (-). Field IDs must be at least 1 character long and at most 128 characters long. Field IDs must also be unique within their template.
tag_template_field

TagTemplateField

Required. The tag template field to create.

CreateTagTemplateRequest

Request message for CreateTagTemplate.

Fields
parent

string

Required. The name of the project and the template location region.
tag_template_id

string

Required. The ID of the tag template to create.

The ID must contain only lowercase letters (a-z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum size is 64 bytes when encoded in UTF-8.
tag_template

TagTemplate

Required. The tag template to create.

CreateTaxonomyRequest

Request message for CreateTaxonomy.

Fields
parent

string

Required. Resource name of the project that the taxonomy will belong to.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.create
taxonomy

Taxonomy

The taxonomy to create.

CrossRegionalSource

Cross-regional source used to import an existing taxonomy into a different region.

Fields
taxonomy

string

Required. The resource name of the source taxonomy to import.

DataSource

Physical location of an entry.

Fields
service

Service

Service that physically stores the data.
resource

string

Full name of a resource as defined by the service. For example:

//bigquery.googleapis.com/projects/{PROJECT_ID}/locations/{LOCATION}/datasets/{DATASET_ID}/tables/{TABLE_ID}
source_entry

string

Output only. Data Catalog entry name, if applicable.
storage_properties

StorageProperties

Detailed properties of the underlying storage.

Service

Name of a service that stores the data.

Enums
SERVICE_UNSPECIFIED Default unknown service.
CLOUD_STORAGE Google Cloud Storage service.
BIGQUERY BigQuery service.

DataSourceConnectionSpec

Specification that applies to a data source connection. Valid only for entries with the DATA_SOURCE_CONNECTION type.

Fields
bigquery_connection_spec

BigQueryConnectionSpec

Fields specific to BigQuery connections.

DatabaseTableSpec

Specification that applies to a table resource. Valid only for entries with the TABLE type.

Fields
type

TableType

Type of this table.
dataplex_table

DataplexTableSpec

Fields specific to a Dataplex table and present only in the Dataplex table entries.

TableType

Type of the table.

Enums
TABLE_TYPE_UNSPECIFIED Default unknown table type.
NATIVE Native table.
EXTERNAL External table.

DataplexExternalTable

External table registered by Dataplex. Dataplex publishes data discovered from an asset into multiple other systems (BigQuery, DPMS) in form of tables. We call them "external tables". External tables are also synced into the Data Catalog. This message contains pointers to those external tables (fully qualified name, resource name et cetera) within the Data Catalog.

Fields
system

IntegratedSystem

Service in which the external table is registered.
fully_qualified_name

string

Fully qualified name (FQN) of the external table.
google_cloud_resource

string

Google Cloud resource name of the external table.
data_catalog_entry

string

Name of the Data Catalog entry representing the external table.

DataplexFilesetSpec

Entry specyfication for a Dataplex fileset.

Fields
dataplex_spec

DataplexSpec

Common Dataplex fields.

DataplexSpec

Common Dataplex fields.

Fields
asset

string

Fully qualified resource name of an asset in Dataplex, to which the underlying data source (Cloud Storage bucket or BigQuery dataset) of the entity is attached.
data_format

PhysicalSchema

Format of the data.
compression_format

string

Compression format of the data, e.g., zip, gzip etc.
project_id

string

Project ID of the underlying Cloud Storage or BigQuery data. Note that this may not be the same project as the correspondingly Dataplex lake / zone / asset.

DataplexTableSpec

Entry specification for a Dataplex table.

Fields
external_tables[]

DataplexExternalTable

List of external tables registered by Dataplex in other systems based on the same underlying data.

External tables allow to query this data in those systems.
dataplex_spec

DataplexSpec

Common Dataplex fields.
user_managed

bool

Indicates if the table schema is managed by the user or not.

DeleteEntryGroupRequest

Request message for DeleteEntryGroup.

Fields
name

string

Required. The name of the entry group to delete.
force

bool

Optional. If true, deletes all entries in the entry group.

DeleteEntryRequest

Request message for DeleteEntry.

Fields
name

string

Required. The name of the entry to delete.

DeletePolicyTagRequest

Request message for DeletePolicyTag.

Fields
name

string

Required. Resource name of the policy tag to delete.

Note: All of its descendant policy tags are also deleted.

Authorization requires the following IAM permission on the specified resource name:

  • datacatalog.taxonomies.update

DeleteTagRequest

Request message for DeleteTag.

Fields
name

string

Required. The name of the tag to delete.

DeleteTagTemplateFieldRequest

Request message for DeleteTagTemplateField.

Fields
name

string

Required. The name of the tag template field to delete.
force

bool

Required. If true, deletes this field from any tags that use it.

Currently, true is the only supported value.

DeleteTagTemplateRequest

Request message for DeleteTagTemplate.

Fields
name

string

Required. The name of the tag template to delete.
force

bool

Required. If true, deletes all tags that use this template.

Currently, true is the only supported value.

DeleteTaxonomyRequest

Request message for DeleteTaxonomy.

Fields
name

string

Required. Resource name of the taxonomy to delete.

Note: All policy tags in this taxonomy are also deleted.

Authorization requires the following IAM permission on the specified resource name:

  • datacatalog.taxonomies.delete

Entry

Entry metadata. A Data Catalog entry represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic) or outside of it. You can use the linked_resource field in the entry resource to refer to the original resource ID of the source system.

An entry resource contains resource details, for example, its schema. Additionally, you can attach flexible metadata to an entry in the form of a Tag.

Fields
name

string

Output only. The resource name of an entry in URL format.

Note: The entry itself and its child resources might not be stored in the location specified in its name.
linked_resource

string

The resource this metadata entry refers to.

For Google Cloud Platform resources, linked_resource is the Full Resource Name. For example, the linked_resource for a table resource from BigQuery is:

//bigquery.googleapis.com/projects/{PROJECT_ID}/datasets/{DATASET_ID}/tables/{TABLE_ID}

Output only when the entry is one of the types in the EntryType enum.

For entries with a user_specified_type, this field is optional and defaults to an empty string.

The resource string must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), periods (.), colons (:), slashes (/), dashes (-), and hashes (#). The maximum size is 200 bytes when encoded in UTF-8.
fully_qualified_name

string

Fully qualified name (FQN) of the resource. Set automatically for entries representing resources from synced systems. Settable only during creation and read-only afterwards. Can be used for search and lookup of the entries.

FQNs take two forms:

  • For non-regionalized resources:

{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  • For regionalized resources:

{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

Example for a DPMS table:

dataproc_metastore:{PROJECT_ID}.{LOCATION_ID}.{INSTANCE_ID}.{DATABASE_ID}.{TABLE_ID}
display_name

string

Display name of an entry.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum size is 200 bytes when encoded in UTF-8. Default value is an empty string.
description

string

Entry description that can consist of several sentences or paragraphs that describe entry contents.

The description must not contain Unicode non-characters as well as C0 and C1 control codes except tabs (HT), new lines (LF), carriage returns (CR), and page breaks (FF). The maximum size is 2000 bytes when encoded in UTF-8. Default value is an empty string.
business_context

BusinessContext

Business Context of the entry. Not supported for BigQuery datasets
schema

Schema

Schema of the entry. An entry might not have any schema attached to it.
source_system_timestamps

SystemTimestamps

Timestamps from the underlying resource, not from the Data Catalog entry.

Output only when the entry has a type listed in the EntryType enum. For entries with user_specified_type, this field is optional and defaults to an empty timestamp.
usage_signal

UsageSignal

Output only. Resource usage statistics.
labels

map<string, string>

Cloud labels attached to the entry.

In Data Catalog, you can create and modify labels attached only to custom entries. Synced entries have unmodifiable labels that come from the source system.
data_source

DataSource

Output only. Physical location of the entry.
personal_details

PersonalDetails

Output only. Additional information related to the entry. Private to the current user.
Union field entry_type. Required. Entry type. entry_type can be only one of the following:
type

EntryType

The type of the entry. Only used for entries with types listed in the EntryType enum.

Currently, only FILESET enum value is allowed. All other entries created in Data Catalog must use the user_specified_type.
user_specified_type

string

Custom entry type that doesn't match any of the values allowed for input and listed in the EntryType enum.

When creating an entry, first check the type values in the enum. If there are no appropriate types for the new entry, provide a custom value, for example, my_special_type.

The user_specified_type string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field system. The source system of the entry. system can be only one of the following:
integrated_system

IntegratedSystem

Output only. Indicates the entry's source system that Data Catalog integrates with, such as BigQuery, Pub/Sub, or Dataproc Metastore.
user_specified_system

string

Indicates the entry's source system that Data Catalog doesn't automatically integrate with.

The user_specified_system string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field type_spec. Type specification. type_spec can be only one of the following:
gcs_fileset_spec

GcsFilesetSpec

Specification that applies to a Cloud Storage fileset. Valid only for entries with the FILESET type.
bigquery_table_spec

BigQueryTableSpec

Specification that applies to a BigQuery table. Valid only for entries with the TABLE type.
bigquery_date_sharded_spec

BigQueryDateShardedSpec

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.

Union field spec. Type- and system-specific information. Specifications for types contain fields common to all entries of a given type, and sub-specifications with fields specific to a given source system.

When extending the API with new types and systems, use this field instead of the legacy type_spec. spec can be only one of the following:
database_table_spec

DatabaseTableSpec

Specification that applies to a table resource. Valid only for entries with the TABLE type.
data_source_connection_spec

DataSourceConnectionSpec

Specification that applies to a data source connection. Valid only for entries with the DATA_SOURCE_CONNECTION type.
routine_spec

RoutineSpec

Specification that applies to a user-defined function or procedure. Valid only for entries with the ROUTINE type.
fileset_spec

FilesetSpec

Specification that applies to a fileset resource. Valid only for entries with the FILESET type.

EntryGroup

Entry group metadata.

An EntryGroup resource represents a logical grouping of zero or more Data Catalog Entry resources.

Fields
name

string

The resource name of the entry group in URL format.

Note: The entry group itself and its child resources might not be stored in the location specified in its name.
display_name

string

A short name to identify the entry group, for example, "analytics data - jan 2011". Default value is an empty string.
description

string

Entry group description. Can consist of several sentences or paragraphs that describe the entry group contents. Default value is an empty string.
data_catalog_timestamps

SystemTimestamps

Output only. Timestamps of the entry group. Default value is empty.

EntryOverview

Entry overview fields for rich text descriptions of entries.

Fields
overview

string

Entry overview with support for rich text.

The overview must only contain Unicode characters, and should be formatted using HTML. The maximum length is 10 MiB as this value holds HTML descriptions including encoded images. The maximum length of the text without images is 100 KiB.

EntryType

The enum field that lists all the types of entry resources in Data Catalog. For example, a BigQuery table entry has the TABLE type.

Enums
ENTRY_TYPE_UNSPECIFIED Default unknown type.
TABLE Output only. The entry type that has a GoogleSQL schema, including logical views.
MODEL

Output only. The type of models.

For more information, see Supported models in BigQuery ML.
DATA_STREAM An entry type for streaming entries. For example, a Pub/Sub topic.
FILESET An entry type for a set of files or objects. For example, a Cloud Storage fileset.
CLUSTER A group of servers that work together. For example, a Kafka cluster.
DATABASE A database.
DATA_SOURCE_CONNECTION Output only. Connection to a data source. For example, a BigQuery connection.
ROUTINE Output only. Routine, for example, a BigQuery routine.
LAKE A Dataplex lake.
ZONE A Dataplex zone.
SERVICE A service, for example, a Dataproc Metastore service.

ExportTaxonomiesRequest

Request message for ExportTaxonomies.

Fields
parent

string

Required. Resource name of the project that the exported taxonomies belong to.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.get
taxonomies[]

string

Required. Resource names of the taxonomies to export.
serialized_taxonomies

bool

Serialized export taxonomies that contain all the policy tags as nested protocol buffers.

ExportTaxonomiesResponse

Response message for ExportTaxonomies.

Fields
taxonomies[]

SerializedTaxonomy

List of taxonomies and policy tags as nested protocol buffers.

FieldType

Fields
Union field type_decl. Required. type_decl can be only one of the following:
primitive_type

PrimitiveType

Primitive types, such as string, boolean, etc.
enum_type

EnumType

An enum type.

EnumType

Fields
allowed_values[]

EnumValue

The set of allowed values for this enum.

This set must not be empty and can include up to 100 allowed values. The display names of the values in this set must not be empty and must be case-insensitively unique within this set.

The order of items in this set is preserved. This field can be used to create, remove, and reorder enum values. To rename enum values, use the RenameTagTemplateFieldEnumValue method.

EnumValue

Fields
display_name

string

Required. The display name of the enum value. Must not be an empty string.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters.

PrimitiveType

Enums
PRIMITIVE_TYPE_UNSPECIFIED The default invalid value for a type.
DOUBLE A double precision number.
STRING An UTF-8 string.
BOOL A boolean value.
TIMESTAMP A timestamp.
RICHTEXT A Richtext description.

FilesetSpec

Specification that applies to a fileset. Valid only for entries with the 'FILESET' type.

Fields
dataplex_fileset

DataplexFilesetSpec

Fields specific to a Dataplex fileset and present only in the Dataplex fileset entries.

GcsFileSpec

Specification of a single file in Cloud Storage.

Fields
file_path

string

Required. Full file path. Example: gs://bucket_name/a/b.txt.
gcs_timestamps

SystemTimestamps

Output only. Creation, modification, and expiration timestamps of a Cloud Storage file.
size_bytes

int64

Output only. File size in bytes.

GcsFilesetSpec

Describes a Cloud Storage fileset entry.

Fields
file_patterns[]

string

Required. Patterns to identify a set of files in Google Cloud Storage.

For more information, see Wildcard Names.

Note: Currently, bucket wildcards are not supported.

Examples of valid file_patterns:

  • gs://bucket_name/dir/*: matches all files in bucket_name/dir directory
  • gs://bucket_name/dir/**: matches all files in bucket_name/dir and all subdirectories
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match the a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt

You can combine wildcards to match complex sets of files, for example:

gs://bucket_name/[a-m]??.j*g
sample_gcs_file_specs[]

GcsFileSpec

Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here.

GetEntryGroupRequest

Request message for GetEntryGroup.

Fields
name

string

Required. The name of the entry group to get.
read_mask

FieldMask

The fields to return. If empty or omitted, all fields are returned.

GetEntryRequest

Request message for GetEntry.

Fields
name

string

Required. The name of the entry to get.

GetPolicyTagRequest

Request message for GetPolicyTag.

Fields
name

string

Required. Resource name of the policy tag.

Authorization requires the following IAM permission on the specified resource name:

  • datacatalog.taxonomies.get

GetTagTemplateRequest

Request message for GetTagTemplate.

Fields
name

string

Required. The name of the tag template to get.

GetTaxonomyRequest

Request message for GetTaxonomy.

Fields
name

string

Required. Resource name of the taxonomy to get.

Authorization requires the following IAM permission on the specified resource name:

  • datacatalog.taxonomies.get

ImportTaxonomiesRequest

Request message for ImportTaxonomies.

Fields
parent

string

Required. Resource name of project that the imported taxonomies will belong to.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.create
Union field source. Source taxonomies to import. source can be only one of the following:
inline_source

InlineSource

Inline source taxonomy to import.
cross_regional_source

CrossRegionalSource

Cross-regional source taxonomy to import.

ImportTaxonomiesResponse

Response message for ImportTaxonomies.

Fields
taxonomies[]

Taxonomy

Imported taxonomies.

InlineSource

Inline source containing taxonomies to import.

Fields
taxonomies[]

SerializedTaxonomy

Required. Taxonomies to import.

IntegratedSystem

This enum lists all the systems that Data Catalog integrates with.

Enums
INTEGRATED_SYSTEM_UNSPECIFIED Default unknown system.
BIGQUERY BigQuery.
CLOUD_PUBSUB Cloud Pub/Sub.
DATAPROC_METASTORE Dataproc Metastore.
DATAPLEX Dataplex.

ListEntriesRequest

Request message for ListEntries.

Fields
parent

string

Required. The name of the entry group that contains the entries to list.

Can be provided in URL format.
page_size

int32

The maximum number of items to return. Default is 10. Maximum limit is 1000. Throws an invalid argument if page_size is more than 1000.
page_token

string

Pagination token that specifies the next page to return. If empty, the first page is returned.
read_mask

FieldMask

The fields to return for each entry. If empty or omitted, all fields are returned.

For example, to return a list of entries with only the name field, set read_mask to only one path with the name value.

ListEntriesResponse

Response message for ListEntries.

Fields
entries[]

Entry

Entry details.
next_page_token

string

Pagination token of the next results page. Empty if there are no more items in results.

ListEntryGroupsRequest

Request message for ListEntryGroups.

Fields
parent

string

Required. The name of the location that contains the entry groups to list.

Can be provided as a URL.
page_size

int32

Optional. The maximum number of items to return.

Default is 10. Maximum limit is 1000. Throws an invalid argument if page_size is greater than 1000.
page_token

string

Optional. Pagination token that specifies the next page to return. If empty, returns the first page.

ListEntryGroupsResponse

Response message for ListEntryGroups.

Fields
entry_groups[]

EntryGroup

Entry group details.
next_page_token

string

Pagination token to specify in the next call to retrieve the next page of results. Empty if there are no more items.

ListPolicyTagsRequest

Request message for ListPolicyTags.

Fields
parent

string

Required. Resource name of the taxonomy to list the policy tags of.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.get
page_size

int32

The maximum number of items to return. Must be a value between 1 and 1000 inclusively. If not set, defaults to 50.
page_token

string

The pagination token of the next results page. If not set, returns the first page.

The token is returned in the response to a previous list request.

ListPolicyTagsResponse

Response message for ListPolicyTags.

Fields
policy_tags[]

PolicyTag

The policy tags that belong to the taxonomy.
next_page_token

string

Pagination token of the next results page. Empty if there are no more results in the list.

ListTagsRequest

Request message for ListTags.

Fields
parent

string

Required. The name of the Data Catalog resource to list the tags of.

The resource can be an Entry or an EntryGroup (without /entries/{entries} at the end).
page_size

int32

The maximum number of tags to return. Default is 10. Maximum limit is 1000.
page_token

string

Pagination token that specifies the next page to return. If empty, the first page is returned.

ListTagsResponse

Response message for ListTags.

Fields
tags[]

Tag

Tag details.
next_page_token

string

Pagination token of the next results page. Empty if there are no more items in results.

ListTaxonomiesRequest

Request message for ListTaxonomies.

Fields
parent

string

Required. Resource name of the project to list the taxonomies of.

Authorization requires the following IAM permission on the specified resource parent:

  • datacatalog.taxonomies.list
page_size

int32

The maximum number of items to return. Must be a value between 1 and 1000 inclusively. If not set, defaults to 50.
page_token

string

The pagination token of the next results page. If not set, the first page is returned.

The token is returned in the response to a previous list request.

ListTaxonomiesResponse

Response message for ListTaxonomies.

Fields
taxonomies[]

Taxonomy

Taxonomies that the project contains.
next_page_token

string

Pagination token of the next results page. Empty if there are no more results in the list.

LookupEntryRequest

Request message for LookupEntry.

Fields
Union field target_name. Required. A full name, SQL name, or a fully qualified name of a Google Cloud Platform resource. target_name can be only one of the following:
linked_resource

string

The full name of the Google Cloud Platform resource the Data Catalog entry represents. For more information, see Full Resource Name.

Full names are case-sensitive. For example:

  • //bigquery.googleapis.com/projects/{PROJECT_ID}/datasets/{DATASET_ID}/tables/{TABLE_ID}
  • //pubsub.googleapis.com/projects/{PROJECT_ID}/topics/{TOPIC_ID}
sql_resource

string

The SQL name of the entry. SQL names are case-sensitive.

Examples:

  • pubsub.topic.{PROJECT_ID}.{TOPIC_ID}
  • pubsub.topic.{PROJECT_ID}.`{TOPIC.ID.SEPARATED.WITH.DOTS}`
  • bigquery.table.{PROJECT_ID}.{DATASET_ID}.{TABLE_ID}
  • bigquery.dataset.{PROJECT_ID}.{DATASET_ID}
  • datacatalog.entry.{PROJECT_ID}.{LOCATION_ID}.{ENTRY_GROUP_ID}.{ENTRY_ID}

Identifiers (*_ID) should comply with the Lexical structure in Standard SQL.
fully_qualified_name

string

Fully qualified name (FQN) of the resource.

FQNs take two forms:

  • For non-regionalized resources:

{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  • For regionalized resources:

{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

Example for a DPMS table:

dataproc_metastore:{PROJECT_ID}.{LOCATION_ID}.{INSTANCE_ID}.{DATABASE_ID}.{TABLE_ID}

ModifyEntryContactsRequest

Request message for ModifyEntryContacts.

Fields
name

string

Required. The full resource name of the entry.
contacts

Contacts

Required. The new value for the Contacts.

ModifyEntryOverviewRequest

Request message for ModifyEntryOverview.

Fields
name

string

Required. The full resource name of the entry.
entry_overview

EntryOverview

Required. The new value for the Entry Overview.

PersonalDetails

Entry metadata relevant only to the user and private to them.

Fields
starred

bool

True if the entry is starred by the user; false otherwise.
star_time

Timestamp

Set if the entry is starred; unset otherwise.

PhysicalSchema

Native schema used by a resource represented as an entry. Used by query engines for deserializing and parsing source data.

Fields

Union field schema.

schema can be only one of the following:
avro

AvroSchema

Schema in Avro JSON format.
thrift

ThriftSchema

Schema in Thrift format.
protobuf

ProtobufSchema

Schema in protocol buffer format.
parquet

ParquetSchema

Marks a Parquet-encoded data source.
orc

OrcSchema

Marks an ORC-encoded data source.
csv

CsvSchema

Marks a CSV-encoded data source.

AvroSchema

Schema in Avro JSON format.

Fields
text

string

JSON source of the Avro schema.

CsvSchema

Marks a CSV-encoded data source.

OrcSchema

Marks an ORC-encoded data source.

ParquetSchema

Marks a Parquet-encoded data source.

ProtobufSchema

Schema in protocol buffer format.

Fields
text

string

Protocol buffer source of the schema.

ThriftSchema

Schema in Thrift format.

Fields
text

string

Thrift IDL source of the schema.

PolicyTag

Denotes one policy tag in a taxonomy, for example, SSN.

Policy tags can be defined in a hierarchy. For example:

+ Geolocation
  + LatLong
  + City
  + ZipCode

Where the "Geolocation" policy tag contains three children.

Fields
name

string

Output only. Resource name of this policy tag in the URL format.

The policy tag manager generates unique taxonomy IDs and policy tag IDs.
display_name

string

Required. User-defined name of this policy tag.

The name can't start or end with spaces and must be unique within the parent taxonomy, contain only Unicode letters, numbers, underscores, dashes and spaces, and be at most 200 bytes long when encoded in UTF-8.
description

string

Description of this policy tag. If not set, defaults to empty.

The description must contain only Unicode characters, tabs, newlines, carriage returns and page breaks, and be at most 2000 bytes long when encoded in UTF-8.
parent_policy_tag

string

Resource name of this policy tag's parent policy tag. If empty, this is a top level tag. If not set, defaults to an empty string.

For example, for the "LatLong" policy tag in the example above, this field contains the resource name of the "Geolocation" policy tag, and, for "Geolocation", this field is empty.
child_policy_tags[]

string

Output only. Resource names of child policy tags of this policy tag.

RenameTagTemplateFieldEnumValueRequest

Request message for RenameTagTemplateFieldEnumValue.

Fields
name

string

Required. The name of the enum field value.
new_enum_value_display_name

string

Required. The new display name of the enum value. For example, my_new_enum_value.

RenameTagTemplateFieldRequest

Request message for RenameTagTemplateField.

Fields
name

string

Required. The name of the tag template field.
new_tag_template_field_id

string

Required. The new ID of this tag template field. For example, my_new_field.

ReplaceTaxonomyRequest

Request message for ReplaceTaxonomy.

Fields
name

string

Required. Resource name of the taxonomy to update.

Authorization requires the following IAM permission on the specified resource name:

  • datacatalog.taxonomies.update
serialized_taxonomy

SerializedTaxonomy

Required. Taxonomy to update along with its child policy tags.

RoutineSpec

Specification that applies to a routine. Valid only for entries with the ROUTINE type.

Fields
routine_type

RoutineType

The type of the routine.
language

string

The language the routine is written in. The exact value depends on the source system. For BigQuery routines, possible values are:

  • SQL
  • JAVASCRIPT
routine_arguments[]

Argument

Arguments of the routine.
return_type

string

Return type of the argument. The exact value depends on the source system and the language.
definition_body

string

The body of the routine.
bigquery_routine_spec

BigQueryRoutineSpec

Fields specific for BigQuery routines.

Argument

Input or output argument of a function or stored procedure.

Fields
name

string

The name of the argument. A return argument of a function might not have a name.
mode

Mode

Specifies whether the argument is input or output.
type

string

Type of the argument. The exact value depends on the source system and the language.

Mode

The input or output mode of the argument.

Enums
MODE_UNSPECIFIED Unspecified mode.
IN The argument is input-only.
OUT The argument is output-only.
INOUT The argument is both an input and an output.

RoutineType

The fine-grained type of the routine.

Enums
ROUTINE_TYPE_UNSPECIFIED Unspecified type.
SCALAR_FUNCTION Non-builtin permanent scalar function.
PROCEDURE Stored procedure.

Schema

Represents a schema, for example, a BigQuery, GoogleSQL, or Avro schema.

Fields
columns[]

ColumnSchema

The unified GoogleSQL-like schema of columns.

The overall maximum number of columns and nested columns is 10,000. The maximum nested depth is 15 levels.

SearchCatalogRequest

Request message for SearchCatalog.

Fields
scope

Scope

Required. The scope of this search request.

The scope is invalid if include_org_ids, include_project_ids are empty AND include_gcp_public_datasets is set to false. In this case, the request returns an error.
query

string

Optional. The query string with a minimum of 3 characters and specific syntax. For more information, see Data Catalog search syntax.

An empty query string returns all data assets (in the specified scope) that you have access to.

A query string can be a simple xyz or qualified by predicates:

  • name:x
  • column:y
  • description:z
page_size

int32

Number of results to return in a single search page.

Can't be negative or 0, defaults to 10 in this case. The maximum number is 1000. If exceeded, throws an "invalid argument" exception.
page_token

string

Optional. Pagination token that, if specified, returns the next page of search results. If empty, returns the first page.

This token is returned in the SearchCatalogResponse.next_page_token field of the response to a previous SearchCatalogRequest call.
order_by

string

Specifies the order of results.

Currently supported case-sensitive values are:

  • relevance that can only be descending
  • last_modified_timestamp [asc|desc] with descending (desc) as default
  • default that can only be descending

If this parameter is omitted, it defaults to the descending relevance.

Scope

The criteria that select the subspace used for query matching.

Fields
include_org_ids[]

string

The list of organization IDs to search within.

To find your organization ID, follow the steps from Creating and managing organizations.
include_project_ids[]

string

The list of project IDs to search within.

For more information on the distinction between project names, IDs, and numbers, see Projects.
include_gcp_public_datasets

bool

If true, include Google Cloud Platform (GCP) public datasets in search results. By default, they are excluded.

See Google Cloud Public Datasets for more information.
restricted_locations[]

string

Optional. The list of locations to search within. If empty, all locations are searched.

Returns an error if any location in the list isn't one of the Supported regions.

If a location is unreachable, its name is returned in the SearchCatalogResponse.unreachable field. To get additional information on the error, repeat the search request and set the location name as the value of this parameter.
starred_only

bool

Optional. If true, search only among starred entries.

By default, all results are returned, starred or not.
include_public_tag_templates
(deprecated)

bool

Optional. This field is deprecated. The search mechanism for public and private tag templates is the same.

SearchCatalogResponse

Response message for SearchCatalog.

Fields
results[]

SearchCatalogResult

Search results.
next_page_token

string

Pagination token that can be used in subsequent calls to retrieve the next page of results.
unreachable[]

string

Unreachable locations. Search results don't include data from those locations.

To get additional information on an error, repeat the search request and restrict it to specific locations by setting the SearchCatalogRequest.scope.restricted_locations parameter.

SearchCatalogResult

Result in the response to a search request.

Each result captures details of one entry that matches the search.

Fields
search_result_type

SearchResultType

Type of the search result.

You can use this field to determine which get method to call to fetch the full resource.
search_result_subtype

string

Sub-type of the search result.

A dot-delimited full type of the resource. The same type you specify in the type search predicate.

Examples: entry.table, entry.dataStream, tagTemplate.
relative_resource_name

string

The relative name of the resource in URL format.

Examples:

  • projects/{PROJECT_ID}/locations/{LOCATION_ID}/entryGroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}
  • projects/{PROJECT_ID}/tagTemplates/{TAG_TEMPLATE_ID}
linked_resource

string

The full name of the Google Cloud resource the entry belongs to.

For more information, see Full Resource Name.

Example:

//bigquery.googleapis.com/projects/PROJECT_ID/datasets/DATASET_ID/tables/TABLE_ID
modify_time

Timestamp

The last modification timestamp of the entry in the source system.
fully_qualified_name

string

Fully qualified name (FQN) of the resource.

FQNs take two forms:

  • For non-regionalized resources:

{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  • For regionalized resources:

{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

Example for a DPMS table:

dataproc_metastore:PROJECT_ID.LOCATION_ID.INSTANCE_ID.DATABASE_ID.TABLE_ID
display_name

string

The display name of the result.
description

string

Entry description that can consist of several sentences or paragraphs that describe entry contents.
Union field system. The source system of the entry. Applicable only when the search_result_type is ENTRY. system can be only one of the following:
integrated_system

IntegratedSystem

Output only. The source system that Data Catalog automatically integrates with, such as BigQuery, Cloud Pub/Sub, or Dataproc Metastore.
user_specified_system

string

Custom source system that you can manually integrate Data Catalog with.

SearchResultType

The resource types that can be returned in search results.

Enums
SEARCH_RESULT_TYPE_UNSPECIFIED Default unknown type.
ENTRY An Entry.
TAG_TEMPLATE A TagTemplate.
ENTRY_GROUP An EntryGroup.

SerializedPolicyTag

A nested protocol buffer that represents a policy tag and all its descendants.

Fields
policy_tag

string

Resource name of the policy tag.

This field is ignored when calling ImportTaxonomies.
display_name

string

Required. Display name of the policy tag. At most 200 bytes when encoded in UTF-8.
description

string

Description of the serialized policy tag. At most 2000 bytes when encoded in UTF-8. If not set, defaults to an empty description.
child_policy_tags[]

SerializedPolicyTag

Children of the policy tag, if any.

SerializedTaxonomy

A nested protocol buffer that represents a taxonomy and the hierarchy of its policy tags. Used for taxonomy replacement, import, and export.

Fields
display_name

string

Required. Display name of the taxonomy. At most 200 bytes when encoded in UTF-8.
description

string

Description of the serialized taxonomy. At most 2000 bytes when encoded in UTF-8. If not set, defaults to an empty description.
policy_tags[]

SerializedPolicyTag

Top level policy tags associated with the taxonomy, if any.
activated_policy_types[]

PolicyType

A list of policy types that are activated per taxonomy.

StarEntryRequest

Request message for StarEntry.

Fields
name

string

Required. The name of the entry to mark as starred.

StarEntryResponse

Response message for StarEntry. Empty for now

StorageProperties

Details the properties of the underlying storage.

Fields
file_pattern[]

string

Patterns to identify a set of files for this fileset.

Examples of a valid file_pattern:

  • gs://bucket_name/dir/*: matches all files in the bucket_name/dir directory
  • gs://bucket_name/dir/**: matches all files in the bucket_name/dir and all subdirectories recursively
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match the a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt
file_type

string

File type in MIME format, for example, text/plain.

SystemTimestamps

Timestamps associated with this resource in a particular system.

Fields
create_time

Timestamp

Creation timestamp of the resource within the given system.
update_time

Timestamp

Timestamp of the last modification of the resource or its metadata within a given system.

Note: Depending on the source system, not every modification updates this timestamp. For example, BigQuery timestamps every metadata modification but not data or permission changes.
expire_time

Timestamp

Output only. Expiration timestamp of the resource within the given system.

Currently only applicable to BigQuery resources.

TableSourceType

Table source type.

Enums
TABLE_SOURCE_TYPE_UNSPECIFIED Default unknown type.
BIGQUERY_VIEW Table view.
BIGQUERY_TABLE BigQuery native table.
BIGQUERY_MATERIALIZED_VIEW BigQuery materialized view.

TableSpec

Normal BigQuery table specification.

Fields
grouped_entry

string

Output only. If the table is date-sharded, that is, it matches the [prefix]YYYYMMDD name pattern, this field is the Data Catalog resource name of the date-sharded grouped entry. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.

Otherwise, grouped_entry is empty.

Tag

Tags contain custom metadata and are attached to Data Catalog resources. Tags conform with the specification of their tag template.

See Data Catalog IAM for information on the permissions needed to create or view tags.

Fields
name

string

The resource name of the tag in URL format where tag ID is a system-generated identifier.

Note: The tag itself might not be stored in the location specified in its name.
template

string

Required. The resource name of the tag template this tag uses. Example:

projects/{PROJECT_ID}/locations/{LOCATION}/tagTemplates/{TAG_TEMPLATE_ID}

This field cannot be modified after creation.
template_display_name

string

Output only. The display name of the tag template.
fields

map<string, TagField>

Required. Maps the ID of a tag field to its value and additional information about that field.

Tag template defines valid field IDs. A tag must have at least 1 field and at most 500 fields.
column

string

Resources like entry can have schemas associated with them. This scope allows you to attach tags to an individual column based on that schema.

To attach a tag to a nested column, separate column names with a dot (.). Example: column.nested_column.

TagField

Contains the value and additional information on a field within a Tag.

Fields
display_name

string

Output only. The display name of this field.
order

int32

Output only. The order of this field with respect to other fields in this tag. Can be set by Tag.

For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order, and field orders within a tag don't have to be sequential.
Union field kind. Required. The value of this field. kind can be only one of the following:
double_value

double

The value of a tag field with a double type.
string_value

string

The value of a tag field with a string type.

The maximum length is 2000 UTF-8 characters.
bool_value

bool

The value of a tag field with a boolean type.
timestamp_value

Timestamp

The value of a tag field with a timestamp type.
enum_value

EnumValue

The value of a tag field with an enum type.

This value must be one of the allowed values listed in this enum.
richtext_value

string

The value of a tag field with a rich text type.

The maximum length is 10 MiB as this value holds HTML descriptions including encoded images. The maximum length of the text without images is 100 KiB.

EnumValue

An enum value.

Fields
display_name

string

The display name of the enum value.

TagTemplate

A tag template defines a tag that can have one or more typed fields.

The template is used to create tags that are attached to GCP resources. Tag template roles provide permissions to create, edit, and use the template. For example, see the TagTemplate User role that includes a permission to use the tag template to tag resources.

Fields
name

string

The resource name of the tag template in URL format.

Note: The tag template itself and its child resources might not be stored in the location specified in its name.
display_name

string

Display name for this template. Defaults to an empty string.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters.
is_publicly_readable

bool

Indicates whether tags created with this template are public. Public tags do not require tag template access to appear in [ListTags][google.cloud.datacatalog.v1.ListTags] API response.

Additionally, you can search for a public tag by value with a simple search query instead of using a tag: predicate.
fields

map<string, TagTemplateField>

Required. Map of tag template field IDs to the settings for the field. This map is an exhaustive list of the allowed fields. The map must contain at least one field and at most 500 fields.

The keys to this map are tag template field IDs. The IDs have the following limitations:

  • Can contain uppercase and lowercase letters, numbers (0-9) and underscores (_).
  • Must be at least 1 character and at most 64 characters long.
  • Must start with a letter or underscore.

TagTemplateField

The template for an individual field within a tag template.

Fields
name

string

Output only. The resource name of the tag template field in URL format. Example:

projects/{PROJECT_ID}/locations/{LOCATION}/tagTemplates/{TAG_TEMPLATE}/fields/{FIELD}

Note: The tag template field itself might not be stored in the location specified in its name.

The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 64 characters.
display_name

string

The display name for this field. Defaults to an empty string.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters.
type

FieldType

Required. The type of value this tag field can contain.
is_required

bool

If true, this field is required. Defaults to false.
description

string

The description for this field. Defaults to an empty string.
order

int32

The order of this field with respect to other fields in this tag template.

For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order and field orders within a tag don't have to be sequential.

Taxonomy

A taxonomy is a collection of hierarchical policy tags that classify data along a common axis.

For example, a "data sensitivity" taxonomy might contain the following policy tags:

+ PII
  + Account number
  + Age
  + SSN
  + Zipcode
+ Financials
  + Revenue

A "data origin" taxonomy might contain the following policy tags:

+ User data
+ Employee data
+ Partner data
+ Public data
Fields
name

string

Output only. Resource name of this taxonomy in URL format.

Note: Policy tag manager generates unique taxonomy IDs.
display_name

string

Required. User-defined name of this taxonomy.

The name can't start or end with spaces, must contain only Unicode letters, numbers, underscores, dashes, and spaces, and be at most 200 bytes long when encoded in UTF-8.
description

string

Optional. Description of this taxonomy. If not set, defaults to empty.

The description must contain only Unicode characters, tabs, newlines, carriage returns, and page breaks, and be at most 2000 bytes long when encoded in UTF-8.
policy_tag_count

int32

Output only. Number of policy tags in this taxonomy.
taxonomy_timestamps

SystemTimestamps

Output only. Creation and modification timestamps of this taxonomy.
activated_policy_types[]

PolicyType

Optional. A list of policy types that are activated for this taxonomy. If not set, defaults to an empty list.

PolicyType

Defines policy types where the policy tags can be used for.

Enums
POLICY_TYPE_UNSPECIFIED Unspecified policy type.
FINE_GRAINED_AC