Package google.cloud.datacatalog.v1

Index

DataCatalog

Data Catalog API service allows clients to discover, understand, and manage their data.

CreateEntry

rpc CreateEntry(CreateEntryRequest) returns (Entry)

Creates an entry. Only entries of 'FILESET' type or user-specified type can be created.

Users should enable the Data Catalog API in the project identified by the parent parameter (see Data Catalog Resource Project for more information).

A maximum of 100,000 entries may be created per entry group.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateEntryGroup

rpc CreateEntryGroup(CreateEntryGroupRequest) returns (EntryGroup)

Creates an EntryGroup.

An entry group contains logically related entries together with Cloud Identity and Access Management policies that specify the users who can create, edit, and view entries within the entry group.

Data Catalog automatically creates an entry group for BigQuery entries ("@bigquery") and Pub/Sub topics ("@pubsub"). Users create their own entry group to contain Cloud Storage fileset entries or custom type entries, and the IAM policies associated with those entries. Entry groups, like entries, can be searched.

A maximum of 10,000 entry groups may be created per organization across all locations.

Users should enable the Data Catalog API in the project identified by the parent parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTag

rpc CreateTag(CreateTagRequest) returns (Tag)

Creates a tag on an Entry. Note: The project identified by the parent parameter for the tag and the tag template used to create the tag must be from the same organization.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTagTemplate

rpc CreateTagTemplate(CreateTagTemplateRequest) returns (TagTemplate)

Creates a tag template. The user should enable the Data Catalog API in the project identified by the parent parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateTagTemplateField

rpc CreateTagTemplateField(CreateTagTemplateFieldRequest) returns (TagTemplateField)

Creates a field in a tag template. The user should enable the Data Catalog API in the project identified by the parent parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteEntry

rpc DeleteEntry(DeleteEntryRequest) returns (Empty)

Deletes an existing entry. Only entries created through CreateEntry method can be deleted. Users should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteEntryGroup

rpc DeleteEntryGroup(DeleteEntryGroupRequest) returns (Empty)

Deletes an EntryGroup. Only entry groups that do not contain entries can be deleted. Users should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTag

rpc DeleteTag(DeleteTagRequest) returns (Empty)

Deletes a tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTagTemplate

rpc DeleteTagTemplate(DeleteTagTemplateRequest) returns (Empty)

Deletes a tag template and all tags using the template. Users should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteTagTemplateField

rpc DeleteTagTemplateField(DeleteTagTemplateFieldRequest) returns (Empty)

Deletes a field in a tag template and all uses of that field. Users should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetEntry

rpc GetEntry(GetEntryRequest) returns (Entry)

Gets an entry.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetEntryGroup

rpc GetEntryGroup(GetEntryGroupRequest) returns (EntryGroup)

Gets an EntryGroup.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetIamPolicy

rpc GetIamPolicy(GetIamPolicyRequest) returns (Policy)

Gets the access control policy for a resource. A NOT_FOUND error is returned if the resource does not exist. An empty policy is returned if the resource exists but does not have a policy set on it.

Supported resources are: - Tag templates. - Entries. - Entry groups. Note, this method cannot be used to manage policies for BigQuery, Pub/Sub and any external Google Cloud Platform resources synced to Data Catalog.

Callers must have following Google IAM permission - datacatalog.tagTemplates.getIamPolicy to get policies on tag templates. - datacatalog.entries.getIamPolicy to get policies on entries. - datacatalog.entryGroups.getIamPolicy to get policies on entry groups.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetTagTemplate

rpc GetTagTemplate(GetTagTemplateRequest) returns (TagTemplate)

Gets a tag template.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListEntries

rpc ListEntries(ListEntriesRequest) returns (ListEntriesResponse)

Lists entries.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListEntryGroups

rpc ListEntryGroups(ListEntryGroupsRequest) returns (ListEntryGroupsResponse)

Lists entry groups.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListTags

rpc ListTags(ListTagsRequest) returns (ListTagsResponse)

Lists the tags on an Entry.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
LookupEntry

rpc LookupEntry(LookupEntryRequest) returns (Entry)

Get an entry by target resource name. This method allows clients to use the resource name from the source Google Cloud Platform service to get the Data Catalog Entry.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
RenameTagTemplateField

rpc RenameTagTemplateField(RenameTagTemplateFieldRequest) returns (TagTemplateField)

Renames a field in a tag template. The user should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
RenameTagTemplateFieldEnumValue

rpc RenameTagTemplateFieldEnumValue(RenameTagTemplateFieldEnumValueRequest) returns (TagTemplateField)

Renames an enum value in a tag template. The enum values have to be unique within one enum field.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
SearchCatalog

rpc SearchCatalog(SearchCatalogRequest) returns (SearchCatalogResponse)

Searches Data Catalog for multiple resources like entries, tags that match a query.

This is a custom method (https://cloud.google.com/apis/design/custom_methods) and does not return the complete resource, only the resource identifier and high level fields. Clients can subsequentally call Get methods.

Note that Data Catalog search queries do not guarantee full recall. Query results that match your query may not be returned, even in subsequent result pages. Also note that results returned (and not returned) can vary across repeated search queries.

See Data Catalog Search Syntax for more information.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
SetIamPolicy

rpc SetIamPolicy(SetIamPolicyRequest) returns (Policy)

Sets the access control policy for a resource. Replaces any existing policy. Supported resources are: - Tag templates. - Entries. - Entry groups. Note, this method cannot be used to manage policies for BigQuery, Pub/Sub and any external Google Cloud Platform resources synced to Data Catalog.

Callers must have following Google IAM permission - datacatalog.tagTemplates.setIamPolicy to set policies on tag templates. - datacatalog.entries.setIamPolicy to set policies on entries. - datacatalog.entryGroups.setIamPolicy to set policies on entry groups.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
TestIamPermissions

rpc TestIamPermissions(TestIamPermissionsRequest) returns (TestIamPermissionsResponse)

Returns the caller's permissions on a resource. If the resource does not exist, an empty set of permissions is returned (We don't return a NOT_FOUND error).

Supported resources are: - Tag templates. - Entries. - Entry groups. Note, this method cannot be used to manage policies for BigQuery, Pub/Sub and any external Google Cloud Platform resources synced to Data Catalog.

A caller is not required to have Google IAM permission to make this request.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateEntry

rpc UpdateEntry(UpdateEntryRequest) returns (Entry)

Updates an existing entry. Users should enable the Data Catalog API in the project identified by the entry.name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateEntryGroup

rpc UpdateEntryGroup(UpdateEntryGroupRequest) returns (EntryGroup)

Updates an EntryGroup. The user should enable the Data Catalog API in the project identified by the entry_group.name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTag

rpc UpdateTag(UpdateTagRequest) returns (Tag)

Updates an existing tag.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTagTemplate

rpc UpdateTagTemplate(UpdateTagTemplateRequest) returns (TagTemplate)

Updates a tag template. This method cannot be used to update the fields of a template. The tag template fields are represented as separate resources and should be updated using their own create/update/delete methods. Users should enable the Data Catalog API in the project identified by the tag_template.name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateTagTemplateField

rpc UpdateTagTemplateField(UpdateTagTemplateFieldRequest) returns (TagTemplateField)

Updates a field in a tag template. This method cannot be used to update the field type. Users should enable the Data Catalog API in the project identified by the name parameter (see Data Catalog Resource Project for more information).

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

BigQueryDateShardedSpec

Spec for a group of BigQuery tables with name pattern [prefix]YYYYMMDD. Context: https://cloud.google.com/bigquery/docs/partitioned-tables#partitioning_versus_sharding

Fields
dataset

string

Output only. The Data Catalog resource name of the dataset entry the current table belongs to, for example, projects/{project_id}/locations/{location}/entrygroups/{entry_group_id}/entries/{entry_id}.
table_prefix

string

Output only. The table name prefix of the shards. The name of any given shard is [table_prefix]YYYYMMDD, for example, for shard MyTable20180101, the table_prefix is MyTable.
shard_count

int64

Output only. Total number of shards.

BigQueryTableSpec

Describes a BigQuery table.

Fields
table_source_type

TableSourceType

Output only. The table source type.
Union field type_spec. Output only. type_spec can be only one of the following:
view_spec

ViewSpec

Table view specification. This field should only be populated if table_source_type is BIGQUERY_VIEW.
table_spec

TableSpec

Spec of a BigQuery table. This field should only be populated if table_source_type is BIGQUERY_TABLE.

ColumnSchema

Representation of a column within a schema. Columns could be nested inside other columns.

Fields
column

string

Required. Name of the column.
type

string

Required. Type of the column.
description

string

Optional. Description of the column. Default value is an empty string.
mode

string

Optional. A column's mode indicates whether the values in this column are required, nullable, etc. Only NULLABLE, REQUIRED and REPEATED are supported. Default mode is NULLABLE.
subcolumns[]

ColumnSchema

Optional. Schema of sub-columns. A column can have zero or more sub-columns.

CreateEntryGroupRequest

Request message for CreateEntryGroup.

Fields
parent

string

Required. The name of the project this entry group is in. Example:

  • projects/{project_id}/locations/{location}

Note that this EntryGroup and its child resources may not actually be stored in the location in this name.
entry_group_id

string

Required. The id of the entry group to create. The id must begin with a letter or underscore, contain only English letters, numbers and underscores, and be at most 64 characters.
entry_group

EntryGroup

The entry group to create. Defaults to an empty entry group.

CreateEntryRequest

Request message for CreateEntry.

Fields
parent

string

Required. The name of the entry group this entry is in. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}

Note that this Entry and its child resources may not actually be stored in the location in this name.
entry_id

string

Required. The id of the entry to create.
entry

Entry

Required. The entry to create.

CreateTagRequest

Request message for CreateTag.

Fields
parent

string

Required. The name of the resource to attach this tag to. Tags can be attached to Entries. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}

Note that this Tag and its child resources may not actually be stored in the location in this name.
tag

Tag

Required. The tag to create.

CreateTagTemplateFieldRequest

Request message for CreateTagTemplateField.

Fields
parent

string

Required. The name of the project and the template location region.

Example:

  • projects/{project_id}/locations/us-central1/tagTemplates/{tag_template_id}
tag_template_field_id

string

Required. The ID of the tag template field to create. Field ids can contain letters (both uppercase and lowercase), numbers (0-9), underscores (_) and dashes (-). Field IDs must be at least 1 character long and at most 128 characters long. Field IDs must also be unique within their template.
tag_template_field

TagTemplateField

Required. The tag template field to create.

CreateTagTemplateRequest

Request message for CreateTagTemplate.

Fields
parent

string

Required. The name of the project and the template location region.

Example:

  • projects/{project_id}/locations/us-central1
tag_template_id

string

Required. The id of the tag template to create.
tag_template

TagTemplate

Required. The tag template to create.

DeleteEntryGroupRequest

Request message for DeleteEntryGroup.

Fields
name

string

Required. The name of the entry group. For example, projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}.
force

bool

Optional. If true, deletes all entries in the entry group.

DeleteEntryRequest

Request message for DeleteEntry.

Fields
name

string

Required. The name of the entry. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}

DeleteTagRequest

Request message for DeleteTag.

Fields
name

string

Required. The name of the tag to delete. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}/tags/{tag_id}

DeleteTagTemplateFieldRequest

Request message for DeleteTagTemplateField.

Fields
name

string

Required. The name of the tag template field to delete. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}/fields/{tag_template_field_id}
force

bool

Required. Currently, this field must always be set to true. This confirms the deletion of this field from any tags using this field. force = false will be supported in the future.

DeleteTagTemplateRequest

Request message for DeleteTagTemplate.

Fields
name

string

Required. The name of the tag template to delete. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}
force

bool

Required. Currently, this field must always be set to true. This confirms the deletion of any possible tags using this template. force = false will be supported in the future.

Entry

Entry Metadata. A Data Catalog Entry resource represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic) or outside of Google Cloud Platform. Clients can use the linked_resource field in the Entry resource to refer to the original resource ID of the source system.

An Entry resource contains resource details, such as its schema. An Entry can also be used to attach flexible metadata, such as a Tag.

Fields
name

string

The Data Catalog resource name of the entry in URL format. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}

Note that this Entry and its child resources may not actually be stored in the location in this name.
linked_resource

string

The resource this metadata entry refers to.

For Google Cloud Platform resources, linked_resource is the full name of the resource. For example, the linked_resource for a table resource from BigQuery is:

  • //bigquery.googleapis.com/projects/projectId/datasets/datasetId/tables/tableId

Output only when Entry is of type in the EntryType enum. For entries with user_specified_type, this field is optional and defaults to an empty string.
display_name

string

Display information such as title and description. A short name to identify the entry, for example, "Analytics Data - Jan 2011". Default value is an empty string.
description

string

Entry description, which can consist of several sentences or paragraphs that describe entry contents. Default value is an empty string.
schema

Schema

Schema of the entry. An entry might not have any schema attached to it.
source_system_timestamps

SystemTimestamps

Timestamps about the underlying resource, not about this Data Catalog entry. Output only when Entry is of type in the EntryType enum. For entries with user_specified_type, this field is optional and defaults to an empty timestamp.
Union field entry_type. Required. Entry type. entry_type can be only one of the following:
type

EntryType

The type of the entry. Only used for Entries with types in the EntryType enum.
user_specified_type

string

Entry type if it does not fit any of the input-allowed values listed in EntryType enum above. When creating an entry, users should check the enum values first, if nothing matches the entry to be created, then provide a custom value, for example "my_special_type". user_specified_type strings must begin with a letter or underscore and can only contain letters, numbers, and underscores; are case insensitive; must be at least 1 character and at most 64 characters long.

Currently, only FILESET enum value is allowed. All other entries created through Data Catalog must use user_specified_type.
Union field system. The source system of the entry. system can be only one of the following:
integrated_system

IntegratedSystem

Output only. This field indicates the entry's source system that Data Catalog integrates with, such as BigQuery or Pub/Sub.
user_specified_system

string

This field indicates the entry's source system that Data Catalog does not integrate with. user_specified_system strings must begin with a letter or underscore and can only contain letters, numbers, and underscores; are case insensitive; must be at least 1 character and at most 64 characters long.
Union field type_spec. Type specification information. type_spec can be only one of the following:
gcs_fileset_spec

GcsFilesetSpec

Specification that applies to a Cloud Storage fileset. This is only valid on entries of type FILESET.
bigquery_table_spec

BigQueryTableSpec

Specification that applies to a BigQuery table. This is only valid on entries of type TABLE.
bigquery_date_sharded_spec

BigQueryDateShardedSpec

Specification for a group of BigQuery tables with name pattern [prefix]YYYYMMDD. Context: https://cloud.google.com/bigquery/docs/partitioned-tables#partitioning_versus_sharding.

EntryGroup

EntryGroup Metadata. An EntryGroup resource represents a logical grouping of zero or more Data Catalog Entry resources.

Fields
name

string

The resource name of the entry group in URL format. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}

Note that this EntryGroup and its child resources may not actually be stored in the location in this name.
display_name

string

A short name to identify the entry group, for example, "analytics data - jan 2011". Default value is an empty string.
description

string

Entry group description, which can consist of several sentences or paragraphs that describe entry group contents. Default value is an empty string.
data_catalog_timestamps

SystemTimestamps

Output only. Timestamps about this EntryGroup. Default value is empty timestamps.

EntryType

Entry resources in Data Catalog can be of different types e.g. a BigQuery Table entry is of type TABLE. This enum describes all the possible types Data Catalog contains.

Enums
ENTRY_TYPE_UNSPECIFIED Default unknown type.
TABLE Output only. The type of entry that has a GoogleSQL schema, including logical views.
MODEL Output only. The type of models, examples include https://cloud.google.com/bigquery-ml/docs/bigqueryml-intro
DATA_STREAM Output only. An entry type which is used for streaming entries. Example: Pub/Sub topic.
FILESET An entry type which is a set of files or objects. Example: Cloud Storage fileset.

FieldType

Fields
Union field type_decl. Required. type_decl can be only one of the following:
primitive_type

PrimitiveType

Represents primitive types - string, bool etc.
enum_type

EnumType

Represents an enum type.

EnumType

Fields
allowed_values[]

EnumValue

The set of allowed values for this enum. This set must not be empty, the display names of the values in this set must not be empty and the display names of the values must be case-insensitively unique within this set. The order of items in this list is preserved. This field can be used to create, remove and reorder enum values. In order to rename enum value use RenameTagTemplateFieldEnumValue method. Can have up to 500 allowed values.

EnumValue

Fields
display_name

string

Required. The display name of the enum value. Must not be an empty string.

PrimitiveType

Enums
PRIMITIVE_TYPE_UNSPECIFIED This is the default invalid value for a type.
DOUBLE A double precision number.
STRING An UTF-8 string.
BOOL A boolean value.
TIMESTAMP A timestamp.

GcsFileSpec

Specifications of a single file in Cloud Storage.

Fields
file_path

string

Required. The full file path. Example: gs://bucket_name/a/b.txt.
gcs_timestamps

SystemTimestamps

Output only. Timestamps about the Cloud Storage file.
size_bytes

int64

Output only. The size of the file, in bytes.

GcsFilesetSpec

Describes a Cloud Storage fileset entry.

Fields
file_patterns[]

string

Required. Patterns to identify a set of files in Google Cloud Storage. See Cloud Storage documentation for more information. Note that bucket wildcards are currently not supported.

Examples of valid file_patterns:

  • gs://bucket_name/dir/*: matches all files within bucket_name/dir directory.
  • gs://bucket_name/dir/**: matches all files in bucket_name/dir spanning all subdirectories.
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt

You can combine wildcards to provide more powerful matches, for example:

  • gs://bucket_name/[a-m]??.j*g
sample_gcs_file_specs[]

GcsFileSpec

Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here.

GetEntryGroupRequest

Request message for GetEntryGroup.

Fields
name

string

Required. The name of the entry group. For example, projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}.
read_mask

FieldMask

The fields to return. If not set or empty, all fields are returned.

GetEntryRequest

Request message for GetEntry.

Fields
name

string

Required. The name of the entry. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}

GetTagTemplateRequest

Request message for GetTagTemplate.

Fields
name

string

Required. The name of the tag template. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}

IntegratedSystem

This enum describes all the possible systems that Data Catalog integrates with.

Enums
INTEGRATED_SYSTEM_UNSPECIFIED Default unknown system.
BIGQUERY BigQuery.
CLOUD_PUBSUB Cloud Pub/Sub.

ListEntriesRequest

Request message for ListEntries.

Fields
parent

string

Required. The name of the entry group that contains the entries, which can be provided in URL format. Example:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}
page_size

int32

The maximum number of items to return. Default is 10. Max limit is 1000. Throws an invalid argument for page_size > 1000.
page_token

string

Token that specifies which page is requested. If empty, the first page is returned.
read_mask

FieldMask

The fields to return for each Entry. If not set or empty, all fields are returned. For example, setting read_mask to contain only one path "name" will cause ListEntries to return a list of Entries with only "name" field.

ListEntriesResponse

Response message for ListEntries.

Fields
entries[]

Entry

Entry details.
next_page_token

string

Token to retrieve the next page of results. It is set to empty if no items remain in results.

ListEntryGroupsRequest

Request message for ListEntryGroups.

Fields
parent

string

Required. The name of the location that contains the entry groups, which can be provided in URL format. Example:

  • projects/{project_id}/locations/{location}
page_size

int32

Optional. The maximum number of items to return. Default is 10. Max limit is 1000. Throws an invalid argument for page_size > 1000.
page_token

string

Optional. Token that specifies which page is requested. If empty, the first page is returned.

ListEntryGroupsResponse

Response message for ListEntryGroups.

Fields
entry_groups[]

EntryGroup

EntryGroup details.
next_page_token

string

Token to retrieve the next page of results. It is set to empty if no items remain in results.

ListTagsRequest

Request message for ListTags.

Fields
parent

string

Required. The name of the Data Catalog resource to list the tags of. The resource could be an Entry or an EntryGroup.

Examples:

  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}
  • projects/{project_id}/locations/{location}/entryGroups/{entry_group_id}/entries/{entry_id}
page_size

int32

The maximum number of tags to return. Default is 10. Max limit is 1000.
page_token

string

Token that specifies which page is requested. If empty, the first page is returned.

ListTagsResponse

Response message for ListTags.

Fields
tags[]

Tag

Tag details.
next_page_token

string

Token to retrieve the next page of results. It is set to empty if no items remain in results.

LookupEntryRequest

Request message for LookupEntry.

Fields
Union field target_name. Required. Represents either the Google Cloud Platform resource or SQL name for a Google Cloud Platform resource. target_name can be only one of the following:
linked_resource

string

The full name of the Google Cloud Platform resource the Data Catalog entry represents. See: https://cloud.google.com/apis/design/resource_names#full_resource_name. Full names are case-sensitive.

Examples:

  • //bigquery.googleapis.com/projects/projectId/datasets/datasetId/tables/tableId
  • //pubsub.googleapis.com/projects/projectId/topics/topicId
sql_resource

string

The SQL name of the entry. SQL names are case-sensitive.

Examples:

  • pubsub.project_id.topic_id
  • pubsub.project_id.`topic.id.with.dots`
  • bigquery.table.project_id.dataset_id.table_id
  • bigquery.dataset.project_id.dataset_id
  • datacatalog.entry.project_id.location_id.entry_group_id.entry_id

*_ids shoud satisfy the standard SQL rules for identifiers. https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical.

RenameTagTemplateFieldEnumValueRequest

Request message for RenameTagTemplateFieldEnumValue.

Fields
name

string

Required. The name of the enum field value. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}/fields/{tag_template_field_id}/enumValues/{enum_value_display_name}
new_enum_value_display_name

string

Required. The new display name of the enum value. For example, my_new_enum_value.

RenameTagTemplateFieldRequest

Request message for RenameTagTemplateField.

Fields
name

string

Required. The name of the tag template. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}/fields/{tag_template_field_id}
new_tag_template_field_id

string

Required. The new ID of this tag template field. For example, my_new_field.

Schema

Represents a schema (e.g. BigQuery, GoogleSQL, Avro schema).

Fields
columns[]

ColumnSchema

Required. Schema of columns. A maximum of 10,000 columns and sub-columns can be specified.

SearchCatalogRequest

Request message for SearchCatalog.

Fields
scope

Scope

Required. The scope of this search request. A scope that has empty include_org_ids, include_project_ids AND false include_gcp_public_datasets is considered invalid. Data Catalog will return an error in such a case.
query

string

Required. The query string in search query syntax. The query must be non-empty.

Query strings can be simple as "x" or more qualified as:

  • name:x
  • column:x
  • description:y

Note: Query tokens need to have a minimum of 3 characters for substring matching to work correctly. See Data Catalog Search Syntax for more information.
page_size

int32

Number of results in the search page. If <=0 then defaults to 10. Max limit for page_size is 1000. Throws an invalid argument for page_size > 1000.
page_token

string

Optional. Pagination token returned in an earlier SearchCatalogResponse.next_page_token, which indicates that this is a continuation of a prior SearchCatalogRequest call, and that the system should return the next page of data. If empty, the first page is returned.
order_by

string

Specifies the ordering of results, currently supported case-sensitive choices are:

  • relevance, only supports descending
  • last_modified_timestamp [asc|desc], defaults to descending if not specified

If not specified, defaults to relevance descending.

Scope

The criteria that select the subspace used for query matching.

Fields
include_org_ids[]

string

The list of organization IDs to search within. To find your organization ID, follow instructions in https://cloud.google.com/resource-manager/docs/creating-managing-organization.
include_project_ids[]

string

The list of project IDs to search within. To learn more about the distinction between project names/IDs/numbers, go to https://cloud.google.com/docs/overview/#projects.
include_gcp_public_datasets

bool

If true, include Google Cloud Platform (GCP) public datasets in the search results. Info on GCP public datasets is available at https://cloud.google.com/public-datasets/. By default, GCP public datasets are excluded.
restricted_locations[]

string

Optional. The list of locations to search within. 1. If empty, search will be performed in all locations; 2. If any of the locations are NOT in the valid locations list, error will be returned; 3. Otherwise, search only the given locations for matching results. Typical usage is to leave this field empty. When a location is unreachable as returned in the SearchCatalogResponse.unreachable field, users can repeat the search request with this parameter set to get additional information on the error.

Valid locations: * asia-east1 * asia-east2 * asia-northeast1 * asia-northeast2 * asia-northeast3 * asia-south1 * asia-southeast1 * australia-southeast1 * eu * europe-north1 * europe-west1 * europe-west2 * europe-west3 * europe-west4 * europe-west6 * global * northamerica-northeast1 * southamerica-east1 * us * us-central1 * us-east1 * us-east4 * us-west1 * us-west2

SearchCatalogResponse

Response message for SearchCatalog.

Fields
results[]

SearchCatalogResult

Search results.
next_page_token

string

The token that can be used to retrieve the next page of results.
unreachable[]

string

Unreachable locations. Search result does not include data from those locations. Users can get additional information on the error by repeating the search request with a more restrictive parameter -- setting the value for SearchDataCatalogRequest.scope.include_locations.

SearchCatalogResult

A result that appears in the response of a search request. Each result captures details of one entry that matches the search.

Fields
search_result_type

SearchResultType

Type of the search result. This field can be used to determine which Get method to call to fetch the full resource.
search_result_subtype

string

Sub-type of the search result. This is a dot-delimited description of the resource's full type, and is the same as the value callers would provide in the "type" search facet. Examples: entry.table, entry.dataStream, tagTemplate.
relative_resource_name

string

The relative resource name of the resource in URL format. Examples:

  • projects/{project_id}/locations/{location_id}/entryGroups/{entry_group_id}/entries/{entry_id}
  • projects/{project_id}/tagTemplates/{tag_template_id}
linked_resource

string

The full name of the cloud resource the entry belongs to. See: https://cloud.google.com/apis/design/resource_names#full_resource_name. Example:

  • //bigquery.googleapis.com/projects/projectId/datasets/datasetId/tables/tableId
Union field system. The source system of the entry. Only applicable when search_result_type is ENTRY. system can be only one of the following:
integrated_system

IntegratedSystem

Output only. This field indicates the entry's source system that Data Catalog integrates with, such as BigQuery or Cloud Pub/Sub.
user_specified_system

string

This field indicates the entry's source system that Data Catalog does not integrate with.

SearchResultType

The different types of resources that can be returned in search.

Enums
SEARCH_RESULT_TYPE_UNSPECIFIED Default unknown type.
ENTRY An Entry.
TAG_TEMPLATE A TagTemplate.
ENTRY_GROUP An EntryGroup.

SystemTimestamps

Timestamps about this resource according to a particular system.

Fields
create_time

Timestamp

The creation time of the resource within the given system.
update_time

Timestamp

The last-modified time of the resource within the given system.
expire_time

Timestamp

Output only. The expiration time of the resource within the given system. Currently only apllicable to BigQuery resources.

TableSourceType

Table source type.

Enums
TABLE_SOURCE_TYPE_UNSPECIFIED Default unknown type.
BIGQUERY_VIEW Table view.
BIGQUERY_TABLE BigQuery native table.
BIGQUERY_MATERIALIZED_VIEW BigQuery materialized view.

TableSpec

Normal BigQuery table spec.

Fields
grouped_entry

string

Output only. If the table is a dated shard, i.e., with name pattern [prefix]YYYYMMDD, grouped_entry is the Data Catalog resource name of the date sharded grouped entry, for example, projects/{project_id}/locations/{location}/entrygroups/{entry_group_id}/entries/{entry_id}. Otherwise, grouped_entry is empty.

Tag

Tags are used to attach custom metadata to Data Catalog resources. Tags conform to the specifications within their tag template.

See Data Catalog IAM for information on the permissions needed to create or view tags.

Fields
name

string

The resource name of the tag in URL format. Example:

  • projects/{project_id}/locations/{location}/entrygroups/{entry_group_id}/entries/{entry_id}/tags/{tag_id}

where tag_id is a system-generated identifier. Note that this Tag may not actually be stored in the location in this name.
template

string

Required. The resource name of the tag template that this tag uses. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}

This field cannot be modified after creation.
template_display_name

string

Output only. The display name of the tag template.
fields

map<string, TagField>

Required. This maps the ID of a tag field to the value of and additional information about that field. Valid field IDs are defined by the tag's template. A tag must have at least 1 field and at most 500 fields.
column

string

Resources like Entry can have schemas associated with them. This scope allows users to attach tags to an individual column based on that schema.

For attaching a tag to a nested column, use . to separate the column names. Example:

  • outer_column.inner_column

TagField

Contains the value and supporting information for a field within a Tag.

Fields
display_name

string

Output only. The display name of this field.
order

int32

Output only. The order of this field with respect to other fields in this tag. It can be set in Tag. For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order, and field orders within a tag do not have to be sequential.
Union field kind. Required. The value of this field. kind can be only one of the following:
double_value

double

Holds the value for a tag field with double type.
string_value

string

Holds the value for a tag field with string type.
bool_value

bool

Holds the value for a tag field with boolean type.
timestamp_value

Timestamp

Holds the value for a tag field with timestamp type.
enum_value

EnumValue

Holds the value for a tag field with enum type. This value must be one of the allowed values in the definition of this enum.

EnumValue

Holds an enum value.

Fields
display_name

string

The display name of the enum value.

TagTemplate

A tag template defines a tag, which can have one or more typed fields. The template is used to create and attach the tag to GCP resources. Tag template roles provide permissions to create, edit, and use the template. See, for example, the TagTemplate User role, which includes permission to use the tag template to tag resources.

Fields
name

string

The resource name of the tag template in URL format. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}

Note that this TagTemplate and its child resources may not actually be stored in the location in this name.
display_name

string

The display name for this template. Defaults to an empty string.
fields

map<string, TagTemplateField>

Required. Map of tag template field IDs to the settings for the field. This map is an exhaustive list of the allowed fields. This map must contain at least one field and at most 500 fields.

The keys to this map are tag template field IDs. Field IDs can contain letters (both uppercase and lowercase), numbers (0-9) and underscores (_). Field IDs must be at least 1 character long and at most 64 characters long. Field IDs must start with a letter or underscore.

TagTemplateField

The template for an individual field within a tag template.

Fields
name

string

Output only. The resource name of the tag template field in URL format. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template}/fields/{field}

Note that this TagTemplateField may not actually be stored in the location in this name.
display_name

string

The display name for this field. Defaults to an empty string.
type

FieldType

Required. The type of value this tag field can contain.
is_required

bool

Whether this is a required field. Defaults to false.
order

int32

The order of this field with respect to other fields in this tag template. For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order, and field orders within a tag do not have to be sequential.

UpdateEntryGroupRequest

Request message for UpdateEntryGroup.

Fields
entry_group

EntryGroup

Required. The updated entry group. "name" field must be set.
update_mask

FieldMask

The fields to update on the entry group. If absent or empty, all modifiable fields are updated.

UpdateEntryRequest

Request message for UpdateEntry.

Fields
entry

Entry

Required. The updated entry. The "name" field must be set.
update_mask

FieldMask

The fields to update on the entry. If absent or empty, all modifiable fields are updated.

The following fields are modifiable: * For entries with type DATA_STREAM: * schema * For entries with type FILESET * schema * display_name * description * gcs_fileset_spec * gcs_fileset_spec.file_patterns * For entries with user_specified_type * schema * display_name * description * user_specified_type * user_specified_system * linked_resource * source_system_timestamps

UpdateTagRequest

Request message for UpdateTag.

Fields
tag

Tag

Required. The updated tag. The "name" field must be set.
update_mask

FieldMask

The fields to update on the Tag. If absent or empty, all modifiable fields are updated. Currently the only modifiable field is the field fields.

UpdateTagTemplateFieldRequest

Request message for UpdateTagTemplateField.

Fields
name

string

Required. The name of the tag template field. Example:

  • projects/{project_id}/locations/{location}/tagTemplates/{tag_template_id}/fields/{tag_template_field_id}
tag_template_field

TagTemplateField

Required. The template to update.
update_mask

FieldMask

Optional. The field mask specifies the parts of the template to be updated. Allowed fields:

  • display_name
  • type.enum_type
  • is_required

If update_mask is not set or empty, all of the allowed fields above will be updated.

When updating an enum type, the provided values will be merged with the existing values. Therefore, enum values can only be added, existing enum values cannot be deleted nor renamed. Updating a template field from optional to required is NOT allowed.

UpdateTagTemplateRequest

Request message for UpdateTagTemplate.

Fields
tag_template

TagTemplate

Required. The template to update. The "name" field must be set.
update_mask

FieldMask

The field mask specifies the parts of the template to overwrite.

Allowed fields:

  • display_name

If absent or empty, all of the allowed fields above will be updated.

ViewSpec

Table view specification.

Fields
view_query

string

Output only. The query that defines the table view.