Index
DataCatalog
(interface)PolicyTagManager
(interface)PolicyTagManagerSerialization
(interface)BigQueryConnectionSpec
(message)BigQueryConnectionSpec.ConnectionType
(enum)BigQueryDateShardedSpec
(message)BigQueryRoutineSpec
(message)BigQueryTableSpec
(message)BusinessContext
(message)CatalogUIExperience
(enum)CloudBigtableInstanceSpec
(message)CloudBigtableInstanceSpec.CloudBigtableClusterSpec
(message)CloudBigtableSystemSpec
(message)CloudSqlBigQueryConnectionSpec
(message)CloudSqlBigQueryConnectionSpec.DatabaseType
(enum)ColumnSchema
(message)ColumnSchema.FieldElementType
(message)ColumnSchema.IndexingType
(enum)ColumnSchema.LookerColumnSpec
(message)ColumnSchema.LookerColumnSpec.LookerColumnType
(enum)CommonUsageStats
(message)Contacts
(message)Contacts.Person
(message)CreateEntryGroupRequest
(message)CreateEntryRequest
(message)CreatePolicyTagRequest
(message)CreateTagRequest
(message)CreateTagTemplateFieldRequest
(message)CreateTagTemplateRequest
(message)CreateTaxonomyRequest
(message)CrossRegionalSource
(message)DataSource
(message)DataSource.Service
(enum)DataSourceConnectionSpec
(message)DatabaseTableSpec
(message)DatabaseTableSpec.DatabaseViewSpec
(message)DatabaseTableSpec.DatabaseViewSpec.ViewType
(enum)DatabaseTableSpec.TableType
(enum)DataplexExternalTable
(message)DataplexFilesetSpec
(message)DataplexSpec
(message)DataplexTableSpec
(message)DatasetSpec
(message)DeleteEntryGroupRequest
(message)DeleteEntryRequest
(message)DeletePolicyTagRequest
(message)DeleteTagRequest
(message)DeleteTagTemplateFieldRequest
(message)DeleteTagTemplateRequest
(message)DeleteTaxonomyRequest
(message)DumpItem
(message)Entry
(message)EntryGroup
(message)EntryOverview
(message)EntryType
(enum)ExportTaxonomiesRequest
(message)ExportTaxonomiesResponse
(message)FeatureOnlineStoreSpec
(message)FeatureOnlineStoreSpec.StorageType
(enum)FieldType
(message)FieldType.EnumType
(message)FieldType.EnumType.EnumValue
(message)FieldType.PrimitiveType
(enum)FilesetSpec
(message)GcsFileSpec
(message)GcsFilesetSpec
(message)GetEntryGroupRequest
(message)GetEntryRequest
(message)GetPolicyTagRequest
(message)GetTagTemplateRequest
(message)GetTaxonomyRequest
(message)ImportTaxonomiesRequest
(message)ImportTaxonomiesResponse
(message)InlineSource
(message)IntegratedSystem
(enum)ListEntriesRequest
(message)ListEntriesResponse
(message)ListEntryGroupsRequest
(message)ListEntryGroupsResponse
(message)ListPolicyTagsRequest
(message)ListPolicyTagsResponse
(message)ListTagsRequest
(message)ListTagsResponse
(message)ListTaxonomiesRequest
(message)ListTaxonomiesResponse
(message)LookerSystemSpec
(message)LookupEntryRequest
(message)ManagingSystem
(enum)MigrationConfig
(message)ModelSpec
(message)ModifyEntryContactsRequest
(message)ModifyEntryOverviewRequest
(message)OrganizationConfig
(message)PersonalDetails
(message)PhysicalSchema
(message)PhysicalSchema.AvroSchema
(message)PhysicalSchema.CsvSchema
(message)PhysicalSchema.OrcSchema
(message)PhysicalSchema.ParquetSchema
(message)PhysicalSchema.ProtobufSchema
(message)PhysicalSchema.ThriftSchema
(message)PolicyTag
(message)ReconcileTagsMetadata
(message)ReconcileTagsMetadata.ReconciliationState
(enum)ReconcileTagsRequest
(message)ReconcileTagsResponse
(message)RenameTagTemplateFieldEnumValueRequest
(message)RenameTagTemplateFieldRequest
(message)ReplaceTaxonomyRequest
(message)RetrieveConfigRequest
(message)RetrieveEffectiveConfigRequest
(message)RoutineSpec
(message)RoutineSpec.Argument
(message)RoutineSpec.Argument.Mode
(enum)RoutineSpec.RoutineType
(enum)Schema
(message)SearchCatalogRequest
(message)SearchCatalogRequest.Scope
(message)SearchCatalogResponse
(message)SearchCatalogResult
(message)SearchResultType
(enum)SerializedPolicyTag
(message)SerializedTaxonomy
(message)ServiceSpec
(message)SetConfigRequest
(message)SqlDatabaseSystemSpec
(message)StarEntryRequest
(message)StarEntryResponse
(message)StorageProperties
(message)SystemTimestamps
(message)TableSourceType
(enum)TableSpec
(message)Tag
(message)TagField
(message)TagField.EnumValue
(message)TagTemplate
(message)TagTemplate.DataplexTransferStatus
(enum)TagTemplateField
(message)TagTemplateMigration
(enum)TaggedEntry
(message)Taxonomy
(message)Taxonomy.PolicyType
(enum)Taxonomy.Service
(message)UnstarEntryRequest
(message)UnstarEntryResponse
(message)UpdateEntryGroupRequest
(message)UpdateEntryRequest
(message)UpdatePolicyTagRequest
(message)UpdateTagRequest
(message)UpdateTagTemplateFieldRequest
(message)UpdateTagTemplateRequest
(message)UpdateTaxonomyRequest
(message)UsageSignal
(message)UsageStats
(message)VertexDatasetSpec
(message)VertexDatasetSpec.DataType
(enum)VertexModelSourceInfo
(message)VertexModelSourceInfo.ModelSourceType
(enum)VertexModelSpec
(message)ViewSpec
(message)
DataCatalog
Data Catalog API service allows you to discover, understand, and manage your data.
CreateEntry |
---|
Creates an entry. You can create entries only with 'FILESET', 'CLUSTER', 'DATA_STREAM', or custom types. Data Catalog automatically creates entries with other types during metadata ingestion from integrated systems. You must enable the Data Catalog API in the project identified by the An entry group can have a maximum of 100,000 entries.
|
CreateEntryGroup |
---|
Creates an entry group. An entry group contains logically related entries together with Cloud Identity and Access Management policies. These policies specify users who can create, edit, and view entries within entry groups. Data Catalog automatically creates entry groups with names that start with the
You can create your own entry groups for Cloud Storage fileset entries and custom entries together with the corresponding IAM policies. User-created entry groups can't contain the Entry groups, like entries, can be searched. A maximum of 10,000 entry groups may be created per organization across all locations. You must enable the Data Catalog API in the project identified by the
|
CreateTag |
---|
Creates a tag and assigns it to:
Note: The project identified by the
|
CreateTagTemplate |
---|
Creates a tag template. You must enable the Data Catalog API in the project identified by the
|
CreateTagTemplateField |
---|
Creates a field in a tag template. You must enable the Data Catalog API in the project identified by the
|
DeleteEntry |
---|
Deletes an existing entry. You can delete only the entries created by the You must enable the Data Catalog API in the project identified by the
|
DeleteEntryGroup |
---|
Deletes an entry group. You must enable the Data Catalog API in the project identified by the
|
DeleteTag |
---|
Deletes a tag.
|
DeleteTagTemplate |
---|
Deletes a tag template and all tags that use it. You must enable the Data Catalog API in the project identified by the
|
DeleteTagTemplateField |
---|
Deletes a field in a tag template and all uses of this field from the tags based on this template. You must enable the Data Catalog API in the project identified by the
|
GetEntry |
---|
Gets an entry.
|
GetEntryGroup |
---|
Gets an entry group.
|
GetIamPolicy |
---|
Gets the access control policy for a resource. May return:
Supported resources are:
Note: This method doesn't get policies from Google Cloud Platform resources ingested into Data Catalog. To call this method, you must have the following Google IAM permissions:
|
GetTagTemplate |
---|
Gets a tag template.
|
ListEntries |
---|
Lists entries. Note: Currently, this method can list only custom entries. To get a list of both custom and automatically created entries, use
|
ListEntryGroups |
---|
Lists entry groups.
|
ListTags |
---|
Lists tags assigned to an
|
LookupEntry |
---|
Gets an entry by its target resource name. The resource name comes from the source Google Cloud Platform service.
|
ModifyEntryContacts |
---|
Modifies contacts, part of the business context of an To call this method, you must have the
|
ModifyEntryOverview |
---|
Modifies entry overview, part of the business context of an To call this method, you must have the
|
ReconcileTags |
---|
|
RenameTagTemplateField |
---|
Renames a field in a tag template. You must enable the Data Catalog API in the project identified by the
|
RenameTagTemplateFieldEnumValue |
---|
Renames an enum value in a tag template. Within a single enum field, enum values must be unique.
|
RetrieveConfig |
---|
Retrieves the configuration related to the migration from Data Catalog to Dataplex for a specific organization, including all the projects under it which have a separate configuration set.
|
RetrieveEffectiveConfig |
---|
Retrieves the effective configuration related to the migration from Data Catalog to Dataplex for a specific organization or project. If there is no specific configuration set for the resource, the setting is checked hierarchicahlly through the ancestors of the resource, starting from the resource itself.
|
SearchCatalog |
---|
Searches Data Catalog for multiple resources like entries and tags that match a query. This is a Custom Method that doesn't return all information on a resource, only its ID and high level fields. To get more information, you can subsequently call specific get methods. Note: Data Catalog search queries don't guarantee full recall. Results that match your query might not be returned, even in subsequent result pages. Additionally, returned (and not returned) results can vary if you repeat search queries. For more information, see Data Catalog search syntax.
|
SetConfig |
---|
Sets the configuration related to the migration to Dataplex for an organization or project.
|
SetIamPolicy |
---|
Sets an access control policy for a resource. Replaces any existing policy. Supported resources are:
Note: This method sets policies only within Data Catalog and can't be used to manage policies in BigQuery, Pub/Sub, Dataproc Metastore, and any external Google Cloud Platform resources synced with the Data Catalog. To call this method, you must have the following Google IAM permissions:
|
StarEntry |
---|
Marks an
|
TestIamPermissions |
---|
Gets your permissions on a resource. Returns an empty set of permissions if the resource doesn't exist. Supported resources are:
Note: This method gets policies only within Data Catalog and can't be used to get policies from BigQuery, Pub/Sub, Dataproc Metastore, and any external Google Cloud Platform resources ingested into Data Catalog. No Google IAM permissions are required to call this method.
|
UnstarEntry |
---|
Marks an
|
UpdateEntry |
---|
Updates an existing entry. You must enable the Data Catalog API in the project identified by the
|
UpdateEntryGroup |
---|
Updates an entry group. You must enable the Data Catalog API in the project identified by the
|
UpdateTag |
---|
Updates an existing tag.
|
UpdateTagTemplate |
---|
Updates a tag template. You can't update template fields with this method. These fields are separate resources with their own create, update, and delete methods. You must enable the Data Catalog API in the project identified by the
|
UpdateTagTemplateField |
---|
Updates a field in a tag template. You can't update the field type with this method. You must enable the Data Catalog API in the project identified by the
|
PolicyTagManager
Policy Tag Manager API service allows you to manage your policy tags and taxonomies.
Policy tags are used to tag BigQuery columns and apply additional access control policies. A taxonomy is a hierarchical grouping of policy tags that classify data along a common axis.
CreatePolicyTag |
---|
Creates a policy tag in a taxonomy.
|
CreateTaxonomy |
---|
Creates a taxonomy in a specified project. The taxonomy is initially empty, that is, it doesn't contain policy tags.
|
DeletePolicyTag |
---|
Deletes a policy tag together with the following:
|
DeleteTaxonomy |
---|
Deletes a taxonomy, including all policy tags in this taxonomy, their associated policies, and the policy tags references from BigQuery columns.
|
GetIamPolicy |
---|
Gets the IAM policy for a policy tag or a taxonomy.
|
GetPolicyTag |
---|
Gets a policy tag.
|
GetTaxonomy |
---|
Gets a taxonomy.
|
ListPolicyTags |
---|
Lists all policy tags in a taxonomy.
|
ListTaxonomies |
---|
Lists all taxonomies in a project in a particular location that you have a permission to view.
|
SetIamPolicy |
---|
Sets the IAM policy for a policy tag or a taxonomy.
|
TestIamPermissions |
---|
Returns your permissions on a specified policy tag or taxonomy.
|
UpdatePolicyTag |
---|
Updates a policy tag, including its display name, description, and parent policy tag.
|
UpdateTaxonomy |
---|
Updates a taxonomy, including its display name, description, and activated policy types.
|
PolicyTagManagerSerialization
Policy Tag Manager Serialization API service allows you to manipulate your policy tags and taxonomies in a serialized format.
Taxonomy is a hierarchical group of policy tags.
ExportTaxonomies |
---|
Exports taxonomies in the requested type and returns them, including their policy tags. The requested taxonomies must belong to the same project. This method generates
|
ImportTaxonomies |
---|
Creates new taxonomies (including their policy tags) in a given project by importing from inlined or cross-regional sources. For a cross-regional source, new taxonomies are created by copying from a source in another region. For an inlined source, taxonomies and policy tags are created in bulk using nested protocol buffer structures.
|
ReplaceTaxonomy |
---|
Replaces (updates) a taxonomy and all its policy tags. The taxonomy and its entire hierarchy of policy tags must be represented literally by This operation automatically does the following:
|
BigQueryConnectionSpec
Specification for the BigQuery connection.
Fields | |
---|---|
connection_ |
The type of the BigQuery connection. |
has_ |
True if there are credentials attached to the BigQuery connection; false otherwise. |
Union field
|
|
cloud_ |
Specification for the BigQuery connection to a Cloud SQL instance. |
ConnectionType
The type of the BigQuery connection.
Enums | |
---|---|
CONNECTION_TYPE_UNSPECIFIED |
Unspecified type. |
CLOUD_SQL |
Cloud SQL connection. |
BigQueryDateShardedSpec
Specification for a group of BigQuery tables with the [prefix]YYYYMMDD
name pattern.
For more information, see Introduction to partitioned tables.
Fields | |
---|---|
dataset |
Output only. The Data Catalog resource name of the dataset entry the current table belongs to. For example:
|
table_ |
Output only. The table name prefix of the shards. The name of any given shard is |
shard_ |
Output only. Total number of shards. |
latest_ |
Output only. BigQuery resource name of the latest shard. |
BigQueryRoutineSpec
Fields specific for BigQuery routines.
Fields | |
---|---|
imported_ |
Paths of the imported libraries. |
BigQueryTableSpec
Describes a BigQuery table.
Fields | |
---|---|
table_ |
Output only. The table source type. |
Union field type_spec . Output only. type_spec can be only one of the following: |
|
view_ |
Table view specification. Populated only if the |
table_ |
Specification of a BigQuery table. Populated only if the |
BusinessContext
Business Context of the entry.
Fields | |
---|---|
entry_ |
Entry overview fields for rich text descriptions of entries. |
contacts |
Contact people for the entry. |
CatalogUIExperience
Configuration related to the opt-in status for the UI switch to Dataplex.
Enums | |
---|---|
CATALOG_UI_EXPERIENCE_UNSPECIFIED |
Default value. The default UI is Dataplex. |
CATALOG_UI_EXPERIENCE_ENABLED |
The UI is Dataplex. |
CATALOG_UI_EXPERIENCE_DISABLED |
The UI is Data Catalog. |
CloudBigtableInstanceSpec
Specification that applies to Instance entries that are part of CLOUD_BIGTABLE
system. (user_specified_type)
Fields | |
---|---|
cloud_ |
The list of clusters for the Instance. |
CloudBigtableClusterSpec
Spec that applies to clusters of an Instance of Cloud Bigtable.
Fields | |
---|---|
display_ |
Name of the cluster. |
location |
Location of the cluster, typically a Cloud zone. |
type |
Type of the resource. For a cluster this would be "CLUSTER". |
linked_ |
A link back to the parent resource, in this case Instance. |
CloudBigtableSystemSpec
Specification that applies to all entries that are part of CLOUD_BIGTABLE
system (user_specified_type)
Fields | |
---|---|
instance_ |
Display name of the Instance. This is user specified and different from the resource name. |
CloudSqlBigQueryConnectionSpec
Specification for the BigQuery connection to a Cloud SQL instance.
Fields | |
---|---|
instance_ |
Cloud SQL instance ID in the format of |
database |
Database name. |
type |
Type of the Cloud SQL database. |
DatabaseType
Supported Cloud SQL database types.
Enums | |
---|---|
DATABASE_TYPE_UNSPECIFIED |
Unspecified database type. |
POSTGRES |
Cloud SQL for PostgreSQL. |
MYSQL |
Cloud SQL for MySQL. |
ColumnSchema
A column within a schema. Columns can be nested inside other columns.
Fields | |
---|---|
column |
Required. Name of the column. Must be a UTF-8 string without dots (.). The maximum size is 64 bytes. |
type |
Required. Type of the column. Must be a UTF-8 string with the maximum size of 128 bytes. |
description |
Optional. Description of the column. Default value is an empty string. The description must be a UTF-8 string with the maximum size of 2000 bytes. |
mode |
Optional. A column's mode indicates whether values in this column are required, nullable, or repeated. Only |
default_ |
Optional. Default value for the column. |
ordinal_ |
Optional. Ordinal position |
highest_ |
Optional. Most important inclusion of this column. |
subcolumns[] |
Optional. Schema of sub-columns. A column can have zero or more sub-columns. |
range_ |
Optional. The subtype of the RANGE, if the type of this field is RANGE. If the type is RANGE, this field is required. Possible values for the field element type of a RANGE include: * DATE * DATETIME * TIMESTAMP |
gc_ |
Optional. Garbage collection policy for the column or column family. Applies to systems like Cloud Bigtable. |
Union field system_spec . Information only applying for columns in Entries from a specific system. system_spec can be only one of the following: |
|
looker_ |
Looker specific column info of this column. |
FieldElementType
Represents the type of a field element.
Fields | |
---|---|
type |
Required. The type of a field element. See |
IndexingType
Specifies inclusion of the column in an index
Enums | |
---|---|
INDEXING_TYPE_UNSPECIFIED |
Unspecified. |
INDEXING_TYPE_NONE |
Column not a part of an index. |
INDEXING_TYPE_NON_UNIQUE |
Column Part of non unique index. |
INDEXING_TYPE_UNIQUE |
Column part of unique index. |
INDEXING_TYPE_PRIMARY_KEY |
Column part of the primary key. |
LookerColumnSpec
Column info specific to Looker System.
Fields | |
---|---|
type |
Looker specific column type of this column. |
LookerColumnType
Column type in Looker.
Enums | |
---|---|
LOOKER_COLUMN_TYPE_UNSPECIFIED |
Unspecified. |
DIMENSION |
Dimension. |
DIMENSION_GROUP |
Dimension group - parent for Dimension. |
FILTER |
Filter. |
MEASURE |
Measure. |
PARAMETER |
Parameter. |
CommonUsageStats
Common statistics on the entry's usage.
They can be set on any system.
Fields | |
---|---|
view_ |
View count in source system. |
Contacts
Contact people for the entry.
Fields | |
---|---|
people[] |
The list of contact people for the entry. |
Person
A contact person for the entry.
Fields | |
---|---|
designation |
Designation of the person, for example, Data Steward. |
email |
Email of the person in the format of |
CreateEntryGroupRequest
Request message for CreateEntryGroup
.
Fields | |
---|---|
parent |
Required. The names of the project and location that the new entry group belongs to. Note: The entry group itself and its child resources might not be stored in the location specified in its name. |
entry_ |
Required. The ID of the entry group to create. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and must start with a letter or underscore. The maximum size is 64 bytes when encoded in UTF-8. |
entry_ |
The entry group to create. Defaults to empty. |
CreateEntryRequest
Request message for CreateEntry
.
Fields | |
---|---|
parent |
Required. The name of the entry group this entry belongs to. Note: The entry itself and its child resources might not be stored in the location specified in its name. |
entry_ |
Required. The ID of the entry to create. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores (_). The maximum size is 64 bytes when encoded in UTF-8. |
entry |
Required. The entry to create. |
CreatePolicyTagRequest
Request message for CreatePolicyTag
.
Fields | |
---|---|
parent |
Required. Resource name of the taxonomy that the policy tag will belong to. Authorization requires the following IAM permission on the specified resource
|
policy_ |
The policy tag to create. |
CreateTagRequest
Request message for CreateTag
.
Fields | |
---|---|
parent |
Required. The name of the resource to attach this tag to. Tags can be attached to entries or entry groups. An entry can have up to 1000 attached tags. Note: The tag and its child resources might not be stored in the location specified in its name. |
tag |
Required. The tag to create. |
CreateTagTemplateFieldRequest
Request message for CreateTagTemplateField
.
Fields | |
---|---|
parent |
Required. The name of the project and the template location region. |
tag_ |
Required. The ID of the tag template field to create. Note: Adding a required field to an existing template is not allowed. Field IDs can contain letters (both uppercase and lowercase), numbers (0-9), underscores (_) and dashes (-). Field IDs must be at least 1 character long and at most 128 characters long. Field IDs must also be unique within their template. |
tag_ |
Required. The tag template field to create. |
CreateTagTemplateRequest
Request message for CreateTagTemplate
.
Fields | |
---|---|
parent |
Required. The name of the project and the template location region. |
tag_ |
Required. The ID of the tag template to create. The ID must contain only lowercase letters (a-z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum size is 64 bytes when encoded in UTF-8. |
tag_ |
Required. The tag template to create. |
CreateTaxonomyRequest
Request message for CreateTaxonomy
.
Fields | |
---|---|
parent |
Required. Resource name of the project that the taxonomy will belong to. Authorization requires the following IAM permission on the specified resource
|
taxonomy |
The taxonomy to create. |
CrossRegionalSource
Cross-regional source used to import an existing taxonomy into a different region.
Fields | |
---|---|
taxonomy |
Required. The resource name of the source taxonomy to import. |
DataSource
Physical location of an entry.
Fields | |
---|---|
service |
Service that physically stores the data. |
resource |
Full name of a resource as defined by the service. For example:
|
source_ |
Output only. Data Catalog entry name, if applicable. |
Union field
|
|
storage_ |
Detailed properties of the underlying storage. |
Service
Name of a service that stores the data.
Enums | |
---|---|
SERVICE_UNSPECIFIED |
Default unknown service. |
CLOUD_STORAGE |
Google Cloud Storage service. |
BIGQUERY |
BigQuery service. |
DataSourceConnectionSpec
Specification that applies to a data source connection. Valid only for entries with the DATA_SOURCE_CONNECTION
type. Only one of internal specs can be set at the time, and cannot be changed later.
Fields | |
---|---|
bigquery_ |
Output only. Fields specific to BigQuery connections. |
DatabaseTableSpec
Specification that applies to a table resource. Valid only for entries with the TABLE
type.
Fields | |
---|---|
type |
Type of this table. |
dataplex_ |
Output only. Fields specific to a Dataplex table and present only in the Dataplex table entries. |
database_ |
Spec what aplies to tables that are actually views. Not set for "real" tables. |
DatabaseViewSpec
Specification that applies to database view.
Fields | |
---|---|
view_ |
Type of this view. |
Union field source_definition . Definition of the view. source_definition can be only one of the following: |
|
base_ |
Name of a singular table this view reflects one to one. |
sql_ |
SQL query used to generate this view. |
ViewType
Concrete type of the view.
Enums | |
---|---|
VIEW_TYPE_UNSPECIFIED |
Default unknown view type. |
STANDARD_VIEW |
Standard view. |
MATERIALIZED_VIEW |
Materialized view. |
TableType
Type of the table.
Enums | |
---|---|
TABLE_TYPE_UNSPECIFIED |
Default unknown table type. |
NATIVE |
Native table. |
EXTERNAL |
External table. |
DataplexExternalTable
External table registered by Dataplex. Dataplex publishes data discovered from an asset into multiple other systems (BigQuery, DPMS) in form of tables. We call them "external tables". External tables are also synced into the Data Catalog. This message contains pointers to those external tables (fully qualified name, resource name et cetera) within the Data Catalog.
Fields | |
---|---|
system |
Service in which the external table is registered. |
fully_ |
Fully qualified name (FQN) of the external table. |
google_ |
Google Cloud resource name of the external table. |
data_ |
Name of the Data Catalog entry representing the external table. |
DataplexFilesetSpec
Entry specyfication for a Dataplex fileset.
Fields | |
---|---|
dataplex_ |
Common Dataplex fields. |
DataplexSpec
Common Dataplex fields.
Fields | |
---|---|
asset |
Fully qualified resource name of an asset in Dataplex, to which the underlying data source (Cloud Storage bucket or BigQuery dataset) of the entity is attached. |
data_ |
Format of the data. |
compression_ |
Compression format of the data, e.g., zip, gzip etc. |
project_ |
Project ID of the underlying Cloud Storage or BigQuery data. Note that this may not be the same project as the correspondingly Dataplex lake / zone / asset. |
DataplexTableSpec
Entry specification for a Dataplex table.
Fields | |
---|---|
external_ |
List of external tables registered by Dataplex in other systems based on the same underlying data. External tables allow to query this data in those systems. |
dataplex_ |
Common Dataplex fields. |
user_ |
Indicates if the table schema is managed by the user or not. |
DatasetSpec
Specification that applies to a dataset. Valid only for entries with the DATASET
type.
Fields | |
---|---|
Union field system_spec . Fields specific to the source system. system_spec can be only one of the following: |
|
vertex_ |
Vertex AI Dataset specific fields |
DeleteEntryGroupRequest
Request message for DeleteEntryGroup
.
Fields | |
---|---|
name |
Required. The name of the entry group to delete. |
force |
Optional. If true, deletes all entries in the entry group. |
DeleteEntryRequest
Request message for DeleteEntry
.
Fields | |
---|---|
name |
Required. The name of the entry to delete. |
DeletePolicyTagRequest
Request message for DeletePolicyTag
.
Fields | |
---|---|
name |
Required. Resource name of the policy tag to delete. Note: All of its descendant policy tags are also deleted. Authorization requires the following IAM permission on the specified resource
|
DeleteTagRequest
Request message for DeleteTag
.
Fields | |
---|---|
name |
Required. The name of the tag to delete. |
DeleteTagTemplateFieldRequest
Request message for DeleteTagTemplateField
.
Fields | |
---|---|
name |
Required. The name of the tag template field to delete. |
force |
Required. If true, deletes this field from any tags that use it. Currently, |
DeleteTagTemplateRequest
Request message for DeleteTagTemplate
.
Fields | |
---|---|
name |
Required. The name of the tag template to delete. |
force |
Required. If true, deletes all tags that use this template. Currently, |
DeleteTaxonomyRequest
Request message for DeleteTaxonomy
.
Fields | |
---|---|
name |
Required. Resource name of the taxonomy to delete. Note: All policy tags in this taxonomy are also deleted. Authorization requires the following IAM permission on the specified resource
|
DumpItem
Wrapper for any item that can be contained in the dump.
Fields | |
---|---|
Union field
|
|
tagged_ |
Entry and its tags. |
Entry
Entry metadata. A Data Catalog entry represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic) or outside of it. You can use the linked_resource
field in the entry resource to refer to the original resource ID of the source system.
An entry resource contains resource details, for example, its schema. Additionally, you can attach flexible metadata to an entry in the form of a Tag
.
Fields | |
---|---|
name |
Output only. Identifier. The resource name of an entry in URL format. Note: The entry itself and its child resources might not be stored in the location specified in its name. |
linked_ |
The resource this metadata entry refers to. For Google Cloud Platform resources,
Output only when the entry is one of the types in the For entries with a The resource string must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), periods (.), colons (:), slashes (/), dashes (-), and hashes (#). The maximum size is 200 bytes when encoded in UTF-8. |
fully_ |
Fully Qualified Name (FQN) of the resource. Set automatically for entries representing resources from synced systems. Settable only during creation, and read-only later. Can be used for search and lookup of the entries. |
display_ |
Display name of an entry. The maximum size is 500 bytes when encoded in UTF-8. Default value is an empty string. |
description |
Entry description that can consist of several sentences or paragraphs that describe entry contents. The description must not contain Unicode non-characters as well as C0 and C1 control codes except tabs (HT), new lines (LF), carriage returns (CR), and page breaks (FF). The maximum size is 2000 bytes when encoded in UTF-8. Default value is an empty string. |
business_ |
Business Context of the entry. Not supported for BigQuery datasets |
schema |
Schema of the entry. An entry might not have any schema attached to it. |
source_ |
Timestamps from the underlying resource, not from the Data Catalog entry. Output only when the entry has a system listed in the |
usage_ |
Resource usage statistics. |
labels |
Cloud labels attached to the entry. In Data Catalog, you can create and modify labels attached only to custom entries. Synced entries have unmodifiable labels that come from the source system. |
data_ |
Output only. Physical location of the entry. |
personal_ |
Output only. Additional information related to the entry. Private to the current user. |
Union field entry_type . Required. Entry type. entry_type can be only one of the following: |
|
type |
The type of the entry. For details, see |
user_ |
Custom entry type that doesn't match any of the values allowed for input and listed in the When creating an entry, first check the type values in the enum. If there are no appropriate types for the new entry, provide a custom value, for example, The
|
Union field system . The source system of the entry. system can be only one of the following: |
|
integrated_ |
Output only. Indicates the entry's source system that Data Catalog integrates with, such as BigQuery, Pub/Sub, or Dataproc Metastore. |
user_ |
Indicates the entry's source system that Data Catalog doesn't automatically integrate with. The
|
Union field system_spec . System specification. Can be used as a complement for spec , when some metadata is relevant for all entries existing within given system system_spec can be only one of the following: |
|
sql_ |
Specification that applies to a relational database system. Only settable when |
looker_ |
Specification that applies to Looker sysstem. Only settable when |
cloud_ |
Specification that applies to Cloud Bigtable system. Only settable when |
Union field type_spec . Type specification. type_spec can be only one of the following: |
|
gcs_ |
Specification that applies to a Cloud Storage fileset. Valid only for entries with the |
bigquery_ |
Output only. Specification that applies to a BigQuery table. Valid only for entries with the |
bigquery_ |
Output only. Specification for a group of BigQuery tables with the For more information, see Introduction to partitioned tables. |
Union field When extending the API with new types and systems, use this field instead of the legacy |
|
database_ |
Specification that applies to a table resource. Valid only for entries with the |
data_ |
Specification that applies to a data source connection. Valid only for entries with the |
routine_ |
Specification that applies to a user-defined function or procedure. Valid only for entries with the |
dataset_ |
Specification that applies to a dataset. |
fileset_ |
Specification that applies to a fileset resource. Valid only for entries with the |
service_ |
Specification that applies to a Service resource. |
model_ |
Model specification. |
feature_ |
FeatureonlineStore spec for Vertex AI Feature Store. |
EntryGroup
Entry group metadata.
An EntryGroup
resource represents a logical grouping of zero or more Data Catalog Entry
resources.
Fields | |
---|---|
name |
Identifier. The resource name of the entry group in URL format. Note: The entry group itself and its child resources might not be stored in the location specified in its name. |
display_ |
A short name to identify the entry group, for example, "analytics data - jan 2011". Default value is an empty string. |
description |
Entry group description. Can consist of several sentences or paragraphs that describe the entry group contents. Default value is an empty string. |
data_ |
Output only. Timestamps of the entry group. Default value is empty. |
transferred_ |
Optional. When set to [true], it means DataCatalog EntryGroup was transferred to Dataplex Catalog Service. It makes EntryGroup and its Entries to be read-only in DataCatalog. However, new Tags on EntryGroup and its Entries can be created. After setting the flag to [true] it cannot be unset. |
EntryOverview
Entry overview fields for rich text descriptions of entries.
Fields | |
---|---|
overview |
Entry overview with support for rich text. The overview must only contain Unicode characters, and should be formatted using HTML. The maximum length is 10 MiB as this value holds HTML descriptions including encoded images. The maximum length of the text without images is 100 KiB. |
EntryType
Metadata automatically ingested from Google Cloud resources like BigQuery tables or Pub/Sub topics always uses enum values from EntryType
as the type of entry.
Other sources of metadata like Hive or Oracle databases can identify the type by either using one of the enum values from EntryType
(for example, FILESET
for a Cloud Storage fileset) or specifying a custom value using the Entry
field user_specified_type
. For more information, see Surface files from Cloud Storage with fileset entries or Create custom entries for your data sources.
Enums | |
---|---|
ENTRY_TYPE_UNSPECIFIED |
Default unknown type. |
TABLE |
The entry type that has a GoogleSQL schema, including logical views. |
MODEL |
The type of models. For more information, see Supported models in BigQuery ML. |
DATA_STREAM |
An entry type for streaming entries. For example, a Pub/Sub topic. |
FILESET |
An entry type for a set of files or objects. For example, a Cloud Storage fileset. |
CLUSTER |
A group of servers that work together. For example, a Kafka cluster. |
DATABASE |
A database. |
DATA_SOURCE_CONNECTION |
Connection to a data source. For example, a BigQuery connection. |
ROUTINE |
Routine, for example, a BigQuery routine. |
LAKE |
A Dataplex lake. |
ZONE |
A Dataplex zone. |
SERVICE |
A service, for example, a Dataproc Metastore service. |
DATABASE_SCHEMA |
Schema within a relational database. |
DASHBOARD |
A Dashboard, for example from Looker. |
EXPLORE |
A Looker Explore. For more information, see Looker Explore API. |
LOOK |
A Looker Look. For more information, see Looker Look API. |
FEATURE_ONLINE_STORE |
Feature Online Store resource in Vertex AI Feature Store. |
FEATURE_VIEW |
Feature View resource in Vertex AI Feature Store. |
FEATURE_GROUP |
Feature Group resource in Vertex AI Feature Store. |
ExportTaxonomiesRequest
Request message for ExportTaxonomies
.
Fields | |
---|---|
parent |
Required. Resource name of the project that the exported taxonomies belong to. Authorization requires the following IAM permission on the specified resource
|
taxonomies[] |
Required. Resource names of the taxonomies to export. |
Union field destination . Required. Export destination for taxonomies. destination can be only one of the following: |
|
serialized_ |
Serialized export taxonomies that contain all the policy tags as nested protocol buffers. |
ExportTaxonomiesResponse
Response message for ExportTaxonomies
.
Fields | |
---|---|
taxonomies[] |
List of taxonomies and policy tags as nested protocol buffers. |
FeatureOnlineStoreSpec
Detail description of the source information of a Vertex Feature Online Store.
Fields | |
---|---|
storage_ |
Output only. Type of underelaying storage for the FeatureOnlineStore. |
StorageType
Type of underlaying storage type.
Enums | |
---|---|
STORAGE_TYPE_UNSPECIFIED |
Should not be used. |
BIGTABLE |
Underlsying storgae is Bigtable. |
OPTIMIZED |
Underlaying is optimized online server (Lightning). |
FieldType
Fields | |
---|---|
Union field type_decl . Required. type_decl can be only one of the following: |
|
primitive_ |
Primitive types, such as string, boolean, etc. |
enum_ |
An enum type. |
EnumType
Fields | |
---|---|
allowed_ |
The set of allowed values for this enum. This set must not be empty and can include up to 100 allowed values. The display names of the values in this set must not be empty and must be case-insensitively unique within this set. The order of items in this set is preserved. This field can be used to create, remove, and reorder enum values. To rename enum values, use the |
EnumValue
Fields | |
---|---|
display_ |
Required. The display name of the enum value. Must not be an empty string. The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters. |
PrimitiveType
Enums | |
---|---|
PRIMITIVE_TYPE_UNSPECIFIED |
The default invalid value for a type. |
DOUBLE |
A double precision number. |
STRING |
An UTF-8 string. |
BOOL |
A boolean value. |
TIMESTAMP |
A timestamp. |
RICHTEXT |
A Richtext description. |
FilesetSpec
Specification that applies to a fileset. Valid only for entries with the 'FILESET' type.
Fields | |
---|---|
dataplex_ |
Fields specific to a Dataplex fileset and present only in the Dataplex fileset entries. |
GcsFileSpec
Specification of a single file in Cloud Storage.
Fields | |
---|---|
file_ |
Required. Full file path. Example: |
gcs_ |
Output only. Creation, modification, and expiration timestamps of a Cloud Storage file. |
size_ |
Output only. File size in bytes. |
GcsFilesetSpec
Describes a Cloud Storage fileset entry.
Fields | |
---|---|
file_ |
Required. Patterns to identify a set of files in Google Cloud Storage. For more information, see Wildcard Names. Note: Currently, bucket wildcards are not supported. Examples of valid
You can combine wildcards to match complex sets of files, for example:
|
sample_ |
Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here. |
GetEntryGroupRequest
Request message for GetEntryGroup
.
Fields | |
---|---|
name |
Required. The name of the entry group to get. |
read_ |
The fields to return. If empty or omitted, all fields are returned. |
GetEntryRequest
Request message for GetEntry
.
Fields | |
---|---|
name |
Required. The name of the entry to get. |
GetPolicyTagRequest
Request message for GetPolicyTag
.
Fields | |
---|---|
name |
Required. Resource name of the policy tag. Authorization requires the following IAM permission on the specified resource
|
GetTagTemplateRequest
Request message for GetTagTemplate
.
Fields | |
---|---|
name |
Required. The name of the tag template to get. |
GetTaxonomyRequest
Request message for GetTaxonomy
.
Fields | |
---|---|
name |
Required. Resource name of the taxonomy to get. Authorization requires the following IAM permission on the specified resource
|
ImportTaxonomiesRequest
Request message for ImportTaxonomies
.
Fields | |
---|---|
parent |
Required. Resource name of project that the imported taxonomies will belong to. Authorization requires the following IAM permission on the specified resource
|
Union field source . Source taxonomies to import. source can be only one of the following: |
|
inline_ |
Inline source taxonomy to import. |
cross_ |
Cross-regional source taxonomy to import. |
ImportTaxonomiesResponse
Response message for ImportTaxonomies
.
Fields | |
---|---|
taxonomies[] |
Imported taxonomies. |
InlineSource
Inline source containing taxonomies to import.
Fields | |
---|---|
taxonomies[] |
Required. Taxonomies to import. |
IntegratedSystem
This enum lists all the systems that Data Catalog integrates with.
Enums | |
---|---|
INTEGRATED_SYSTEM_UNSPECIFIED |
Default unknown system. |
BIGQUERY |
BigQuery. |
CLOUD_PUBSUB |
Cloud Pub/Sub. |
DATAPROC_METASTORE |
Dataproc Metastore. |
DATAPLEX |
Dataplex. |
CLOUD_SPANNER |
Cloud Spanner |
CLOUD_BIGTABLE |
Cloud Bigtable |
CLOUD_SQL |
Cloud Sql |
LOOKER |
Looker |
VERTEX_AI |
Vertex AI |
ListEntriesRequest
Request message for ListEntries
.
Fields | |
---|---|
parent |
Required. The name of the entry group that contains the entries to list. Can be provided in URL format. |
page_ |
The maximum number of items to return. Default is 10. Maximum limit is 1000. Throws an invalid argument if |
page_ |
Pagination token that specifies the next page to return. If empty, the first page is returned. |
read_ |
The fields to return for each entry. If empty or omitted, all fields are returned. For example, to return a list of entries with only the |
ListEntriesResponse
Response message for ListEntries
.
Fields | |
---|---|
entries[] |
Entry details. |
next_ |
Pagination token of the next results page. Empty if there are no more items in results. |
ListEntryGroupsRequest
Request message for ListEntryGroups
.
Fields | |
---|---|
parent |
Required. The name of the location that contains the entry groups to list. Can be provided as a URL. |
page_ |
Optional. The maximum number of items to return. Default is 10. Maximum limit is 1000. Throws an invalid argument if |
page_ |
Optional. Pagination token that specifies the next page to return. If empty, returns the first page. |
ListEntryGroupsResponse
Response message for ListEntryGroups
.
Fields | |
---|---|
entry_ |
Entry group details. |
next_ |
Pagination token to specify in the next call to retrieve the next page of results. Empty if there are no more items. |
ListPolicyTagsRequest
Request message for ListPolicyTags
.
Fields | |
---|---|
parent |
Required. Resource name of the taxonomy to list the policy tags of. Authorization requires the following IAM permission on the specified resource
|
page_ |
The maximum number of items to return. Must be a value between 1 and 1000 inclusively. If not set, defaults to 50. |
page_ |
The pagination token of the next results page. If not set, returns the first page. The token is returned in the response to a previous list request. |
ListPolicyTagsResponse
Response message for ListPolicyTags
.
Fields | |
---|---|
policy_ |
The policy tags that belong to the taxonomy. |
next_ |
Pagination token of the next results page. Empty if there are no more results in the list. |
ListTagsRequest
Request message for ListTags
.
Fields | |
---|---|
parent |
Required. The name of the Data Catalog resource to list the tags of. The resource can be an |
page_ |
The maximum number of tags to return. Default is 10. Maximum limit is 1000. |
page_ |
Pagination token that specifies the next page to return. If empty, the first page is returned. |
ListTagsResponse
Response message for ListTags
.
Fields | |
---|---|
tags[] |
|
next_ |
Pagination token of the next results page. Empty if there are no more items in results. |
ListTaxonomiesRequest
Request message for ListTaxonomies
.
Fields | |
---|---|
parent |
Required. Resource name of the project to list the taxonomies of. Authorization requires the following IAM permission on the specified resource
|
page_ |
The maximum number of items to return. Must be a value between 1 and 1000 inclusively. If not set, defaults to 50. |
page_ |
The pagination token of the next results page. If not set, the first page is returned. The token is returned in the response to a previous list request. |
filter |
Supported field for filter is 'service' and value is 'dataplex'. Eg: service=dataplex. |
ListTaxonomiesResponse
Response message for ListTaxonomies
.
Fields | |
---|---|
taxonomies[] |
Taxonomies that the project contains. |
next_ |
Pagination token of the next results page. Empty if there are no more results in the list. |
LookerSystemSpec
Specification that applies to entries that are part LOOKER
system (user_specified_type)
Fields | |
---|---|
parent_ |
ID of the parent Looker Instance. Empty if it does not exist. Example value: |
parent_ |
Name of the parent Looker Instance. Empty if it does not exist. |
parent_ |
ID of the parent Model. Empty if it does not exist. |
parent_ |
Name of the parent Model. Empty if it does not exist. |
parent_ |
ID of the parent View. Empty if it does not exist. |
parent_ |
Name of the parent View. Empty if it does not exist. |
LookupEntryRequest
Request message for LookupEntry
.
Fields | |
---|---|
project |
Project where the lookup should be performed. Required to lookup entry that is not a part of |
location |
Location where the lookup should be performed. Required to lookup entry that is not a part of |
Union field target_name . Required. A full name, SQL name, or a fully qualified name of a Google Cloud Platform resource. target_name can be only one of the following: |
|
linked_ |
The full name of the Google Cloud Platform resource the Data Catalog entry represents. For more information, see Full Resource Name. Full names are case-sensitive. For example:
|
sql_ |
The SQL name of the entry. SQL names are case-sensitive. Examples:
Identifiers ( |
fully_ |
Fully Qualified Name (FQN) of the resource. FQNs take two forms:
Example for a DPMS table:
|
ManagingSystem
This enum describes all the systems that manage Taxonomy and PolicyTag resources in DataCatalog.
Enums | |
---|---|
MANAGING_SYSTEM_UNSPECIFIED |
Default value |
MANAGING_SYSTEM_DATAPLEX |
Dataplex. |
MANAGING_SYSTEM_OTHER |
Other |
MigrationConfig
The configuration related to the migration to Dataplex applied to an organization or project. It is the response message for SetConfig
and RetrieveEffectiveConfig
.
Fields | |
---|---|
tag_ |
Opt-in status for the migration of Tag Templates to Dataplex. |
catalog_ |
Opt-in status for the UI switch to Dataplex. |
ModelSpec
Specification that applies to a model. Valid only for entries with the MODEL
type.
Fields | |
---|---|
Union field system_spec . System spec system_spec can be only one of the following: |
|
vertex_ |
Specification for vertex model resources. |
ModifyEntryContactsRequest
Request message for ModifyEntryContacts
.
Fields | |
---|---|
name |
Required. The full resource name of the entry. |
contacts |
Required. The new value for the Contacts. |
ModifyEntryOverviewRequest
Request message for ModifyEntryOverview
.
Fields | |
---|---|
name |
Required. The full resource name of the entry. |
entry_ |
Required. The new value for the Entry Overview. |
OrganizationConfig
The configuration related to the migration from Data Catalog to Dataplex that has been applied to an organization and any projects under it. It is the response message for RetrieveConfig
.
Fields | |
---|---|
config |
Map of organizations and project resource names and their configuration. The format for the map keys is |
PersonalDetails
Entry metadata relevant only to the user and private to them.
Fields | |
---|---|
starred |
True if the entry is starred by the user; false otherwise. |
star_ |
Set if the entry is starred; unset otherwise. |
PhysicalSchema
Native schema used by a resource represented as an entry. Used by query engines for deserializing and parsing source data.
Fields | |
---|---|
Union field
|
|
avro |
Schema in Avro JSON format. |
thrift |
Schema in Thrift format. |
protobuf |
Schema in protocol buffer format. |
parquet |
Marks a Parquet-encoded data source. |
orc |
Marks an ORC-encoded data source. |
csv |
Marks a CSV-encoded data source. |
AvroSchema
Schema in Avro JSON format.
Fields | |
---|---|
text |
JSON source of the Avro schema. |
CsvSchema
This type has no fields.
Marks a CSV-encoded data source.
OrcSchema
This type has no fields.
Marks an ORC-encoded data source.
ParquetSchema
This type has no fields.
Marks a Parquet-encoded data source.
ProtobufSchema
Schema in protocol buffer format.
Fields | |
---|---|
text |
Protocol buffer source of the schema. |
ThriftSchema
Schema in Thrift format.
Fields | |
---|---|
text |
Thrift IDL source of the schema. |
PolicyTag
Denotes one policy tag in a taxonomy, for example, SSN.
Policy tags can be defined in a hierarchy. For example:
+ Geolocation
+ LatLong
+ City
+ ZipCode
Where the "Geolocation" policy tag contains three children.
Fields | |
---|---|
name |
Identifier. Resource name of this policy tag in the URL format. The policy tag manager generates unique taxonomy IDs and policy tag IDs. |
display_ |
Required. User-defined name of this policy tag. The name can't start or end with spaces and must be unique within the parent taxonomy, contain only Unicode letters, numbers, underscores, dashes and spaces, and be at most 200 bytes long when encoded in UTF-8. |
description |
Description of this policy tag. If not set, defaults to empty. The description must contain only Unicode characters, tabs, newlines, carriage returns and page breaks, and be at most 2000 bytes long when encoded in UTF-8. |
parent_ |
Resource name of this policy tag's parent policy tag. If empty, this is a top level tag. If not set, defaults to an empty string. For example, for the "LatLong" policy tag in the example above, this field contains the resource name of the "Geolocation" policy tag, and, for "Geolocation", this field is empty. |
child_ |
Output only. Resource names of child policy tags of this policy tag. |
ReconcileTagsMetadata
Long-running operation
metadata message returned by the ReconcileTags
.
Fields | |
---|---|
state |
State of the reconciliation operation. |
errors |
Maps the name of each tagged column (or empty string for a sole entry) to tagging operation |
ReconciliationState
Enum holding possible states of the reconciliation operation.
Enums | |
---|---|
RECONCILIATION_STATE_UNSPECIFIED |
Default value. This value is unused. |
RECONCILIATION_QUEUED |
The reconciliation has been queued and awaits for execution. |
RECONCILIATION_IN_PROGRESS |
The reconciliation is in progress. |
RECONCILIATION_DONE |
The reconciliation has been finished. |
ReconcileTagsRequest
Request message for ReconcileTags
.
Fields | |
---|---|
parent |
Required. Name of |
tag_ |
Required. The name of the tag template, which is used for reconciliation. |
force_ |
If set to |
tags[] |
A list of tags to apply to an entry. A tag can specify a tag template, which must be the template specified in the |
ReconcileTagsResponse
Long-running operation
response message returned by ReconcileTags
.
Fields | |
---|---|
created_ |
Number of tags created in the request. |
updated_ |
Number of tags updated in the request. |
deleted_ |
Number of tags deleted in the request. |
RenameTagTemplateFieldEnumValueRequest
Request message for RenameTagTemplateFieldEnumValue
.
Fields | |
---|---|
name |
Required. The name of the enum field value. |
new_ |
Required. The new display name of the enum value. For example, |
RenameTagTemplateFieldRequest
Request message for RenameTagTemplateField
.
Fields | |
---|---|
name |
Required. The name of the tag template field. |
new_ |
Required. The new ID of this tag template field. For example, |
ReplaceTaxonomyRequest
Request message for ReplaceTaxonomy
.
Fields | |
---|---|
name |
Required. Resource name of the taxonomy to update. Authorization requires the following IAM permission on the specified resource
|
serialized_ |
Required. Taxonomy to update along with its child policy tags. |
RetrieveConfigRequest
Request message for RetrieveConfig
.
Fields | |
---|---|
name |
Required. The organization whose config is being retrieved. |
RetrieveEffectiveConfigRequest
Request message for RetrieveEffectiveConfig
.
Fields | |
---|---|
name |
Required. The resource whose effective config is being retrieved. |
RoutineSpec
Specification that applies to a routine. Valid only for entries with the ROUTINE
type.
Fields | |
---|---|
routine_ |
The type of the routine. |
language |
The language the routine is written in. The exact value depends on the source system. For BigQuery routines, possible values are:
|
routine_ |
Arguments of the routine. |
return_ |
Return type of the argument. The exact value depends on the source system and the language. |
definition_ |
The body of the routine. |
Union field system_spec . Contains fields specific to the source system. system_spec can be only one of the following: |
|
bigquery_ |
Fields specific for BigQuery routines. |
Argument
Input or output argument of a function or stored procedure.
Fields | |
---|---|
name |
The name of the argument. A return argument of a function might not have a name. |
mode |
Specifies whether the argument is input or output. |
type |
Type of the argument. The exact value depends on the source system and the language. |
Mode
The input or output mode of the argument.
Enums | |
---|---|
MODE_UNSPECIFIED |
Unspecified mode. |
IN |
The argument is input-only. |
OUT |
The argument is output-only. |
INOUT |
The argument is both an input and an output. |
RoutineType
The fine-grained type of the routine.
Enums | |
---|---|
ROUTINE_TYPE_UNSPECIFIED |
Unspecified type. |
SCALAR_FUNCTION |
Non-builtin permanent scalar function. |
PROCEDURE |
Stored procedure. |
Schema
Represents a schema, for example, a BigQuery, GoogleSQL, or Avro schema.
Fields | |
---|---|
columns[] |
The unified GoogleSQL-like schema of columns. The overall maximum number of columns and nested columns is 10,000. The maximum nested depth is 15 levels. |
SearchCatalogRequest
Request message for SearchCatalog
.
Fields | |
---|---|
scope |
Required. The scope of this search request. The |
query |
Optional. The query string with a minimum of 3 characters and specific syntax. For more information, see Data Catalog search syntax. An empty query string returns all data assets (in the specified scope) that you have access to. A query string can be a simple
|
page_ |
Upper bound on the number of results you can get in a single response. Can't be negative or 0, defaults to 10 in this case. The maximum number is 1000. If exceeded, throws an "invalid argument" exception. |
page_ |
Optional. Pagination token that, if specified, returns the next page of search results. If empty, returns the first page. This token is returned in the |
order_ |
Specifies the order of results. Currently supported case-sensitive values are:
Search queries don't guarantee full recall. Results that match your query might not be returned, even in subsequent result pages. Additionally, returned (and not returned) results can vary if you repeat search queries. If you are experiencing recall issues and you don't have to fetch the results in any specific order, consider setting this parameter to If this parameter is omitted, it defaults to the descending |
admin_ |
Optional. If set, use searchAll permission granted on organizations from |
Scope
The criteria that select the subspace used for query matching.
Fields | |
---|---|
include_ |
The list of organization IDs to search within. To find your organization ID, follow the steps from Creating and managing organizations. |
include_ |
The list of project IDs to search within. For more information on the distinction between project names, IDs, and numbers, see Projects. |
include_ |
If See Google Cloud Public Datasets for more information. |
restricted_ |
Optional. The list of locations to search within. If empty, all locations are searched. Returns an error if any location in the list isn't one of the Supported regions. If a location is unreachable, its name is returned in the |
starred_ |
Optional. If By default, all results are returned, starred or not. |
include_public_tag_templates |
Optional. This field is deprecated. The search mechanism for public and private tag templates is the same. |
SearchCatalogResponse
Response message for SearchCatalog
.
Fields | |
---|---|
results[] |
Search results. |
total_ |
The approximate total number of entries matched by the query. |
next_ |
Pagination token that can be used in subsequent calls to retrieve the next page of results. |
unreachable[] |
Unreachable locations. Search results don't include data from those locations. To get additional information on an error, repeat the search request and restrict it to specific locations by setting the |
SearchCatalogResult
Result in the response to a search request.
Each result captures details of one entry that matches the search.
Fields | |
---|---|
search_ |
Type of the search result. You can use this field to determine which get method to call to fetch the full resource. |
search_ |
Sub-type of the search result. A dot-delimited full type of the resource. The same type you specify in the Examples: |
relative_ |
The relative name of the resource in URL format. Examples:
|
linked_ |
The full name of the Google Cloud resource the entry belongs to. For more information, see Full Resource Name. Example:
|
modify_ |
The last modification timestamp of the entry in the source system. |
fully_ |
Fully qualified name (FQN) of the resource. FQNs take two forms:
Example for a DPMS table:
|
display_ |
The display name of the result. |
description |
Entry description that can consist of several sentences or paragraphs that describe entry contents. |
Union field system . The source system of the entry. Applicable only when the search_result_type is ENTRY . system can be only one of the following: |
|
integrated_ |
Output only. The source system that Data Catalog automatically integrates with, such as BigQuery, Cloud Pub/Sub, or Dataproc Metastore. |
user_ |
Custom source system that you can manually integrate Data Catalog with. |
SearchResultType
The resource types that can be returned in search results.
Enums | |
---|---|
SEARCH_RESULT_TYPE_UNSPECIFIED |
Default unknown type. |
ENTRY |
An Entry . |
TAG_TEMPLATE |
A TagTemplate . |
ENTRY_GROUP |
An EntryGroup . |
SerializedPolicyTag
A nested protocol buffer that represents a policy tag and all its descendants.
Fields | |
---|---|
policy_ |
Resource name of the policy tag. This field is ignored when calling |
display_ |
Required. Display name of the policy tag. At most 200 bytes when encoded in UTF-8. |
description |
Description of the serialized policy tag. At most 2000 bytes when encoded in UTF-8. If not set, defaults to an empty description. |
child_ |
Children of the policy tag, if any. |
SerializedTaxonomy
A nested protocol buffer that represents a taxonomy and the hierarchy of its policy tags. Used for taxonomy replacement, import, and export.
Fields | |
---|---|
display_ |
Required. Display name of the taxonomy. At most 200 bytes when encoded in UTF-8. |
description |
Description of the serialized taxonomy. At most 2000 bytes when encoded in UTF-8. If not set, defaults to an empty description. |
policy_ |
Top level policy tags associated with the taxonomy, if any. |
activated_ |
A list of policy types that are activated per taxonomy. |
ServiceSpec
Specification that applies to a Service resource. Valid only for entries with the SERVICE
type.
Fields | |
---|---|
Union field system_spec . System spec system_spec can be only one of the following: |
|
cloud_ |
Specification that applies to Instance entries of |
SetConfigRequest
Request message for SetConfig
.
Fields | |
---|---|
name |
Required. The organization or project whose config is being specified. |
Union field configuration . The configuration field to set. configuration can be only one of the following: |
|
tag_ |
Opt-in status for the migration of Tag Templates to Dataplex. |
catalog_ |
Opt-in status for the UI switch to Dataplex. |
SqlDatabaseSystemSpec
Specification that applies to entries that are part SQL_DATABASE
system (user_specified_type)
Fields | |
---|---|
sql_ |
SQL Database Engine. enum SqlEngine { UNDEFINED = 0; MY_SQL = 1; POSTGRE_SQL = 2; SQL_SERVER = 3; } Engine of the enclosing database instance. |
database_ |
Version of the database engine. |
instance_ |
Host of the SQL database enum InstanceHost { UNDEFINED = 0; SELF_HOSTED = 1; CLOUD_SQL = 2; AMAZON_RDS = 3; AZURE_SQL = 4; } Host of the enclousing database instance. |
StarEntryRequest
Request message for StarEntry
.
Fields | |
---|---|
name |
Required. The name of the entry to mark as starred. |
StarEntryResponse
This type has no fields.
Response message for StarEntry
. Empty for now
StorageProperties
Details the properties of the underlying storage.
Fields | |
---|---|
file_ |
Patterns to identify a set of files for this fileset. Examples of a valid
|
file_ |
File type in MIME format, for example, |
SystemTimestamps
Timestamps associated with this resource in a particular system.
Fields | |
---|---|
create_ |
Creation timestamp of the resource within the given system. |
update_ |
Timestamp of the last modification of the resource or its metadata within a given system. Note: Depending on the source system, not every modification updates this timestamp. For example, BigQuery timestamps every metadata modification but not data or permission changes. |
expire_ |
Output only. Expiration timestamp of the resource within the given system. Currently only applicable to BigQuery resources. |
TableSourceType
Table source type.
Enums | |
---|---|
TABLE_SOURCE_TYPE_UNSPECIFIED |
Default unknown type. |
BIGQUERY_VIEW |
Table view. |
BIGQUERY_TABLE |
BigQuery native table. |
BIGQUERY_MATERIALIZED_VIEW |
BigQuery materialized view. |
TableSpec
Normal BigQuery table specification.
Fields | |
---|---|
grouped_ |
Output only. If the table is date-sharded, that is, it matches the
Otherwise, |
Tag
Tags contain custom metadata and are attached to Data Catalog resources. Tags conform with the specification of their tag template.
See Data Catalog IAM for information on the permissions needed to create or view tags.
Fields | |
---|---|
name |
Identifier. The resource name of the tag in URL format where tag ID is a system-generated identifier. Note: The tag itself might not be stored in the location specified in its name. |
template |
Required. The resource name of the tag template this tag uses. Example:
This field cannot be modified after creation. |
template_ |
Output only. The display name of the tag template. |
fields |
Required. Maps the ID of a tag field to its value and additional information about that field. Tag template defines valid field IDs. A tag must have at least 1 field and at most 500 fields. |
dataplex_ |
Output only. Denotes the transfer status of the Tag Template. |
Union field Deleting the scope from the parent resource deletes all tags attached to that scope. These fields cannot be updated after creation. |
|
column |
Resources like entry can have schemas associated with them. This scope allows you to attach tags to an individual column based on that schema. To attach a tag to a nested column, separate column names with a dot ( |
TagField
Contains the value and additional information on a field within a Tag
.
Fields | |
---|---|
display_ |
Output only. The display name of this field. |
order |
Output only. The order of this field with respect to other fields in this tag. Can be set by For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order, and field orders within a tag don't have to be sequential. |
Union field kind . Required. The value of this field. kind can be only one of the following: |
|
double_ |
The value of a tag field with a double type. |
string_ |
The value of a tag field with a string type. The maximum length is 2000 UTF-8 characters. |
bool_ |
The value of a tag field with a boolean type. |
timestamp_ |
The value of a tag field with a timestamp type. |
enum_ |
The value of a tag field with an enum type. This value must be one of the allowed values listed in this enum. |
richtext_ |
The value of a tag field with a rich text type. The maximum length is 10 MiB as this value holds HTML descriptions including encoded images. The maximum length of the text without images is 100 KiB. |
EnumValue
An enum value.
Fields | |
---|---|
display_ |
The display name of the enum value. |
TagTemplate
A tag template defines a tag that can have one or more typed fields.
The template is used to create tags that are attached to Google Cloud resources. Tag template roles provide permissions to create, edit, and use the template. For example, see the TagTemplate User role that includes a permission to use the tag template to tag resources.
Fields | |
---|---|
name |
Identifier. The resource name of the tag template in URL format. Note: The tag template itself and its child resources might not be stored in the location specified in its name. |
display_ |
Display name for this template. Defaults to an empty string. The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters. |
is_ |
Indicates whether tags created with this template are public. Public tags do not require tag template access to appear in [ListTags][google.cloud.datacatalog.v1.ListTags] API response. Additionally, you can search for a public tag by value with a simple search query in addition to using a |
fields |
Required. Map of tag template field IDs to the settings for the field. This map is an exhaustive list of the allowed fields. The map must contain at least one field and at most 500 fields. The keys to this map are tag template field IDs. The IDs have the following limitations:
|
dataplex_ |
Optional. Transfer status of the TagTemplate |
DataplexTransferStatus
This enum describes TagTemplate transfer status to Dataplex service.
Enums | |
---|---|
DATAPLEX_TRANSFER_STATUS_UNSPECIFIED |
Default value. TagTemplate and its tags are only visible and editable in DataCatalog. |
MIGRATED |
TagTemplate and its tags are auto-copied to Dataplex service. Visible in both services. Editable in DataCatalog, read-only in Dataplex. Deprecated: Individual TagTemplate migration is deprecated in favor of organization or project wide TagTemplate migration opt-in. |
TRANSFERRED |
TagTemplate and its tags are auto-copied to Dataplex service. Visible in both services. Editable in Dataplex, read-only in DataCatalog. |
TagTemplateField
The template for an individual field within a tag template.
Fields | |
---|---|
name |
Identifier. The resource name of the tag template field in URL format. Example:
Note: The tag template field itself might not be stored in the location specified in its name. The name must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_), and must start with a letter or underscore. The maximum length is 64 characters. |
display_ |
The display name for this field. Defaults to an empty string. The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum length is 200 characters. |
type |
Required. The type of value this tag field can contain. |
is_ |
If true, this field is required. Defaults to false. |
description |
The description for this field. Defaults to an empty string. |
order |
The order of this field with respect to other fields in this tag template. For example, a higher value can indicate a more important field. The value can be negative. Multiple fields can have the same order and field orders within a tag don't have to be sequential. |
TagTemplateMigration
Configuration related to the opt-in status for the migration of TagTemplates to Dataplex.
Enums | |
---|---|
TAG_TEMPLATE_MIGRATION_UNSPECIFIED |
Default value. Migration of Tag Templates from Data Catalog to Dataplex is not performed. |
TAG_TEMPLATE_MIGRATION_ENABLED |
Migration of Tag Templates from Data Catalog to Dataplex is enabled. |
TAG_TEMPLATE_MIGRATION_DISABLED |
Migration of Tag Templates from Data Catalog to Dataplex is disabled. |
TaggedEntry
Wrapper containing Entry and information about Tags that should and should not be attached to it.
Fields | |
---|---|
present_ |
Optional. Tags that should be ingested into the Data Catalog. Caller should populate template name, column and fields. |
absent_ |
Optional. Tags that should be deleted from the Data Catalog. Caller should populate template name and column only. |
Union field entry . Required. Entry to be ingested. entry can be only one of the following: |
|
v1_ |
Non-encrypted Data Catalog v1 Entry. |
Taxonomy
A taxonomy is a collection of hierarchical policy tags that classify data along a common axis.
For example, a "data sensitivity" taxonomy might contain the following policy tags:
+ PII
+ Account number
+ Age
+ SSN
+ Zipcode
+ Financials
+ Revenue
A "data origin" taxonomy might contain the following policy tags:
+ User data
+ Employee data
+ Partner data
+ Public data
Fields | |
---|---|
name |
Identifier. Resource name of this taxonomy in URL format. Note: Policy tag manager generates unique taxonomy IDs. |
display_ |
Required. User-defined name of this taxonomy. The name can't start or end with spaces, must contain only Unicode letters, numbers, underscores, dashes, and spaces, and be at most 200 bytes long when encoded in UTF-8. The taxonomy display name must be unique within an organization. |
description |
Optional. Description of this taxonomy. If not set, defaults to empty. The description must contain only Unicode characters, tabs, newlines, carriage returns, and page breaks, and be at most 2000 bytes long when encoded in UTF-8. |
policy_ |
Output only. Number of policy tags in this taxonomy. |
taxonomy_ |
Output only. Creation and modification timestamps of this taxonomy. |
activated_ |
Optional. A list of policy types that are activated for this taxonomy. If not set, defaults to an empty list. |
service |
Output only. Identity of the service which owns the Taxonomy. This field is only populated when the taxonomy is created by a Google Cloud service. Currently only 'DATAPLEX' is supported. |
PolicyType
Defines policy types where the policy tags can be used for.
Enums | |
---|---|
POLICY_TYPE_UNSPECIFIED |
Unspecified policy type. |
FINE_GRAINED_ACCESS_CONTROL |
Fine-grained access control policy that enables access control on tagged sub-resources. |
Service
The source system of the Taxonomy.
Fields | |
---|---|
name |
The Google Cloud service name. |
identity |
The service agent for the service. |
UnstarEntryRequest
Request message for UnstarEntry
.
Fields | |
---|---|
name |
Required. The name of the entry to mark as not starred. |
UnstarEntryResponse
This type has no fields.
Response message for UnstarEntry
. Empty for now
UpdateEntryGroupRequest
Request message for UpdateEntryGroup
.
Fields | |
---|---|
entry_ |
Required. Updates for the entry group. The |
update_ |
Names of fields whose values to overwrite on an entry group. If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied. |
UpdateEntryRequest
Request message for UpdateEntry
.
Fields | |
---|---|
entry |
Required. Updates for the entry. The |
update_ |
Names of fields whose values to overwrite on an entry. If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied. You can modify only the fields listed below. For entries with type
For entries with type
For entries with
|
UpdatePolicyTagRequest
Request message for UpdatePolicyTag
.
Fields | |
---|---|
policy_ |
The policy tag to update. You can update only its description, display name, and parent policy tag fields. Authorization requires the following IAM permission on the specified resource
|
update_ |
Specifies the fields to update. You can update only display name, description, and parent policy tag. If not set, defaults to all updatable fields. For more information, see FieldMask. |
UpdateTagRequest
Request message for UpdateTag
.
Fields | |
---|---|
tag |
Required. The updated tag. The "name" field must be set. |
update_ |
Names of fields whose values to overwrite on a tag. Currently, a tag has the only modifiable field with the name In general, if this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied. |
UpdateTagTemplateFieldRequest
Request message for UpdateTagTemplateField
.
Fields | |
---|---|
name |
Required. The name of the tag template field. |
tag_ |
Required. The template to update. |
update_ |
Optional. Names of fields whose values to overwrite on an individual field of a tag template. The following fields are modifiable:
If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied with one exception: when updating an enum type, the provided values are merged with the existing values. Therefore, enum values can only be added, existing enum values cannot be deleted or renamed. Additionally, updating a template field from optional to required is not allowed. |
UpdateTagTemplateRequest
Request message for UpdateTagTemplate
.
Fields | |
---|---|
tag_ |
Required. The template to update. The |
update_ |
Names of fields whose values to overwrite on a tag template. Currently, only If this parameter is absent or empty, all modifiable fields are overwritten. If such fields are non-required and omitted in the request body, their values are emptied. Note: Updating the |
UpdateTaxonomyRequest
Request message for UpdateTaxonomy
.
Fields | |
---|---|
taxonomy |
The taxonomy to update. You can update only its description, display name, and activated policy types. Authorization requires the following IAM permission on the specified resource
|
update_ |
Specifies fields to update. If not set, defaults to all fields you can update. For more information, see FieldMask. |
UsageSignal
The set of all usage signals that Data Catalog stores.
Note: Usually, these signals are updated daily. In rare cases, an update may fail but will be performed again on the next day.
Fields | |
---|---|
update_ |
The end timestamp of the duration of usage statistics. |
usage_ |
Output only. BigQuery usage statistics over each of the predefined time ranges. Supported time ranges are |
common_ |
Common usage statistics over each of the predefined time ranges. Supported time ranges are |
favorite_ |
Favorite count in the source system. |
UsageStats
Detailed statistics on the entry's usage.
Usage statistics have the following limitations:
- Only BigQuery tables have them.
- They only include BigQuery query jobs.
- They might be underestimated because wildcard table references are not yet counted. For more information, see Querying multiple tables using a wildcard table
Fields | |
---|---|
total_ |
The number of successful uses of the underlying entry. |
total_ |
The number of failed attempts to use the underlying entry. |
total_ |
The number of cancelled attempts to use the underlying entry. |
total_ |
Total time spent only on successful uses, in milliseconds. |
VertexDatasetSpec
Specification for vertex dataset resources.
Fields | |
---|---|
data_ |
The number of DataItems in this Dataset. Only apply for non-structured Dataset. |
data_ |
Type of the dataset. |
DataType
Type of data stored in the dataset.
Enums | |
---|---|
DATA_TYPE_UNSPECIFIED |
Should not be used. |
TABLE |
Structured data dataset. |
IMAGE |
Image dataset which supports ImageClassification, ImageObjectDetection and ImageSegmentation problems. |
TEXT |
Document dataset which supports TextClassification, TextExtraction and TextSentiment problems. |
VIDEO |
Video dataset which supports VideoClassification, VideoObjectTracking and VideoActionRecognition problems. |
CONVERSATION |
Conversation dataset which supports conversation problems. |
TIME_SERIES |
TimeSeries dataset. |
DOCUMENT |
Document dataset which supports DocumentAnnotation problems. |
TEXT_TO_SPEECH |
TextToSpeech dataset which supports TextToSpeech problems. |
TRANSLATION |
Translation dataset which supports Translation problems. |
STORE_VISION |
Store Vision dataset which is used for HITL integration. |
ENTERPRISE_KNOWLEDGE_GRAPH |
Enterprise Knowledge Graph dataset which is used for HITL labeling integration. |
TEXT_PROMPT |
Text prompt dataset which supports Large Language Models. |
VertexModelSourceInfo
Detail description of the source information of a Vertex model.
Fields | |
---|---|
source_ |
Type of the model source. |
copy |
If this Model is copy of another Model. If true then |
ModelSourceType
Source of the model.
Enums | |
---|---|
MODEL_SOURCE_TYPE_UNSPECIFIED |
Should not be used. |
AUTOML |
The Model is uploaded by automl training pipeline. |
CUSTOM |
The Model is uploaded by user or custom training pipeline. |
BQML |
The Model is registered and sync'ed from BigQuery ML. |
MODEL_GARDEN |
The Model is saved or tuned from Model Garden. |
GENIE |
The Model is saved or tuned from Genie. |
CUSTOM_TEXT_EMBEDDING |
The Model is uploaded by text embedding finetuning pipeline. |
MARKETPLACE |
The Model is saved or tuned from Marketplace. |
VertexModelSpec
Specification for vertex model resources.
Fields | |
---|---|
version_ |
The version ID of the model. |
version_ |
User provided version aliases so that a model version can be referenced via alias |
version_ |
The description of this version. |
vertex_ |
Source of a Vertex model. |
container_ |
URI of the Docker image to be used as the custom container for serving predictions. |
ViewSpec
Table view specification.
Fields | |
---|---|
view_ |
Output only. The query that defines the table view. |