REST Resource: projects.locations.entryGroups.entries

Resource: Entry

Entry metadata. A Data Catalog entry represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic) or outside of it. You can use the linkedResource field in the entry resource to refer to the original resource ID of the source system.

An entry resource contains resource details, for example, its schema. Additionally, you can attach flexible metadata to an entry in the form of a Tag.

JSON representation 
{
  "name": string,
  "linkedResource": string,
  "fullyQualifiedName": string,
  "displayName": string,
  "description": string,
  "schema": {
    object (Schema)
  },
  "sourceSystemTimestamps": {
    object (SystemTimestamps)
  },
  "dataSource": {
    object (DataSource)
  },

  // Union field entry_type can be only one of the following:
  "type": enum (EntryType),
  "userSpecifiedType": string
  // End of list of possible types for union field entry_type.

  // Union field system can be only one of the following:
  "integratedSystem": enum (IntegratedSystem),
  "userSpecifiedSystem": string
  // End of list of possible types for union field system.

  // Union field type_spec can be only one of the following:
  "gcsFilesetSpec": {
    object (GcsFilesetSpec)
  },
  "bigqueryTableSpec": {
    object (BigQueryTableSpec)
  },
  "bigqueryDateShardedSpec": {
    object (BigQueryDateShardedSpec)
  }
  // End of list of possible types for union field type_spec.
  "databaseTableSpec": {
    object (DatabaseTableSpec)
  }
}
Fields
name

string

Output only. The resource name of an entry in URL format.

Note: The entry itself and its child resources might not be stored in the location specified in its name.

linkedResource

string

The resource this metadata entry refers to.

For Google Cloud Platform resources, linkedResource is the Full Resource Name. For example, the linkedResource for a table resource from BigQuery is:

//bigquery.googleapis.com/projects/{PROJECT_ID}/datasets/{DATASET_ID}/tables/{TABLE_ID}

Output only when the entry is one of the types in the EntryType enum.

For entries with a userSpecifiedType, this field is optional and defaults to an empty string.

The resource string must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), periods (.), colons (:), slashes (/), dashes (-), and hashes (#). The maximum size is 200 bytes when encoded in UTF-8.
fullyQualifiedName

string

Fully qualified name (FQN) of the resource. Set automatically for entries representing resources from synced systems. Settable only during creation and read-only afterwards. Can be used for search and lookup of the entries.

FQNs take two forms:

  • For non-regionalized resources:

{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  • For regionalized resources:

{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

Example for a DPMS table:

dataproc_metastore:{PROJECT_ID}.{LOCATION_ID}.{INSTANCE_ID}.{DATABASE_ID}.{TABLE_ID}
displayName

string

Display name of an entry.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum size is 200 bytes when encoded in UTF-8. Default value is an empty string.
description

string

Entry description that can consist of several sentences or paragraphs that describe entry contents.

The description must not contain Unicode non-characters as well as C0 and C1 control codes except tabs (HT), new lines (LF), carriage returns (CR), and page breaks (FF). The maximum size is 2000 bytes when encoded in UTF-8. Default value is an empty string.
schema

object (Schema)

Schema of the entry. An entry might not have any schema attached to it.
sourceSystemTimestamps

object (SystemTimestamps)

Timestamps from the underlying resource, not from the Data Catalog entry.

Output only when the entry has a type listed in the EntryType enum. For entries with userSpecifiedType, this field is optional and defaults to an empty timestamp.
dataSource

object (DataSource)

Output only. Physical location of the entry.
Union field entry_type. Required. Entry type. entry_type can be only one of the following:
type

enum (EntryType)

The type of the entry. Only used for entries with types listed in the EntryType enum.

Currently, only FILESET enum value is allowed. All other entries created in Data Catalog must use the userSpecifiedType.
userSpecifiedType

string

Custom entry type that doesn't match any of the values allowed for input and listed in the EntryType enum.

When creating an entry, first check the type values in the enum. If there are no appropriate types for the new entry, provide a custom value, for example, my_special_type.

The userSpecifiedType string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field system. The source system of the entry. system can be only one of the following:
integratedSystem

enum (IntegratedSystem)

Output only. Indicates the entry's source system that Data Catalog integrates with, such as BigQuery, Pub/Sub, or Dataproc Metastore.
userSpecifiedSystem

string

Indicates the entry's source system that Data Catalog doesn't automatically integrate with.

The userSpecifiedSystem string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field type_spec. Type specification. type_spec can be only one of the following:
gcsFilesetSpec

object (GcsFilesetSpec)

Specification that applies to a Cloud Storage fileset. Valid only for entries with the FILESET type.
bigqueryTableSpec

object (BigQueryTableSpec)

Specification that applies to a BigQuery table. Valid only for entries with the TABLE type.
bigqueryDateShardedSpec

object (BigQueryDateShardedSpec)

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.
databaseTableSpec

object (DatabaseTableSpec)

Specification that applies to a table resource. Valid only for entries with the TABLE type.

EntryType

The enum field that lists all the types of entry resources in Data Catalog. For example, a BigQuery table entry has the TABLE type.

Enums
ENTRY_TYPE_UNSPECIFIED Default unknown type.
TABLE Output only. The entry type that has a GoogleSQL schema, including logical views.
MODEL

Output only. The type of models.

For more information, see Supported models in BigQuery ML.
DATA_STREAM An entry type for streaming entries. For example, a Pub/Sub topic.
FILESET An entry type for a set of files or objects. For example, a Cloud Storage fileset.
DATABASE A database.
SERVICE A service, for example, a Dataproc Metastore service.

IntegratedSystem

This enum lists all the systems that Data Catalog integrates with.

Enums
INTEGRATED_SYSTEM_UNSPECIFIED Default unknown system.
BIGQUERY BigQuery.
CLOUD_PUBSUB Cloud Pub/Sub.
DATAPROC_METASTORE Dataproc Metastore.

GcsFilesetSpec

Describes a Cloud Storage fileset entry.

JSON representation 
{
  "filePatterns": [
    string
  ],
  "sampleGcsFileSpecs": [
    {
      object (GcsFileSpec)
    }
  ]
}
Fields
filePatterns[]

string

Required. Patterns to identify a set of files in Google Cloud Storage.

For more information, see Wildcard Names.

Note: Currently, bucket wildcards are not supported.

Examples of valid filePatterns:

  • gs://bucket_name/dir/*: matches all files in bucket_name/dir directory
  • gs://bucket_name/dir/**: matches all files in bucket_name/dir and all subdirectories
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match the a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt

You can combine wildcards to match complex sets of files, for example:

gs://bucket_name/[a-m]??.j*g
sampleGcsFileSpecs[]

object (GcsFileSpec)

Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here.

GcsFileSpec

Specification of a single file in Cloud Storage.

JSON representation 
{
  "filePath": string,
  "gcsTimestamps": {
    object (SystemTimestamps)
  },
  "sizeBytes": string
}
Fields
filePath

string

Required. Full file path. Example: gs://bucket_name/a/b.txt.
gcsTimestamps

object (SystemTimestamps)

Output only. Creation, modification, and expiration timestamps of a Cloud Storage file.
sizeBytes

string (int64 format)

Output only. File size in bytes.

BigQueryTableSpec

Describes a BigQuery table.

JSON representation 
{
  "tableSourceType": enum (TableSourceType),

  // Union field type_spec can be only one of the following:
  "viewSpec": {
    object (ViewSpec)
  },
  "tableSpec": {
    object (TableSpec)
  }
  // End of list of possible types for union field type_spec.
}
Fields
tableSourceType

enum (TableSourceType)

Output only. The table source type.
Union field type_spec. Output only. type_spec can be only one of the following:
viewSpec

object (ViewSpec)

Table view specification. Populated only if the tableSourceType is BIGQUERY_VIEW.
tableSpec

object (TableSpec)

Specification of a BigQuery table. Populated only if the tableSourceType is BIGQUERY_TABLE.

TableSourceType

Table source type.

Enums
TABLE_SOURCE_TYPE_UNSPECIFIED Default unknown type.
BIGQUERY_VIEW Table view.
BIGQUERY_TABLE BigQuery native table.
BIGQUERY_MATERIALIZED_VIEW BigQuery materialized view.

ViewSpec

Table view specification.

JSON representation 
{
  "viewQuery": string
}
Fields
viewQuery

string

Output only. The query that defines the table view.

TableSpec

Normal BigQuery table specification.

JSON representation 
{
  "groupedEntry": string
}
Fields
groupedEntry

string

Output only. If the table is date-sharded, that is, it matches the [prefix]YYYYMMDD name pattern, this field is the Data Catalog resource name of the date-sharded grouped entry. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.

Otherwise, groupedEntry is empty.

BigQueryDateShardedSpec

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.

JSON representation 
{
  "dataset": string,
  "tablePrefix": string,
  "shardCount": string
}
Fields
dataset

string

Output only. The Data Catalog resource name of the dataset entry the current table belongs to. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.
tablePrefix

string

Output only. The table name prefix of the shards.

The name of any given shard is [tablePrefix]YYYYMMDD. For example, for the MyTable20180101 shard, the tablePrefix is MyTable.
shardCount

string (int64 format)

Output only. Total number of shards.

DatabaseTableSpec

Specification that applies to a table resource. Valid only for entries with the TABLE type.

JSON representation 
{
  "type": enum (TableType)
}
Fields
type

enum (TableType)

Type of this table.

TableType

Type of the table.

Enums
TABLE_TYPE_UNSPECIFIED Default unknown table type.
NATIVE Native table.
EXTERNAL External table.

Schema

Represents a schema, for example, a BigQuery, GoogleSQL, or Avro schema.

JSON representation 
{
  "columns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
columns[]

object (ColumnSchema)

The unified GoogleSQL-like schema of columns.

The overall maximum number of columns and nested columns is 10,000. The maximum nested depth is 15 levels.

ColumnSchema

A column within a schema. Columns can be nested inside other columns.

JSON representation 
{
  "column": string,
  "type": string,
  "description": string,
  "mode": string,
  "subcolumns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
column

string

Required. Name of the column.

Must be a UTF-8 string without dots (.). The maximum size is 64 bytes.
type

string

Required. Type of the column.

Must be a UTF-8 string with the maximum size of 128 bytes.
description

string

Optional. Description of the column. Default value is an empty string.

The description must be a UTF-8 string with the maximum size of 2000 bytes.
mode

string

Optional. A column's mode indicates whether values in this column are required, nullable, or repeated.

Only NULLABLE, REQUIRED, and REPEATED values are supported. Default mode is NULLABLE.
subcolumns[]

object (ColumnSchema)

Optional. Schema of sub-columns. A column can have zero or more sub-columns.

DataSource

Physical location of an entry.

JSON representation 
{
  "service": enum (Service),
  "resource": string
}
Fields
service

enum (Service)

Service that physically stores the data.
resource

string

Full name of a resource as defined by the service. For example:

//bigquery.googleapis.com/projects/{PROJECT_ID}/locations/{LOCATION}/datasets/{DATASET_ID}/tables/{TABLE_ID}

Service

Name of a service that stores the data.

Enums
SERVICE_UNSPECIFIED Default unknown service.
CLOUD_STORAGE Google Cloud Storage service.
BIGQUERY BigQuery service.

Methods

create

 Creates an entry.

delete

 Deletes an existing entry.

get

 Gets an entry.

getIamPolicy

 Gets the access control policy for a resource.

list

 Lists entries.

patch

 Updates an existing entry.

testIamPermissions

 Gets your permissions on a resource.