REST Resource: projects.locations.entryGroups.entries

Resource: Entry

Entry metadata. A Data Catalog entry represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic) or outside of it. You can use the linkedResource field in the entry resource to refer to the original resource ID of the source system.

An entry resource contains resource details, for example, its schema. Additionally, you can attach flexible metadata to an entry in the form of a Tag.

JSON representation 
{
  "name": string,
  "linkedResource": string,
  "fullyQualifiedName": string,
  "displayName": string,
  "description": string,
  "schema": {
    object (Schema)
  },
  "sourceSystemTimestamps": {
    object (SystemTimestamps)
  },
  "usageSignal": {
    object (UsageSignal)
  },
  "labels": {
    string: string,
    ...
  },
  "dataSource": {
    object (DataSource)
  },

  // Union field entry_type can be only one of the following:
  "type": enum (EntryType),
  "userSpecifiedType": string
  // End of list of possible types for union field entry_type.

  // Union field system can be only one of the following:
  "integratedSystem": enum (IntegratedSystem),
  "userSpecifiedSystem": string
  // End of list of possible types for union field system.

  // Union field type_spec can be only one of the following:
  "gcsFilesetSpec": {
    object (GcsFilesetSpec)
  },
  "bigqueryTableSpec": {
    object (BigQueryTableSpec)
  },
  "bigqueryDateShardedSpec": {
    object (BigQueryDateShardedSpec)
  }
  // End of list of possible types for union field type_spec.

  // Union field spec can be only one of the following:
  "databaseTableSpec": {
    object (DatabaseTableSpec)
  },
  "dataSourceConnectionSpec": {
    object (DataSourceConnectionSpec)
  },
  "routineSpec": {
    object (RoutineSpec)
  }
  // End of list of possible types for union field spec.
}
Fields
name

string

Output only. The resource name of an entry in URL format.

Note: The entry itself and its child resources might not be stored in the location specified in its name.

linkedResource

string

The resource this metadata entry refers to.

For Google Cloud Platform resources, linkedResource is the Full Resource Name. For example, the linkedResource for a table resource from BigQuery is:

//bigquery.googleapis.com/projects/{PROJECT_ID}/datasets/{DATASET_ID}/tables/{TABLE_ID}

Output only when the entry is one of the types in the EntryType enum.

For entries with a userSpecifiedType, this field is optional and defaults to an empty string.

The resource string must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), periods (.), colons (:), slashes (/), dashes (-), and hashes (#). The maximum size is 200 bytes when encoded in UTF-8.
fullyQualifiedName

string

Fully qualified name (FQN) of the resource. Set automatically for entries representing resources from synced systems. Settable only during creation and read-only afterwards. Can be used for search and lookup of the entries.

FQNs take two forms:

  • For non-regionalized resources:

{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  • For regionalized resources:

{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

Example for a DPMS table:

dataproc_metastore:{PROJECT_ID}.{LOCATION_ID}.{INSTANCE_ID}.{DATABASE_ID}.{TABLE_ID}
displayName

string

Display name of an entry.

The name must contain only Unicode letters, numbers (0-9), underscores (_), dashes (-), spaces ( ), and can't start or end with spaces. The maximum size is 200 bytes when encoded in UTF-8. Default value is an empty string.
description

string

Entry description that can consist of several sentences or paragraphs that describe entry contents.

The description must not contain Unicode non-characters as well as C0 and C1 control codes except tabs (HT), new lines (LF), carriage returns (CR), and page breaks (FF). The maximum size is 2000 bytes when encoded in UTF-8. Default value is an empty string.
schema

object (Schema)

Schema of the entry. An entry might not have any schema attached to it.
sourceSystemTimestamps

object (SystemTimestamps)

Timestamps from the underlying resource, not from the Data Catalog entry.

Output only when the entry has a type listed in the EntryType enum. For entries with userSpecifiedType, this field is optional and defaults to an empty timestamp.
usageSignal

object (UsageSignal)

Output only. Resource usage statistics.
labels

map (key: string, value: string)

Cloud labels attached to the entry.

In Data Catalog, you can create and modify labels attached only to custom entries. Synced entries have unmodifiable labels that come from the source system.
dataSource

object (DataSource)

Output only. Physical location of the entry.
Union field entry_type. Required. Entry type. entry_type can be only one of the following:
type

enum (EntryType)

The type of the entry. Only used for entries with types listed in the EntryType enum.

Currently, only FILESET enum value is allowed. All other entries created in Data Catalog must use the userSpecifiedType.
userSpecifiedType

string

Custom entry type that doesn't match any of the values allowed for input and listed in the EntryType enum.

When creating an entry, first check the type values in the enum. If there are no appropriate types for the new entry, provide a custom value, for example, my_special_type.

The userSpecifiedType string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field system. The source system of the entry. system can be only one of the following:
integratedSystem

enum (IntegratedSystem)

Output only. Indicates the entry's source system that Data Catalog integrates with, such as BigQuery, Pub/Sub, or Dataproc Metastore.
userSpecifiedSystem

string

Indicates the entry's source system that Data Catalog doesn't automatically integrate with.

The userSpecifiedSystem string has the following limitations:

  • Is case insensitive.
  • Must begin with a letter or underscore.
  • Can only contain letters, numbers, and underscores.
  • Must be at least 1 character and at most 64 characters long.
Union field type_spec. Type specification. type_spec can be only one of the following:
gcsFilesetSpec

object (GcsFilesetSpec)

Specification that applies to a Cloud Storage fileset. Valid only for entries with the FILESET type.
bigqueryTableSpec

object (BigQueryTableSpec)

Specification that applies to a BigQuery table. Valid only for entries with the TABLE type.
bigqueryDateShardedSpec

object (BigQueryDateShardedSpec)

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.

Union field spec. Type- and system-specific information. Specifications for types contain fields common to all entries of a given type, and sub-specifications with fields specific to a given source system.

When extending the API with new types and systems, use this field instead of the legacy type_spec. spec can be only one of the following:
databaseTableSpec

object (DatabaseTableSpec)

Specification that applies to a table resource. Valid only for entries with the TABLE type.
dataSourceConnectionSpec

object (DataSourceConnectionSpec)

Specification that applies to a data source connection. Valid only for entries with the DATA_SOURCE_CONNECTION type.
routineSpec

object (RoutineSpec)

Specification that applies to a user-defined function or procedure. Valid only for entries with the ROUTINE type.

EntryType

The enum field that lists all the types of entry resources in Data Catalog. For example, a BigQuery table entry has the TABLE type.

Enums
ENTRY_TYPE_UNSPECIFIED Default unknown type.
TABLE Output only. The entry type that has a GoogleSQL schema, including logical views.
MODEL

Output only. The type of models.

For more information, see Supported models in BigQuery ML.
DATA_STREAM An entry type for streaming entries. For example, a Pub/Sub topic.
FILESET An entry type for a set of files or objects. For example, a Cloud Storage fileset.
CLUSTER A group of servers that work together. For example, a Kafka cluster.
DATABASE A database.
DATA_SOURCE_CONNECTION Output only. Connection to a data source. For example, a BigQuery connection.
ROUTINE Output only. Routine, for example, a BigQuery routine.
SERVICE A service, for example, a Dataproc Metastore service.

IntegratedSystem

This enum lists all the systems that Data Catalog integrates with.

Enums
INTEGRATED_SYSTEM_UNSPECIFIED Default unknown system.
BIGQUERY BigQuery.
CLOUD_PUBSUB Cloud Pub/Sub.
DATAPROC_METASTORE Dataproc Metastore.

GcsFilesetSpec

Describes a Cloud Storage fileset entry.

JSON representation 
{
  "filePatterns": [
    string
  ],
  "sampleGcsFileSpecs": [
    {
      object (GcsFileSpec)
    }
  ]
}
Fields
filePatterns[]

string

Required. Patterns to identify a set of files in Google Cloud Storage.

For more information, see Wildcard Names.

Note: Currently, bucket wildcards are not supported.

Examples of valid filePatterns:

  • gs://bucket_name/dir/*: matches all files in bucket_name/dir directory
  • gs://bucket_name/dir/**: matches all files in bucket_name/dir and all subdirectories
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match the a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt

You can combine wildcards to match complex sets of files, for example:

gs://bucket_name/[a-m]??.j*g
sampleGcsFileSpecs[]

object (GcsFileSpec)

Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here.

GcsFileSpec

Specification of a single file in Cloud Storage.

JSON representation 
{
  "filePath": string,
  "gcsTimestamps": {
    object (SystemTimestamps)
  },
  "sizeBytes": string
}
Fields
filePath

string

Required. Full file path. Example: gs://bucket_name/a/b.txt.
gcsTimestamps

object (SystemTimestamps)

Output only. Creation, modification, and expiration timestamps of a Cloud Storage file.
sizeBytes

string (int64 format)

Output only. File size in bytes.

BigQueryTableSpec

Describes a BigQuery table.

JSON representation 
{
  "tableSourceType": enum (TableSourceType),

  // Union field type_spec can be only one of the following:
  "viewSpec": {
    object (ViewSpec)
  },
  "tableSpec": {
    object (TableSpec)
  }
  // End of list of possible types for union field type_spec.
}
Fields
tableSourceType

enum (TableSourceType)

Output only. The table source type.
Union field type_spec. Output only. type_spec can be only one of the following:
viewSpec

object (ViewSpec)

Table view specification. Populated only if the tableSourceType is BIGQUERY_VIEW.
tableSpec

object (TableSpec)

Specification of a BigQuery table. Populated only if the tableSourceType is BIGQUERY_TABLE.

TableSourceType

Table source type.

Enums
TABLE_SOURCE_TYPE_UNSPECIFIED Default unknown type.
BIGQUERY_VIEW Table view.
BIGQUERY_TABLE BigQuery native table.
BIGQUERY_MATERIALIZED_VIEW BigQuery materialized view.

ViewSpec

Table view specification.

JSON representation 
{
  "viewQuery": string
}
Fields
viewQuery

string

Output only. The query that defines the table view.

TableSpec

Normal BigQuery table specification.

JSON representation 
{
  "groupedEntry": string
}
Fields
groupedEntry

string

Output only. If the table is date-sharded, that is, it matches the [prefix]YYYYMMDD name pattern, this field is the Data Catalog resource name of the date-sharded grouped entry. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.

Otherwise, groupedEntry is empty.

BigQueryDateShardedSpec

Specification for a group of BigQuery tables with the [prefix]YYYYMMDD name pattern.

For more information, see Introduction to partitioned tables.

JSON representation 
{
  "dataset": string,
  "tablePrefix": string,
  "shardCount": string
}
Fields
dataset

string

Output only. The Data Catalog resource name of the dataset entry the current table belongs to. For example:

projects/{PROJECT_ID}/locations/{LOCATION}/entrygroups/{ENTRY_GROUP_ID}/entries/{ENTRY_ID}.
tablePrefix

string

Output only. The table name prefix of the shards.

The name of any given shard is [tablePrefix]YYYYMMDD. For example, for the MyTable20180101 shard, the tablePrefix is MyTable.
shardCount

string (int64 format)

Output only. Total number of shards.

DatabaseTableSpec

Specification that applies to a table resource. Valid only for entries with the TABLE type.

JSON representation 
{
  "type": enum (TableType)
}
Fields
type

enum (TableType)

Type of this table.

TableType

Type of the table.

Enums
TABLE_TYPE_UNSPECIFIED Default unknown table type.
NATIVE Native table.
EXTERNAL External table.

DataSourceConnectionSpec

Specification that applies to a data source connection. Valid only for entries with the DATA_SOURCE_CONNECTION type.

JSON representation 
{
  "bigqueryConnectionSpec": {
    object (BigQueryConnectionSpec)
  }
}
Fields
bigqueryConnectionSpec

object (BigQueryConnectionSpec)

Fields specific to BigQuery connections.

BigQueryConnectionSpec

Specification for the BigQuery connection.

JSON representation 
{
  "connectionType": enum (ConnectionType),
  "hasCredential": boolean,
  "cloudSql": {
    object (CloudSqlBigQueryConnectionSpec)
  }
}
Fields
connectionType

enum (ConnectionType)

The type of the BigQuery connection.
hasCredential

boolean

True if there are credentials attached to the BigQuery connection; false otherwise.
cloudSql

object (CloudSqlBigQueryConnectionSpec)

Specification for the BigQuery connection to a Cloud SQL instance.

ConnectionType

The type of the BigQuery connection.

Enums
CONNECTION_TYPE_UNSPECIFIED Unspecified type.
CLOUD_SQL Cloud SQL connection.

CloudSqlBigQueryConnectionSpec

Specification for the BigQuery connection to a Cloud SQL instance.

JSON representation 
{
  "instanceId": string,
  "database": string,
  "type": enum (DatabaseType)
}
Fields
instanceId

string

Cloud SQL instance ID in the format of project:location:instance.
database

string

Database name.
type

enum (DatabaseType)

Type of the Cloud SQL database.

DatabaseType

Supported Cloud SQL database types.

Enums
DATABASE_TYPE_UNSPECIFIED Unspecified database type.
POSTGRES Cloud SQL for PostgreSQL.
MYSQL Cloud SQL for MySQL.

RoutineSpec

Specification that applies to a routine. Valid only for entries with the ROUTINE type.

JSON representation 
{
  "routineType": enum (RoutineType),
  "language": string,
  "routineArguments": [
    {
      object (Argument)
    }
  ],
  "returnType": string,
  "definitionBody": string,
  "bigqueryRoutineSpec": {
    object (BigQueryRoutineSpec)
  }
}
Fields
routineType

enum (RoutineType)

The type of the routine.
language

string

The language the routine is written in. The exact value depends on the source system. For BigQuery routines, possible values are:

  • SQL
  • JAVASCRIPT
routineArguments[]

object (Argument)

Arguments of the routine.
returnType

string

Return type of the argument. The exact value depends on the source system and the language.
definitionBody

string

The body of the routine.
bigqueryRoutineSpec

object (BigQueryRoutineSpec)

Fields specific for BigQuery routines.

RoutineType

The fine-grained type of the routine.

Enums
ROUTINE_TYPE_UNSPECIFIED Unspecified type.
SCALAR_FUNCTION Non-builtin permanent scalar function.
PROCEDURE Stored procedure.

Argument

Input or output argument of a function or stored procedure.

JSON representation 
{
  "name": string,
  "mode": enum (Mode),
  "type": string
}
Fields
name

string

The name of the argument. A return argument of a function might not have a name.
mode

enum (Mode)

Specifies whether the argument is input or output.
type

string

Type of the argument. The exact value depends on the source system and the language.

Mode

The input or output mode of the argument.

Enums
MODE_UNSPECIFIED Unspecified mode.
IN The argument is input-only.
OUT The argument is output-only.
INOUT The argument is both an input and an output.

BigQueryRoutineSpec

Fields specific for BigQuery routines.

JSON representation 
{
  "importedLibraries": [
    string
  ]
}
Fields
importedLibraries[]

string

Paths of the imported libraries.

Schema

Represents a schema, for example, a BigQuery, GoogleSQL, or Avro schema.

JSON representation 
{
  "columns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
columns[]

object (ColumnSchema)

The unified GoogleSQL-like schema of columns.

The overall maximum number of columns and nested columns is 10,000. The maximum nested depth is 15 levels.

ColumnSchema

A column within a schema. Columns can be nested inside other columns.

JSON representation 
{
  "column": string,
  "type": string,
  "description": string,
  "mode": string,
  "subcolumns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
column

string

Required. Name of the column.

Must be a UTF-8 string without dots (.). The maximum size is 64 bytes.
type

string

Required. Type of the column.

Must be a UTF-8 string with the maximum size of 128 bytes.
description

string

Optional. Description of the column. Default value is an empty string.

The description must be a UTF-8 string with the maximum size of 2000 bytes.
mode

string

Optional. A column's mode indicates whether values in this column are required, nullable, or repeated.

Only NULLABLE, REQUIRED, and REPEATED values are supported. Default mode is NULLABLE.
subcolumns[]

object (ColumnSchema)

Optional. Schema of sub-columns. A column can have zero or more sub-columns.

UsageSignal

The set of all usage signals that Data Catalog stores.

Note: Usually, these signals are updated daily. In rare cases, an update may fail but will be performed again on the next day.

JSON representation 
{
  "updateTime": string,
  "usageWithinTimeRange": {
    string: {
      object (UsageStats)
    },
    ...
  }
}
Fields
updateTime

string (Timestamp format)

The end timestamp of the duration of usage statistics.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
usageWithinTimeRange

map (key: string, value: object (UsageStats))

Usage statistics over each of the predefined time ranges.

Supported time ranges are {"24H", "7D", "30D"}.

UsageStats

Detailed statistics on the entry's usage.

Usage statistics have the following limitations:

JSON representation 
{
  "totalCompletions": number,
  "totalFailures": number,
  "totalCancellations": number,
  "totalExecutionTimeForCompletionsMillis": number
}
Fields
totalCompletions

number

The number of successful uses of the underlying entry.
totalFailures

number

The number of failed attempts to use the underlying entry.
totalCancellations

number

The number of cancelled attempts to use the underlying entry.
totalExecutionTimeForCompletionsMillis

number

Total time spent only on successful uses, in milliseconds.

DataSource

Physical location of an entry.

JSON representation 
{
  "service": enum (Service),
  "resource": string
}
Fields
service

enum (Service)

Service that physically stores the data.
resource

string

Full name of a resource as defined by the service. For example:

//bigquery.googleapis.com/projects/{PROJECT_ID}/locations/{LOCATION}/datasets/{DATASET_ID}/tables/{TABLE_ID}

Service

Name of a service that stores the data.

Enums
SERVICE_UNSPECIFIED Default unknown service.
CLOUD_STORAGE Google Cloud Storage service.
BIGQUERY BigQuery service.

Methods

create

 Creates an entry.

delete

 Deletes an existing entry.

get

 Gets an entry.

getIamPolicy

 Gets the access control policy for a resource.

list

 Lists entries.

patch

 Updates an existing entry.

testIamPermissions

 Gets your permissions on a resource.