REST Resource: projects.locations.entryGroups.entries

Resource: Entry

Entry Metadata. A Data Catalog Entry resource represents another resource in Google Cloud Platform (such as a BigQuery dataset or a Pub/Sub topic), or outside of Google Cloud Platform. Clients can use the linkedResource field in the Entry resource to refer to the original resource ID of the source system.

An Entry resource contains resource details, such as its schema. An Entry can also be used to attach flexible metadata, such as a Tag.

JSON representation 
{
  "name": string,
  "linkedResource": string,
  "displayName": string,
  "description": string,
  "schema": {
    object (Schema)
  },
  "sourceSystemTimestamps": {
    object (SystemTimestamps)
  },
  "usageSignal": {
    object (UsageSignal)
  },

  // Union field entry_type can be only one of the following:
  "type": enum (EntryType),
  "userSpecifiedType": string
  // End of list of possible types for union field entry_type.

  // Union field system can be only one of the following:
  "integratedSystem": enum (IntegratedSystem),
  "userSpecifiedSystem": string
  // End of list of possible types for union field system.

  // Union field type_spec can be only one of the following:
  "gcsFilesetSpec": {
    object (GcsFilesetSpec)
  },
  "bigqueryTableSpec": {
    object (BigQueryTableSpec)
  },
  "bigqueryDateShardedSpec": {
    object (BigQueryDateShardedSpec)
  }
  // End of list of possible types for union field type_spec.
}
Fields
name

string

Output only. Identifier. The Data Catalog resource name of the entry in URL format. Example:

  • projects/{projectId}/locations/{location}/entryGroups/{entryGroupId}/entries/{entryId}

Note that this Entry and its child resources may not actually be stored in the location in this name.
linkedResource

string

The resource this metadata entry refers to.

For Google Cloud Platform resources, linkedResource is the full name of the resource. For example, the linkedResource for a table resource from BigQuery is:

  • //bigquery.googleapis.com/projects/projectId/datasets/datasetId/tables/tableId

Output only when Entry is of type in the EntryType enum. For entries with userSpecifiedType, this field is optional and defaults to an empty string.
displayName

string

Display information such as title and description. A short name to identify the entry, for example, "Analytics Data - Jan 2011". Default value is an empty string.
description

string

Entry description, which can consist of several sentences or paragraphs that describe entry contents. Default value is an empty string.
schema

object (Schema)

Schema of the entry. An entry might not have any schema attached to it.
sourceSystemTimestamps

object (SystemTimestamps)

Output only. Timestamps about the underlying resource, not about this Data Catalog entry. Output only when Entry is of type in the EntryType enum. For entries with userSpecifiedType, this field is optional and defaults to an empty timestamp.
usageSignal

object (UsageSignal)

Output only. Statistics on the usage level of the resource.
Union field entry_type. Required. Entry type. entry_type can be only one of the following:
type

enum (EntryType)

The type of the entry. Only used for Entries with types in the EntryType enum.
userSpecifiedType

string

Entry type if it does not fit any of the input-allowed values listed in EntryType enum above. When creating an entry, users should check the enum values first, if nothing matches the entry to be created, then provide a custom value, for example "my_special_type". userSpecifiedType strings must begin with a letter or underscore and can only contain letters, numbers, and underscores; are case insensitive; must be at least 1 character and at most 64 characters long.

Currently, only FILESET enum value is allowed. All other entries created through Data Catalog must use userSpecifiedType.
Union field system. The source system of the entry. system can be only one of the following:
integratedSystem

enum (IntegratedSystem)

Output only. This field indicates the entry's source system that Data Catalog integrates with, such as BigQuery or Pub/Sub.
userSpecifiedSystem

string

This field indicates the entry's source system that Data Catalog does not integrate with. userSpecifiedSystem strings must begin with a letter or underscore and can only contain letters, numbers, and underscores; are case insensitive; must be at least 1 character and at most 64 characters long.
Union field type_spec. Type specification information. type_spec can be only one of the following:
gcsFilesetSpec

object (GcsFilesetSpec)

Specification that applies to a Cloud Storage fileset. This is only valid on entries of type FILESET.
bigqueryTableSpec

object (BigQueryTableSpec)

Specification that applies to a BigQuery table. This is only valid on entries of type TABLE.
bigqueryDateShardedSpec

object (BigQueryDateShardedSpec)

Specification for a group of BigQuery tables with name pattern [prefix]YYYYMMDD. Context: https://cloud.google.com/bigquery/docs/partitioned-tables#partitioning_versus_sharding.

EntryType

Entry resources in Data Catalog can be of different types e.g. a BigQuery Table entry is of type TABLE. This enum describes all the possible types Data Catalog contains.

Enums
ENTRY_TYPE_UNSPECIFIED Default unknown type.
TABLE Output only. The type of entry that has a GoogleSQL schema, including logical views.
MODEL Output only. The type of models. https://cloud.google.com/bigquery-ml/docs/bigqueryml-intro
DATA_STREAM Output only. An entry type which is used for streaming entries. Example: Pub/Sub topic.
FILESET An entry type which is a set of files or objects. Example: Cloud Storage fileset.

IntegratedSystem

This enum describes all the possible systems that Data Catalog integrates with.

Enums
INTEGRATED_SYSTEM_UNSPECIFIED Default unknown system.
BIGQUERY BigQuery.
CLOUD_PUBSUB Cloud Pub/Sub.

GcsFilesetSpec

Describes a Cloud Storage fileset entry.

JSON representation 
{
  "filePatterns": [
    string
  ],
  "sampleGcsFileSpecs": [
    {
      object (GcsFileSpec)
    }
  ]
}
Fields
filePatterns[]

string

Required. Patterns to identify a set of files in Google Cloud Storage. See Cloud Storage documentation for more information. Note that bucket wildcards are currently not supported.

Examples of valid filePatterns:

  • gs://bucket_name/dir/*: matches all files within bucket_name/dir directory.
  • gs://bucket_name/dir/**: matches all files in bucket_name/dir spanning all subdirectories.
  • gs://bucket_name/file*: matches files prefixed by file in bucket_name
  • gs://bucket_name/??.txt: matches files with two characters followed by .txt in bucket_name
  • gs://bucket_name/[aeiou].txt: matches files that contain a single vowel character followed by .txt in bucket_name
  • gs://bucket_name/[a-m].txt: matches files that contain a, b, ... or m followed by .txt in bucket_name
  • gs://bucket_name/a/*/b: matches all files in bucket_name that match a/*/b pattern, such as a/c/b, a/d/b
  • gs://another_bucket/a.txt: matches gs://another_bucket/a.txt

You can combine wildcards to provide more powerful matches, for example:

  • gs://bucket_name/[a-m]??.j*g
sampleGcsFileSpecs[]

object (GcsFileSpec)

Output only. Sample files contained in this fileset, not all files contained in this fileset are represented here.

GcsFileSpec

Specifications of a single file in Cloud Storage.

JSON representation 
{
  "filePath": string,
  "gcsTimestamps": {
    object (SystemTimestamps)
  },
  "sizeBytes": string
}
Fields
filePath

string

Required. The full file path. Example: gs://bucket_name/a/b.txt.
gcsTimestamps

object (SystemTimestamps)

Output only. Timestamps about the Cloud Storage file.
sizeBytes

string (int64 format)

Output only. The size of the file, in bytes.

BigQueryTableSpec

Describes a BigQuery table.

JSON representation 
{
  "tableSourceType": enum (TableSourceType),

  // Union field type_spec can be only one of the following:
  "viewSpec": {
    object (ViewSpec)
  },
  "tableSpec": {
    object (TableSpec)
  }
  // End of list of possible types for union field type_spec.
}
Fields
tableSourceType

enum (TableSourceType)

Output only. The table source type.
Union field type_spec. Output only. type_spec can be only one of the following:
viewSpec

object (ViewSpec)

Table view specification. This field should only be populated if tableSourceType is BIGQUERY_VIEW.
tableSpec

object (TableSpec)

Spec of a BigQuery table. This field should only be populated if tableSourceType is BIGQUERY_TABLE.

TableSourceType

Table source type.

Enums
TABLE_SOURCE_TYPE_UNSPECIFIED Default unknown type.
BIGQUERY_VIEW Table view.
BIGQUERY_TABLE BigQuery native table.
BIGQUERY_MATERIALIZED_VIEW BigQuery materialized view.

ViewSpec

Table view specification.

JSON representation 
{
  "viewQuery": string
}
Fields
viewQuery

string

Output only. The query that defines the table view.

TableSpec

Normal BigQuery table spec.

JSON representation 
{
  "groupedEntry": string
}
Fields
groupedEntry

string

Output only. If the table is a dated shard, i.e., with name pattern [prefix]YYYYMMDD, groupedEntry is the Data Catalog resource name of the date sharded grouped entry, for example, projects/{projectId}/locations/{location}/entrygroups/{entryGroupId}/entries/{entryId}. Otherwise, groupedEntry is empty.

BigQueryDateShardedSpec

Spec for a group of BigQuery tables with name pattern [prefix]YYYYMMDD. Context: https://cloud.google.com/bigquery/docs/partitioned-tables#partitioning_versus_sharding

JSON representation 
{
  "dataset": string,
  "tablePrefix": string,
  "shardCount": string
}
Fields
dataset

string

Output only. The Data Catalog resource name of the dataset entry the current table belongs to, for example, projects/{projectId}/locations/{location}/entrygroups/{entryGroupId}/entries/{entryId}.
tablePrefix

string

Output only. The table name prefix of the shards. The name of any given shard is [tablePrefix]YYYYMMDD, for example, for shard MyTable20180101, the tablePrefix is MyTable.
shardCount

string (int64 format)

Output only. Total number of shards.

Schema

Represents a schema (e.g. BigQuery, GoogleSQL, Avro schema).

JSON representation 
{
  "columns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
columns[]

object (ColumnSchema)

Required. Schema of columns. A maximum of 10,000 columns and sub-columns can be specified.

ColumnSchema

Representation of a column within a schema. Columns could be nested inside other columns.

JSON representation 
{
  "column": string,
  "type": string,
  "description": string,
  "mode": string,
  "subcolumns": [
    {
      object (ColumnSchema)
    }
  ]
}
Fields
column

string

Required. Name of the column.
type

string

Required. Type of the column.
description

string

Optional. Description of the column. Default value is an empty string.
mode

string

Optional. A column's mode indicates whether the values in this column are required, nullable, etc. Only NULLABLE, REQUIRED and REPEATED are supported. Default mode is NULLABLE.
subcolumns[]

object (ColumnSchema)

Optional. Schema of sub-columns. A column can have zero or more sub-columns.

UsageSignal

The set of all usage signals that we store in Data Catalog.

JSON representation 
{
  "updateTime": string,
  "usageWithinTimeRange": {
    string: {
      object (UsageStats)
    },
    ...
  }
}
Fields
updateTime

string (Timestamp format)

The timestamp of the end of the usage statistics duration.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
usageWithinTimeRange

map (key: string, value: object (UsageStats))

Usage statistics over each of the pre-defined time ranges, supported strings for time ranges are {"24H", "7D", "30D"}.

UsageStats

Detailed counts on the entry's usage. Caveats: - Only BigQuery tables have usage stats - The usage stats only include BigQuery query jobs - The usage stats might be underestimated, e.g. wildcard table references are not yet counted in usage computation https://cloud.google.com/bigquery/docs/querying-wildcard-tables

JSON representation 
{
  "totalCompletions": number,
  "totalFailures": number,
  "totalCancellations": number,
  "totalExecutionTimeForCompletionsMillis": number
}
Fields
totalCompletions

number

The number of times that the underlying entry was successfully used.
totalFailures

number

The number of times that the underlying entry was attempted to be used but failed.
totalCancellations

number

The number of times that the underlying entry was attempted to be used but was cancelled by the user.
totalExecutionTimeForCompletionsMillis

number

Total time spent (in milliseconds) during uses the resulted in completions.

Methods

create

 Creates an entry.

delete

 Deletes an existing entry.

get

 Gets an entry.

getIamPolicy

 Gets the access control policy for a resource.

list

 Lists entries.

patch

 Updates an existing entry.

testIamPermissions

 Returns the caller's permissions on a resource.