Fully qualified names

Fully qualified names are globally unique, case-sensitive resource identifiers that you can use with Dataplex Universal Catalog to refer to Google Cloud and non-Google Cloud resources.

For example, fully qualified names are used for systems that support the automatic ingestion of metadata from Google Cloud. If you catalog your non-Google Cloud resources in Dataplex Universal Catalog, you can provide fully qualified names when you create the custom entries that represent those third-party resources.

Fully qualified names are composed of a prefix followed by segments delimited with dots. The maximum length is 4,000 characters. Surround segments that contain reserved characters, such as colons, dots, and whitespace, with backticks. Use double backticks to specify a backtick within a segment.

Predefined formats for Google Cloud resources

The following tables list the predefined formats for fully qualified names that are supported for Google Cloud resources.

BigQuery

Resource Format
Dataset bigquery:{projectId}.{datasetId}
Table bigquery:{projectId}.{datasetId}.{tableId}
View bigquery:{projectId}.{datasetId}.{viewId}
Model bigquery:{projectId}.{datasetId}.{modelId}
Sharded table bigquery:sharded:{projectId}.{datasetId}.{tablePrefix}
Routine bigquery:routine:{projectId}.{datasetId}.{routineId}
Connection bigquery:connection:{projectId}.{location}.{connectionId}

BigQuery sharing (formerly Analytics Hub)

Resource Format
Exchange analytics_hub:{projectId}.{location}.{exchangeId}
Listing analytics_hub:{projectId}.{location}.{exchangeId}.{listingId}

Bigtable

Resource Format
Instance bigtable:{projectId}.{instanceId}
Table bigtable:{projectId}.{instanceId}.{tableId}

Cloud SQL for MySQL

Resource Format
Instance cloudsql_mysql:{projectId}.{location}.{instanceId}
Schema cloudsql_mysql:{projectId}.{location}.{instanceId}.{schemaId}
Table cloudsql_mysql:{projectId}.{location}.{instanceId}.{schemaId}.{tableId}

Cloud SQL for PostgreSQL

Resource Format
Instance cloudsql_postgresql:{projectId}.{location}.{instanceId}
Database cloudsql_postgresql:{projectId}.{location}.{instanceId}.{databaseId}
Schema cloudsql_postgresql:{projectId}.{location}.{instanceId}.{databaseId}.{schemaId}
Table cloudsql_postgresql:{projectId}.{location}.{instanceId}.{databaseId}.{schemaId}.{tableId}

Cloud Storage

Resource Format
Bucket gcs:{bucketName}
Path gcs:{bucketName}.{virtualPath}

Dataplex Universal Catalog

Resource Format
Lake dataplex:{projectId}.{location}.{lakeId}
Zone dataplex:{projectId}.{location}.{lakeId}.{zoneId}
Table dataplex:{projectId}.{location}.{lakeId}.{zoneId}.{tableId}
Fileset dataplex:{projectId}.{location}.{lakeId}.{zoneId}.{filesetId}

Dataproc Metastore

Resource Format
Service dataproc_metastore:{projectId}.{location}.{instanceId}
Database dataproc_metastore:{projectId}.{location}.{instanceId}.{databaseId}
Table dataproc_metastore:{projectId}.{location}.{instanceId}.{databaseId}.{tableId}

Looker

Resource Format
Instance looker:{instanceId}
Dashboard looker:dashboard:{instanceId}.{folderId}.{dashboardId}
Explore looker:explore:{instanceId}.{folderId}.{exploreId}
Look looker:look:{instanceId}.{folderId}.{lookId}

Pub/Sub

Resource Format
Topic pubsub:topic:{projectId}.{topicId}
Subscription pubsub:subscription:{projectId}.{subscriptionId}

Spanner

Resource Format
Instance spanner:{projectId}.{instanceConfigId}.{instanceId}
Database spanner:{projectId}.{instanceConfigId}.{instanceId}.{databaseId}
Table spanner:{projectId}.{instanceConfigId}.{instanceId}.{databaseId}.{tableId}
View spanner:{projectId}.{instanceConfigId}.{instanceId}.{databaseId}.{viewId}

Vertex AI

Resource Format
Model vertex_ai:model:{projectId}.{locationId}.{modelId}.{version}
Dataset vertex_ai:dataset:{projectId}.{locationId}.{datasetId}
FeatureGroup vertex_ai:featuregroup:{projectId}.{locationId}.{featureGroupId}
FeatureOnlineStore vertex_ai:featureonlinestore:{projectId}.{locationId}.{featureOnlineStoreId}
FeatureView vertex_ai:featureonlinestore:{projectId}.{locationId}.{featureOnlineStoreId}.{FeatureViewId}

Predefined formats for non-Google Cloud resources

The following tables list the predefined formats for fully qualified names that are supported for third-party resources.

Apache Cassandra

Resource Format
Cluster cassandra:{dns}
Keyspace cassandra:{dns}.{keyspaceId}
Table cassandra:{dns}.{keyspaceId}.{tableId}

Apache Hive

Resource Format
Instance hive_metastore:{dns}
Database hive_metastore:{dns}.{database}
Table hive_metastore:{dns}.{database}.{table}

Apache Kafka

Resource Format
Cluster kafka:{dns}
Topic kafka:{dns}.{topicId}

AWS Athena

Resource Format
Catalog athena:{catalogId}.{region}
Database athena:{catalogId}.{region}.{databaseId}
Table athena:{catalogId}.{region}.{databaseId}.{tableId}

AWS Redshift

Resource Format
Instance redshift:{clusterId}.{region}.{port}
Database redshift:{clusterId}.{region}.{port}.{databaseId}
Schema redshift:{clusterId}.{region}.{port}.{databaseId}.{schemaId}
Table redshift:{clusterId}.{region}.{port}.{databaseId}.{schemaId}.{tableId}

AWS S3

Resource Format
Bucket s3:{bucket}
Path s3:{bucket}.{virtualPath}

AWS Glue

Resource Format
Table aws_glue:table:{region}.{accountId}.{dbName}.{tableName}

Azure Blob Storage

Resource Format
Account abs:{account}
Container abs:{account}.{container}
Path abs:{account}.{container}.{virtualPath}

Azure Cosmos DB

Resource Format
Account cosmos-db:{accountName}
Database cosmos-db:{accountName}.{databaseId}
Container cosmos-db:{accountName}.{databaseId}.{containerId}

Azure Data Explorer

Resource Format
Cluster kusto:{clusterName}.{region}
Database kusto:{clusterName}.{region}.{databaseId}
Table kusto:{clusterName}.{region}.{databaseId}.{tableId}

Databricks

Resource Format
Metastore databricks:{metastoreId}
Catalog databricks:{metastoreId}.{catalogId}
Schema databricks:{metastoreId}.{catalogId}.{schemaId}
Table databricks:table:{metastoreId}.{catalogId}.{schemaId}.{tableId}
View databricks:view:{metastoreId}.{catalogId}.{schemaId}.{viewId}
Routine databricks:routine:{metastoreId}.{catalogId}.{schemaId}.{routineId}
Volume databricks:volume:{metastoreId}.{catalogId}.{schemaId}.{volumeId}
Model databricks:model:{metastoreId}.{catalogId}.{schemaId}.{modelId}

Databricks File System

Resource Format
Instance dbfs:{workspace}
Path dbfs:{workspace}.{path}

Generic File System

Resource Format
Instance filesystem:{dns}
Path filesystem:{dns}.{path}

Hadoop Distributed File System

Resource Format
Instance hdfs:{dns}
Path hdfs:{dns}.{path}

IBM DB2

Resource Format
Instance db2:{dns}
Database db2:{dns}.{databaseId}
Schema db2:{dns}.{databaseId}.{schemaId}
Table db2:{dns}.{databaseId}.{schemaId}.{tableId}

IBM Informix

Resource Format
Instance informix:{dns}
Database informix:{dns}.{databaseId}
Schema informix:{dns}.{databaseId}.{schemaId}
Table informix:{dns}.{databaseId}.{schemaId}.{tableId}

Microsoft SQL Server

Resource Format
Instance sqlserver:{dns}
Database sqlserver:{dns}.{databaseId}
Schema sqlserver:{dns}.{databaseId}.{schemaId}
Table sqlserver:{dns}.{databaseId}.{schemaId}.{tableId}

MySQL

Resource Format
Instance mysql:{dns}
Schema mysql:{dns}.{schemaId}
Table mysql:{dns}.{schemaId}.{tableId}

Oracle Database

Resource Format
Instance oracle:{dns}
Database oracle:{dns}.{databaseId}
Schema oracle:{dns}.{databaseId}.{schemaId}
Table oracle:{dns}.{databaseId}.{schemaId}.{tableId}

PostgreSQL

Resource Format
Instance postgresql:{dns}
Database postgresql:{dns}.{databaseId}
Schema postgresql:{dns}.{databaseId}.{schemaId}
Table postgresql:{dns}.{databaseId}.{schemaId}.{tableId}

Salesforce

Resource Format
SObject salesforce:{environment}.{orgId}.{sobject}

Snowflake

Resource Format
Account snowflake:{accountName}
Database snowflake:{accountName}.{databaseId}
Schema snowflake:{accountName}.{databaseId}.{schemaId}
Table snowflake:{accountName}.{databaseId}.{schemaId}.{tableId}

Tableau

Resource Format
Server tableau:{dns}
Site tableau:{dns}.{site}
Project tableau:project:{dns}.{site}.{projectId}
Data source tableau:datasource:{dns}.{site}.{projectId}.{datasourceId}
Workbook tableau:workbook:{dns}.{site}.{projectId}.{workbookId}
Sheet tableau:sheet:{dns}.{site}.{projectId}.{workbookId}.{sheetId}
Dashboard tableau:dashboard:{dns}.{site}.{projectId}.{workbookId}.{dashboardId}

Trino

Resource Format
Instance trino:{dns}
Catalog trino:{dns}.{catalogId}
Schema trino:{dns}.{catalogId}.{schemaId}
Table trino:{dns}.{catalogId}.{schemaId}.{tableId}

Custom formats

You can create a custom fully qualified name format by using the custom: prefix. Custom formats are not validated, and might not be compatible with Dataplex Universal Catalog features. To ease future migration, create custom formats that are similar to predefined formats.

Resources that are referenced in OpenLineage messages with the namespace custom are interpreted as custom fully qualified names.