Package google.cloud.metastore.v1

Index

DataprocMetastore

Configures and manages metastore services. Metastore services are fully managed, highly available, autoscaled, autohealing, OSS-native deployments of technical metadata management software. Each metastore service exposes a network endpoint through which metadata queries are served. Metadata queries can originate from a variety of sources, including Apache Hive, Apache Presto, and Apache Spark.

The Dataproc Metastore API defines the following resource model:

  • The service works with a collection of Google Cloud projects, named: /projects/*
  • Each project has a collection of available locations, named: /locations/* (a location must refer to a Google Cloud region)
  • Each location has a collection of services, named: /services/*
  • Dataproc Metastore services are resources with names of the form:

/projects/{project_number}/locations/{location_id}/services/{service_id}.

CreateMetadataImport

rpc CreateMetadataImport(CreateMetadataImportRequest) returns (Operation)

Creates a new MetadataImport in a given project and location.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

CreateService

rpc CreateService(CreateServiceRequest) returns (Operation)

Creates a metastore service in a project and location.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

DeleteService

rpc DeleteService(DeleteServiceRequest) returns (Operation)

Deletes a single service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

ExportMetadata

rpc ExportMetadata(ExportMetadataRequest) returns (Operation)

Exports metadata from a service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

GetMetadataImport

rpc GetMetadataImport(GetMetadataImportRequest) returns (MetadataImport)

Gets details of a single import.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

GetService

rpc GetService(GetServiceRequest) returns (Service)

Gets the details of a single service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

ListMetadataImports

rpc ListMetadataImports(ListMetadataImportsRequest) returns (ListMetadataImportsResponse)

Lists imports in a service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

ListServices

rpc ListServices(ListServicesRequest) returns (ListServicesResponse)

Lists services in a project and location.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

UpdateMetadataImport

rpc UpdateMetadataImport(UpdateMetadataImportRequest) returns (Operation)

Updates a single import. Only the description field of MetadataImport is supported to be updated.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

UpdateService

rpc UpdateService(UpdateServiceRequest) returns (Operation)

Updates the parameters of a single service.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

CreateMetadataImportRequest

Request message for DataprocMetastore.CreateMetadataImport.

Fields
parent

string

Required. The relative resource name of the service in which to create a metastore import, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.imports.create
metadata_import_id

string

Required. The ID of the metadata import, which is used as the final component of the metadata import's name.

This value must be between 1 and 64 characters long, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.

metadata_import

MetadataImport

Required. The metadata import to create. The name field is ignored. The ID of the created metadata import must be provided in the request's metadata_import_id field.

request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

CreateServiceRequest

Request message for DataprocMetastore.CreateService.

Fields
parent

string

Required. The relative resource name of the location in which to create a metastore service, in the following form:

projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.services.create
service_id

string

Required. The ID of the metastore service, which is used as the final component of the metastore service's name.

This value must be between 2 and 63 characters long inclusive, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.

service

Service

Required. The Metastore service to create. The name field is ignored. The ID of the created metastore service must be provided in the request's service_id field.

request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

DatabaseDumpSpec

The specification of database dump to import from or export to.

Type

The type of the database dump.

Enums
TYPE_UNSPECIFIED The type of the database dump is unknown.
MYSQL Database dump is a MySQL dump file.

DeleteServiceRequest

Request message for DataprocMetastore.DeleteService.

Fields
name

string

Required. The relative resource name of the metastore service to delete, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.services.delete
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

ExportMetadataRequest

Request message for DataprocMetastore.ExportMetadata.

Fields
service

string

Required. The relative resource name of the metastore service to run export, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}.

request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID. A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

database_dump_type

Type

Optional. The type of the database dump. If unspecified, defaults to MYSQL.

destination_gcs_folder

string

A Cloud Storage URI of a folder, in the format gs://<bucket_name>/<path_inside_bucket>. A sub-folder <export_folder> containing exported files will be created below it.

GetMetadataImportRequest

Request message for DataprocMetastore.GetMetadataImport.

Fields
name

string

Required. The relative resource name of the metadata import to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{import_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.imports.get

GetServiceRequest

Request message for DataprocMetastore.GetService.

Fields
name

string

Required. The relative resource name of the metastore service to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.services.get

HiveMetastoreConfig

Specifies configuration information specific to running Hive metastore software as the metastore service.

Fields
version

string

Immutable. The Hive metastore schema version.

config_overrides

map<string, string>

A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore (configured in hive-site.xml). The mappings override system defaults (some keys cannot be overridden).

kerberos_config

KerberosConfig

Information used to configure the Hive metastore service as a service principal in a Kerberos realm. To disable Kerberos, use the UpdateService method and specify this field's path (hive_metastore_config.kerberos_config) in the request's update_mask while omitting this field from the request's service.

KerberosConfig

Configuration information for a Kerberos principal.

Fields
keytab

Secret

A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center (KDC).

principal

string

A Kerberos principal that exists in the both the keytab the KDC to authenticate as. A typical principal is of the form primary/instance@REALM, but there is no exact format.

krb5_config_gcs_uri

string

A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/to/krb5.conf, although the file does not need to be named krb5.conf explicitly.

ListMetadataImportsRequest

Request message for DataprocMetastore.ListMetadataImports.

Fields
parent

string

Required. The relative resource name of the service whose metadata imports to list, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.imports.list
page_size

int32

Optional. The maximum number of imports to return. The response may contain less than the maximum number. If unspecified, no more than 500 imports are returned. The maximum value is 1000; values above 1000 are changed to 1000.

page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListServices call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListServices must match the call that provided the page token.

filter

string

Optional. The filter to apply to list results.

order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListMetadataImportsResponse

Response message for DataprocMetastore.ListMetadataImports.

Fields
metadata_imports[]

MetadataImport

The imports in the specified service.

next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.

unreachable[]

string

Locations that could not be reached.

ListServicesRequest

Request message for DataprocMetastore.ListServices.

Fields
parent

string

Required. The relative resource name of the location of metastore services to list, in the following form:

projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.services.list
page_size

int32

Optional. The maximum number of services to return. The response may contain less than the maximum number. If unspecified, no more than 500 services are returned. The maximum value is 1000; values above 1000 are changed to 1000.

page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListServices call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListServices must match the call that provided the page token.

filter

string

Optional. The filter to apply to list results.

order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListServicesResponse

Response message for DataprocMetastore.ListServices.

Fields
services[]

Service

The services in the specified location.

next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.

unreachable[]

string

Locations that could not be reached.

LocationMetadata

Metadata about the service in a location.

Fields
supported_hive_metastore_versions[]

HiveMetastoreVersion

The versions of Hive Metastore that can be used when creating a new metastore service in this location. The server guarantees that exactly one HiveMetastoreVersion in the list will set is_default.

HiveMetastoreVersion

A specification of a supported version of the Hive Metastore software.

Fields
version

string

The semantic version of the Hive Metastore software.

is_default

bool

Whether version will be chosen by the server if a metastore service is created with a HiveMetastoreConfig that omits the version.

MaintenanceWindow

Maintenance window. This specifies when Dataproc Metastore may perform system maintenance operation to the service.

Fields
hour_of_day

Int32Value

The hour of day (0-23) when the window starts.

day_of_week

DayOfWeek

The day of week, when the window starts.

MetadataExport

The details of a metadata export operation.

Fields
start_time

Timestamp

Output only. The time when the export started.

end_time

Timestamp

Output only. The time when the export ended.

state

State

Output only. The current state of the export.

database_dump_type

Type

Output only. The type of the database dump.

destination_gcs_uri

string

Output only. A Cloud Storage URI of a folder that metadata are exported to, in the form of gs://<bucket_name>/<path_inside_bucket>/<export_folder>, where <export_folder> is automatically generated.

State

The current state of the metadata export.

Enums
STATE_UNSPECIFIED The state of the metadata export is unknown.
RUNNING The metadata export is running.
SUCCEEDED The metadata export completed successfully.
FAILED The metadata export failed.
CANCELLED The metadata export is cancelled.

MetadataImport

A metastore resource that imports metadata.

Fields
name

string

Immutable. The relative resource name of the metadata import, of the form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{metadata_import_id}.

description

string

The description of the metadata import.

create_time

Timestamp

Output only. The time when the metadata import was created.

update_time

Timestamp

Output only. The time when the metadata import was last updated.

state

State

Output only. The current state of the metadata import.

database_dump

DatabaseDump

Immutable. A database dump from a pre-existing metastore's database.

DatabaseDump

A specification of the location of and metadata about a database dump from a relational database management system.

Fields
database_type
(deprecated)

DatabaseType

The type of the database.

gcs_uri

string

A Cloud Storage object or folder URI that specifies the source from which to import metadata. It must begin with gs://.

type

Type

Optional. The type of the database dump. If unspecified, defaults to MYSQL.

DatabaseType

The type of the database.

Enums
DATABASE_TYPE_UNSPECIFIED The type of the source database is unknown.
MYSQL The type of the source database is MySQL.

State

The current state of the metadata import.

Enums
STATE_UNSPECIFIED The state of the metadata import is unknown.
RUNNING The metadata import is running.
SUCCEEDED The metadata import completed successfully.
UPDATING The metadata import is being updated.
FAILED The metadata import failed, and attempted metadata changes were rolled back.

MetadataManagementActivity

The metadata management activities of the metastore service.

Fields
metadata_exports[]

MetadataExport

Output only. The latest metadata exports of the metastore service.

restores[]

Restore

Output only. The latest restores of the metastore service.

OperationMetadata

Represents the metadata of a long-running operation.

Fields
create_time

Timestamp

Output only. The time the operation was created.

end_time

Timestamp

Output only. The time the operation finished running.

target

string

Output only. Server-defined resource path for the target of the operation.

verb

string

Output only. Name of the verb executed by the operation.

status_message

string

Output only. Human-readable status of the operation, if any.

requested_cancellation

bool

Output only. Identifies whether the caller has requested cancellation of the operation. Operations that have successfully been cancelled have [Operation.error][] value with a google.rpc.Status.code of 1, corresponding to Code.CANCELLED.

api_version

string

Output only. API version used to start the operation.

Restore

The details of a metadata restore operation.

Fields
start_time

Timestamp

Output only. The time when the restore started.

end_time

Timestamp

Output only. The time when the restore ended.

state

State

Output only. The current state of the restore.

backup

string

Output only. The relative resource name of the metastore service backup to restore from, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}/backups/{backup_id}.

type

RestoreType

Output only. The type of restore.

details

string

Output only. The restore details containing the revision of the service to be restored to, in format of JSON.

RestoreType

The type of restore. If unspecified, defaults to METADATA_ONLY.

Enums
RESTORE_TYPE_UNSPECIFIED The restore type is unknown.
FULL The service's metadata and configuration are restored.
METADATA_ONLY Only the service's metadata is restored.

State

The current state of the restore.

Enums
STATE_UNSPECIFIED The state of the metadata restore is unknown.
RUNNING The metadata restore is running.
SUCCEEDED The metadata restore completed successfully.
FAILED The metadata restore failed.
CANCELLED The metadata restore is cancelled.

Secret

A securely stored value.

Fields
cloud_secret

string

The relative resource name of a Secret Manager secret version, in the following form:

projects/{project_number}/secrets/{secret_id}/versions/{version_id}.

Service

A managed metastore service that serves metadata queries.

Fields
name

string

Immutable. The relative resource name of the metastore service, of the form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

create_time

Timestamp

Output only. The time when the metastore service was created.

update_time

Timestamp

Output only. The time when the metastore service was last updated.

labels

map<string, string>

User-defined labels for the metastore service.

network

string

Immutable. The relative resource name of the VPC network on which the instance can be accessed. It is specified in the following form:

projects/{project_number}/global/networks/{network_id}.

endpoint_uri

string

Output only. The URI of the endpoint used to access the metastore service.

port

int32

The TCP port at which the metastore service is reached. Default: 9083.

state

State

Output only. The current state of the metastore service.

state_message

string

Output only. Additional information about the current state of the metastore service, if available.

artifact_gcs_uri

string

Output only. A Cloud Storage URI (starting with gs://) that specifies where artifacts related to the metastore service are stored.

tier

Tier

The tier of the service.

maintenance_window

MaintenanceWindow

The one hour maintenance window of the metastore service. This specifies when the service can be restarted for maintenance purposes in UTC time.

uid

string

Output only. The globally unique resource identifier of the metastore service.

metadata_management_activity

MetadataManagementActivity

Output only. The metadata management activities of the metastore service.

release_channel

ReleaseChannel

Immutable. The release channel of the service. If unspecified, defaults to STABLE.

hive_metastore_config

HiveMetastoreConfig

Configuration information specific to running Hive metastore software as the metastore service.

ReleaseChannel

Release channels bundle features of varying levels of stability. Newer features may be introduced initially into less stable release channels and can be automatically promoted into more stable release channels.

Enums
RELEASE_CHANNEL_UNSPECIFIED Release channel is not specified.
CANARY The CANARY release channel contains the newest features, which may be unstable and subject to unresolved issues with no known workarounds. Services using the CANARY release channel are not subject to any SLAs.
STABLE The STABLE release channel contains features that are considered stable and have been validated for production use.

State

The current state of the metastore service.

Enums
STATE_UNSPECIFIED The state of the metastore service is unknown.
CREATING The metastore service is in the process of being created.
ACTIVE The metastore service is running and ready to serve queries.
SUSPENDING The metastore service is entering suspension. Its query-serving availability may cease unexpectedly.
SUSPENDED The metastore service is suspended and unable to serve queries.
UPDATING The metastore service is being updated. It remains usable but cannot accept additional update requests or be deleted at this time.
DELETING The metastore service is undergoing deletion. It cannot be used.
ERROR The metastore service has encountered an error and cannot be used. The metastore service should be deleted.

Tier

Available service tiers.

Enums
TIER_UNSPECIFIED The tier is not set.
DEVELOPER The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept.
ENTERPRISE The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads.

UpdateMetadataImportRequest

Request message for DataprocMetastore.UpdateMetadataImport.

Fields
update_mask

FieldMask

Required. A field mask used to specify the fields to be overwritten in the metadata import resource by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.

metadata_import

MetadataImport

Required. The metadata import to update. The server only merges fields in the import if they are specified in update_mask.

The metadata import's name field is used to identify the metastore import to be updated.

Authorization requires the following IAM permission on the specified resource metadataImport:

  • metastore.imports.update
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

UpdateServiceRequest

Request message for DataprocMetastore.UpdateService.

Fields
update_mask

FieldMask

Required. A field mask used to specify the fields to be overwritten in the metastore service resource by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.

service

Service

Required. The metastore service to update. The server only merges fields in the service if they are specified in update_mask.

The metastore service's name field is used to identify the metastore service to be updated.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.update
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.