REST Resource: projects.locations.services

Resource: Service

A managed metastore service that serves metadata queries.

JSON representation 
{
  "name": string,
  "createTime": string,
  "updateTime": string,
  "labels": {
    string: string,
    ...
  },
  "network": string,
  "endpointUri": string,
  "port": integer,
  "state": enum (State),
  "stateMessage": string,
  "artifactGcsUri": string,
  "tier": enum (Tier),
  "metadataIntegration": {
    object (MetadataIntegration)
  },
  "maintenanceWindow": {
    object (MaintenanceWindow)
  },
  "uid": string,
  "metadataManagementActivity": {
    object (MetadataManagementActivity)
  },
  "releaseChannel": enum (ReleaseChannel),
  "encryptionConfig": {
    object (EncryptionConfig)
  },
  "networkConfig": {
    object (NetworkConfig)
  },
  "databaseType": enum (DatabaseType),
  "telemetryConfig": {
    object (TelemetryConfig)
  },
  "scalingConfig": {
    object (ScalingConfig)
  },
  "scheduledBackup": {
    object (ScheduledBackup)
  },
  "deletionProtection": boolean,

  // Union field metastore_config can be only one of the following:
  "hiveMetastoreConfig": {
    object (HiveMetastoreConfig)
  }
  // End of list of possible types for union field metastore_config.
}
Fields
name

string

Immutable. The relative resource name of the metastore service, in the following format:

projects/{projectNumber}/locations/{locationId}/services/{serviceId}.
createTime

string (Timestamp format)

Output only. The time when the metastore service was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The time when the metastore service was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
labels

map (key: string, value: string)

User-defined labels for the metastore service.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
network

string

Immutable. The relative resource name of the VPC network on which the instance can be accessed. It is specified in the following form:

projects/{projectNumber}/global/networks/{network_id}.
endpointUri

string

Output only. The URI of the endpoint used to access the metastore service.
port

integer

The TCP port at which the metastore service is reached. Default: 9083.
state

enum (State)

Output only. The current state of the metastore service.
stateMessage

string

Output only. Additional information about the current state of the metastore service, if available.
artifactGcsUri

string

Output only. A Cloud Storage URI (starting with gs://) that specifies where artifacts related to the metastore service are stored.
tier

enum (Tier)

The tier of the service.
metadataIntegration

object (MetadataIntegration)

Optional. The setting that defines how metastore metadata should be integrated with external services and systems.
maintenanceWindow

object (MaintenanceWindow)

The one hour maintenance window of the metastore service. This specifies when the service can be restarted for maintenance purposes in UTC time. Maintenance window is not needed for services with the SPANNER database type.
uid

string

Output only. The globally unique resource identifier of the metastore service.
metadataManagementActivity

object (MetadataManagementActivity)

Output only. The metadata management activities of the metastore service.
releaseChannel

enum (ReleaseChannel)

Immutable. The release channel of the service. If unspecified, defaults to STABLE.
encryptionConfig

object (EncryptionConfig)

Immutable. Information used to configure the Dataproc Metastore service to encrypt customer data at rest. Cannot be updated.
networkConfig

object (NetworkConfig)

The configuration specifying the network settings for the Dataproc Metastore service.
databaseType

enum (DatabaseType)

Immutable. The database type that the Metastore service stores its data.
telemetryConfig

object (TelemetryConfig)

The configuration specifying telemetry settings for the Dataproc Metastore service. If unspecified defaults to JSON.
scalingConfig

object (ScalingConfig)

Scaling configuration of the metastore service.
scheduledBackup

object (ScheduledBackup)

Optional. The configuration of scheduled backup for the metastore service.
deletionProtection

boolean

Optional. Indicates if the dataproc metastore should be protected against accidental deletions.
Union field metastore_config. Configuration properties specific to the underlying metastore service technology (the software that serves metastore queries). metastore_config can be only one of the following:
hiveMetastoreConfig

object (HiveMetastoreConfig)

Configuration information specific to running Hive metastore software as the metastore service.

HiveMetastoreConfig

Specifies configuration information specific to running Hive metastore software as the metastore service.

JSON representation 
{
  "version": string,
  "configOverrides": {
    string: string,
    ...
  },
  "kerberosConfig": {
    object (KerberosConfig)
  },
  "endpointProtocol": enum (EndpointProtocol),
  "auxiliaryVersions": {
    string: {
      object (AuxiliaryVersionConfig)
    },
    ...
  }
}
Fields
version

string

Immutable. The Hive metastore schema version.
configOverrides

map (key: string, value: string)

A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore (configured in hive-site.xml). The mappings override system defaults (some keys cannot be overridden). These overrides are also applied to auxiliary versions and can be further customized in the auxiliary version's AuxiliaryVersionConfig.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
kerberosConfig

object (KerberosConfig)

Information used to configure the Hive metastore service as a service principal in a Kerberos realm. To disable Kerberos, use the services.patch method and specify this field's path (hiveMetastoreConfig.kerberos_config) in the request's updateMask while omitting this field from the request's service.
endpointProtocol

enum (EndpointProtocol)

The protocol to use for the metastore service endpoint. If unspecified, defaults to THRIFT.
auxiliaryVersions

map (key: string, value: object (AuxiliaryVersionConfig))

A mapping of Hive metastore version to the auxiliary version configuration. When specified, a secondary Hive metastore service is created along with the primary service. All auxiliary versions must be less than the service's primary version. The key is the auxiliary service name and it must match the regular expression a-z?. This means that the first character must be a lowercase letter, and all the following characters must be hyphens, lowercase letters, or digits, except the last character, which cannot be a hyphen.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.

KerberosConfig

Configuration information for a Kerberos principal.

JSON representation 
{
  "keytab": {
    object (Secret)
  },
  "principal": string,
  "krb5ConfigGcsUri": string
}
Fields
keytab

object (Secret)

A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center (KDC).
principal

string

A Kerberos principal that exists in the both the keytab the KDC to authenticate as. A typical principal is of the form primary/instance@REALM, but there is no exact format.
krb5ConfigGcsUri

string

A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/to/krb5.conf, although the file does not need to be named krb5.conf explicitly.

Secret

A securely stored value.

JSON representation 
{

  // Union field value can be only one of the following:
  "cloudSecret": string
  // End of list of possible types for union field value.
}
Fields

Union field value.

value can be only one of the following:
cloudSecret

string

The relative resource name of a Secret Manager secret version, in the following form:

projects/{projectNumber}/secrets/{secret_id}/versions/{version_id}.

EndpointProtocol

Protocols available for serving the metastore service endpoint.

Enums
ENDPOINT_PROTOCOL_UNSPECIFIED The protocol is not set.
THRIFT Use the legacy Apache Thrift protocol for the metastore service endpoint.
GRPC Use the modernized gRPC protocol for the metastore service endpoint.

AuxiliaryVersionConfig

Configuration information for the auxiliary service versions.

JSON representation 
{
  "version": string,
  "configOverrides": {
    string: string,
    ...
  },
  "networkConfig": {
    object (NetworkConfig)
  }
}
Fields
version

string

The Hive metastore version of the auxiliary service. It must be less than the primary Hive metastore service's version.
configOverrides

map (key: string, value: string)

A mapping of Hive metastore configuration key-value pairs to apply to the auxiliary Hive metastore (configured in hive-site.xml) in addition to the primary version's overrides. If keys are present in both the auxiliary version's overrides and the primary version's overrides, the value from the auxiliary version's overrides takes precedence.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
networkConfig

object (NetworkConfig)

Output only. The network configuration contains the endpoint URI(s) of the auxiliary Hive metastore service.

NetworkConfig

Network configuration for the Dataproc Metastore service.

JSON representation 
{
  "consumers": [
    {
      object (Consumer)
    }
  ]
}
Fields
consumers[]

object (Consumer)

Immutable. The consumer-side network configuration for the Dataproc Metastore instance.

Consumer

Contains information of the customer's network configurations.

JSON representation 
{
  "endpointUri": string,
  "endpointLocation": string,

  // Union field vpc_resource can be only one of the following:
  "subnetwork": string
  // End of list of possible types for union field vpc_resource.
}
Fields
endpointUri

string

Output only. The URI of the endpoint used to access the metastore service.
endpointLocation

string

Output only. The location of the endpoint URI. Format: projects/{project}/locations/{location}.

Union field vpc_resource.

vpc_resource can be only one of the following:
subnetwork

string

Immutable. The subnetwork of the customer project from which an IP address is reserved and used as the Dataproc Metastore service's endpoint. It is accessible to hosts in the subnet and to all hosts in a subnet in the same region and same network. There must be at least one IP address available in the subnet's primary range. The subnet is specified in the following form:

projects/{projectNumber}/regions/{region_id}/subnetworks/{subnetwork_id}

State

The current state of the metastore service.

Enums
STATE_UNSPECIFIED The state of the metastore service is unknown.
CREATING The metastore service is in the process of being created.
ACTIVE The metastore service is running and ready to serve queries.
SUSPENDING The metastore service is entering suspension. Its query-serving availability may cease unexpectedly.
SUSPENDED The metastore service is suspended and unable to serve queries.
UPDATING The metastore service is being updated. It remains usable but cannot accept additional update requests or be deleted at this time.
DELETING The metastore service is undergoing deletion. It cannot be used.
ERROR The metastore service has encountered an error and cannot be used. The metastore service should be deleted.
AUTOSCALING The Dataproc Metastore service 2 is being scaled up or down.
MIGRATING The metastore service is processing a managed migration.

Tier

Available service tiers.

Enums
TIER_UNSPECIFIED The tier is not set.
DEVELOPER The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept.
ENTERPRISE The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads.

MetadataIntegration

Specifies how metastore metadata should be integrated with external services.

JSON representation 
{
  "dataCatalogConfig": {
    object (DataCatalogConfig)
  }
}
Fields
dataCatalogConfig

object (DataCatalogConfig)

Optional. The integration config for the Data Catalog service.

DataCatalogConfig

Specifies how metastore metadata should be integrated with the Data Catalog service.

JSON representation 
{
  "enabled": boolean
}
Fields
enabled

boolean

Optional. Defines whether the metastore metadata should be synced to Data Catalog. The default value is to disable syncing metastore metadata to Data Catalog.

MaintenanceWindow

Maintenance window. This specifies when Dataproc Metastore may perform system maintenance operation to the service.

JSON representation 
{
  "hourOfDay": integer,
  "dayOfWeek": enum (DayOfWeek)
}
Fields
hourOfDay

integer

The hour of day (0-23) when the window starts.
dayOfWeek

enum (DayOfWeek)

The day of week, when the window starts.

MetadataManagementActivity

The metadata management activities of the metastore service.

JSON representation 
{
  "metadataExports": [
    {
      object (MetadataExport)
    }
  ],
  "restores": [
    {
      object (Restore)
    }
  ]
}
Fields
metadataExports[]

object (MetadataExport)

Output only. The latest metadata exports of the metastore service.
restores[]

object (Restore)

Output only. The latest restores of the metastore service.

MetadataExport

The details of a metadata export operation.

JSON representation 
{
  "startTime": string,
  "endTime": string,
  "state": enum (State),
  "databaseDumpType": enum (Type),

  // Union field destination can be only one of the following:
  "destinationGcsUri": string
  // End of list of possible types for union field destination.
}
Fields
startTime

string (Timestamp format)

Output only. The time when the export started.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

Output only. The time when the export ended.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
state

enum (State)

Output only. The current state of the export.
databaseDumpType

enum (Type)

Output only. The type of the database dump.

Union field destination.

destination can be only one of the following:
destinationGcsUri

string

Output only. A Cloud Storage URI of a folder that metadata are exported to, in the form of gs://<bucket_name>/<path_inside_bucket>/<export_folder>, where <export_folder> is automatically generated.

State

The current state of the metadata export.

Enums
STATE_UNSPECIFIED The state of the metadata export is unknown.
RUNNING The metadata export is running.
SUCCEEDED The metadata export completed successfully.
FAILED The metadata export failed.
CANCELLED The metadata export is cancelled.

Restore

The details of a metadata restore operation.

JSON representation 
{
  "startTime": string,
  "endTime": string,
  "state": enum (State),
  "backup": string,
  "type": enum (RestoreType),
  "details": string,
  "backupLocation": string
}
Fields
startTime

string (Timestamp format)

Output only. The time when the restore started.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

Output only. The time when the restore ended.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
state

enum (State)

Output only. The current state of the restore.
backup

string

Output only. The relative resource name of the metastore service backup to restore from, in the following form:

projects/{projectId}/locations/{locationId}/services/{serviceId}/backups/{backupId}.
type

enum (RestoreType)

Output only. The type of restore.
details

string

Output only. The restore details containing the revision of the service to be restored to, in format of JSON.
backupLocation

string

Optional. A Cloud Storage URI specifying where the backup artifacts are stored, in the format gs://<bucket_name>/<path_inside_bucket>.

State

The current state of the restore.

Enums
STATE_UNSPECIFIED The state of the metadata restore is unknown.
RUNNING The metadata restore is running.
SUCCEEDED The metadata restore completed successfully.
FAILED The metadata restore failed.
CANCELLED The metadata restore is cancelled.

RestoreType

The type of restore. If unspecified, defaults to METADATA_ONLY.

Enums
RESTORE_TYPE_UNSPECIFIED The restore type is unknown.
FULL The service's metadata and configuration are restored.
METADATA_ONLY Only the service's metadata is restored.

ReleaseChannel

Release channels bundle features of varying levels of stability. Newer features may be introduced initially into less stable release channels and can be automatically promoted into more stable release channels.

Enums
RELEASE_CHANNEL_UNSPECIFIED Release channel is not specified.
CANARY The CANARY release channel contains the newest features, which may be unstable and subject to unresolved issues with no known workarounds. Services using the CANARY release channel are not subject to any SLAs.
STABLE The STABLE release channel contains features that are considered stable and have been validated for production use.

EncryptionConfig

Encryption settings for the service.

JSON representation 
{
  "kmsKey": string
}
Fields
kmsKey

string

The fully qualified customer provided Cloud KMS key name to use for customer data encryption, in the following format:

projects/{projectNumber}/locations/{locationId}/keyRings/{key_ring_id}/cryptoKeys/{crypto_key_id}.

DatabaseType

The backend database type for the metastore service.

Enums
DATABASE_TYPE_UNSPECIFIED The DATABASE_TYPE is not set.
MYSQL MySQL is used to persist the metastore data.
SPANNER Spanner is used to persist the metastore data.

TelemetryConfig

Telemetry Configuration for the Dataproc Metastore service.

JSON representation 
{
  "logFormat": enum (LogFormat)
}
Fields
logFormat

enum (LogFormat)

The output format of the Dataproc Metastore service's logs.

LogFormat

Enums
LOG_FORMAT_UNSPECIFIED The LOG_FORMAT is not set.
LEGACY Logging output uses the legacy textPayload format.
JSON Logging output uses the jsonPayload format.

ScalingConfig

Represents the scaling configuration of a metastore service.

JSON representation 
{
  "autoscalingConfig": {
    object (AutoscalingConfig)
  },

  // Union field scaling_model can be only one of the following:
  "instanceSize": enum (InstanceSize),
  "scalingFactor": number
  // End of list of possible types for union field scaling_model.
}
Fields
autoscalingConfig

object (AutoscalingConfig)

Optional. The autoscaling configuration.
Union field scaling_model. Represents either a predetermined instance size or a numeric scaling factor. scaling_model can be only one of the following:
instanceSize

enum (InstanceSize)

An enum of readable instance sizes, with each instance size mapping to a float value (e.g. InstanceSize.EXTRA_SMALL = scalingFactor(0.1))
scalingFactor

number

Scaling factor, increments of 0.1 for values less than 1.0, and increments of 1.0 for values greater than 1.0.

InstanceSize

Metastore instance sizes.

Enums
INSTANCE_SIZE_UNSPECIFIED Unspecified instance size
EXTRA_SMALL Extra small instance size, maps to a scaling factor of 0.1.
SMALL Small instance size, maps to a scaling factor of 0.5.
MEDIUM Medium instance size, maps to a scaling factor of 1.0.
LARGE Large instance size, maps to a scaling factor of 3.0.
EXTRA_LARGE Extra large instance size, maps to a scaling factor of 6.0.

AutoscalingConfig

Represents the autoscaling configuration of a metastore service.

JSON representation 
{
  "autoscalingFactor": number,
  "autoscalingEnabled": boolean,
  "limitConfig": {
    object (LimitConfig)
  }
}
Fields
autoscalingFactor

number

Output only. The scaling factor of a service with autoscaling enabled.
autoscalingEnabled

boolean

Optional. Whether or not autoscaling is enabled for this service.
limitConfig

object (LimitConfig)

Optional. The LimitConfig of the service.

LimitConfig

Represents the autoscaling limit configuration of a metastore service.

JSON representation 
{
  "maxScalingFactor": number,
  "minScalingFactor": number
}
Fields
maxScalingFactor

number

Optional. The highest scaling factor that the service should be autoscaled to.
minScalingFactor

number

Optional. The lowest scaling factor that the service should be autoscaled to.

ScheduledBackup

This specifies the configuration of scheduled backup.

JSON representation 
{
  "enabled": boolean,
  "cronSchedule": string,
  "timeZone": string,
  "nextScheduledTime": string,
  "backupLocation": string,
  "latestBackup": {
    object (LatestBackup)
  }
}
Fields
enabled

boolean

Optional. Defines whether the scheduled backup is enabled. The default value is false.
cronSchedule

string

Optional. The scheduled interval in Cron format, see https://en.wikipedia.org/wiki/Cron The default is empty: scheduled backup is not enabled. Must be specified to enable scheduled backups.
timeZone

string

Optional. Specifies the time zone to be used when interpreting cronSchedule. Must be a time zone name from the time zone database (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g. America/Los_Angeles or Africa/Abidjan. If left unspecified, the default is UTC.
nextScheduledTime

string (Timestamp format)

Output only. The time when the next backups execution is scheduled to start.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
backupLocation

string

Optional. A Cloud Storage URI of a folder, in the format gs://<bucket_name>/<path_inside_bucket>. A sub-folder <backup_folder> containing backup files will be stored below it.
latestBackup

object (LatestBackup)

Output only. The details of the latest scheduled backup.

LatestBackup

The details of the latest scheduled backup.

JSON representation 
{
  "backupId": string,
  "startTime": string,
  "state": enum (State),
  "duration": string
}
Fields
backupId

string

Output only. The ID of an in-progress scheduled backup. Empty if no backup is in progress.
startTime

string (Timestamp format)

Output only. The time when the backup was started.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
state

enum (State)

Output only. The current state of the backup.
duration

string (Duration format)

Output only. The duration of the backup completion.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

State

The current state of the backup.

Enums
STATE_UNSPECIFIED The state of the backup is unknown.
IN_PROGRESS The backup is in progress.
SUCCEEDED The backup completed.
FAILED The backup failed.

Methods

alterLocation

 Alter metadata resource location.

alterTableProperties

 Alter metadata table properties.

cancelMigration

 Cancels the ongoing Managed Migration process.

completeMigration

 Completes the managed migration process.

create

 Creates a metastore service in a project and location.

delete

 Deletes a single service.

exportMetadata

 Exports metadata from a service.

get

 Gets the details of a single service.

getIamPolicy

 Gets the access control policy for a resource.

list

 Lists services in a project and location.

moveTableToDatabase

 Move a table to another database.

patch

 Updates the parameters of a single service.

queryMetadata

 Query Dataproc Metastore metadata.

restore

 Restores a service from a backup.

setIamPolicy

 Sets the access control policy on the specified resource.

startMigration

 Starts the Managed Migration process.

testIamPermissions

 Returns permissions that a caller has on the specified resource.