Choose an endpoint protocol for Dataproc Metastore

This page explains how to choose an endpoint protocol for Dataproc Metastore.

You first choose your endpoint protocol when you create a Dataproc Metastore service. This choice affects the features that you can integrate and use with your service. It's important to note that once you choose an endpoint protocol, there are some restrictions if you want to change it. Make sure you choose the appropriate protocol for your needs.

To understand the differences between endpoint protocols, see About endpoint protocols.

Before you begin

Required Roles

To get the permission that you need to create a Dataproc Metastore, ask your administrator to grant you the following IAM roles on your project, based on the principle of least privilege:

For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the metastore.services.create permission, which is required to create a Dataproc Metastore.

You might also be able to get this permission with custom roles or other predefined roles.

For more information about specific Dataproc Metastore roles and permissions, see Manage Dataproc access with IAM.

Choose an endpoint protocol

The following examples show an abbreviated version of the steps that you follow to choose an endpoint protocol. For complete step-by-step instructions on the entire process you must follow, see Create a Dataproc Metastore.

For a new Dataproc Metastore service

To create a Dataproc Metastore service and choose the endpoint protocol for the first time, follow these instructions:

Console

  1. In the Google Cloud console, open the Dataproc Metastore page:

    Open Dataproc Metastore

  2. In the Dataproc Metastore Navigation menu, click Create.

    The Create service page opens.

  3. In the Endpoint protocol section, select Thrift or gRPC.

    Thrift is the default preselected endpoint protocol.

    You can only update an existing endpoint protocol from Thrift to gRPC. If you want to move from gRPC to Thrift, create a new Dataproc Metastore service.

  4. Choose the remaining configurations for your service, as needed.

  5. To create and start the service, click Submit.

    If you selected a gRPC endpoint, you must grant additional IAM permissions to your service. For more information, see Access gRPC endpoints.

gcloud CLI

  1. To choose an endpoint protocol, run the following gcloud metastore services create command:

    gcloud metastore services create SERVICE \
   --endpoint-protocol=ENDPOINT_PROTOCOL ; default="thrift"

    Replace the following:

    • SERVICE_ID: The name or ID for your Dataproc Metastore service.

    • ENDPOINT_PROTOCOL: The endpoint protocol that you want to use for your Dataproc Metastore. You can specify either thrift or grpc.

      Thrift is the default preselected endpoint protocol.

      You can only update an existing endpoint protocol from Thrift to gRPC. If you want to move from gRPC to Thrift, you must create a new service.

  2. Verify that your service was successfully created.

    If you selected a gRPC endpoint protocol, you must add additional IAM permissions to your account. For more information, see Access gRPC endpoints.

For an existing Dataproc Metastore service

To update the endpoint protocol of an existing Dataproc Metastore service, follow these instructions:

Console

  1. In the Google Cloud console, open the Dataproc Metastore page:

    Open Dataproc Metastore

  2. On the Dataproc Metastore page, click the service to update.

    The Service detail page opens.

  3. On the Configuration tab, click Edit.

    The Edit service page opens.

  4. In the Endpoint protocol section, select gRPC.

  5. To update the service, click Submit.

    After updating to the gRPC endpoint protocol, you must add additional IAM permissions to your account. For more information, see Access gRPC endpoints.

gcloud CLI

  1. To update an endpoint protocol, run the following gcloud metastore services update command:

    gcloud metastore services update SERVICE \
   --endpoint-protocol=ENDPOINT_PROTOCOL

    Replace the following:

    • SERVICE: The name of your Dataproc Metastore service.
    • ENDPOINT_PROTOCOL: The endpoint protocol that you want update for your Dataproc Metastore service. You can only specify grpc.

  2. Verify that your update was successful.

    After updating to a gRPC endpoint, you must grant additional IAM permissions to your service. For more information, see Access gRPC endpoints.

Find your endpoint URI value

Console

You can view your endpoint URI on the Dataproc Metastore page.

  • For Thrift endpoints, The URL value starts with thrift://
  • For gRPC endpoints, the URL value starts with https://

The following image shows an example of a Dataproc Metastore service configured with a gRPC endpoint protocol.

Service detail URL and hive.metastore.warehouse.dir values

gcloud CLI

To view your endpoint URI, run the following gcloud metastore services describe command:

gcloud metastore services describe SERVICE --format="get(endpointUri)"

Replace the following:

  • SERVICE: The name of your Dataproc Metastore service.

What's next