VPC Service Controls with Dataproc Metastore

To further secure your Dataproc Metastore services, you can protect them using VPC Service Controls (VPC-SC).

VPC Service Controls helps mitigate the risk of data exfiltration. Using VPC Service Controls, you can add projects to service perimeters that protect resources and services from requests that cross the perimeter.

To learn more about VPC Service Controls, see Overview of VPC Service Controls.

Dataproc Metastore resources are exposed on the metastore.googleapis.com API, which allows you to perform service-level operations, such as the creation and deletion of services.

You set up VPC Service Controls with Dataproc Metastore by restricting connectivity to this API surface.

Configuring the Virtual Private Cloud (VPC) network

You can configure the VPC network to restrict Private Google Access with respect to a service perimeter. This ensures that hosts on your VPC or on-premises network can only communicate with Google APIs and services that are supported by VPC Service Controls in ways which conform to the associated perimeter's policy.

For more information, see Setting up private connectivity to Google APIs and services.

Creating a service perimeter

During this procedure, you select the Dataproc Metastore projects that you want the VPC service perimeter to protect.

To create a service perimeter, follow the instructions in Creating a service perimeter.

Adding more projects to the service perimeter

To add existing Dataproc Metastore projects to the perimeter, follow the instructions in Updating a service perimeter.

Adding the Dataproc Metastore and Cloud Storage APIs to the service perimeter

To mitigate the risk of your data being exfiltrated from Dataproc Metastore, for example, when using Dataproc Metastore import or export APIs, you must restrict both the Dataproc Metastore API and the Cloud Storage API.

To add Dataproc Metastore and Cloud Storage APIs as restricted services:

Console

  1. In the Cloud Console, open the VPC Service Controls page:

    Go to the VPC Service Controls page in the Cloud Console

  2. On the VPC Service Controls page, in the table, click the name of the service perimeter that you want to modify.

  3. Click Edit Perimeter.

  4. On the Edit VPC Service Perimeter page, click Add Services.

  5. Add Dataproc Metastore API and Cloud Storage API.

  6. Click Save.

gcloud

Run the following gcloud access-context-manager perimeters update command:

 gcloud access-context-manager perimeters update PERIMETER_ID 
--policy=POLICY_ID
--add-restricted-services=metastore.googleapis.com,storage.googleapis.com

Replace the following:

  • PERIMETER_ID: The ID of the perimeter or the fully qualifed identifier for the perimeter.
  • POLICY_ID: The ID of the access policy.

Creating an access level

Optionally, to permit external access to protected resources inside a perimeter, you can use access levels. Access levels apply only to requests for protected resources coming from outside the service perimeter. You can't use access levels to give protected resources permission to access data and services outside the perimeter.

See Allowing access to protected resources from outside a perimeter.

What's next