VPC Service Controls with Dataproc Metastore

To further secure your Dataproc Metastore services, you can protect them using VPC Service Controls (VPC-SC).

VPC Service Controls helps mitigate the risk of data exfiltration. Using VPC Service Controls, you can add projects to service perimeters that protect resources and services from requests that cross the perimeter.

To learn more about VPC Service Controls, see Overview of VPC Service Controls.

Dataproc Metastore resources are exposed on the metastore.googleapis.com API, which allows you to perform service-level operations, such as the creation and deletion of services.

You set up VPC Service Controls with Dataproc Metastore by restricting connectivity to this API surface.

Configure the Virtual Private Cloud (VPC) network

You can configure the VPC network to restrict Private Google Access with respect to a service perimeter. This ensures that hosts on your VPC or on-premises network can only communicate with Google APIs and services that are supported by VPC Service Controls in ways which conform to the associated perimeter's policy.

For more information, see Setting up private connectivity to Google APIs and services.

Create a service perimeter

During this procedure, you select the Dataproc Metastore projects that you want the VPC service perimeter to protect.

To create a service perimeter, follow the instructions in Creating a service perimeter.

Add more projects to the service perimeter

To add existing Dataproc Metastore projects to the perimeter, follow the instructions in Updating a service perimeter.

Note: If your Dataproc Metastore service uses a network belonging to a different project, and Compute Engine is protected by the service perimeter, then the metastore project and network project must be in the same perimeter.

Add the Dataproc Metastore and Cloud Storage APIs to the service perimeter

To mitigate the risk of your data being exfiltrated from Dataproc Metastore, for example, when using Dataproc Metastore import or export APIs, you must restrict both the Dataproc Metastore API and the Cloud Storage API.

Note: You can only import or export data from a Cloud Storage bucket that is in a project that resides in the same service perimeter as the Dataproc Metastore service. If you've configured Kerberos for a service, then the Secret Manager secret and krb5.conf Cloud Storage object must belong to a project that resides in the same service perimeter as the service.

To add Dataproc Metastore and Cloud Storage APIs as restricted services:

Console

1. In the Google Cloud console, open the VPC Service Controls page:

   Go to the VPC Service Controls page in the Google Cloud console

2. On the VPC Service Controls page, in the table, click the name of the service perimeter that you want to modify.

3. Click Edit Perimeter.

4. On the Edit VPC Service Perimeter page, click Add Services.

5. Add Dataproc Metastore API and Cloud Storage API.

6. Click Save.

gcloud

Run the following gcloud access-context-manager perimeters update command:

 gcloud access-context-manager perimeters update PERIMETER_ID 

     --policy=POLICY_ID 

     --add-restricted-services=metastore.googleapis.com,storage.googleapis.com
 

Replace the following:

• PERIMETER_ID: The ID of the perimeter or the fully qualifed identifier for the perimeter.
• POLICY_ID: The ID of the access policy.

Create an access level

Optionally, to permit external access to protected resources inside a perimeter, you can use access levels. Access levels apply only to requests for protected resources coming from outside the service perimeter. You can't use access levels to give protected resources permission to access data and services outside the perimeter.

See Allowing access to protected resources from outside a perimeter.

What's next

• VPC Service Controls
• Service access
• Dataproc Metastore IAM and access control