Choose the database type

You can choose the database type for your Dataproc Metastore service to fit your performance needs. You can choose between a MySQL or Spanner database type.

MySQL is the default database type. It's supported for all Hive versions. Spanner-backed Dataproc Metastore services are supported for Hive versions 2.3.6 and 3.1.2.

Differences between MySQL and Spanner database types

The MySQL database type uses Cloud SQL, while the Spanner database type uses Cloud Spanner. MySQL supports encryption features, but lacks scalability. Spanner provides more reliability with the trade-off of increased latency.

The following table lists the differences between MySQL and Spanner database types:

MySQL Spanner
Reliability (uptime) Cloud SQL SLO 99.95%* Cloud Spanner SLO 99.99%*
Latency No change in latency Increased latency of about 50%
CMEK Supported Not Supported
Maintenance windows Required Not Required

Notes:

  • *The Cloud SQL and Cloud Spanner SLOs don't directly translate to Dataproc Metastore SLOs. Dataproc Metastore SLOs are not affected by your database type selection.
  • There is no pricing difference between the two database types.

Choose the database type for a new Dataproc Metastore service

You can choose the database type for your Dataproc Metastore service when you create it.

Console

  1. In the console, open the Dataproc Metastore page:

    Open Dataproc Metastore in the console

  2. At the top of the Dataproc Metastore page, click the Create button. The Create service page opens.

  3. Configure your service as desired.

  4. Under Database type, select MySQL or Spanner. MySQL is the default database type.

  5. Click Submit.

gcloud

  1. Run the following gcloud beta metastore services create command:

    gcloud beta metastore services create SERVICE \
   --location=LOCATION \
   --database-type=DATABASE_TYPE; default="mysql"

  2. Verify that the creation was successful.

Database type caveats

Spanner-backed Dataproc Metastore services have the following caveats:

  • Only support Hive versions 2.3.6 and 3.1.2.

  • Don't support Dataproc Metastore service to Data Catalog sync.

  • Not supported with Auxiliary versions.

  • Don't support import, export, backup, and restore.

What's next