Using customer-managed encryption keys (CMEK)

This page describes how to set up a service account and keys for customer-managed encryption keys, and how to create an instance that uses a customer-managed encryption key. To learn more about using customer-managed encryption keys with Cloud SQL, see Overview of customer-managed encryption keys.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to the project selector page

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Install and initialize the Cloud SDK.
  5. Make sure you have the Cloud SQL Admin role on your user account.

    Go to the IAM page

  6. Enable the Cloud Key Management Service API.

    Enable the API

  7. Enable the Cloud SQL Admin API.

    Enable the API

Workflow for creating a Cloud SQL instance with CMEK

  1. gcloud and API users only: Create a service account for each project that requires customer-managed encryption keys.
  2. Create a keyring and key, and set the location for each key.
  3. gcloud and API users only: Grant the key access to the service account.
  4. Copy or write down the key ID (KMS_RESOURCE_ID) and location for the key, and the ID (KMS_RESOURCE_ID) for the keyring. You need this information when granting the key access to the service account.
  5. Go to a project and create a Cloud SQL instance with the following options:
    1. The same location as the customer-managed encryption key
    2. The customer-managed key configuration
    3. The customer-managed encryption key ID

Your Cloud SQL instance is now enabled with CMEK.

Creating a service account

You need to create a service account for each project that requires customer-managed encryption keys.

To allow a user to manage service accounts, grant one of the following roles:

  • Service Account User (roles/iam.serviceAccountUser): Includes permissions to list service accounts, get details about a service account, and impersonate a service account.
  • Service Account Admin (roles/iam.serviceAccountAdmin): Includes permissions to list service accounts and get details about a service account. Also includes permissions to create, update, and delete service accounts, and to view or change the Cloud SQL for MySQL policy on a service account.

Currently, you can only use gcloud command-line tool commands to create the type of service account you need for customer-managed encryption keys. If you are using the Console, Cloud SQL automatically creates this service account for you.

To create a service account with gcloud command-line tool, run the following command:

gcloud beta services identity create --service=sqladmin.googleapis.com \
             --project=[USER_PROJECT]

The previous command returns a service account name, using the following format:

service-[PROJECT_NUMBER]@gcp-sa-cloud-sql.iam.gserviceaccount.com

For example:

service-534582264470@gcp-sa-cloud-sql.iam.gserviceaccount.com

You use this service account name during the procedure in Granting the key access to the service account.

Creating a key

You can create the key in the same Google Cloud project as the Cloud SQL instance or in a separate user project. The Cloud KMS key ring location must match the region where you want to create Cloud SQL instance. A multi-region or global region key will not work. The Cloud SQL instance create request fails if the regions don't match.

To create a Cloud KMS key:

Console

  1. Go to the Cryptographic keys page.
    Go to the Cryptographic Keys page
  2. Click CREATE KEY RING.
  3. Add a Key ring name. Write down this name because you need it when granting the key access to the service account.
  4. Add a Key ring location.
  5. Click CREATE. The Create key page opens.
  6. Add a Key name.
  7. Select a Purpose (symmetric or asymmetric).
  8. Select a Rotation period and Starting on date.
  9. Click CREATE.
  10. On the Keys table, click the three dots in the last column, and select Copy Resource ID or write it down. This is the KMS_RESOURCE_ID. You need the KMS_RESOURCE_ID when granting the key access to the service account.

gcloud

  1. Create a new key ring.
         gcloud kms keyrings create [KEYRING_NAME] --location [LOCATION]
      
    Write down this name because you need it when granting the key access to the service account.
  2. Create a key on the key ring.
         gcloud kms keys create [KEY_NAME] --location [LOCATION] \
               --keyring [KEYRING_NAME] --purpose encryption
         
    Write down this name because you need it when granting the key access to the service account.

Granting the key access to the service account

You only need to perform this procedure if you are using gcloud or the API.

To grant access to the service account:

  gcloud kms keys add-iam-policy-binding 
[KEY_NAME] --location [LOCATION] --keyring [KEY_RING_NAME]
--member serviceAccount:service-[PROJECT_NUMBER]@gcp-sa-cloud-sql.iam.gserviceaccount.com
--role roles/cloudkms.cryptoKeyEncrypterDecrypter

Creating a Cloud SQL instance with CMEK

To create an instance with customer-managed encryption keys:

Console

  1. Go to the Cloud SQL Instances page in the Google Cloud Console.

    Go to the Cloud SQL Instances page

  2. Click Create instance.
  3. Choose the database engine.
  4. Enter a name for the instance. Do not include sensitive or personally identifiable information in your instance name; it is externally visible. You do not need to include the project ID in the instance name. This is created automatically where appropriate (for example, in the log files).
  5. Enter the password for the 'root'@'%' user.
  6. Set the region for your instance. Place your instance in the same region as the resources that access it. In most cases, you don't need to specify a zone.
  7. Under Configuration options, select all your configurations options until you reach Machine type and storage.
  8. Expand Machine type and storage.
  9. Under Encryption, choose Customer-managed key.
  10. Select the KMS key from the dropdown menu or manually enter the KMS_RESOURCE_ID. Only KMS keys in the same project and region as the instance are displayed. To choose a KMS key belonging to a different project but in the same region, select Don't see your key? Enter key resource ID and enter the KMS_RESOURCE_ID captured earlier.
  11. If the service account does not have permission to encrypt/decrypt with the selected key, a message displays. If this happens, click Grant to grant the service account the IAM role on the selected KMS key.
  12. Once the configuration options are selected, click Create.
  13. You see a message explaining the implications of using customer-managed encryption key. Read and acknowledge it to proceed further with instance creation.

gcloud

  gcloud sql instances create [INSTANCE_NAME] \
  --project [PROJECT_ID] \
  --disk-encryption-key [KMS_RESOURCE_ID]
  --database-version=[VERSION]
  --tier= [MACHINE_TYPE] \
  --region=[REGION] \
  --root-password=[INSERT-PASSWORD-HERE]
  

REST v1beta4

To create an instance with customer-managed encryption keys, pass diskEncryptionConfiguration to the command.

Before using any of the request data below, make the following replacements:

  • project-id: The project ID
  • instance-id: The instance ID
  • region: The region
  • database-version: Emum version string (for example, MYSQL_8_0)
  • kms-resource-id: The ID you received from creating a key.

HTTP method and URL:

POST https://www.googleapis.com/sql/v1beta4/projects/project-id/instances

Request JSON body:

{
  "name":"instance-id",
  "region":"region",
  "databaseVersion": "database-version",
  "diskEncryptionConfiguration" :
  {
      "kmsKeyName" : "kms-resource-id"
  },
  "settings":
  {
     "backupConfiguration":
      {
         "binaryLogEnabled": true,
         "enabled":true
      }
   }
}

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://www.googleapis.com/sql/v1beta4/projects/project-id/instances",
  "status": "PENDING",
  "user": "user@example.com",
  "insertTime": "2020-01-16T02:32:12.281Z",
  "operationType": "UPDATE",
  "name": "operation-id",
  "targetId": "instance-id",
  "selfLink": "https://www.googleapis.com/sql/v1beta4/projects/project-id/operations/operation-id",
  "targetProject": "project-id"
}
This REST API uses Instances:Insert.

Creating a backup for a CMEK-enabled instance

When you create a backup of a Cloud SQL instance, the backup is encrypted with the same primary key version as the primary instance's customer-managed key.

See Creating and managing on-demand and automatic backups.

You will see a message on the Create a backup form that says: "Your backup will be encrypted with the primary version of this instance's customer-managed encryption key. If anyone destroys or disables this key version, all backup data encrypted using that key version will be permanently lost. You can check the primary version for the key in Cloud KMS."

On the backups page, the list of backups enabled with customer-managed encryption keys have two extra columns. One column that shows that the backup is for a CMEK-enabled instance, and a column that displays the key version used to encrypt the backup.

Creating a replica for a CMEK-enabled instance

When you create a read replica of a Cloud SQL instance in the same region, it inherits the same customer-managed encryption key as the parent instance. If you create a read replica in a different region, you are given a new list of customer-managed encryption keys to select from. Each region uses its own set of keys.

See Creating read replicas.

On the Create read replica page, you see the following information:

  1. The Encryption shows customer-managed.
  2. The Encryption key for the replica is displayed.
  3. There is a message saying "Your replica will be encrypted with the primary instance's customer-managed key. If anyone destroys this key, all data encrypted with it will be permanently lost."

For a cross-region replica, you see the following message:

"Your cross-region replica will be encrypted with the selected customer-managed key. If anyone destroys this key, all data encrypted with it will be permanently lost."

Creating a clone of a CMEK-enabled instance

When you create a clone of Cloud SQL instance, it inherits the same customer-managed encryption key as the one used to encrypt the source instance.

See Cloning instances.

On the Create clone page, you see the following information:

  1. The Encryption shows customer-managed.
  2. The Encryption key for the clone is displayed.
  3. There is a message saying "Your clone will be encrypted with the source instance's customer-managed key. If anyone destroys this key, all data encrypted with it will be permanently lost."

Viewing key information for a CMEK-enabled instance

Once you successfully create a Cloud SQL instance, you can look at the instance list or the instance overview page to see that it was created using a customer-managed encryption key. The details also show the key that was used to create the instance.

  1. Go to the Cloud SQL Instances page.

    Go to the Cloud SQL Instances page

  2. In the Instances list, scroll to the right until you see the Encryption column. In this column, you see Google-managed and Customer-managed.

  3. Click an instance name to open its Overview page. The customer-managed encryption key is listed in the Configuration pane.

Disabling and re-enabling key versions

See the following topics:

Troubleshooting

This section describes things to try when you get an error message while setting up or using CMEK-enabled instances.

Cloud SQL administrator operations, such as create, clone, or update, might fail due to Cloud KMS errors, and missing roles or permissions. Common reasons for failure include a missing Cloud KMS key version, a disabled or destroyed Cloud KMS key version, insufficient IAM permissions to access the Cloud KMS key version, or the Cloud KMS key version is in a different region than the Cloud SQL instance. Use the following troubleshooting table to diagnose and resolve common problems.

Customer-managed encryption keys troubleshooting table

For this error... The issue might be... Try this...
Per-product, per-project service account not found The service account name is incorrect. Make sure you created a service account for the correct user project.

GO TO THE SERVICE ACCOUNTS PAGE.

Cannot grant access to the service account The user account does not have permission to grant access to this key version. Add the Organization Administrator role on your user or service account.

GO TO THE IAM ACCOUNTS PAGE

Cloud KMS key version is destroyed The key version is destroyed. If the key version is destroyed, you cannot use it to encrypt or decrypt data.
Cloud KMS key version is disabled The key version is disabled. Re-enable the Cloud KMS key version.

GO TO THE CRYPTO KEYS PAGE

Insufficient permission to use the Cloud KMS key The cloudkms.cryptoKeyEncrypterDecrypter role is missing on the user or service account you are using to run operations on Cloud SQL instances, or the Cloud KMS key version doesn't exist. Add the cloudkms.cryptoKeyEncrypterDecrypter role on your user or service account.

GO TO THE IAM ACCOUNTS PAGE


If the role is already on your account, see Creating a key to learn how to create a new key version. See note.
Cloud KMS key is not found The key version does not exist. Create a new key version. See Creating a key. See note.
Cloud SQL instance and Cloud KMS key version are in different regions The Cloud KMS key version and Cloud SQL instance must be in the same region. It does not work if the Cloud KMS key version is in a global region or multi-region. Create a key version in the same region where you want to create instances. See Creating a key. See note.

What's next