This page describes how to use customer-managed encryption keys (CMEK) for Spanner.
To learn more about CMEK, see Customer-managed encryption keys (CMEK).
Create a CMEK-enabled database
Create a key in Cloud Key Management Service (Cloud KMS). Spanner supports:
The key must be in the same location as your Spanner instance. For example, if your Spanner instance configuration is
us-west1
, then your key ring location must also beus-west1
.Not every Spanner multi-region instance configuration has a corresponding key ring location in Cloud KMS. You will not be able to create CMEK-enabled databases in those Spanner instances.
If you already have a Cloud KMS key in the correct location, you can skip this step.
For more information, see the following resources:
Grant Spanner access to the key.
In Cloud Shell, create and display the service agent, or display it if the account already exists:
gcloud beta services identity create --service=spanner.googleapis.com \ --project=my_spanner_project
If you're prompted to install the gcloud Beta Commands component, type
Y
. After installation, the command is automatically restarted.The gcloud services identity command creates or gets the service agent that Spanner can use to access the Cloud KMS key on your behalf.
The service account ID is formatted like an email address:
Service identity created: service-xxx@gcp-sa-spanner.iam.gserviceaccount.com
Grant the
cloudkms.cryptoKeyEncrypterDecrypter
role to the service account:gcloud kms keys add-iam-policy-binding my_kms_key \ --location my_kms_key_location \ --keyring my_kms_key_ring \ --project=my_kms_project \ --member serviceAccount:service-xxx@gcp-sa-spanner.iam.gserviceaccount.com \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter
You will see:
Updated IAM policy for key [my-kms-key]
This role ensures the service account has permission to both encrypt and decrypt with the Cloud KMS key. For more information, see Cloud KMS Permissions and Roles.
Create the database and specify your Cloud KMS key.
Console
1. Go to the Spanner Instances page in the Google Cloud console.
2. Click the instance name in which to create the database.
3. Click Create database and fill out the required fields.
4. Click Show encryption options.
5. Select Use a customer-managed encryption key (CMEK).
6. Select your key from the drop-down list.
The list of keys is limited to the current Google Cloud project. To use a key from a different Google Cloud project, create the database using gcloud instead of the Google Cloud console.
Once the database is created, you can verify that the database is CMEK-enabled by viewing the Database details page.
gcloud
gcloud spanner databases create example_db \ --project=my_spanner_project \ --instance=my_spanner_instance \ --ddl="CREATE TABLE Users (Id INT64 NOT NULL, FirstName STRING(100) NOT NULL, LastName STRING(100) NOT NULL,) PRIMARY KEY (Id)" \ --kms-project=my_kms_project \ --kms-location=my_kms_key_location \ --kms-keyring=my_kms_key_ring \ --kms-key=my_kms_key
To verify that a database is CMEK-enabled:
gcloud spanner databases describe example_db \ --project=my_spanner_project \ --instance=my_spanner_instance
CMEK-enabled databases include a field for
encryptionConfig
, as shown in this example:encryptionConfig: kmsKeyName:projects/my-kms-project/locations/eur5/keyRings/my-kms-key-ring/cryptoKeys/my-kms-key name: projects/my-spanner-project/instances/my-instance/databases/my-db state: READY
C#
using Google.Cloud.Spanner.Admin.Database.V1; using Google.Cloud.Spanner.Common.V1; using System; using System.Threading.Tasks; public class CreateDatabaseWithEncryptionKeyAsyncSample { public async Task<Database> CreateDatabaseWithEncryptionKeyAsync(string projectId, string instanceId, string databaseId, CryptoKeyName kmsKeyName) { // Create a DatabaseAdminClient instance that can be used to execute a // CreateDatabaseRequest with custom encryption configuration options. DatabaseAdminClient databaseAdminClient = DatabaseAdminClient.Create(); // Define create table statement for table #1. var createSingersTable = @"CREATE TABLE Singers ( SingerId INT64 NOT NULL, FirstName STRING(1024), LastName STRING(1024), ComposerInfo BYTES(MAX) ) PRIMARY KEY (SingerId)"; // Define create table statement for table #2. var createAlbumsTable = @"CREATE TABLE Albums ( SingerId INT64 NOT NULL, AlbumId INT64 NOT NULL, AlbumTitle STRING(MAX) ) PRIMARY KEY (SingerId, AlbumId), INTERLEAVE IN PARENT Singers ON DELETE CASCADE"; // Create the CreateDatabase request with encryption configuration and execute it. var request = new CreateDatabaseRequest { ParentAsInstanceName = InstanceName.FromProjectInstance(projectId, instanceId), CreateStatement = $"CREATE DATABASE `{databaseId}`", ExtraStatements = { createSingersTable, createAlbumsTable }, EncryptionConfig = new EncryptionConfig { KmsKeyNameAsCryptoKeyName = kmsKeyName, }, }; var operation = await databaseAdminClient.CreateDatabaseAsync(request); // Wait until the operation has finished. Console.WriteLine("Waiting for the operation to finish."); var completedResponse = await operation.PollUntilCompletedAsync(); if (completedResponse.IsFaulted) { Console.WriteLine($"Error while creating database: {completedResponse.Exception}"); throw completedResponse.Exception; } var database = completedResponse.Result; Console.WriteLine($"Database {database.Name} created with encryption key {database.EncryptionConfig.KmsKeyName}"); return database; } }
C++
Go
Java
Node.js
PHP
Python
Ruby
# project_id = "Your Google Cloud project ID" # instance_id = "Your Spanner instance ID" # database_id = "Your Spanner database ID" # kms_key_name = "Database eencryption KMS key" require "google/cloud/spanner" spanner = Google::Cloud::Spanner.new project: project_id instance = spanner.instance instance_id job = instance.create_database database_id, statements: [ "CREATE TABLE Singers ( SingerId INT64 NOT NULL, FirstName STRING(1024), LastName STRING(1024), SingerInfo BYTES(MAX) ) PRIMARY KEY (SingerId)", "CREATE TABLE Albums ( SingerId INT64 NOT NULL, AlbumId INT64 NOT NULL, AlbumTitle STRING(MAX) ) PRIMARY KEY (SingerId, AlbumId), INTERLEAVE IN PARENT Singers ON DELETE CASCADE" ], encryption_config: { kms_key_name: kms_key_name } puts "Waiting for create database operation to complete" job.wait_until_done! database = job.database puts "Database #{database.database_id} created with encryption key #{database.encryption_config.kms_key_name}"
View the key versions in use
Information on key versions comes from the database's encryption_info
field.
When a database's key version changes, the change is not immediately propagated
to encryption_info
. There might be a delay before the change is reflected in
the info field.
Console
Encryption information is displayed on the Database details page.
gcloud
Get a database's encryption_info
by calling
databases describe
or
databases list
. For example:
gcloud spanner databases describe example_db \
--project=my_spanner_project \
--instance=my_spanner_instance
Here's the output:
name: projects/my-project/instances/test-instance/databases/example-db
encryptionConfig:
kmsKeyName: projects/google.com:cloud-spanner-demo/locations/us-central1/keyRings/cmek_demo/cryptoKeys/backup-key
encryptionInfo:
encryptionType: CUSTOMER_MANAGED_ENCRYPTION
Disable the key
Disable the key version(s) that are in use by following these instructions for each key version.
Wait for the change to take effect. Disabling a key can take up to 3 hours to propagate.
Confirm that data is no longer accessible:
gcloud spanner databases execute-sql example_db \ --project=my_spanner_project \ --instance=my_spanner_instance \ --sql='SELECT * FROM Users'
You will see the following error:
KMS key required by the Spanner resource is not accessible.
Enable the key
Enable the key versions that are in use by the database by following these instructions for each key version.
Wait for the change to take effect. Enabling a key can take up to 3 hours to propagate.
Confirm that data is accessible:
gcloud spanner databases execute-sql example_db \ --project=my_spanner_project \ --instance=my_spanner_instance \ --sql='SELECT * FROM Users'
If the change has taken effect, the command will execute successfully.
Back up a database
By default, backups created from a database use the same encryption config as the database itself. You can optionally specify a different encryption config for a backup.
To create a backup:
Console
Go to the Database details page in the Google Cloud console.
In the Backup/Restore tab, click Create.
Enter a backup name and select an expiration date.
Select Use a customer-managed encryption key (CMEK) and choose a key from the drop-down list.
Click Create.
The Backups table displays encryption information for each backup.
gcloud
gcloud spanner backups create my_backup \
--project=my_spanner_project \
--instance=my_spanner_instance \
--database=example_db \
--retention-period=1y \
--encryption-type=customer_managed_encryption \
--kms-project=my_kms_project \
--kms-location=my_kms_key_location \
--kms-keyring=my_kms_key_ring \
--kms-key=my_kms_key
--async
To verify that the backup created is CMEK encrypted:
gcloud spanner backups describe my_backup \
--project=my_spanner_project \
--instance=my_spanner_instance
C#
using Google.Cloud.Spanner.Admin.Database.V1;
using Google.Cloud.Spanner.Common.V1;
using Google.Protobuf.WellKnownTypes;
using System;
using System.Threading.Tasks;
public class CreateBackupWithEncryptionKeyAsyncSample
{
public async Task<Backup> CreateBackupWithEncryptionKeyAsync(string projectId, string instanceId, string databaseId, string backupId, CryptoKeyName kmsKeyName)
{
// Create a DatabaseAdminClient instance.
DatabaseAdminClient databaseAdminClient = DatabaseAdminClient.Create();
// Create the CreateBackupRequest with encryption configuration.
CreateBackupRequest request = new CreateBackupRequest
{
ParentAsInstanceName = InstanceName.FromProjectInstance(projectId, instanceId),
BackupId = backupId,
Backup = new Backup
{
DatabaseAsDatabaseName = DatabaseName.FromProjectInstanceDatabase(projectId, instanceId, databaseId),
ExpireTime = DateTime.UtcNow.AddDays(14).ToTimestamp(),
},
EncryptionConfig = new CreateBackupEncryptionConfig
{
EncryptionType = CreateBackupEncryptionConfig.Types.EncryptionType.CustomerManagedEncryption,
KmsKeyNameAsCryptoKeyName = kmsKeyName,
},
};
// Execute the CreateBackup request.
var operation = await databaseAdminClient.CreateBackupAsync(request);
Console.WriteLine("Waiting for the operation to finish.");
// Poll until the returned long-running operation is complete.
var completedResponse = await operation.PollUntilCompletedAsync();
if (completedResponse.IsFaulted)
{
Console.WriteLine($"Error while creating backup: {completedResponse.Exception}");
throw completedResponse.Exception;
}
var backup = completedResponse.Result;
Console.WriteLine($"Backup {backup.Name} of size {backup.SizeBytes} bytes " +
$"was created at {backup.CreateTime} " +
$"using encryption key {kmsKeyName}");
return backup;
}
}
C++
Go
Java
Node.js
PHP
Python
Ruby
# project_id = "Your Google Cloud project ID"
# instance_id = "Your Spanner instance ID"
# database_id = "Your Spanner database ID"
# backup_id = "Your Spanner backup ID"
# kms_key_name = "Your backup encryption database KMS key"
require "google/cloud/spanner"
spanner = Google::Cloud::Spanner.new project: project_id
client = spanner.client instance_id, database_id
instance = spanner.instance instance_id
database = instance.database database_id
expire_time = Time.now + 14 * 24 * 3600 # 14 days from now
encryption_config = {
encryption_type: :CUSTOMER_MANAGED_ENCRYPTION,
kms_key_name: kms_key_name
}
job = database.create_backup backup_id, expire_time, version_time: version_time, encryption_config: encryption_config
puts "Backup operation in progress"
job.wait_until_done!
backup = instance.backup backup_id
puts "Backup #{backup.backup_id} of size #{backup.size_in_bytes} bytes was created at #{backup.create_time} using encryption key #{kms_key_name}"
Restore from a backup
By default, databases that are restored from a backup use the same encryption config as the backup itself, but you can override this behavior by specifying a different encryption config for the restored database. If the backup is protected by CMEK, the key version that was used to create the backup must be available so that it can be decrypted.
To restore a database:
Console
Go to the Instance details page in the Cloud console.
In the Backup/Restore tab, select a backup and click Restore.
Select the instance to restore and name the restored database.
If you wish to use CMEK with the restored database, select Use a customer-managed encryption key (CMEK) and select a key from the drop-down list.
gcloud
gcloud spanner databases restore --async \
--project=my_spanner_project \
--destination-instance=destination_instance \
--destination-database=example_db_restored \
--source-instance=my_spanner_instance \
--source-backup=my_backup
Verify that the restored database is CMEK encrypted:
gcloud spanner databases describe example_db_restored \
--project=my_spanner_project \
--instance=destination_instance
For more information, see Restoring a database from a backup.
C#
using Google.Cloud.Spanner.Admin.Database.V1;
using Google.Cloud.Spanner.Common.V1;
using System;
using System.Threading.Tasks;
public class RestoreDatabaseWithEncryptionAsyncSample
{
public async Task<Database> RestoreDatabaseWithEncryptionAsync(string projectId, string instanceId, string databaseId, string backupId, CryptoKeyName kmsKeyName)
{
// Create a DatabaseAdminClient instance.
DatabaseAdminClient databaseAdminClient = DatabaseAdminClient.Create();
// Create the RestoreDatabaseRequest with encryption configuration.
RestoreDatabaseRequest request = new RestoreDatabaseRequest
{
ParentAsInstanceName = InstanceName.FromProjectInstance(projectId, instanceId),
DatabaseId = databaseId,
BackupAsBackupName = BackupName.FromProjectInstanceBackup(projectId, instanceId, backupId),
EncryptionConfig = new RestoreDatabaseEncryptionConfig
{
EncryptionType = RestoreDatabaseEncryptionConfig.Types.EncryptionType.CustomerManagedEncryption,
KmsKeyNameAsCryptoKeyName = kmsKeyName,
}
};
// Execute the RestoreDatabase request.
var operation = await databaseAdminClient.RestoreDatabaseAsync(request);
Console.WriteLine("Waiting for the operation to finish.");
// Poll until the returned long-running operation is complete.
var completedResponse = await operation.PollUntilCompletedAsync();
if (completedResponse.IsFaulted)
{
Console.WriteLine($"Error while restoring database: {completedResponse.Exception}");
throw completedResponse.Exception;
}
var database = completedResponse.Result;
var restoreInfo = database.RestoreInfo;
Console.WriteLine($"Database {restoreInfo.BackupInfo.SourceDatabase} " +
$"restored to {database.Name} " +
$"from backup {restoreInfo.BackupInfo.Backup} " +
$"using encryption key {database.EncryptionConfig.KmsKeyName}");
return database;
}
}
C++
Go
Java
Node.js
PHP
Python
Ruby
# project_id = "Your Google Cloud project ID"
# instance_id = "Your Spanner instance ID"
# database_id = "Your Spanner database ID of where to restore"
# backup_id = "Your Spanner backup ID"
# kms_key_name = "Your backup encryption database KMS key"
require "google/cloud/spanner"
spanner = Google::Cloud::Spanner.new project: project_id
instance = spanner.instance instance_id
backup = instance.backup backup_id
encryption_config = {
encryption_type: :CUSTOMER_MANAGED_ENCRYPTION,
kms_key_name: kms_key_name
}
job = backup.restore database_id, encryption_config: encryption_config
puts "Waiting for restore backup operation to complete"
job.wait_until_done!
database = job.database
restore_info = database.restore_info
puts "Database #{restore_info.backup_info.source_database_id} was restored to #{database.database_id} from backup #{restore_info.backup_info.backup_id} using encryption key #{database.encryption_config.kms_key_name}"
View audit logs for the Cloud KMS key
Make sure that logging is enabled for the Cloud KMS API in your project.
Go to the Logs Explorer in the Cloud console.
Limit the log entries to your Cloud KMS key by adding the following lines to the Query builder:
resource.type="cloudkms_cryptokey" resource.labels.location="my_kms_key_location" resource.labels.key_ring_id="my_kms_key_ring" resource.labels.crypto_key_id="my_kms_key"
Under normal operations, encrypt and decrypt actions are logged with
INFO
severity. These entries are logged as the zones in your Spanner instance poll the Cloud KMS key about every 5 minutes.If Spanner fails to access the key, the operations are logged as
ERROR
.