Cannot create table with customer-managed encryption

Problem

When trying to create Bigtable table using customer-managed encryption keys (CMEK) the operation fails due to the following error:

rpc error: code = Canceled desc = Operation successfully rolled back : Failed to create the replicated table. Request failed due to a problematic CMEK state

Environment

  • Cloud Bigtable
  • Service identity

Solution

You must follow the instructions described in Use customer-managed encryption keys to create a CMEK key for each region that the instance's clusters will be in:

  1. Create (or retrieve) a Bigtable service agent.
  2. Create a CMEK key.
  3. Configure IAM settings for that key.
  4. The following gcloud command is used to create Bigtable service agent which resulted in an error:
    $ gcloud beta services identity create --service=bigtable.googleapis.com --project xxxx
    
    ERROR: (gcloud.beta.services.identity.create) INVALID_ARGUMENT: Service bigtable.googleapis.com has not been configured for service identities.
  5. Create the service identity with the correct name as bigtableadmin.googleapis.com.

Cause

Using the wrong Bigtable service name bigtable.googleapis.com while creating Bigtable service agent causes Bigtable service agent creation to fail. Since the Bigtable service agent is not created successfully, it further leads to an error while creation of Bigtable table using customer-managed encryption keys (CMEK).
$ cbt -project ${PROJECT_ID} -instance ${BIGTABLE_INSTANCE_NAME} createtable ${BIGTABLE_TABLE_NAME}

Creating table: rpc error: code = Canceled desc = Operation successfully rolled back : Failed to create the replicated table. Request failed due to a problematic CMEK state