In some large-scale storage systems, like at Google, a key hierarchy with multiple layers is used to encrypt data.
The key used to encrypt data itself is called a data encryption key (DEK). For easy access, the DEK is stored near the data that it encrypts. The DEK is encrypted (or wrapped) by a key encryption key (KEK). Having a smaller number of KEKs than DEKs and using a central Key Management Service makes storing and encrypting data at scale more manageable, and means the central keystore is a singular point to more easily audit and restrict data access. This process is known as envelope encryption.
This is treated very briefly in an (otherwise very detailed) NIST document section 220.127.116.11.
Depending on your situation, and the volume of data you are encrypting, you may
choose to use a similar model. Cloud KMS was designed to manage KEKs, and thus
the maximum data input size for
Decrypt functions is 64 KiB.
However, for data that you know will not approach that limit, you could use
Cloud KMS to encrypt and decrypt data directly.