Introducing delayed destruction for Secret Manager, a new way to protect your secrets

Secret Manager is a fully-managed, scalable service for storing, operating, auditing and accessing secrets used across Google Cloud services including GKE and Compute Engine. A critical part of any secrets management strategy is managing deletion and destruction of secrets. To provide customers with advanced capabilities in this area, we are pleased to announce the general availability of delayed destruction of secret versions for Secret Manager. This new capability helps to ensure that secret material cannot be erroneously deleted — either by accident or as part of an intended malicious attack. 

How It Works

There are two main resources within Secret Manager — secrets and secret versions. A secret is a project-global object containing a collection of metadata and secret versions. The secret version is where the actual secret data, such as credentials, API keys or certificates is stored. 

While managing the secret material lifecycle in Secret Manager, customers have faced some challenges:

• Destruction of a secret version is an irreversible step. This means there is no way to recover your secret material if it is destroyed.

• Lack of actionable alerting if there is an attempt to destroy any of your critical secrets, which reduces the chance of timely intervention from an administrator.

To address these challenges, we have introduced two capabilities as part of the delayed destruction of secrets: a customizable delay duration to prevent immediate destruction of secret versions, as well as a new Pub/Sub event notification that alerts you when a destroy action is attempted.

Before the introduction of delayed destruction, a secret version could move between enabled, disabled, and destroyed phases. Disabled is a reversible state which prevents access to the secret version. By default, the destroyed state is an immediate and irreversible state.

Delayed destruction introduces a new flow that disables immediate destruction of secret versions. This provides a fallback option in case of an unexpected incident and additional peace of mind that data is protected from threats.

Now, when a destroy action is taken , a soft-delete will be performed on the particular version by moving the status of the version to disabled. The secrets remain in an archival period for N days. This period can be configured by administrators using the TTL_DURATION field. During this archival period, an administrator can choose to revive the secret version by re-enabling it and moving to an enabled state. After the delay period expires, the secret version is permanently destroyed. 

Observability and alerting via Pub/Sub notification 

Observability is a key element of a well-defined security posture. To inform organizational admins and SecOps specialists about any attempts to destroy a secret version, we have added a new optional Pub/Sub notification named SECRET_VERSION_DESTROY_SCHEDULED.Once enabled, any scheduled destruction will notify the appropriate Pub/Sub topic, allowing the on-call personnel to analyze the change and if necessary restore the secret version instead of allowing destruction to proceed.    

Complement delayed destruction with least privilege access

Enabling the delayed destruction feature alone will not provide you complete defense against destruction of secret material. It is important that you set appropriate access controls on your secrets. Even with the feature enabled, an administrator of the secret can still choose to delete the complete secret if they wish. Therefore it is imperative that admin access to secrets is only granted to a tightly restricted, highly trusted group of users. For regular lifecycle management operations on secrets, users should only be provided with restricted roles, which grant them the least privileges required to complete the task:

• Secret Manager Admin: Granted to admins who have full control over secret, versions and its lifecycle. Typically this role would be restricted to a security admin group and not broadly distributed.

• Secret Version Manager / Secret Version Adder: Granted to devops engineers or service accounts performing lifecycle management on secret versions (add, enable, disable, destroy) or rotation workflows. These roles do not allow any modification of secret properties.

Addressing customer requirements 

Feedback from customers who have collaborated with us on the development of delayed destruction has helped strengthen resilience against cyber attacks and streamline backups.

“The ‘delay destruction of secret versions’ capability in Secret Manager helps us defend against disruptive attacks like ransomware or accidental deletion of important secrets in a simple and effective manner. As a financial services company, the security and resilience of our IT platforms is at the heart of what we do,” said Dominik Raub, chief information security officer, Crypto Finance AG, a Deutsche Börse Group company.

“We store credentials and key material in Secret Manager to make it available to applications in a secure manner. Since these secrets are critical to the operation of our applications and infrastructure, they need to be protected against malicious or accidental deletion,” he said. “In the past this was a real challenge on all cloud platforms we are working with. We approached our partners at Google about this issue and were very pleased with their collaborative mindset and the speed at which they delivered a solution. Now we can protect our secrets against deletion with just a few clicks instead of implementing complex backup procedures. And all that without compromising the protection afforded by the Secret Manager.”

Get started with Secret Manager today

Delayed destruction of secret versions in Secret Manager is now generally available with Google Cloud console, API, gcloud and client library support. To learn more about delayed destruction capabilities, please visit the Secret Manager documentation.