This topic discusses support for event notifications in Secret Manager.
Overview
Event notifications sends information about changes to your secrets and secret versions to Pub/Sub. These notifications can be used to trigger arbitrary workflows, such as restarting an application when a new secret version is added, or notifying security engineers when a secret is deleted. For more information on how to use these notifications to trigger workflows, see the Pub/Sub documentation.
How event notifications work in Secret Manager
Secrets can be configured with a list of up to 10 Pub/Sub topics. Whenever an operation is performed that modifies the secret or one of its versions, Secret Manager will automatically publish a message to each of the Pub/Sub topics on that secret. Get, List, and Access calls do not result in message publications.
Pub/Sub messages have a set of "attribute" key-value pairs containing metadata about the event, as well as a "data" field containing a full JSON serialization of the Secret or SecretVersion resource which was created or modified. This JSON is a UTF-8 encoded string that represents the Secret or SecretVersion resource in exactly the form specified by the Secret Manager public API, encoded in JSON as specified in the proto3 JSON Mapping.
Event types
The following is a list of event types currently supported by Secret Manager:
Event type | Description |
---|---|
SECRET_CREATE |
Sent when a new secret is successfully created. |
SECRET_UPDATE |
Sent when a new secret is successfully updated. |
SECRET_DELETE |
Sent when a secret is deleted, either because of a user-initiated request or secret expiration. |
SECRET_VERSION_ADD |
Sent when a new secret version is successfully created. |
SECRET_VERSION_ENABLE |
Sent when a secret version is enabled. |
SECRET_VERSION_DISABLE |
Sent when a secret version is disabled. |
SECRET_VERSION_DESTROY |
Sent when a secret version is destroyed. |
SECRET_ROTATE |
Sent when it is time to rotate a secret. See Creating and managing rotation policies on secrets for more info. |
TOPIC_CONFIGURED |
This is a test message with no body or attributes other than the eventType: TOPIC_CONFIGURED attribute. Sent when a secret is created or updated with a list of Pub/Sub topics, but does not indicate that the create or update operation was successful. A SECRET_CREATE or SECRET_UPDATE message will be sent immediately afterwards if the operation was successful. You should not use this event for your workflows; instead use SECRET_CREATE or SECRET_UPDATE . Whenever topics are updated on a secret, a TOPIC_CONFIGURED test message is sent to all topics on the secret, including ones that were already present. |
Notification format
Notifications sent to the Pub/Sub topic consist of two parts:
- Attributes: A set of key:value pairs describing the event.
- Data: A string that contains the metadata of the changed object.
Attributes
Attributes are key:value pairs contained in notifications sent by
Secret Manager to your Pub/Sub topic. All notifications other than TOPIC_CONFIGURED
test messages always
contain the following set of key:value pairs, regardless of the notification's
data:
Attribute name | Example | Description |
---|---|---|
eventType | SECRET_CREATE |
The type of event that has just occurred. See Event types for a list of possible values. |
dataFormat | JSON_API_V1 |
The format of the object data. |
secretId | projects/p/secrets/my-secret |
The name of the secret on which the event occurred. |
timestamp | 2021-01-20T11:17:45.081104-08:00 |
The time the event occurred. |
In addition, notifications sometimes contain the following set of key:value pairs:
Attribute name | Example | Description |
---|---|---|
versionId | projects/p/secrets/my-secret/versions/456 |
The name of the secret version on which the event occurred. Only present on SECRET_VERSION_ADD , SECRET_VERSION_ENABLE , SECRET_VERSION_DISABLE , and SECRET_VERSION_DESTROY event notifications. |
deleteType | REQUESTED |
Whether the delete was requested by a user (REQUESTED ) or due to secret expiration (EXPIRATION ). Only present on SECRET_DELETE event notifications. |
Data
The data field is a UTF-8 string that contains the metadata of the changed object. Data is either a Secret or Secret Version.
For SECRET_DELETE
notifications, the metadata contained in the data field represents
the object metadata as it was before the delete. For all other notifications, the metadata included in the
data field represents the object metadata after the
change occurs.
Limitations
Event notifications is available only in the Secret Manager v1
API and gCloud.
Before you begin
You may choose to store all resources in the same project or to store secrets and Pub/Sub topics in separate projects. Complete the following prerequisites to set up Secret Manager and Pub/Sub:
Secret Manager:
- Create or use an existing project to hold your Secret Manager resources.
- If necessary, complete the steps in the Configuring Secret Manager section of the Secret Manager quickstart.
Pub/Sub:
- Create or use an existing project to hold your Pub/Sub resources.
- If necessary, enable the Pub/Sub API.
Set the following variables to the project IDs of your Secret Manager and Pub/Sub projects.
Command-Line
$ export SM_PROJECT_ID="..."
$ export PUBSUB_PROJECT_ID="..."
Authenticate to Google Cloud:
gcloud
$ gcloud auth login
Creating a service account
You need to create a service account for each project that requires
secrets with event notifications. Currently, you can only use gcloud
command-line tool
commands to create the type of service account you need for event notifications.
To create a service account with gcloud
command-line tool, run the following command:
gcloud
$ gcloud services identity create \
--service "secretmanager.googleapis.com" \
--project "${SM_PROJECT_ID}"
The previous command returns a service account name, using the following format:
service-[PROJECT_NUMBER]@gcp-sa-secretmanager.iam.gserviceaccount.com
You will grant this service account permission to publish on the Pub/Sub topics which will be configured on your secrets.
Save the service account name as an environment variable:
Command-Line
# This is from the output of the command above
$ export SM_SERVICE_ACCOUNT="service-...."
The environment variables for the Secret Manager project, Pub/Sub project, and Secret Manager service account must be set the entire time you are following this procedure.
Create Pub/Sub topics
Follow the Pub/Sub quickstart to create topics in your Pub/Sub project in the Cloud Console. Alternatively, you can create topics with gcloud
command-line tool as in this example.
gcloud
$ export TOPIC_NAME="projects/${PUBSUB_PROJECT_ID}/topics/my-topic"
$ gcloud pubsub topics create ${TOPIC_NAME}
Repeat this multiple times if you wish to create multiple Pub/Sub topics on the secret.
Grant the service account for Secret Manager permission to publish on the topics just created. This can be done through the Cloud Console or with gcloud
command-line tool. The following command grants the Pub/Sub Publisher role (roles/pubsub.publisher
) on the my-topic
Pub/Sub topic to the service account.
gcloud
$ gcloud pubsub topics add-iam-policy-binding ${TOPIC_NAME} \
--member "serviceAccount:${SM_SERVICE_ACCOUNT}" \
--role "roles/pubsub.publisher"
Create Pub/Sub subscriptions
In order to view the messages published to a topic, you must also create a subscription to the topic. Follow the Pub/Sub quickstart to create subscriptions in your Pub/Sub project in the Cloud Console. Alternatively, you can create subscriptions with gcloud
command-line tool as in this example.
gcloud
$ export SUBSCRIPTION_NAME="projects/${PUBSUB_PROJECT_ID}/subscriptions/my-subscription"
$ gcloud pubsub subscriptions create ${SUBSCRIPTION_NAME} --topic ${TOPIC_NAME}
Create a secret with topics configured
Create a secret with a list of up to 10 topics configured. All topics configured on a secret will receive event notifications when the secret or one of its versions is changed. The following command creates a secret with my-topic
configured.
gcloud
$ gcloud secrets create my-secret \
--project "${SM_PROJECT_ID}" \
--topics ${TOPIC_NAME}
API
$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets?secretId=my-secret" \
--request "POST" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $(gcloud auth print-access-token)" \
--data-binary @- <<EOF
{
"replication":{
"automatic":{}
},
"topics":{
"name": "${TOPIC_NAME}"
}
}
EOF
Updating secret topics
Modify the Pub/Sub topics configured on a secret by updating the secret with the new Pub/Sub topic resource names. With gcloud
command-line tool you can add or remove one or more topics from a secret, as well as clear all topics from the secret.
Add topics
Adds one or more topics to a secret. Adding a topic which is already present will have no effect.
gcloud
$ gcloud secrets update my-secret \
--project "${SM_PROJECT_ID}" \
--add-topics "projects/${PUBSUB_PROJECT_ID}/topics/my-topic-2,projects/${PUBSUB_PROJECT_ID}/topics/my-topic-3"
Remove topics
Removes one or more topics from a secret. Removing a topic which is not present will have no effect.
gcloud
$ gcloud secrets update my-secret \
--project "${SM_PROJECT_ID}" \
--remove-topics "projects/${PUBSUB_PROJECT_ID}/topics/my-topic-2,projects/${PUBSUB_PROJECT_ID}/topics/my-topic-3"
Clear topics
Remove all topics from a secret.
gcloud
$ gcloud secrets update my-secret \
--project "${SM_PROJECT_ID}" \
--clear-topics
When updating topics with the API, you must specify the full set of topics for the secret with each PATCH
request (you cannot specify particular topics to add or remove, but must provide the full list each time).
API
$ curl "https://secretmanager.googleapis.com/v1/projects/${SM_PROJECT_ID}/secrets/my-secret?updateMask=topics" \
--request "PATCH" \
--header "Authorization: Bearer $(gcloud auth print-access-token)" \
--header "Content-Type: application/json" \
--data-binary @- <<EOF
{
"topics":[
{
"name": "${TOPIC_NAME}"
},
{
"name": "projects/${PUBSUB_PROJECT_ID}/topics/my-topic-2"
},
{
"name": "projects/${PUBSUB_PROJECT_ID}/topics/my-topic-3"
}
]
}
EOF
Consume event notifications with Cloud Functions
Event notifications can be used to trigger arbitrary workflows by creating cloud functions to consume the Pub/Sub messages. See the Cloud Functions documentation for a full guide. The sample code below is for a cloud function that prints secret metadata whenever an UPDATE_SECRET event is published to the topic.
Node.js
/**
* Triggered from a message on a Cloud Pub/Sub topic.
* The printed value will be visible in Cloud Logging
* (https://cloud.google.com/functions/docs/monitoring/logging).
*
* @param {!Object} event Event payload.
* @param {!Object} context Metadata for the event.
*/
exports.smEventsFunction = (event, context) => {
event_type = event.attributes.eventType;
if (event_type == 'SECRET_UPDATE') {
secret_id = event.attributes.secretId;
const secret_metadata = Buffer.from(event.data, 'base64').toString();
console.log(`Secret ${secret_id} was updated. Its new metadata is: ${secret_metadata}`);
}
};
Python
import base64
def sm_events_function(event, unused_context):
"""Triggered from a message on a Cloud Pub/Sub topic.
The printed value will be visible in Cloud Logging
(https://cloud.google.com/functions/docs/monitoring/logging).
Args:
event (dict): Event payload.
unused_context (google.cloud.functions.Context): Metadata for the event.
"""
event_type = event['attributes']['eventType']
if event_type == 'SECRET_UPDATE':
secret_id = event['attributes']['secretId']
secret_metadata = base64.b64decode(event['data']).decode('utf-8')
print('Secret {} was updated. Its new metadata is: {}'.format(secret_id, secret_metadata))
Go
package p
import (
"context"
"log"
)
type attributes struct {
SecretId string `json:"secretId"`
EventType string `json:"eventType"`
}
// PubSubMessage is the payload of a Cloud Pub/Sub event.
type PubSubMessage struct {
Attributes attributes `json:"attributes"`
Data []byte `json:"data"`
}
// Triggered from a message on a Cloud Pub/Sub topic.
// The printed value will be visible in Cloud Logging
// (https://cloud.google.com/functions/docs/monitoring/logging).
func SmEventsFunction(ctx context.Context, m PubSubMessage) error {
if m.Attributes.EventType == "SECRET_UPDATE" {
log.Printf("Secret %s was updated. Its new metadata is: %s", m.Attributes.SecretId, string(m.Data))
}
return nil
}
Java
package com.example;
import com.example.Example.PubSubMessage;
import com.google.cloud.functions.BackgroundFunction;
import com.google.cloud.functions.Context;
import java.util.Base64;
import java.util.Map;
import java.util.logging.Logger;
// Triggered from a message on a Cloud Pub/Sub topic.
// The printed value will be visible in Cloud Logging
// (https://cloud.google.com/functions/docs/monitoring/logging).
public class Example implements BackgroundFunction<PubSubMessage> {
private static final Logger logger = Logger.getLogger(Example.class.getName());
@Override
public void accept(PubSubMessage message, Context context) {
if (message.attributes.get("eventType").equals("SECRET_UPDATE")) {
String secretId = message.attributes.get("secretId");
String data = new String(Base64.getDecoder().decode(message.data));
logger.info(String.format("Secret %s was updated. Its new metadata is: %s", secretId, data));
}
}
public static class PubSubMessage {
String data;
Map<String, String> attributes;
String messageId;
String publishTime;
}
}
C#
using CloudNative.CloudEvents;
using Google.Cloud.Functions.Framework;
using Google.Events.Protobuf.Cloud.PubSub.V1;
using System;
using System.Threading;
using System.Threading.Tasks;
// Triggered from a message on a Cloud Pub/Sub topic.
// The printed value will be visible in Cloud Logging
// (https://cloud.google.com/functions/docs/monitoring/logging).
namespace PubSubSample
{
public class Function : ICloudEventFunction<MessagePublishedData>
{
public Task HandleAsync(CloudEvent cloudEvent, MessagePublishedData data, CancellationToken cancellationToken)
{
if (data.Message.Attributes["eventType"] == "SECRET_UPDATE") {
string secretId = data.Message.Attributes["secretId"];
string secretMetadata = data.Message.TextData;
Console.WriteLine($"Secret {secretId} was updated. Its new metadata is: {secretMetadata}");
}
return Task.CompletedTask;
}
}
}
Ruby
require "functions_framework"
require "base64"
# Triggered from a message on a Cloud Pub/Sub topic.
# The printed value will be visible in Cloud Logging
# (https://cloud.google.com/functions/docs/monitoring/logging).
FunctionsFramework.cloud_event "sm_events_function" do |event|
message = event.data["message"]
if message["attributes"]["eventType"] == "SECRET_UPDATE"
secret_id = message["attributes"]["secretId"]
message_data = Base64.decode64 message["data"]
FunctionsFramework.logger.info "Secret %s was updated. Its new metadata is: %s" % [secret_id, message_data]
end
end
What's next?
- Learn more about managing secrets and secret versions.
- Learn more about managing access to secrets.