- notification - Configure object change notification
notification - Configure object change notification
gsutil notification create -f (json|none) [-p prefix] [-t topic] \ [-m key:value]... [-e eventType]... bucket_url gsutil notification delete (notificationConfigName|bucket_url)... gsutil notification list bucket_url... gsutil notification watchbucket [-i id] [-t token] app_url bucket_url gsutil notification stopchannel channel_id resource_id
The notification command is used to configure Google Cloud Storage support for sending notifications to Cloud Pub/Sub as well as to configure the object change notification feature.
The "create", "list", and "delete" sub-commands deal with configuring Cloud Storage integration with Google Cloud Pub/Sub.
The create sub-command creates a notification config on a bucket, establishing a flow of event notifications from Cloud Storage to a Cloud Pub/Sub topic. As part of creating this flow, the create command also verifies that the destination Cloud Pub/Sub topic exists, creating it if necessary, and verifies that the Cloud Storage bucket has permission to publish events to that topic, granting the permission if necessary.
If a destination Cloud Pub/Sub topic is not specified with the -t flag, Cloud Storage will by default choose a topic name in the default project whose ID is the same the bucket name. For example, if the default project ID specified is 'default-project' and the bucket being configured is gs://example-bucket, the create command will use the Cloud Pub/Sub topic "projects/default-project/topics/example-bucket".
In order to enable notifications, a special Cloud Storage service account unique to each project must have the IAM permission "projects.topics.publish". This command will check to see if that permission exists and, if not, will attempt to grant it.
You can create multiple notification configurations for a bucket, but their triggers cannot overlap such that a single event could send multiple notifications. Attempting to create a notification configuration that overlaps with an existing notification configuration results in an error.
Begin sending notifications of all changes to the bucket example-bucket to the Cloud Pub/Sub topic projects/default-project/topics/example-bucket:
gsutil notification create -f json gs://example-bucket
The same as above, but specifies the destination topic ID 'files-to-process' in the default project:
gsutil notification create -f json \ -t files-to-process gs://example-bucket
The same as above, but specifies a Cloud Pub/Sub topic belonging to the specific cloud project 'example-project':
gsutil notification create -f json \ -t projects/example-project/topics/files-to-process gs://example-bucket
Create a notification config that will only send an event when a new object has been created:
gsutil notification create -f json -e OBJECT_FINALIZE gs://example-bucket
Create a topic and notification config that will only send an event when an object beginning with "photos/" is affected:
gsutil notification create -p photos/ gs://example-bucket
List all of the notificationConfigs in bucket example-bucket:
gsutil notification list gs://example-bucket
Delete all notitificationConfigs for bucket example-bucket:
gsutil notification delete gs://example-bucket
Delete one specific notificationConfig for bucket example-bucket:
gsutil notification delete \ projects/_/buckets/example-bucket/notificationConfigs/1
The create sub-command has the following options
Specify an event type filter for this notification config. Cloud Storage will only send notifications of this type. You may specify this parameter multiple times to allow multiple event types. If not specified, Cloud Storage will send notifications for all event types. The valid types are:
OBJECT_FINALIZE - An object has been created. OBJECT_METADATA_UPDATE - The metadata of an object has changed. OBJECT_DELETE - An object has been permanently deleted. OBJECT_ARCHIVE - A live Cloud Storage object has been archived.
|-f||Specifies the payload format of notification messages. Must be either "json" for a payload matches the object metadata for the JSON API, or "none" to specify no payload at all. In either case, notification details are available in the message attributes.|
|-m||Specifies a key:value attribute that will be appended to the set of attributes sent to Cloud Pub/Sub for all events associated with this notification config. You may specify this parameter multiple times to set multiple attributes.|
|-p||Specifies a prefix path filter for this notification config. Cloud Storage will only send notifications for objects in this bucket whose names begin with the specified prefix.|
|-s||Skips creation and permission assignment of the Cloud Pub/Sub topic. This is useful if the caller does not have permission to access the topic in question, or if the topic already exists and has the appropriate publish permission assigned.|
|-t||The Cloud Pub/Sub topic to which notifications should be sent. If not specified, this command will choose a topic whose project is your default project and whose ID is the same as the Cloud Storage bucket name.|
Once the create command has succeeded, Cloud Storage will publish a message to the specified Cloud Pub/Sub topic when eligible changes occur. In order to receive these messages, you must create a Pub/Sub subscription for your Pub/Sub topic. To learn more about creating Pub/Sub subscriptions, see the Pub/Sub Subscriber Overview.
You can create a simple Pub/Sub subscription using the
tool. For example, to create a new subscription on the topic "myNewTopic" and
attempt to pull messages from it, you could run:
gcloud beta pubsub subscriptions create --topic myNewTopic testSubscription gcloud beta pubsub subscriptions pull --auto-ack testSubscription
The list sub-command provides a list of notification configs belonging to a given bucket. The listed name of each notification config can be used with the delete sub-command to delete that specific notification config.
For listing Object Change Notifications instead of Cloud Pub/Sub notification subscription configs, add a -o flag.
Fetch the list of notification configs for the bucket example-bucket:
gsutil notification list gs://example-bucket
The same as above, but for Object Change Notifications instead of Cloud Pub/Sub notification subscription configs.
gsutil notification list -o gs://example-bucket
Fetch the notification configs in all buckets matching a wildcard:
gsutil notification list gs://example-*
Fetch all of the notification configs for buckets in the default project:
gsutil notification list gs://*
The delete sub-command deletes notification configs from a bucket. If a notification config name is passed as a parameter, that notification config alone will be deleted. If a bucket name is passed, all notification configs associated with that bucket will be deleted.
Cloud Pub/Sub topics associated with this notification config will not be deleted by this command. Those must be deleted separately, for example with the gcloud command gcloud beta pubsub topics delete.
Object Change Notification subscriptions cannot be deleted with this command. For that, see the command gsutil notification stopchannel.
Delete a single notification config (with ID 3) in the bucket example-bucket:
gsutil notification delete projects/_/buckets/example-bucket/notificationConfigs/3
Delete all notification configs in the bucket example-bucket:
gsutil notification delete gs://example-bucket
Object Change Notifications
For more information on the Object Change Notification feature, please see the Object Change Notification docs.
The "watchbucket" and "stopchannel" sub-commands enable and disable Object Change Notifications.
The watchbucket sub-command can be used to watch a bucket for object changes. A service account must be used when running this command.
The app_url parameter must be an HTTPS URL to an application that will be notified of changes to any object in the bucket. The URL endpoint must be a verified domain on your project. See Notification Authorization for details.
The optional id parameter can be used to assign a unique identifier to the created notification channel. If not provided, a random UUID string will be generated.
The optional token parameter can be used to validate notifications events. To do this, set this custom token and store it to later verify that notification events contain the client token you expect.
Watch the bucket example-bucket for changes and send notifications to an application server running at example.com:
gsutil notification watchbucket https://example.com/notify \ gs://example-bucket
Assign identifier my-channel-id to the created notification channel:
gsutil notification watchbucket -i my-channel-id \ https://example.com/notify gs://example-bucket
Set a custom client token that will be included with each notification event:
gsutil notification watchbucket -t my-client-token \ https://example.com/notify gs://example-bucket
The stopchannel sub-command can be used to stop sending change events to a notification channel.
The channel_id and resource_id parameters should match the values from the response of a bucket watch request.
Stop the notification event channel with channel identifier channel1 and resource identifier SoGqan08XDIFWr1Fv_nGpRJBHh8:
gsutil notification stopchannel channel1 SoGqan08XDIFWr1Fv_nGpRJBHh8
Notifications And Parallel Composite Uploads
By default, gsutil enables parallel composite uploads for large files (see gsutil help cp), which means that an upload of a large object can result in multiple temporary component objects being uploaded before the actual intended object is created. Any subscriber to notifications for this bucket will then see a notification for each of these components being created and deleted. If this is a concern for you, note that parallel composite uploads can be disabled by setting "parallel_composite_upload_threshold = 0" in your boto config file. Alternately, your subscriber code can filter out gsutil's parallel composite uploads by ignoring any notification about objects whose names contain (but do not start with) the following string: