Google Cloud Storage Triggers

Cloud Functions can respond to change notifications emerging from Google Cloud Storage. These notifications can be configured to trigger in response to various events inside a bucket—object creation, deletion, archiving and metadata updates.

Event types

Cloud Storage events used by Cloud Functions are based on Cloud Pub/Sub Notifications for Google Cloud Storage and can be configured in a similar way.

Supported trigger type values are:

• google.storage.object.finalize

• google.storage.object.delete

• google.storage.object.archive

• google.storage.object.metadataUpdate

Note: Cloud Functions can only be triggered by Cloud Storage buckets in the same Google Cloud Platform project.

Object Finalize

Trigger type value: google.storage.object.finalize

This event is sent when a new object is created (or an existing object is overwritten, and a new generation of that object is created) in the bucket.

For example, the following function logs relevant data when an event occurs:

Node.js 8/10

functions/node8/index.js

View on GitHub Feedback

/**
 * Generic background Cloud Function to be triggered by Cloud Storage.
 *
 * @param {object} data The event payload.
 * @param {object} context The event metadata.
 */
exports.helloGCSGeneric = (data, context) => {
  const file = data;
  console.log(`  Event ${context.eventId}`);
  console.log(`  Event Type: ${context.eventType}`);
  console.log(`  Bucket: ${file.bucket}`);
  console.log(`  File: ${file.name}`);
  console.log(`  Metageneration: ${file.metageneration}`);
  console.log(`  Created: ${file.timeCreated}`);
  console.log(`  Updated: ${file.updated}`);
};

Node.js 6 (Deprecated)

functions/helloworld/index.js

View on GitHub Feedback

/**
 * Generic background Cloud Function to be triggered by Cloud Storage.
 *
 * @param {object} event The Cloud Functions event.
 * @param {function} callback The callback function.
 */
exports.helloGCSGeneric = (event, callback) => {
  const file = event.data;

  console.log(`  Event: ${event.eventId}`);
  console.log(`  Event Type: ${event.eventType}`);
  console.log(`  Bucket: ${file.bucket}`);
  console.log(`  File: ${file.name}`);
  console.log(`  Metageneration: ${file.metageneration}`);
  console.log(`  Created: ${file.timeCreated}`);
  console.log(`  Updated: ${file.updated}`);

  callback();
};

Python

functions/gcs/main.py

View on GitHub Feedback

def hello_gcs_generic(data, context):
    """Background Cloud Function to be triggered by Cloud Storage.
       This generic function logs relevant data when a file is changed.

    Args:
        data (dict): The Cloud Functions event payload.
        context (google.cloud.functions.Context): Metadata of triggering event.
    Returns:
        None; the output is written to Stackdriver Logging
    """

    print('Event ID: {}'.format(context.event_id))
    print('Event type: {}'.format(context.event_type))
    print('Bucket: {}'.format(data['bucket']))
    print('File: {}'.format(data['name']))
    print('Metageneration: {}'.format(data['metageneration']))
    print('Created: {}'.format(data['timeCreated']))
    print('Updated: {}'.format(data['updated']))

Go

functions/helloworld/storage_generic/hello_storage_generic.go

View on GitHub Feedback

// Package helloworld provides a set of Cloud Functions samples.
package helloworld

import (
	"context"
	"fmt"
	"log"
	"time"

	"cloud.google.com/go/functions/metadata"
)

// GCSEvent is the payload of a GCS event. Please refer to the docs for
// additional information regarding GCS events.
type GCSEvent struct {
	Bucket         string    `json:"bucket"`
	Name           string    `json:"name"`
	Metageneration string    `json:"metageneration"`
	ResourceState  string    `json:"resourceState"`
	TimeCreated    time.Time `json:"timeCreated"`
	Updated        time.Time `json:"updated"`
}

// HelloGCSInfo prints information about a GCS event.
func HelloGCSInfo(ctx context.Context, e GCSEvent) error {
	meta, err := metadata.FromContext(ctx)
	if err != nil {
		return fmt.Errorf("metadata.FromContext: %v", err)
	}
	log.Printf("Event ID: %v\n", meta.EventID)
	log.Printf("Event type: %v\n", meta.EventType)
	log.Printf("Bucket: %v\n", e.Bucket)
	log.Printf("File: %v\n", e.Name)
	log.Printf("Metageneration: %v\n", e.Metageneration)
	log.Printf("Created: %v\n", e.TimeCreated)
	log.Printf("Updated: %v\n", e.Updated)
	return nil
}

To deploy the function with an object finalize trigger, run the following command in the directory that contains the function code:

Node.js 8

gcloud functions deploy helloGCSGeneric --runtime nodejs8 --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event google.storage.object.finalize

Node.js 10 (Beta)

gcloud functions deploy helloGCSGeneric --runtime nodejs10 --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event google.storage.object.finalize

Node.js 6 (Deprecated)

gcloud functions deploy helloGCSGeneric --runtime nodejs6 --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event google.storage.object.finalize

Python

gcloud functions deploy hello_gcs_generic --runtime python37 --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event google.storage.object.finalize

Go

gcloud functions deploy HelloGCSInfo --runtime go111 --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event google.storage.object.finalize

where YOUR_TRIGGER_BUCKET_NAME is the name of the Cloud Storage Bucket that the function will monitor.

Object Delete

Trigger type value: google.storage.object.delete

This event is sent when an object is permanently deleted. Depending on the object versioning setting for a bucket this means:

• For versioning buckets, this is only sent when a version is permanently deleted (but not when an object is archived).

• For non-versioning buckets, this is sent when an object is deleted or overwritten.

Object Archive

Trigger type value: google.storage.object.archive

This event is sent when a live version of an object is archived or deleted.

This event is only sent for versioning buckets.

Object Metadata Update

Trigger type value: google.storage.object.metadataUpdate

This event is sent when the metadata of an existing object changes.

Event structure

Storage event data is delivered in the Cloud Storage object format.

Event delivery mechanism

The events are delivered with Pub/Sub notifications from Cloud Storage. Setting up too many functions against the same bucket may exhaust the notification limit for the bucket and make it impossible to create the function, as indicated by the error Cloud Storage bucket ...: Pub/Sub notification limit reached. See Google Cloud Storage Cloud Pub/Sub Notifications documentation for more information about notification limits.

Legacy Cloud Storage triggers

The gcloud command below deploys a function that is triggered by legacy object change notifications on a specific bucket. Generally, Cloud Pub/Sub Notifications are easier to use, more flexible, and more powerful than object change notifications. However, these legacy notifications are supported for legacy functions already consuming these events.

gcloud functions deploy YOUR_FUNCTION_NAME --trigger-resource YOUR_TRIGGER_BUCKET_NAME --trigger-event providers/cloud.storage/eventTypes/object.change

Argument	Description
--trigger-resource NAME	The name of the Cloud Storage bucket the function watches for changes.
--trigger-event NAME	The name of the event type that the function wishes to receive. In this case, it is the legacy object.change event.

Next steps

See the Cloud Storage Tutorial for an example of how to implement a background function that is triggered by Cloud Storage.