Cloud Storage Tutorial

This simple tutorial demonstrates writing, deploying, and triggering a Background Cloud Function with a Cloud Storage trigger.

Objectives

Costs

This tutorial uses billable components of Cloud Platform, including:

  • Google Cloud Functions
  • Google Cloud Storage

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a GCP project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your project.

    Learn how to enable billing

  4. Enable the Cloud Functions and Cloud Storage APIs.

    Enable the APIs

  5. Install and initialize the Cloud SDK.
  6. Update and install gcloud components:
    gcloud components update &&
    gcloud components install beta
  7. Prepare your environment for Node.js development.

    Go to the setup guide

Preparing the application

  1. Create a Cloud Storage bucket to upload a test file, where [YOUR_TRIGGER_BUCKET_NAME] is a globally unique bucket name:

    gsutil mb gs://[YOUR_TRIGGER_BUCKET_NAME]
    

  2. Create a directory on your local system for the application code:

    Linux or Mac OS X

    Create the directory:

    mkdir ~/gcf_gcs

    Move into the directory:

    cd ~/gcf_gcs

    Windows(CMD)

    Create the directory:

    mkdir %HOMEDRIVE%%HOMEPATH%\gcf_gcs

    Move into the directory:

    cd %HOMEDRIVE%%HOMEPATH%\gcf_gcs

Deploying and triggering the function

Currently, Cloud Storage functions are based on Pub/Sub notifications from Cloud Storage and support similar event types:

The following sections describe how to deploy and trigger a function for each of these event types.

Object Finalize

Object finalize events trigger when a "write" of a Cloud Storage Object is successfully finalized. In particular, this means that creating a new object or overwriting an existing object triggers this event. Archive and metadata update operations are ignored by this trigger.

Object Finalize: deploying the function

To deploy the function, create an index.js file in the gcf_gcs directory with the following contents:

Node.js

/**
 * Generic background Cloud Function to be triggered by Cloud Storage.
 *
 * @param {object} event The Cloud Functions event.
 * @param {function} callback The callback function.
 */
exports.helloGCSGeneric = (event, callback) => {
  const file = event.data;
  const context = event.context;

  console.log(`Event ${context.eventId}`);
  console.log(`  Event Type: ${context.eventType}`);
  console.log(`  Bucket: ${file.bucket}`);
  console.log(`  File: ${file.name}`);
  console.log(`  Metageneration: ${file.metageneration}`);
  console.log(`  Created: ${file.timeCreated}`);
  console.log(`  Updated: ${file.updated}`);

  callback();
};

The helloGCSGeneric function is exported by the module and is executed when the new object is finalized in a bucket.

Next, run the following command in the gcf_gcs directory:

gcloud beta functions deploy helloGCSGeneric --trigger-resource [YOUR_TRIGGER_BUCKET_NAME] --trigger-event google.storage.object.finalize

where [YOUR_TRIGGER_BUCKET_NAME] is the name of the Cloud Storage Bucket that triggers the function.

Object Finalize: triggering the function

To trigger the function:

  1. Create an empty gcf-test.txt file in the gcf_gcs directory.

  2. Upload the file to Cloud Storage in order to trigger the function:

    gsutil cp gcf-test.txt gs://[YOUR_TRIGGER_BUCKET_NAME]
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of your Cloud Storage bucket where you will upload a test file.

  3. Check the logs to make sure the executions have completed:

    gcloud beta functions logs read --limit 50
    

Object Delete

Object delete events are most useful for non-versioning buckets. They are triggered when an old version of an object is deleted. In addition, they are triggered when an object is overwritten. Object delete triggers can also be used with versioning buckets, triggering when a version of an object is permanently deleted.

Object Delete: deploying the function

To deploy the function:

  1. Create an index.js file in the gcf_gcs directory with the same helloGCSGeneric function as in the finalize example. This function will be executed when an object is deleted from the bucket.

  2. Run the following command in the gcf_gcs directory:

    gcloud beta functions deploy helloGCSGeneric --trigger-resource [YOUR_TRIGGER_BUCKET_NAME] --trigger-event google.storage.object.delete
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of the Cloud Storage Bucket that triggers the function.

Object Delete: triggering the function

To trigger the function:

  1. Create an empty file called gcf-test.txt in the gcf_gcs directory.

  2. Make sure that your bucket is non-versioning:

    gsutil versioning set off gs://[YOUR_TRIGGER_BUCKET_NAME]
    
  3. Upload the file to Cloud Storage in order to trigger the function:

    gsutil cp gcf-test.txt gs://[YOUR_TRIGGER_BUCKET_NAME]
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of your Cloud Storage bucket where you will upload a test file. At this point the function should not execute yet.

  4. Delete the file:

    gsutil rm gs://[YOUR_TRIGGER_BUCKET_NAME]/gcf-test.txt
    
  5. Check the logs to make sure the executions have completed:

    gcloud beta functions logs read --limit 50
    

Note that the function may take some time to finish executing.

Object Archive

Object archive events can be used only with versioning buckets. They are triggered when an old version of an object is archived. In particular, this means that when an object is overwritten or deleted, an archive event is triggered.

Object Archive: deploying the function

To deploy the function:

  1. Create an index.js file in the gcf_gcs directory with the same helloGCSGeneric function as in the finalize example. This function will be executed when an object is archived.

  2. Run the following command in the gcf_gcs directory:

    gcloud beta functions deploy helloGCSGeneric --trigger-resource [YOUR_TRIGGER_BUCKET_NAME] --trigger-event google.storage.object.archive
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of the Cloud Storage Bucket that triggers the function.

Object Archive: triggering the function

To trigger the function:

  1. Create an empty gcf-test.txt file in the gcf_gcs directory.

  2. Make sure that your bucket has versioning enabled:

    gsutil versioning set on gs://[YOUR_TRIGGER_BUCKET_NAME]
    
  3. Upload the file to Cloud Storage in order to trigger the function:

    gsutil cp gcf-test.txt gs://[YOUR_TRIGGER_BUCKET_NAME]
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of your Cloud Storage bucket where you will upload a test file. At this point the function should not execute yet.

  4. Archive the file:

    gsutil rm gs://[YOUR_TRIGGER_BUCKET_NAME]/gcf-test.txt
    
  5. Watch the logs to make sure the executions have completed:

    gcloud beta functions logs read --limit 50
    

Object Metadata Update

Metadata update events are triggered when the metadata of existing object is updated.

Object Metadata Update: deploying the function

To deploy the function:

  1. Create an index.js file in the gcf_gcs directory with the same helloGCSGeneric function as in the finalize example. This function will be executed when an object's metadata changes.

  2. Run the following command in the gcf_gcs directory:

    gcloud beta functions deploy helloGCSGeneric --trigger-resource [YOUR_TRIGGER_BUCKET_NAME] --trigger-event google.storage.object.metadataUpdate
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of the Cloud Storage Bucket that triggers the function.

Object Metadata Update: triggering the function

To trigger the function:

  1. Create an empty gcf-test.txt file in the gcf_gcs directory.

  2. Make sure that your bucket is non-versioning:

    gsutil versioning set off gs://[YOUR_TRIGGER_BUCKET_NAME]
    
  3. Upload the file to Cloud Storage in order to trigger the function:

    gsutil cp gcf-test.txt gs://[YOUR_TRIGGER_BUCKET_NAME]
    

    where [YOUR_TRIGGER_BUCKET_NAME] is the name of your Cloud Storage bucket where you will upload a test file. At this point the function should not execute yet.

  4. Update the metadata of the file:

    gsutil -m setmeta -h "Content-Type:text/plain" gs://[YOUR_TRIGGER_BUCKET_NAME]/gcf-test.txt
    
  5. Watch the logs to make sure the executions have completed:

    gcloud beta functions logs read --limit 50
    

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

To delete the project:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click
      Delete project
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting the Cloud Functions

Deleting Cloud Functions does not remove any resources stored in Cloud Storage.

To delete a Cloud Function, run the following command:

gcloud beta functions delete [NAME_OF_FUNCTION]

where [NAME_OF_FUNCTION] is the name of the function to delete.

You can also delete Cloud Functions from the Google Cloud Platform Console.

Send feedback about...

Cloud Functions Documentation