Cloud Pub/Sub Tutorial

This simple tutorial demonstrates writing, deploying, and triggering an Event-Driven Cloud Function with a Cloud Pub/Sub trigger.

If you are new to Pub/Sub and want to learn more, see the Pub/Sub documentation, particularly managing topics and subscriptions. See Google Cloud Pub/Sub Triggers for an overview of working with Pub/Sub topics and subscriptions in Cloud Functions.

If you're looking for code samples for using Pub/Sub itself, please visit the Google Cloud sample browser.

Note: You can also use HTTP-triggered functions to listen to Pub/Sub push subscriptions. Using an HTTP function enables a single function to subscribe to multiple Pub/Sub topics.

Objectives

• Write and deploy an Event-Driven Cloud Function.
• Trigger the function by publishing a message to a Pub/Sub topic.

Costs

This tutorial uses billable components of Cloud Platform, including:

• Cloud Functions
• Pub/Sub

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

4. Enable the Cloud Functions and Cloud Pub/Sub APIs.

   Enable the APIs

5. Install and initialize the Cloud SDK.

If you already have the Cloud SDK installed, update it by running the following command:

gcloud components update

6. Prepare your development environment.

   Node.js

   Go to the Node.js setup guide

   Python

   Go to the Python setup guide

   Go

   Go to the Go setup guide

   Java

   Go to the Java setup guide

   C#

   Go to the .NET setup guide

   Ruby

   Go to the Ruby setup guide

   PHP

   Go to Using PHP on Google Cloud

Preparing the application

1. Clone the sample app repository to your local machine:

   Node.js

   git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Python

   git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Go

   git clone https://github.com/GoogleCloudPlatform/golang-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Java

   git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   C#

   git clone https://github.com/GoogleCloudPlatform/dotnet-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Ruby

   git clone https://github.com/GoogleCloudPlatform/ruby-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   PHP

   git clone https://github.com/GoogleCloudPlatform/php-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

2. Change to the directory that contains the Cloud Functions sample code for accessing Pub/Sub:

   Node.js

   cd nodejs-docs-samples/functions/helloworld/

   Python

   cd python-docs-samples/functions/helloworld/

   Go

   cd golang-samples/functions/helloworld/

   Java

   cd java-docs-samples/functions/helloworld/hello-pubsub/

   C#

   cd dotnet-docs-samples/functions/helloworld/HelloPubSub/

   Ruby

   cd ruby-docs-samples/functions/helloworld/pubsub/

   PHP

   cd php-docs-samples/functions/helloworld_pubsub/

3. Take a look at the sample code:

   Node.js

   functions/helloworld/index.js

   View on GitHub

   /**
    * Background Cloud Function to be triggered by Pub/Sub.
    * This function is exported by index.js, and executed when
    * the trigger topic receives a message.
    *
    * @param {object} message The Pub/Sub message.
    * @param {object} context The event metadata.
    */
   exports.helloPubSub = (message, context) => {
     const name = message.data
       ? Buffer.from(message.data, 'base64').toString()
       : 'World';
   
     console.log(`Hello, ${name}!`);
   };

   Python

   functions/helloworld/main.py

   View on GitHub

   def hello_pubsub(event, context):
       """Background Cloud Function to be triggered by Pub/Sub.
       Args:
            event (dict):  The dictionary with data specific to this type of
                           event. The `@type` field maps to
                            `type.googleapis.com/google.pubsub.v1.PubsubMessage`.
                           The `data` field maps to the PubsubMessage data
                           in a base64-encoded string. The `attributes` field maps
                           to the PubsubMessage attributes if any is present.
            context (google.cloud.functions.Context): Metadata of triggering event
                           including `event_id` which maps to the PubsubMessage
                           messageId, `timestamp` which maps to the PubsubMessage
                           publishTime, `event_type` which maps to
                           `google.pubsub.topic.publish`, and `resource` which is
                           a dictionary that describes the service API endpoint
                           pubsub.googleapis.com, the triggering topic's name, and
                           the triggering event type
                           `type.googleapis.com/google.pubsub.v1.PubsubMessage`.
       Returns:
           None. The output is written to Cloud Logging.
       """
       import base64
   
       print("""This Function was triggered by messageId {} published at {} to {}
       """.format(context.event_id, context.timestamp, context.resource["name"]))
   
       if 'data' in event:
           name = base64.b64decode(event['data']).decode('utf-8')
       else:
           name = 'World'
       print('Hello {}!'.format(name))

   Go

   functions/helloworld/hello_pubsub.go

   View on GitHub

   
   // Package helloworld provides a set of Cloud Functions samples.
   package helloworld
   
   import (
   	"context"
   	"log"
   )
   
   // PubSubMessage is the payload of a Pub/Sub event.
   // See the documentation for more details:
   // https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
   type PubSubMessage struct {
   	Data []byte `json:"data"`
   }
   
   // HelloPubSub consumes a Pub/Sub message.
   func HelloPubSub(ctx context.Context, m PubSubMessage) error {
   	name := string(m.Data) // Automatically decoded from base64.
   	if name == "" {
   		name = "World"
   	}
   	log.Printf("Hello, %s!", name)
   	return nil
   }
   

   Java

   functions/helloworld/hello-pubsub/src/main/java/functions/HelloPubSub.java

   View on GitHub

   
   import com.google.cloud.functions.BackgroundFunction;
   import com.google.cloud.functions.Context;
   import com.google.events.cloud.pubsub.v1.Message;
   import java.nio.charset.StandardCharsets;
   import java.util.Base64;
   import java.util.logging.Level;
   import java.util.logging.Logger;
   
   public class HelloPubSub implements BackgroundFunction<Message> {
     private static final Logger logger = Logger.getLogger(HelloPubSub.class.getName());
   
     @Override
     public void accept(Message message, Context context) {
       String name = "world";
       if (message != null && message.getData() != null) {
         name = new String(
             Base64.getDecoder().decode(message.getData().getBytes(StandardCharsets.UTF_8)),
             StandardCharsets.UTF_8);
       }
       logger.info(String.format("Hello %s!", name));
       return;
     }
   }

   C#

   functions/helloworld/HelloPubSub/Function.cs

   View on GitHub

   using CloudNative.CloudEvents;
   using Google.Cloud.Functions.Framework;
   using Google.Events.Protobuf.Cloud.PubSub.V1;
   using Microsoft.Extensions.Logging;
   using System.Threading;
   using System.Threading.Tasks;
   
   namespace HelloPubSub
   {
       public class Function : ICloudEventFunction<MessagePublishedData>
       {
           private readonly ILogger _logger;
   
           public Function(ILogger<Function> logger) =>
               _logger = logger;
   
           public Task HandleAsync(CloudEvent cloudEvent, MessagePublishedData data, CancellationToken cancellationToken)
           {
               string nameFromMessage = data.Message?.TextData;
               string name = string.IsNullOrEmpty(nameFromMessage) ? "world" : nameFromMessage;
               _logger.LogInformation("Hello {name}", name);
               return Task.CompletedTask;
           }
       }
   }

   Ruby

   functions/helloworld/pubsub/app.rb

   View on GitHub

   require "functions_framework"
   require "base64"
   
   FunctionsFramework.cloud_event "hello_pubsub" do |event|
     # The event parameter is a CloudEvents::Event::V1 object.
     # See https://cloudevents.github.io/sdk-ruby/latest/CloudEvents/Event/V1.html
     name = Base64.decode64 event.data["message"]["data"] rescue "World"
   
     # A cloud_event function does not return a response, but you can log messages
     # or cause side effects such as sending additional events.
     logger.info "Hello, #{name}!"
   end

   PHP

   functions/helloworld_pubsub/index.php

   View on GitHub

   
   use Google\CloudFunctions\CloudEvent;
   
   function helloworldPubsub(CloudEvent $event): void
   {
       $log = fopen(getenv('LOGGER_OUTPUT') ?: 'php://stderr', 'wb');
   
       $cloudEventData = $event->getData();
       $pubSubData = base64_decode($cloudEventData['message']['data']);
   
       $name = $pubSubData ? htmlspecialchars($pubSubData) : 'World';
       fwrite($log, "Hello, $name!" . PHP_EOL);
   }

   Note: The Pub/Sub message's data is stored as a base64-encoded string in the data property of the PubsubMessage object. To extract this data, you may need to decode it as shown in the examples above. The context parameter is described here.

Deploying the function

To deploy the function with a Pub/Sub trigger, run the following command in the directory that contains the sample code (or in the case of Java, the pom.xml file):

Node.js

gcloud functions deploy helloPubSub \
--runtime nodejs14 \
--trigger-topic YOUR_TOPIC_NAME

You can use the following values for the --runtime flag to specify your preferred Node.js version:

• nodejs16 (preview)
• nodejs14 (recommended)
• nodejs12
• nodejs10

Python

gcloud functions deploy hello_pubsub \
--runtime python39 \
--trigger-topic YOUR_TOPIC_NAME

You can use the following values for the --runtime flag to specify your preferred Python version:

• python39 (recommended)
• python38
• python37

Go

gcloud functions deploy HelloPubSub \
--runtime go113 \
--trigger-topic YOUR_TOPIC_NAME

You can use the following values for the --runtime flag to specify your preferred Go version:

• go116 (preview)
• go113 (recommended)
• go111

Java

gcloud functions deploy java-pubsub-function \
--entry-point functions.HelloPubSub \
--runtime java11 \
--memory 512MB \
--trigger-topic YOUR_TOPIC_NAME

C#

gcloud functions deploy csharp-pubsub-function \
--entry-point HelloPubSub.Function \
--runtime dotnet3 \
--trigger-topic YOUR_TOPIC_NAME

Ruby

gcloud functions deploy hello_pubsub --runtime ruby27 \
--trigger-topic YOUR_TOPIC_NAME

You can use the following values for the --runtime flag to specify your preferred Ruby version:

• ruby27 (recommended)
• ruby26

PHP

gcloud functions deploy helloworldPubsub --runtime php74 \
--trigger-topic YOUR_TOPIC_NAME

where YOUR_TOPIC_NAME is the name of the Pub/Sub topic to which the function will be subscribed.

If YOUR_TOPIC_NAME doesn't already exist, this command creates it for you. You can also create a topic before you run the deploy command, using either the Cloud Console or the following gcloud command:

gcloud pubsub topics create YOUR_TOPIC_NAME

Note: Cloud Functions automatically creates push subscriptions that forward messages from the specified topic to your function.

Triggering the function

1. Publish a message to your Pub/Sub topic. In this example, the message is a name that the function will include in a greeting:

   gcloud pubsub topics publish YOUR_TOPIC_NAME --message YOUR_NAME

   Replace YOUR_TOPIC_NAME with the name of your Pub/Sub topic, and YOUR_NAME with an arbitrary string.

2. Check the logs to be sure the executions have completed:

   gcloud functions logs read --limit 50
   

   Note: Logs might take a few moments to appear. If you don't see them immediately, check again in a minute or two.

You can also publish a message to a Pub/Sub topic from within a function.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Cloud Console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting the Cloud Function

Deleting Cloud Functions does not remove any resources stored in Cloud Storage.

To delete the Cloud Function you created in this tutorial, run the following command:

Node.js

gcloud functions delete helloPubSub 

Python

gcloud functions delete hello_pubsub 

Go

gcloud functions delete HelloPubSub 

Java

gcloud functions delete java-pubsub-function 

C#

gcloud functions delete csharp-pubsub-function 

Ruby

gcloud functions delete hello_pubsub 

PHP

gcloud functions delete helloworldPubsub 

You can also delete Cloud Functions from the Google Cloud Console.