Cloud Pub/Sub Tutorial

This simple tutorial demonstrates writing, deploying, and triggering an Event-Driven Cloud Function with a Cloud Pub/Sub trigger.

If you are new to Pub/Sub and want to learn more, see the Pub/Sub documentation, particularly managing topics and subscriptions. See Google Cloud Pub/Sub Triggers for an overview of working with Pub/Sub topics and subscriptions in Cloud Functions.

If you're looking for code samples for using Pub/Sub itself, please visit the Google Cloud sample browser.

Objectives

Costs

This tutorial uses billable components of Cloud Platform, including:

  • Cloud Functions
  • Pub/Sub

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the Cloud Functions and Cloud Pub/Sub APIs.

    Enable the APIs

  5. Install and initialize the Cloud SDK.
  6. If you already have the Cloud SDK installed, update it by running the following command:

    gcloud components update
  7. Prepare your development environment.

Preparing the application

  1. Clone the sample app repository to your local machine:

    Node.js

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Python

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Go

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Java

    git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    C#

    git clone https://github.com/GoogleCloudPlatform/dotnet-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Ruby

    git clone https://github.com/GoogleCloudPlatform/ruby-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    PHP

    git clone https://github.com/GoogleCloudPlatform/php-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

  2. Change to the directory that contains the Cloud Functions sample code for accessing Pub/Sub:

    Node.js

    cd nodejs-docs-samples/functions/helloworld/

    Python

    cd python-docs-samples/functions/helloworld/

    Go

    cd golang-samples/functions/helloworld/

    Java

    cd java-docs-samples/functions/helloworld/hello-pubsub/

    C#

    cd dotnet-docs-samples/functions/helloworld/HelloPubSub/

    Ruby

    cd ruby-docs-samples/functions/helloworld/pubsub/

    PHP

    cd php-docs-samples/functions/helloworld_pubsub/

  3. Take a look at the sample code:

    Node.js

    /**
     * Background Cloud Function to be triggered by Pub/Sub.
     * This function is exported by index.js, and executed when
     * the trigger topic receives a message.
     *
     * @param {object} message The Pub/Sub message.
     * @param {object} context The event metadata.
     */
    exports.helloPubSub = (message, context) => {
      const name = message.data
        ? Buffer.from(message.data, 'base64').toString()
        : 'World';
    
      console.log(`Hello, ${name}!`);
    };

    Python

    def hello_pubsub(event, context):
        """Background Cloud Function to be triggered by Pub/Sub.
        Args:
             event (dict):  The dictionary with data specific to this type of
                            event. The `@type` field maps to
                             `type.googleapis.com/google.pubsub.v1.PubsubMessage`.
                            The `data` field maps to the PubsubMessage data
                            in a base64-encoded string. The `attributes` field maps
                            to the PubsubMessage attributes if any is present.
             context (google.cloud.functions.Context): Metadata of triggering event
                            including `event_id` which maps to the PubsubMessage
                            messageId, `timestamp` which maps to the PubsubMessage
                            publishTime, `event_type` which maps to
                            `google.pubsub.topic.publish`, and `resource` which is
                            a dictionary that describes the service API endpoint
                            pubsub.googleapis.com, the triggering topic's name, and
                            the triggering event type
                            `type.googleapis.com/google.pubsub.v1.PubsubMessage`.
        Returns:
            None. The output is written to Cloud Logging.
        """
        import base64
    
        print("""This Function was triggered by messageId {} published at {} to {}
        """.format(context.event_id, context.timestamp, context.resource["name"]))
    
        if 'data' in event:
            name = base64.b64decode(event['data']).decode('utf-8')
        else:
            name = 'World'
        print('Hello {}!'.format(name))

    Go

    
    // Package helloworld provides a set of Cloud Functions samples.
    package helloworld
    
    import (
    	"context"
    	"log"
    )
    
    // PubSubMessage is the payload of a Pub/Sub event.
    // See the documentation for more details:
    // https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
    type PubSubMessage struct {
    	Data []byte `json:"data"`
    }
    
    // HelloPubSub consumes a Pub/Sub message.
    func HelloPubSub(ctx context.Context, m PubSubMessage) error {
    	name := string(m.Data) // Automatically decoded from base64.
    	if name == "" {
    		name = "World"
    	}
    	log.Printf("Hello, %s!", name)
    	return nil
    }
    

    Java

    
    import com.google.cloud.functions.BackgroundFunction;
    import com.google.cloud.functions.Context;
    import com.google.events.cloud.pubsub.v1.Message;
    import java.nio.charset.StandardCharsets;
    import java.util.Base64;
    import java.util.logging.Level;
    import java.util.logging.Logger;
    
    public class HelloPubSub implements BackgroundFunction<Message> {
      private static final Logger logger = Logger.getLogger(HelloPubSub.class.getName());
    
      @Override
      public void accept(Message message, Context context) {
        String name = "world";
        if (message != null && message.getData() != null) {
          name = new String(
              Base64.getDecoder().decode(message.getData().getBytes(StandardCharsets.UTF_8)),
              StandardCharsets.UTF_8);
        }
        logger.info(String.format("Hello %s!", name));
        return;
      }
    }

    C#

    using CloudNative.CloudEvents;
    using Google.Cloud.Functions.Framework;
    using Google.Events.Protobuf.Cloud.PubSub.V1;
    using Microsoft.Extensions.Logging;
    using System.Threading;
    using System.Threading.Tasks;
    
    namespace HelloPubSub
    {
        public class Function : ICloudEventFunction<MessagePublishedData>
        {
            private readonly ILogger _logger;
    
            public Function(ILogger<Function> logger) =>
                _logger = logger;
    
            public Task HandleAsync(CloudEvent cloudEvent, MessagePublishedData data, CancellationToken cancellationToken)
            {
                string nameFromMessage = data.Message?.TextData;
                string name = string.IsNullOrEmpty(nameFromMessage) ? "world" : nameFromMessage;
                _logger.LogInformation("Hello {name}", name);
                return Task.CompletedTask;
            }
        }
    }

    Ruby

    require "functions_framework"
    require "base64"
    
    FunctionsFramework.cloud_event "hello_pubsub" do |event|
      # The event parameter is a CloudEvents::Event::V1 object.
      # See https://cloudevents.github.io/sdk-ruby/latest/CloudEvents/Event/V1.html
      name = Base64.decode64 event.data["message"]["data"] rescue "World"
    
      # A cloud_event function does not return a response, but you can log messages
      # or cause side effects such as sending additional events.
      logger.info "Hello, #{name}!"
    end

    PHP

    
    use Google\CloudFunctions\CloudEvent;
    
    function helloworldPubsub(CloudEvent $event): void
    {
        $log = fopen(getenv('LOGGER_OUTPUT') ?: 'php://stderr', 'wb');
    
        $cloudEventData = $event->getData();
        $pubSubData = base64_decode($cloudEventData['message']['data']);
    
        $name = $pubSubData ? htmlspecialchars($pubSubData) : 'World';
        fwrite($log, "Hello, $name!" . PHP_EOL);
    }

Deploying the function

To deploy the function with a Pub/Sub trigger, run the following command in the directory that contains the sample code (or in the case of Java, the pom.xml file):

Node.js

gcloud functions deploy helloPubSub \
--runtime nodejs14 \
--trigger-topic YOUR_TOPIC_NAME
You can use the following values for the --runtime flag to specify your preferred Node.js version:
  • nodejs10
  • nodejs12
  • nodejs14

Python

gcloud functions deploy hello_pubsub \
--runtime python39 \
--trigger-topic YOUR_TOPIC_NAME
You can use the following values for the --runtime flag to specify your preferred Python version:
  • python37
  • python38
  • python39

Go

gcloud functions deploy HelloPubSub \
--runtime go113 \
--trigger-topic YOUR_TOPIC_NAME
You can use the following values for the --runtime flag to specify your preferred Go version:
  • go111 (Deprecated)
  • go113

Java

gcloud functions deploy java-pubsub-function \
--entry-point functions.HelloPubSub \
--runtime java11 \
--memory 512MB \
--trigger-topic YOUR_TOPIC_NAME

C#

gcloud functions deploy csharp-pubsub-function \
--entry-point HelloPubSub.Function \
--runtime dotnet3 \
--trigger-topic YOUR_TOPIC_NAME

Ruby

gcloud functions deploy hello_pubsub --runtime ruby27 \
--trigger-topic YOUR_TOPIC_NAME
You can use the following values for the --runtime flag to specify your preferred Ruby version:
  • ruby26
  • ruby27

PHP

gcloud functions deploy helloworldPubsub --runtime php74 \
--trigger-topic YOUR_TOPIC_NAME

where YOUR_TOPIC_NAME is the name of the Pub/Sub topic to which the function will be subscribed.

If YOUR_TOPIC_NAME doesn't already exist, this command creates it for you. You can also create a topic before you run the deploy command, using either the Cloud Console or the following gcloud command:

gcloud pubsub topics create YOUR_TOPIC_NAME

Triggering the function

  1. Publish a message to your Pub/Sub topic. In this example, the message is a name that the function will include in a greeting:

    gcloud pubsub topics publish YOUR_TOPIC_NAME --message YOUR_NAME

    Replace YOUR_TOPIC_NAME with the name of your Pub/Sub topic, and YOUR_NAME with an arbitrary string.

  2. Check the logs to be sure the executions have completed:

    gcloud functions logs read --limit 50
    

You can also publish a message to a Pub/Sub topic from within a function.

Cleaning up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting the Cloud Function

Deleting Cloud Functions does not remove any resources stored in Cloud Storage.

To delete the Cloud Function you created in this tutorial, run the following command:

Node.js

gcloud functions delete helloPubSub 

Python

gcloud functions delete hello_pubsub 

Go

gcloud functions delete HelloPubSub 

Java

gcloud functions delete java-pubsub-function 

C#

gcloud functions delete csharp-pubsub-function 

Ruby

gcloud functions delete hello_pubsub 

PHP

gcloud functions delete helloworldPubsub 

You can also delete Cloud Functions from the Google Cloud Console.