Quickstart: building a functioning Cloud Pub/Sub system


This quickstart walks you through setting up a simple set of applications that communicate by sending messages through Cloud Pub/Sub rather than synchronous RPCs. By decoupling applications, messaging:

  • Makes applications more robust
  • Might simplify development

For example, the caller (publisher) does not need the receiver (subscriber) to be up and available. It simply sends a message to Cloud Pub/Sub. Nor does the publisher need to know which and how many subscriber applications need to receive the message. As a result, the service can be relied upon to deliver the message to one or more subscriber applications whenever they are available.

This quickstart is intended for Python developers using macOS.


  • A Google Account
  • A macOS X system with Python and Git installed
  • Up to an hour of time to complete

System overview

In this quickstart, you start a publisher application that sends a "Hello, World!" message to two subscribers, as illustrated below:

Diagram of the
  topic, its attached subscriptions, and the publisher and subscriber
  applications that send message to and receive messages from Cloud

The two subscriber applications use the same code, but you start them at different times. This process demonstrates how Cloud Pub/Sub enables asynchronous communication. To build this system, you:

  1. Create the Cloud Pub/Sub topic and subscriptions needed.
  2. Create a service account that the applications use for authentication.
  3. Set up Cloud IAM permissions.
  4. Start three independent applications: one publisher and two subscribers.

Quickstart setup

Set up your GCP project and Cloud Pub/Sub topic and subscriptions

  1. Log into Google Cloud Platform Console.

    Go to the Google Cloud Platform Console

    If you're new to Cloud, click Activate and follow the prompts to set up your Cloud account.

    At the time of this quickstart's creation, there is no cost for the first fraction of the free monthly data allotment. See the Cloud Pub/Sub pricing page for details. This quickstart also includes cleanup instructions.

  2. Select an existing project or create a new one. A default project is created for you the first time you use GCP.

    In the Home section of the GCP Console, make a note of the Project ID. You use this value to set your current Cloud Storage project during the Cloud SDK initialization process. You also pass this ID to the Python script as you start the publisher and subscriber applications.

  3. Go to the Cloud Pub/Sub section of the Google Cloud Platform Console.

    Go to the Cloud Pub/Sub section

    Follow the prompt to enable the API.

  4. Click Create a topic. Publishing applications send messages to topics. Use hello_topic as the Name.

  5. Click the topic name, then click Create Subscription:

    1. Name the subscription sub_one. Do not change any of the default settings. You are creating a StreamingPull subscription, which is a type of pull subscription.

    2. Use the same procedure to create another subscription attached to hello_topic, named sub_two.

      You can click the topic name in the Topics view to see the new subscriptions or you can change to the Subscriptions view.

At this point, your Cloud Pub/Sub environment is ready to manage message flow between the quickstart's publishing and subscribing applications.

Create service account credentials

  1. Go to the Service accounts section of the console.

    Go to Cloud IAM service accounts

  2. Select your project and click Create Service Account.

  3. Enter a Service account name, such as pubsub-quickstart.

  4. Click Create.

  5. For the quickstart, the service account needs publishing and subscribing permissions. Use the Select a role dropdown to add the Cloud Pub/Sub Publisher role.

    The Service
account permissions dialog, using the string 'pub' to filter for
Pub/Sub roles

  6. Click Add another role and add Cloud Pub/Sub Subscriber.

    The Service
account permissions dialog, with Pub/Sub Publisher and Pub/Sub subscriber,
clicking the Continue button

  7. Click Continue. You do not need to grant users access to this service account.

  8. Click Create Key. The key is used by the client library to access the Cloud Pub/Sub API.

  9. Select JSON and click Create.

    The key is sent to your Downloads folder. For the purposes of this quickstart, you can leave it there.

  10. Rename the key file to ~/Downloads/key.json.

Install the Cloud SDK

  1. Follow the instructions for installing and initializing the Cloud SDK.

    • While initializing the Cloud SDK, select the option to enter a project ID and enter the ID of the project that you created or chose in the setup section.

    • You can return to this quickstart after you install and initialize the Cloud SDK. You do not need to install other components or download the Cloud Client Libraries.

    After you have installed the Cloud SDK, you can use the gcloud command-line tool perform Cloud Pub/Sub operations in Compute Engine.

  2. Start a new terminal before using these gcloud commands:

    gcloud pubsub topics list
    gcloud pubsub subscriptions list

    You can also use the gcloud config set project PROJECT_ID to change the project from the one you set up during initialization.

Get the latest Python and set up a virtual environment

This quickstart provides a usage example, so you don't need to follow the example shown in the virtual environment setup section. You can return to this quickstart after installing the virtual environment.

Check out the publisher and subscriber code

  1. Create a project folder to contain the Cloud Pub/Sub Python files needed for this quickstart. Then change to it and download the code:

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
  2. Close any open terminals before proceeding.

Set up three terminals

  1. Start one terminal for each quickstart application (one publisher and two subscribers). In each of the terminals, perform all of the operations in this section. For convenience, we refer to these terminals as:

    • publisher terminal
    • sub_one terminal
    • sub_two terminal
  2. Create a Python virtual environment and activate it.

    • In the first terminal, run this command:

      virtualenv --python python2.7 pyenv-qs && source pyenv-qs/bin/activate
    • In the other two terminals, the following command is sufficient:

      source pyenv-qs/bin/activate

    After you run the activate command, your command prompt should include (pyenv-qs) $.

    You can also point your virtual environment to a different Python version.

  3. Ensure that you are using the virtual environment as described in the setup guide:

    pip install --upgrade google-cloud-pubsub

    Associate the JSON key with the service account. You assigned the key Cloud Pub/Sub roles when you created the service account credentials.

    export GOOGLE_APPLICATION_CREDENTIALS=~/Downloads/key.json
  4. Set up an environment variable with your current project ID. This gcloud command determines your currently selected project ID and sets it as a variable:

    export PROJECT=`gcloud config get-value project`

    To verify that your current GCP is correctly registered as this variable:

    echo $PROJECT
  5. Change to your project folder, then navigate to the quickstart sample folder:

    cd python-docs-samples/pubsub/cloud-client/quickstart/

Start the apps and observe message flow

Start the Subscriber 1 application

In the sub_one terminal, start Subscriber 1:

python sub.py $PROJECT sub_one

Once started, this application polls the Cloud Pub/Sub sub_one subscription.

Subscriber 1
  application begins listening for messages on the sub_one subscription.

Start the Publisher application

In the publisher terminal, start the Publisher application:

python pub.py $PROJECT hello_topic
  • The Publisher application sends a "Hello, World!" message to Cloud Pub/Sub while remaining unaware of any existing subscriptions. The application also assigns a message ID.

  • The Subscriber 1 application receives the 'Hello World' message, prints it, and sends an acknowledgment to Cloud Pub/Sub.

  • The Publisher application prints the acknowledgement. The acknowledgment tells Cloud Pub/Sub that the message has been processed successfully and does not need to be re-sent to this or any other sub_one subscriber.

  • Cloud Pub/Sub removes the message from sub_one.

The Publisher
  application publishes the message and assigns a message ID. The Subscriber 1
  application receives the 'Hello World' message and sends an

Start the Subscriber 2 application

In the sub_two terminal, start Subscriber 2:

python sub.py $PROJECT sub_two

This subscriber receives messages delivered to the sub_two subscription. Subscriber 2 reuses the sub.py script. The difference is that Subscriber 2 isn't started until after the Publisher has sent the message to the topic and subscriptions. If Publisher were calling Subscriber 2 directly, the publishing application would either have to wait until Subscriber 2 comes up or it would have to time out. Cloud Pub/Sub manages this process by effectively saving the message for Subscriber 2.

Subscriber 2
  starts listening and receives the message that was waiting for it in

You are now ready to develop with Cloud Pub/Sub!

How did it go?

Additional resources and links are available on the Cloud Pub/Sub support page.

Clean up

  1. Stop all running applications.

  2. Delete the ~/pubsub-quickstart directory from your local environment.

  3. Shut down the quickstart project in the IAM & admin section of the Google Cloud Platform Console.

  4. Remove the service account credentials: rm ~/Downloads/key.json

What's next

Here are some things that you can try:

  • Examine the quickstart's pub.py and sub.py code and browse other Cloud Pub/Sub samples on github. As an exercise, create a version of pub.py that publishes the local time every second.

  • Learn to batch messages.

  • Process high-volume Cloud Pub/Sub subscriptions with Cloud Dataflow

  • Using Push subscriptions, receive messages that trigger App Engine endpoints or Cloud Functions.

  • Retrieve previously acknowledged messages using replay. By default, Cloud Pub/Sub removes acknowledged messages from subscriptions. In this quickstart, for instance, you would not be able to rerun sub.py to receive the "Hello, World!" message again. The replay feature allows you to set up subscriptions so that you can receive messages after they have been acknowledged.

هل كانت هذه الصفحة مفيدة؟ يرجى تقييم أدائنا:

إرسال تعليقات حول...

Cloud Pub/Sub Documentation