Writing and Responding to Pub/Sub Messages

Region ID

The REGION_ID is a code that Google assigns based on the region you select when you create your app. Including REGION_ID.r in App Engine URLs is optional for existing apps and will soon be required for all new apps.

To ensure a smooth transition, we are slowly updating App Engine to use region IDs. If we haven't updated your Google Cloud project yet, you won't see a region ID for your app. Since the ID is optional for existing apps, you don't need to update URLs or make other changes once the region ID is available for your existing apps.

Learn more about region IDs.

Pub/Sub provides reliable, many-to-many, asynchronous messaging between applications. Publisher applications can send messages to a topic, and other applications can subscribe to that topic to receive the messages.

This document describes how to use the Cloud Client Library to send and receive Pub/Sub messages in a Python 3.7 app.

Prerequisites

  • Follow the instructions in "Hello, World!" for Python 3.7 on App Engine to set up your environment and project, and to understand how App Engine Python 3.7 apps are structured.
  • Write down and save your project ID, because you will need it to run the sample application described in this document.

Cloning the sample app

Copy the sample apps to your local machine, and navigate to the pubsub directory:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples
cd python-docs-samples/appengine/standard_python37/pubsub

Creating a topic and subscription

Create a topic and subscription, which includes specifying the endpoint to which the Pub/Sub server should send requests:

gcloud pubsub topics create YOUR_TOPIC_NAME
gcloud pubsub subscriptions create YOUR_SUBSCRIPTION_NAME \
    --topic YOUR_TOPIC_NAME \
    --push-endpoint \
    https://YOUR_PROJECT_ID.REGION_ID.r.appspot.com/pubsub/push?token=YOUR_TOKEN \
    --ack-deadline 10

Replace YOUR_TOKEN with a secret random token. The push endpoint uses this to verify requests.

Setting environment variables

Edit the app.yaml file to set the environment variables for your project ID, topic, and verification token:

env_variables:
    PUBSUB_TOPIC: your-topic
    # This token is used to verify that requests originate from your
    # application. It can be any sufficiently random string.
    PUBSUB_VERIFICATION_TOKEN: 1234abc

Code review

The sample app uses the Cloud Client Libraries.

The sample app uses the values you set in the app.yaml file to configure environment variables. The push request handler uses these values to confirm that the request came from Pub/Sub and originated from a trusted source:

app.config['PUBSUB_VERIFICATION_TOKEN'] = \
    os.environ['PUBSUB_VERIFICATION_TOKEN']
app.config['PUBSUB_TOPIC'] = os.environ['PUBSUB_TOPIC']

The sample app maintains a global list to store messages received by this instance:

MESSAGES = []
The receive_messages_handler() method receives pushed messages and adds them to the MESSAGES global list:

@app.route('/push-handlers/receive_messages', methods=['POST'])
def receive_messages_handler():
    # Verify that the request originates from the application.
    if (request.args.get('token', '') !=
            current_app.config['PUBSUB_VERIFICATION_TOKEN']):
        return 'Invalid request', 400

    # Verify that the push request originates from Cloud Pub/Sub.
    try:
        # Get the Cloud Pub/Sub-generated JWT in the "Authorization" header.
        bearer_token = request.headers.get('Authorization')
        token = bearer_token.split(' ')[1]
        TOKENS.append(token)

        # Verify and decode the JWT. `verify_oauth2_token` verifies
        # the JWT signature, the `aud` claim, and the `exp` claim.
        # Note: For high volume push requests, it would save some network
        # overhead if you verify the tokens offline by downloading Google's
        # Public Cert and decode them using the `google.auth.jwt` module;
        # caching already seen tokens works best when a large volume of
        # messages have prompted a single push server to handle them, in which
        # case they would all share the same token for a limited time window.
        claim = id_token.verify_oauth2_token(token, requests.Request(),
                                             audience='example.com')
        # Must also verify the `iss` claim.
        if claim['iss'] not in [
            'accounts.google.com',
            'https://accounts.google.com'
        ]:
            raise ValueError('Wrong issuer.')
        CLAIMS.append(claim)
    except Exception as e:
        return 'Invalid token: {}\n'.format(e), 400

    envelope = json.loads(request.data.decode('utf-8'))
    payload = base64.b64decode(envelope['message']['data'])
    MESSAGES.append(payload)
    # Returning any 2xx status indicates successful receipt of the message.
    return 'OK', 200

The index() method interacts with the App Engine web app to publish new messages and display received messages:

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'GET':
        return render_template('index.html', messages=MESSAGES, tokens=TOKENS,
                               claims=CLAIMS)

    data = request.form.get('payload', 'Example payload').encode('utf-8')

    # Consider initializing the publisher client outside this function
    # for better latency performance.
    publisher = pubsub_v1.PublisherClient()
    topic_path = publisher.topic_path(app.config['GCLOUD_PROJECT'],
                                      app.config['PUBSUB_TOPIC'])
    future = publisher.publish(topic_path, data)
    future.result()
    return 'OK', 200

Running the sample locally

When running locally, you can use the Cloud SDK to provide authentication to use Google Cloud APIs. Assuming you set up your environment as described in Prerequisites, you have already run the gcloud init command, which provides this authentication.

Install dependencies, preferably in a virtual environment.

Mac OS / Linux

  1. Create an isolated Python environment in a directory external to your project and activate it:
    python3 -m venv env
    source env/bin/activate
  2. Navigate to your project directory and install dependencies:
    cd YOUR_PROJECT
    pip install  -r requirements.txt

Windows

Use PowerShell to run your Python packages.

  1. Locate your installation of PowerShell.
  2. Right-click on the shortcut to PowerShell and start it as an administrator.
  3. Create an isolated Python environment in a directory external to your project and activate it:
    python -m venv env
    env\Scripts\activate
  4. Navigate to your project directory and install dependencies:
    cd YOUR_PROJECT
    pip install -r requirements.txt

Then set environment variables before starting your application:

export GOOGLE_CLOUD_PROJECT=[your-project-id]
export PUBSUB_VERIFICATION_TOKEN=[your-verification-token]
export PUBSUB_TOPIC=[your-topic]
python main.py

Simulating push notifications

The application can send messages locally, but it is not able to receive push messages locally. You can, however, simulate a push message by making an HTTP request to the local push notification endpoint. The sample includes the file sample_message.json.

You can use curl or a httpie client to send an HTTP POST request:

curl -i -H "Content-Type: application/json" --data @sample_message.json "localhost:8080/pubsub/push?token=[your-token]"

Or

http POST ":8080/pubsub/push?token=[your-token]" < sample_message.json

Response:

HTTP/1.0 200 OK
Content-Length: 2
Content-Type: text/html; charset=utf-8
Date: Mon, 10 Aug 2015 17:52:03 GMT
Server: Werkzeug/0.10.4 Python/2.7.10

OK

After the request completes, you can refresh localhost:8080 and see the message in the list of received messages.

Running on App Engine

To deploy the demo app to App Engine by using the gcloud command-line tool, you run the following command from the directory where your app.yaml file is located:

gcloud app deploy

You can now access the application at https://PROJECT_ID.REGION_ID.r.appspot.com. You can use the form to submit messages, but there's no guarantee of which instance of your application will receive the notification. You can send multiple messages and refresh the page to see the received message.