Using Cloud Pub/Sub with Cloud Run tutorial

This tutorial shows how to write, deploy, and call a Cloud Run service from a Cloud Pub/Sub push subscription.

You can use this tutorial with Cloud Run or Cloud Run for Anthos on Google Cloud.

Objectives

  • Write, build, and deploy a service to Cloud Run or Cloud Run for Anthos on Google Cloud
  • Call the service by publishing a message to a Cloud Pub/Sub topic.

Costs

This tutorial uses billable components of Cloud Platform, including:

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Google アカウントにログインします。

    Google アカウントをまだお持ちでない場合は、新しいアカウントを登録します。

  2. GCP プロジェクトを選択または作成します。

    プロジェクト セレクタのページに移動

  3. Google Cloud Platform プロジェクトに対して課金が有効になっていることを確認します。 詳しくは、課金を有効にする方法をご覧ください。

  4. Enable the Cloud Run API
  5. Install and initialize the Cloud SDK.
  6. Install the gcloud beta component: 
    gcloud components install beta
  7. For Cloud Run for Anthos on Google Cloud install the gcloud kubectl component: 
    gcloud components install kubectl
  8. Update components: 
    gcloud components update
  9. If you are using Cloud Run for Anthos on Google Cloud, create a new cluster using the instructions in Setting up Cloud Run for Anthos on Google Cloud.

Setting up gcloud defaults

To configure gcloud with defaults for your Cloud Run service:

  1. Set your default project:

    gcloud config set project PROJECT-ID

    Replace PROJECT-ID with the name of the project you created for this tutorial.

  2. If you are using Cloud Run (fully managed), configure gcloud for your chosen region:

    gcloud config set run/region REGION

    Replace REGION with the supported Cloud Run region of your choice.

  3. If you are using Cloud Run for Anthos on Google Cloud, configure gcloud for your cluster:

    gcloud config set run/cluster CLUSTER-NAME
gcloud config set run/cluster_location REGION

    Replace

    • CLUSTER-NAME with the name you used for your cluster,
    • REGION with the supported cluster location of your choice.

Cloud Run locations

Cloud Run is regional, which means the infrastructure that runs your Cloud Run services is located in a specific region and is managed by Google to be redundantly available across all the zones within that region.

Meeting your latency, availability, or durability requirements are primary factors for selecting the region where your Cloud Run services are run. You can generally select the region nearest to your users but you should consider the location of the other GCP products that are used by your Cloud Run service. Using GCP products together across multiple locations can affect your service's latency as well as cost.

Cloud Run is available in the following regions:

  • asia-northeast1 (Tokyo)
  • europe-west1 (Belgium)
  • us-central1 (Iowa)
  • us-east1 (South Carolina)

If you already created a Cloud Run service, you can view the region in the Cloud Run dashboard in the GCP Console.

Creating a Cloud Pub/Sub topic

The sample service is triggered by messages published to a Cloud Pub/Sub topic, so you'll need to create a topic in Cloud Pub/Sub.

To create a new Cloud Pub/Sub topic, use the command:

gcloud pubsub topics create myRunTopic

You can use myRunTopic or replace with a topic name unique within your GCP project.

Retrieving the code sample

To retrieve the code sample for use:

  1. Clone the sample app repository to your local machine:

    Node.js 

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Python 

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Go 

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

    Java 

    git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

  2. Change to the directory that contains the Cloud Run sample code:

    Node.js 

    cd nodejs-docs-samples/run/pubsub/

    Python 

    cd python-docs-samples/run/pubsub/

    Go 

    cd golang-samples/run/pubsub/

    Java 

    cd java-docs-samples/run/pubsub/

Looking at the code

The code for this tutorial consists of the following:

  • A server that handles incoming requests.

    Node.js

    To keep the Node.js service easy to test, the server configuration is separate from the server startup.

    The Node.js web server is set up in app.js.

    run/pubsub/app.js 
    const express = require('express');
const bodyParser = require('body-parser');
const app = express();

app.use(bodyParser.json());
    The web server is started in index.js:
    run/pubsub/index.js 
    const app = require('./app.js');
const PORT = process.env.PORT || 8080;

app.listen(PORT, () =>
  console.log(`nodejs-pubsub-tutorial listening on port ${PORT}`)
);

    Python

    run/pubsub/main.py 
    import base64
from flask import Flask, request
import os
import sys

app = Flask(__name__)

    Go

    run/pubsub/main.go 
    
// Sample run-pubsub is a Cloud Run service which handles Pub/Sub messages.
package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"os"
)

func main() {
	http.HandleFunc("/", HelloPubSub)
	// Determine port for HTTP service.
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
		log.Printf("Defaulting to port %s", port)
	}
	// Start HTTP server.
	log.Printf("Listening on port %s", port)
	log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

    Java

    run/pubsub/src/main/java/com/example/cloudrun/PubSubApplication.java 
    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class PubSubApplication {
  public static void main(String[] args) {
    SpringApplication.run(PubSubApplication.class, args);
  }
}

  • A handler that processes the Cloud Pub/Sub message and logs a greeting.

    Node.js

    run/pubsub/app.js 
    app.post('/', (req, res) => {
  if (!req.body) {
    const msg = 'no Pub/Sub message received';
    console.error(`error: ${msg}`);
    res.status(400).send(`Bad Request: ${msg}`);
    return;
  }
  if (!req.body.message) {
    const msg = 'invalid Pub/Sub message format';
    console.error(`error: ${msg}`);
    res.status(400).send(`Bad Request: ${msg}`);
    return;
  }

  const pubSubMessage = req.body.message;
  const name = pubSubMessage.data
    ? Buffer.from(pubSubMessage.data, 'base64')
        .toString()
        .trim()
    : 'World';

  console.log(`Hello ${name}!`);
  res.status(204).send();
});

    Python

    run/pubsub/main.py 
    @app.route('/', methods=['POST'])
def index():
    envelope = request.get_json()
    if not envelope:
        msg = 'no Pub/Sub message received'
        print(f'error: {msg}')
        return f'Bad Request: {msg}', 400

    if not isinstance(envelope, dict) or 'message' not in envelope:
        msg = 'invalid Pub/Sub message format'
        print(f'error: {msg}')
        return f'Bad Request: {msg}', 400

    pubsub_message = envelope['message']

    name = 'World'
    if isinstance(pubsub_message, dict) and 'data' in pubsub_message:
        name = base64.b64decode(pubsub_message['data']).decode('utf-8').strip()

    print(f'Hello {name}!')

    # Flush the stdout to avoid log buffering.
    sys.stdout.flush()

    return ('', 204)

    Go

    run/pubsub/main.go 
    
// PubSubMessage is the payload of a Pub/Sub event.
type PubSubMessage struct {
	Message struct {
		Data []byte `json:"data,omitempty"`
		ID   string `json:"id"`
	} `json:"message"`
	Subscription string `json:"subscription"`
}

// HelloPubSub receives and processes a Pub/Sub push message.
func HelloPubSub(w http.ResponseWriter, r *http.Request) {
	var m PubSubMessage
	body, err := ioutil.ReadAll(r.Body)
	if err != nil {
		log.Printf("iotuil.ReadAll: %v", err)
		http.Error(w, "Bad Request", http.StatusBadRequest)
		return
	}
	if err := json.Unmarshal(body, &m); err != nil {
		log.Printf("json.Unmarshal: %v", err)
		http.Error(w, "Bad Request", http.StatusBadRequest)
		return
	}

	name := string(m.Message.Data)
	if name == "" {
		name = "World"
	}
	log.Printf("Hello %s!", name)
}

    Java

    run/pubsub/src/main/java/com/example/cloudrun/PubSubController.java 
    import java.util.Base64;
import org.apache.commons.lang3.StringUtils;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

// PubsubController consumes a Pub/Sub message.
@RestController
public class PubSubController {
  @RequestMapping(value = "/", method = RequestMethod.POST)
  public ResponseEntity receiveMessage(@RequestBody Body body) {
    // Get PubSub message from request body.
    Body.Message message = body.getMessage();
    if (message == null) {
      String msg = "Bad Request: invalid Pub/Sub message format";
      System.out.println(msg);
      return new ResponseEntity(msg, HttpStatus.BAD_REQUEST);
    }

    String data = message.getData();
    String target =
        !StringUtils.isEmpty(data) ? new String(Base64.getDecoder().decode(data)) : "World";
    String msg = "Hello " + target + "!";

    System.out.println(msg);
    return new ResponseEntity(msg, HttpStatus.OK);
  }
}

    You must code the service to return an accurate HTTP response code. Success codes, such as HTTP 200 or 204, acknowledge complete processing of the Cloud Pub/Sub message. Error codes, such as HTTP 400 or 500, indicate the message will be retried, as described in Receiving messages using Push guide.

  • A Dockerfile that defines the operating environment for the service. The contents of the Dockerfile vary by language.

    Node.js

    run/pubsub/Dockerfile 
    
# Use the official lightweight Node.js 10 image.
# https://hub.docker.com/_/node
FROM node:10-slim

# Create and change to the app directory.
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./

# Install dependencies.
# If you add a package-lock.json speed your build by switching to 'npm ci'.
# RUN npm ci --only=production
RUN npm install --production

# Copy local code to the container image.
COPY . .

# Run the web service on container startup.
CMD [ "npm", "start" ]

    Python

    run/pubsub/Dockerfile 
    
# Use the official Python image.
# https://hub.docker.com/_/python
FROM python:3.7

# Copy application dependency manifests to the container image.
# Copying this separately prevents re-running pip install on every code change.
COPY requirements.txt ./

# Install production dependencies.
RUN pip install -r requirements.txt

# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./

# Run the web service on container startup.
# Use gunicorn webserver with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 main:app

    Go

    run/pubsub/Dockerfile 
    
# Use the offical Golang image to create a build artifact.
# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang
FROM golang:1.12 as builder

# Copy local code to the container image.
WORKDIR /app
COPY . .

# Build hellopubsub inside the container.
RUN CGO_ENABLED=0 GOOS=linux go build -v -o hellopubsub

# Use a Docker multi-stage build to create a lean production image.
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM alpine:3.10

# Install certificates for secure communication with network services.
RUN apk add --no-cache ca-certificates

# Copy the binary to the production image from the builder stage.
COPY --from=builder /app/hellopubsub .

# Run the web service on container startup.
CMD ["/hellopubsub"]

    Java

    run/pubsub/Dockerfile 
    
# Use the official maven/Java 11 image to create a build artifact.
# https://hub.docker.com/_/maven
FROM maven:3.6.1-jdk-11 as builder

# Copy local code to the container image.
WORKDIR /app
COPY pom.xml .
COPY src ./src

# Build a release artifact.
RUN mvn package -DskipTests

# Use AdoptOpenJDK for base image.
# It's important to use OpenJDK 8u191 or above that has container support enabled.
# https://hub.docker.com/r/adoptopenjdk/openjdk11
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7-slim

# Copy the jar to the production image from the builder stage.
COPY --from=builder /app/target/pubsub-*.jar /pubsub.jar

# Run the web service on container startup.
CMD ["java","-Dserver.port=${PORT}","-jar","/pubsub.jar"]

For details on how to authenticate the origin of Cloud Pub/Sub requests, read the section below on Integrating with Cloud Pub/Sub.

Shipping the code

Shipping code consists of three steps: building a container image with Cloud Build, uploading the container image to Container Registry, and deploying the container image to Cloud Run or Cloud Run for Anthos on Google Cloud.

To ship your code:

  1. Build your container and publish on Container Registry:

    gcloud builds submit --tag gcr.io/PROJECT-ID/pubsub

    Where PROJECT-ID is your GCP project ID, and pubsub is the name you want to give your service.

    Upon success, you should see a SUCCESS message containing the ID, creation time, and image name. The image is stored in Container Registry and can be re-used if desired.

  2. Run the following command to deploy your app:

    gcloud beta run deploy pubsub-tutorial --image gcr.io/PROJECT-ID/pubsub

    Replace PROJECT-ID with your GCP project ID. pubsub is the container name and pubsub-tutorial is the name of the service. Notice that the container image is deployed to the service and region (Cloud Run) or cluster (Cloud Run for Anthos on Google Cloud) that you configured previously under Setting up gcloud

    If deploying to Cloud Run, respond n, "No", to the "allow unauthenticated" prompt. By keeping the service private you can rely Cloud Run's automatic Cloud Pub/Sub integration to authenticate requests. See Integrating with Cloud Pub/Sub for more details on how this is configured. See Managing Access for more details on IAM-based authentication.

    Wait until the deployment is complete: this can take about half a minute. On success, the command line displays the service URL. This URL is used to configure a Cloud Pub/Sub subscription.

  3. If you want to deploy a code update to the service, repeat the previous steps. Each deployment to a service creates a new revision and automatically starts serving traffic when ready.

Integrating with Cloud Pub/Sub

Now that we have deployed our Cloud Run service, we will configure Cloud Pub/Sub to push messages to it.

To integrate the service with Cloud Pub/Sub:

  1. Enable your project to create Cloud Pub/Sub authentication tokens:

    gcloud projects add-iam-policy-binding PROJECT-ID \
     --member=serviceAccount:service-PROJECT-NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \
     --role=roles/iam.serviceAccountTokenCreator

    Replace

    • PROJECT-ID with your GCP project ID.
    • PROJECT-NUMBER with your GCP project number.

  2. Create or select a service account to represent the Cloud Pub/Sub subscription identity.

    gcloud iam service-accounts create cloud-run-pubsub-invoker \
     --display-name "Cloud Run Pub/Sub Invoker"

    You can use cloud-run-pubsub-invoker or replace with a name unique within your Google Cloud Platform project.

  3. Create a Cloud Pub/Sub subscription with the service account using the appropriate tab depending on whether you are using Cloud Run or Cloud Run for Anthos on Google Cloud.

    Cloud Run

    1. Give the invoker service account permission to invoke your pubsub-tutorial service:

      gcloud beta run services add-iam-policy-binding pubsub-tutorial \
   --member=serviceAccount:cloud-run-pubsub-invoker@PROJECT-ID.iam.gserviceaccount.com \
   --role=roles/run.invoker

      It can take several minutes for the IAM changes to propagate. In the meantime you might see HTTP 403 errors in the service logs.

    2. Create a Cloud Pub/Sub subscription with the service account:

      gcloud beta pubsub subscriptions create myRunSubscription --topic myRunTopic \
   --push-endpoint=SERVICE-URL/ \
   --push-auth-service-account=cloud-run-pubsub-invoker@PROJECT-ID.iam.gserviceaccount.com

      Replace

      • myRunTopic with the topic you previously created.
      • SERVICE-URL with the HTTPS URL provided on deploying the service. This URL works even if you have also added a domain mapping.
      • PROJECT-ID with your GCP project ID.

      The --push-account-service-account flag activates the Cloud Pub/Sub push functionality for Authentication and authorization.

      Your Cloud Run service domain is automatically registered for use with Cloud Pub/Sub subscriptions.

      For Cloud Run only, there is a built-in authentication check that the token is valid and an authorization check that the service account has permission to invoke the Cloud Run service.

    Cloud Run on GKE

    1. Change the default domain of your cluster or add a domain mapping to your service.

    2. Register domain ownership for Cloud Pub/Sub.

    3. Enable HTTPS on your cluster.

    4. Add code to validate the authentication token attached to Cloud Pub/Sub messages. Sample code is provided in Authentication and authorization by the push endpoint.

    The authentication must ensure that the token is valid and associated with the expected service account. Unlike Cloud Run (fully managed), Cloud Run for Anthos on Google Cloud has no built-in authorization check that the token is valid or that the service account has authorization to invoke the Cloud Run for Anthos on Google Cloud service.

    1. Create a Cloud Pub/Sub subscription with the service account:

      gcloud beta pubsub subscriptions create myRunSubscription --topic myRunTopic \
     --push-endpoint=SERVICE-URL/ \
     --push-auth-service-account=cloud-run-pubsub-invoker@PROJECT-ID.iam.gserviceaccount.com

      Replace

      • myRunTopic with the topic you previously created.
      • SERVICE-URL with your custom service URL. Specify https as the protocol.
      • PROJECT-ID with your GCP project ID.

      The --push-account-service-account flag activates the Cloud Pub/Sub push functionality for Authentication and authorization.

Your service is now fully integrated with Cloud Pub/Sub.

Trying it out

To test the end-to-end solution:

  1. Send a Cloud Pub/Sub message to the topic:

    gcloud pubsub topics publish myRunTopic --message "Runner"

    You can also publish messages programmatically instead of using the command-line as shown in this tutorial. For more information, see Publishing messages.

  2. Navigate to the service logs:

    1. Navigate to the Google Cloud Platform Console
    2. Click the pubsub-tutorial service.

    3. Select the Logs tab.

      Logs might take a few moments to appear. If you don't see them immediately, check again after a few moments.

  3. Look for the "Hello Runner!" message.

Cleaning up

To walk through a more in-depth use case of using Cloud Run with Cloud Pub/Sub, skip cleanup for now and continue with the Image Processing with Cloud Run tutorial.

If you created a new project for this tutorial, delete the project. If you used an existing project and wish to keep it without the changes added in this tutorial, delete resources created for the tutorial.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. GCP Console で [プロジェクト] ページに移動します。

    プロジェクト ページに移動

  2. プロジェクト リストで、削除するプロジェクトを選択し、[削除] をクリックします。
  3. ダイアログでプロジェクト ID を入力し、[シャットダウン] をクリックしてプロジェクトを削除します。

Deleting tutorial resources

  1. Delete the Cloud Run service you deployed in this tutorial:

    gcloud beta run services delete SERVICE-NAME

    Where SERVICE-NAME is your chosen service name.

    You can also delete Cloud Run services from the Google Cloud Platform Console.

  2. Remove the gcloud default configurations you added during tutorial setup.

    If you use Cloud Run (fully managed), remove the region setting:

     gcloud config unset run/region

    If you use Cloud Run for Anthos on Google Cloud, remove the cluster configuration:

     gcloud config unset run/cluster run/cluster
 gcloud config unset run/cluster run/cluster_location

  3. Remove the project configuration:

     gcloud config unset project

  4. Delete other Google Cloud Platform resources created in this tutorial:

What's next

このページは役立ちましたか？評価をお願いいたします。

フィードバックを送信...

このページ
ドキュメントのフィードバック
Cloud Run Documentation
サービスのフィードバック