Using Cloud Pub/Sub with Cloud Run tutorial

は、

This tutorial shows how to write, deploy, and call a Cloud Run service from a Cloud Pub/Sub push subscription.

You can use this tutorial with Cloud Run or Cloud Run on GKE.

Objectives

  • Write, build, and deploy a service to Cloud Run or Cloud Run on GKE
  • Call the service by publishing a message to a Cloud Pub/Sub topic.

Costs

This tutorial uses billable components of Cloud Platform, including:

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

  1. Google アカウントにログインします。

    Google アカウントをまだお持ちでない場合は、新しいアカウントを登録します。

  2. GCP プロジェクトを選択または作成します。

    [リソースの管理] ページに移動

  3. プロジェクトに対して課金が有効になっていることを確認します。

    課金を有効にする方法について

  4. Enable the Cloud Run API
  5. Install and initialize the Cloud SDK.
  6. Install the gcloud beta component: 
    gcloud components install beta
  7. For Cloud Run on GKE install the gcloud kubectl component: 
    gcloud components install kubectl
  8. Update components: 
    gcloud components update
  9. If you are using Cloud Run on GKE, create a new cluster using the instructions in Setting up Cloud Run on GKE.

Setting up gcloud defaults

To configure gcloud with defaults for your Cloud Run service:

  1. Set your default project:

    gcloud config set project [PROJECT-ID]

    Replace [PROJECT-ID] with the name of the project you created for this tutorial.

  2. If you are using Cloud Run, configure gcloud for your chosen region:

     gcloud config set run/region us-central1

    Replace us-central1 with the supported Cloud Run region of your choice.

  3. If you are using Cloud Run on GKE, configure gcloud for your cluster:

    gcloud config set run/cluster [CLUSTER-NAME]
gcloud config set run/cluster_location us-central1-a

    Replace [CLUSTER-NAME] with the name you used for your cluster, and, if necessary, replace us-central1-a with the supported cluster location of your choice.

Cloud Run locations

Cloud Run is regional, which means the infrastructure that runs your Cloud Run services is located in a specific region and is managed by Google to be redundantly available across all the zones within that region.

Meeting your latency, availability, or durability requirements are primary factors for selecting the region where your Cloud Run services are run. You can generally select the region nearest to your users but you should consider the location of the other GCP products that are used by your Cloud Run service. Using GCP products together across multiple locations can affect your service's latency as well as cost.

Cloud Run is available in the following regions:

  • us-central1 (Iowa)

If you already created a Cloud Run service, you can view the region in the Cloud Run dashboard in the GCP Console.

Creating a Cloud Pub/Sub topic

The sample service is triggered by messages published to a Cloud Pub/Sub topic, so you'll need to create a topic in Cloud Pub/Sub.

To create a new Cloud Pub/Sub topic, use the command:

gcloud pubsub topics create myRunTopic

You can use myRunTopic or replace with a topic name unique within your GCP project.

Retrieving the code sample

To retrieve the code sample for use:

  1. Download the sample service code:

  2. Decompress the archive. For example, use these standard Linux and macOS command-line utilities:

    Tarball

    tar -xzvf [FILE].tar.gz

    Zip file

    unzip [FILE].zip

    Where [FILE] is the filename of the downloaded archive without the file extension.

Looking at the code

The code for this tutorial consists of the following:

  • A server that handles incoming HTTP requests.

    Node.js

    To keep the Node.js service easy to test, the server configuration is separate from the server startup.

    The Node.js web server is set up in app.js. 

    const express = require('express')
const bodyParser = require('body-parser');
const app = express()

app.use(bodyParser.json())
    The web server is started in index.js: 
    const app = require('./app.js');
const PORT = process.env.PORT || 8080;

app.listen(PORT, () => console.log(`nodejs-pubsub-tutorial listening on port ${PORT}`))

    Go 

    // Sample run-pubsub is a Cloud Run service which handles Pub/Sub messages.
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"os"
)

func main() {
	http.HandleFunc("/", HelloPubSub)
	// Determine port for HTTP service.
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
		log.Printf("Defaulting to port %s", port)
	}
	// Start HTTP server.
	log.Printf("Listening on port %s", port)
	log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

  • A Hello World function that handles the Cloud Pub/Sub message and logs a greeting.

    Node.js 

    app.post('/', (req, res) => {

  if (!req.body) {
    const msg = 'no Pub/Sub message received'
    console.error(`error: ${msg}`)
    res.status(400).send(`Bad Request: ${msg}`)
    return
  }
  if (!req.body.message) {
    const msg = 'invalid Pub/Sub message format'
    console.error(`error: ${msg}`)
    res.status(400).send(`Bad Request: ${msg}`)
    return
  }

  const pubSubMessage = req.body.message
  const name = pubSubMessage.data
    ? Buffer.from(pubSubMessage.data, 'base64').toString().trim()
    : 'World'

  console.log(`Hello ${name}!`)
  res.status(204).send()
})

    Go

    go/pubsub/main.go 
    // PubSubMessage is the payload of a Pub/Sub event. Please refer to the docs for
// additional information regarding Pub/Sub events.
type PubSubMessage struct {
	Message struct {
		Data []byte `json:"data,omitempty"`
		ID   string `json:"id"`
	} `json:"message"`
	Subscription string `json:"subscription"`
}

// HelloPubSub consumes a Pub/Sub message.
func HelloPubSub(w http.ResponseWriter, r *http.Request) {
	// Parse the Pub/Sub message.
	var m PubSubMessage

	if err := json.NewDecoder(r.Body).Decode(&m); err != nil {
		log.Printf("json.NewDecoder: %v", err)
		http.Error(w, "Bad Request", http.StatusBadRequest)
		return
	}

	name := string(m.Message.Data)
	if name == "" {
		name = "World"
	}
	log.Printf("Hello %s!", name)
}

    You must code the service to return an accurate HTTP response code. Success codes, such as HTTP 200 or 204, acknowledge complete processing of the Cloud Pub/Sub message. Error codes, such as HTTP 400 or 500, indicate the message will be retried, as described in Receiving messages using Push guide.

  • A Dockerfile that defines the operating environment for the service. The contents of the Dockerfile vary by language.

    Node.js

    nodejs/pubsub/Dockerfile 
    # Use the official Node.js 10 image.
# https://hub.docker.com/_/node
FROM node:10

# Create and change to the app directory.
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./

# Install production dependencies.
RUN npm install --only=production

# Copy local code to the container image.
COPY . .

# Run the web service on container startup.
CMD [ "npm", "start" ]

    Go

    go/pubsub/Dockerfile 
    # Use the offical Golang image to create a build artifact.
# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang
FROM golang as builder

# Copy local code to the container image.
WORKDIR /app
COPY . .

# Build the outyet command inside the container.
# (You may fetch or manage dependencies here,
# either manually or with a tool like "godep".)
RUN CGO_ENABLED=0 GOOS=linux go build -v -o hellopubsub

# Use a Docker multi-stage build to create a lean production image.
# https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds
FROM alpine

# Copy the binary to the production image from the builder stage.
COPY --from=builder /app/hellopubsub .

# Run the web service on container startup.
CMD ["/hellopubsub"]

For details on how to authenticate the origin of Cloud Pub/Sub requests, read the section below on Integrating with Cloud Pub/Sub.

Shipping the code

Shipping code consists of three steps: building a container image with Cloud Build, uploading the container image to Container Registry, and deploying the container image to Cloud Run or Cloud Run on GKE.

To ship your code:

  1. Build your container and publish on Container Registry.

    gcloud builds submit --tag gcr.io/[PROJECT-ID]/pubsub

    Where [PROJECT-ID] is your GCP project ID, and pubsub is the name you want to give your service.

    Upon success, you should see a SUCCESS message containing the ID, creation time, and image name. The image is stored in Container Registry and can be re-used if desired.

  2. Run the following command to deploy your app:

    gcloud beta run deploy pubsub-tutorial --image gcr.io/[PROJECT-ID]/pubsub

    Replace [PROJECT-ID] with your GCP project ID. pubsub is the container name and pubsub-tutorial is the name of the service. Notice that the container image is deployed to the service and region (Cloud Run) or cluster (Cloud Run on GKE) that you configured previously under Setting up gcloud

    If deploying to Cloud Run, respond n, "No", to the "allow unauthenticated" prompt. By keeping the service private you can rely Cloud Run's automatic Cloud Pub/Sub integration to authenticate requests. See Integrating with Cloud Pub/Sub for more details on how this is configured. See Managing Access for more details on IAM-based authentication.

    Wait until the deployment is complete: this can take about half a minute. On success, the command line displays the service URL. This URL is used to configure a Cloud Pub/Sub subscription.

  3. If you want to deploy a code update to the service, repeat the previous steps. Each deployment to a service creates a new revision and automatically starts serving traffic when ready.

Integrating with Cloud Pub/Sub

Now that we have deployed our Cloud Run service, we will configure Cloud Pub/Sub to push messages to it.

To integrate the service with Cloud Pub/Sub:

  1. Enable your project to create Cloud Pub/Sub authentication tokens:

    gcloud projects add-iam-policy-binding [PROJECT-ID] \
     --member=serviceAccount:service-[PROJECT-NUMBER]@gcp-sa-pubsub.iam.gserviceaccount.com \
     --role=roles/iam.serviceAccountTokenCreator

    Replace

    • [PROJECT-ID] with your GCP project ID.
    • [PROJECT-NUMBER] with your GCP project number.

  2. Create or select a service account to represent the Cloud Pub/Sub subscription identity.

    gcloud iam service-accounts create cloud-run-pubsub-invoker \
     --display-name "Cloud Run Pub/Sub Invoker"

    You can use cloud-run-pubsub-invoker or replace with a name unique within your Google Cloud Platform project.

  3. Create a Cloud Pub/Sub subscription with the service account using the appropriate tab depending on whether you are using Cloud Run or Cloud Run on GKE.

    Cloud Run

    1. Give the invoker service account permission to invoke your pubsub-tutorial service:

      gcloud beta run services add-iam-policy-binding pubsub-tutorial \
   --member=serviceAccount:cloud-run-pubsub-invoker@[PROJECT-ID].iam.gserviceaccount.com \
   --role=roles/run.invoker

      It can take several minutes for the IAM changes to propagate. In the meantime you might see HTTP 403 errors in the service logs.

    2. Create a Cloud Pub/Sub subscription with the service account:

      gcloud beta pubsub subscriptions create myRunSubscription --topic myRunTopic \
   --push-endpoint=[SERVICE-URL]/ \
   --push-auth-service-account=cloud-run-pubsub-invoker@[PROJECT-ID].iam.gserviceaccount.com

      Replace

      • myRunTopic with the topic you previously created.
      • [SERVICE-URL] with the HTTPS URL provided on deploying the service. This URL works even if you have also added a domain mapping.
      • [PROJECT-ID] with your GCP project ID.

      The --push-account-service-account flag activates the Cloud Pub/Sub push functionality for Authentication and authorization.

      Your Cloud Run service domain is automatically registered for use with Cloud Pub/Sub subscriptions.

      For Cloud Run only, there is a built-in authentication check that the token is valid and an authorization check that the service account has permission to invoke the Cloud Run service.

    Cloud Run on GKE

    1. Change the default domain of your cluster or add a domain mapping to your service.

    2. Register domain ownership for Cloud Pub/Sub.

    3. Enable HTTPS on your cluster.

    4. Add code to validate the authentication token attached to Cloud Pub/Sub messages. Sample code is provided in Authentication and authorization by the push endpoint.

    The authentication must ensure that the token is valid and associated with the expected service account. Unlike the fully managed version of Cloud Run, Cloud Run on GKE has no built-in authorization check that the token is valid or that the service account has authorization to invoke the Cloud Run on GKE service.

    1. Create a Cloud Pub/Sub subscription with the service account:

      gcloud beta pubsub subscriptions create myRunSubscription --topic myRunTopic \
     --push-endpoint=[SERVICE-URL]/ \
     --push-auth-service-account=cloud-run-pubsub-invoker@[PROJECT-ID].iam.gserviceaccount.com

      Replace

      • myRunTopic with the topic you previously created.
      • [SERVICE-URL] with your custom service URL. Specify https as the protocol.
      • [PROJECT-ID] with your GCP project ID.

      The --push-account-service-account flag activates the Cloud Pub/Sub push functionality for Authentication and authorization.

Your service is now fully integrated with Cloud Pub/Sub.

Trying it out

To test the end-to-end solution:

  1. Send a Cloud Pub/Sub message to the topic:

    gcloud pubsub topics publish myRunTopic --message "Runner"

    You can also publish messages programmatically instead of using the command-line as shown in this tutorial. For more information, see Publishing messages.

  2. Navigate to the service logs:

    1. Navigate to the Google Cloud Platform Console
    2. Click the pubsub-tutorial service.

    3. Select the Logs tab.

      Logs might take a few moments to appear. If you don't see them immediately, check again after a few moments.

  3. Look for the "Hello Runner!" message.

Cleaning up

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. GCP Console で [プロジェクト] ページに移動します。

    プロジェクト ページに移動

  2. プロジェクト リストで、削除するプロジェクトを選択し、[削除] をクリックします。
  3. ダイアログでプロジェクト ID を入力し、[シャットダウン] をクリックしてプロジェクトを削除します。

Deleting the Cloud Run service

Note that deleting Cloud Run services do not remove any resources stored in Cloud Storage. If you want those deleted, you must delete them separately.

To delete the service you deployed in this tutorial:

gcloud beta run services delete [SERVICE-NAME]

Where [SERVICE-NAME] is your chosen service name.

You can also delete Cloud Run or Cloud Run on GKE services from the Google Cloud Platform Console.

What's next

このページは役立ちましたか？評価をお願いいたします。

フィードバックを送信...

このページ
ドキュメントのフィードバック
Cloud Run Documentation
サービスのフィードバック