This tutorial shows how to write, deploy, and call a Knative serving service from a Pub/Sub push subscription.
Objectives
- Write, build, and deploy a service to Knative serving
- Call the service by publishing a message to a Pub/Sub topic.
Costs
In this document, you use the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage,
use the pricing calculator.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
- Enable the Knative serving API
- Install and initialize the gcloud CLI.
- Install the
kubectl
component:gcloud components install kubectl
- Update components:
gcloud components update
- If you are using Knative serving, create a new cluster using the instructions in Setting up Knative serving.
Setting up gcloud defaults
To configure gcloud with defaults for your Knative serving service:
Set your default project:
gcloud config set project PROJECT_ID
Replace PROJECT_ID with the name of the project you use for this tutorial.
Configure gcloud for your cluster:
gcloud config set run/platform gke gcloud config set run/cluster CLUSTER-NAME gcloud config set run/cluster_location REGION
Replace:
- CLUSTER-NAME with the name you used for your cluster,
- REGION with the supported cluster location of your choice.
Creating a Pub/Sub topic
The sample service is triggered by messages published to a Pub/Sub topic, so you'll need to create a topic in Pub/Sub.
To create a new Pub/Sub topic, use the command:
gcloud pubsub topics create myRunTopic
You can use myRunTopic or replace with a topic name unique within your Google Cloud project.
Retrieving the code sample
To retrieve the code sample for use:
Clone the sample app repository to your local machine:
Node.js
git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git
Alternatively, you can download the sample as a zip file and extract it.
Python
git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
Alternatively, you can download the sample as a zip file and extract it.
Go
git clone https://github.com/GoogleCloudPlatform/golang-samples.git
Alternatively, you can download the sample as a zip file and extract it.
Java
git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git
Alternatively, you can download the sample as a zip file and extract it.
Change to the directory that contains the Knative serving sample code:
Node.js
cd nodejs-docs-samples/run/pubsub/
Python
cd python-docs-samples/run/pubsub/
Go
cd golang-samples/run/pubsub/
Java
cd java-docs-samples/run/pubsub/
Looking at the code
The code for this tutorial consists of the following:
A server that handles incoming requests.
Node.js
To keep the Node.js service easy to test, the server configuration is separate from the server startup.The Node.js web server is set up in
The web server is started inapp.js
.index.js
:Python
Go
Java
A handler that processes the Pub/Sub message and logs a greeting.
Node.js
Python
Go
Java
You must code the service to return an accurate HTTP response code. Success codes, such as HTTP
200
or204
, acknowledge complete processing of the Pub/Sub message. Error codes, such as HTTP400
or500
, indicate the message will be retried, as described in Receiving messages using Push guide.A
Dockerfile
that defines the operating environment for the service. The contents of theDockerfile
vary by language.Node.js
Python
Go
Java
This sample uses Jib to build Docker images using common Java tools. Jib optimizes container builds without the need for a Dockerfile or having Docker installed. Learn more about building Java containers with Jib.
For details on how to authenticate the origin of Pub/Sub requests, read the section below on Integrating with Pub/Sub.
Shipping the code
Shipping code consists of three steps: building a container image with Cloud Build, uploading the container image to Container Registry, and deploying the container image to Knative serving.
To ship your code:
Build your container and publish on Container Registry:
Node.js
gcloud builds submit --tag gcr.io/PROJECT_ID/pubsub
Where PROJECT_ID is your Google Cloud project ID, and
pubsub
is the name you want to give your service.Upon success, you should see a SUCCESS message containing the ID, creation time, and image name. The image is stored in Container Registry and can be re-used if desired.
Python
gcloud builds submit --tag gcr.io/PROJECT_ID/pubsub
Where PROJECT_ID is your Google Cloud project ID, and
pubsub
is the name you want to give your service.Upon success, you should see a SUCCESS message containing the ID, creation time, and image name. The image is stored in Container Registry and can be re-used if desired.
Go
gcloud builds submit --tag gcr.io/PROJECT_ID/pubsub
Where PROJECT_ID is your Google Cloud project ID, and
pubsub
is the name you want to give your service.Upon success, you should see a SUCCESS message containing the ID, creation time, and image name. The image is stored in Container Registry and can be re-used if desired.
Java
mvn compile jib:build -Dimage=gcr.io/PROJECT_ID/pubsub
Where PROJECT_ID is your Google Cloud project ID, and
pubsub
is the name you want to give your service.Upon success, you should see a BUILD SUCCESS message. The image is stored in Container Registry and can be re-used if desired.
Run the following command to deploy your app:
gcloud run deploy pubsub-tutorial --image gcr.io/PROJECT_ID/pubsub
Replace PROJECT_ID with your Google Cloud project ID.
pubsub
is the container name andpubsub-tutorial
is the name of the service. Notice that the container image is deployed to the service and cluster that you configured previously under Setting up gcloudWait until the deployment is complete: this can take about half a minute. On success, the command line displays the service URL. This URL is used to configure a Pub/Sub subscription.
If you want to deploy a code update to the service, repeat the previous steps. Each deployment to a service creates a new revision and automatically starts serving traffic when ready.
Integrating with Pub/Sub
Now that we have deployed our Knative serving service, we will configure Pub/Sub to push messages to it.
To integrate the service with Pub/Sub:
Enable Pub/Sub to create authentication tokens in your project:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member=serviceAccount:service-PROJECT-NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \ --role=roles/iam.serviceAccountTokenCreator
Replace
- PROJECT_ID with your Google Cloud project ID.
- PROJECT-NUMBER with your Google Cloud project number.
Create or select a service account to represent the Pub/Sub subscription identity.
gcloud iam service-accounts create cloud-run-pubsub-invoker \ --display-name "Cloud Run for Anthos Pub/Sub Invoker"
You can use
cloud-run-pubsub-invoker
or replace with a name unique within your Google Cloud project.Create a Pub/Sub subscription with the service account:
Enable auto TLS and HTTPS for your cluster and add a domain mapping to your service.
Register domain ownership for Pub/Sub.
Add code to validate the authentication token attached to Pub/Sub messages. Sample code is provided in Authentication and authorization by the push endpoint.
The authentication must ensure that the token is valid and associated with the expected service account. Unlike Cloud Run, Knative serving has no built-in authorization check that the token is valid or that the service account has authorization to invoke the Knative serving service.
Create a Pub/Sub subscription with the service account:
gcloud pubsub subscriptions create myRunSubscription --topic myRunTopic \ --push-endpoint=SERVICE-URL/ \ --push-auth-service-account=cloud-run-pubsub-invoker@PROJECT_ID.iam.gserviceaccount.com
Replace
- myRunTopic with the topic you previously created.
- SERVICE-URL with your custom service URL.
Specify
https
as the protocol. - PROJECT_ID with your Google Cloud project ID.
The
--push-auth-service-account
flag activates the Pub/Sub push functionality for Authentication and authorization.
Your service is now fully integrated with Pub/Sub.
Trying it out
To test the end-to-end solution:
Send a Pub/Sub message to the topic:
gcloud pubsub topics publish myRunTopic --message "Runner"
You can also publish messages programmatically instead of using the command-line as shown in this tutorial. For more information, see Publishing messages.
Navigate to the service logs:
Navigate to the Knative serving page in the Google Cloud console:
Click the
pubsub-tutorial
service.Select the Logs tab.
Logs might take a few moments to appear. If you don't see them immediately, check again after a few moments.
Look for the "Hello Runner!" message.
Clean up
To walk through a more in-depth use case of using Knative serving with Pub/Sub, skip cleanup for now and continue with the Image processing tutorial.
If you created a new project for this tutorial, delete the project. If you used an existing project and wish to keep it without the changes added in this tutorial, delete resources created for the tutorial.
Deleting the project
The easiest way to eliminate billing is to delete the project that you created for the tutorial.
To delete the project:
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Deleting tutorial resources
Delete the Knative serving service you deployed in this tutorial:
gcloud run services delete SERVICE-NAME
Where SERVICE-NAME is your chosen service name.
You can also delete Knative serving services from the Google Cloud console:
Remove the gcloud default configurations you added during the tutorial setup:
gcloud config unset run/platform gcloud config unset run/cluster gcloud config unset run/cluster_location
Remove the project configuration:
gcloud config unset project
Delete other Google Cloud resources created in this tutorial:
- Delete the Pub/Sub topic
myRunTopic
- Delete the Pub/Sub subscription
myRunSubscription
- Delete the container image named
gcr.io/PROJECT_ID/pubsub
from Container Registry. - Delete the invoker service account
cloud-run-pubsub-invoker@PROJECT_ID.iam.gserviceaccount.com
- If you created a cluster for this tutorial, delete the cluster
- Delete the Pub/Sub topic
What's next
- Expand the sample service deployed in this tutorial to add image processing functionality that modifies images uploaded to Cloud Storage.
- Learn more about how topics fit into Pub/Sub architecture and how to manage topics
- Learn more about Pub/Sub subscriptions in managing subscriptions.
- Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.