Many apps need to do background processing outside of the context of a web request. This tutorial creates a web app that lets users input text to translate, and then displays a list of previous translations. The translation is done in a background process to avoid blocking the user's request.
The following diagram illustrates the translation request process.
Here is the sequence of events for how the tutorial app works:
- Visit the web page to see a list of previous translations, stored in Firestore.
- Request a translation of text by entering an HTML form.
- The translation request is published to Pub/Sub.
- A Cloud Run app receives the Pub/Sub message.
- The Cloud Run app uses Cloud Translation to translate the text.
- The Cloud Run app stores the result in Firestore.
This tutorial is intended for anyone who is interested in learning about background processing with Google Cloud. No prior experience is required with Pub/Sub, Firestore, App Engine, or Cloud Run. However, to understand all of the code, some experience with PHP, JavaScript, and HTML is helpful.
Objectives
- Understand and deploy Cloud Run services.
- Understand and deploy an App Engine app.
- Try the app.
Costs
In this document, you use the following billable components of Google Cloud:
To generate a cost estimate based on your projected usage,
use the pricing calculator.
When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, see Clean up.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Firestore, Cloud Run, Pub/Sub, and Cloud Translation APIs.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Firestore, Cloud Run, Pub/Sub, and Cloud Translation APIs.
-
In the Google Cloud console, open the app in Cloud Shell.
Cloud Shell provides command-line access to your cloud resources directly from the browser. Open Cloud Shell in your browser and click Proceed to download the sample code and change into the app directory.
-
In Cloud Shell, configure the
gcloud
tool to use your Google Cloud project:# Configure gcloud for your project gcloud config set project YOUR_PROJECT_ID
Understanding the Cloud Run backend
You define a single PHP function translateString
, and configure your
Cloud Run service to respond to a Pub/Sub message
by invoking this function.
The function must import several dependencies in order to connect with Firestore and Translation.
Cloud Run starts by initializing the Firestore and Pub/Sub clients. Then, it parses the Pub/Sub message data to get the text to translate and the desired target language.
The Translation API is used to translate the string to the desired language.
The function comes up with a unique name for the translation request to make sure we don't store any duplicate translations. Then, it translates in a Firestore transaction to make sure concurrent executions don't accidentally run the same translation twice.
Building and Deploying the Cloud Run backend
Build the Cloud Run app in the
backend
directory:gcloud builds submit backend/ \ --tag gcr.io/PROJECT_ID/background-function
Deploy the Cloud Run app using the image tag from the previous step:
gcloud run deploy background-processing-function --platform managed \ --image gcr.io/PROJECT_ID/background-function --region REGION
Where
REGION
is a Google Cloud region.When your deployment finishes, you will see a URL for your deployed app in the command output. For example:
Service [background-processing-function] revision [default-00002-vav] has been deployed and is serving 100 percent of traffic at https://default-c457u4v2ma-uc.a.run.app
Copy this URL for the next step.
Setting up the Pub/Sub subscription
Your Cloud Run app will receive messages from
Pub/Sub whenever a message is published to the topic translate
.
A built-in authentication check ensures the Pub/Sub message contains a valid authorization token from a service account which has permission to invoke your Cloud Run backend.
The next steps walk you through setting up the Pub/Sub topic, subscription, and service account for making authentiated calls to your Cloud Run backend. Read more about this integration in Authenticating service-to-service.
Create the topic
translate
for publishing new translation requests:gcloud pubsub topics create translate
Enable your project to create Pub/Sub authentication tokens:
gcloud projects add-iam-policy-binding PROJECT_ID \ --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com \ --role=roles/iam.serviceAccountTokenCreator
Where
PROJECT_NUMBER
is your Google Cloud project number, which can be found by runninggcloud projects describe PROJECT_ID | grep projectNumber
.Create or select a service account to represent the Pub/Sub subscription identity.
gcloud iam service-accounts create cloud-run-pubsub-invoker \ --display-name "Cloud Run Pub/Sub Invoker"
Note: You can use
cloud-run-pubsub-invoker
or replace with a name unique within your Google Cloud project.Give the invoker service account permission to invoke your
background-processing-function
service:gcloud run services add-iam-policy-binding background-processing-function \ --member=serviceAccount:cloud-run-pubsub-invoker@PROJECT_ID.iam.gserviceaccount.com \ --role=roles/run.invoker --platform managed --region REGION
It can take several minutes for the Identity and Access Management changes to propagate. In the meantime you might see
HTTP 403
errors in the service logs.Create a Pub/Sub subscription with the service account:
gcloud pubsub subscriptions create run-translate-string --topic translate \ --push-endpoint=CLOUD_RUN_URL \ --push-auth-service-account=cloud-run-pubsub-invoker@PROJECT_ID.iam.gserviceaccount.com
Where
CLOUD_RUN_URL
is the HTTPS URL you copied after building and deploying your backend.The
--push-account-service-account
flag activates the Pub/Sub push functionality for Authentication and authorization.Your Cloud Run service domain is automatically registered for use with Pub/Sub subscriptions.
Understanding the app
There are two main components for the web app:
-
A PHP HTTP server to handle web requests. The server has the following two
endpoints:
-
/
: Lists all of the existing translations and shows a form users can submit to request new translations. -
/request-translation
: Form submissions are sent to this endpoint, which publishes the request to Pub/Sub to be translated asynchronously.
-
- An HTML template that is filled in with the existing translations by the PHP server.
The HTTP server
In the
app
directory,index.php
starts by setting up the Lumen app and registering HTTP handlers:The index handler (
/
) gets all existing translations from Firestore and renders a template with the list:The request translation handler, registered at
/request-translation
, parses the HTML form submission, validates the request, and publishes a message to Pub/Sub:
The HTML template
The HTML template is the basis for the HTML page shown to the user so they can see previous translations and request new ones. The template is filled in by the HTTP server with the list of existing translations.
-
The
<head>
element of the HTML template includes metadata, style sheets, and JavaScript for the page:The page pulls in Material Design Lite (MDL) CSS and JavaScript assets. MDL lets you add a Material Design look and feel to your websites.
The page uses JQuery to wait for the document to finish loading and set a form submission handler. Whenever the request translation form is submitted, the page does minimal form validation to check that the value isn't empty, and then sends an asynchronous request to the
/request-translation
endpoint.Finally, an MDL snackbar appears to indicate if the request was successful or if it encountered an error.
- The HTML body of the page uses an MDL layout and several MDL components to display a list of translations and a form to request additional translations:
Running the app in Cloud Shell
Before trying to deploy the web app, install the dependencies and run it locally.
First, install the dependencies with Composer. The gRPC extension for PHP is required, and is pre-installed on Cloud Shell.
composer install -d app
Next, run the PHP built-in web server to serve your app:
APP_DEBUG=true php -S localhost:8080 -t app
The
APP_DEBUG=true
flag will display any exceptions that occur.In Cloud Shell, click Web preview , and select Preview on port 8080. This opens a new window with your running app.
Deploying the web app
You can use the App Engine standard environment to build and deploy an app that runs reliably under heavy load and with large amounts of data.
This tutorial uses the App Engine standard environment to deploy the HTTP frontend.
The app.yaml
configures the App Engine app:
-
From the same directory as the
app.yaml
file, deploy your app to the App Engine standard environment:gcloud app deploy
Testing the app
After you've deployed the Cloud Run function and App Engine app, try requesting a translation.
-
To view the app in your browser,enter the following URL:
https://PROJECT_ID.REGION_ID.r.appspot.com
Replace the following:
PROJECT_ID
: Your Google Cloud project IDREGION_ID
: A code that App Engine assigns to your app
There is a page with an empty list of translations and a form to request new translations.
-
In the Text to translate field, enter some text to translate, for
example,
Hello, World
. - Select a language from the drop-down list that you want to translate the text to.
- Click Submit.
- To refresh the page, click Refresh refresh. There is a new row in the translation list. If you don't see a translation, wait a few more seconds and try again. If you still don't see a translation, see the next section about debugging the app.
Debugging the app
If you cannot connect to your App Engine app or don't see new translations, check the following:
- Check that the
gcloud
deploy commands successfully completed and didn't output any errors. If there were errors (for example,message=Build failed
), fix them, and try building and deploying the Cloud Run app and deploying the App Engine app again. In the Google Cloud console, go to the Logs Explorer page.
- In the Recently selected resources drop-down list, click GAE Application, and then click All module_id. You see a list of requests from when you visited your app. If you don't see a list of requests, confirm you selected All module_id from the drop-down list. If you see error messages printed to the Google Cloud console, check that your app's code matches the code in the section about understanding the web app.
- In the Recently selected resources drop-down list, click Cloud Run Revision, and then click All logs. You should see a POST request sent to your deployed app's URL. If not, check that the Cloud Run and App Engine app are using the same Pub/Sub topic, and that a Pub/Sub subscription exists to push to your Cloud Run endpoint.
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.
Delete the Google Cloud project
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Delete the tutorial resources
Delete the App Engine app you created in this tutorial:
- In the Google Cloud console, go to the Versions page for App Engine.
- Select the checkbox for the non-default app version that you want to delete.
- To delete the app version, click Delete.
Delete the Cloud Run service you deployed in this tutorial:
gcloud run services delete background-processing-function
You can also delete Cloud Run services from the Google Cloud console.
Delete other Google Cloud resources created in this tutorial:
- Delete the Pub/Sub topic
translate
- Delete the Pub/Sub subscription
run-translate-string
- Delete the container image named
gcr.io/PROJECT_ID/background-processing
from Container Registry. - Delete the invoker service account
cloud-run-pubsub-invoker@PROJECT_ID.iam.gserviceaccount.com
- Delete the Pub/Sub topic