Community Tutorials

Export a custom AutoML Tables model and serve it with Cloud Run

Author(s): @amygdala , Published: 2020-08-18

Amy Unruh | Developer Advocate | Google

Contributed by Google employees.

With AutoML Tables, you can automatically build and deploy state-of-the-art machine-learning models using your own structured data. See this notebook for a walkthrough.

AutoML Tables includes a feature with which you can export your full custom model, packaged such that you can serve it with a Docker container. This lets you serve your models anywhere that you can run a container.

This tutorial shows you how to package an exported AutoML Tables model to serve on Cloud Run. With Cloud Run, your model serving automatically scales up with traffic and scales down to 0 when it’s not being used. This tutorial also shows how you can examine your trained custom model in TensorBoard.

This tutorial uses the Cloud Console, but you could also accomplish the same steps through the command-line interface or using the AutoML Tables client libraries.

Note: This tutorial applies to the AutoML Tables service as accessed here: https://console.cloud.google.com/automl-tables/. Export of the (Preview) AutoML Tabular models requires a slightly different process. We intend to update this tutorial soon to include both.

About the dataset and scenario

The Cloud Public Datasets Program makes available public datasets that are useful for experimenting with machine learning. Just as in Explaining model predictions on structured data, this tutorial uses data that is essentially a join of two public datasets stored in BigQuery—London bike rentals and NOAA weather data—with some additional processing to clean up outliers and derive additional GIS and day-of-week fields.

You use this dataset to build a regression model to predict the duration of a bike rental based on information about the start and end stations, the day of the week, the weather on that day, and other data. If you were running a bike rental company, for example, these predictions and their explanations could help you to anticipate demand and plan how to stock each location.

You can use AutoML Tables for tasks as varied as asset valuations, fraud detection, credit risk analysis, customer retention prediction, and analyzing item layouts in stores.

Create a dataset

The first step in training a Tables model is to create a dataset using your data. This tutorial uses the bike rentals and weather dataset described above. You can also follow along with your own tabular dataset, but in that case you need to construct your own prediction instances, too.

Go to the Tables page in the Cloud Console, and enable the API.
Create a new Tables dataset.
Import your data into the dataset:
1. On the Import tab, select Import data from BigQuery.
2. Enter aju-dev-demos as the BigQuery project ID, london_bikes_weather as the dataset ID, and bikes_weather as the table name.
3. Click Import.

Edit the dataset’s schema

After the import is complete, you edit the dataset schema to change a few of the inferred types.

On the Train tab, make sure that your schema matches the screenshot below:

Change bike_id, end_station_id, start_station_id, and loc_cross to be of type Categorical.
Select duration in the Target column section.

Adjust the dataset schema

Useful statistics are generated for the columns, including correlation statistics with the target column, which can help you determine which columns you want to use as model inputs.

Train the Tables model

Now you're ready to train a model on the dataset.

For this example, you train a model to predict ride duration given all the other dataset inputs, so you train a regression model.

Enter a training budget of 1 hour, and include all available feature columns, as shown in the following screenshot.

Train a model to predict ride

Export the trained model

After the model is trained, you export the result, so that it can be served from any environment in which you can run a container. Alternatively, you could deploy your model to AI Platform for online prediction.

For details about the export process, see Exporting models.

Steps in this procedure use gsutil. To run these commands, you need gcloud installed. You can run these commands from the Cloud Shell instead of your local machine if you don't want to install the SDK locally.

On the Test & Use tab, under the Use your model heading, click the Container card to export your trained model to be run from a Docker container.
Browse to select the Google Cloud Storage folder into which you want to export your model, and click the Export button.

You need to use a regional Cloud Storage bucket, in the same region as your model.

Consider creating a sub-folder for the model export in the Cloud Storage bucket, so that if you have multiple exports, you can keep track of them.
When the export is finished, create a local directory (for example, bikes_weather) to hold your model.
Copy the download command provided in the Cloud Console, which will look something like the following:

gsutil cp -r gs://[YOUR_STORAGE_BUCKET]/model_export_1//* ./download_dir
Edit this command as follows:
1. Add quotation marks around the gs URI.
2. Remove one of the end slashes.
3. Edit download_dir to point to the directory that you created.
The result should look something like the following:
```
gsutil cp -r 'gs://[YOUR_STORAGE_BUCKET/model_export_1/*' ./bikes_weather
```
Run the command from the parent directory of your bikes_weather directory.

The exported model is copied to ./bikes_weather.

Test your exported model locally

After you've downloaded your model, you can run and test it locally. This provides a good check before deploying to Cloud Run. The process is described in detail in the AutoML Tables documentation.

Change to the bikes_weather directory.

You should see a model_export subdirectory, which is the result of your download.
Rename the model_export/tbl/tf_saved_model* subdirectory to remove the timestamp suffix.
Create and run a container to serve your new trained model:
```
docker run -v `pwd`/model-export/tbl/[YOUR_RENAMED_DIRECTORY]:/models/default/0000001 -p 8080:8080 -it gcr.io/cloud-automl-tables-public/model_server
```
This starts up a model server to which you can send requests. This command uses the gcr.io/cloud-automl-tables-public/model_server container image and mounts your local directory.
Download or navigate to the instances.json file, which holds data for three prediction instances for the bikes and weather model.
From the directory where you placed instances.json, run the following command:
```
curl -X POST --data @instances.json http://localhost:8080/predict
```
You’ll get back predictions for all of the instances in the JSON file.

The actual duration for the third instance is 1200.

View information about your exported model in TensorBoard

In this section, you view your exported custom model in TensorBoard.

Viewing your exported model in TensorBoard requires a conversion step. You need to have TensorFlow 1.14 or 1.15 installed to run the the conversion script.

Download or navigate to the convert_oss.py script, and copy it to the parent directory of model_export.
Create a directory for the output (for example, converted_export):
```
mkdir converted_export
```

Run the script:

python ./convert_oss.py --saved_model ./model-export/tbl/<your_renamed_directory>/saved_model.pb --output_dir converted_export

Point TensorBoard to the converted model:
```
tensorboard --logdir=converted_export
```
View the exported custom Tables model in Tensorboard.

You will see a rendering of the model graph, and you can pan and zoom to view model sub-graphs in more detail.

Zoom in to see part of the model graph in more detail.

Create a Cloud Run service based on your exported model

Note: Currently, this part of the tutorial doesn't work properly because of a change in the model_server base image, though you can still use your created container image locally. We intend to update this tutorial soon with a fix.

At this point, you have a trained model that you've exported and tested locally. You are almost ready to deploy it to Cloud Run. As the last step of preparation, you create a container image that uses gcr.io/cloud-automl-tables-public/model_server as a base image and adds the model directory, and you push that image to the Google Container Registry, so that Cloud Run can access it.

Build a container to use for Cloud Run

In the same bikes_weather directory that holds the model_export subdirectory, create a file called Dockerfile that contains the following two lines, replacing [YOUR_RENAMED_DIRECTORY] with the path to your exported model, the same path that you used in a previous step when running locally:
```
FROM gcr.io/cloud-automl-tables-public/model_server

ADD model-export/tbl/[YOUR_RENAMED_DIRECTORY] /models/default/0000001
```
The template is in Dockerfile.template, too.
Build a container from the Dockerfile (in this example called bw-serve):
```
docker build . -t gcr.io/[YOUR_PROJECT_ID]/bw-serve
```
Push the container to the Google Container Registry:
```
docker push gcr.io/[YOUR_PROJECT_ID]/bw-serve
```

If you get an error, you may need to configure Docker to use gcloud to authenticate requests to Container Registry.

Alternately, you can use Cloud Build to build the container instead, as follows:

gcloud builds submit --tag gcr.io/[YOUR_PROJECT_ID]/bw-serve .

Create your Cloud Run service

Now you're ready to deploy the container to Cloud Run, where you can scalably serve it for predictions.

Go to the Cloud Run page in the Cloud Console.
Click Start using if necessary.
Click Create service.
For the container URL, enter the name of the container that you just built above.
Select Cloud Run (fully managed).
Enter a service name, which can be anything you like.
Select Require Authentication.
Click Show optional revision settings.
Change Memory allocated to 2GiB.
Leave the rest of the settings at their default values, and click Create.

Send prediction requests to the Cloud Run service

After your Cloud Run service is deployed, you can send prediction requests to it. Your new service has a URL that starts with your service name and ends with run.app. You can send JSON predictions to the Cloud Run service just as with the local server you tested earlier; but with Cloud Run, the service will scale up and down based on demand.

Assuming that you selected the Require Authentication option, you can make prediction requests like this:

curl -X POST -H \
"Authorization: Bearer $(gcloud auth print-identity-token)" --data @./instances.json \
https:/[YOUR_SERVICE_URL]/predict

It may take a second or two for the first request to return, but subsequent requests will be faster.

If you set up your Cloud Run service endpoint so that it does not require authentication, you don’t need to include the authorization header in your curl request.

What’s next?

In this tutorial, you saw how to export a custom AutoML Tables trained model, view model information in TensorBoard, and build a container image that lets you serve the model from any environment. Then you saw how you can deploy that image to Cloud Run for scalable serving. See the Cloud Run documentation for more information on how to configure your prediction endpoint for end-user or service-to-service authentication.

Once you’ve built a model-serving container image, you can deploy it to other environments as well. For example, if you have installed Knative serving on a Kubernetes cluster, you can create a Knative service like this, using the same container image (replacing [YOUR_PROJECT_ID] with your project ID):

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: bikes-weather
spec:
  template:
    spec:
      containers:
        - image: gcr.io/[YOUR_PROJECT_ID]/bw-serve

Though the example model for this tutorial fits on a 2-GiB Cloud Run instance, you might have models that are too large for the managed Cloud Run service, and serving it with Kubernetes/GKE is a good alternative.

If you’re curious about the details of your custom model, you can use Cloud Logging to view information about your AutoML Tables model. Using Cloud Logging, you can see the final model hyperparameters and the hyperparameters and object values used during model training and tuning.

You may also be interested in exploring the updated AutoML Tables client libraries, which make it easy for you to train and use Tables programmatically, or reading about how to create a contextual bandit model pipeline using AutoML Tables, without needing a specialist for tuning or feature engineering.

Submit a tutorial

Share step-by-step guides

Submit a tutorial

Request a tutorial

Ask for community help

Submit a request

View tutorials

Search Google Cloud tutorials

View tutorials

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.