Export a custom AutoML Tables model and serve it with Cloud Run
Contributed by Google employees.
AutoML Tables includes a feature with which you can export your full custom model, packaged such that you can serve it with a Docker container. This lets you serve your models anywhere that you can run a container.
This tutorial shows you how to package an exported AutoML Tables model to serve on Cloud Run. With Cloud Run, your model serving automatically scales up with traffic and scales down to 0 when it’s not being used. This tutorial also shows how you can examine your trained custom model in TensorBoard.
About the dataset and scenario
The Cloud Public Datasets Program makes available public datasets that are useful for experimenting with machine learning. Just as in Explaining model predictions on structured data, this tutorial uses data that is essentially a join of two public datasets stored in BigQuery—London bike rentals and NOAA weather data—with some additional processing to clean up outliers and derive additional GIS and day-of-week fields.
You use this dataset to build a regression model to predict the duration of a bike rental based on information about the start and end stations, the day of the week, the weather on that day, and other data. If you were running a bike rental company, for example, these predictions and their explanations could help you to anticipate demand and plan how to stock each location.
You can use AutoML Tables for tasks as varied as asset valuations, fraud detection, credit risk analysis, customer retention prediction, and analyzing item layouts in stores.
Create a dataset
The first step in training a Tables model is to create a dataset using your data. This tutorial uses the bike rentals and weather dataset described above. You can also follow along with your own tabular dataset, but in that case you need to construct your own prediction instances, too.
Go to the Tables page in the Cloud Console, and enable the API.
Create a new Tables dataset.
Import your data into the dataset:
- On the Import tab, select Import data from BigQuery.
aju-dev-demosas the BigQuery project ID,
london_bikes_weatheras the dataset ID, and
bikes_weatheras the table name.
- Click Import.
Edit the dataset’s schema
After the import is complete, you edit the dataset schema to change a few of the inferred types.
On the Train tab, make sure that your schema matches the screenshot below:
loc_crossto be of type Categorical.
durationin the Target column section.
Useful statistics are generated for the columns, including correlation statistics with the target column, which can help you determine which columns you want to use as model inputs.
Train the Tables model
Now you're ready to train a model on the dataset.
For this example, you train a model to predict ride duration given all the other dataset inputs, so you train a regression model.
Enter a training budget of 1 hour, and include all available feature columns, as shown in the following screenshot.
Export the trained model
After the model is trained, you export the result, so that it can be served from any environment in which you can run a container. Alternatively, you could deploy your model to AI Platform for online prediction.
For details about the export process, see Exporting models.
Steps in this procedure use
gsutil. To run these commands, you need
gcloud installed. You can run these commands from
the Cloud Shell instead of your local machine if you don't want to install the SDK locally.
On the Test & Use tab, under the Use your model heading, click the Container card to export your trained model to be run from a Docker container.
Browse to select the Google Cloud Storage folder into which you want to export your model, and click the Export button.
You need to use a regional Cloud Storage bucket, in the same region as your model.
Consider creating a sub-folder for the model export in the Cloud Storage bucket, so that if you have multiple exports, you can keep track of them.
When the export is finished, create a local directory (for example,
bikes_weather) to hold your model.
Copy the download command provided in the Cloud Console, which will look something like the following:
gsutil cp -r gs://[YOUR_STORAGE_BUCKET]/model_export_1//* ./download_dir
Edit this command as follows:
- Add quotation marks around the
- Remove one of the end slashes.
download_dirto point to the directory that you created.
The result should look something like the following:
gsutil cp -r 'gs://[YOUR_STORAGE_BUCKET/model_export_1/*' ./bikes_weather
- Add quotation marks around the
Run the command from the parent directory of your
The exported model is copied to
Test your exported model locally
After you've downloaded your model, you can run and test it locally. This provides a good check before deploying to Cloud Run. The process is described in detail in the AutoML Tables documentation.
Change to the
You should see a
model_exportsubdirectory, which is the result of your download.
model_export/tbl/tf_saved_model*subdirectory to remove the timestamp suffix.
Create and run a container to serve your new trained model:
docker run -v `pwd`/model-export/tbl/[YOUR_RENAMED_DIRECTORY]:/models/default/0000001 -p 8080:8080 -it gcr.io/cloud-automl-tables-public/model_server
This starts up a model server to which you can send requests. This command uses the
gcr.io/cloud-automl-tables-public/model_servercontainer image and mounts your local directory.
Download or navigate to the
instances.jsonfile, which holds data for three prediction instances for the bikes and weather model.
From the directory where you placed
instances.json, run the following command:
curl -X POST --data @instances.json http://localhost:8080/predict
You’ll get back predictions for all of the instances in the JSON file.
The actual duration for the third instance is 1200.
View information about your exported model in TensorBoard
In this section, you view your exported custom model in TensorBoard.
Viewing your exported model in TensorBoard requires a conversion step. You need to have TensorFlow 1.14 or 1.15 installed to run the the conversion script.
Download or navigate to the
convert_oss.pyscript, and copy it to the parent directory of
Create a directory for the output (for example,
Run the script:
python ./convert_oss.py --saved_model ./model-export/tbl/<your_renamed_directory>/saved_model.pb --output_dir converted_export
Point TensorBoard to the converted model:
View the exported custom Tables model in Tensorboard.
You will see a rendering of the model graph, and you can pan and zoom to view model sub-graphs in more detail.
Zoom in to see part of the model graph in more detail.
Create a Cloud Run service based on your exported model
At this point, you have a trained model that you've exported and tested locally. You are almost ready to deploy it to
Cloud Run. As the last step of preparation, you create a container image that uses
gcr.io/cloud-automl-tables-public/model_server as a base image and adds the model directory, and you push that image to the
Google Container Registry, so that Cloud Run can access it.
Build a container to use for Cloud Run
In the same
bikes_weatherdirectory that holds the
model_exportsubdirectory, create a file called
Dockerfilethat contains the following two lines, replacing
[YOUR_RENAMED_DIRECTORY]with the path to your exported model, the same path that you used in a previous step when running locally:
FROM gcr.io/cloud-automl-tables-public/model_server ADD model-export/tbl/[YOUR_RENAMED_DIRECTORY]/models/default/0000001
The template is in
Build a container from the
Dockerfile(in this example called
docker build . -t gcr.io/[YOUR_PROJECT_ID]/bw-serve
Push the container to the Google Container Registry:
docker push gcr.io/[YOUR_PROJECT_ID]/bw-serve
If you get an error, you may need to configure Docker to use
authenticate requests to Container Registry.
Alternately, you can use Cloud Build to build the container instead, as follows:
gcloud builds submit --tag gcr.io/[YOUR_PROJECT_ID]/bw-serve .
Create your Cloud Run service
Now you're ready to deploy the container to Cloud Run, where you can scalably serve it for predictions.
Go to the Cloud Run page in the Cloud Console.
Click Start using if necessary.
Click Create service.
For the container URL, enter the name of the container that you just built above.
Select Cloud Run (fully managed).
Enter a service name, which can be anything you like.
Select Require Authentication.
Click Show optional revision settings.
Change Memory allocated to 2GiB.
Leave the rest of the settings at their default values, and click Create.
Send prediction requests to the Cloud Run service
After your Cloud Run service is deployed, you can send prediction requests to it. Your new service has a URL that starts with your service name and ends
run.app. You can send JSON predictions to the Cloud Run service just as with the local server you tested earlier; but with Cloud Run, the service will
scale up and down based on demand.
Assuming that you selected the Require Authentication option, you can make prediction requests like this:
curl -X POST -H \ "Authorization: Bearer $(gcloud auth print-identity-token)" --data @./instances.json \ https:/[YOUR_SERVICE_URL]/predict
It may take a second or two for the first request to return, but subsequent requests will be faster.
If you set up your Cloud Run service endpoint so that it does not require authentication, you don’t need to include the authorization header in your
In this tutorial, you saw how to export a custom AutoML Tables trained model, view model information in TensorBoard, and build a container image that lets you serve the model from any environment. Then you saw how you can deploy that image to Cloud Run for scalable serving. See the Cloud Run documentation for more information on how to configure your prediction endpoint for end-user or service-to-service authentication.
Once you’ve built a model-serving container image, you can deploy it to other environments as well. For example, if you have installed
Knative serving on a Kubernetes cluster, you can create a Knative service like this, using the
same container image (replacing
[YOUR_PROJECT_ID] with your project ID):
apiVersion: serving.knative.dev/v1 kind: Service metadata: name: bikes-weather spec: template: spec: containers: - image: gcr.io/[YOUR_PROJECT_ID]/bw-serve
Though the example model for this tutorial fits on a 2-GiB Cloud Run instance, you might have models that are too large for the managed Cloud Run service, and serving it with Kubernetes/GKE is a good alternative.
If you’re curious about the details of your custom model, you can use Cloud Logging to view information about your AutoML Tables model. Using Cloud Logging, you can see the final model hyperparameters and the hyperparameters and object values used during model training and tuning.
You may also be interested in exploring the updated AutoML Tables client libraries, which make it easy for you to train and use Tables programmatically, or reading about how to create a contextual bandit model pipeline using AutoML Tables, without needing a specialist for tuning or feature engineering.