Image Classification using Flowers dataset

This tutorial uses the Flowers dataset to build a customized image classification model via transfer learning and the existent Inception-v3 model in order to correctly label different types of flowers using Cloud Machine Learning Engine.

The sample code you will walk through and results you will monitor consist of four parts: data preprocessing, model training with the transformed data, model deployment, and prediction request steps. All parts will be completed in the cloud.

What you will build

You will run sample code in order to preprocess data with Cloud Dataflow and then use that transformed data to train a model with Cloud ML Engine. You will then deploy the trained model to Cloud ML Engine and test the model by sending a prediction request to it.

In this sample dataset you only have a small set of images (~3,600). Without more data it isn’t possible to use machine learning techniques to adequately train an accurate classification model from scratch. Instead, you’ll use an approach called transfer learning. In transfer learning you use a pre-trained model to extract image features that you will use to train a new classifier. In this tutorial in particular you’ll use a pre-trained model called Inception.

Objectives

In this introductory, end-to-end walkthrough you will use python code to:

  • Perform data preprocessing in the cloud, reading original data files and using Cloud Dataflow to convert them into TFRecord format for training.
  • Run training using Cloud Machine Learning Engine to obtain optimal model specifications.
  • Deploy the trained model.
  • Request a prediction from the trained model and observe the resulting accuracy.

Costs

This walkthrough uses billable components of Google Cloud Platform, including:

  • Cloud Dataflow for:
    • Preprocessing data
  • Cloud Machine Learning Engine for:
    • Training
    • Making a prediction request
  • Google Cloud Storage for:
    • Storing input data for training
    • Staging the trainer package
    • Writing training artifacts

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Cloud Platform users might be eligible for a free trial.

Before you begin

Download the sample from the GitHub repository

macOS

  1. Download and extract the Cloud ML Engine sample zip file.

  2. Open a terminal window and navigate to the directory that contains the extracted cloudml-samples-master directory.

  3. Navigate to the cloudml-samples-master > flowers directory. The commands in this walkthrough must be run from the flowers directory.

    cd cloudml-samples-master/flowers
    

Cloud Shell

  1. Download the Cloud ML Engine sample zip file.

    wget https://github.com/GoogleCloudPlatform/cloudml-samples/archive/master.zip
    
  2. Unzip the file to extract the cloudml-samples-master directory.

    unzip master.zip
    
  3. Navigate to the cloudml-samples-master > flowers directory. The commands in this walkthrough must be run from the flowers directory.

    cd cloudml-samples-master/flowers
    

Install dependencies

The sample provides a requirements.txt file that you can use to install the dependencies required by the project.

pip install --user -r requirements.txt

Set up and test your Cloud environment

Complete the following steps to set up a GCP account, activate the Cloud ML Engine API, and install and activate the Cloud SDK.

Set up your GCP project

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a GCP project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your project.

    Learn how to enable billing

  4. Enable the Cloud Machine Learning Engine and Compute Engine APIs.

    Enable the APIs

  5. Set up authentication:
    1. In the GCP Console, go to the Create service account key page.

      Go to the Create Service Account Key page
    2. From the Service account drop-down list, select New service account.
    3. In the Service account name field, enter a name .
    4. From the Role drop-down list, select Project > Owner.

      Note: The Role field authorizes your service account to access resources. You can view and change this field later by using GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
    5. Click Create. A JSON file that contains your key downloads to your computer.
  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

  7. Install and initialize the Cloud SDK.

Set up your environment

Choose one of the options below to set up your environment locally on macOS or in a remote environment on Cloud Shell.

For macOS users, we recommend that you set up your environment using the MACOS tab below. Cloud Shell, shown on the CLOUD SHELL tab, is available on macOS, Linux, and Windows. Cloud Shell provides a quick way to try Cloud Machine Learning Engine, but isn’t suitable for ongoing development work.

macOS

  1. Check Python installation
    Confirm that you have Python installed and, if necessary, install it.

    python -V
  2. Check pip installation
    pip is Python’s package manager, included with current versions of Python. Check if you already have pip installed by running pip --version. If not, see how to install pip.

    You can upgrade pip using the following command:

    pip install -U pip

    See the pip documentation for more details.

  3. Install virtualenv
    virtualenv is a tool to create isolated Python environments. Check if you already have virtualenv installed by running virtualenv --version. If not, install virtualenv:

    pip install --user --upgrade virtualenv

    To create an isolated development environment for this guide, create a new virtual environment in virtualenv. For example, the following command activates an environment named cmle-env:

    virtualenv cmle-env
    source cmle-env/bin/activate
  4. For the purposes of this tutorial, run the rest of the commands within your virtual environment.

    See more information about using virtualenv. To exit virtualenv, run deactivate.

Cloud Shell

  1. Open the Google Cloud Platform Console.

    Google Cloud Platform Console

  2. Click the Activate Google Cloud Shell button at the top of the console window.

    Activate Google Cloud Shell

    A Cloud Shell session opens inside a new frame at the bottom of the console and displays a command-line prompt. It can take a few seconds for the shell session to be initialized.

    Cloud Shell session

    Your Cloud Shell session is ready to use.

  3. Configure the gcloud command-line tool to use your selected project.

    gcloud config set project [selected-project-id]

    where [selected-project-id] is your project ID. (Omit the enclosing brackets.)

Verify the Google Cloud SDK components

To verify that the Google Cloud SDK components are installed:

  1. List your models:

    gcloud ml-engine models list
  2. If you have not created any models before, the command returns an empty list:

    Listed 0 items.

    After you start creating models, you can see them listed by using this command.

  3. If you have installed gcloud previously, update gcloud:

    gcloud components update

Install TensorFlow

To install TensorFlow, run the following command:

pip install --user --upgrade tensorflow

For more information about installing TensorFlow, see the TensorFlow documentation.

To troubleshoot your TensorFlow installation, see TensorFlow's guide to common installation problems.

Run a simple TensorFlow Python program

Run a simple Python program to test your installation of TensorFlow. If you are using Cloud Shell, note that TensorFlow is already installed.

  1. Start a Python interactive shell.

    python
    
  2. Import TensorFlow.

    >>> import tensorflow as tf

  3. Create a constant that contains a string.

    >>> hello = tf.constant('Hello, TensorFlow!')

  4. Create a TensorFlow session.

    >>> sess = tf.Session()

    You can ignore the warnings that the TensorFlow library wasn't compiled to use certain instructions.

  5. Display the value of hello.

    >>> print(sess.run(hello))

    If successful, the system outputs:

    Hello, TensorFlow!

  6. Stop the Python interactive shell.

    >>> exit()

Set up your Cloud Storage bucket

This section shows you how to create a new bucket. You can use an existing bucket, but if it is not part of the project you are using to run Cloud ML Engine, you must explicitly grant access to the Cloud ML Engine service accounts.

  1. Specify a name for your new bucket. The name must be unique across all buckets in Cloud Storage.

    BUCKET_NAME="your_bucket_name"

    For example, use your project name with -mlengine appended:

    PROJECT_ID=$(gcloud config list project --format "value(core.project)")
    BUCKET_NAME=${PROJECT_ID}-mlengine
  2. Check the bucket name that you created.

    echo $BUCKET_NAME
  3. Select a region for your bucket and set a `REGION` environment variable.

    Warning: You must specify a region (like us-central1) for your bucket, not a multi-region location (like us). See the available regions for Cloud ML Engine services. For example, the following code creates REGION and sets it to us-central1.

    REGION=us-central1
  4. Create the new bucket:

    gsutil mb -l $REGION gs://$BUCKET_NAME

    Note: Use the same region where you plan on running Cloud ML Engine jobs. The example uses us-central1 because that is the region used in the getting-started instructions.

Declaring variables

Start by declaring all variables and making them read-only.

 declare -r BUCKET_NAME="gs://${your_bucket_name}"
 declare -r REGION=“your_valid_region”
 declare -r PROJECT_ID=$(gcloud config list project --format "value(core.project)")
 declare -r JOB_NAME="flowers_${USER}_$(date +%Y%m%d_%H%M%S)"
 declare -r GCS_PATH="${BUCKET_NAME}/${USER}/${JOB_NAME}"
 declare -r DICT_FILE=gs://cloud-ml-data/img/flower_photos/dict.txt

 declare -r MODEL_NAME=flowers
 declare -r VERSION_NAME=v1

 echo
 echo "Using job id: " $JOB_NAME
 set -v -e

Note that the BUCKET_NAME and REGION variables are user specific, so you must explicitly fill in the name of your own project's bucket and region. For help with choosing a region, see the guide to available regions for Cloud ML Engine services.

Preprocessing training and evaluation data in the cloud

For the sake of this tutorial the original dataset of labeled flower images has been randomly split into training and evaluation datasets. Of the original data, 90% is reserved for training and 10% is reserved for evaluation. You start with these two separate files which are stored in a Google-owned Cloud Storage bucket. You then preprocess these two Google-hosted datasets to extract the image features from the bottleneck layer (typically the penultimate layer just before the final output layer that actually does the classification) of the Inception network. You save the output of this preprocessing in your own Cloud Storage bucket and use these files for training.

Preprocessing takes about 60 minutes per dataset to complete.

Evaluation data preprocessing

Start with the evaluation data preprocessing.

python trainer/preprocess.py \
    --input_dict "$DICT_FILE" \
    --input_path "gs://cloud-ml-data/img/flower_photos/eval_set.csv" \
    --output_path "${GCS_PATH}/preproc/eval" \
    --cloud

Monitor preprocessing

To monitor preprocessing progress you can either use the command line tool for text log entries or Dataflow's web-based monitoring user interface in the console which provides a graphical representation of each pipeline and details about your job's status and execution.

Print log entries with the following commands:

gcloud dataflow jobs list
export JOB_ID="{corresponding_id}"
gcloud beta dataflow logs list $JOB_ID --importance=detailed

Or navigate to Dataflow's monitoring user interface:

Cloud Dataflow Monitoring UI

Training data preprocessing

Preprocess training data in the same manner as the evaluation data.

python trainer/preprocess.py \
    --input_dict "$DICT_FILE" \
    --input_path "gs://cloud-ml-data/img/flower_photos/train_set.csv" \
    --output_path "${GCS_PATH}/preproc/train" \
    --cloud

Monitoring progress is the same as the evaluation data.

Run model training in the cloud

Once preprocessing on Cloud ML Engine has fully completed you are ready to train a simple classifier. Verify that preprocessing has finished by navigating to the Dataflow console page and checking the job status.

Cloud Dataflow Monitoring UI

Run model training in the cloud with the following command:

gcloud ml-engine jobs submit training "$JOB_NAME" \
    --stream-logs \
    --module-name trainer.task \
    --package-path trainer \
    --staging-bucket "$BUCKET_NAME" \
    --region "$REGION" \
    --runtime-version=1.4\
    -- \
    --output_path "${GCS_PATH}/training" \
    --eval_data_paths "${GCS_PATH}/preproc/eval*" \
    --train_data_paths "${GCS_PATH}/preproc/train*"

Monitor training progress

Similar to preprocessing monitoring, you can monitor training progress via simple text logs in the command line or by launching TensorBoard visualization tool and going to the summary logs produced during training.

Stream logs with the command line:

gcloud ml-engine jobs stream-logs "$JOB_NAME"

Or open TensorBoard visualization tool:

macOS

  1. Launch TensorBoard:

    OUTPUT_PATH = "${GCS_PATH}/training"
    
    python -m tensorflow.tensorboard --logdir=$OUTPUT_PATH
    
  2. Once you start running TensorBoard, you can access it in your browser at http://localhost:6006

Cloud Shell

  1. Launch TensorBoard:

    OUTPUT_PATH = "${GCS_PATH}/training"
    
    python -m tensorflow.tensorboard --logdir=$OUTPUT_PATH --port=8080
    
  2. Select "Preview on port 8080" from the Web Preview menu at the top of the command line.

You can shut down TensorBoard at any time by typing ctrl+c on the command line.

Deploying and using the model for prediction

You now have a SavedModel stored on Google Cloud Storage ready to be exported. Think of the "model" that is being created with this command as a container for the optimal computation artifact (Tensorflow graphs) that were created during training.

  1. Use the gcloud ml-engine models create command to name and export the SavedModel to Cloud ML Engine.

    gcloud ml-engine models create "$MODEL_NAME" \
      --regions "$REGION"
    
  2. Create the first version of the model. Creating a version actually deploys your uniquely defined model to a Cloud instance, and gets it ready to serve (predict).

    gcloud ml-engine versions create "$VERSION_NAME" \
      --model "$MODEL_NAME" \
      --origin "${GCS_PATH}/training/model" \
      --runtime-version=1.4
    

Cloud ML Engine automatically sets the first version of your model to the default.

Send a prediction request to the trained model

After you deploy your model you are able to test its online prediction capability. You do this by downloading an example flower (daisy) image from the eval set and then send a request message.

  1. Copy the image to your local disk.

      gsutil cp \
        gs://cloud-ml-data/img/flower_photos/daisy/100080576_f52e8ee070_n.jpg \
        daisy.jpg
    

    In order to make a prediction request the above JPEG file must be encoded (using base64 encoding) in a JSON string.

  2. Create a request message locally in JSON format from the downloaded daisy JPEG file.

      python -c 'import base64, sys, json; img = base64.b64encode(open(sys.argv[1], "rb").read()); print json.dumps({"key":"0", "image_bytes": {"b64": img}})' daisy.jpg &> request.json
    

    The above code is Cloud ML Engine online prediction in action.

    If you are deploying a new version of your model you might have to wait for your request to be fulfilled. This could take up to 10 minutes. To check the status of your deployed version navigate to ML Engine > Models on Google Cloud Platform Console.

    After a few minutes, the model will become available for use via Cloud ML Engine’s prediction API. When it does use the predict command to send a prediction request to Cloud ML Engine for the given instance.

  3. Call the prediction service API to get classifications.

      gcloud ml-engine predict --model ${MODEL_NAME} --json-instances request.json
    

    The response should look something like this, where prediction number corresponds to flower type (daisy - 0, dandelion - 1, roses - 2, sunflowers - 3, tulips - 4):

KEY  PREDICTION                SCORES
0      0                    [0.9980067610740662, 0.00011650333908619359,
0.00028453863342292607, 0.0006193328299559653, 0.0009433324448764324,
2.9501752578653395e-05]

The prediction accuracy after running this code should be approximately 99.8%.

Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

To delete the project:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete project. After selecting the checkbox next to the project name, click
      Delete project
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

  1. Open a terminal window (if not already open).

  2. Use the gsutil rm command with the -r flag to delete the directory that contains your most recent job:

    gsutil rm -r gs://$BUCKET_NAME/$JOB_NAME
    

If successful, the command returns a message similar to:

Removing gs://my-awesome-bucket/just-a-folder/cloud-storage.logo.png#1456530077282000...
Removing gs://my-awesome-bucket/...

Repeat the command for any other directories that you created for this sample.

Alternately, if you have no other data stored in the bucket, you can run the gsutil rm -r command on the bucket itself.

What's next

You've now completed a walkthrough of a Cloud ML Engine code sample that uses flower image data for preprocessing, training, deployment and prediction. You ran code to preprocess the image data using Cloud Dataflow, performed training in the cloud, then deployed a model and used it to get an online prediction.

The following resources can help you continue learning about Cloud ML Engine.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud ML Engine for TensorFlow