Creating an Object Detection Application Using TensorFlow

This tutorial describes how to install and run an object detection application. The application uses TensorFlow and other public API libraries to detect multiple objects in an uploaded image.

Objectives

This is a basic tutorial designed to familiarize you with TensorFlow applications. When you are finished, you should be able to:

• Create a virtual machine (VM) using Compute Engine.
• Install the Object Detection API library.
• Install and launch an object detection web application.
• Test the web application with uploaded images.

Costs

This tutorial uses billable components of Google Cloud, including:

• Compute Engine
• Persistent Disk

The estimated price to run this tutorial, assuming you use every resource for an entire day, is approximately $1.36 based on this pricing calculator.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

TensorFlow architecture overview

The object detection application uses the following components:

• TensorFlow. An open source machine learning library developed by researchers and engineers within Google's Machine Intelligence research organization. TensorFlow runs on multiple computers to distribute the training workloads.

• Object Detection API. An open source framework built on top of TensorFlow that makes it easy to construct, train, and deploy object detection models.

• Pre-trained object detection models. The Object Detection API provides pre-trained object detection models for users running inference jobs. Users are not required to train models from scratch.

Local implementation

The following diagram shows how this tutorial is implemented. The web application is deployed to a VM instance running on Compute Engine.

When the client uploads an image to the application, the application runs the inference job locally. The pre-trained model returns the labels of detected objects, and the image coordinates of the corresponding objects. Using these values, the application generates new images populated with rectangles around the detected objects. Separate images are generated for each object category, allowing the client to discriminate between selected objects.

Remote implementation

You can deploy the pre-trained model on AI Platform to provide an API service for inference. If you do, the web application sends an API request to detect objects in the uploaded image, instead of running the inference job locally.

TensorFlow allows you to choose which platform to execute inference jobs on depending on your business needs. This flexibility shows the advantage of Google Cloud Platform and TensorFlow as an open platform for machine learning.

Pre-trained models

You can use the pre-trained models with the Object Detection API. They are trained with the COCO dataset and are capable of detecting general objects in 80 categories.

The COCO mAP column shows the model's accuracy index. Higher numbers indicate better accuracy. As speed increases, accuracy decreases.

Model name	Speed	COCO mAP
Download ssd_mobilenet_v1_coco	fast	21
Download ssd_inception_v2_coco	fast	24
Download rfcn_resnet101_coco	medium	30
Download faster_rcnn_resnet101_coco	medium	32
Download faster_rcnn_inception_resnet_v2_atrous_coco	slow	37

Launch a VM instance

1. In the Cloud Console, go to the VM instances page.

   Go to VM instances

2. Click Create instance.
3. Set Machine type to 8 vCPUs.
4. Click the Customize link next in the Machine type section.
5. In the Memory section, replace 30 with 8.
6. In the Firewall section, select Allow HTTP traffic.
7. Click the Management, security, disks, networking, sole tenancy link, then click the Networking tab.
8. In the Network interfaces section, next to the default row, click Edit.
9. Select Create IP address from the External IP drop-down list to assign a static IP address. In the Name field, enter staticip,and then click Reserve.
10. Click Create to create the instance.

SSH into the instance

1. Next to the instance name, click SSH.

2. Enter the following command to switch to the root user:

   sudo -i
   

Install the Object Detection API library

1. Install the prerequisite packages.

   apt-get update

   apt-get install -y protobuf-compiler python3-pil python3-lxml python3-pip python3-dev git

   pip3 install -U pip

   python3 -m pip install Flask==1.1.1 WTForms==2.2.1 Flask_WTF==0.14.2 Werkzeug==0.16.0 tensorflow==2.0.0

2. Install the Object Detection API library.

   cd /opt

   git clone https://github.com/tensorflow/models

   cd models/research

   protoc object_detection/protos/*.proto --python_out=.

Install and launch the web application

1. Install the application.

   cd $HOME
   git clone https://github.com/GoogleCloudPlatform/tensorflow-object-detection-example
   cp -a tensorflow-object-detection-example/object_detection_app_p3 /opt/
   chmod u+x /opt/object_detection_app_p3/app.py
   cp /opt/object_detection_app_p3/object-detection.service /etc/systemd/system/
   

2. The application provides a simple user authentication mechanism. You can change the username and password by modifying the /opt/object_detection_app_p3/decorator.py file.

   USERNAME = 'username'
   PASSWORD = 'passw0rd'

3. Launch the application.

   systemctl daemon-reload
   systemctl enable object-detection
   systemctl start object-detection
   systemctl status object-detection
   

   The last command outputs the application status, as in the following example:

   ● object-detection.service - Object Detection API Demo
      Loaded: loaded (/etc/systemd/system/object-detection.service; enabled; vendor preset: enabled)
      Active: active (running) since Thu 2020-07-02 23:55:17 UTC; 38s ago
    Main PID: 17136 (python3)
       Tasks: 29 (limit: 4915)
      CGroup: /system.slice/object-detection.service
              └─17136 python3 /opt/object_detection_app_p3/app.py
   
   Jul 02 23:55:32 od-test app.py[17136]: 2020-07-02 23:55:32.930129: I tensorflow/core/platform/cpu_feature_guard.cc:142] You
   Jul 02 23:55:32 od-test app.py[17136]: 2020-07-02 23:55:32.936310: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94
   Jul 02 23:55:32 od-test app.py[17136]: 2020-07-02 23:55:32.937050: I tensorflow/compiler/xla/service/service.cc:168] XLA se
   Jul 02 23:55:32 od-test app.py[17136]: 2020-07-02 23:55:32.937078: I tensorflow/compiler/xla/service/service.cc:175]   Stre
   Jul 02 23:55:40 od-test app.py[17136]:  * Serving Flask app "app" (lazy loading)
   Jul 02 23:55:40 od-test app.py[17136]:  * Environment: production
   Jul 02 23:55:40 od-test app.py[17136]:    WARNING: This is a development server. Do not use it in a production deployment.
   Jul 02 23:55:40 od-test app.py[17136]:    Use a production WSGI server instead.
   Jul 02 23:55:40 od-test app.py[17136]:  * Debug mode: off
   Jul 02 23:55:40 od-test app.py[17136]:  * Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
   

   The application loads the model binary immediately after launch. It will take a minute to start serving requests from clients. You'll see the message Running on http://0.0.0.0:80/ (Press CTRL+C to quit) when it's ready.

Test the web application

1. Using a web browser, access the static IP address that was assigned when you launched the VM instance.

2. Enter the username and password that you configured during the application installation process. The default username is username, and the default password is passw0rd.

3. Upload an image file with a JPEG, JPG, or PNG extension. The application shows the result of the object detection inference. Depending on the size of the image, it might take up to 30 seconds to upload the image.

   

   The object names detected by the model are shown in the application window.

4. Click an object name to display rectangles surrounding the corresponding objects in the image. The rectangle thickness increases with object identification confidence. In the preceding image, a fork, cup, dining table, person, and knife, are detected.

5. Click Cup. Rectangles display around all detected cups in the image.

6. Click Original to see the original image.

Test this model's accuracy by uploading images that contain different types of objects.

Change the inference model

The application can use pretrained models. They have different characteristics in terms of accuracy and speed.

1. Choose one of COCO-trained models from the Tensorflow detection model zoo. (The Outputs column should be Boxes.)

2. Copy the URL of the model from a link on the Model name column.

3. Open /opt/object_detection_app_p3/app.py and replace the URL in the following line with the URL that you copied:

   MODEL_URL = 'http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet50_coco_2018_01_28.tar.gz'

4. Restart the application:

   systemctl restart object-detection
   

Cleaning up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Caution: Deleting a project has the following effects:
• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.
1. In the Cloud Console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

• Learn how to use AI Platform to train your model to make predictions on your own dataset.

• Learn how to conduct transfer learning using your own dataset with the Object Detection API.

• Try out other Google Cloud features for yourself. Have a look at our tutorials.