Jump to Content
Google Cloud

How Aucnet leveraged TensorFlow to transform their IT engineers into machine learning engineers

August 17, 2017
佐藤一憲

Google Cloud デベロッパーアドボケイト

Using deep learning is not rocket science

Many businesses believe that adopting deep learning or machine learning (ML) techniques means hiring a fleet of cutting edge data scientists with PhDs. But this is simply not true. Creating new deep learning models and theories is hard, but using the existing, popular deep learning models is actually not rocket science. In fact, a typical IT engineer can learn the basics of ML, including how to integrate and use the well-known ML and deep learning algorithms and techniques, in order to build a machine learning solution. In short, a company’s IT engineers can be trained to become ML engineers.

One company that did exactly that is Japanese car auction service Aucnet. Aucnet is one of the largest real-time auction service providers in the world, handling four million auctions every year. But one of their pain points was image classification. Entering a single car to auction required uploading 20 photos from various angles with labels like "front view," "side view," "tire," "handle," "seat" and so on. This time consuming task could take up to 20 minutes per car.

But Aucnet had an idea. By training their existing IT engineers in the basics of ML, they were able to build Konpeki (click the link to try the demo yourself), a real-time car image recognition system, powered by TensorFlow. By integrating various deep learning technologies, ML APIs and Google Cloud services, they shortened the amount of time it took to list a car auction to just a few minutes, down from 20. Also, Aucnet was able to apply Cloud Machine Learning Engine to increase the speed of ML training process by 86 times.

If you’re interested in experimenting with Konpeki, try visiting the site and loading a set of car images. Here’s a little more on how Aucnet built Konpeki.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-1wy9d.max-500x500.PNG

Konpeki, a car image recognition system for used car auctions

Deep learning turns Aucnet's enterprise assets into a golden opportunity

When Kuroyanagi, General Manager of Aucnet IBS Inc., heard of AlphaGo, the AI computer program that beat the world’s number one-ranked Go player, he wondered if deep learning technology could be the solution to Aucnet’s image classification problem. It seemed to him that the technology could turn their enormous collection of high quality used car images (12 million images each year) and metadata into a valuable asset for the company.

Kuroyanagi started the Konpeki project in January 2016, building a PoC system by combining TensorFlow, Cloud Vision API and Compute Engine.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-43t5s.max-400x400.PNG

Kuroyanagi (2nd from left) and his project members

Partnering with the TensorFlow community and ecosystem

But when Aucnet started the project, no one on their team had experience with data analytics, machine learning or deep learning. So they began searching for a partner with deep knowledge and experience to provide a production ML solution.

To find this partner, they turned to the developer community. In Japan, the TensorFlow User Group (TFUG) has been active since 2016, with 2800+ members. TFUG sponsors several different sub-meetups that focus on various topics, such as business ML use cases, technical deep dives, hardware acceleration and embedded systems and more. At these meetings, TensorFlow users get together to share use cases and tips and tricks.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-3f31u.max-400x400.PNG

TensorFlow User Group has been actively hosting many different meetups

Through TFUG and Google, Aucnet discovered BrainPad, a leading company in the Japanese TensorFlow community. Together, BrainPad and Aucnet spent several weeks conducting TensorFlow workshops, discussing design and building the PoC system.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-6ec7w.max-500x500.PNG

System diagram of Konpeki system

One important technique BrainPad brought to Aucnet is transfer learning. If you want to train a deep neural network model in image recognition from scratch, it’d take several weeks and millions of training images. But with transfer learning, you can reuse a pre-trained image recognition model combined with a smaller neural network model.

In Konpeki, Aucnet used the pre-trained Inception-v3 model combined with a smaller neural network model, which significantly reduced the number of training images required. The current system uses about 200 photos for each car model, with a total of 150,000 images used to classify more than 700 different car models at high accuracy.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-2gw7d.max-600x600.PNG

Konpeki integrated with the existing Salesforce dashboard

Turning IT engineers into ML engineers

After the initial workshop with BrainPad, Aucnet’s IT engineers learned how to use the technology quickly. This allowed them to start leading the development of the new system themselves, improving its accuracy through trial and error. In this way, Aucnet’s IT engineers were transformed into ML engineers.

Here are some examples of solutions Aucnet’s ML engineers developed to solve their technical challenges:

  • System integration with Salesforce: Aucnet’s ML engineers needed to think not just of what ML could do, but also how it would be used in the real world. To that end, they built Konpeki to be integrated with the existing Salesforce dashboard Aucnet’s used car dealers use to upload their vehicles.
  • Continuous Integration (CI) of ML models: Just like in software development, Aucnet had to to build a CI framework for production deployment of ML models. This meant the continuous training of models with the latest data, as well as automated evaluation, version control and deployment of models.
  • Distributed training with ML Engine for 86X faster performance: In the beginning, Aucnet used TensorFlow running on a single Compute Engine instance to train the model, but it took 43 hours and over 15,000 images. To shorten the time, Aucnet began using Cloud Machine Learning Engine, a fully-managed distributed training and prediction runtime for TensorFlow. Machine Learning Engine made the training process 86 times faster.
  • Adjusting the Inception v3 model so it could recognize the difference between left and right camera angles: The Inception v3 model is trained to recognize objects at any angle. This made it hard for Aucnet to classify whether an image was from the left or right angle. To solve for this, Aucnet’s ML engineers created a standalone model just for recognizing the camera angle in a photo.
  • Splitting models into different parts, then combining it with Vision API for higher accuracy: Initially, Aucnet was using a single big model for classifying all car types at once. With the current system, they split the models into multiple parts: models for recognizing the angle of a car image ( front, side, rear, etc.), and models for recognizing 40 different car parts (steering wheel, tires, etc). Also, they used Vision API to detect the car manufacturer. By combining all of these methods, the system can classify the exact car model far more accurately.

https://storage.googleapis.com/gweb-cloudblog-publish/images/aucnet-5y1g3.max-600x600.PNG

Combining many models and Vision API to achieve higher accuracy

Based on what they learned, Aucnet’s ML engineers are now evolving their system to make it into a generic object recognition platform. As the first step, they’ve even started a trial to adapt it to recognize branded products for auction.

Conclusion

Through the TensorFlow community, Aucnet was able to link up with ML experts BrainPad in order to pursue their own ML solution. This collaboration allowed them to convert their existing IT engineering work force into ML engineers and create a valuable asset for the company with Konpeki. Interested in creating your own homegrown ML workforce, or becoming an ML engineer yourself? The TensorFlow community is a great starting point. You can learn more at tensorflow.org.

Posted in