TNO: Making roads safer with TensorFlow and deep learning

About TNO

The Netherlands Organization for Applied Scientific Research (TNO) is an independent research organization established by law. TNO connects people and knowledge to create innovations that boost the sustainable competitive strength of industry and the well-being of society. TNO creates independent and reliable solutions to the grand challenges society faces.

Industries: Other
Location: The Netherlands
Products: TensorFlow

TNO researchers use TensorFlow to identify unsafe traffic situations where additional safety measures could save lives. Next, TNO is adapting its AI system to deploy on Google Cloud Platform.

  • Filters out relevant video so that one week of analysis can be done in one hour
  • Deploys GPUs in the cloud so clients no longer need specialist hardware
  • Helps implement research with extensive TensorFlow community on GitHub

One week of videos analyzed in one hour

It takes time for discoveries at a lab or university to make an impact in people’s lives, which is why the organisation TNO was founded in 1932. TNO bridges the gap between new research and its real-world implementation for clients across the Netherlands and abroad. Today, the organisation has over 2,700 professionals working on cutting-edge projects in energy, industry, health, defense, and urbanization.

European traffic safety statistics show that in just four years, between 2009 and 2013, more than 26,000 vulnerable road users such as pedestrians and cyclists were killed and over 250,000 seriously injured in road accidents across seven European countries. Many of the incidents took place on dangerous urban intersections where TNO estimates that improved road safety measures, such as pedestrian crossings, could save 5,500 people a year from injury or death. But before clients can put those measures in place, they have to identify which intersections and situations pose a hazard and what specific measures could improve these.

“We get information from video using our video analysis pipeline where deep learning, implemented in TensorFlow takes a crucial part. This usually requires specialised hardware, which not all of our clients have access to or can maintain, so cloud products make it easy to scale to their locations, whether they’re in Bangladesh or Barcelona.”

Paul van den Haak, Deputy Research Manager, TNO

That’s where the InDeV project comes in. InDeV is an international collaboration of researchers, including TNO, from six different European countries and Canada. It was started to develop new ways of measuring traffic safety because statistics on traffic safety were unreliable, insufficiently detailed, and hard to collect. By observing behavior in critical traffic situations there is no need for accident statistics to study traffic (un)safety. The job of the Intelligent Imaging experts of TNO is to apply machine learning to video of accident hot spots to rate intersections on a scale according to their safety.

“We combine our knowledge about traffic behavior and traffic safety with our knowledge of image processing to classify the safety of traffic junctions,” explains Paul van den Haak at the TNO Intelligent Imaging group. “We get information from video using our video analysis pipeline where deep learning, implemented in TensorFlow takes a crucial part. This usually requires specialised hardware, which not all of our clients have access to or can maintain, so cloud products make it easy to scale to their locations, whether they’re in Bangladesh or Barcelona.”

Filtering weeks of footage

Reviewing video footage is a time-intensive and expensive process. Trained personnel need to look through weeks of video of street intersections to gather data for research into traffic safety. A single intersection needs to be monitored for three weeks with two cameras to create an estimation of its safety, adding up to six weeks of footage, which can take six weeks of work to analyse. Typically, less than one percent of the recorded material is actually of interest to researchers.

“I worked with many different frameworks and found TensorFlow easy to use, especially when combined with a high-level library such as Keras, and very easy to implement in a cloud environment. It’s important that I can use a framework to implement and deploy a new idea as fast as possible.”

Maarten Kruithof, Data Scientist, TNO

To isolate the useful footage from the rest, TNO looked to use deep learning to detect dangerous situations by identifying pedestrians, cyclists, and vehicles in footage and analyzing their trajectories. “Identifying road users is the most computation intensive step in our entire process,” says Paul. “And to analyse those large amounts of data we need to use a GPU. That means the analysis can either be done on the specialised hardware that we have in-house, or on cloud servers.”

TNO processes footage through a neural network based on TensorFlow. “I was drawn to TensorFlow by its ease of use and availability of high-level libraries such as Keras,” says Maarten Kruithof, data scientist at TNO. “I worked with many different frameworks and found TensorFlow easy to use, especially when combined with a high-level library such as Keras, and very easy to implement in a cloud environment. It’s important that a framework is ready for me to implement and deploy a new idea as fast as I can. With TensorFlow, I can do that.”

TNO uses TensorFlow to localize objects in video, and because it is available on the cloud, TNO can bring it to market quickly, unburdened by the need for additional, expensive GPU hardware. “Our clients don't need to buy specialised equipment,” says Maarten. “They can just connect to our virtual machine and plug and play. Google’s machine learning platform lets us scale easily when more data becomes available, and the flexibility of the platform helps us speed up our develop-test-adjust cycle.”

Faster research, faster production

TNO always ensures that it works with the very latest versions of technology as well as keeping on top of developments. Bringing products to market is a crucial part of the organisation’s mission. Using TensorFlow helps TNO do both, as Maarten explains:

“When a new paper is published, people quickly implement it in TensorFlow and put their work on GitHub. That makes it much faster to make a demonstration version of an application. And there is such a large community working on TensorFlow that it’s easy for us to find new research.”

Maarten Kruithof, Data Scientist, TNO

“Before GitHub we had to read articles, then implement those methods ourselves for use in applications. One of the nice things with TensorFlow is that when a new paper is published, people quickly implement it in TensorFlow and put their work on GitHub. That makes it much faster to make a demonstration version of an application. And there is such a large community working on TensorFlow that it’s easy for us to find new research, because it will invariably be implemented within one or two weeks of publication.”

Using TensorFlow can tighten up the gap between demonstration model and market-ready versions, too. “Working with TensorFlow in the cloud means our test models are actually almost ready for production, without the mess of large scripts related to our in-house hardware,” says Maarten.

A week’s work in one hour

With TNO’s neural network based on TensorFlow, researchers report that it takes only one hour to review footage that would previously have taken a week to inspect. Now the team is adapting the system for deployment on Google Cloud Platform. “In the end we want to develop a toolbox that municipalities can use to have the traffic situation in their city analysed, without buying a new computer,” says Paul. “With Google Cloud Platform they could use our scientific models and algorithms without having to buy all kinds of expensive and complex hardware.”

“We're seeing deep learning explode in terms of people at TNO wanting to use it,” says Paul. “That means our capacity for this research needs to be very flexible, and with Google Cloud Platform we could just scale up or scale down, without installing and provisioning new GPUs for new students.”

About TNO

The Netherlands Organization for Applied Scientific Research (TNO) is an independent research organization established by law. TNO connects people and knowledge to create innovations that boost the sustainable competitive strength of industry and the well-being of society. TNO creates independent and reliable solutions to the grand challenges society faces.

Industries: Other
Location: The Netherlands
Google Cloud Platform logo

12 Months FREE TRIAL

Try Kubernetes Engine, BigQuery, and other Cloud Platform products with $300 in free credit and 12 months.

TRY IT FREE
Google Cloud Platform logo

12 Months FREE TRIAL

Try Kubernetes Engine, BigQuery, and other Cloud Platform products with $300 in free credit and 12 months.

TRY IT FREE