Creating millions of virtual worlds for the next AI breakthrough

In 2012, the availability of the ImageNet training data set drove a dramatic shift in the development of artificial intelligence, most notably with the first truly successful vision AI models. For the first time, researchers could create machine learning (ML) models that could recognize objects with human-level accuracy. 

While ImageNet spearheaded the current AI revolution, there are still many domains where researchers do not have enough available data to create high quality models, because it either doesn’t exist, is not accessible or is too expensive to obtain. For example, a key factor limiting the creation of self-driving cars is the availability of driving data across many possible driving conditions, including accidents or near misses. Obviously we cannot recreate traffic accidents in real life to obtain data to train our self-driving AI models. But we can create simulations that help us do that—which can be transformative.

Types of simulations
Ever since computers were invented, we have used them for simulations. Some of the very first computers were built explicitly to simulate complex physical phenomena, from ballistics to weather. And many of the largest supercomputers today are used for simulation as well, from finding new chemical compounds to predicting climate change.

I will distinguish two different types of simulations: high precision scientific (or engineering) simulation and large scale probabilistic simulation. 

Scientific and engineering simulations typically require a very high degree of accuracy. If we want to understand how weather works, we need to model fluid dynamic equations, including factors such as temperature and density, and have a very high resolution model of the world. If we are building a new turbine fan, it’s necessary to understand at a very fine level of detail the fan’s physical properties. Factors such as thermal conductivity or elasticity may play an important role when the fan is rotating at tens of thousands of revolutions per minute and operating at temperatures of thousands of degrees Fahrenheit. Microscopic and quantum effects may be critical to understanding a particular physical system.

In probabilistic simulations, we typically have rougher models of reality, but we need to explore a vast amount of options. Many of these simulations originate from traditional Monte Carlo methods (popularized by Ulam and others). Here, we try to obtain an approximate solution, or range of solutions, by running through many slight variations of a simulation scenario. If we are trying to guess whether a particular city grid will see a lot of traffic, we can simulate thousands of different random scenarios with various numbers of vehicles, destinations and speeds. The actual physical characteristics of the car engines don’t matter as much as the higher level properties of the simulated world.

Scientific simulations typically need moderate amounts of very precise data, are very hard to build and scale, and may require very expensive dedicated supercomputers. Large scale probabilistic simulation, on the other hand, is inherently parallel, and can be built and scaled up effectively in the cloud. Because of this, these simulations can be an ideal data source for the creation of training sets for AI models, as well as to enable the end to end testing of AI systems.

Simulation Use Cases
The use case for simulations goes beyond self-driving cars; it is a new paradigm for the creation of AI. We can solve problems where we have a lot of examples of how humans do it or where we can simulate the environment with a high degree of fidelity.

Some of the most interesting problems that can be solved by simulations are in the robotics space. In principle, it’s easier to build a digital twin of a factory and program or train robots safely in this space rather than create them in a real physical environment. This is true especially in situations where the robots need to interact safely with human workers or where the parameter space is too large, such as grippers for the thousands of items that may be in a retail warehouse. In these types of applications, there is no easy way to program the robots for the vast number of potentially encountered scenarios. 

Even beyond robotics, there are many new areas where we can apply the same simulation approach. In retail, we can simulate movement of people to design a floor plan. In manufacturing, we can help assembly line design. In logistics and supply chain, we can create transportation networks that are more efficient and resilient. And, we have barely started to explore the applications of simulation in the healthcare space. If we were able to simulate the chemical interactions inside a cell, we could, for example, predict the potential effect of a new drug. And even if the simulation is not 100% precise, we could still use it to help guide the testing process of new substances, saving years of development and hundreds of millions of dollars in the process.

At Google Cloud, we’re excited about simulation and its tremendous potential to help us build AI systems. I will speak about simulation, in particular simulation for manufacturing next November 3 at Gartner EMEA. If you are attending, please reach out—you can always connect with me on LinkedIn and Twitter. You can find more details about Unity Simulation here.