Google Cloud helps ADAM and THOR find asteroids

Asteroids are relatively small objects that orbit the Sun. Besides being fascinating objects, there is real value in understanding where they are and their trajectories both to better understand the composition and formation of our solar system and to make sure none of them gets too close to our dear planet anytime soon.

Being small, they’re not easy to see and their orbits are often far from the ideal Keplerian ellipses due to the many perturbations that affect them. Discovering and tracking them often requires specialized surveys made with some of our most powerful telescopes over a long time.

The Asteroid Institute, with the help of Google Cloud, has found a new way of discovering hidden asteroids in existing datasets. These asteroids were hidden not because they couldn’t be seen (they were there all along in existing observational images of the sky), but because we previously did not have the computational power to recognize them in those datasets.  

Google Cloud worked with researchers from The Asteroid Institute, a program of the non-profit B612 Foundation, to apply modern cloud computing tools and algorithms to deep field sky surveys to find and map asteroids in our solar system. The Asteroid Institute announced today that the new method has found and validated 104 new asteroids, opening the possibility of using this method to find hundreds or thousands of new asteroids hiding in old datasets, as well as changing the way we search for asteroids with new telescopes.

“We’ve proven a new, computationally driven method of discovering asteroids,” said Dr. Ed Lu, Executive Director of the Asteroid Institute and former NASA astronaut, in a conversation with Vint Cerf, Vice President and Chief Internet Evangelist at Google, about the new computation method for discovering asteroids. “The Minor Planet Center confirmed and added these newly discovered asteroids to its registry … opening the door for Asteroid Institute-supported researchers to submit thousands of additional new discoveries. This has never been done before, since previous methods have relied on specific telescopic survey operations. Instead, we've been able to identify and track asteroids using enormous computational power.”

ADAM and THOR: Hunting the sky together

The Asteroid Institute employs a cloud-based astrodynamics platform called the Asteroid Discovery Analysis and Mapping (ADAM) to hunt for asteroids in our solar system. ADAM is an open-source computational system that runs astrodynamics algorithms at massive scale in Google Cloud. 

The novel algorithm used to discover these new asteroids is called Tracklet-less Heliocentric Orbit Recovery (THOR). The algorithm links points of light in different sky images that are consistent with asteroid orbits. Having multiple observations of an asteroid taken over a range of time is essential to calculating its orbit. THOR can identify asteroids and calculate their orbits meeting criteria by the Minor Planet Center to recognize them as tracked asteroids. 

For its initial demonstration, researchers from the Asteroid Institute and the University of Washington searched a 30-day window of images from the NOIRLab Source Catalog (NSC), a collection of nearly 68 billion observations taken by the National Optical Astronomy Observatory telescopes between 2012 and 2019.

Finding and tracking asteroids is a difficult problem for a few reasons. Space is huge, even when we know where we are supposed to look. The relative luminosity (the intrinsic brightness of a celestial object) is low in comparison to planets in our solar system or faraway stars. Also, the Earth is constantly moving through the solar system, as are the asteroids. Linking different images of the sky to previous signals in the data increases the variables and makes the entire endeavor extremely computationally expensive when dealing with large datasets.

“First, linking asteroid detections across multiple nights is difficult due to the sheer number of possible linkages, made even more challenging by the presence of false positives,” researchers wrote in a paper on the introduction of THOR in May 2021. “Second, the motion of the observer makes the linking problem nonlinear as minor planets will exhibit higher order motion on the topocentric sky over the course of weeks.” 

The way asteroid detection has been performed before was to require multiple images of the same area of the sky be taken within roughly an hour to ensure that moving asteroids only traverse a small distance in the sky and so can be linked together easily into a so-called tracklet.  Then if multiple tracklets are taken over a longer period (a week or more), an orbit can be determined. However, these constraints on the telescope observing pattern mean that most astronomical datasets are unsuitable for asteroid detection. THOR removes the need to rely on the asteroid tracklets in the data (thus “Tracklet-less”). By relaxing the constraint for very closely spaced observations, individual asteroids can traverse arbitrary distances in the sky.  THOR is able to link these far flung observations by assuming a series of test orbits and transforming sky images into the frame of these test orbits.     

“This provides a path to scanning an otherwise voluminous 6D phase space with a finite number of test orbits and at feasible computational cost,” researchers wrote.

One of the primary benefits of the THOR approach is that it can be used on datasets beyond the scope of just one celestial survey, opening up the possibility of investigating a diverse array of datasets and making connections between them. Thus, THOR becomes an independent agent from the observer (the telescope survey data) and able to make connections between different datasets from different times.  Not only that, but with THOR, future asteroid search telescopes can be designed with more efficient observing cadences (i.e. without the need for rapid repeated images of the same area of sky).  

One interesting note from THOR is that it is not based on machine learning or neural networks. The power of THOR is in taking data from the test orbits and applying statistics plus physics, through linear and clustering algorithms and then using a massive amount of computational power through the cloud. One area where machine learning can be applied in the future will be in making the search through the infinite possible orbits smarter and faster, in the same way that DeepMind’s AlphaGo algorithm efficiently determines which move to make next on a Go board among a very large number of possibilities.

Scaling ADAM with Google Cloud

THOR runs on the ADAM, which is The Asteroid Institute’s astrodynamics-as-a-service platform powered by Google Cloud, including the scalable computational and storage capabilities in Compute Engine, Cloud Storage, and Google Kubernetes Engine.

In 2019 and 2020, Google Cloud’s Office of the CTO conducted architectural sessions and organized hackathons with solution architects at Google Cloud to find the best way to implement ADAM on Google Cloud at scale. Google Cloud also provided cloud credits and technical support for the ADAM platform’s current development and future work.

“The potential of this software ecosystem also stretches far beyond historical data – with additional development, ADAM::THOR will be able to perform real-time asteroid discovery on observations as they come in from telescopes around the globe,” said Joachim Moeyens, co-creator of THOR and an Asteroid Institute Fellow.

While THOR makes digging through existing astronomical datasets computationally feasible, it is important to consider that the data consists of a massive amount of space, searching for continuous orbits. It’s still a very large number of candidate orbits that need to be searched and thus requires the type of storage capacity and computing resources that only the public cloud can provide. The raw astronomical data is stored in Google Cloud Storage, with refined data later abstracted into BigQuery. Google Cloud enables ADAM to scale by allowing it to run on thousands of machines simultaneously, making it possible to analyze the data in a reasonable amount of time. 

One of the most powerful aspects of the partnership between B612 and Google Cloud is to make ADAM a “discovery-as-a-service” platform available for future researchers.

“With the arrival of the public cloud, researchers can now get access to large computational resources when they need them with much less overall cost and time penalty,” said Vint Cerf. “And we're seeing that large datasets can be applied to new problems.”