Unacast: Scaling for rapid growth with a flexible pipeline primed for geospatial data

About Unacast

Unacast is a Norwegian company based out of New York, leading a global push for accuracy and transparency in location data through the proprietary Real World Graph®. Unacast has won multiple awards for both its products and its innovative culture based on decency and diversity. Previous to Unacast, the founders were part of the founding team at TIDAL, acquired by Jay Z in 2015.

Industries: Technology
Location: Norway

Unacast scales its geospatial services seamlessly from millions to billions of rows of data ingested with Google Cloud Platform, developing a DaaS and PaaS offering with its modular managed services.

Google Cloud Results

  • Enables cost-effective and complex geospatial analysis with easy joining of location data using BigQuery GIS
  • Supports rapid growth with frictionless scaling from millions to billions of data rows ingested daily into BigQuery
  • Facilitates developing new products including a Platform-as-a-Service model using modular managed services

Easy storage of over 1PB of data in BigQuery

Unacast aggregates various real-world data sources such as GPS and telecommunications data drawn from multiple partners at scale, into a data set that is vast, precise, and verified. Through its Real World Graph® platform, it offers unique location data insights with transformative applications for many industries including city development, transportation planning, and retail analytics.

"The real world is chaotic," explains Kjartan Slette, Co-founder and COO at Unacast. "We want to understand the physical world better, providing a service that can deliver both density and precision, and respond to the needs of multiple industries. To do that, we need to gather extreme volumes of data from different kinds of signals with variable quality attributes. Aside from this, we want to be as transparent as possible about the quality and sources of that data."

"When you're trying to build data models on top of the real world, scalability is always going to be a concern. We wanted a platform that was ready to scale very quickly, enabling us to expand from our initial marketing focus to other verticals and models of data provision."

Kjartan Slette, Co-founder and COO, Unacast

In order to deliver accurate and verifiable insights on top of massive amounts of data from the real world, Unacast needed a platform that was flexible and offered optimal scaling. Google Cloud Platform (GCP) provides the scalability and transparency it was looking for.

"When you're trying to build data models on top of the real world, scalability is always going to be a concern," says Kjartan. "We wanted a platform that was ready to scale very quickly, enabling us to expand from our initial marketing focus to other verticals and models of data provision."

Achieving frictionless scaling with managed services

To support Unacast in quickly developing new products and services, it needed an agile infrastructure. It wanted to be able to scale easily, while also reducing DevOps to a minimum so its team had time to work on development.

To do that, Unacast chose to build its data pipeline on GCP, using App Engine, Cloud Pub/Sub, Cloud Dataflow, and BigQuery, and using Cloud Composer for workflow orchestration and monitoring. Unacast also chose to build its architecture on Google Kubernetes Engine with Kubernetes from the outset, and the platform now scales painlessly according to the company's needs.

Because all the GCP services are managed, Unacast is able to keep time spent on operations and monitoring down to a minimum. Cloud Dataflow delivers powerful responses to advanced jobs such as streaming, batch processing, large mapping analysis, and major import jobs. Over the past three years BigQuery has established itself at the core of the company's work life, as the volumes of data have grown rapidly. Unacast now has over a petabyte of data stored in BigQuery and in Cloud Storage.

"As a company that works with geospatial data, BigQuery GIS has been a dream come true. It just makes everything easier: it's faster and more cost effective, and it's enabled us to do things we simply previously could not do."

Andreas Heim, VP of Engineering, Unacast

"Three years ago, we were loading tens of millions of rows into BigQuery every day. This has now increased to around twelve billion rows of data daily," says Andreas Heim, VP of Engineering at Unacast. "Despite that rapid growth, we've had almost no friction in scaling up our ingestion by two orders of magnitude. We didn't fully realize how powerful BigQuery was until we started using it extensively. The sheer speed, flexibility and user-defined functions are exceptional. We use it to prototype pipelines for data processing, analytics, learning, and as our main storage."

Simplifying geospatial joins, using BigQuery GIS

For its geospatial data analysis, Unacast has recently started using BigQuery GIS. Before, in order to perform certain geospatial calculations Unacast loaded data into Cloud Dataflow for processing prior to implementing user-defined functions in BigQuery. Now, the team is taking advantage of native GIS support in BigQuery to not only optimize how they perform critical geospatial joins, but to unlock additional value.

"As a company that works with geospatial data, BigQuery GIS has been a dream come true," says Andreas. "It just makes everything easier: it's faster and more cost effective, and it's enabled us to do things we simply previously could not do. For example, we can work with historical data in a cost-effective manner, which means delivering better insights for our customers."

"Using our own infrastructure as a model, we're now externalizing it, which is a completely new income stream for us. With Google Cloud Platform, it's easy to isolate individual services and create a bespoke offering for individual clients."

Kjartan Slette, Co-founder and COO, Unacast

Expanding to a Platform-as-a-Service model

Thanks to GCP, Unacast has been able to effortlessly scale its infrastructure as the volume of data it handles has grown. "The services are really well designed and fit together perfectly," says Andreas. "That has enabled us to move fast. We're onboarding data faster and pushing out new features faster. For example, Cloud Composer has replaced ten different services that were doing the same work and has simplified the onboarding of employees."

Now, Unacast is planning to expand its use of big data services on GCP with Cloud Bigtable. "We're really interested in Cloud Bigtable at the moment, as a way of increasing the speed and reducing the cost of scanning data," says Andreas. "We're also experimenting with machine learning, and we know Cloud AutoML is ready and waiting for us to plug in when we find the right use case for it."

Using GCP infrastructure in a modular way, Unacast has also been able to develop a new entry point to its services, offering its platform as a service. "Using our own infrastructure as a model, we're now externalizing it, which is a completely new income stream for us," explains Kjarten. "With Google Cloud Platform, it's easy to isolate individual services and create a bespoke offering for individual clients."

"We want to be a business that can solve absolutely anyone's location problems, even companies that aren't set up for sharing data," he adds. "With GCP, we have the tools we need to scale further, and use our new PaaS services to develop even closer relationships with our partners."

About Unacast

Unacast is a Norwegian company based out of New York, leading a global push for accuracy and transparency in location data through the proprietary Real World Graph®. Unacast has won multiple awards for both its products and its innovative culture based on decency and diversity. Previous to Unacast, the founders were part of the founding team at TIDAL, acquired by Jay Z in 2015.

Industries: Technology
Location: Norway
Google Cloud Platform logo

12 Months FREE TRIAL

Try Kubernetes Engine, BigQuery, and other Cloud Platform products with $300 in free credit and 12 months.

TRY IT FREE
Google Cloud Platform logo

12 Months FREE TRIAL

Try Kubernetes Engine, BigQuery, and other Cloud Platform products with $300 in free credit and 12 months.

TRY IT FREE