Delivery Hero: Serving up data across the company to ease decision-making with Google Cloud
About Delivery Hero
Launched in 2011, local delivery platform operator Delivery Hero strives to deliver anything, from meals to medicine. The Berlin-based company operates across Asia, Europe, Latin America, the Middle East, and Africa, processing close to 800 million orders per quarter.
Tell us your challenge. We're here to help.
Contact us
About Aliz
Google Cloud partner Aliz specializes in machine learning, data analytics, and cloud migration.
With a BigQuery-powered data mesh, Delivery Hero makes data accessible to teams across the company, enabling them to deliver data-driven solutions faster and leverage data for machine learning.
Google Cloud results
- Laying the groundwork for Delivery Hero’s machine learning initiatives by making data universally accessible
- Optimizing costs by shifting to a data mesh system where data units have control over their own query costs
- Increasing the interoperability, usability, security, scalability, and quality of data with the global infrastructure provided by Data Hub
- Using a BigQuery-powered data mesh to end time-consuming nightly data imports joining data from different data warehouses
- Facilitating enforcement of strict security and access standards with the granular security features of BigQuery
20+ data warehouses replaced with single global data mesh
The COVID-19 pandemic triggered a huge upsurge in the demand for food delivery services as various global lockdowns forced people to stay at home and order in, while the ease and speed of delivery has ensured that demand for these services continues to grow. Revenue in the online food delivery market is projected to reach $312,153 million in 2022. But it’s not just meals people want to be delivered. Intuitive apps have made it simple to order anything online, from groceries to medicine. And 791 million of these orders were processed by the Delivery Hero platforms in the third quarter of 2021.
Striving to be the world’s leading local delivery platform, with a continually growing global team, Delivery Hero works with an extensive network of restaurants, shops, riders, and partners. In 2019, the Berlin-based company opened its first Dmarts, delivery-only supermarkets that guarantee rapid delivery of groceries and household goods to customers around the world.
“To address the problems we were facing, we needed to make data globally and universally accessible, and to do this we decided to adopt the data mesh approach using Google Cloud.”
—Mathias Nitzsche, VP Engineering Global Data, Delivery HeroMaking data more accessible
Delivery Hero sees constant innovation as the core of its growth. This growth comes from multiple platforms within the group and is led by multiple business units, which get to work relatively autonomously. Their independence makes Delivery Hero more agile and innovative, but it requires a data infrastructure to match. This used to be a challenge in the past: as teams were all using their own tools and built their own data warehouses, it was difficult for them to discover, access, and combine data across functions, slowing down progress.
“Whenever someone had an idea, so much time was spent on figuring out where the data was, how to access it, how to understand its quality, and what security and compliance precautions were necessary,” says Mathias Nitzsche, VP Engineering at Delivery Hero’s Global Data team. “To address the problems we were facing, we needed to make data globally and universally accessible, and to do this we decided to adopt the data mesh approach using Google Cloud. As part of this effort, Delivery Hero also created a global catalog that centrally documents all available data, its meaning, its quality, and its source.”
Providing a shared data infrastructure with BigQuery
In line with Delivery Hero’s multicloud strategy, Nitzsche and his Global Data team look for the best vendor for every data use case. For Delivery Hero’s data mesh, BigQuery offered a solution that covered all the bases. Says Nitzsche: “We wanted to provide a common infrastructure, which is used by all data owners, as well as the necessary tooling and governance to make all data across Delivery Hero accessible with high interoperability, security, and scalability. BigQuery was the best tool on the market delivering on that very core idea.”
To build what Delivery Hero terms its Data Hub, which would replace its web of data warehouses, Delivery Hero worked with Google Cloud partner Aliz. “With Aliz, we didn’t have to waste a lot of time setting up the parameters of our collaboration,” says Nitzsche. “They focused on our challenges and helped us come up with solutions. It was an incredibly flexible and pragmatic collaboration. We’re really fortunate to have Aliz on our side.”
“Previously, we had a nightly web of data imports, and data could be delayed for up to 48 hours when imported from another data warehouse With our Data Hub on BigQuery, that work has disappeared because all our units have live access to the latest data from another warehouse in the data mesh.”
—Mathias Nitzsche, VP Engineering Global Data, Delivery HeroConnecting multiple data warehouses in a data mesh
Thanks to the Data Hub, all business units now have their own Google Cloud projects, where they manage their own data. There is a centrally developed airflow solution, using clusters hosted on Google Kubernetes Engine, which is independently deployed to each business unit’s Google Cloud project. This contains the business unit’s BigQuery, and the BigQuery functionality makes it possible to combine data from different BigQuery projects in the same region, creating Delivery Hero’s decentralized data mesh. In the future, there are plans to build on this using Analytics Hub.
The new data accessibility has been a paradigm shift for the company. Prior to the Data Hub, Delivery Hero’s business units had to constantly import data from other warehouses to solve the most basic problems. Details about customers, groceries, and logistics, for example, involved combining data from three data warehouses, which was time-consuming and tedious.
“Previously, we had a nightly web of data imports, and data could be delayed for up to 48 hours when imported from another data warehouse,” says Nitzsche. “With our Data Hub on BigQuery, that work has disappeared because all our units have live access to the latest data from another warehouse in the data mesh. This makes our teams a lot faster.”
Removing multiple scalability bottlenecks at once
Delivery Hero’s old approach to data sharing wasn’t just time consuming, but also hard to scale. As databases grew, systems administrators had to scale them manually and keep export pipelines intact so different business units could keep sharing data. “Over the years, this infrastructure created multiple scalability challenges we had to keep track of,” says Nitzsche. “Today, our solution scales automatically, and all these challenges are gone. It’s like moving from on-premises servers into the cloud, there’s a whole problem you simply no longer have to worry about.”
The many teams at Delivery Hero running data warehouses also had to foot the bill for the growing costs related to their consumers’ queries. BigQuery enabled the many teams running data warehouses to shift the responsibility for scaling the data warehouse to Google, and now consumers are responsible for paying for queries, which optimizes costs.
“Instead of the data provider team having to pay for however their data is accessed and used, the shift is that now the individual business units pay, which automatically incentives them to make reasonable queries and optimizations,” says Nitzsche.
Less hassle, more security
Security is also paramount for Delivery Hero, and with built-in security features from BigQuery, Data Hub makes it much easier to enforce strict security and access standards. “When you’re constantly importing and exporting data via storage buckets, securing that data becomes incredibly difficult,” says Nitzsche. “That’s another thing we no longer have to worry about. With the BigQuery column and row-level security features, we can define exactly who has access to what and even mask certain data fields to protect sensitive information.”
“Now that we’ve made our data universally accessible, we’re continuing our journey with Google Cloud to build our machine learning platform. It’ll help us to really leverage the power of our data and unleash new solutions that will ensure Delivery Hero remains the world’s leading local delivery platform.”
—Mathias Nitzsche, VP Engineering Global Data, Delivery HeroUnleashing the power of data with machine learning
Nitzsche’s Global Data team is an enabler for Delivery Hero’s business units, who can now leverage the newly built Data Hub to build better data-based products faster and define more consistent KPIs and definitions across the company. “Data scientists now have much bigger amounts of data at their fingertips, and they can just play with it and find insights, which would have been very hard to find just a year ago,” says Nitzsche.
A lot of these insights are bolstering the company’s growing machine learning capabilities. From route management for drivers and order predictions in logistics to better recommendations and personalization on the website, machine learning use cases are driving growth and delivering value across the business. The plan is to use machine learning to create a taxonomy of its products that will help Delivery Hero make better use of its sales data and improve communication. To build this powerful machine learning platform, Nitzsche and his team are using Vertex AI.
Data Hub has laid the groundwork for these developments, enabling and accelerating innovation across Delivery Hero’s business units. But for Nitzsche and his team, that’s only the beginning. “Now that we’ve made our data universally accessible, we’re continuing our journey with Google Cloud to build our machine learning platform. It’ll help us to really leverage the power of our data and unleash new solutions that will ensure Delivery Hero remains the world’s leading local delivery platform.”
Tell us your challenge. We're here to help.
Contact us
About Delivery Hero
Launched in 2011, local delivery platform operator Delivery Hero strives to deliver anything, from meals to medicine. The Berlin-based company operates across Asia, Europe, Latin America, the Middle East, and Africa, processing close to 800 million orders per quarter.
About Aliz
Google Cloud partner Aliz specializes in machine learning, data analytics, and cloud migration.