Moloco handles 5 million+ ad requests per second with Cloud Bigtable

Editor’s note: Here we explore how the machine learning firm, Moloco, uses Google Cloud database services and AI platform to help mobile app advertisers create targeted ad campaigns using ML-based predictions and win ad placement bids all in under 100 milliseconds.

Moloco is a machine learning company that currently works with customers who need solutions for online advertising and ecommerce. Our company runs a mobile audience platform for user acquisition, engagement, and monetization. We help app advertisers acquire, re-engage, and retain high-value mobile app users at scale. We build custom machine learning (ML) models for clients that help them optimize their campaigns and predict performance, so they can hit their goals quickly and efficiently.

With the help of Cloud Bigtable, the scalable, fully managed NoSQL database, we’ve dramatically increased the number of mobile user bid requests we can handle—from more than 550,000 per second to over 5 million requests per second. Bigtable and other integrated Google Cloud data (such as Google Kubernetes Engine - GKE, and Memorystore) and machine learning services help our customers optimize the outcome from ad campaigns.

Putting the smarts in advertising technology services 

We’re in the business of helping our customers run the most efficient ad campaigns possible by predicting the probability of advertising impressions per placement and using ML-based insights to recommend adjustments. We help companies make informed decisions about ad placement by analyzing data to determine how well an ad will perform when shown. Using these insights, we continuously update our bid processing engine and keep improving the performances of our customers’ ad campaigns through internal recommendations. Our flagship solution is Moloco Cloud DSP, a cloud-based programmatic advertising platform and Moloco Retail Media, a solution that enables retailers to offer ad solutions to their sellers. Both utilize the Moloco Machine Learning Engine to power performance.

For example, an app advertiser might come to us for assistance running a campaign so that they can expose their ad to billions of potential users across the globe. We help them create and implement an ad-spot buying strategy to meet their business objectives, which is based on our proprietary understanding of mobile app users, advertisers, campaign outcomes, and market conditions. To develop this understanding, we need to constantly ingest new information from data sources across the mobile app ecosystems and use our machine learning models over the combined data to improve predictions. We gather mobile app impressions, mobile app installations, and post-installation events as provided by a customer’s measurement partner (MMPs) to understand the most likely outcomes possible from different ad placements and price them. 

Keeping up in a speedy market

Our data pipelines and machine learning models need to be running at top speed every second to support the scale of data ingestion, volume of bid requests, and continuous model training that we promise our customers. Most ad exchanges put 150 -millisecond time limitations on bid requests, meaning we have to return our bid responses within that time window or we lose the ad opportunity. That time frame includes any network propagation latency from and to the exchange itself or on our side, so we set an internal deadline at 100 milliseconds--that’s 100 milliseconds to process the request, sort campaigns, apply ML models for inference to evaluate campaigns against the request, adjust our predictions based on inventory types and bid floors, and finally return bid responses.

If we tried to create the whole solution ourselves, it would have taken us years to build. We knew we wanted a platform partner we can trust to keep us running on the cutting edge of technology without needing to dedicate an internal team to basic infrastructure. We chose Bigtable and additional Google Cloud compute, storage, and networking services because they support the speed and scale we need to be competitive, and they integrate seamlessly with the other Google services, like machine learning training and inference framework, that partially support our intelligent end-to-end ad tech service.

By building our ad tech platform using Google Cloud, we’ve expanded the number of bid requests we can handle, from 550,000 to over 5 million per second, that’s an increase of more than 9 times from where we started about two years ago. 

Processing ad requests at unmatched speed and scale

Incoming bid requests first hit our software and services on Google Kubernetes Engine which consist of over 3,000 virtual machines (VMs) within the Kubernetes cluster. If the request passes the sanity check and throttling rules, we evaluate it against our entire campaign history for a cohort of users, including successful and unsuccessful bidding history. We have billions of these records, all of which are stored in Bigtable and Memorystore. We use Bigtable as a long-term, persistent storage for such records.Bigtable shines when the underlying data can be stored in a sparse manner, which fits our use case perfectly, and when the data needs to be accessed with low latency.We use SSD storage for Bigtable as low latency is critical to our application. Our workload on Bigtable keeps varying with daily and weekly patterns. Bigtable allowed us to scale the instance size as tightly as possible without causing reliability issues. The autoscaling enabled us to improve our cost efficiency and reduce the operation overhead. All in all, using Bigtable has helped us create more value for customers through a unified data lifecycle. 

Such records are combined with key fields in the request, to be used as input features to our proprietary machine learning models designed with the TensorFlow platform. These models predict various metrics such as win probabilities (if we were to bid at certain price points) and chance of conversion. As bid requests go through multiple stages of the bidding process, we retain detailed logs (e.g., we export Load Balancer logs to BigQuery and GCS as well as use Stream Insert of BigQuery). Then, using the unified ML Ops platform, Vertex AI, we train our ML models with the new logs. 

We host machine learning models on a separate set of VMs for real-time inference through TensorFlow, so the bidding machines and the inference machines communicate through gRPC and protocol buffers. Incoming bid requests also carry useful contextual information, such as whether a user engages with an ad while reading an article. That, and a lot of other context from the other tables on Bigtable are extracted as features. Then we feed these features into the inference machine through gRPC.  

The models run on the defined feature set to understand an ad spot’s relevance  in this current context and for a large number of campaigns and provide a prediction for each of those campaigns. That's the result that comes back into the bidding system, which then converts probabilities into actual USD prices that we can compare against the bid floor we’ve chosen. Finally, we choose the best campaign and then participate in the bidding by responding back with a bid response.

We couldn’t perform at this scale or speed without Google Cloud as our backbone. Google Cloud databases, storage services, and Google Kubernetes Engine integrate with the larger ecosystem of market-leading services that make up Google Cloud services, making it easy for us to build fast data pipelines that create real time insights, so we can make better business decisions.

With the help of Google Cloud, Moloco can now process hundreds of billions of  bid requests every day thanks to Bigtable, which offers low-latency and high scalability of accessing our history data. We’re able to transform our data and our customers’ data into valuable insights. What’s more, with powerful, integrated tools like Bigtable, Vertex AI, and BigQuery we can easily manage our data and develop new machine learning models that continue to optimize ads for our customers. 

Learn more about Moloco and Cloud Bigtable. And check out our blog with helpful graphics that answers the question, How BIG is Cloud Bigtable?