Databases

Statsig supports up to 7.5 million QPS with Memorystore for Redis Cluster

March 21, 2024

Jason Wang

Software Engineer at Statsig

Brent Echols

Software Engineer at Statsig

Try Gemini 1.5 models

Google's most advanced multimodal models in Vertex AI

Try it

Editor’s note: Established in 2021, Statsig helps companies ship, test, and manage software and application features with confidence. Facing bottlenecks and connectivity issues, the company realized it needed a performant, reliable, scalable, and fully managed Redis service — and Memorystore for Redis Cluster ticked all the boxes. With real-time analytics capabilities and robust storage (99.99% SLA) at a lower cost, Memorystore provides a higher queries per second (QPS) capacity. This allows Statsig to refocus on its core mission: building a full product observability platform that maximizes impact.

At Statsig, we’re passionate about experimentation. We want to make it easy for companies to test, iterate, and deploy product features, gaining key insights into performance and user behavior in the process.

Statsig is a feature-flag, experimentation, and product analytics platform that helps our users make data-driven decisions to drive better user experience in their software and applications. Our platform's ability to update and manage all that data in real time is critical. As our team grew from eight engineers with an open-source Redis cache, we sought an operational database capable of handling our expansion and query demands.

Experimenting with a new database

We used to rely on the caching solution provided by a different cloud provider as part of our data store. However, there were throughput bottlenecks, connectivity issues, latency slowdowns, and high costs that didn’t return value as we had hoped. It became difficult to see how our previous system would remain sustainable under the strain of increased loads and higher demand.

We decided to follow in the footprints of our platform and experiment: We went looking for a solution that combined robust efficiency, clustering capabilities, and features at a lower cost point. We chose Memorystore for Redis Cluster because it allowed us to accomplish our business goals without compromising on cost or predictability.

And since most of our applications are stateless and easily handle changes in the cache, our migration to Memorystore for Redis Cluster was a simple, smooth chance to bring our operations in line with our business strategy.

Elevating real-time data management

Memorystore for Redis Cluster has become an invaluable asset for Statsig, delivering robust scalability and versatility for our operations. We effectively utilize its capabilities for things like regional feature flagging, events and metrics storage, real-time analytics, and read-through caching. We have also been using Memorystore for our core Statsig console features such as real-time health checks and event sampling/deduplication in our streaming pipeline.

Memorystore for Redis Cluster's high availability (99.99% SLA) ensures the consistent performance and reliability critical to our services. The flexibility to seamlessly scale in or out empowers us to dynamically adapt our cluster size as needed.

And the results are undeniable, with measurable improvements observed across key areas.

Improved database performance - With Memorystore for Redis Cluster, we are more confident in the caching layer and its support for more use cases and higher queries per second (QPS). We are now able to easily handle an average of 1.5 million QPS, and up to 7.5 million QPS at peak, keeping pace with our customers as they scale.

Greater scalability - Memorystore for Redis Cluster’s ability to scale in or out with zero downtime has allowed us to support higher QPS and a range of use cases, positioning us to grow our customer base and services.

Cost efficiency and reliability - We've achieved a considerable reduction in costs while maintaining high-quality service. The efficiency of our database running on Memorystore has translated to a 70% reduction in the cost of Redis, as compared to the costs of running the same workloads with our previous cloud provider. Memorystore’s reliable performance has also helped with our real-time data processing needs.

Enhanced management and monitoring - The transition to Memorystore has simplified our database management, saving us from diagnosing consistent Redis connection issues. Memorystore’s easy-to-use monitoring tools reduced the time our developers spend on database issues and freed them up to focus on platform innovation.

Integrated security - A little extra peace of mind never hurts. Memorystore's seamless integration with Google Cloud VPC enhances our security posture.

Looking forward with Memorystore

As our customer base grows, we are expanding our use of Memorystore for Redis Cluster to enable smarter and faster product development, and are confident that it will remain a central component of our services — a database we can count on to support higher loads and increased demand. Looking forward, we’ll keep finding use cases for Memorystore’s robust features and dependable scalability.