Endear seamlessly integrates vast data sources with AlloyDB
JP Grace
CTO, Endear
Editor’s note: Endear is an omnichannel customer relationship management (CRM) platform that fuels hyper-personalization and customer insights. When its existing database struggled to meet increasing demand, the company recognized the need to migrate to a robust, scalable solution. Today, Endear uses AlloyDB for PostgreSQL, a fully managed Google Cloud database service that provides the superior performance, scale, and availability needed to continue delivering powerful solutions to retail brands.
As a small team managing big data at Endear, adaptability and scalability are key to our vision of staying at the forefront of CRM technology. Our work involves aggregating and integrating customer data from multiple silos, including ecommerce platforms, point-of-sale (POS) systems, marketing platforms, customer service platforms, loyalty platforms, and more.
This landscape poses a multifaceted challenge: We need to ensure data is handled accurately, securely, and efficiently, so we can offer real-time actionable insights to our clients. Achieving this at scale grew difficult with our current database service, prompting us to search for a new solution that could meet our standards of technical innovation, strategic foresight, and operational excellence.
While we explored other database solutions, including CockroachDB and several Google Cloud services, we ultimately decided on AlloyDB for its advanced data management and scaling capabilities. It’s been the backbone of our data strategy ever since.
How Google Cloud became our VIP
Years ago, we chose Cloud SQL as our database solution at Endear because it aligned with our workload demands and budget. As we grew and expanded our architecture, we realized we had outgrown Cloud SQL. That said, we soon found that Google Cloud was still the best fit in terms of pricing and scalability.
During our selection process, we assessed solutions from other vendors, looking at cost, performance, flexibility, and operational simplicity. Only AlloyDB offered the optimal balance in accordance with our needs. For example, CockroachDB offered scalability, but its pricing structure would have increased costs over time. We also assessed Spanner, known for its strong consistency and horizontal scalability, and Bigtable, recognized for its high throughput and scalability for analytics workloads. While both are powerful products, AlloyDB's blend of features offered a better fit in terms of performance for our specific workload patterns.
-
As a result, we decided to migrate to AlloyDB to support our growth. Its PostgreSQL compatibility gave us an open, standards-based relational database that could scale to thousands of transactions per second with performance that is often only possible with expensive, proprietary alternatives. As a fully managed service, AlloyDB also offered significant reductions in the operational overhead involved in provisioning, scaling, and maintenance. Plus, we could leverage our familiarity with Google Cloud’s ecosystem, reducing the learning curve and offering coherence across our technology stack. Staying inside Google's networks (VPCs) not only ensured the security and privacy of our connections but helped us avoid the added latency, cost, and complexity of communicating with 3P services over the internet.
The smooth move to AlloyDB
Migrating from Cloud SQL to AlloyDB was remarkably seamless, thanks to Google Cloud’s Database Migration Service, which handled much of the work for us. The first step involved setting up a replication slot in our existing Cloud SQL database and pointing it to our new AlloyDB cluster. This allowed us to establish a real-time data sync between the two databases to prevent loss during migration.
Next, we managed database connections with PgBouncer, a connection pooler for PostgreSQL. This ensured that we had no downtime, as we could quickly swap out the connection strings in our secrets manager to redirect traffic from the old Cloud SQL database to the new AlloyDB cluster. We also conducted thorough performance testing on the new AlloyDB database to verify it could handle our workload and performance requirements without any issues.
All that was left was to toggle the new database to live by updating our applications’ configurations to use AlloyDB as the primary database. The experience was positive from beginning to end and allowed us to start using AlloyDB with minimal impact on our operations. Most importantly, we were able to quickly get back to building the custom capabilities that our brands care about.
Today, we know that AlloyDB was the right choice. After just one year, we’ve already seen significant improvements.
-
6X the number of connections: Last year, we had one PgBouncerconnection pool that could serve roughly 400-500 connections. Now, we have three separate PgBouncer connection pools for our AlloyDB cluster, managing on average just under 1000 database connections.
-
6X the number of transactions per second: Currently, we peak at about 5,000 transactions per second on our primary cluster and 10,000 transactions per second on our read cluster. Prior to our migration, we were only able to run around 1,500 transactions per second.
-
Super fast response times: AlloyDB can sustain and scale our workloads easily, delivering a P99 aggregated query latency under 10ms and helping us maintain a great experience for our end users.
Keeping it in the Google Cloud family
AlloyDB was the perfect fit for our data architecture because managed databases play a crucial role in supporting various applications and use cases in our platform. In addition to AlloyDB, which handles our application and integration data, we also use the following Google Cloud services:
-
A managed Elasticsearch cluster in Google Cloud enhances search functionality and audience segmentation in our application.
-
BigQuery helps us perform complex queries on massive datasets with its powerful data warehouse capabilities.
-
Firestore provides real-time database features, enabling seamless synchronization and instant updates across our app.
-
Pub/Sub powers real-time messaging.
-
Google Cloud Storage provides durable and highly available object storage.
The strategic use of managed services from Google’s Data Cloud enables us to focus on delivering value to our customers without having to handle the complexities of data management and scaling.
Preparing our platform and customers for the future
As we look ahead, our new focus is on maintaining operational agility, addressing bottlenecks as they arise, and seamlessly integrating new capabilities.
Perhaps most exciting is our plan to integrate artificial intelligence (AI) into our products. AI presents a unique set of requirements, particularly in terms of data storage, processing, and analysis capabilities. Google Cloud databases will be instrumental in ensuring the seamless flow of data between our applications and AI models while providing robust tools and security features for data management and analysis. For example, we’re interested in seeing how AlloyDB can accelerate decision-making and put more decisions directly in the hands of retail store associates by running fast aggregated queries and making the results available with less friction.
Our vision of continuous growth and innovation is closely tied to the capabilities and support provided by Google Cloud databases. We see Google Cloud not just as a platform but as a partner in our journey towards embracing new technologies like AI, helping us to address future challenges and unlock new opportunities. With Google Cloud, we are confident in our ability to adapt, scale, and lead in our industry, thanks to its solid foundation.
