How Cloud Spanner helped Merpay easily scale to millions of users

Editor’s note: To launch a new mobile payment platform, Mercari needed a database solution strong on scalability, availability, and performance. Here’s how Cloud Spanner delivered those results. 

E-commerce companies need to connect customers to their services securely, reliably, with zero downtime. When Mercari, Inc. launched a new mobile payment platform, we chose Cloud Spanner as part of our data portfolio, which provided us with easy scalability to handle millions of new users, a fully managed service that minimized overhead costs, and deep integration with other Google Cloud services. 

Mercari, Inc., Japan’s largest C2C marketplace, launched its app in 2013, which allows 18.2million monthly users to easily and securely buy new and used items. Mercari expanded into the United States and in 2014 and launched Merpay, a mobile payment service that can be used through Mercari in Japan in 2019. With more than 8.5 million users, Merpay is now accepted at 1.8 million merchants and e-commerce sites in Japan, supporting payments via the DOCOMO ID contactless system and QR code.

Prioritizing availability and scalability

When we started building Merpay, we were looking for a new database. In the past, Mercari had used MySQL with bare metal hardware. Because of the amount of data, we required additional expertise to manage and maintain the hardware, software and MySQL implementation. Having built much of the microservices architecture for our Mercari app using Google Kubernetes Engine (GKE), it was natural to look at Google Cloud’s managed services when deciding upon our new database infrastructure for Merpay. 

For the database, we were focusing especially on requirements around availability, scalability, and performance. To do a single payment transaction, there were multiple steps each with writes, requiring high-write-throughput and low-latency from the database. Needing a solution that would support reliable payment processing 24/7/365, we chose Spanner, which offers up to 99.999% availability with zero downtime for planned maintenance and schema changes. 

We worked closely with Google Cloud’s Premium Support, Technical Account Management (TAM), and Strategic Cloud Engineer and Cloud Consultant teams to implement Spanner, including separating the payment processing— originally one of the functions in Mercari—as a microservice.

A nod to easier nodes and better scale

After launch, the number of Merpay users reached 2 million in only a few months. We had 45 Spanner nodes at the time of launch of Merpay and about 50 nodes four months later. Because we’d optimized the application side during those four months, we didn’t have to add many nodes to keep up with the growing traffic. 

Whenever we needed to scale up the serving and storage resources in our instances, Spanner made it easy to increase nodes as needed in the Cloud Console. To optimize costs, it’s easy to add nodes during a marketing campaign and remove them afterward. Even if the traffic count is different from expected, we can change the number of nodes immediately. That's one incredible convenience of Cloud Spanner.

Building powerful pipelines

Our data pipelines are used for KPI analytics, fraud detection, credit scoring, and customer support use cases. Because Cloud Spanner integrates so easily with other Google Cloud data services, the data in Spanner is easily accessible for analytics. From the Spanner database for each microservice, we create both batch and streaming data pipelines using Pub/Sub and Dataflow, as well as Apache Flink, and load the data into BigQuery and Google Cloud Storage. We’re able to quickly aggregate the data required for data platforms such as BigQuery and Cloud Storage. The original data is stored in Spanner and managed by microservices, but analysis through BigQuery is centrally managed by the Data Platform team. This team determines the confidentiality level of data and centrally manages BigQuery permissions using Terraform. By centrally managing data access permissions on the data platform rather than on individual microservices, we’re able to set appropriate security for individual users.

Our full data portfolio also includes Looker, which we use for product analysis, accounting analysis, test performance visualization, operation monitoring, development efficiency analysis, and HR analysis. We also use Dataproc, Cloud Composer, Data Catalog, and Data Studio. The Dataflow template created for the pipeline is also published as OSS. 

With Spanner and other Google Cloud services, our Merpay platform is flexible, secure, scalable and highly available. As Spanner eliminates overhead, we can devote engineering resources to developing new tools and solutions for our customers. We’re looking ahead at providing cryptocurrency service, for example, and are now working on a project to migrate Mercari’s monolithic system that is still on premises entirely over to Google Cloud.

Read more about Mercari and Merpay. Or check out our recent blog: three reasons to consider Cloud Spanner for your next project.