Carousell: Creating an ecosystem of second-hand goods for a greener future with Google Cloud
About Carousell
Carousell is an online classifieds marketplace that lets anyone buy and sell items. Launched in August 2012, Carousell began in Singapore and is now one of the world's largest and fastest-growing marketplaces in eight markets across Southeast Asia, Taiwan and Hong Kong. With a focus on sustainability, Carousell launched its Carousell Green initiative to encourage customers to buy and sell pre-owned items and minimize waste.
Tell us your challenge. We're here to help.
Contact usCarousell scales its online classifieds marketplace on GKE for a smooth user experience, analyzes massive datasets on BigQuery, and deploys ML models with Tensorflow on Google Cloud to improve the way people buy and sell things.
Google Cloud results
- Optimizes costs with Spot VMs to invest savings in features that create value for customers
- Saves up to 70% of engineers time so they can focus on innovation rather than infrastructure tasks
- Provides a single source of truth with real-time data on BigQuery to support decision-making across the business
Scalable architecture to support tens of millions of users
Selling second-hand electronics online may seem commonplace today, but back in 2012, this practice was not as popular. It was for this reason that the founders of Carousell started the mobile classifieds marketplace that now connects tens of millions of buyers and sellers. Since then, the platform has expanded to include the buying and selling of almost anything and everything, from fashion and entertainment, even property. On the platform, sellers can easily list products and services through its web and mobile app, while buyers can search for products with keywords and images in milliseconds.
Since its launch, the business acquired similar and complementary organizations across the region in recent years. Now, the Carousell Group comprises companies and platforms such as Carousell, Mudah in Malaysia, Chotot in Vietnam, and OneKyat in Myanmar.
Carousell takes a sustainable approach to e-commerce by encouraging consumers to buy second-hand items as the first choice instead of purchasing new ones. In the beginning, Carousell operated a variety of hosted services. However, as the marketplace expanded, this infrastructure was unable to meet business requirements for a stable, scalable, reliable, and cost-effective technology platform. This is why the team decided to turn to Google Cloud.
"Google Cloud provided a fine degree of control over instance configurations at an attractive price point and delivered the freedom to adapt our cloud architecture easily to accommodate changes to the business," says Sanjeev Jaiswal, Senior Director of Engineering at Carousell. "We also saw using Kubernetes as a key piece of the puzzle in helping us reduce our maintenance requirements."
The business quickly migrated its applications and workloads to the cloud platform. The engineering team completed the initial project in two weeks before fine-tuning the architecture to optimize performance, reliability, and cost-effectiveness.
"Our goal is to move all production workloads to a standard technology stack by 2022 to maximize resources and reduce costs," says Sanjeev Jaiswal, Senior Director of Engineering at Carousell. "It's difficult to find people with experience across multiple cloud environments. By having all backend operations on Google Cloud, engineers can ramp up on the technologies quickly and confidently."
Google Cloud also offers an architecture that can scale with Carousell's growth. This makes it possible for the site reliability engineering (SRE) team to adjust resources to meet demand without having to deal with issues related to server provisioning and staffing shortages at the data center. "Instead of planning capacity for six months, we can support long-term strategies without infrastructure constraints," Jaiswal adds.
With the time saved, the engineering team can now participate in design discussions and work on major projects. "With Google Cloud, the team can spend up to 70% of their time working on major projects such as creating a configuration that allows us to reproduce the Carousell app environment for testing, experimentation, and innovation," says Jaiswal.
Carousell leverages the interoperability between Google Cloud products to support its compute and data workloads so the platform can run smoothly. The company uses Compute Engine to access compute resources to run PostgreSQL databases instances, Cloud Load Balancing to distribute cloud resources effectively, and App Engine to develop and host applications in the cloud. When someone lists an item or service on Carousell, the static assets such as images and videos are saved in Cloud Storage and accessed in relevant search results. Persistent Disk provides block storage for Compute Engine instances, and Dataflow runs batch processing jobs.
"It's difficult to find people with experience across multiple cloud environments. By having all backend operations on Google Cloud, engineers can ramp up on the technologies quickly and confidently."
—Sanjeev Jaiswal, Senior Director of Engineering, CarousellRunning cost-efficient Kubernetes workloads with Spot VMs
The business started with Docker to isolate applications and services from their environments and run a Kubernetes cluster on Compute Engine. However, the time and resource savings available by automating the management and scaling of clusters through Google Kubernetes Engine (GKE) proved compelling. Carousell now uses the service to run all its in-house applications and code. The Carousell platform backend and messaging layer built-in Apache Kafka now runs in Kubernetes.
With GKE, the business can scale seamlessly to support demand peaks and sustained growth. The service stops and restarts Spot Virtual Machines (VMs), previously known as preemptible VMs, to minimize costs as part of the 50 to 100 cluster size changes required during intensive customer usage. "Google Kubernetes Engine enables our engineers to focus 90% of their time on a single code base and let the product handle application scaling for them," says Jaiswal.
Carousell uses Spot VMs to run more than 200 GKE nodes at a lower cost, creating an affordable and flexible environment for developing applications. "Stateless and batch workloads can be handled easily by Spot VMs at a low cost," says Jaiswal. "If Spot VMs are terminated due to system events, we have replacement Spot VMs available. In the worst case scenario, we can fall back to on-demand VMs."
One such use case for Spot VMs is a self-service discovery portal to improve developer productivity. It can be daunting for developers new to the company to know how each of the 150 microservices functions. If a service goes down, they may scramble to find critical information such as what the service does or who the owner is to troubleshoot the outage. The developer portal aims to simplify service discovery so developers can focus on code and development rather than the infrastructure pieces.
"We want to make sure every employee can access the same data we use to make decisions and measure key metrics for each product and feature we have. Having a BigQuery data warehouse means we can handle large volumes of data and concurrent queries, so that we can focus on query optimization rather than scalability."
—Sanjeev Jaiswal, Senior Director of Engineering, CarousellImproving product performance with real-time insights on BigQuery
Carousell receives more than 150 million searches every month. Marketing campaigns and publicity events can result in a spike in traffic. As the company expands regionally, the number of visitors and searches will likely increase exponentially. For a positive user experience, Carousell keeps a close tab on application performance to prevent issues that affect service.
In general, the team uses BigQuery to store everything from structural schemas with its backend services, to event logs to analyze user behaviours. These data are pulled on a scheduled basis or through streaming pipelines for real-time reporting.
The SRE team can gather data such as error rates and latency from more than 150 microservices in one place. The 10 engineering teams across the company can also monitor microservices performance using Looker Studio to visualize BigQuery data. For performance improvement, engineering leaders meet weekly to troubleshoot and resolve incident and site issues.
"We want to make sure every employee can access the same data we use to make decisions and measure key metrics for each product and feature we have," says Jaiswal "Having a BigQuery data warehouse means we can handle large volumes of data and concurrent queries, so that we can focus on query optimization rather than scalability."
In addition, the product team uses Looker with BigQuery to visualize marketing metrics such as customer retention rate and customer acquisition cost to understand customer behavior changes and fine-tune campaigns.
Solving business problems with data science on Google Cloud ML Engine
AI is crucial for Carousell's innovation in the consumer-to-consumer classifieds space. The company deploys machine learning (ML) models to improve user experiences, such as personalized product recommendations for buyers, image search and price suggestions for sellers, and fraud detection.
Training an ML model is compute-intensive, so Carousell uses Google Cloud for cost and performance optimization. The team processes the data they need on BigQuery, or rely on Apache Spark for certain use-cases. The data science team then trains their ML models with TensorFlow on Google Cloud, and Google Cloud ML Engine. Once the models are trained, they are tested on live experiments by bucketing users in segments to see the performance where engineers can then decide on whether to roll out or disable them.
"With Google Cloud, the team can spend up to 70% of their time working on major projects such as creating a configuration that allows us to reproduce the entire Carousell app environment for testing, experimentation, and innovation."
—Sanjeev Jaiswal, Senior Director of Engineering, CarousellPromoting a culture of continuous learning and collaboration with Google Workspace
For productivity and prompt, reliable communication within its business, customers, and stakeholders, Carousell continues to rely on Google Workspace. Every month, Carousell organizes an engineering day with different topics where employees can join in person or via video-conferencing on Google Meet. Engineers across different time zones can view each other's availability and schedule meetings on Calendar to discuss project requirements and other development work. "The nice thing about Google Workspace is that it's an inexpensive set of tools that enables you to scale email and other productivity tools in a cost-effective way," says Jaiswal.
Jaiswal believes that Google Cloud will continue to underpin Carousell's quest to become a world leader in classifieds marketplaces, delivering a solid foothold in ASEAN and a presence in the rest of the world. "Google Cloud will continue to be the foundation of all our engineering work and allow us to scale applications to users worldwide," he says.
Tell us your challenge. We're here to help.
Contact us
About Carousell
Carousell is an online classifieds marketplace that lets anyone buy and sell items. Launched in August 2012, Carousell began in Singapore and is now one of the world's largest and fastest-growing marketplaces in eight markets across Southeast Asia, Taiwan and Hong Kong. With a focus on sustainability, Carousell launched its Carousell Green initiative to encourage customers to buy and sell pre-owned items and minimize waste.