Reduced 98th percentile query times from over 40 seconds to significantly faster loads for critical dashboards
Shifted BI team focus from "firefighting" report errors to strategic Gen AI initiatives
Enforced data quality for Conversational Analytics through automated SQL and description checks using Looker Continuous Integration
Carousell optimized its data operations by implementing a multi-instance Looker architecture and a robust CI/CD pipeline, significantly improving dashboard performance and data trust. This transformation enables faster executive decision-making and prepares the organization for a future driven by Conversational Analytics.
Carousell optimized its data operations by implementing a multi-instance Looker architecture and a robust CI/CD pipeline, significantly improving dashboard performance and data trust. This transformation enables faster executive decision-making and prepares the organization for a future driven by Conversational Analytics.
Optimizing performance and ensuring trusted data
Carousell, a premier classifieds business focused on Southeast Asia, reached a pivotal moment in its data journey as its reporting needs grew in scale and complexity. Initially, the team successfully utilized Looker to build extensive reporting and dashboards that served every department, from Finance to Marketing. However, the sheer volume of data—often involving tables in the hundreds of terabytes—presented a unique opportunity to optimize how Looker processed granular information. To maintain the high-speed experience stakeholders expected, Carousell recognized the need to refine their architectural strategy to better support large-scale, derived entities such as platform transactions.
The team identified that while LookML offered immense flexibility for defining business logic, shifting heavy computational tasks further "left" into the ETL layer would unlock even greater performance for their most critical dashboards. Previously, dashboards were connected to very heavy tables and complex LookML was used for heavy lifting, which impacted the speed of insights for business stakeholders. Furthermore, as the number of developers grew, Carousell sought to implement a more collaborative, governed workflow.
The goal was to move beyond a rapid, direct-to-production model where developers could push changes without PR approval—a practice originally intended to minimize turnaround time but which occasionally led to errors. Carousell aimed to transition toward a structured environment that utilized automated checks to ensure the highest data quality for executive leadership.
Looker, unlike other visualization tools, provides flexibility for developers and the ability to standardize metrics through LookML, which is crucial for trusted data. We decided to move towards pre-aggregating data and shifting heavy computation to ETL to focus LookML on visual and UI changes.
Shishir Nehete
Data Analytics Lead, Carousell
By evolving their setup, Carousell aimed to empower analysts with the deep exploration capabilities they loved, while providing the Executive team with streamlined, pre-aggregated strategic views that deliver insights at the speed of business.
A dual-instance approach for speed and innovation
To resolve the tension between deep exploration and rapid reporting, Carousell implemented a multi-instance architecture. They designated a new, dedicated Looker instance strictly for aggregated dashboards, executive reviews, and platform health monitoring. This ensured that critical Weekly Business Reviews were powered by pre-aggregated tables in BigQuery, guaranteeing fast performance for leadership. Meanwhile, the legacy instance was retained to allow stakeholders to explore granular data and experiment with new metrics without impacting the speed or stability of production reports.
The new CI/CD capabilities have allowed the team to focus more on Gen AI initiatives rather than 'firefighting'... The new setup, including description fields and production connection checks, ensures higher quality data and makes it very unlikely that an Explore will take a long time to load.
Wei Jie Ng
Business Intelligence Engineer, Carousell
A cornerstone of this new architecture was the implementation of a rigorous Continuous Integration and Continuous Deployment (CI/CD) pipeline on the new instance. Wei Jie Ng, a BI Engineer at Carousell, integrated tools like Looker CI and LAMS (Look At Me Sideways) to automate quality control. Unlike the old instance, where developers could push directly to production, the new pipeline enforces mandatory checks. For example, the system now automatically catches SQL syntax errors—such as a misspelled country name—and blocks the merge until fixed, ensuring that broken code never reaches production dashboards.
The governance controls go beyond simple syntax checks. The team utilized LAMS to enforce strict style guides, requiring descriptions for every dimension and ensuring all models connect to the production database.
This strict governance was a strategic move to prepare for Generative AI. By enforcing detailed metadata and descriptions, Carousell is building a semantic "business layer" or "cubes" on top of their data pipelines where heavy logic is defined. This ensures that when Looker’s Conversational Analytics interacts with the data, it is aware of the whole entity, allowing it to accurately interpret natural language queries like "give me monthly revenue in USD for a specific category".
The results of this transformation have been profound. Query times dropped significantly, with the 98th percentile times reducing from over forty seconds to less than ten, enhancing the experience for business users. Culturally, the stability provided by CI/CD has shifted the conversation during business reviews. Instead of asking "Is this a data issue?" stakeholders now focus on "What is causing these changes in the metrics?".
Crucially, this architecture has freed the BI team from constant "firefighting" and troubleshooting broken reports. Instead of fixing errors, the engineering team now focuses on deploying Gen AI initiatives. They are currently experimenting with Looker agents that can connect different Explores, such as joining transaction and revenue tables, to tailor insights for specific teams like Sales. Carousell anticipates a significant change in stakeholder behavior, moving from manual dashboard clicks to direct questions via the conversational interface, empowered by a trusted, high-speed data foundation.
Carousell is a classifieds marketplace in Southeast Asia that inspires the world to start selling and buying secondhand.
Industry: Retail
Location: Singapore
Products: Google Cloud, Looker, BigQuery
