Oxylabs: Making effective operational decisions with illuminating data insights
About Oxylabs
Oxylabs is a market-leading proxy and web-scraping solution service for large-scale public data gathering. The company's mission is to make big data accessible to all businesses, with a strong emphasis on ethical business practice and legal compliance, and it has more than 1,000 clients.
Tell us your challenge. We're here to help.
Contact usOxylabs' on-premises data solution wasn't providing timely insights for informed decisions. Migrating to BigQuery gave Oxylabs the analytics for effective decision making for itself and its customers.
Google Cloud results
- Query response time down from days to hours
- Seamless data transfer from Google Ads and Google Analytics into BigQuery
- Reduced maintenance time means developers are free to focus on smart data analysis
Oxylabs gains operational insights with BigQuery
When city center outlets shut during Covid-19 lockdowns, people turned to the internet to source food, clothing, and everything else in between. While the resulting online retail boom has slowed with the reopening of physical stores, ecommerce is still predicted to make up 20.8% of total retail in 2023, rising to 23% in 2025, marking a 5.2% increase in just five years.
As more businesses move online, it becomes harder for companies to gather information about what their competitors are doing and what their customers want. Oxylabs is on a mission to help online retailers and other digital businesses (including cybersecurity, brand protection, and many more) overcome that challenge. By gathering publicly listed data from websites, Oxylabs helps companies to unearth hidden gems of business intelligence that can help them understand their markets and react quickly. "Companies need to have this visibility of market data to compete effectively," says Juras Juršėnas, COO, Oxylabs.
"For us, it was a matter of principle. If we were providing data to others to make better decisions, it stood to reason that we should have the same capability internally. When you're stuck in line waiting for information, you know that something's got to give."
—Juras Juršėnas, COO, OxylabsOxylabs realized that its on-premises solution was holding it back, and that if it was going to continue to scale up and add new products and features, it would need a scalable tech stack that allowed it to use its data to its full potential. Oxylabs turned to Google Cloud.
"For us, it was a matter of principle," Juršėnas observes. "If we were providing data to others to make better decisions, it stood to reason that we should have the same capability internally. When you're stuck in line waiting for information, you know that something's got to give."
Searching for a solution to fit Oxylabs' tech stack
As Juršėnas and his team searched for a cloud-based data solution to generate the insights the company required, they ran tests with multiple providers.
"We liked BigQuery a lot in terms of query speed, and the user interface is really simple to use," says Juršėnas. "BigQuery is also well integrated into the Google ecosystem, which means we can seamlessly transfer data from other Google products, such as Google Ads and Google Analytics, to BigQuery, saving us development time and resources."
Security was another important factor. Oxylabs prides itself on its ethical approach, scraping only publicly available web pages and strictly adhering to General Data Protection Regulation (GDPR). Google Cloud data security and governance features, such as its identity and access management features, reassured Juršėnas that BigQuery was the right choice for Oxylabs.
"We opened an account and started to try things out, and it all just worked," says Juršėnas. "The time-to-value was really impressive. There was no admin to get started, no extra management to make the most of the data. We could start off by setting up this small instance and then just expand on it as we went."
A new view of the business with powerful data insights
The company needs to maintain infrastructure in many countries around the world, so that its proxies can access locally restricted data, but found it difficult to see and analyze all of this expenditure in one place. Oxylabs now uses BigQuery to analyze company data and gain the kind of operational insights that were impossible with its previous data capabilities.
"Now, with BigQuery and its automated scaling, we have fact tables with over a trillion records and it takes virtually no time to transform or aggregate those," says Juršėnas.
"BigQuery allows us to do this sort of analysis very quickly, on an ongoing basis, without requiring lots of developers to maintain it. Leaving our developers free to focus on getting better at analytics, asking the right questions and preparing a smart analysis."
—Juras Juršėnas, COO, OxylabsOxylabs can also analyze the data from its services themselves to ensure that they are running efficiently. For example, if a retail customer is using its Scraper API product to gather data from a competitor's ecommerce site, Oxylabs can use BigQuery to analyze the response times and control the quality of results.
"BigQuery allows us to do this sort of analysis very quickly, on an ongoing basis, without requiring lots of developers to maintain it," says Juršėnas. "This leaves our developers free to focus on getting better at analytics, asking the right questions and preparing a smart analysis."
How customers benefit from a new data culture
According to Juršėnas, having easy access to all its data in one place has changed the culture at Oxylabs, allowing them to become more analytical.
It has also allowed Oxylabs to pass on more benefits to its clients by analyzing how customers use its services, and advising them on how they could make better use of them, as well as enabling Oxylabs to focus its marketing efforts to gain new customers.
"Once we had all this usage data, we also connected it to our CRM data and our web acquisition data, and it allowed us to solve much more interesting questions," says Juršėnas. "For example, we can run cohort analysis and see the effectiveness of our different marketing campaigns on the long-term value of the customer. And based on this, we can optimize our marketing to acquire more of these customers."
Getting information into the hands of those who need it most
Oxylabs has also begun providing new data services to its clients with the help of BigQuery. While many clients use an API to gain information from a particular web page, others have such a large number of queries that they would rather not use an API for each one. Instead, Oxylabs has begun using BigQuery to process the data and pass it on directly to the client on a regular basis, so that they can start working with it immediately.
Moreover, the time Oxylabs has saved in maintaining and operating its previous data stack can now be used cleaning and preparing data for its customers. Those customers can then spend more time using and analyzing that data and less time cleaning it, which not only helps Oxylabs to stand out from its competitors, but saves its customers significant time in DevOps.
"Now we have Google Cloud, it has given us new opportunities to provide different services to clients in terms of analyzing data. For us, it's about making sure everyone has all the necessary information to make better decisions. We think the world would be better that way."
—Juras Juršėnas, COO, OxylabsFor Juršėnas, BigQuery has ultimately enabled Oxylabs to keep pace with the growth of digital businesses, enabling it to streamline its business with valuable operational insights, while at the same time rolling out new services to customers that enable them to remain competitive.
"Before, we had one way of doing things that didn't allow us to do what we wanted with data," Juršėnas explains. "Now we have Google Cloud, it has given us new opportunities to provide different services to clients in terms of analyzing data. And because it is so easy to build new features with BigQuery, we plan to keep on developing more data-intensive solutions for our customers. For us, it's about making sure everyone has all the necessary information to make better decisions. We think the world would be better that way."
Tell us your challenge. We're here to help.
Contact us
About Oxylabs
Oxylabs is a market-leading proxy and web-scraping solution service for large-scale public data gathering. The company's mission is to make big data accessible to all businesses, with a strong emphasis on ethical business practice and legal compliance, and it has more than 1,000 clients.