Lightricks: Creating and scaling state-of-the-art content creation tools with Google Cloud
About Lightricks
Lightricks creates content creation tools, including its flagship products Facetune2, Videoleap and Photoleap. Its apps have been downloaded over 500 million times and it has over five million paying subscribers worldwide.
Tell us your challenge. We're here to help.
Contact us
About DoiT International
DoiT is a global tech consultancy that develops the technology and expertise needed to solve complex cloud challenges.
Lightricks harnesses the power of BigQuery and Dataflow to ingest billions of events per week, without impacting the user experience.
Google Cloud results
- Ingesting around a billion events a day with BigQuery and Dataflow
- Developing a working containerized infrastructure on Google Kubernetes Engine in weeks
- Enabling users to prepare scheduled queries minutes before an event is registered and sent
- Making compute resources available whenever they are needed
Processing vast amounts of data without inhibiting the UX
Content editing was once a time-consuming and tedious process. But since its launch in 2013, Israel-headquartered Lightricks has been on a mission to bring intuitive content-editing tools to professional video makers, graphic designers, and web builders, with its leading products that include Facetune, Videoleap, and Photoleap. “We build apps that harness powerful technology to make great user experiences,” explains Yaron Inger, CTO and co-founder of Lightricks.
“We have tools, for example, that enable our users to whiten their teeth in a photo, and to perform these types of functions, we build sophisticated algorithms that understand how to recognize what you're trying to edit and then automatically perform the edit,” says Inger.
“Google Cloud had the tools to empower our business intelligence, developers, and data science teams, and to put more functionality into the hands of our subscribers.”
—Dror Gilboa, Data Platform Team Lead, LightricksLightricks quickly found a market for its apps. It grew fast, and its original system, which relied on a hybrid of cloud services teamed with on-premises infrastructure for GPU usage, struggled to meet demand and to support increasingly complex machine learning and analytics needs. To enhance its unique selling proposition and its user experience and to enable unlimited scale, the team considered all the options and decided that Google Cloud was best placed to help it meet its ambitious goals.
“Google Cloud had the tools to empower our business intelligence, developers, and data science teams, and to put more functionality into the hands of our subscribers,” explains Dror Gilboa, Data Platform Team Lead at Lightricks. “BigQuery was one of the first tools we used, and it allowed us to collect analytics and really helped us improve the app’s user experience.”
“BigQuery was one of the first tools we used, and it allowed us to collect analytics and really helped us improve the app’s user experience.”
—Dror Gilboa, Data Platform Team Lead, LightricksLetting BigQuery take the data strain
The Lightricks apps ingest and analyze a lot of largely mobile data (such as user behavior) from customers, often in near real time. “We have business intelligence use cases, to optimize online ad campaigns, for example, where we need to create reports on a few terabytes of data and act on it almost instantaneously,” says Gilboa. “BigQuery, together with Dataflow, automates the ingesting and analyzing of these. This makes the ingestions of thousands of events per second very easy to perform at scale. The data connected to user behavior goes through Dataflow and is then ingested into BigQuery. It is automatic and allows us to look at the data from different angles at a very high scale.”
Lightricks is a big fan of the autoscale feature of Dataflow. With previous third-party systems it had used for uploading data, it often found data blocked because it had reached its limit. “We’re always increasing the number of apps and events we run,” Gilboa says. “We’re ingesting around 10,000 events per second or a billion a day. And we now have a platform that supports that growth by design.
“We can, for example, find a great user who came from a particular marketing campaign and then, using Dataflow, instantly target more users via the same route. Automating this work takes the strain off our in-house team.”
Letting a small team think big
Lightricks often refers back to the possibilities that Google Cloud opens up for its small workforce. Vika Kam, VP Engineering Creators Services and Community at Lightricks, reveals: “After only a few weeks with just a few engineers and DevOps people, we had a working Kubernetes infrastructure on Google Kubernetes Engine. We would never have achieved this with our old system; we always had problems and needed more people to maintain the infrastructure.”
Most of these issues resulted from having no separation between storage and compute, explains Inger, and infuriatingly they always seemed to arise at inopportune times. “Once we were trying to close a funding round and we needed loads of reports sent to future investors and the cluster shut down, and one Christmas when everyone ran queries to fetch data and make business decisions, it couldn’t cope either.”
Previously the infrastructure was an obstacle to getting things done, now it’s an enabler, says Inger. “All these services reduce friction, allowing our developers to focus on delivering business value to our users instead of maintaining our infrastructure. We leave that to Google Cloud now, and what’s great is Google Cloud is constantly adding features to GKE, and we get to use them without having to build them.”
Amplifying machine learning
GKE has enhanced Lightricks’ machine learning capabilities too. Lightricks began experimenting with training machine learning models in the cloud in 2014, but as the need grew, the team had problems with compute availability. This changed following the move to Google Cloud. “The compute capacity we get from Google Cloud is much better,” says Ofir Bibi, VP Research at Lightricks. “Compute resources are there whenever we need them. We initially created another Kubernetes cluster and used that alongside another cloud vendor, but the team shifted out of choice to Google Cloud, because it was smoother and more available.”
Inger adds that GKE is cost-effective too. “GKE means we now have fewer people investing their time in configuring clusters and upgrading them,” he says. “In days you can have a cluster set up, and be building Docker images from their repositories and set deployment pipeline.”
Lightricks has worked with tech consultancy DoiT International since 2016, and Bibi says DoiT has been particularly supportive of its machine learning program. “When, for instance, we wanted to create a cluster on GKE and attach it to our machine learning systems, DoiT ensured that our data lakes that we use for research and our on-premises compute functionality synched. We have an elaborate machine learning construction and DoiT provides ongoing support for everything from architecture to problem-solving.”
Lightricks’ marketing, product optimization, and recommendation engine teams are all creating machine learning models on Compute Engine, but they are slowly migrating to managed services on Vertex AI so that they can scale their models even faster. These machine learning models will recommend posts to serve to users on the app feed, so that they are fed the most relevant templates and tutorials and ingest analytics events to optimize user interactions such as notifications.
“We will be sending more data and improving the data science and machine learning models that we run. With the support of Google Cloud, we know that we can achieve this quickly, cost-effectively, and at scale.”
—Yaron Inger, CTO and co-founder, LightricksSecuring third-party services
As Lightricks gets into more backend procedures and serving and inference for recommendation systems, it is building services on top of open-source technologies. Kam really appreciates the number of services built on top of standard open source technologies on Google Cloud and how easy it is to integrate with third-party services. “Not only does Google Cloud make it easy for us to configure our third-party services, such as Cloudinary or Elasticsearch, it does this by providing us with the means to forward traffic outside of our private network without exposing anything to the public Internet. We trust it to keep us safe.”
Being able to continue developing and adding to its app line, while keeping everything secure, is Lightricks’ ambition for 2022. Inger concludes: “We face a dramatic shift in the marketplace. Our creators want to grow their audiences, get more traction, and monetize their content. To support this, we’re building on top of our existing content creation tools to keep offering state-of-the-art services and to develop our creators’ platform.
“And the major growth for us in 2022 will be in the backend services that we provide to users, like the ability to have a shared profile across apps and to upload media and share it. This means we will be sending more data and improving the data science and machine learning models that we run. With the support of Google Cloud, we know that we can achieve this quickly, cost-effectively, and at scale.”
Tell us your challenge. We're here to help.
Contact us
About Lightricks
Lightricks creates content creation tools, including its flagship products Facetune2, Videoleap and Photoleap. Its apps have been downloaded over 500 million times and it has over five million paying subscribers worldwide.
About DoiT International
DoiT is a global tech consultancy that develops the technology and expertise needed to solve complex cloud challenges.