How SEEN scaled output 89x and reduced GPU costs by 66% using Google Kubernetes Engine

At SEEN, we’re in the personalized video business.

We serve a wide range of clients who need to render and stream a high volume of unique, high definition videos that leverage data points — like name, gender, purchase history, etc. — to speak directly to their customers and constituents.

For some of these campaigns, we need to render and stream hundreds of thousands — or millions — of videos in just a few days, or even in just a few hours. To do so, we leverage an adaptable, scalable, and efficient cloud-based architecture built on Google Cloud and Google Kubernetes Engine (GKE).

In this blog, we’ll walk you through:

• Why we needed to replace our legacy bare metal architecture with a new cloud-based architecture built on Google Cloud and GKE

• Why we chose Google Cloud as our cloud services partner

• How Google Cloud helped us design and implement our new architecture, and what it looks like

• The quantitative and qualitative benefits we’ve experienced leveraging Google Cloud and GKE

Let’s dig in.

Our challenge: Rendering personalized videos at speed and scale

Every startup needs an ambitious goal.

At SEEN, we want to render and stream millions of individual personalized videos to millions of individual people in just a few seconds.

This goal is ambitious, but it isn’t just another “moonshot” designed to sound impressive — achieving this goal is critical to our company’s growth and future.

Here’s why.

When we first launched SEEN, we attracted smaller clients who needed to render and stream a relatively low volume of personalized videos.

To serve these clients, we built a basic architecture based on colocated machines. Our system worked — it could render and stream thousands of cinema-quality personalized videos sent over the course of a few weeks— but it was highly manual and time consuming to operate, and it slowed us down and didn’t scale very well.

This legacy architecture became a real problem as we grew and began to attract larger companies with larger projects. To serve these new clients we needed to generate and stream many more personalized videos at a much faster rate. For example, one client needed 1.8 million videos in a few days, while another — a fitness app — needed 100 million videos in one day. 

Projects at that speed and scale were impossible to consider with our legacy architecture, but to grow SEEN to the next level we needed to find a way to capture these enterprise clients and deliver on these types of projects. To do that, we needed to rebuild our old system from the ground up with a modern, efficient, and scalable cloud-based architecture.

Here's how we did it.

Searching for a new partner: How (and why) we chose Google Cloud

As soon as we realized we needed to rebuild our fundamental architecture we began to search for a cloud services provider. We knew that finding the right partner — with the right products and support — could solve a lot of problems for us and accelerate the process of designing, building, and deploying our new video rendering system. 

To start our search, we drew up a list of what we were looking for in an ideal cloud services partner. Our list included:

• The basics, like sustainability, security, and a local presence in Europe to help mitigate GDPR and other concerns (most of our clients were in Europe at the time).

• Next-gen hardware like NVIDIA GPUs, Kubernetes products, and emerging technical solutions like GPU time sharing and with more NVIDIA products on the roadmap. 

• White-glove support with direct collaboration and guidance to help us solve both general and niche problems in our system.

This last point was key to deciding which service provider we would choose. At the time we had a small-but-mighty team of four developers. While they had significant expertise with the cloud and rendering engines, at the time they just didn’t have the bandwidth necessary to bring our vision to life. 

With these requirements in hand, we reached out to cloud services providers, including Google Cloud.

We met with other providers, but Google Cloud felt different from the start. We met with them in-person, where we held productive conversations about our existing legacy architecture. We presented our new architecture’s design, and they provided insights on how one of their solutions — Google Kubernetes Engine (GKE) — would fit well within it.

Looking back, the feedback and treatment we received from Google Cloud was miles ahead of other vendors. It felt like they really understood our use case, and really wanted to partner with us one-on-one to bring our vision to life. In addition, they had the right product suite for our needs and offered us early access to features that would fit our scalability needs such as emerging GPUs from NVIDIA. 

The choice was clear. We selected Google Cloud and got to work.

Our new solution: How we render personalized videos today

 Google Cloud’s hands-on support and personalized attention didn’t stop after we signed our contract. Their teams have remained hands-on throughout the entire process of designing, deploying, and extending our new architecture.

As we built our new system, Google Cloud consultants:

• Constantly reviewed and commented on our architecture designs and ongoing implementation process

• Pointed us to products that became key components of our new architecture, such as GPU timesharing

• Provided input as we solved small problems like configuring NVIDIA drivers to work with Kubernetes, and rewriting parts of our rendering engine to leverage Google Kubernetes Engine (GKE)

• Continued to give us early previews of upcoming Kubernetes and GPU features, as well as their newest NVIDIA chips and machines 

Our engineers worked hand-in-hand with Google Cloud's consultants to redesign our architecture from the ground up. Ultimately, we built on Google Cloud and created a cloud-based architecture capable of generating unique, personalized videos at a scale and speed we never achieved before. 

While our new architecture is proprietary — and we need to keep it close-to-the-chest for competitive reasons — we can share a few of its key technical components. 

• We created worker pods that each perform a single rendering task at a time, and pull their jobs from Pub/Sub.

• We use GKE’s auto-scaling to rapidly scale from 1 to a virtually infinite number of nodes, and tailor their performance at the level of individual workloads. This gives us granular and responsive control over the compute power we deploy (and over our costs).

• We use GPU time-sharing between pods to leverage our compute resources as efficiently and effectively as possible — increasing our average GPU utilization by 1.6x and lowering our costs by 66%. 

By leveraging these services, our system now runs much faster, and is more efficient, scalable, reliable, and adaptable than before. With it we’ve transformed the products we offer, the scale and speed we can promise, and the clients we can serve — and it’s already delivered some significant technical and business outcomes. 

What we've achieved by partnering with Google Cloud

By partnering Google Cloud's consultants and rebuilding our architecture on Google Cloud and GKE, we’ve achieved outcomes that include:

• Generating 8800% more videos rendered per hour. With our old system, we could only render ~6,000 videos per hour. In a recent test of our new system, we effortlessly rendered 540,000 videos in one hour (and we know it's capable of rendering far, far more in that time frame).

• Developing enterprise-class capacity. With our new system, we can now render and stream millions of personalized videos at speed and scale. With it we’ve been able to serve leading global companies like BMW, WWF, Redbull, Red Cross, Coop, ICA, and Action Against Hunger — and take on bigger projects, like sending 2+ million personalized videos for the largest food retailer in Sweden.

• Clearing bottlenecks to our business. Before, we could only generate videos for one or two campaigns at a time. Now, we can comfortably handle as many campaigns as we’d like at one time — which means we no longer need to turn down projects and have increased our revenue.

• Building and offering new products. We can expand our portfolio with powerful new products. For example, we can now build real-time personalized videos that stream seconds after the viewer inputs their data and presses “play” letting us scale new, dynamic products.

• Improving system visibility. We’ve increased our logs, alerts, and overall visibility into render status for our huge campaigns (something that was difficult with our legacy architecture).

In sum: By partnering with Google Cloud, we have laid the groundwork to achieve our most ambitious technical and business goals — and we have already dramatically improved how we render and stream a huge volume of cinema-quality personalized videos, and how we serve our biggest and most demanding clients.

Bring these results to your organization

Today, we’re building more of our fundamental architecture on Google’s Cloud services, and improving its core performance so we can scale our personalized video production to insane levels. Our big, ambitious goal no longer seems out of reach. In fact, we now believe we will soon be able to render our cinema-quality personalized videos in .2 seconds or less. 

To get there, we’re continuing our close partnership with the Google Cloud team. They are helping us develop new scaling strategies, giving us critical best practices to apply to our architecture, and guiding our development of new products like real-time rendering.

Together, we’ve transformed how our core products function — and you can do the same by partnering with Google Cloud’s teams, by building on Google Cloud and by using GKE. It’s worked for us — big time — and we know it can work for you too.

^{Special thank you to Alina Bylkova, Lekë Dobruna, Gabor Lossos, Vigan Sokoli, John Rowley, and Michael Ivanov.}