ITV: Delivering a higher quality live viewing experience through data
About ITV
ITV is an integrated producer broadcaster, and the largest commercial broadcaster in the United Kingdom, with a 45 percent share of the UK TV advertising market and 27 million registered users on ITV Hub, its online video-on-demand (VoD) service.
Tell us your challenge. We're here to help.
Contact usITV, the United Kingdom's leading commercial broadcaster, created a custom viewer analytics solution with managed services to track unprecedented, unpredictable audience numbers in close to real time.
Google Cloud results
- Built a custom viewer analytics solution in three months
- Alerts on live online viewing issues within two minutes, instead of ten
- Scales automatically to match unprecedented audiences online
Scales real-time analytics on 1M viewers
The summer of 2018 was an extraordinary time for UK broadcaster ITV. An average of almost four million viewers a day tuned in to its hit reality show Love Island, and as the England football team progressed further through the World Cup than it had in decades, record-breaking audiences of up to 27 million people watched their matches live on ITV.
For ITV's online service ITV Hub, that meant extra pressure to help ensure that shows streamed correctly to an increased audience. Broadcasters typically rely on data analytics to track the performance of their platforms and use that feedback to optimize services for viewers. At ITV, however, that conventional approach to data analytics could not respond fast enough in live broadcasts watched by millions nationwide, as Simon Forman, Head of Behavioral Data Engineering at ITV explains:
"We collect data on the video quality people receive on the platform, and other statistics, and then we make that information available to different teams within the company. Usually, our system alerts us to problems like signal quality issues within ten minutes of them occurring, but the audiences for live programs this summer were so large that we needed to track the service on ITV Hub much more closely. Operational teams didn't want to wait ten minutes to spot a problem, they wanted to know straight away."
"We now do data analytics like no other broadcaster, because we've built it all ourselves, for ourselves, without worrying about infrastructure. That's made us far more agile. At other major broadcasters, switching analytics tools is taking more than a year. We did it in three months."
—Andy Burnett, Director of Direct to Consumer Technology and Operations, ITVThat's why Simon and his team chose to build CROCUS, a solution on Google Cloud, using managed services that scale automatically to match demand. Usage data from viewers is queued in Cloud Pub/Sub to compensate for irregular delivery, before feeding into Cloud Dataflow, which transforms it and feeds it back to Cloud Pub/Sub, ready for presentation on a dashboard, all in as little as 30 seconds.
Cloud Functions provides serverless application hosting, meaning the team only needed to write the minimum code necessary for the solution to operate. In addition to appearing on a live dashboard, viewer statistics and reports are sent through to BigQuery to be stored with raw information, for later analysis, auditing, and debugging purposes.
In just three months, ITV was able to create a new analytics solution that could scale to match demand, however unpredictable. This meant that the company could process and present viewership data in close to real time, regardless of how many viewers tuned in.
"The sheer volume of viewers was much higher than usual, while the data we received was needed sooner and subject to much greater scrutiny," says Simon. "For the England semi-final, one million people tuned in to ITV Hub, many times more than we'd previously had for a live broadcast online."
Scaling analytics to match massive, unpredictable demand
Online media broadcasters rely on data to provide feedback on their services, but that feedback typically arrives with a considerable delay, from ten minutes to one day after problems occur. For large, live online audiences, ITV needed a solution that could collect data on viewer behavior much more quickly, to be ready for analysts within seconds.
"Our analysts were very attached to their own set of analytics tools, which they wanted to use on top of BigQuery. But after a while, they began querying data directly through BigQuery itself. It's easy to use and fast on complex queries, and they use Looker Studio to display results."
—Simon Forman, Head of Behavioural Data Engineering, ITV"Our analysts look for any deviation from the normal pattern of viewing behavior," says Simon. "A normal football game looks like a steady build-up of audience size, that dips at half-time as people go off to make tea, and then returns for the second half. Love Island has its own, very recognizable profile. People are creatures of habit, and if there's any change in that profile, we know that something's wrong."
Because it was impossible to know in advance just how many people would tune in to each program, the data analytics solution had to be ready to scale, whether for 200,000 viewers, or 2 million. Trying to preempt demand ran the risk of not providing enough capacity to process signals in time, or providing too much, which would mean spending money on idle capacity.
"The beauty of the system is that it is so scalable," says Simon. "We don't have to make any changes ourselves for all of these components to handle extra data, no matter how much demand changes, or how suddenly. Everything scales up automatically, then scales back down again."
For Andy Burnett, Director of Direct to Consumer Technology and Operations at ITV, CROCUS gives ITV an edge over its competitors in the industry:
"We now do data analytics like no other broadcaster, because we've built it all ourselves, for ourselves, without worrying about infrastructure. That's made us far more agile. At other major broadcasters, switching analytics tools is taking more than a year. We did it in three months."
A simpler, more powerful way to analyze data
By making the most of managed services on Google Cloud, ITV can run CROCUS with minimal effort. "We don't need a big team, because we're not managing physical servers or even virtual servers," says Andy. "We just click a button and away it goes. The maintenance overhead is tiny compared to other services in the video platform business."
"CROCUS has been pivotal in advancing our business intelligence. We have created the ability to democratize data to our organization from a descriptive and diagnostic perspective. Now we can venture into the world of predictive and prescriptive analytics, where we see huge opportunities for ITV."
—Rob Hobson, Director of Business Planning and Insight, ITVThe analytics solution has delivered unexpected benefits, too, especially through BigQuery, which was originally set up only to store data for later analysis, and mainly chosen for its simple connection with Cloud Dataflow. "Our analysts were very attached to their own set of analytics tools, which they wanted to use on top of BigQuery," says Simon. "But after a while, they began querying data directly through BigQuery itself. It's easy to use and fast on complex queries, and they use Looker Studio to display results."
For Rob Hobson, ITV's Director of Business Planning and Insight, the straightforward accessibility of the new data platform marks a major development for the role of business analytics at the company. "CROCUS has been pivotal in advancing our business intelligence," says Rob. "We have created the ability to democratize data to our organization from a descriptive and diagnostic perspective. Now we can venture into the world of predictive and prescriptive analytics, where we see huge opportunities for ITV."
Working towards a unified, cross-platform view of the audience
By making data more reliable and minimizing maintenance, CROCUS means teams have fewer distractions and can spend more time on the tasks they were hired to do. "It's the job of our Behavioral Data Engineering team to make sure that data is true, good quality, clean, and usable, and it's the job of our Analytics team to go and do analytics," says Andy. "That seems like an obvious thing to state, but before we built CROCUS the Analytics team was constantly checking and questioning the reliability of data as it had gone through our system. It was demoralizing for them. But now we're attracting good analysts and data scientists because they see the technology and say, 'Great – I'm in.'"
Now ITV is working to collect data on normal broadcast television usage, as well as online. "One million people watched the England semi-final this summer," says Simon. "But 26 million watched it on our traditional television broadcast services. Our next step is to bring a large percentage of that traditional TV broadcast audience into our analytics, which we can then use to make recommendations and personalize features for viewers, based on their habits and the way they like to watch TV. We're quite confident that the Google Cloud tools we're using will handle that very well, even at that scale."
"CROCUS is in rude health," says Andy. "Now it's time for it to power our advertising and linear businesses. Other broadcasters are following a well-trodden, conventional path on data analytics. We've gone off on a tangent. Within UK broadcasting, I think we've stolen a march on our competitors."
Tell us your challenge. We're here to help.
Contact us
About ITV
ITV is an integrated producer broadcaster, and the largest commercial broadcaster in the United Kingdom, with a 45 percent share of the UK TV advertising market and 27 million registered users on ITV Hub, its online video-on-demand (VoD) service.