Choreograph: Scaling advertising audience solutions globally with Google Cloud
About Choreograph
Choreograph™ is WPP’s global data products and technology company fueling growth for the world’s most admired brands. Built on the most advanced global data foundation, Choreograph is powered by 850 technologists, product developers, and data scientists that orchestrate data capabilities across GroupM and WPP into an end-to-end data enablement solution.
Tell us your challenge. We're here to help.
Contact usWith big changes in the industry, advertisers must adapt fast. Choreograph (part of WPP) used Google Cloud for infrastructure and Bigtable and Dataproc to deliver its cutting-edge adtech platform rapidly to global agencies.
Google Cloud results
- Rearchitected for the cloud, shifting highly clustered systems to managed services
- Shifted APAC, AMIA, and North America data centers to the cloud in four-month sprints
- Right-sized their system requirements, cutting their initial cloud costs by 50%, all without having to re-platform their applications
50% cloud cost savings
Evolving adtech at the world’s largest advertising group stresses on-premises infrastructure
In the world of advertising and marketing, executing campaigns effectively means reaching targeted audiences who are most likely to act, wherever they are on the web. For Choreograph™—WPP’s global data products and technology company that is fueling growth for the world’s most admired brands, this means providing their agencies with world-class technology to reach those audiences. Therefore, Choreograph developed its data enrichment and modeling product, data iq, formerly known as mPlatform, which helps advertisers reach audiences via real-time bidding (RTB) on a demand-side platform (DSP).
Data iq ingests anonymized data from publishers, campaign data, and third-party data, such as websites visited, campaign clicks, impressions, and even purchases made on other platforms. Choreograph uses it to build profiles to determine audience and market segments that will deliver the highest probability of action on a media ad.
Data iq was designed with global growth in mind. Its data pipelines could be reused and reapplied across different regions, and it provided support for varying data flow across 62 countries. Choreograph built four data centers around the globe, successfully creating an on-premises infrastructure that could scale without requiring more staff to support more data—at least for a while.
The situation changed when Choreograph deployed data iq into WPP’s worldwide network of agencies. Suddenly, the customer base dramatically increased, and more hardware was necessary to support the shift. When an unacceptable delay of as much as four weeks to ship, rack, stack, and implement hardware was imminent, Choreograph began to look at moving to the cloud.
“As hardcore infrastructure people, we wanted to talk bits and bytes and networks, not services. With Google, we reached that level right away. We talked about cores and memory and network throughput. That got us excited.”
—Bob Hammond, CTO, ChoreographGoogle Cloud: A skeptic’s best choice
With millions of data transactions per second flowing into data iq that needed to be processed in near real time to update the DSPs, the system ran very hot, all the time. Therefore, the Choreograph team had some doubts about the cloud. “Our architecture was really close to the hardware. We didn’t virtualize. My initial impression was running on the cloud was never going to work,” explains Choreograph CTO, Bob Hammond, who led the development of data iq.
Nevertheless, Choreograph started evaluating cloud options. Quickly, Google Cloud emerged as the right cloud for the company. Choreograph already had a strong partnership with Google, and Google understood the Choreograph team’s communication preferences. “As hardcore infrastructure people, we wanted to talk bits and bytes and networks, not services,” Bob recalls. “With Google Cloud, we reached that level right away. We talked about cores and memory and network throughput. That got us excited.”
Another determining factor was the fact that with Google Cloud, there are no service packages. Service packages scale everything at once. By contrast, Google Cloud provides tuning flexibility. This flexibility was a necessity for data iq, which requires high-speed memory networks, but not as much CPU. With Google Cloud, Choreograph could tune down cores and crank up memory specific to its needs without having to acquire thousands of extra CPUs as part of a service package. This led to 50% operational savings over the course of the next 12 months, allowing this investment to be diverted into projects focused on the future of identity and the depreciation of cookies. And because Choreograph used containers, Google Kubernetes Engine (GKE) was an obvious choice for the managed environment they needed for deploying, managing, and scaling containerized applications.
“Boom—we flipped the DNS, everyone rolled over one night, came in Monday morning, and they had no idea they were even running on Google Cloud.”
—Bob Hammond, CTO, ChoreographFrom on-premises naysayers to cloud rainmakers
Choreograph’s first step was to prove that Google Cloud would be able to support their high-demand system requirements. Bob and his team took the data center for Choreograph’s largest region, North America, and mirrored its exact setup in Google Cloud. Everything went off without a hitch, and they had the evidence that they could run data iq’s highly clustered architecture on the cloud.
With this proof, Choreograph set out to rearchitect and reengineer the cloud implementation for even greater efficiency. The large, non-standard cluster systems used with data iq required software that was not designed for the cloud. Therefore, the cost to run them on Google Cloud was extremely high.
Bob assembled a team of top engineers across disciplines, and named it Rainmaker. They worked alongside support from the Google Cloud Professional Services Organization (PSO) to migrate Choreograph’s largest clustered systems to managed services and to move from a number of third-party technology stacks to Google Cloud-based services. The team from Google Cloud PSO helped with rearchitecting, final architecture and security setup, and more as Choreograph moved from Couchbase to Cloud Bigtable and from Hadoop to Dataproc. The result? Choreograph saw a decrease in costs of over 50% by migrating from the data centers.
“The most important piece of advice I would give other organizations on the same journey is to make sure you go back and right-size your deployments in relation to your business needs.”
—Bob Hammond, CTO, ChoreographFull migration at top speed takes down the data center—and costs
With everything reengineered for the North America region, Choreograph began the full migration. Starting with their smallest region, APAC, the Rainmaker team moved each of the other data centers to the cloud. Each migration was on a four-month schedule, or two 60-day sprints. In the first month, the team installed the new system, which was the rearchitected platform migrated from Couchbase, Hadoop, and more to Google Cloud services. The second month was spent synching all the data. In the third month, the team flipped the DNS and let it run concurrently with the data center in case there were issues. In the fourth month, the Rainmakers tore down the data center.
“It was beautiful,” remembers Bob. “Boom—we flipped the DNS, everyone rolled over one night, came in Monday morning, and they had no idea they were even running on Google Cloud.”
With a successful shift and reengineering, Choreograph immediately saw lower system costs. Today, in the years since implementation, Choreograph has been able to save even more. Thanks to the flexibility of Google Cloud, Choreograph right-sized its system requirements, adjusting settings based on core, memory, and network consumption. This cut their initial cloud costs in half, all without having to replatform their applications.
“The most important piece of advice I would give other organizations on the same journey is to make sure you go back and right-size your deployments in relation to your business needs. Sometimes, it’s only possible to see what your configurations need to be after your applications are running in production environments,” advises Bob.
“Over the past few years, we radically changed how we do what we do. It made our engineering and technical operations and data science groups that much better operationally. We never could have done that on a physical infrastructure.”
—Bob Hammond, CTO, ChoreographMeeting the future with the power of cloud
The advertising and marketing sector is rapidly changing. By migrating to a Google Cloud infrastructure, which always stays up-to-date, Choreograph has positioned itself for success in its market.
“Over the past few years, we radically changed how we do what we do,” says Bob. “It made our engineering and technical operations and data science groups that much better operationally. We never could have done that on a physical infrastructure.” Now Choreograph is exploring other Google Cloud managed services for their platform, such as BigQuery for data warehousing.
Tell us your challenge. We're here to help.
Contact us
About Choreograph
Choreograph™ is WPP’s global data products and technology company fueling growth for the world’s most admired brands. Built on the most advanced global data foundation, Choreograph is powered by 850 technologists, product developers, and data scientists that orchestrate data capabilities across GroupM and WPP into an end-to-end data enablement solution.