Square Enix’s journey of building an AI-driven Customer Data Platform

Editor’s note: In this post, we’ll learn from Japanese video game and entertainment company Square Enix about its journey building a customer data platform with Google Cloud to enhance its first-party data strategy by collecting and connecting its marketing, player, and game data.

In today’s data-driven world, understanding who your customers are is more critical than ever. For game developers and publishers, in particular, gaining a complete picture of who your players can quite literally be a game changer — transforming games that never quite find the right audience into standout titles. Data can unlock a trove of valuable insights that help gaming companies gain a deeper understanding of their product discovery journey and player base, unlocking insights into playing style, playtime metrics and more to create engaging, personalized communications.

Unfortunately, traditional data analysis is often siloed with player information scattered across multiple systems and platforms, making it cumbersome and time-consuming to achieve a consolidated view of a player’s interactions and habits. That’s where Customer Data Platforms (CDPs) come in, acting as a central hub that can bring all your player data together and deliver a unified player profile that empowers game developers to truly get to know their players and create games that resonate. Developing a CDP for games is especially complex. This is due to the sheer number of players a game can attract and the massive volume of data each player generates.

When set up and configured correctly, CDPs are an integral part in driving player-centric experiences and game design across your business. Typically, CDPs comprise four key areas:

• Player data aggregation: Collecting, integrating, and storing all forms of player data, including platform preferences, in-game achievements, playstyle behavior, and social interactions.
• Cross-platform player identification: Unifying player data across different gaming platforms, devices (PC, console, mobile), in-game IDs, and email addresses into a comprehensive player profile.
• Game analytics and AI insights: Utilizing advanced AI and machine learning models to deliver out-of-the-box insights that power predictive player behavior, forecasting churn, in-game and outer-game engagement, and optimal communication strategies.
• Player activation: Utilizing preference based segmentation to engage players with targeted messaging across different channels including in-game notifications, emails and social media. Offering personalized catalog title recommendations to players can also help foster lifelong relationships.

While organizations are able to purchase many of these technologies off the shelf, there are many benefits to building your own CDP and creating a more tailored platform to meet your specific needs and use cases. In this post, we’ll share our own journey of how we built our own CDP with Google Cloud at Square Enix. 

Building an AI-driven CDP with Google Cloud

As an early adopter of data collection, analysis and machine learning, the Square Enix Analytics and Insights department has been at the center of bringing business value to our organization through data-driven game innovations, such as using telemetry data to discover new markets for games and inform patching decisions about areas where players get stuck.

Our European division — Square Enix West — faced a significant shift in data strategy in 2017, driven by the General Data Protection Regulation (GDPR) due to come into force the following year. Recognizing the changing landscape and policy around marketing data, our team, along with technology leadership, saw the need to focus on collecting and connecting our first-party data across multiple sources — thus began the journey to aggregate marketing, player, and game data under a single roof. 

Given that gaming is often a lifelong passion, the ability to analyze fan’s engagement level with Square Enix’s services over periods of five to ten years and across multiple data sources is crucial. Our vision was to craft a solution that would allow the team to conduct end-to-end analysis of gamers’ behavior, including touchpoints, purchases, gameplay and retention. We knew that the right CDP tool would be essential to service players at every step along their discovery and buying journey.

Using Google Cloud services, particularly serverless, we built a proprietary system called Single Gamer View (SGV), which delivers both CDP and customer relationship management features. The new solution unifies all of our databases, provides marketing automation, and feeds data back into marketing platforms to drive fan engagement.

We built pipelines leveraging PubSub and Dataflow to collect data into BigQuery for data processing. This data comes from sources across Square Enix, including game data, website data from Google Analytics 360, aggregated sales data, digital advertising data, and even metrics from emails and surveys. We run AI and machine learning models daily, leveraging Vertex AI for training and managing models for larger-scale tasks.

In addition to building SGV, the alignment of all team members — from marketing to analytics to engineering to machine learning experts — is key. Our projects begin with data analysts, who draft proposals and estimate the value of the project. Community managers and brand teams then review these proposals to ensure alignment with player needs. Data scientists, following an agile approach, start with rule-based segmentation, and later transition to more advanced machine learning methods once early strategies prove successful. Finally, our data engineering and data protection teams provide oversight throughout, ensuring data security and effective deployment. 

How SGV delivers AI-driven marketing

Today, the significance of SGV is set to grow even more, especially as we move toward the phasing-out of third-party cookies in 2024. This shift will highlight the importance of having robust internal systems for gathering and effectively using data to nurture relationships with fans. 

The value of having an integrated CDP is often poorly understood until it is tied into marketing activations, fan retention, and marketing activity modeling. Consequently, here are three core initiatives we championed from the beginning and continue to support.

1. Marketing automation and feedback loop
Having all our marketing and game data in one place allowed us to develop and train more advanced ML-powered personalization systems. For instance, we designed a marketing automation framework with a feedback loop to test out different marketing actions. The process uses a contextual bandit model pipeline built with AutoML Tables and helps drive our marketing email campaigns, encouraging better retention, demo trials, and survey responses. 

SGV also helped us build a self-optimizing recommendation system, which can detect when a player has reached a milestone in the game, compares them to similar players, and sends recommendations tailored to the individual. The system, which leverages BigQuery ML to make recommendations, receives feedback on whether the demo recommended was played and if so, how long the player played.

2. Fan onboarding and retention
With SGV, we have improved Square Enix’s ability to understand and nurture life-long relationships with fans. Game teams can create more personalized customer experiences, which better service players’ needs by leveraging ML-based segmentations powered by BigQuery ML to identify users by play style, class or character preferences, or content they find interesting. For example, we can now provide players who enjoy player versus player (PvP) conflict with specific news regarding PvP content updates. This personalization not only improves the user experience but also helps to deepen their relationship and retention with the game. In fact, our teams saw a significant increase in players returning to specific Square Enix games after implementing this new data-driven approach.

3. Marketing ROI challenges 
With evolving digital advertising standards and increased privacy regulations, particularly the phase-out of third-party cookies, digital tracking methodologies have become less viable. This has led to the resurgence of Marketing Mix Modeling (MMM). MMM is a statistical analysis method used to estimate the impact of various marketing tactics on sales and determine the effectiveness of each promotion.

Organizations without a centralized data repository would have to start an MMM project by first defining the necessary data, locating it, acquiring it, and then cleaning it. Using traditional data analysis methods, just preparing the data for modeling could take several months. However, as we have major data points inside SGV, data scientists can now quickly access data and develop proof-of-concept models. Additionally, leveraging tools like Supermetrics and Google Trends through Google Cloud, we can now pull more granular marketing data from our marketing sources and gain more valuable insights. 

Bringing gaming fan engagement to the next level with Google Cloud

Overall, SGV has been quite literally a game changer for Square Enix, helping us to increase player engagement and enhance marketing ROI measurement. By integrating Google Cloud tools, advanced analytics, and proprietary data, we have consistently delivered impressive results. Our overall email marketing performance has increased by 20%, with some campaigns seeing as much as a 150% increase particularly when our recommendation engine effectively addresses long-tail needs. We have also seen a 10% increase in game retention and completion rates. 

In addition, building a custom CDP tailored to our specific requirements has increased our work efficiency. Our teams benefit from an automated optimization system, allowing us to save time managing tasks like promoting catalog titles, sending retention emails, and collecting feedback surveys. 

The next step is to continue enhancing the management and scalability of our machine learning processes. We believe that effectively using Vertex AI will be crucial during this next phase, especially as we tackle the challenge of moving from experimentation to production. For example, we are already considering how to use Google Cloud’s new generative AI features in BigQuery to enrich and help automate our existing data processes. We are also keen to see how new data scientist-friendly features, such as managed notebooks and BigQuery DataFrames, can help us accelerate the development and initial adaptation of notebook code. While notebooks may not fully meet these standards on their own, these tools can simplify experimentation and transition to production without needing specialized expertise.

While the full impact of AI in the gaming industry remains to be seen, we are looking forward to seeing (and also shaping) a future where we can finally bridge the “last mile” and deliver new innovative experiences to our players. 

Interested in starting your own customer data platform journey? Learn more about how Google Cloud can help you prepare for a privacy-centric future.