Ubisoft: Using machine learning to transform our understanding of the ancient world
About Ubisoft
French video games company Ubisoft is known for its original and immersive world-creating franchises, including Watch Dogs and Assassin's Creed.
Tell us your challenge. We're here to help.
Contact us
About Psycle
Digital agency Psycle Interactive helps businesses with the adoption of new technology and has built everything from elearning platforms to chatbots.
Ubisoft built a toolset to help Egyptologists translate Middle Egyptian hieroglyphics, crowdsourcing data with a drawing tool using Google Firebase and recognizing hieroglyphics using Cloud AutoML.
Google Cloud results
- Builds a machine learning model capable of recognizing hieroglyphics using Cloud AutoML
- Creates tools replicating the translation process used by Egyptologists with a platform designed for deployment on Google Cloud
- Enables the crowdsourcing of data using a custom-built drawing tool on Google Firebase
80K glyphs contributed by crowdsourcing in one night
What if video games could not only build new worlds for their players, but transform our understanding of the world we live in? By creating innovative, exciting games franchises such as Far Cry and Assassin's Creed, Ubisoft has grown to become the third largest independent video games publisher in the world with a projected turnover of €2 billion in its fiscal year 2019. "We put human beings at the center of everything we do," explains Guillaume Carmona, VP Marketing, EMEA at Ubisoft. "Innovation is key, not only in terms of the games we produce, but also the way we do business. Beyond just providing entertainment, we want to enrich people's lives."
Assassin's Creed: Origins, the tenth installment in the series, came out in 2017 and was inspired by the landscape and history of the Ptolemaic period of Ancient Egypt. It was also the starting point for a project with Google ZOO. "For Origins, we used elements of history to build a world for our players. But we also wanted to contribute something back, and that was how the idea for The Hieroglyphics Initiative came about," says Pierre Miazga, Associated Director, Marketing at Ubisoft. "Working together with Google ZOO, we asked the question: what if we could find a way to help translate hieroglyphics using artificial intelligence?"
"Rather than just build a data repository, we wanted to create a toolset that could then be adapted and developed by both the AI and the Egyptology communities. This kind of research project is all about engaging people and pushing the limits of what is possible."
—Guillaume Carmona, VP Marketing, EMEA, UbisoftDuring the development of Origins, Ubisoft involved Egyptologists in order to incorporate accurate historical details. For the Hieroglyphics Initiative, they worked in close collaboration with Egyptologist and PhD student Perrine Poiron (UQAM/Sorbonne) to understand what would be most useful to the community. "We chose to focus on hieroglyphics from the Middle Egyptian language, which was used from 2000 BC to around the fourth century AD," says Pierre. "We then identified the different stages of the translation process: transcribing individual glyphs, sequencing them, and then translating them."
Ubisoft brought on board digital agency Psycle Interactive to help. "From the start, the biggest challenge for the project was data," says Alex Fry, Director at Psycle Interactive. "There is no centralized digital repository for information on hieroglyphics, with almost all translations stored in books and research papers. We needed to find a way to gather enough data before we could even begin to train the machine learning models to recognize hieroglyphics." Mobilizing the gaming community to draw hieroglyphics to train custom machine learning models on Google Cloud was the answer.
"Rather than just build a data repository, we wanted to create a toolset that could then be adapted and developed by both the AI and the Egyptology communities," says Guillaume. "This kind of research project is all about engaging people and pushing the limits of what is possible."
Involving Ubisoft's players in the project
For the first stage of the translation project, Ubisoft needed to find a way of extracting enough glyphs and sequences from source images to train the models to recognize them. "There are over 850 signs that the model needs to recognize, and in order for a neural network to recognize a glyph, we need at least 50 examples of it," says Pierre. "In September 2017, we launched the project at the British Museum in London and put out a call to invite anyone interested in the idea to help."
To collect the necessary data, Ubisoft and Psycle built a drawing tool using Firebase, which feeds images into a Firestore Cloud Storage database and uses Cloud Firestore to synch user data. Users access the tool online using Google Chrome via their Google or Ubisoft account, and using the tool, they can trace individual glyphs.
"The advantage of Cloud AutoML is that it provides our developers with a user-friendly experience, handling complex processes such as data pre-processing, model design and deployment, and delivering quick results using the latest models."
—Alex Fry, Director, Psycle InteractiveProducing optimal custom ML models
"Once we were in a position to assemble a dataset from the crowdsourced drawings and elements shared by our scientific partners, we began building a tool to associate glyph sequences recognized by the tool with the most probable meanings, working with Ancient Egyptian dictionaries," says Alex. "This is also accessed via Chrome, and uses App Engine to provide services that support the user at various stages of the translation process."
"Developing Machine Learning models is a very time-consuming process, requiring a lot of computational power and machine learning expertise," he explains. "The advantage of Cloud AutoML is that it provides our developers with a user-friendly experience, handling complex processes such as data pre-processing, model design and deployment, and delivering quick results using the latest models."
"The other advantage is the quality of the models produced," says Alex. "Cloud AutoML uses AI to build AI: it optimizes data as it ingests it, producing optimal parameters or even changing the model. That leads to excellent results."
Digital humanities for Egyptology
As well as consulting with Egyptologist Perrine Poiron, Ubisoft engaged the broader Egyptology community around the world on the project, including academics from as far afield as Sydney, Berlin, Harvard University, and the University of Oxford.
"We needed to understand how these researchers work and the tools they need, and then engage them in testing the tools and providing feedback," says Pierre. "In the long term, it will be the scientific community who feeds the pipeline and develops new applications for the tools."
Senior lecturer in the Department of Ancient History at Macquarie University (Sydney, Australia) Dr Alex Woods says "The work of Ubisoft's Hieroglyphics Initiative creates a unique opportunity to cross disciplinary boundaries and apply tools and technologies from digital humanities to the discipline of Egyptology. Starting with the hieroglyphs from the tombs at Beni Hassan, this project provides a framework and set of purpose-built tools to digitally capture and analyze hieroglyphic data. Working with this tool enables educators to infuse cutting-edge technology into our teaching and opens opportunities for research collaborations and partnerships."
"We have produced a set of innovative tools that assist Egyptologists throughout the translation process, and a platform that will grow as the scientific community adds their own data. We are delivering The Hieroglyphics Initiative as an open-source project, and are certain that new and surprising uses will emerge."
—Pierre Miazga, Associated Director, Marketing, UbisoftOpen-source tools for the community
The ambitions of the project have captured the imagination of the Ubisoft community. "In terms of engagement, the project has been a huge success," says Pierre. "When we launched, 10,000 people collected 80,000 glyph drawings in a single night."
"It's an interesting example of how to solve the problem of collecting enough data without putting pressure on your resources or requiring too much of people," comments Sam Mayo, Creative Technology Lead for Google ZOO.
Since launching the Hieroglyphics Initiative in September 2017, the team has collected sufficient data to create a glyph facsimile tool and a sequencing tool. "For an Egyptologist, the first step of translation is to create a facsimile of the glyphs from the source image. The tool we have developed using Cloud AutoML is already able to speed up that process. That will give Egyptologists more time to focus on addressing more interesting and complex problems."
The next stage will be to develop the tool collaboratively, to enable complete end-to-end translation. "We have produced a set of innovative tools that assist Egyptologists throughout the translation process, and a platform that will grow as the scientific community adds their own data," says Pierre. "We are delivering The Hieroglyphics Initiative as an open-source project, and are certain that new and surprising uses will emerge."
Tell us your challenge. We're here to help.
Contact us
About Ubisoft
French video games company Ubisoft is known for its original and immersive world-creating franchises, including Watch Dogs and Assassin's Creed.
About Psycle
Digital agency Psycle Interactive helps businesses with the adoption of new technology and has built everything from elearning platforms to chatbots.