A film studio at your fingers: Why AI is just the latest technology for bringing our stories to life

Cinema has always been about pushing the boundaries of reality. Unlike books or radio, which live in our minds, films make dreams a reality, bringing them to life for all to see. 

What also sets film and television apart from other artforms is that it typically takes thousands of dreamers to make a feature: writers, directors, actors, camera crews, editors, animators, prop and sound departments, even electricians, Teamsters, and caterers are involved, plus so many more. It’s arguably the most collaborative creative process there is.

The other key ingredient is technology. It made films possible in the first place and consistently helped us translate these visions into a shared reality — from the advent of the magic lantern to innovations like sound, color, special effects, and CGI. Each breakthrough has expanded our ability to bring stories to the screen.

Collaboration and innovation. These are the brains and heart — plus a little courage — behind almost every great cinematic experience.

And that holds true today, as the industry embarks on our latest leap forward in the art of filmmaking: generative AI.

For proof, look no further than Sphere in Las Vegas. This week, The Wizard of Oz at Sphere will premiere, showcasing a wholly new entertainment experience, going far beyond any experience that’s come before. Our task was to take the original motion picture and expand it to fill Sphere’s colossal 160,000-square-foot LED screen, along with 4D elements developed by the Sphere Studios team. The goal is to transport audiences to the familiar world of Oz in a way they never could have imagined. (Tickets are now on sale, for those who, like me, can’t wait to see it).

As much as the venue or the experience itself, what makes The Wizard of Oz at Sphere special is that this beloved film is once again a vehicle for breaking new ground, showcasing the creative and artistic possibilities of AI in the same way it introduced viewers to the vibrant hues of Technicolor nearly 90 years ago. 

As the team prepares to raise the curved curtain in Vegas, I can tell you without reservation: This project would not have been possible without the wizardry of AI in the hands of thousands of artists and technologists — a sentiment I expect we’ll be hearing more often, both in Hollywood and around the world.

There’s no place like AI for movie magic

With the release of Google AI platforms like Gemini and Veo, we’re already witnessing never-before-imagined creations blossom across our screens. 

Luminaries like director Darren Aronofsky and Tribeca Film CEO, co-founder, and producer Jane Rosenthal have embraced the technology to test the limits of our art. As Eliza McNitt, a director working with Aronofsky’s gen-AI production studio put it, “Veo is a generative video model, but to me, it’s another lens through which I get to imagine the universe around me.”

It took similarly new ways of thinking, seeing, and creating to even begin to realize The Wizard of Oz at Sphere. You can read all about it on Google’s Keyword blog, but in short, the Sphere team tasked us and Magnopus with solving three creative challenges related to the original film that could really only be achieved with gen AI.

• Super resolution: To fill Sphere’s massive curved 16K LED screen, we had to upscale the grainy 35mm celluloid filmstock from 1939. Trained on petabytes of video at a variety of resolutions, Veo can understand and reconstruct precise character details required for the specific needs of the project. This has allowed us to fill in missing details while preserving the original natural features and nuance of the characters, making the existing Technicolor frames pop like never before. 
• Outpainting: The team also needed to expand the scope of scenes to fit the scale of Sphere. We wanted to make these iconic characters feel deep and real, to enhance the immersive experience. AI outpainting, which intelligently expands videos beyond their existing frames, has enabled us to recreate character details that were cropped in the original shots by the original framing. 
• Performance generation: Similarly, we’re utilizing a feature we call performance generation to interpolate between certain shots and frames, filling in incomplete performances on screen. This approach enabled us to extend action and movement into the new expanded environments, closing gaps in the traditional editing of the film that features cuts or reverse shots to capture the action.

For example, in many scenes in the film, the characters are not all in the same frame at the same time due to the original photography and editing of the film. The size of Sphere’s display plane is better suited to a continuous shot with all the characters still in view. Through a combination of performance generation and outpainting, we’re able to keep Dorothy and her friends together in the frame for the whole scene. And it’s all in incredible detail, thanks to super-resolution.

To further inform the accuracy and detail of this work, a vast collection of supplementary training material was built to help us fine-tune our models, which not only includes the original film but also set photography, promotional materials, archival sketches, even the notes from an early Technicolor cinematographer on camera specs.

Creativity and collaboration as easy as clicking your heels

In the great tradition of filmmaking and artmaking, these new technologies are paving the way for groundbreaking experiences like The Wizard of Oz at Sphere. With its advanced reasoning, multimodal capabilities, and natural language interfaces, gen AI is fast becoming a core building block across the entire media supply chain, enhancing everything from content production to monetization and audience experience. 

At its core, gen AI is about empowering professional creatives to produce previously unattainable artistic outcomes. The entertainment industry is never short on ideas, but almost always on time and money. With AI, it’s possible to create multiple versions of elements and performances quicker than ever, enhancing and accelerating the storytelling and editing abilities of any creative team.

The Wizard of Oz at Sphere has been an epic production undertaking, achieved with still new and largely experimental technology. The technical and creative challenges rivaled any project I’ve ever worked on. For those behind the scenes, witnessing the potential for gen AI to achieve new cinematic outputs and work in new ways has been almost as exciting as seeing the experience come to life at Sphere.

For filmmakers thinking about new projects, this underscores the potential of AI. It offers more efficient avenues to create convincing shots and atmospheric elements that might otherwise be out of reach — taking their visual storytelling to truly new places.

Paving the way for a new era of filmmaking

More than ever, AI is part of filmmaking’s democratizing moment, making techniques and tools once reserved exclusively for industry professionals widely available to anyone with a device and a vision. Studios and independent film companies can also compete in new ways, revolutionizing both their efficiency and big-picture creativity.

But to be truly bold with AI, you’ll need to be responsible from the start. 

These breakthroughs are exhilarating, and at the same time, AI poses new challenges especially as advances and novel uses continue to emerge. Gen AI, in particular, is making it easier than ever to tell stories and free our imaginations from real-world production limitations, but it also raises concerns about its ethical and creative impact. As part of Google’s mission to make the world’s information universally accessible and useful, we are striving to make AI helpful for everyone — and we’ve been clear on our AI principles from day one.

While Google Cloud is as hands-on a partner as our entertainment customers request, we neither own the content nor dictate creative control for projects like The Wizard of Oz at Sphere or the hundreds of other media and entertainment endeavors we’ve supported. Studios and customers retain full ownership of their data and intellectual property, and we don’t use customer data to train our models.

This stance is crucial, especially amid ongoing discussions around data ownership and the ethical sourcing of AI models. We’re also deeply committed to transparency and clarity regarding the training data we use for our models.

The Wizard of Oz at Sphere represents a powerful example of how AI, when used responsibly and collaboratively, can unlock unprecedented creative potential. The AI tools and techniques we developed throughout this ambitious project weren’t just about making things look bigger or better at a new scale; they were integral to delivering the hyper-realistic detail that will make this immersive experience so unique and memorable. 

Ultimately, AI isn’t here to replace human artistry — it’s here to amplify it. Humans have always been storytellers, and technology has just been a means for sharing our stories, and our experiences, in new and better ways. Now, with powerful AI tools right at our fingertips, the possibilities for telling our stories have never been greater.