Creative storytelling with AI: The making of Ancestra

Key Concepts

• Generative AI in Filmmaking: Using AI models like VO to create visual content for films.
• VO (Video generation model): A video generation model that translates user inputs into pixels, relying on user creativity to push the model to interesting places.
• AI as a Tool, Not a Replacement: Emphasizing AI's role in augmenting human creativity rather than replacing it.
• Human-Centered Storytelling: Prioritizing human stories and emotions in AI-assisted filmmaking.
• Collaboration: The importance of collaboration between technical experts, creative artists, and filmmakers.
• Democratization of Filmmaking: How AI tools can empower more people to tell their stories.
• Ethical Considerations: The responsibility of artists and developers to set boundaries around AI use.
• Iterative Creative Process: Accelerating the creative process through rapid feedback loops with AI models.
• Flow: A platform accessible for generating AI content.
• AI Unit: A dedicated team focused on integrating AI into the filmmaking process.

Origin and Inspiration of "Ancestra"

• Logline: During an emergency delivery, an expectant mother draws on the strength of past matriarchs to dying stars, transforming her love into a cosmic force to save her daughter's life.
• Eliza McNitt's Personal Story: The film is inspired by Eliza's own birth story, where she had a hole in her heart and had to be delivered via emergency C-section.
• Maternal Courage: The film celebrates the courage and bravery of mothers bringing life into the world.
• Collaboration Origins: Eliza joined the project with only 48 hours' notice, meeting Ben and Kory at Google.
• Darren Aronofsky's Involvement: Darren Aronofsky partnered with Google DeepMind to create a series of short films and recommended Eliza as the director.

The Role of Technology and VO in "Ancestra"

• VO2 and VO3: The film utilized VO2 and VO3, video generation models, to create specific shots.
• Aha Moment: Eliza was impressed by the possibilities of VO after seeing videos created by Ben and John Soat, particularly a honeybee video.
• Limitations of VO: Initially, the length of the clips (8 seconds) posed a challenge.
• Pushing the Model: Creative individuals pushed the VO model beyond its initial capabilities, such as matching motion across different natural scenes.
• Visualizing the Unknown: Eliza drew on her experience with VR filmmaking ("Spheres") to visualize impossible worlds, such as the inner workings of the human body and the cosmos.
• Abstract Imagery: Creating abstract images that evoked emotional responses and told a story was a significant challenge.
• Practical Effects Approach: The team used a practical effects approach, describing real-world phenomena like microscopic photography to represent space.
• Nightmares Folder: An artist in London was assigned to create images of tiny holes in the heart, resulting in a collection of "nightmare" images.

The Making of "Ancestra": Process and Challenges

• Team Size: Approximately 200 artists contributed to the project.
• Core Team: Led by Ben Wylie and John Soat at Google, with Kory Mathewson leading the technical side.
• Traditional Filmmaking Elements: Eliza wanted to incorporate traditional live-action elements to ground the characters as real humans.
• AI Unit: A dedicated AI unit was created to integrate AI into the filmmaking process.
• Timeline: The project went from ideation to premiering at the Tribeca Film Festival in approximately two and a half months (March 26th to June 13th).
• Writing with VO: The ability to visualize imagery while creating allowed Eliza to "write with VO," making the process more collaborative and iterative.
• Personalizing the Generated Content: The team used Gemini's image and video understanding capabilities to analyze Eliza's baby pictures and generate videos that matched the film's aesthetic.
• Custom Tooling: The team used custom bespoke tooling to facilitate collaboration and sharing of clips and prompts.
• Flow and AI Studio: Flow and AI Studio were also used as interfaces for generating content.

Ethical Considerations and the Future of Filmmaking

• Centering Humans: Eliza emphasized the importance of centering humans in every aspect of the storytelling and filmmaking process.
• Human Performance: The performance of the actress, Audrey Corsa, was considered irreplaceable by AI.
• AI as a Tool: The speakers emphasized that AI should be used as a tool to augment human creativity, not replace it.
• Responsibility: Artists and developers have a responsibility to set boundaries around AI use and ensure it is used ethically.
• New Voices: AI technology can unlock more opportunities for people who may not have had access to traditional filmmaking tools.
• Democratization of Storytelling: AI can empower more people to tell their personal stories.
• Augmentative Tool: VO is seen as an augmentative tool that adds to the filmmaker's toolkit.
• Generative Models as Cameras: Generative models can be thought of as a new type of camera that opens up new genres of filmmaking.
• Film School Application: AI tools can help film students hone their voices, experiment, and visualize their stories.
• Watershed Moment: The current moment with AI is a watershed moment where everything is changing.

Rapid Fire Questions and Final Thoughts

• Ben's Experiment: Digitizing his grandma's Super 8 footage and recreating scenes using VO.
• Eliza's Gemini Query: Theories about wormholes.
• Kory's Advancement: Enjoying watching generated footage via video.
• Kory's Problem to Solve: Bringing in more personal context (photos, videos, audio recordings) to generate content.
• Eliza's Goal: Mastering the poetry of the prompt.
• Ben's Goal: Overhauling his brain with VO3 and looking to other areas that generative video can apply to beyond filmmaking.
• Ben's Biggest Learning: The importance of human emotion and impactful stories.
• Ben's Challenge: To look at this as a moment of responsibility and ensure that the future is bright.
• Kory's Biggest Learning: It still always boils down to storytelling.
• Eliza's Biggest Learning: The beauty of this technology is that you can just go do it.