Adobe is developing “IP-safe” gen AI models for the entertainment industry

As Hollywood continues to embrace generative AI, Adobe is taking steps to make its Firefly suite of creative tools the go-to for studios’ entertainment production needs. Timed to this year’s Sundance Film Festival, Adobe has announced that it is working with a number of studios, directors, and talent agencies to develop “private, IP-safe” Firefly Foundry gen AI “omni-models.” According to the company, Firefly Foundry models are meant to “accelerate creativity without eroding ownership or creative intent” while generating different kinds of assets like audio-aware videos and 3D / vector graphics that can be seamlessly integrated into workflows that use other Adobe products like Premiere. Unlike other models on the market that are fed large datasets scraped from the internet, Firefly Foundry models — which are being marketed to businesses as opposed to regular consumers — are unique to each of Adobe’s clients and only trained on IP that clients own the rights to. What Adobe is offering is similar to the general idea behind genAI startup Asteria. But as a legacy company with decades of experience building these kinds of products, Adobe seems much better positioned to actually be able to pull it off. Adobe says that this ensures Firefly Foundry models enable “responsible AI adoption across every stage of production” from pre-visualization to final stage edits. And the company is banking on its legacy of creating some of the entertainment industry’s most widely used creative tools to sell customers on the idea of adopting this new technology. Speaking to The Verge, Adobe’s vice president of genAI new business ventures Hannah Elsakr explained that Firefly Foundry was borne out of the company’s previous work with large companies using Firefly’s older, less customizable models. In those instances, the models were limited in their ability to generate assets beyond static images, or to understand the fine details of clients’ IP. Those limitations were rooted in the fact that Adobe’s more basic gen AI models were trained solely on inputs that Adobe had bought the rights to use. But that wasn’t quite enough to create the kinds of assets Adobe’s clients wanted going forward. “Global companies like The Home Depot and Disney said that they needed more,” Elsakr said. “They needed a creative world that understood multiple products, characters, and the physics of how those characters move — both for video and for 3D. That’s where Firefly Foundry comes in.” To hype Firefly Foundry up, Adobe is already collaborating with talent agencies, including Creative Artists Agency, United Talent Agency, and William Morris Endeavor. The company is also working with Suicide Squad director David Ayer and Jaume Collet-Serra of Black Adam fame. In addition to partnering with production houses B5 Studios, Promise Advanced Imagination, and Cantina Creative Adobe has partnered with Parsons School of Design and Whistling Woods International to “develop research, educational resources, and curriculum focused on the role of AI in creative fields.” Firefly Foundry sounds like an approach to generative AI that megacorporations might be keen on adopting if only to avoid any potential IP infringement that could occur with regular models. And with its focus on getting the next generation of creative professionals using these tools early, the company might be setting itself up to be a long-term winner in the AI arms race. Correction, January 22nd: An earlier version of this article misspelled Elsakr’s last name. It is “Elsakr,” not “Elaskr.”