Adobe is working on a mysterious text-to-video generative AI product that will let users create video clips with nothing but text prompts. If the feature sounds familiar, that’s because Adobe won’t be the first to launch a product like this. OpenAI showed off Sora demos earlier this year, allowing people to create videos using text prompts. Of note, Sora has been trained on actual videos from creators.
It’s too early to say which product will be better. Adobe’s unnamed tool isn’t out, and Sora is still being tested ahead of a wider release. But a report indicates that Adobe might take a better route to devloping a text-to-video service: It’ll actually pay creators to license their clips so they can be used to train the AI model.
According to Adobe documents that Bloomberg saw, Adobe has started purchasing videos for its Sora rival. Adobe is reportedly offering its network of creators $120 for “videos of people engaged in everyday actions such as walking or expressing emotions including joy and anger.”
Adobe also wants anatomy shots of feet, hands, and eyes, the report says. People interacting with objects like smartphones or fitness equipment are also priorities. But Adobe doesn’t want copyrighted content, nuditity, or other potentially offensive material.
Adobe’s offer reportedly works out to about $2.62 per minute for creators. But fees to creators could go up to $7.25 per minute of video that Adobe will use to train its Sora equivalent.
ChatGPT creator OpenAI stunned the world a few weeks ago with Sora, its text-to-video AI tool that generates clips that look almost like the real thing. A few weeks later, OpenAI CTO Mira Murati announced plans to release Sora to the public later this year.
Despite her executive role at the company, Murati wouldn’t say how OpenAI trained Sora. She wouldn’t answer questions about what sorts of videos OpenAI used to train Sora either because she didn’t know or because something fishy might have happened.
That is, OpenAI might have trained Sora on publicly available clips from social media or even sites like YouTube without actually getting approvals from content creators or social networks.
While that’s speculation, Murati’s way of dodging the question didn’t do OpenAI any favors, particularly in a world that is hyperfocused on copyright issues regarding AI products.
Murati eventually confirmed OpenAI used videos from Shutterstock, but wouldn’t say whether YouTube and Meta clips were also used for training Sora.
It’s unclear when Adobe’s Sora text-to-video product will launch or how long it will take to be ready for public testing. Per Bloomberg, Adobe has already confirmed it’s working on video-generation AI tools and plans to discuss them later this year.
Adobe already has genAI tools for generating images and performing edits. A text-to-video AI program that rivals Sora seems like the logical next step.