OpenAI is capitalizing on bad taste with its new text-to-video generator, Sora 2.
The new app allows users to share AI-generated videos on a TikTok-style feed. If you hoped that people would be using it to create great art, prepare to be disappointed — and nowhere is that failure clearer than in videos that resurrect sloppified versions of deceased celebrities.
We’ve seen videos of Michael Jackson rapping, for instance, as well as Tupac Shakur hanging out in North Korea and John F. Kennedy rambling about Black Friday deals.
The ethics of puppeteering the likeness of a dead person are dicey at best. But for a glimpse of the real rock bottom of the nascent Sora 2 content ecosystem, consider a wave of videos that show famed theoretical physicist Stephen Hawking — who used a wheelchair since the late 1960s due to the motor neuron disease amyotrophic lateral sclerosis (ALS), which ultimately led to his death in 2018 — being brutalized in a range of horrifying ways.
In one video, a forklift delivers Hawking’s wheelchair into a WWE-style ring, where he’s immediately knocked to the ground by burly wrestlers.
“This shouldn’t even be legal!” exclaims an announcer in the AI-generated clip.
In another video, Hawking takes blow after blow to the face from a UFC fighter. “Hawking’s in trouble,” the announcer yells, as the physicist topples out of his wheelchair.
In another, Hawking is trampled by a raging bull. In another, he’s attacked by a crocodile that drags him into a muddy river. In yet another, he drops into a skateboarding halfpipe and wipes out badly, smoke pouring from his wheelchair.
There’s a particular irony in AI being used to depict horrific violence against Hawking, who was an outspoken critic of the risks of AI during his life.
It’s also a distasteful illustration of tensions around Sora 2 that OpenAI has yet to meaningfully address. While its safety documentation for Sora 2 states that the company will “take measures to block depictions of public figures,” videos of deceased celebrities are running rampant on social media.
In a statement to PCMag, the company said that it would “allow the generation of historical figures,” suggesting an exception for the departed.
If the company is opting to allow videos of dead public figures, is anything fair game, regardless how violent or grotesque? If the company is allowing footage of a man in a wheelchair being beaten and bludgeoned, it’s hard to imagine what isn’t allowed.
Futurism has reached out to OpenAI and Stephen Hawking’s estate for comment.
In theory, living people should have control over their likeness through so-called “cameos” through which users can opt in to having their face and voice be depicted in AI videos by other users.
“With cameos, you can drop yourself straight into any Sora scene with remarkable fidelity after a short one-time video-and-audio recording in the app to verify your identity and capture your likeness,” the company’s announcement reads.
But while the company claims users will have full “control” of their “likeness end-to-end,” we’ve already seen plenty of evidence that the company’s guardrails are failing to stop nonconsensual deepfakes.
Besides the resurrection of dead celebrities, users have also generated many clips of copyrighted characters and other intellectual property.
As the chaos has unfolded, OpenAI CEO Sam Altman promised in a blog post that rightsholders will be given “more granular control over generation of characters, similar to the opt-in model for likeness but with additional controls.”
The announcement seemingly contradicted earlier reporting that OpenAI was going to force rightsholders to opt out of having their protected IP be used on the app.
Altman has yet to comment on users generating distasteful videos of deceased celebrities. But it’s likely only a matter of time until their estates start asking some serious questions.
More on Sora 2: OpenAI’s Sora 2 Already Melting Down Into Outrageous Drama