Just two days ago, OpenAI released its latest text-to-video-and audio-generator, Sora 2, which comes with a built-in — and currently invite-only — TikTok-style social app for sharing AI videos.
It didn’t take long for users to notice that the company did little to ensure it didn’t spit out grotesque — and possibly copyright-infringing — materials.
In one video that went viral on X-formerly-Twitter, a cleanly-animated clip shows Nickelodeon’s SpongeBob SquarePants cooking up blue crystals in a meth lab under the sea, much in the style of AMC’s iconic “Breaking Bad” television series.
It’s not just unmistakably SpongeBob and his friend Patrick visually taking on the role of Walter White and Jesse Pinkman in “Breaking Bad”; the clip features fully-fledged, AI-generated takes on the characters’ iconic voices as well.
“We don’t call it ‘stuff,’ Patrick,” the anthropomorphized sponge tells its compatriot. “It’s Blue Barnacle. Purest you’ll find under the sea.”
The clip demonstrates how little OpenAI has done to prevent its tool from being used to generate copyright-protected material. It’s a hotly debated topic that has already led to major Hollywood studios suing AI generator companies. Last month, Warner Bros. Discovery sued Midjourney for infringement, joining Disney and NBCU, which had teamed up to sue the firm for similar reasons in June.
For Sora 2, OpenAI has also said that rights holders will have to actively opt out of having their copyrighted materials appear in generations, as the Wall Street Journal reported earlier this week.
“Given the intense competition in the space, I think they think, ‘maybe we will ask for forgiveness instead of asking for permission,'” Georgetown Law School professor Kristelia García told the newspaper.
OpenAI has repeatedly argued that the use of copyrighted material falls under “fair use,” a thorny doctrine that allows the use of copyrighted material without permission for transformative purposes like commentary, criticism, or news reporting.
OpenAI has also leaned on a familiar argument: that if the United States doesn’t do it, China will.
“If the PRC’s developers have unfettered access to data and American companies are left without fair use access, the race for AI is effectively over,” reads an OpenAI policy proposal to the White House’s Office of Science and Technology.
While the debate rages on, OpenAI’s hot new video generator is already clogging up social media feeds with a torrent of AI slop.
A search on X reveals a litany of other highly detailed AI-generated clips featuring SpongeBob Squarepants and his co-stars. One video shows the animated character being taught the programming language C++. Another clip shows him mining cryptocurrencies while surrounded by computer hardware.
Others went in an even edgier direction, as with a clip of a “German-dubbed Nazi SpongeBob.”
Futurism has reached out to Nickelodeon for comment.
Beyond copyright-infringing content, Sora 2 has also been used to generate other controversial videos, such as a 1990s TV commercial for a children’s toy inspired by the island of deceased pedophile Jeffrey Epstein.
OpenAI’s own Sora developer, Gabriel Petersson, also shared an AI-generated video of OpenAI CEO Sam Altman getting caught after trying to steal a GPU from a Target store, raising concerns that the tool could be used to impersonate others or even frame them for a crime they didn’t commit.
The Washington Post‘s Drew Harwell also demonstrated in a video how easy it is to generate fake body cam footage of a Black person being arrested after being accused of shoplifting.
“Here’s a video I made of Sam [Altman] leading an ICE raid,” Harwell said in the video, “and again, none of this was blocked.”
OpenAI claimed in a blog post accompanying its Sora 2 announcement that it has “made sure safety is built in from the very start” by “filtering harmful content” and adding “audio safeguards” that block “attempts to generate music that imitates living artists or existing works.”
It also said that it will “honor takedown requests from creators who believe a Sora output infringes on their work.”
Given what we’ve seen so far, those requests could soon start to pour in.
More on Sora 2: Sora 2 Is Creating Ads for “Epstein Island” Children’s Toys