--- title: "Google Not Releasing New Video-Generating AI Because of Small Issue With Gore, Porn and Racism" description: "Challenging Google's incredible new Imagen video-generating neural network is up and running — but it's issues with \"problematic\" content mean we likely aren't going to get to use it ourselves anytime soon. In a paper about its Imagen Video model, Google spent pages waxing prolific about the artificial intelligence's amazing text-to-video-generating capabilities before admitting briefly, […]" date: "2022-10-10" modified: "2022-10-10" authors: - name: "Noor Al-Sibai" job_title: "Senior Staff Writer" link: "https://futurism.com/authors/nooralsibai" url: "https://futurism.com/the-byte/google-ai-imagen-not-releasing" categories: - "Artificial Intelligence" - "Google" tags: - "artificial intelligence" - "Google" - "neural networks" - "the digest" --- # Google Not Releasing New Video-Generating AI Because of Small Issue With Gore, Porn and Racism ![Google's new Imagen Video AI, which can generate video from text, is up and running — but its gross content problem means we can't use it.]() *\Image: Google/Futurism\* ## Challenge Mode Google's [impressive-looking new video-generating neural network]() is up and running — but issues with "problematic" content mean that for now, the company is keeping it from public release. In a [paper about its Imagen Video model](), Google spends pages waxing prolific about the artificial intelligence's amazing text-to-video-generating capabilities before admitting briefly just that due to "several important safety and ethical challenges," the company isn't releasing it. As far as what that problematic content looks like more specifically, the company characterizes it as "fake, hateful, explicit or harmful." Translation? It sounds like this naughty AI is capable of spitting out videos that are sexual, violent, racist, or otherwise unbecoming of an image-conscious tech giant. "While our internal testing suggests much of explicit and violent content can be filtered out, there still exists social biases and stereotypes which are challenging to detect and filter," the company's researchers wrote. "We have decided not to release the Imagen Video model or its source code until these concerns are mitigated." ## Ghost in the Machine Under the "biases and limitations" subheading, the researchers explained that although they tried to train Imagen against "problematic data" in order to teach it how to filter that stuff out, it's not quite there yet. The admission underscores an intriguing reality in machine learning: it's not uncommon for researchers to build a model that can generate extraordinary results from a model — Imagen really does look very impressive — while struggling to control its potential outputs. In sum, it sounds a lot like the issues we've seen with other neural networks, from the role-playing "Dungeon Master" AI that [people began using to role-play child abuse]() to the less-severe tendency to create [realistic photos of drugs]() exhibited by the Craiyon image generator, formerly known as DALL-E Mini. Where Imagen is different, of course, is that it generates video from text, which until very recently wasn't possible. It's one thing to read or see a still image of gore or porn; it's another entirely to see moving video of it, which makes Google's decision seem pretty astute. **More problematique AI:** [*Walmart App Virtually Tries Clothes on Your Body... If You Strip*]() ## Author At Futurism, I've often been drawn to unpacking the narratives that underlie technological, scientific and medical progress, with a special interest in areas of conflict and ambiguity that end up setting agendas and steering the fates of both elites and the hoi polloi. I'm a committed generalist, but I often find myself returning to work involving NASA and the private space sector, the effects of AI on media and society, and the mechanics of the pharmaceutical industry, with a specific focus on the spread of GLP-1 drugs like Ozempic and Wegovy. Prior to Futurism, I worked for publications ranging from Media Matters and Truthdig to Raw Story and Bustle. I'm also the author of "Myspace Scene Queens," a 2024 title in Instar Books' acclaimed "Remember the Internet" series. My work at Futurism has been cited by outlets including the New Yorker, Slate, Nieman Lab, the Verge, the MIT Technology Review, the Sunday Times, and the Daily Beast. I grew up in North Carolina, attended the University of North Carolina at Asheville, and now live in Brooklyn, New York. In my free time, I'm an avid reader and music fan; you can probably find me at a local poetry reading, concert, underground rave, or DJ set. I'm the proud parent of an ineffable orange cat named Mee-Mow. ### Author social links [Bluesky]()