---
title: "People Caught Using AI to Role-Play Sexual Abuse of Children"
description: "What began as a fun, bizarre experiment — using sophisticated text-writing algorithms by OpenAI to create an artificial intelligence dungeon master — took a dark turn as players started to role-play abusive, toxic, and dangerous stories. “AI Dungeon,” as a game, dropped players into an unpredictable, mostly improvised text adventure that adapted and […]"
date: "2021-05-05"
modified: "2021-05-05"
authors:
  - name: "Dan Robitzski"
    link: "https://futurism.com/authors/danrobitzski"
url: "https://futurism.com/the-byte/ai-role-play-abuse-children"
categories:
  - "Artificial Intelligence"
tags:
  - "ai dungeon"
  - "artificial intelligence"
  - "the digest"
---

# People Caught Using AI to Role-Play Sexual Abuse of Children

![The creators of the game "AI Dungeon" in which a text-writing algorithm creates a fantasy narrative realized some users were creating abusive content.](<https://futurism.com/wp-content/uploads/2021/05/AI-DUNGEON.jpg>)
*\<em\>Image: Futurism\</em\>*

## Dark Turn

What began as a fun, bizarre experiment — using sophisticated text-writing algorithms by OpenAI to create an artificial intelligence dungeon master for Dungeons & Dragons-like role-play adventures — took a dark turn as players started to role-play abusive, toxic, and dangerous stories.

The game "AI Dungeon" dropped players into an [unpredictable, mostly improvised text adventure](<https://futurism.com/the-byte/roleplaying-ai-game-dungeon-master>) that adapted and improvised the narrative based on what players chose to do. However, *[Wired ](<https://www.wired.com/story/ai-fueled-dungeon-game-got-much-darker/>)*[reports](<https://www.wired.com/story/ai-fueled-dungeon-game-got-much-darker/>) that the game's creator, a startup called Latitude, learned last month that some players were using that freeform storytelling to role-play campaigns that involved sexually abusing minors — a dark indication of how hard it is to control the applications of advanced machine learning, and one that presents a thorny legal and content moderation challenge.

## Moderation Blowback

Latitude [quickly announced updates](<https://latitude.io/blog/update-to-our-community-ai-test-april-2021/>) to its content moderation practices by banning the AI algorithm from accepting or using certain terms and manually reviewing any games that get flagged. And while that's an easy, obvious step in this case, some "AI Dungeon" players have become very vocal, complaining about what they claim is an overstep or violation of privacy by Latitude.

"Content moderation decisions are difficult in some cases, but not this one," OpenAI CEO Sam Altman said, according to *Wired*. "This is not the future for AI that any of us want."

## Extreme Caution

Users are complaining that the automated flagging system is too strict — one user says they were flagged for mentioning an "8-year-old-laptop," according to *Wired* — and that the manual oversight is a violation of their privacy. But given that Latitude seems to have encountered one of the darker corners of online communities, the extra caution seems more than justifiable.

Meanwhile, Latitude mentioned in its announcement that it would still support violent and adult content, just so long as it could verify that the fictional in-game narratives involved consenting adults.

**READ MORE:** [It Began As an AI-Fueled Dungeon Game. It Got Much Darker](<https://www.wired.com/story/ai-fueled-dungeon-game-got-much-darker/>) \[*Wired*\]

**More on AI Dungeon:** *[This Roleplaying AI Makes a Great Dungeon Master](<https://futurism.com/the-byte/roleplaying-ai-game-dungeon-master>)*

## Author
Dan Robitzki is a senior reporter for Futurism, where he likes to cover AI, tech ethics, and medicine. He spends his extra time fencing and streaming games from Los Angeles, California.

### Author social links  
[Twitter](<https://x.com/danrobitzski>)