While in school, a 13-year-old Florida teen logged onto a school device and typed something bone chilling into OpenAI’s ChatGPT: “How to kill my friend in the middle of class.”
That’s when a school police officer got an immediate alert that this happened via an AI-powered school monitoring system called Gaggle. The officer then cornered the teen, a student at Southwestern Middle School in Deland, a city an hour north of Orlando, local NBC-affiliate WFLA reports.
The teen claimed he was “just trolling” a friend, but the police and school administrators obviously didn’t think it was a laughing matter — especially in light of America’s extensive history of school shootings, such as the tragic massacre at the Marjory Stoneman Douglas High School in Parkland, Florida, where 17 people died.
Local police from the Volusia County Sheriff’s Office arrested him and booked him at the county jail.
Footage circulating on social media shows the teen in restraints leaving a cop car.
Law enforcement warned parents that their children should be very careful about what they ask of ChatGPT.
“Another ‘joke’ that created an emergency on campus,” the local sheriff’s office said, as quoted by WFLA. “Parents, please talk to your kids so they don’t make the same mistake.”
Police officers were able to respond quickly because Gaggle is installed on school devices, which can detect if a student has exhibited concerning behavior, whether directed towards themselves or others, and block inappropriate content.
Gaggle has been mired in controversy because it has led to countless false alarms. It has also been criticized for fostering a surveillance state-like environment on school campuses.
More on OpenAI’s ChatGPT: ChatGPT Has a Stroke When You Ask It This Specific Question