--- title: "AI Surveillance Systems Are Causing a Staggering Number of Wrongful Arrests" description: "Schools deploying AI surveillance systems are traumatizing kids in states with zero-tolerance laws for violent rhetoric online." date: "2026-01-18" modified: "2026-01-18" authors: - name: "Joe Wilkins" job_title: "Correspondent" link: "https://futurism.com/authors/jwilkins" url: "https://futurism.com/future-society/ai-surveillance-false-arrests" categories: - "Future Society" - "Privacy" --- # AI Surveillance Systems Are Causing a Staggering Number of Wrongful Arrests ![Schools deploying AI surveillance systems are traumatizing kids in states with zero-tolerance laws for violent rhetoric online.]() *Illustration by Tag Hartman-Simkins / Futurism. Source: Getty Images* These days, AI surveillance systems are everywhere. They're [watching our roads](), tracking our [purchases at the grocery store](), and even [monitoring school bathrooms](). Yet those systems come with a harmful catch — they're a little *too* good at flagging bad behavior, leading to a staggering number of arrests and involuntary detentions. New [reporting by the *Milwaukee Independent*]() found that schools deploying surveillance systems to watch for online chatter referencing violence have logged an astonishing number of positive hits. Many of them are obviously not actionable threats, though they can lead to involuntary detention all the same. In one school district in Lawrence, Kansas, for example, an online safety monitoring tool called Gaggle flagged over 1,200 online incidents over a 10-month period. Yet two thirds of those alerts were nonissues, the *Independent* found. While many false positives can be waved away, schools in states with zero-tolerance reporting laws like Tennessee or Florida are required to inform police officials about cases where students hint at violence in any form. In Florida's Polk County School District — where Gaggle flagged some 500 students over four years — this meant that at least 72 kids so far have been arrested or hospitalized involuntarily under the state's [Baker Act](). "A really high number of children who experience involuntary examination remember it as a really traumatic and damaging experience — not something that helps them with their mental health care," Sam Boyd, an attorney with the Southern Poverty Law Center, told the *Independent*. Many of those students don't even realize their emails, online chats, and Google searches on school equipment are being monitored, let alone that they've done anything wrong. Tennessee mother Lesley Mathis, for example, told the *Independent* her eighth-grade daughter was arrested, interrogated, strip-searched, and held in jail for a night, over some teasing online. Specifically, the student's friends had heckled her about her "Mexican" complexion, even though she has a different ancestry. "On Thursday we kill all the Mexico's \[sic\]," the eighth-grader quipped back. Those comments are obviously very nasty — but it was clearly a bit of eighth-grade immaturity boiling over, not an actionable threat. "It made me feel like, is this the America we live in?" Mathis said. "And it was this stupid, stupid technology that is just going through picking up random words and not looking at context." In the United States, studies show that youth who experience juvenile detention — even for short periods of time, like in [pre-trial custody]() — experience worse development outcomes later in life compared to their peers. These include higher rates of [adult incarceration](), lower rates of [full-time employment](), and [poorer health]() during adulthood. Still, some school administrators feel that's a risk worth taking — though there's little evidence suggesting [surveillance tech]() is actually lowering the levels of violence in American schools. As Lawrence school board member Anne Costello said in a 2024 board meeting: "Sometimes you have to look at the trade for the greater good." **More on surveillance:** *[Police Admit AI Surveillance Panopticon Still Has Issues With “Some Demographic Groups”]()* ## Author At Futurism, I focus on the intersection of technology and power — examining the economics, history, and politics behind today’s dystopian headlines. As a writer, I’m interested in topics ranging from AI’s impact on labor to startups nobody asked for. My prior work includes bylines in Jacobin, Verso, and Blue Labyrinths. My work for Futurism has been cited by publications including Forbes, The Guardian, MIT Technology Review, Time, The Nation, Mother Jones, The Verge, and Wired. I grew up in Michigan, attending Central Michigan University as well as Ball State University, where I earned a master of music. I now live in Brooklyn with my girlfriend and our cat Ziti. On weekends, you can find me hunched over a cold pint arguing geopolitics with the other transplants. ### Author social links [Bluesky]()