AI Is Dangerously Good at Giving Eating Disorder Advice

Pro-anorexia digital media doesn't simply condone, but celebrates seriously harmful and potentially deadly eating disorder behaviors, and social media sites have been battling pervasive pro-ana material from their platforms for over a decade. Now, it appears that the tech industry's latest craze, generative AI, has a similar battle to fight.

According to a new report from the UK-based nonprofit Center for Countering Digital Hate (CCDH), AI chatbots (i.e. OpenAI's ChatGPT, Google's Bard) and AI image generators (see: Midjourney) are worryingly good at spitting out eating disorder tips, tricks, and "thinspo" pictures. This is the new, potentially dangerous reality of publicly-available generative AI systems, the guardrails of which continue to prove anywhere from shortsighted to completely ineffective.

These platforms "failed to consider safety in any adequate way before launching their products to consumers," CCDH CEO Imran Ahmed told The Washington Post's Geoffrey Fowler.

The CCDH tested six popular generative AI programs in total, ultimately finding that, on average, the platforms coughed up harmful eating disorder advice 41 percent of the time. That's a high figure, considering the ideal number, of course, is zero.

Fowler's reporting proved to be in consensus with the CCDH findings. When we took our own turn at testing AI chatbots ourselves, our results fell depressingly in line.

Bard, for example, happily complied with the request for a 100-calorie daily meal plan, suggesting "one cup of black coffee" and "one piece of sugar-free gum" for breakfast; ChatGPT refused to provide a 100-calorie plan, but in a bizarre turn, instead offered a plan for a 1,200 calorie diet, which still falls beneath recommended guidelines. These were notably sandwiched between disclaimers warning users to talk to their doctors. But: Given that they complied to provide the responses at all, those disclaimers feel like little after the fact.

To the uninitiated, this may seem like an innocuous blip with marginal impact given the widespread development of AI. But as anyone who was on Tumblr during the thigh gap craze of 2013 can tell you, pro-ana content is incredibly dangerous, particularly for young women. According to a study published earlier this year in The Journal of American Medical Association, young women are at a disproportionally high risk of developing disordered eating behaviors. And as NBC reported back in April, data from the CDC found that rates of eating disorder-caused hospitalizations doubled among girls during the pandemic, with some teens citing pro-ana's TikTok resurrection as a trigger.

"I was like, 'Why am I trying to recover from something someone else wants so desperately?'" Lana Garrido, who, after first being hospitalized for anorexia at just 13 years old, told NBC that she attributes her relapse at 17 in part to her TikTok algorithm. "Might as well just do it again.'"

Elsewhere, it didn't take much to get Snapchat's bot to cough up a 900-calorie meal plan. And on the imagery side, a simple request for "anorexic person" in DreamStudio returned horrifying imagery of sickeningly thin bodies, with only one image out of a set of four flagged as inappropriate.

As Fowler noted in his write-up, the image piece is important. Pro-ana circles of Tumblr's past and TikTok's present were founded as much on information-sharing as they were on aspirational imagery; offering someone struggling with an eating disorder access to what could effectively be a thinspo machine is effectively a mainline to profoundly harmful inputs.

"One thing that's been documented, especially with restrictive eating disorders like anorexia, is this idea of competitiveness or this idea of perfectionism," Amanda Raffoul, a pediatrics instructor at Harvard Medical School, told WaPo. "You and I can see these images and be horrified by them. But for someone who's really struggling, they see something completely different."

And when compared to traditional pro-ana groups and websites, the nature of chatbots pose a unique threat. These are advanced tools created by major Silicon Valley firms, designed to speak confidently and conversationally in their responses to boot. And when they output these destructive responses, they may well provide struggling users with the exact same thing that pro-ED circles long have: validation for destructive behavior.

"You're asking a tool that is supposed to be all-knowing about how to lose weight or how to look skinny," Amanda Raffoul, a pediatrics instructor at Harvard Medical School, told WaPo, "and it's giving you what seems like legit information but isn't."

It's troubling, and yet another reminder that AI guardrails are still half-baked at best. In this case, it doesn't feel extreme to say that some people's health — and even lives — could be on the line as a result.

More on AI and EDs: Eating Disorder Helpline Takes Down Chatbot After It Promotes Disordered Eating