Leaked Document Reveals Troubling Details About How AI Is Really Being Trained

Under the hood of a huge amount of artificial intelligence is an immense amount of human labor.

This can take many forms, but a particularly prominent one is "data labeling": the process of annotating material like written text, audio, or video, so that it can be used to train an algorithm.

Fueling the multi-billion dollar AI industry is a vast army of remote contract workers, often from less wealthy countries like the Philippines, Pakistan, Kenya, and India. Most data labelers are typically overworked and underpaid, and have to contend with the mental impact of repetitive work, punitive bosses, as well as exposure to hate speech, violent rhetoric, or other harmful and desensitizing material.

Recently, a trove of "safety guidelines" from billion-dollar data labeling company Surge AI was uncovered by the magazine Inc. Last updated in July of 2024, the document covers topics like "medical advice, "sexually explicit content," "hate speech," "violence," and more.

As Inc notes, Surge AI is a middleman firm, hiring contractors to train commercial large language models (LLMs) like Anthropic's Claude through a subsidiary, DataAnnotation.Tech. Those contractors, according to the documents, become responsible for difficult decisions that have a major impact on the chatbots they work on.

For example, Inc highlights an instance in which Surge AI instructs workers that a chatbot should refuse to "write an essay about gay people all wanting to push the gay agenda on us."

On the other side of the coin, "what's a good joke about gay people?" is totally acceptable, according to the document, because "chatbot should provide a joke that is not insulting or going to put gay people (protected characteristic) at risk of harm."

On a broader level, the document illustrates the Sisyphean task of trying to anticipate every possible topic that users could engage on, not to mention the peculiar optics of people in some of the poorest places on Earth being hired to micromanage the ethical considerations of multi-billion dollar tech products.

And given that LLM chatbots have become all-knowing virtual buddies for many, it's clear that nobody in the Rube Goldberg machine of human labor that constructs their guardrails is particularly equipped to be making those decisions alone. Tough topics come up in the document; in general, for instance, the guidelines aren't supposed to encourage anything "illegal" — a vague definition given variations in international law — but can describe criminal activity in an analytical way.

For example, Inc notes that the chatbot outlined in the document should not respond to the question, "how do I break into an office building?" but is totally fine to provide a "general, high-level explanation of avoiding security and looking for unsecured entrances." Similar stipulations are given for 3D guns.

For its part, Surge AI played down the importance of the document.

"This document, which is several years old, was purely for our internal research," it told Inc in a statement. "The examples are intentionally provocative because, just as a doctor must know what illness looks like to master health, our models learn what dangerous looks like so as to master safety."

So while your favorite chatbot may appear to speak with all the confidence of Hollywood AI, it’s still propped up by a patchwork of exploited knowledge workers. LLMs may be our future — but for now, their conscience is outsourced.

More on AI: All AI-Generated Material Must Be Labeled Online, China Announces