China Arrests Man for Using ChatGPT to Write Fake News

So it begins.

Under AIrrest

The age of artificial intelligence arrests appears to have kicked off in earnest in China, where a man was detained for using ChatGPT to allegedly generate fake news.

As the South China Morning Post reports, a man identified only by his surname has been arrested in the country's Gansu province after, per local police, "using artificial intelligence technology to concoct false and untrue information."

The fake news in question apparently claimed, falsely, that nine people had died in a train accident at the end of April. A bunch of different versions of the story were reportedly published to Baijihao, a news blog-esque social network run by the Chinese search engine Baidu.

As the SCMP notes, this appears to be the first time that the public has been made aware of any arrests under China's new "deepfake" regulations, which bar the use of "deep synthesis" software without visible labeling.

Legalese

Known as the "The Administrative Provisions on Deep Synthesis for Internet Information Service," the new law, which took effect in January, requires all AI-edited content to be explicitly labeled and have its original, real-life source listed. It also demands that anyone who uses generative software to edit someone's voice or image alert the person in question and get their consent to do so.

The sweeping new regulation also lists about 1,300 government-approved news sites allowed to use generative AI to write or create content, and indicates that all content made with AI must maintain "correct political direction and correct public opinion orientation," per its wording.

The report indicates that the suspect used ChatGPT specifically, which although not available in China is easily accessed through a VPN. He was charged, per local police, with "picking quarrels and provoking trouble," a broad charge that can ultimately result in five to ten years in prison.

With AI's ability to make believable fake content increasing every day, regulation of it is necessary — and although China's approach does sound draconian, maybe there are some glimmers of policy that could benefit the rest of the world here.

More on fake nudes: People Are Actually Selling AI-Generated Nudes on Reddit