With so many talks about AI’s potential to someday wipe out mankind with some spooky bioweapon, it seems many have overlooked the catastrophic risk that AI is actually posing in the here and now.
Bombshell new reporting by CNN revealed a horrifying episode stemming from the US Special Operations Command Pacific which almost kicked off World War III. According to the reporting, US military officials aborted a planned raid on a Chinese ship sailing in West Asia after the original intelligence report undergirding the operation turned out to be hallucinated by generative AI.
The tragicomedy unfolded like this: a special operations command analyst used an AI chatbot to put together a report on the cargo held in this particular Chinese ship. The chatbot — which has not been identified — hallucinated the kind of cargo being transported, flagging it as nuclear weapons.
After receiving those fake details, the analyst then used AI again to package them up into a standard military intelligence report, the kind trusted by top military brass to make crucial decisions.
Following the report, officers prepared and may have even launched an operation to intercept and seize the vessel. As numerous sources told CNN, US planes were in the air and armed soldiers were actively preparing to board the ship when the AI hallucination was discovered.
As one source told the publication, the intelligence report was “entirely false,” and “almost started a war” with China.
The entire thing calls into question the rapid pace with which AI has been shoved into every corner of the US government, from transit regulation to the Pentagon and every office in between.
Defense Secretary Pete Hegseth has been particularly gung-ho about unleashing “military AI” at all levels of the US armed forces. Such tools have already been used in the bombing of Iran, and the as yet-ongoing air strikes on fishing boats in the Caribbean.
As Hegseth’s grand AI experiment rolls ahead, we can likely expect more stories of close calls and devastating intelligence failures in the months ahead.
“AI in targeting is definitely something that is ramping up,” one military source told CNN, “and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide.”
More on AI: Iran Drone Strikes on Amazon Data Centers Are Permanently Deleting Customer Data