Folks, I’m sipping my coffee and reading about a recent incident where a Chinese ship in the Middle East was supposedly transporting components of a nuclear weapons program, and I’m thinking to myself, “This sounds like a recipe for disaster.” The US military sprang into action, planning to intercept the vessel, with armed personnel preparing to board the ship and military planes taking to the air. But, as it often does, the story takes a turn – it was all based on a false report generated with the help of a chatbot that inaccurately identified the material the ship was carrying.
The report was “entirely false,” according to one source, but it “almost started a war.” Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations. The analyst who generated the report used a chatbot to query some intelligence reporting on the ship’s manifest, which originated with US Special Operations Command Pacific, based in Hawaii. The bot fused together open-source intelligence with secret signals intelligence and reached its fateful conclusion about the material the ship was carrying.
It’s astonishing to think that the US military is pushing to integrate AI into nearly every facet of their work, from analyzing raw intelligence to selecting targets for strikes. The rationale is that AI can help the military make battlefield decisions faster, but this episode underscores the profound risks of using this technology for targeting in the middle of a war. Analysts have long feared that AI could lead to a catastrophic miscalculation if nation states are relying on poor or corrupted data.
The US military’s rapid adoption of AI is decentralized, with different parts of the government using different tools under different orders and safety standards. There’s no one set of standards for how the US verifies the information generated by these tools. The constellation of different AI systems being deployed by disparate corners of the military and intelligence community means that the relative reliability and functionality vary widely.
For weeks, Washington policymakers have been intensely debating AI after a series of dire warnings from Silicon Valley engineers and tech CEOs. But this incident highlights a different, more immediate risk: human beings making disastrous decisions based on inaccurate or misleading information generated by AI or other automated systems.
The kind of “hallucination” that the tool used by the analyst in this case conjured has not been an isolated incident across the intelligence community. Older intelligence officials, even those who broadly support the use of AI, are concerned that AI has put pressure on analysts to produce and disseminate intelligence faster, opening the door for mistakes. As one source said, “AI allows you to get to a bad idea faster.”
In the end, it’s clear that the US military needs to be careful when using AI in targeting, as the risks of fatal mistakes are very real. As I finish my coffee, I’m left thinking that maybe we should take a step back and make sure we’re not relying too heavily on technology that can lead us astray. After all, as the saying goes, “garbage in, garbage out” – and in this case, the garbage almost led to a war. 🙄
Armchair patriot. Believes in the free market, cold beer, and that there’s always a guy named George behind every CNN segment.
Former remote-throwing champion turned #1 couch commentator on liberal panic in the media. Born in Texas (or so his mug says), he earned a degree in Fake Newsology & Beer Philosophy from YouTube University.
⚖️ ENTER THE LEGALIZER
Think the government has no idea what it's doing? Put your case before the Federal Case Terminal and find out.
