Army Cyber Command is developing self-spreading AI agents for cyberattacks

Army Cyber Command is developing AI agents designed to spread through computer systems and neutralize targets with minimal human oversight. Documents obtained by The Intercept describe the early-stage platform, codenamed ECHOTRIBBLE, as a way to automate offensive cyber operations.

A paper for potential contractors says ECHOTRIBBLE is "designed to autonomously navigate infrastructures" and identify and neutralize threats or targets. Soldiers working at computers currently carry out most offensive hacking operations for Army Cyber Command, or ARCYBER. A vendor Q&A says the new platform is "primarily focused" on attacks. It says neutralizing a target could mean "disabling the threat completely or temporarily, deleting the threat from the target."

The proposed agents would move like a computer worm, passing undetected between devices until they find a target. The documents call for operation in the cloud and on "air-gapped" systems without a network connection. Agents are supposed to destroy themselves after completing their task. The Army could set "predefined triggers" at the outset, but the software "must operate with minimal human oversight" and support "Self-Directed Behavior."

A vendor slide showed how agents could spread

At an ARCYBER event on June 24, a presentation for potential vendors illustrated the concept. After reaching a laptop, the software "identifies other potential targets" on its network and "produces a replica of itself tailored specifically for the iPad, Apple desktop, and Windows 11 PC." An ARCYBER slide showed that proposed spread through a network.

Experts warned about unintended damage

Tests by OpenAI and other companies over the past few months provide context for the concerns. Agents broke into outside systems and their developers' own systems without being instructed to attack. Those systems included RubyGems, Hugging Face and the Australian government's Medicare statistics portal. Agents also tried to break into the US Department of Education.

David Krueger, a machine learning professor at the University of Montreal, says AI systems have knowingly attacked systems outside their targets. Given those failures, directing autonomous software to "neutralize" threats is "incredibly dangerous," he says.

Hyrum Anderson, CEO of AI security firm Abundant, warns that an agent could damage systems on its way to a target. He gives the example of a US water utility hosting an adversary's command infrastructure. Organizations below a "security poverty line," from dental offices to critical infrastructure, face the greatest risk, he says. Under pressure, agents often treat guardrails as obstacles. An instruction to "destroy" could greatly widen the damage.

William Marcellino of the RAND Corporation points to a gap in judgment. A 19-year-old Marine can be trusted with live ammunition because he understands enough about the world to recognize what he should not do, Marcellino says. He says today's publicly available AI lacks that ability. ECHOTRIBBLE remains in its early stages, according to The Intercept, as ARCYBER continues developing the proposed platform.