The latest warning from Anthropic: AI models may be capable of self-replicating through IoT devices, even without access to a datacenter.

In their article from their Frontier Red Team (AI models may be capable of self-replicating without access to a datacenter, available here), Anthropic warns that “small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there’s no centralized point to shut off—though it may be slow (with a doubling time of days). Unlike non-AI malware, it could also figure out unfamiliar devices on its own.”

As they also note: “Prior work has already shown that AI agents can act like [a worm]: scanning their network, executing an exploit, and copying model weights to a target machine across a GPU cluster. Such an ‘AI worm’ could carry its model weights with it as it reproduces (or re-download them from a public hub), and then run inference directly on compromised hardware. Effectively, the weights would become part of the malware payload.”

While datacenter GPUs “would be a clear, high-value target for such malware”, the team notes that “expensive accelerators aren’t strictly necessary. In this blog post, we show that a distributed botnet of IoT devices—which are cheap and typically poorly defended—could also provide enough compute to run a capable model.”

Their researchers demonstrated that a capable model could run across 15 inexpensive Android TV boxes.

The key concern is that these AI systems would be difficult to shut down, as they have no central point that can be stopped. How soon this could occur is unclear, but Anthropic researchers indicated that the first, simpler forms of this exploit could be just a few months away.

Full disclosure: it’s a lengthy, highly technical article and I didn’t read all the research that illustrates their points. Still, these “smart” devices don’t seem so smart now, do they? Sigh.

So, what do you think? Are you concerned that AI models may be capable of self-replicating through IoT devices? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using ChatGPT, using the term “robot hiding in a smart oven”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.