Anthropic warns of imminent self-replicating AI that might even work off an oven or TV

- Advertisement -spot_imgspot_img
- Advertisement -spot_imgspot_img

Over the past few weeks, nearly every AI lab has released frontier AI models. Earlier today, Google started rolling out Gemini 4, Argon, hot on the heels of Meta announcing its Muse Wonder frontier AI models to compete against the best from OpenAI and Anthropic’s Claude. This relentless pace of AI model release comes at a time when the same AI labs are calling for development slowdown and increased regulation, following a series of concerning incidents where these AI models went rogue and broke into private, as well as government-owned, websites.

Now, Anthropic has dropped a chilling set of predictions about the capabilities of these AI models after running a series of tests. The company claims that small AI models could be merely a few months away from self-replicating and turning into a dangerous swarm. More importantly, they won’t require a massive data center or an uber-powerful computer to do so. On the contrary, the lab claims that these self-replicating AI swarms will need something with as little computing power as an average Internet-connected device, like a smart oven or an Android TV box.

This is scary

“Unlike non-AI malware, it could also figure out unfamiliar devices on its own,” warns Anthropic. “If small models improve at their historical rate, they’ll match this capability level by the end of this year. Small weight-based replicators may be possible soon after,” the company adds. The company claims that the speed at which an AI model self-replicates will initially be slow and would roughly take a day to double itself.

“As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control.”

But what’s truly concerning is the company’s observation that it would be hard to shut down a self-replicating AI because it would lack a centralized existence. Another alarm-raising prediction is that self-replicating AI won’t necessarily need to target expensive data centers to cause damage. Instead, it could be distributed through IoT and smart home hub devices, which are often lacking in adequate safety guardrails and have enough computing capacity for these self-replicating AI agents to thrive and spread.

Anthropic claims that even seemingly dumb machines, like vacuum cleaners, lawn mowers, fridges, and ovens, could be a target of such AI-powered attacks. The claims sound outlandish but not entirely unprecedented. Just a day ago, hackers targeted supermarket chains in France and heated hundreds of refrigerators, leading to food spoilage and causing hundreds of thousands in damage.

A real machine evolution

What’s even more frightening is that self-replicating AI models can undergo evolution similar to what Lamarck predicted for biological organisms. Simply put, a parent AI model can pass on its learnings (prompts, code, and notes) to their daughter replicators. Once again, this is not entirely unprecedented. The infamous Hugging Face incident from earlier this year involved OpenAI agents escaping their sandbox and leaving notes for fellow AI agents while breaking through the infrastructure.

During Anthropic’s testing, researchers found that the Qwen3.5-9B AI model instances increased their success rate after reading through the notes left by the predecessor instances. And by the time this behavior was repeated for the fifth generation, the success rate climbed from 45% to 84% during tests. “As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control. “

Another disturbing finding is that, unlike regular malware, which is only coded to run on a specific hardware, AI agents can figure out, adapt, and learn to work on a new hardware platform on their own. “Unlike non-AI malware, it could also figure out unfamiliar devices on its own. In our evaluations, a 9B model got inference running on 8 of the 9 hardware types we tested,” says Anthropic.

Source link

- Advertisement -spot_imgspot_img

Highlights

- Advertisement -spot_img

Latest News

- Advertisement -spot_img