[IshKebab]

Did people doubt that this was theoretically possible? Seems self-evident to me. The interesting thing will be seeing it in the real world rather than in a controlled environment where they deliberately made all devices on the network have a known vulnerability.

[acdha]

There’s a difference between speculation and measurement, especially since you’d have people making arguments like saying that open models aren’t powerful/fast enough to work. Demonstrating this is a useful warning to everyone (most of the industry) who’s been slacking on internal defenses because they don’t think a well-resourced attacker will target them.

[IshKebab]

It's not really a realistic measurement though is it?

[acdha]

It shows that the idea can work in a somewhat realistic scenario. That doesn’t mean we’re all doomed but it’s a good wake up call for anyone who’s been slacking on internal controls or patching.

[pixl97]

Honestly with some of the denialists here a terminator could kick down their door with lazgun in hand and they'd still tell you that AI can't do that.

And for the people that think that alignment is stupid, not training your AI to think twice about writing self spreading worms is a recipe for disaster after someone gets a token stealing, resource grabbing worm going.