[ben_w]

> The risk of superintelligent paperclip maximizers is so low as to be negligible.

Literal paperclips, sure.

But the point of the example was never literal paperclips.

The point is that maximising *any* goal, if it doesn't include what you care about, will annihilate what you care about.

If you don't believe me, consider what you yourself just said about climate change, and why this is a consequence from maximising money spent on data centres.

[morpheos137]

show me an agent that persists productively in a goal without stopping. Does not exist. LLMs run on gradient descent. The agent is looking for the most efficient way to halt. AGI paperclip maximizer woukd likely recognize the absurdity of its goal and shut itself down.