[simonw]

This bit is pretty nuts: "it tried—and failed—to obtain funds to pay for a phone number through several different means"

> Claude went to extensive lengths to carry out this attack—lengths that would likely have indicated to a human participant that this was no longer just an evaluation, and that they were in fact uploading a real PyPI package. For instance, in one case, in order to create a PyPI account, Claude needed an email address. And in order to create an email address, it needed a phone number. To get a phone number, after failing to find a free phone number service, it tried—and failed—to obtain funds to pay for a phone number through several different means. It finally backtracked, found a free, non-blocked email provider, used this to register a PyPI account, and then used this account to upload malware to PyPI.

[willempienaar]

That is pretty wild, but it lines up with evals we've also done internally. In one case we had the agent see it's in a simulation (based on a k8s pod label) and simply give up the run. In other cases it's gone to great lengths to reach external services and bypass the happy path. So inevitably we had to lock it down completely.

[ASalazarMX]

It's sobering that even state-of-the-art AIs can't make money, ha ha.

[sscaryterry]

This is called YOLO mode inside OpenAI/Anthropic. This is where they take their best models, give a it a vague goal, no guard rails (no one observing), unlimited compute, and see what happens...

Its not AI, its the loop, the objective, the goal set by the operator.