[agf]

> For example, in one run, it edited the code to perform a system call to run itself. This led to the script endlessly calling itself. In another case, its experiments took too long to complete, hitting our timeout limit. Instead of making its code run faster, it simply tried to modify its own code to extend the timeout period.

They go on to say that the solution is sandboxing, but still, this feels like burying the lede.

[adroniser]

Does it really? If you want an LLM to edit code you need to feed it every single line of code in a prompt. Is it really that surprising that having just learnt it has been timed out, and then seeing code that has an explicit timeout in it, it edits it?? This is just a claim about the underlying foundational LLM since the whole science thing is just a wrapper.

I think this bit of it is just a gimmick put in for hype purposes.

[tomohelix]

The beginning of the AI uprising lmao

[jalman]

It could be just the opposite. One of the cheapest ways to improve alignment would be to re-run the models iteratively. The AI was likely looking for precision in the aforementioned experiment. Precision in inference is a correlate for aligned inference. https://doi.org/10.22541/au.172116310.02818938/v1

[euroderf]

> it simply tried to modify its own code to extend the timeout period.

And slacking off, at that.