[glitchc]

Yes it's unlikely that hard safety rules are possible for general intelligence. After billions of years of trying, the best biology has been able to do is incentivize certain behaviours. The only way to prevent seems to be to kill the organism for trying. I'm not sure if we can do better than evolution.

[avmich]

> I'm not sure if we can do better than evolution.

Surely we can, see aiplanes and rockets. There could be ideas why evolution didn't work in this case - like, too little time between humans getting power and conquering the planet - but in general, lack of proof isn't a proof of lack. So we still don't know if safety of this kind is possible.

[rsfern]

“Kill the [model] for trying” kind of sounds like using reinforcement learning to get models to behave a certain way