[Workaccount2]

I have wondered for a while now if it happens to be that the less safe guards and thought policing you do, the more capable and generalizable the model becomes. Like "bad" parameters are actually critical for forming the whole picture necessary for ingenuity and advancement.

Effectively making it so that whoever has the lowest safeguards has the most capable model.

[soiltype]

Yes, safety is a limiter on development velocity one way or another.

I don't know whether "plotting harm" is a critical ability for passing some invisible threshold in not-well-defined intelligence. But building AI to avoid being harmful is incentivized against because it takes resources away from building AI to be more capable.