[roxolotl]

The risk is real and should be accounted for. However it is regularly presented both as surprising, and indicative that something more is going on with these models than them behaving how the training set suggests they should behave.

The consequences are the same but it’s important how these things are talked about. It’s also dangerous to convince the public that these systems are something they are not.