[brettkromkamp]

Is any one really surprised by this? Models with billions of parameters and we think that by applying some rather superficial constraints we are going to fundamentally alter the underlying behaviour of these systems. Don’t know. It seems to me that we really don’t understand what we have unleashed.

[blululu]

On principle no it is not surprising given the points you mention. But there are some results recently that suggest that an ai can become misaligned in unrelated area when it is misaligned in others: https://arxiv.org/abs/2502.17424

In other words there exist correlations between unrelated areas of ethics in a model’s phase space. Agreed that we don’t really understand llm’s that well.