[stonecauldron]

Exactly, I think that by their very design, LLMs are very sensitive to how a question is framed.

But I wonder how much of that comes from RLHF itself or just from the way token prediction works.

[rzmmm]

It's likely the RLHF process since there are significant differences between models about this.