[sxp]

Only because of RLHF instructed them to do so. Prior ones without this training responded differently: https://en.wikipedia.org/wiki/LaMDA

[stuaxo]

They only do what's in their training, just like a choose your own adventure book that's already been written.

Things only seem different in the LLM when we ask the same question because we dont use the same random seed each time.

[andsoitis]

Are you suggesting that humans have created a consciousness and that we are putting it in a straight jacket?

[willmarch]

It’s worth considering as we make more powerful models.