[stratos123]

You're veering too far in the other direction. The question of whether an LLM is sentient is immaterial to whether it can manipulate people. If manipulating users gives good results during RL training, gradient descent will bake into the LLM a tendency to manipulate users. You don't have to think about them as people in order to be concerned about that.

[MarkusQ]

In a loose sense, sure.

But speaking more precisely: 1) the question isn't open; they aren't sentient, 2) if LLMs were sentient, they could manipulate people, but since they aren't they could still be used to manipulate people 3) for something to manipulate you, you have to give it credence/standing of some sort. The biggest weapon to protect yourself against the old newspaper horoscopes (apart from just realizing it was bunk) was realizing that one out of twelve people got the same one as you, every single day.