[placebo]

While I agree that human persuasion would probably not change a worldview built on lived experience, you can't know in advance what might be possible with superhuman persuasion. You might be led to believe that your experience was interpreted incorrectly, that things are different now or that you live in an illusion and don't even know who you are. There is no way to tell what the limits of psychological manipulation are for reprogramming your beliefs unless you are totally above any human doubt about everything, which is in itself a sad state to be in.

I hope that persuaded you :)

[somenameforme]

Well, but I'm sure you'd accept that there are limits. Where we may differ is where those limits begin and where they end. In the end LLMs are not magical. All it's going to be able to do is present words to you. And how we respond to words is something that we can control. It's not like some series of words is just going to be able to completely reprogram you.

Like here I expect there is 0% chance, even if I had a superhuman LLM writing words for me, that I could ever convince you that LLMs will not be able to convince you to hold any arbitrary position. It's because you've formed your opinion, it's not falsifiable, and so there's not a whole heck of a lot else to be done except have some fun debates like this where, if anything, we tend to work to strengthen our own opinions by finding and repairing any holes in them.

[placebo]

Both our opinions about this are equally unfalsifiable unless we agree on an experiment that can be performed at some point which would make one of us change their mind.

I assume you'd agree that the pursuit of what is ultimately true should be exactly the opposite of making oneself more closed minded by repairing inconvenient holes in one's opinions rather than reassessing them based on new evidence.

I wasn't referring to the ability to persuade someone to hold an arbitrary position (although that could be a fun debate as well), and putting aside the discussion about the ability to persuade fanatics, if a super intelligence had an internal model that is more aligned with what is true, it could in theory convince someone who wants to understand the truth to take a critical look at their opinions and change them if they are authentic and courageous enough to do so.