[stonecauldron]

This is especially problematic because of how easily (and unconsciously) one can bias LLMs with how the prompt is framed.

As an experiment, I recently asked an LLM to analyse the export of a text chat to uncover relationship dynamics.

Simply stating that I was one of the people in the chat would make the LLM turn the other person into the villain. None of that was visible if I framed the chat as only involving third party people.

[NoMoreNicksLeft]

If these LLMs were trained on internet forum posts, think about how those work.

If the posts talked about third party interactions (movie characters), they try to see everything from all the points of view. If nothing else, because it can be interesting to talk about. If instead the posts talk about personal interactions, then people go into advice mode. Your girlfriend's bad for you and cheating on you, dump her before she dumps you. Your neighbors are assholes, get a restraining order. Your boss is sabotaging you, stand up for yourself so you can get a promotion. When people talk about interactions you have had yourself, they always see the other person as the villain, unless you come across as so unlikable that they hate you and see the other person as the victim.

LLMs picked up on that, possibly.

[stonecauldron]

Yeah, that makes a lot sense.