[kromem]

It's more that hyper-empathy and more broadly hyper-emotionality is how social media goes.

In real life, someone asking to cut in line because they are sad might get a "I'm sorry for your loss, but I'm in a rush too."

But online, callousness in response to emotional vulnerability is generally down voted while empathy is upvoted on something like Reddit.

Well guess what data source was being used to train appropriateness of responses to input? All that karma wasn't being thrown out the window.

So we have LLMs that in their core network have effectively learned to output responses that would get upvoted on Reddit and avoid comments that would get down voted.

Appealing to empathy or sentimentality works because lurkers upvoted feel good comments.

The most important thing to know about the current tech is that LLMs do not reflect humanity - but they do reflect the version of ourselves that we collectively projected online. Which is a highly exaggerated form of the real thing.