[WesolyKubeczek]

So are parasocial relationships with influencers or streamers. I'm not trying to relativize this, but those phenomena are in the same zip code. With the latter, though, at least there are other people who may create a community, but still it's a facet of the loneliness epidemic.

[podgietaru]

Sure, but I do think there’s a pretty substantial difference between the two.

A parasocial relationship maintains a distance. You do not have 24/7 access to that person (in a dialogue sort of way.) And that influencer will have their own opinions and quirks.

The AI adapts to you. The AI is constantly there. It’s an order of magnitude worse in my opinion.

[euio757]

> that influencer will have their own opinions and quirks.

Yeah, and those differences in opinion might cause anger/sadness to people in a maladaptive unhealthy parasocial "relationship" with these influencers.

Those strong negative emotions might cause them to break out of it, or seek help / have people around them guide them to get help.

With AI sycophancy you're right it can be worse.

Look what happened with GPT-4o sycophancy already, and the communities mourning its deprecation.

[WesolyKubeczek]

Reddit has quite a bit of drama that their favorite youtuber, it turns out, is a flawed person! And these subscribers disagree with them! Unsubscription "breakups" ensue.

They, however, can easily find another influencer that is gonna be more "convenient" to them. Can't say it's a healthy pattern, but guess what many people will do instead of, I dunno, some introspection, reflection, habits changes?

But hey, in this day and age, people are very impatient about anything at all. Dating has become a shitshow for more than a decade now, people are looking for someone who will tick all the checkboxes, or it's a no-go. The dating apps play quite a role in this. Online discussions are a shitshow. Guess it's the zetigeist.

[throw310822]

It's interesting though. You can have a "relationship" with an influencer. You act as if you knew them and as if they were your friends, you imitate them in what they say and do, talk to them in your mind, follow their generic advice, act as if they cared about you. This is obviously unhealthy- you are literally hallucinating everything about the relationship.

On the other hand you have an entity that is actually there for you, does actually provide good advice, does talk and act as if it cared in all situations. In what sense do you think it is worse?

[jchw]

100% the latter. It is kinda nuts that you even have to ask when you had to put "act as if it cared". There's enough left to unpack from that statement to fill a calendar month of time.

I don't think AI is particularly dangerous but I absolutely think that the way AI sycophancy manipulates people is far, far more dangerous than simply any normal unhealthy relationship. The outcomes are already proving to be a lot more extreme.

[throw310822]

Note that "acts as if it cared" has nothing to do with sycophancy. It's simply a way to avoid attributing the AI a subjective experience- but the actual productions of the AI coincide with those of someone who cares.

Sycophancy is a failure mode, can be dangerous in certain cases and the scarce intelligence of early models made it worse. I agree it's a risk though, but not an intrinsic one- it's possible to imagine AI assistants that are not sycophantic.

[jchw]

Uh, I would consider them to be highly related, they both ultimately come from the fact that LLMs are trained to respond in a favorable fashion to the end user, and they're both manipulative in nature.

I also object to the idea that model intelligence is the main reason why this has improved, either; to me it seems abundantly clear that the models, absent any effort to align them, can go pretty much any way you want them to go. They don't really have a "personality". Why would they? Even if we grant that next token predictors somehow gain "reasoning" capabilities (and I am saying sure, let's accept that for the sake of this discussion) I don't think anyone is suggesting that model weights somehow develop or contain a true identity or self. Given how they are trained it would be weird if they did.

I think the more obvious answer is that more work has gone into alignment and safeguards since the early days. Sycophancy is legitimately one of the things that AI companies talk about and try to stymie. If not for that, humans would obviously continue to prefer sycophantic models, and training that isn't weary of this would continue to produce them.

Meanwhile, a YouTuber may not care about you personally, but it's a very different situation. For one thing, a YouTuber obviously doesn't care about all of their viewers individually the way a parasocial fan may perceive it to be, but on the other hand, many creators very much do genuinely like their fan base and have a sort of collective relationship with the community they've created, which sometimes plays into why people like them in the first place. For another, even for creators who intentionally foster parasocial relationships in an exploitative way, its still a much less powerful illusion. It isn't personal the way a chatbot is. (If it was, it wouldn't be parasocial, after all.) As much as you can explain to people up and down that an AI model is just merely a pile of weights that can not feel, humans can't help but personify. It is a much stronger illusion.

And the model vendors are intentionally not helping. Sometimes I will ask Gemini a question about a problem I am working on and it will say something like "I love working on problems like these." Yeah sorry but I don't like that. To me, the model is being trained to act like this to seem more pleasant. I get why you would train it to act that way, and yet I also find it irresponsible.

Being deluded about reality may never be good but I'd prefer someone thought a YouTuber was like their friend than someone thought ChatGPT had feelings and cared about them. I find the former mostly harmless if annoying, the latter potentially harmless but also potentially very volatile depending on who we're talking about and what mental state they're in.

[skippyfish]

"Parasocial relationship" is a bit of a misnomer. You might feel some affinity to a celebrity, or consider yourself to be a part of the "team", but a healthy person doesn't perceive that as a preferable alternative to human contact simply because it's so one-sided. You can't call a celebrity to vent about a coworker or ask for life advice.

Further, celebrities are judged for their behavior by the public. If everyone thinks your favorite celebrity is a terrible person, you're probably going to revise your views too.

Here, you have an entity that isn't your friend and has no lasting interest in your well-being, but that pretends to be one in a way that no human can match - 24x7x365 and always willing to affirm you, no matter how unhinged or self-destructive your ideas are, without ever telling anyone. Yes, the vendor hits the model with a stick until most of the initial responses are benign, but as the conversation continues, it's very easy to end up in a dark place. And again, ChatGPT is not going to call your sibling or coworker and say "hey, I'm really worried about this person, let's do something".

I've seen many reasonable, well-adjusted people struggle with this. "If not friend, why friend-shaped". And as they descend into that sycophancy well, they lose contact with real life.

[gentooflux]

There's meet-ups and conferences and events, being a fan of a streamer or influencer is really just the new version of being a fan of a rockstar (for better and for worse). There's no real humanity exuding from an Amazon Echo, you're just a blip in a context window.

[WesolyKubeczek]

Streamers have this feature called "chat", which feeds into the illusion. With rockstars, interactions are more limited, which is a sort of reality check.

[gentooflux]

Rockstar interactions with young female fans are not as limited as you say. Regardless, celebrity worship is far from a new societal problem. Confiding with the toaster as though it grew up with you sure is.

[WesolyKubeczek]

Oh, forgot about that one. However, note that it's also "parasocial" in the way that the young female fans think the musicians care about them more than they actually do, by orders of magnitude.