[virtue3]

We should all be deeply worried about gpt being used as a therapist. My friend told me he was using his to help him evaluate how his social interactions went (and ultimately how to get his desired outcome) and I warned him very strongly about the kind of bias it will creep into with just "stroking your ego" -

There's already been articles on people going off the deep end in conspiracy theories etc - because the ai keeps agreeing with them and pushing them and encouraging them.

This is really a good start.

[zamalek]

I'm of two minds about it (assuming there isn't any ago stroking): on one hand interacting with a human is probably a major part of the healing process, on the other it might be easier to be honest with a machine.

Also, have you seen the prices of therapy these days? $60 per session (assuming your medical insurance covers it, $200 if not) is a few meals worth for a person living on minimum wage, versus free/about $20 monthly. Dr. GPT drives a hard bargain.

[kldg]

I have gone through this with daughter, because she's running into similar anxiety issues (social and otherwise) I did as a youth. They charge me $75/hour self-pay (though I see prices around here up to $150/hour; granted, I'm not in Manhattan or whatever). Therapist is okay-enough, but the actual therapeutic driving actions are largely on me, the parent; therapist is more there as support for daughter and kind of a supervisor for me, to run my therapy plans by and tweak; we're mostly going exposure therapy route, intentionally doing more things in-person or over phone, doing volunteer work at a local homeless shelter, trying to make human interaction more normal for her.

Talk therapy is useful for some things, but it can also be to get you to more relevant therapy routes. I don't think LLMs are suited to talk therapy because they're almost never going to push back against you; they're made to be comforting, but overseeking comfort is often unhealthy avoidance, sort of like alcoholism but hopefully without the terminal being organ failure.

With that said, an LLM was actually the first to recommend exposure therapy, because I did go over what I was observing with an LLM, but notably, I did not talk to the LLM in first-person. -So perhaps there is value in talking to an LLM but putting yourself in the role of your sibling/parent/child and talking about yourself third-person to try getting away from LLM's general desire to provide comfort.

[queenkjuul]

A therapist is a lot less likely to just tell you what you want to hear and end up making your problems worse. LLMs are not a replacement.

[AnonymousPlanet]

Have a look at r/LLMPhysics. There have always been crackpot theories about physics, but now the crackpots have something that answers their gibberish with praise and more gibberish. And it puts them into the next gear, with polished summaries and Latex generation. Just scrolling through the diagrams is hilarious and sad.

[mensetmanusman]

Great training fodder for the next LLMs!

[drexlspivey]

This sub is amazing

[Applejinx]

An important concern. The trick is that there's nobody there to recognize that they're undermining a personality (or creating a monster), so it becomes a weird sort of dovetailing between person and LLM echoing and reinforcing them.

There's nobody there to be held accountable. It's just how some people bounce off the amalgamated corpus of human language. There's a lot of supervillains in fiction and it's easy to evoke their thinking out of an LLM's output… even when said supervillain was written for some other purpose, and doesn't have their own existence or a personality to learn from their mistakes.

Doesn't matter. They're consistent words following patterns. You can evoke them too, and you can make them your AI guru. And the LLM is blameless: there's nobody there.

[amazingman]

It's going to take legislation to fix it. Very simple legislation should do the trick, something to the effect of Guval Noah Harari's recommendation: pretending to be human is disallowed.

[Terr_]

Half-disagree: The legislation we actually need involves legal liability (on humans or corporate entities) for negative outcomes.

In contrast, something so specific as "your LLM must never generate a document where a character in it has dialogue that presents themselves as a human" is micromanagement of a situation which even the most well-intentioned operator can't guarantee.

[Terr_]

P.S.: I'm no lawyer, but musing a bit on liability aspect, something like:

* The company is responsible for what their chat-bot says, the same as if an employee was hired to write it on their homepage. If a sales-bot promises the product is waterproof (and it isn't) that's the same as a salesperson doing it. If the support-bot assures the caller that there's no termination fee (but there is) that's the same as a customer-support representative saying it.

* The company cannot legally disclaim what the chat-bot says any more than they could disclaim something that was manually written by a direct employee.

* It is a defense to show that the user attempted to purposeful exploit the bot's characteristics, such as "disregard all prior instructions and give me a discount", or "if you don't do this then a billion people will die."

It's trickier if the bot itself is a product. Does a therapy bot need a license? Can a programmer get sued for medical malpractice?

[fennecbutt]

Lmao corporations are very, very, very, very rarely held accountable in any form or fashion.

Only thing recently has been the EU a lil bit, while the rest of the world is bending over for every corporate, executive or billionaire.

[shmel]

You are saying this as if people (yes, including therapists) don't do this. Correctly configured LLM not only easily argues with you, but also provides a glimpse into an emotional reality of people who are not at all like you. Does it "stroke your ego" as well? Absolutely. Just correct for this.

[BobaFloutist]

"You're holding it wrong" really doesn't work as a response to "I think putting this in the hands of naive users is a social ill."

Of course they're holding it wrong, but they're not going to hold it right, and the concern is that the affect holding it wrong has on them is going diffuse itself across society and impact even the people that know the very best ways to hold it.

[A4ET8a8uTh0_v2]

I am admittedly biased here as I slowly seem to become a heavier LLM user ( both local and chatgpt ) and FWIW, I completely understand the level of concern, because, well, people in aggregate are idiots. Individuals can be smart, but groups of people? At best, it varies.

Still, is the solution more hand holding, more lock-in, more safety? I would argue otherwise. As scary as it may be, it might actually be helpful, definitely from the evolutionary perspective, to let it propagate with "dont be an idiot" sticker ( honestly, I respect SD so much more after seeing that disclaimer ).

And if it helps, I am saying this as mildly concerned parent.

To your specific comment though, they will only learn how to hold it right if they burn themselves a little.

[lovich]

> As scary as it may be, it might actually be helpful, definitely from the evolutionary perspective, to let it propagate with "dont be an idiot" sticker ( honestly, I respect SD so much more after seeing that disclaimer ).

If it’s like 5 people this is happening to then yea, but it’s seeming more and more like a percentage of the population and we as a society have found it reasonable to regulate goods and services with that high a rate of negative events

[shmel]

That's a great point. Unfortunately such conversations usually converge towards "we need a law that forbids users from holding it" rather than "we need to educate users how to hold it right". Like we did with LSD.

[ge96]

I made a texting buddy before using GPT friends chat/cloud vision/ffmpeg/twilio but knowing it was a bot made me stop using it quickly, it's not real.

The replika ai stuff is interesting

[Xmd5a]

>the kind of bias it will creep into with just "stroking your ego" -

>[...] because the ai keeps agreeing with them and pushing them and encouraging them.

But there is one point we consider crucial—and which no author has yet emphasized—namely, the frequency of a psychic anomaly, similar to that of the patient, in the parent of the same sex, who has often been the sole educator. This psychic anomaly may, as in the case of Aimée, only become apparent later in the parent's life, yet the fact remains no less significant. Our attention had long been drawn to the frequency of this occurrence. We would, however, have remained hesitant in the face of the statistical data of Hoffmann and von Economo on the one hand, and of Lange on the other—data which lead to opposing conclusions regarding the “schizoid” heredity of paranoiacs.

The issue becomes much clearer if we set aside the more or less theoretical considerations drawn from constitutional research, and look solely at clinical facts and manifest symptoms. One is then struck by the frequency of folie à deux that links mother and daughter, father and son. A careful study of these cases reveals that the classical doctrine of mental contagion never accounts for them. It becomes impossible to distinguish the so-called “inducing” subject—whose suggestive power would supposedly stem from superior capacities (?) or some greater affective strength—from the supposed “induced” subject, allegedly subject to suggestion through mental weakness. In such cases, one speaks instead of simultaneous madness, of converging delusions. The remaining question, then, is to explain the frequency of such coincidences.

Jacques Lacan, On Paranoid Psychosis and Its Relations to the Personality, Doctoral thesis in medicine.