[bjornsing]

Or maybe it just feels that way because the world is being fixed at unprecedented rates?

For example I had the misfortune to needing help from the mental health community a decade ago. I sometimes think about how much easier that part of my life would have been if TikTok and Claude had existed back then. It’s easy to forget, but many of the institutions we’ve built over centuries are deeply dysfunctional and very much in need of disruption.

The night often seems darkest just before dawn.

(With that said, I agree there are risks of the kind you describe.)

[ehnto]

I don't think you should be getting mental health advice from LLMs, it has been shown that their sycophantic nature reinforces your own diagnoses not that of a professionals. Using an LLM in a field you're knowledgeable in would make you think twice about their accuracy in fields you are unable to know the correctness of personally. It will all sound very convincing, but may be a fundamentally flawed diagnoses.

Another way of thinking about, a non-trivial amount of training data is internet comments and blogs, which have an alarming amount of self diagnoses, non-professional diagnoses, and totally fabricated facts about mental illness.

[true_religion]

I agree. I was using an LLM to test some assumptions I was making about a chemical combination, and although it's reasoning was logical it made an error about how pH works when you combine two acids of similar strength then it held onto that incorrect conclusion for the rest of the chat.

Since I wasn't asking anything related to pH, I skimmed past that section and didn't notice the error until much later in the chat when the LLM decided to build upon erroneous reasoning.

I think someone who hadn't studied chemistry would've relied on its' answer since all the rest of the logic would've been correct if the solution really did become more acidic.

[hyperadvanced]

Agree, pointing to TikTok and LLMs as a “disrupter” to the prevailing psychotherapeutic industry is crazy town. Those media reify / amplify the exact same crazy of their training set into absurdity.

[bjornsing]

To be clear I have nothing against the prevailing psychotherapeutic industry. I’m very thankful for that part of the mental health community. I think it’s rather well-aligned with TikTok too, btw.

[hyperadvanced]

I’m fairly opposed to both, but respect the difference of opinion

[lelanthran]

> I don't think you should be getting mental health advice from LLMs, it has been shown that their sycophantic nature reinforces your own diagnoses not that of a professionals.

And therapists don't? All the data used in diagnosis is self-reported feelings. All the progress made from seeing a therapist is also self-reported feelings.

Random chance might make more accurate diagnoses than any based on self-reported feelings.

[ehnto]

To an extent, but the words you give an LLM are the entirety of what it has to go on, and again they are sycophantic. If you tell it you have depression, and you insist, it will do its best to agree. In the exact same conversation, you could later convince it you have schizophrenia instead. A human wouldn't buy it.

A trained psychologist is going to use their procedural training to diagnose you. Not text input, they are asking you questions with subtext, and you may not even realise what they learned about you from your answer. With an LLM, you are loading it's context with your world view and it will go off that.

[bjornsing]

I’m well aware of the risks. But I think you have an unrealistically rosy picture of professionals’ diagnoses.

[dxdm]

Replacing expert opinion with engagement-baiting content from feed machines and hallucinating matrices seems to me is a part of the problem, and not the solution.

Especially using TikTok to try and improve mental health issues seems a bit like trying to fight fire with a (edit: spelling) hose of jet fuel.

[bjornsing]

Sometimes it’s exactly the right medicine. :)

[queenkjuul]

I very much think you're wrong

[ehnto]

There's certainly no shortage of issues with the mental health professional field, that much is true. I hope you are doing well regardless. I suspect we're going to see this story play out a lot, where the limitations of AI should be a major limiting factor, but people will get results anyway.

I imagine it will rely a lot on the pilot, and how well they understand those limitations. Perhaps the bigger risks are those without good understanding of LLMs who just treat it like an all knowing expert human.

[bjornsing]

If you are only capable of blind trust or distrust then there’s a huge difference between human experts and LLMs (and it’s of course wise to put your blind trust in a human, not an LLM). But if you have more of a ”trust no one” or ”trust but verify” mentality in general then it’s not so clear cut. The LLMs have their advantages. For example every chat with an LLM is an independent sample, whereas once a doctor has diagnosed you it’s very hard to get them to consider evidence contradictory to that diagnosis.

[icehawk]

I am really beginning to think that the Gell-Mann amnesia effect has a shining example in LLM usage.

Its obvious there's no intelligence behind it when you make it try to do something with data that gets parsed by a computer.

[atoav]

If there is one horror I have it is one of the really mentally unwell people I knew when growing up pairing up with an LLM that feeds into everything they say. LLMs are great for truth seekers, who know how to deal with the yes-sayer-mentality of the chatbot. If there is one type of person I am 100% sure cannot deal with that it would be mental health patients.

That doesn't mean I don't think LLMs couldn't be used for therapy with great success. Just not general purpose models without a lot of tweaking.

[bjornsing]

I fully agree that the night seems dark. But having recovered from a state that many institutionalized professionals deemed hopeless I have a different perspective.

Have you seen One Flew Over the Cuckoo's Nest? I think it serves as a good model to explain why we can both be right. You’re looking at the average effect of the institution across every minor character that appears in frame and deem it positive, and it was. I’m looking at what happened to Jack Nicholson and see the institution as deeply dysfunctional. If you think about it I think you’ll agree that I’m not wrong about that.

[atoav]

I agree that institutionalized everything can develope blind spots that makes those institutions work broadly, but can't help (and in extreme cases hurts) individuals within said institutional context — although that doesn't really contradict my point.

Just because getting kicked into the head occasionally helps someone with a neorological defect, doesn't mean we should recommend getting kicked in the head by a horse as a remedy, if you follow my point. It means we need to figure out why it helped those people and maybe find a way that doesn't require an equiestrian.

And in the case of therapy we already have a lot of knowledge what works and what doesn't. Therapeutic psychiatrists typically can't give patients the time and attention they would require for structural reasons, so LLMs have real potential here. But that LLM needs to know how to deal with patients in different conditions. E.g. imagine a patient with a schizophenic disorder convincing a LLM into feeding into their paranoid schizophrenia. My experience with LLMs so far is, that they would happily just do this, if you're persistent enough..

[mdavid626]

I hope you’re not seriously comparing TikTok/LLM-s with proper mental health institutions.

[ImPleadThe5th]

Man we are cooked.

[mdavid626]

Indeed, we are.