[atoav]

If there is one horror I have it is one of the really mentally unwell people I knew when growing up pairing up with an LLM that feeds into everything they say. LLMs are great for truth seekers, who know how to deal with the yes-sayer-mentality of the chatbot. If there is one type of person I am 100% sure cannot deal with that it would be mental health patients.

That doesn't mean I don't think LLMs couldn't be used for therapy with great success. Just not general purpose models without a lot of tweaking.

[bjornsing]

I fully agree that the night seems dark. But having recovered from a state that many institutionalized professionals deemed hopeless I have a different perspective.

Have you seen One Flew Over the Cuckoo's Nest? I think it serves as a good model to explain why we can both be right. You’re looking at the average effect of the institution across every minor character that appears in frame and deem it positive, and it was. I’m looking at what happened to Jack Nicholson and see the institution as deeply dysfunctional. If you think about it I think you’ll agree that I’m not wrong about that.

[atoav]

I agree that institutionalized everything can develope blind spots that makes those institutions work broadly, but can't help (and in extreme cases hurts) individuals within said institutional context — although that doesn't really contradict my point.

Just because getting kicked into the head occasionally helps someone with a neorological defect, doesn't mean we should recommend getting kicked in the head by a horse as a remedy, if you follow my point. It means we need to figure out why it helped those people and maybe find a way that doesn't require an equiestrian.

And in the case of therapy we already have a lot of knowledge what works and what doesn't. Therapeutic psychiatrists typically can't give patients the time and attention they would require for structural reasons, so LLMs have real potential here. But that LLM needs to know how to deal with patients in different conditions. E.g. imagine a patient with a schizophenic disorder convincing a LLM into feeding into their paranoid schizophrenia. My experience with LLMs so far is, that they would happily just do this, if you're persistent enough..