[TeMPOraL]

Yes, but Do Not Harm is often quite complex to figure out in practice - that's why it's "a big concept in medical ethics", and not something trivial that doesn't need to be talked about much.

> However, Mostly Good doesn't scale, because often patients need to seek medical attention more than once. Each visit then becomes an increasing risk factor of being hurt by treatment, eventually making malpractice inevitable.

True if the problem requires indefinite amount of repeat visits; false the chance of getting cured and not needing any more visits is big enough that the risk factor doesn't get a chance to grow. "Mostly good" is all we can ever get in the real world anyway; whether that's OK or not depends on the value of "mostly" in any given case.

Pedantics aside, my point is that it's better to have bad solution than to have none at all, and if some people get hurt applying it, the first instinct should be work on reducing the chance and magnitude of damage, instead of giving up entirely.

In this case: therapy is too expensive for most people who need it. The Web, and now LLMs, are poor substitutes for therapy, but they're accessible. Eliminating self-serve therapy substitutes won't make their users to come to mental health specialists instead; it'll just leave them with no help at all.

[octokatt]

> false the chance of getting cured and not needing any more visits is big enough that the risk factor doesn't get a chance to grow

In general, most mental health issues are chronic, and need multiple visits. In this case, that's part of the problem, as C.AI becomes more trusted over time, but the chance of a hallucination occurring grows with the number of visits, both increasing the eventual likelihood of an incident and compounding the harm.

> the first instinct should be work on reducing the chance and magnitude of damage, instead of giving up entirely

The first instinct should be to prove this attempt at therapeutic C.AI _actually works_. Then we can work on improvement, but right now, that hasn't happened. We don't let people practice medicine without a license for some very good reasons.

My fears for AI therapy becoming a more accepted practice is that we know, from research, the act of _having a person care_ is part of what makes therapy work. Humans are social creatures, for better and for worse, and loneliness is almost as good a predictor for bad health outcomes as smoking. [1] We have no meaningful evidence that an AI is going to cure loneliness, nor lead to better health outcomes, but we do know that protracted social media usage does worsen mental health outcomes. [2]

As well, having a cheap substitute means the focus will be on getting the cheap substitute to work, instead of addressing the larger ongoing mental health epidemic and health care cost crisis. Getting the AI to work will become the new focus, rather than getting to the root cause and creating meaningful policies to create a better society where health care isn't out of reach and strong, healthy friendships with people are supported by infrastructure.

C.AI replacing therapists is huge step towards a future where people are unable to afford to talk to another person. This attempt to substitute therapists, who are professional people who care, is a huge step towards that future. Do you want a future where Medicare only gives access to a chatbot, rather than a human professional? I don't want that, and I hope you don't either.

[1] https://www.pbs.org/newshour/health/loneliness-poses-health-... [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC9915628/

[TeMPOraL]

> This attempt to substitute therapists, who are professional people who care, is a huge step towards that future. Do you want a future where Medicare only gives access to a chatbot, rather than a human professional? I don't want that, and I hope you don't either.

No, I don't want to do that. And I 100% agree with you on the importance of human connection - as you wrote, "we know, from research, the act of _having a person care_ is part of what makes therapy work."

Yes, exactly that. I'd dare say it's the most important part.

However.

The future you fear has already come to pass. In the developed world, human labor tends to be the most expensive part of any product or service. That includes therapy. That 1:1 time with a trained specialist is expensive, and you need a lot of it. Most people already can't afford it at all.

I don't want LLMs to replace therapists, and I wouldn't even be talking positively about their utility in this space, if a person like me could just go into therapy whenever they needed it. Alas, we can't. It's way too expensive, way too infrequent, and takes way too much time (billed time) to get to the point where you can even hope to start to make progress. So as much as I'm typically against replacing skilled labor with self-service tech solutions, with therapy we may be forced to do it simply to have anything available to people at all.