[octokatt]

"Above all else, do no harm" - Hippocratic Oath (translated)

The reason why a big concept in medical ethics is Do No Harm is because it is very easy to justify some bad outcomes for the sake of many good ones, similar to what is happening now with this application of character.ai -- it's easy to justify Mostly Good.

However, Mostly Good doesn't scale, because often patients need to seek medical attention more than once. Each visit then becomes an increasing risk factor of being hurt by treatment, eventually making malpractice inevitable.

_That's not good._ Move fast and break things cannot apply to the ethical practice of medicine, which is what is currently being attempted.

[heroprotagonist]

Unfortunately, "do no harm" has become "cover your ass". This often translates to a slew of expensive test procedures before one can get diagnosed and treated, which deters patients from seeking medical care.

eg, at one point I had to go through ~$2500 out of pocket sleep studies (after insurance, probably 2-3x that in actual price before deductible limit kicked in), in order to get a $35/month (copay limit, $150 ticket price) _subscription rental_ of a CPAP machine.

A year or three later I found out I could have bought a similar CPAP machine with autocalibration for about $400 online. I would not have had to spend a few nights in sleep studies or repeated doctor visits to "justify" the continued rental.

Plenty of people who should have known the easier and cheaper route, and were explicitly supposed to be my expert advisers in the subject, never pointed this out. They all just padded their bottom line citing the "need to be sure" as rationale for the tests before risking me being unnecessarily uncomfortable at night if I hooked up a machine I didn't need to.

It was mostly the insurance company being ripped off, but a significant chunk of time, money, and effort came from me. I was not amused at the outcome.

[octokatt]

Hard agree. Have gotten to play this dance with getting migraines treated. To get the treatment I know works, I had to try taking a medication both me and my doctor were reasonably certain would make me sick.

Sure enough, the medication made me so nauseous I missed two days of work. Then I got to stop taking the medication, and go through withdrawal. Thanks, insurance companies.

[TeMPOraL]

Yes, but Do Not Harm is often quite complex to figure out in practice - that's why it's "a big concept in medical ethics", and not something trivial that doesn't need to be talked about much.

> However, Mostly Good doesn't scale, because often patients need to seek medical attention more than once. Each visit then becomes an increasing risk factor of being hurt by treatment, eventually making malpractice inevitable.

True if the problem requires indefinite amount of repeat visits; false the chance of getting cured and not needing any more visits is big enough that the risk factor doesn't get a chance to grow. "Mostly good" is all we can ever get in the real world anyway; whether that's OK or not depends on the value of "mostly" in any given case.

Pedantics aside, my point is that it's better to have bad solution than to have none at all, and if some people get hurt applying it, the first instinct should be work on reducing the chance and magnitude of damage, instead of giving up entirely.

In this case: therapy is too expensive for most people who need it. The Web, and now LLMs, are poor substitutes for therapy, but they're accessible. Eliminating self-serve therapy substitutes won't make their users to come to mental health specialists instead; it'll just leave them with no help at all.

[octokatt]

> false the chance of getting cured and not needing any more visits is big enough that the risk factor doesn't get a chance to grow

In general, most mental health issues are chronic, and need multiple visits. In this case, that's part of the problem, as C.AI becomes more trusted over time, but the chance of a hallucination occurring grows with the number of visits, both increasing the eventual likelihood of an incident and compounding the harm.

> the first instinct should be work on reducing the chance and magnitude of damage, instead of giving up entirely

The first instinct should be to prove this attempt at therapeutic C.AI _actually works_. Then we can work on improvement, but right now, that hasn't happened. We don't let people practice medicine without a license for some very good reasons.

My fears for AI therapy becoming a more accepted practice is that we know, from research, the act of _having a person care_ is part of what makes therapy work. Humans are social creatures, for better and for worse, and loneliness is almost as good a predictor for bad health outcomes as smoking. [1] We have no meaningful evidence that an AI is going to cure loneliness, nor lead to better health outcomes, but we do know that protracted social media usage does worsen mental health outcomes. [2]

As well, having a cheap substitute means the focus will be on getting the cheap substitute to work, instead of addressing the larger ongoing mental health epidemic and health care cost crisis. Getting the AI to work will become the new focus, rather than getting to the root cause and creating meaningful policies to create a better society where health care isn't out of reach and strong, healthy friendships with people are supported by infrastructure.

C.AI replacing therapists is huge step towards a future where people are unable to afford to talk to another person. This attempt to substitute therapists, who are professional people who care, is a huge step towards that future. Do you want a future where Medicare only gives access to a chatbot, rather than a human professional? I don't want that, and I hope you don't either.

[1] https://www.pbs.org/newshour/health/loneliness-poses-health-... [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC9915628/

[TeMPOraL]

> This attempt to substitute therapists, who are professional people who care, is a huge step towards that future. Do you want a future where Medicare only gives access to a chatbot, rather than a human professional? I don't want that, and I hope you don't either.

No, I don't want to do that. And I 100% agree with you on the importance of human connection - as you wrote, "we know, from research, the act of _having a person care_ is part of what makes therapy work."

Yes, exactly that. I'd dare say it's the most important part.

However.

The future you fear has already come to pass. In the developed world, human labor tends to be the most expensive part of any product or service. That includes therapy. That 1:1 time with a trained specialist is expensive, and you need a lot of it. Most people already can't afford it at all.

I don't want LLMs to replace therapists, and I wouldn't even be talking positively about their utility in this space, if a person like me could just go into therapy whenever they needed it. Alas, we can't. It's way too expensive, way too infrequent, and takes way too much time (billed time) to get to the point where you can even hope to start to make progress. So as much as I'm typically against replacing skilled labor with self-service tech solutions, with therapy we may be forced to do it simply to have anything available to people at all.