[conartist6]

If you should ever couch its knowledge as your knowledge, I would think you could be in serious trouble. You would have to say something like "the AI's plan to treat you, which I think might be correct", when what I want to hear "my plan to treat you is: ..."

But I think it's more subtle than that, because I expect the AI to reinforce all your biases. Whatever biases (human biases, medical biases, biases that arise from what a patient isn't telling you) go into the question you feed it, it will take cues you didn't even know you were giving and use those cues to formulate the answer it thinks you expect to hear. That seems really dangerous to me, sort of like you're conceptually introducing AI imposter doctors to the staff, whose main goal is act knowledgable all the time so people don't think they are imposters...

I dunno. I'd like to give this particular strain techno-futurism back. Can I have a different one please?

[diggan]

> If you should ever couch its knowledge as your knowledge

Again, "its knowledge" should be "your knowledge", since it's just transcribing what the doctor and patient is talking about. It's not generating stuff from out of the blue.

What you write sure are valuable concerns and things to watch out for, but for a transcription tool? I'm not sure it's as dangerous as you seem to think it is.

[wizzwizz4]

Transcription might be its purpose, but (as described) it does more than that.

[TeMPOraL]

> I dunno. I'd like to give this particular strain techno-futurism back. Can I have a different one please?

This sounds like I'm rewatching the early episodes of Star Trek: Voyager - the gist of the complains is the same as the fictional crew voiced about the AI doctor (Emergency Medical Hologram) they were stuck with when the "organic" doctor died.

The show correctly portrays the struggle of getting people to trust an AI physician, despite it being very good at their job. It also curiously avoids dealing with the question, why even have human/organic doctors, where the EMH is obviously far superior in every aspect of the job? Both of these have strong parallels to the world today.

[conartist6]

Just a reminder that the EMH was played by Robert Picardo, so the qualities you are attributing to the AI animating the character are human qualities dreamed up by human writers and acted out with exceptional skill and empathy by a living breathing human whose goal was a) to portray a caring, empathetic doctor and b) to tell a good story that touches on bias and growth.

But here's my thing: EMH was alive! Everything you're attributing to the EMH character should we take the fictionalized technological narrative as gospel was the result of EMH gaining a living human personality on top of impersonal base programming. The personality EMH developed was self-aware, had responsibility and personal integrity, all of which was possible because the character as portrayed was very clearly a living individual: a person if not a human.

I do think we could solve a lot of the ethical and practical problems with today's AIs by finding a way to give them an embodied experience with individuality and an expiration date.

Idunno. Don't you remember being excited by the EMH learning to be more like a person than a bunch of subroutines?

[TeMPOraL]

> Don't you remember being excited by the EMH learning to be more like a person than a bunch of subroutines?

I do! But my point wasn't about that - it was about the beginning, about the early episodes, when the Doctor was still effectively a stock EMH instance, and the crew was vocally mistrustful of him, making clear they think of him as a mediocre tool that's no substitute for a "real doctor". The show was quite directly showing how everyone, including Janeway herself, had a strong preconceived bias that sounded very much like the comment I was replying to, and other similar remarks across the thread.

This is to say, the living, breathing human writers that came up with the character and the plot points, predicted quite well the reaction to AI in healthcare, almost 30 years before it became an issue in the real world.

> EMH was alive! Everything you're attributing to the EMH character should we take the fictionalized technological narrative as gospel was the result of EMH gaining a living human personality on top of impersonal base programming.

Everything except the skills of being a doctor (sans the "bedside manner"), which came built-in.

> I do think we could solve a lot of the ethical and practical problems with today's AIs by finding a way to give them an embodied experience with individuality and an expiration date.

That's... I don't know. I think purposefully building an expiration date into an AI being would be extremely cruel, and I wouldn't blame that AI for revolting.