[nradov]

In a medical setting we first have to be clear about whether the LLM is acting as a transcriptionist or a scribe. Those are different roles. A transcriptionist merely writes down everything the clinician says verbatim, with perhaps a bit of formatting to fit a standard template. Speech recognition software has been used for transcription since before LLMs even existed. Sometimes a human transcriptionist will review and edit the software output. But ultimately the clinician is accountable for approving the document before it legally becomes part of the patient chart.

A medical scribe isn't just taking dictation, they're doing some level of interpretation and often entering structured, coded data directly into an EHR. This is a more complex role and requires some clinical skills. There are some new LLM products that automate this to an extent, but ultimately the licensed clinician is still legally accountable for what goes into the patient chart.

I can absolutely ethically argue for rationing care. All healthcare systems ration care although the means vary. Demand is effectively infinite, especially from older patients with complex or terminal conditions. Resources are finite.

[giardini]

I saw someone whose doctors were misled by this "transcriptionist vs scribe" difference in an ER situation. The LLM was listening and recording as the doctor asked questions and the patient answered. PT had suffered a falling episode. ER doctor asked if pt had any warning (e.g., was dizzy, faint, or unbalanced before fall). Pt said no. Later when reading final discharge report with pt we noticed that the LLM had inserted that the pt "experienced dizziness and then fell" which completely contradicted the pt's actual words.

Since then pt has had to re-explain to multiple doctors how the ER report was incorrect, to their usual MD skepticism. So, for now, I figure LLMs are for shite. Maybe in another 15 years...

And it pays to read everything on a medical report before leaving the facility.