[Zenzero]

Anything deeper than surface level in medicine.

Try getting it to properly select crystalloids with proper additives for a patient with a given history and lab results and watch in horror as it confidently gives instructions that would kill the patient.

What is even more irritating is that I had gpt4 debate me on things that it was completely wrong about and it was only when I responded with a stern rebuke that it hit me with the usual "Apologies for the misunderstanding..."

[mensetmanusman]

LLMs are not good at answering expert level questions at the forefront of human knowledge.

[Zenzero]

Unfortunately it would be considered basic medicine in this case.

[mensetmanusman]

Is it basic but not documented? Basic to me means the first google search result is generally correct.

[Zenzero]

That's not how medicine operates.

Medical problems are highly contextual, so you are not going to get much valuable information at the level of what a doctor is thinking from the first page of Google. That doesn't mean it isn't a simple within our area of expertise.

[mensetmanusman]

In my area of expertise, a well formulated google search can result in a page 1 full of academic articles on the general topic, but there isn’t necessarily consensus. This might be a case of the curse of knowledge :)

[mordymoop]

To be fair, I have not found MDs to be particularly reliable for answering basic questions about medicine either.

[Zenzero]

OK. I can't speak for what you've experienced. I can only offer what I see from LLMs given what I know.