[thayne]

Part of the problem is due to the marketing of LLMs as more capable and trustworthy than they really are.

And the safety testing actually makes this worse, because it leads people to trust that LLMs are less likely to give dangerous advice, when they could still do so.

[jdross]

Spend 15 minutes talking to a person in their 20's about how they use ChatGPT to work through issues in their personal lives and you'll see how much they already trust the "advice" and other information produced by LLMs.

Manipulation is a genuine concern!

[jpeeler]

Netflix needs to do a Black Mirror episode where either a sentient AI pretends that it's "dumber" than it is while secretly plotting to overthrow humanity. Either that or a LLM is hacked by deep state actors that provides similar manipulated advice.

[thrtythreeforty]

Isn't this just a redux of all the "hard take off" creative writing?

[seam_carver]

One of the story arcs in “The Phoenix” by Osama Tezuka is on a similar topic.

[justacrow]

It's not just young people. My boss (originally a programmer) agreed with me that there's lots of problems using ChatGPT for our products and programs as it gives the wrong answers too often, but tgen 30 seconds later told me that it was apparently great at giving medical advice.

...later someone higher-up decided that it's actually great at programming as well, and so now we all believe it's incredibly useful and necessary for us to be able to do our daily work

[literalAardvark]

Most doctors will prescribe antibiotics for viral infections just to get you out and the next guy in, they have zero interest in sitting there to troubleshoot with you.

For this reason o3 is way better than most of the doctors I've had access to, to the point where my PCP just writes whatever I brought in because she can't follow 3/4 of it.

Yes, the answers are often wrong and incomplete, and it's up to you to guide the model to sort it out, but it's just like vibe coding: if you put in the steering effort, you can get a decent output.

Would it be better if you could hire an actual professional to do it? Of course. But most of us are priced out of that level of care.

[andsoitis]

> Most doctors will prescribe antibiotics for viral infections just to get you out and the next guy in

Where do you get this data from?

[somenameforme]

Family in my case. There are two reasons they do this. A lot of people like medicine - they think it justifies the cost of the visit, and there's a real placebo effect (which is not an oxymoron as many might think).

The second is that many viral infections can, in rare scenarios, lead to bacterial infections. For instance a random flu can leave one more susceptible to developing pneumonia. Throwing antibiotics at everything is a defensive measure to help ward of malpractice lawsuits. Even if frivolous, it's something no doctor wants to deal with, but some absurd number - something like 1 in 15 per year, will.

[literalAardvark]

Lived experience. I'm not in the US and neither are most doctors.

[bdangubic]

I can co-sign this being bi-coastal. in the US not once have I or my 12-year old kid been prescribed antibiotics. on three ocassions in europe I had to take my kid to the doctor and each time antibiotics were prescribed (never consumed)

[asadotzler]

Your claim of most here is not only unsupported, it's completely wrong.

[literalAardvark]

I'd like to see your support for that very confident take.

In my experience it's not only correct, but so common that it's hard not to get a round of antibiotics to go.

The only caveat is that I'm in the EU, not the US.

[DiscourseFan]

LLMs are really good at medical diagnostics, though…

[amluto]

No. They are sort of good at reading a list of symptoms and having a decent chance of coming up with whatever their training set thinks is a likely diagnosis. They seem to be rather erratic, though, and they also make stuff up that would be trivially rejected by anyone with actual understanding of medicine (and by the same LLM if you quiz it harder!). They also seem to suffer from what I believe is known as the “reverse problem”: if the training set contains “condition A causes symptoms B and C” but not “if you have symptoms B and C then condition A should be considered”, then LLMs seem very likely to miss the diagnosis. Interestingly, real doctors seem to have the same problem, and I suspect that medicine is full of supposedly rare conditions that are actually not so rare but that most doctors are incapable of diagnosing.

[brookst]

Can you point to a specific bit of marketing that says to take whatever medications a LLM suggests, or other similar overreach?

People keep talking about this “marketing”, and I have yet to see a single example.