[ryandrake]

Failure cases aren't just "patient died." They also include all the times where ChatGPT's "advice" aligned with their doctor's advice, and when ChatGPT's advice was just totally wrong and the patient correctly ignored it. Nobody knows how numerous these cases are.

[ctoth]

So your failure cases are now "it agreed with the doctor" and "the patient correctly identified bad advice."

Where's the failure?

[ryandrake]

These are failures to provide useful advice over and above what could be gotten from a professional. In the sense that ChatGPT is providing net-neutral (maybe slightly positive since it confirms the doctor's diagnosis) or net-negative benefits (in the case that it's just wasting the user's time with garbage).