[bokonist]

Recently I had a major shoulder operation. I was shocked at how little information I had about who was a good surgeon. If I didn't have a family member who worked in the complaints department of the local big hospital, I would have had no way of knowing who was considered good or bad.

My surgeon told me that I had a 90% percent chance of success, and a 1% chance of nasty complications like nerve damage. The literature on the procedure said that typical success rates were 75% and the complications rate more like 5%. Was my surgeon particular good? Did I have a better shot because I was young and healthy? Or was my surgeon suffering from the Lake Wobegon affect and overconfidence? There was no way for me to know as a patient.

That said, naive score keeping could go very awry, for very obvious reasons that others in thread have mentioned.

Here is the system I would like to see. Tell me how this could get gamed:

Surgeons should be required to give official, written, probabilities to all potential patients. So for instance, a surgeon might say that there is a 90% chance that I can play football again, a 1% chance that my arm will end up worse than before the surgery, and a .01% chance of death.

Then surgeons should simply be measured against their own predictions. When I go to a surgeon, I should have access to that data. The surgeon has no incentive to be overly conservative with the probabilities - because then I will go to a surgeon who is more skilled, can predict better outcomes, and the track record to prove it. Nor does the surgeon have an incentive to be overly optimistic, because then they will get dinged for not scoring according to their own predictions. Nor does the surgeon have an incentive to turn away high-risk patients, they just need to state the risks accurately.

The patient wins because the patient can finally have the most accurate as possible information about the risks and benefits of a surgery, and can get multiple opinions, compare them, and have good data about which surgeons are reliable in their predictions.

[imaginenore]

Why ask for the doctor's opinion about the data, and not for the data itself?

And what's the punishment if the doctor's estimate is wrong? Or he/she is lying?

Doctors have enough shit on their plate, we simply need the access to the hospital data with the doctor names attached.

[bokonist]

I ask for the doctor's official opinion/prediction about the probabilities for my own surgery. This must be the doctor's opinion because every person and every surgery is different. A doctor's job is to analyze each person's situation, and then use their experience, knowledge, and professional judgement to give the patient the doctor's best estimate of the benefits and risks of a given procedure. This is not "adding sh*t to their plate", this is formalizing a core job function of every doctor.

The patient would get the doctor's prediction track record directly from the hospital or a third party monitoring agency. That way the patient knows if the doctor is generally accurate in their predictions, or if they are consistently overconfident in their own abilities.

If a doctor was wrong once, they are wrong. If they are consistently wrong, then that shows up in their stats. Patients will no longer trust their predictions, and will seek other doctors. The doctor will have to really improve their prediction ability (a good thing) or else go out of business for lack of patients.

[pedrosorio]

I don't see how this solves the problem of some doctors tackling harder/easier cases.

How do you distinguish two doctors with high prediction ability and low success rates (compared to the average for that procedure) if one is bad (and she knows it) and the other is tackling harder cases (and is actually one of the best in the field for cases with high probability of complications)?

Without input from other doctors (or simply using a lot of data where you can correlate hard procedures with other factors in the patient data) you'll never be able to distinguish the two doctors mentioned above.

[bokonist]

"How do you distinguish two doctors with high prediction ability and low success rates (compared to the average for that procedure)"

The doctor is never compared against the "average for that procedure" for exactly the reasons you give. The doctor is only compared against that doctor's own predictions.

So as a patient in need of a surgery, you would get opinions from 3-4 different surgeons, each one would offer their personal outcome probabilities. The patient gets access to that doctor's stats that score their actual track record against their own predictions. The patient should then choose the surgeon who gives the best odds but also has a track record of hitting their predictions.

A doctor who tackles hard cases should still have a good success rate against their own predictions. Such a doctor will just lower their predictions according to the riskiness of the case. If the doctor is good, such a doctor will still get business, because the skilled doctor will still offer better odds (odds that the patient can actually trust) than can be reliably offered by a less skilled doctor.

The one weakness of my system is that it does not give any sort of global score. There is still the problem of having to find 3 to 4 good surgeons to ask for an opinion in the first place. But at least once you have gotten to that point, you can have trustworthy predictions upon which to base your decisions.

[sokoloff]

Doctors in such a system may still have an incentive to over negatively predict the tough cases, and specifically by an amount greater than their peers.

The hope would be "send this tough or impossible case to someone else" such that the doctor's success rate stats will remain high and the outcome prediction stats would be unaffected (as the "trial" would go to another doctor).

I'm all for having more information available, and when my extended family faces a serious medical concern, we seek out friends and family in medicine, asking "if you faced this situation, what doctor would you trust?" I don't know of a way to globally institutionalize that process.

[MarkMc]

But the track record for such a doctor will clearly see that they aren't tackling the difficult cases. Would a doctor want a high success rate if it comes with a reputation for only taking easy cases?

Even if the answer to the above question is Yes, the reduced number of doctors willing to tackle difficult cases will be able to charge higher fees. So at some point you would reach a market equilibrium where the desire to tackle easy cases is balanced by the desire to earn a higher income. That is, when compared to the current system the easy cases will become cheaper and the more difficult cases will become more expensive - but maybe that is an acceptable outcome if it means the system as a whole is more efficient?

[mjevans]

Even if this does work, all it validates (measures) is the accuracy of guessing outcome. It does not (as mentioned via reference to global score) actually measure what the best odds for likelyhood are.

The system also does not account for new entry in to the surgical field, nor doctors changing in skill and accuracy over time (the data doesn't have aging parameters).

[MarkMc]

You can distinguish the two doctors very easily: for any given patient, the doctor with low success rates will give that patient lower odds than the doctor who only tackles harder cases.