[packeted]

Most patient-reported outcome measures are designed to provide an objective assessment of a patient's health status, for example in urology many of the surveys ask specific questions about urinary symptoms, eg. how many times did you have to get up in the night to urinate, or in orthopedics whether you had difficulty performing specific tasks related to your joint.

Some aspects are always going to be subjective (eg. impact on quality of life or pain) and IMHO that's OK and we should absolutely attempt to measure them, not least because that information could help inform the treatment itself. Also, by measuring the changes in response over time for a patient, you can attempt to control for individual biases.

I agree that patient satisfaction surveys (in the UK, categorized as patient reported experience measures) can be very prone to bias and while important, are not necessarily correlated with outcomes.

[nradov]

The trouble with that is it doesn't establish causality. Did the patient's symptoms improve because of the provider's intervention or in spite of it? Plus you can't force patients to respond to the survey so there's no way to know if you have a representative sample. In my subjective experience, patients that are happy and improving are more likely to answer those questions than patients who are unhappy and ill.

[Retric]

X people per hour for over a few years averages out much random noise. Even surgeons see on average multiple people per day over a few years which is still recent enough to be relevant.

[bigfudge]

The real issue here is the selection bias in the caseloads of good vs poor physicians. In psychological treatment teams it's common for the caseload to be (implicitly or explicitly) allocated based on the perceived skill of team members. Note — this may not necessarily correlate with their actual skill, but it still screws up any estimates of provider performance unless you have very good prognostic indicators of outcome from before treatment (and you likely won't).

[nradov]

No it doesn't average out at all due to persistent differences in patient populations across providers which can't be adequately controlled for using the available data.

[Retric]

You misunderstand. You have a representative sample of the patients seen by that doctor. Individual differences like patient weight may be very important at the individual level, but across a thousands of people that's far less important than the overall differences across populations.

[nradov]

No that's simply not how it works and won't give you an accurate picture of quality differences between providers. I don't know how to make it any more clear.

[Retric]

Comparisons between providers is a separate issue.

Suppose you assigned people randomly to 101 doctors from 2 populations (A,B). Now suppose A was 10x as likely to die. D(0) get's 0% of A's and 100% B's. D(1) is 1A and 99B. All the way to D(100) that only get's B's.

In that admittedly simplified example you could determine that D(0) did a better job than D(100) by only getting 8x as many deaths even if 9.8x may be statistically irrelevant.

Yes, the real world is vastly more complex. But, while that may make a strict ordering impossible you can likely find out the best doctor is likely in the top quarter and the worst doctor very likely in the bottom quarter, which can be useful.

Picture a score card that said 80% chance in (0% - 20%], 15% chance in the (25%-50%], etc. That's not exactly meaningless information.

[imglorp]

I imagine the time aspect is especially important in some areas. For example patient impressions of orthopedic surgery is guaranteed to be terrible in the short term, but it's the long term outcome--eg mobility--that might really matter, after everything heals and recovers.