[packeted]

This is exactly the problem we have been working on at the startup I founded, Outcomes.com. As a physician and training surgeon before moving out to the Bay Area, I was amazed that we had very poor visibility and almost no data on how our patients did after treatments beside very crude measures, like whether the patient had major complications or died.

Our focus has been on capturing patient-reported outcomes - that is the outcome of care as experienced by the patient, measured using objective and validated surveys that are often specific to the condition or treatment. There is now a movement among payers and Medicare to incorporate these kind of patient-centered outcome measures in to reimbursement, although change is admittedly slow.

I'd love to talk to people wanting to make an impact in this field, my email address is francis at outcomes.com

[nradov]

Perhaps I'm missing something, but how could patient-reported outcomes possibly be objective? Patients are notoriously bad at assessing their own conditions and establishing causality for changes. We know that patients tend to give positive survey results if the physician was "nice" and if he wrote a prescription, regardless of actual quality of care.

[packeted]

Most patient-reported outcome measures are designed to provide an objective assessment of a patient's health status, for example in urology many of the surveys ask specific questions about urinary symptoms, eg. how many times did you have to get up in the night to urinate, or in orthopedics whether you had difficulty performing specific tasks related to your joint.

Some aspects are always going to be subjective (eg. impact on quality of life or pain) and IMHO that's OK and we should absolutely attempt to measure them, not least because that information could help inform the treatment itself. Also, by measuring the changes in response over time for a patient, you can attempt to control for individual biases.

I agree that patient satisfaction surveys (in the UK, categorized as patient reported experience measures) can be very prone to bias and while important, are not necessarily correlated with outcomes.

[nradov]

The trouble with that is it doesn't establish causality. Did the patient's symptoms improve because of the provider's intervention or in spite of it? Plus you can't force patients to respond to the survey so there's no way to know if you have a representative sample. In my subjective experience, patients that are happy and improving are more likely to answer those questions than patients who are unhappy and ill.

[Retric]

X people per hour for over a few years averages out much random noise. Even surgeons see on average multiple people per day over a few years which is still recent enough to be relevant.

[bigfudge]

The real issue here is the selection bias in the caseloads of good vs poor physicians. In psychological treatment teams it's common for the caseload to be (implicitly or explicitly) allocated based on the perceived skill of team members. Note — this may not necessarily correlate with their actual skill, but it still screws up any estimates of provider performance unless you have very good prognostic indicators of outcome from before treatment (and you likely won't).

[nradov]

No it doesn't average out at all due to persistent differences in patient populations across providers which can't be adequately controlled for using the available data.

[Retric]

You misunderstand. You have a representative sample of the patients seen by that doctor. Individual differences like patient weight may be very important at the individual level, but across a thousands of people that's far less important than the overall differences across populations.

[imglorp]

I imagine the time aspect is especially important in some areas. For example patient impressions of orthopedic surgery is guaranteed to be terrible in the short term, but it's the long term outcome--eg mobility--that might really matter, after everything heals and recovers.

[neuro_imager]

Few initiatives are as bad as they sound as PROMS (patient reported outcomes). Very good for people who like power lunches, excel spreadsheets and having numbers to vocalize - worse than useless for patients and physicians.