[discardorama]

I've been interested in application of ML to healthcare for nearly 20 years. In the early days, researchers just refused to share data; they were worried that you (using some fancy math or ML) would upstage them. I actually heard this first-hand from a faculty member.

Fast forward to today, and there is more openness. I did skim the paper mentioned here, but did not see any links to the actual data, which is a shame.

[JamesBarney]

I think this is still true today. I worked at large research institution, and the doctors were cut-throat when it came to keeping other people from using their data.

My understanding is that 70% of the difficulty medical research is collecting the data. So once you've done that you want to make sure you've protected your investment of time and energy.

From an individual stand point I understand why they do it. From a societal standpoint it's such a waste.

[jfoutz]

Don't they have to publish the data with the papers? There are so many ways to screw up analysis. That's ignoring collection problems...

I'm guessing they don't, which makes findings, kinda, suspect.

[JamesBarney]

HIPAA compliance issues makes this an issue.

Also their trove of data is not a one shot. They might collect data a mountain of data on Leukemia but publish one study on Leukemia's correlation with high power lines. Publishing this paper wouldn't necessitate opening up all of their data from either a horizontal or vertical perspective.

[jfoutz]

HIPPA makes perfect sense.

I see your point about using the data for other research. Hopefully reviewers of the paper, at least, get to look at the data.