Hacker News new | ask | show | jobs
by ChrisLovejoy 1740 days ago
I suspect there must be some confounder here - like the positioning used for the CXRs correlating with race, based on the methodology used in a particular region / hospital.

Seems the most likely explanation for it still working even when pixellated as 8x8?
1 comments
OneEyedRobot 1740 days ago
That was exactly my thinking. I wouldn't publish anything until the pixellated versions were better understood.
link
SpicyLemonZest 1740 days ago
Well, isn’t the point of publishing to get help figuring it out from other researchers in the field? I agree it’s very likely that there’s some kind of explainable trick the AI is using, but there’s no guarantee it’s an easy trick that the authors could have figured out.
link
OneEyedRobot 1740 days ago
I'd warm up to that concept if the article was: "We don't know what in the hell is going on here. Here's our source code and data set of x-rays and race. What do you think?"

It could be that in the realm of machine learning, most of what is going on is people turning random knobs on a big machine and getting mysterious results. It's the birth of science without understanding.

link
SpicyLemonZest 1740 days ago
That's precisely what the researchers are saying. In the underlying paper, they conclude that "this capability is extremely difficult to isolate or mitigate", call for "further investigation and research into the human-hidden but model-decipherable information", and suggest medical imaging people should "consider the use of deep learning models with extreme caution" until future research produces a better understanding of what's happening.
link
OneEyedRobot 1740 days ago
They always call for 'further investigation'.

Looking at this: https://arxiv.org/pdf/2107.10356.pdf

My general impression (no more than that) is a whole bunch of people crowding into a paper. The paper is mostly applying trivial image processing functions and seeing how some software they don't understand is responding. The main aim is pearl-clutching about 'bias' rather than any kind of understanding. God knows what they're going to do when any medical exam includes some kind of deep dive into the patient's genetics.

No surprises. It's the nature of the era.

link