[msapaydin]

I think that not being able to reproduce the results claimed in a paper is not specific to ML research. While working as a post-doc at a top university research lab, i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper, and that the primary author went on to become a prof at a top university in the US. In short, scientific fraud is also quite common, in most academic papers.

[sillysaurusx]

Thank you!!

i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper,

This was my exact experience. I didn’t understand why I kept having it, and kept blaming myself for not being careful enough. My code must be wrong, or the data, or something.

Nah. It was the idea.

Kept feeling like a kick in the gut, until here we are today, when I’m warning everyone that Karras, of all people, might publish such a thing.

I really appreciate that you posted this, because I’m so happy I wasn’t alone in the feeling of “what’s going on, here…?”

[gonehome]

That seems like a worthwhile thing to publicize in and of itself?

The replication crisis in psychology threw out 50% or so of supposed scientific results.

If this (or just straight fraud) is common elsewhere, it seems like knowing about that would be a good thing for science.

[msapaydin]

I agree, most top conferences nowadays publish reviews openly, and address this issue I think. Also it is easier said than done, this is so endemic in so many different academic settings, not just in the US, but also in Europe.