Hacker News new | ask | show | jobs
by bildung 1913 days ago
The ML field alone is full of unreproducable papers.

Example: "“Probably 50%-75% of all papers are unreproducible. It’s sad, but it’s true,” another user wrote. “Think about it, most papers are ‘optimized’ to get into a conference. More often than not the authors know that a paper they’re trying to get into a conference isn’t very good! So they don’t have to worry about reproducibility because nobody will try to reproduce them.” https://bdtechtalks.com/2021/03/01/papers-without-code-machi...

1 comments

On the Papers without Code page, however, I see only 11 submitted papers, out of which 5 have been marked as resolved: https://airtable.com/shrWz8OF3uMZ8G4cY/tbl5ZzB7ahIui1EoD

This does not seem very good data to empirically underlay the claim of 50%-75% papers being irreproducible. I don't doubt that there are (too) many of them, but having some real data before making strong claims about numbers would be more credible.