Hacker News new | ask | show | jobs
by renewiltord 1907 days ago
The difference with CS is that there's no one to trust. Everything you need is right there.

Other disciplines have people fake results. You can't fake a result in CS since people only really attack decidable problems, and you need no empirical results for any. It's like how even though mathematicians ultimately do make errors, Mathematics has no replication crisis.

However, most so-called 'hard' science are also full of knowledge that poorly models the world. Sadly, since most people who say this are also climate kooks, everyone will label you as a climate kook if you say this, though.

But anyway, here's a fun thread https://twitter.com/MicrobiomDigest/status/13740469245510328...

People constantly downvote me on HN for claiming that there are photoshopped papers out there. I'm not complaining about the downvotes. We make the culture of this place together and I can accept if I say things that you believe aren't right. But I am right.

Edit since rate limited: oh, I didn't consider ML. Okay, consider my position reversed. Yeah, the empirical sciences do have reproducibility issues.

1 comments

The ML field alone is full of unreproducable papers.

Example: "“Probably 50%-75% of all papers are unreproducible. It’s sad, but it’s true,” another user wrote. “Think about it, most papers are ‘optimized’ to get into a conference. More often than not the authors know that a paper they’re trying to get into a conference isn’t very good! So they don’t have to worry about reproducibility because nobody will try to reproduce them.” https://bdtechtalks.com/2021/03/01/papers-without-code-machi...

On the Papers without Code page, however, I see only 11 submitted papers, out of which 5 have been marked as resolved: https://airtable.com/shrWz8OF3uMZ8G4cY/tbl5ZzB7ahIui1EoD

This does not seem very good data to empirically underlay the claim of 50%-75% papers being irreproducible. I don't doubt that there are (too) many of them, but having some real data before making strong claims about numbers would be more credible.