[lusus_naturae]

> The point of doing "reproducible science" is not that I write a paper, you email me asking how did I come up with my number, and I email you back an explanation. No! The important details should be in the paper already.

This is how it works, people email each other all the time. Why shouldn’t you? You can’t imagine every bit of information that someone would want, and papers have page length limits so you make choices about what to cut.

[michaelt]

Don't you think that gives the people who are falsifying results rather an easy time?

If Bob falsifies data and someone e-mails him, asking him to send them rope to hang him with, he can simply delete the e-mail.

Or claim he forgot the details of the analysis. Or claim it was handled by a grad student who left. Or claim the info was lost when a hard drive broke. Or claim the data was the intellectual property of College A and he can't access it now he's at College B. Or claim privacy or copyright rules cover the key data. Or that they don't have a license to the software that can open the data files any more. Or any of a dozen other things.

[topaz0]

Any of those responses (or non-response) by Bob would justify the asker in their skepticism of the falsified results. Giving somebody a chance to respond is not the same as relaxing your standards of evidence, it's just an acknowledgement that there might be explanations you haven't thought of and giving an opportunity for those to be brought up.

[lusus_naturae]

All anyone can do is try to use that research in their own work, and see if their work supports findings from prior work. Sometimes it doesn’t, I am not sure if that means someone lied on purpose. It’s possible they were bad at interpreting the results, or they made bad assumptions. I think poor research standards are the main reason for the reproducibility crises, and not people lying on purpose.

Typically bad research assumptions or implementations are rooted out during peer review, but it’s an imperfect process.

I do think there needs to be a dedicated non-profit and neutral organization solely responsible for reproducing scientific results for all fields, and assigning a reproducibility score to research finding. This could become an entire field by itself, and would have its own complications, but the reproducibility crises does exist and needs a solution.

[hnfong]

A paper should have enough information for an independent researcher to reproduce the results.

Otherwise, years later, if the author dies, the paper would be basically worthless for reproduction.

(We're still talking about reproducibility crisis, right?)