Hacker News new | ask | show | jobs
by yayday 3637 days ago
You're already moving your goalposts. Your first post said replicates the finding. If I simply post my results and someone else looks at the data and agrees with me then are they really replicating my finding or did they, oh what's the word… peer review my work and agreed?
1 comments

I really dislike the idea that "Your code runs on your data" is a form of replication.
It's a minimum standard and one that many published works cannot meet. Universal enforcement of this minimum would itself be a big step forward.
It is a minimum standard, and a useful one. It mostly annoys me when certain groups of advocates seem to treat it as an end in and of itself, and evidence that something is "repeatable" and will end the problems in obtaining scientific evidence.

I come from a field where something isn't a reproduced until it's also found in an entirely different study on a different population.

Oh, same here. Two of my current manuscripts have now been replicated in 3-5 independent clinical cohorts. The thing is, if the field won't even recognize that "ability to produce the same outputs given your inputs" is critical, good fucking luck with things like "actually replicates in a separate population".

Neuroscience is particularly horrendous about this.