[nonbel]

Yes, we want to know what observations we can "hang our hats on", so that we can come up with actual theories (not vague crap like "this drug makes that disease happen less"). The more replications published the better until it is being done in undergrad/high school classrooms.

You can't rely on anything in the current half-assed, no replication environment. In many areas it is literally not worth coming up with an mathematical model to explain the data because it is all so questionable.

[IanCal]

I'm not really sure how that addresses the issues raised.

You're saying it's better overall if more replications are published. I'm talking about the problems that are stopping us from getting to that point. The current incentives do not align well with the desire to have more replications done, and simple changes could easily backfire.

> (not vague crap like "this drug makes that disease happen less")

I simply do not agree that if you think this and find some results that point in that direction you should not publish. I see absolutely no reason to save electrons to improve some average of papers, I'd much rather work isn't re-done repeatedly. Perhaps publishing something vague with some backing (e.g. we think X does Y and the data is at least consistent with this, and we can't think of what else makes sense) gives enough for someone else to build on and do a more rigorous investigation.

[nonbel]

In my initial post I distinguish between two steps. The first is data. If you collected some data and can describe the methods well enough for others to replicate it then go ahead and publish it. There is no need to start theorizing about things like "the drug caused the difference" vs "there was a confounder that caused the difference".

Explaining the observations is a separate step from coming up with a reliable way to generate a pattern in the data.