[BeetleB]

> The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level, for heaven’s sake.

This may vary based on discipline, but in both the subdisciplines of experimental and theoretical physics I was involved in: No - very few will provide the data/derivation. My professors were very open about this: They don't want to lose their competitive edge. Almost no experimentalist I knew could take papers from his/her field and reproduce the results, because the papers lacked enough detail to do so. They would mention a technique, but there are lots and lots of nuances involved when building equipment to carry out the technique[1], and these are intentionally excluded. It's unlikely you'll be able to build the equipment the same way the original authors would.

[1] Most experimental physics involves building your own equipment, or at the least modifying existing equipment.

[AOsborn]

Serious question: if the experiment can't be reproduced to be verified, how does the paper provide more proof than a blanket 'trust me'?

From someone who isn't an academic, isn't this letting politics come before science?

[BeetleB]

> if the experiment can't be reproduced to be verified, how does the paper provide more proof than a blanket 'trust me'?

It doesn't, and it is a big "trust me". People review papers based on the merits of the idea and methodology, and then tend to trust the results. Of course, if the results are very "significant" (e.g. cold fusion), then it will be scrutinized more, people will fail to reproduce, and they will harass the author. 99% of papers don't fall in this category, though.

> isn't this letting politics come before science?

Yep. The games at play are often: "How do I write my paper in the most convincing way?" and "As a referee, this paper is hurting the research work I am currently doing. What is the best way to reject this paper?"

The extremely annoying part was I felt I was back to taking literature courses, where I'm graded on very subjective metrics. It was horrible, especially when all my work was extremely objective. However, the publishing system is not incentivized to be that objective.

Simple example: A colleague's paper was rejected because he explained a phenomenon using method A, and the referee complained there was no mention of method B. Method B was the hot topic of the day. Neither method A nor method B had good empirical data to support it - it was almost purely theoretical at that point. But that community was gravitating towards method B, and really did not want to see alternative explanations.

[AOsborn]

Thanks, interesting to consider.

[aiphex]

If you are working in the same niche, you will likely know the tricks of the trade. You usually have an idea of what your peers are working on and it's often a race to see who gets a paper out first. This is what conferences are for as well, to try and figure out what your peers are up to.

[Terr_]

Sounds like there's a need for some sort of time-based escrow on "the full paper"... Of course, game-theory strikes again because nobody will want to pay for it.