Hacker News new | ask | show | jobs
by subroutine 1656 days ago
While I agree with the premise of this article - that the scientific paper is obsolete - the proposed alternative does not address the primary inefficiencies with our current system. Dynamic 'papers' with illustrative examples are surely an improvement over current manuscripts. However, a flashy paper should be well down the list of goals for a research program. At the top of the list should be to make an important discovery or breakthrough.

Currently there is way too much trivial shit being published in the ever-expanding number of scientific journals.

Our current model for career advancement in academia is partially to blame for this. Landing a tenure track job requires having a prolific publication record. And once you've landed that coveted ladder-rank position, the pressure to publish only heats up, with tenure on the line. And then even after you've secured tenure, career advancement still depends heavily on the ol' publication record. Pre-tenure the pressure to publish in high-impact journals is immense; it's the kind of pressure that drives otherwise honest people to consider partaking in fraud (and unfortunately those who stick to their morals often lose out to those who fabricate results to some degree). Post-tenure the pressure changes from securing high-impact papers to just getting on whatever papers you can. Review boards for associate professors would more readily give a promotion to someone with 20 meaningless papers over the last three years than someone with 2 papers in CNS journals over the same timeframe, even though a single paper in Cell/Nature/Science is typically more impactful than 100 papers in Frontiers or other similarly dogshit journals. So post-tenure pay raises are based on getting as many words in print as possible, using the least amount of effort to do so.

I almost forgot where I was going with this... right, so, in my opinion we need to switch from a publication-based mindset to a discovery-based mindset. We (the public) provide the NIH with $30 billion dollars per year, with the idea that such an investment will lead to medical breakthroughs, discoveries, and other innovations that can concretely improve our health and wellbeing. However, so much of that is wasted on the idiosyncrasies of career advancement inside the ivory tower.

If I were the director of the NIH my sole purpose would be to end this nonsense. And my first order of action would be no longer accepting grant applications from individual PIs. I would only entertain grant applications from a small force of scientists (4-8 lab equivalents) with thorough and cogent plans for making breakthroughs on cancer or heart disease etc. I would change the minimum R01 funding amount from <$500k to >$10 million dollars. I would change the grant renewal timelines from every year to every 5 years but require yearly progress updates to ensure the proposed experiment were soundly conducted. I would encourage the equal reporting of both positive and null findings. Performance would not in any way be based on positive findings, only that the experiments were carefully run. I would require that all raw data be deposited into publicly accessible repos (not at the end of the study, not yearly, but whenever data is generated it should be made accessible asap). I would encourage that research groups bother with drafting/submitting interim manuscripts (interim meaning prior to completion of the full 5-10 year study) only if they find something important, otherwise just provide a comprehensive writeup at the end of the study. This final writeup would not be submitted to a 3rd party journal. It would be posted directly on the NIH website I'd have created for such results reporting. Naturally this would also be publicly accessible. That would be a start...

1 comments

if you can magically predict result of research project, this will work!
No that's how it currently works. What I propose aims to fix that. You must have missed this entire section...

"I would encourage the equal reporting of both positive and null findings. Performance would not in any way be based on positive findings, only that the experiments were carefully run."

Would you have considered the LHC experiments "failures" had we not found evidence of the Higgs Boson? I certainly wouldn't have. What matters most is that important questions backed by sound theoretical reasoning are addressed carefully, collaboratively, openly, and with plenty of long-term support. If we do that, we get genuine answers - such answers represent important information whether or not they support our original hypotheses.