| > the accept rate is about 10% after multiple rounds of review/revision This is part of what I'm driving at here - "accepted for publication in this journal" is not the same as "peer reviewed". Broadly speaking when someone says that a paper has been peer reviewed, they only mean that a third party has examined the paper and certified that they did not find any significant issues with the methodology. The GP didn't say "this hasn't even been accepted for publication" - they said "this hasn't even been peer reviewed yet". Those are very different things! I would see acceptance into a top-tier journal - or even a reputable journal - as a very, very strong signal. I'm still curious what proportion of papers meet the bar for inclusion into a journal but are filtered out by the peer review process. I suspect it's vanishingly small. > Then when you don't get accepted there you go to a 'just publish my paper' journal [...] Right, which is why I always take the conclusions of a paper with a grain of salt inversely proportional to the prestige of the journal in which it was published. If I'm looking at a paper in Nature, I'm as confident as is reasonable that the conclusions of the paper are correct (or at least well-founded). If I'm looking at a paper that hasn't been published in a journal of any type, then I essentially discard their conclusion and consider the data and the author's analysis (with a healthy dose of skepticism). In this case, even as someone who is definitely not a professional in the field, I identified what I believe to be glaring methodological errors: https://news.ycombinator.com/item?id=23690529 |
Not anymore, I'd say. Both The New England Journal of Medicine and Lancet were fooled by the Surgisphere scandal, and they didn't even notice the statistical errors, let alone the fabrication of the data.
Lancet took years to remove the Wakefield paper on autism and vaccines, and still has other papers with questionable statistics up.
What I consider a strong signal is good data with clear conclusions and well-outlined limitations (these are often as important as the results themselves). Peer review can help with that, but it's not a magic bullet. In that sense top tier journals can be "dangerous" because sometimes they focus on good storytelling instead of presenting the data correctly.