[adev_]

I'm gonna get enemies for that ....

But let's face it: the Academia peer review process is pretty rotten.

- Many organisations could not care less about whatever crap you publish as long as you publish. And if possible a lot.

- Many journal's reviewers are friend of friends and you can often easily guess their names, even in double blinded review.

- Many reviewers care more about you quoting their own paper than giving any relevant feedback.

- There is plenty "pay2publish" journals that care about quantity not quality that are commonly used by academia to "force" validate crappy researches/PHds

- The publishing world is full of oversized-ego individuals that will destroy a paper just because it comes from a competing lab/department/professor and not because it is scientifically irrelevant.

And that is only a little subset of the problems...

[dash2]

My experience is different. Peer review is far from perfect but:

* Most of my papers have had thoughtful comments. Typically, they improved the paper, sometimes a lot.

* I have very rarely been asked to quote a specific paper from a specific author. Of course, pointing out important references that the paper has missed is part of peer review, so this would not necessarily be evidence of corruption.

* There are loads of crap pay-to-publish predatory journals, but publishing in those journals would harm my career, not help it. Nobody cares about them.

* I've not experienced people rejecting papers from rival labs or departments. I don't say it doesn't happen, but I haven't experienced it.

* I myself try hard to give high-quality reviews that explore the paper's value in depth.

I am from one particular discipline, country and subfield. Others may have very different experiences.

Again, I am not claiming academic peer review and publishing doesn't need a lot of improvement. Removing the parasitic mainstream publishers like Elsevier and Springer would be a great start.

[woofie11]

It varies field-by-field. In my field, peer review is a joke. I'm glad it's not a joke in your field yet, but incentive structures say it eventually will be. If incentives don't change, it's a question of when, not if.

[andi999]

What incentive structures? When I get a paper for review, I do my (time limited) best. Which incentive will change me?

[woofie11]

Academic hiring and promotion.

High-quality academic curation doesn't effect anything in your career. On the other hand, the impact of your own research does does affect it. I'd like my taxpayer dollars to go a little less towards publish-or-perish, and a little more towards QA to make sure the papers coming out are credible.

[ketzu]

I agree with the general point of your comment, but my personal experience with the usefulness of reviews is not so great.

Nearly all reviews were superficial, not a single review ever found any technical flaw, even when some were discovered later. The acceptance recommendations seem to be governed by the overall impression the reviewer had from the paper and not by any factual points they make in the review. This manifested in getting 5 reviews at one conference, where all reviewers made nearly the same points in their review: all about the presentation of the paper. The paper received all of the five recommendations: Accept, weak accept, borderline, weak reject, reject. (The most critical and useful with a small correction of one formula was the "Accept" review.)

Many reviewers also leave the impression they didn't read the paper in full or at all, or ran out of time while for the review, so looked for a reason to reject the paper and didn't look any further.

Note: I am not saying my papers should have been accepted, 2 had serious issues I glad I discovered later! I also had a few very useful reviews to improve the presentation of my paper.

[timkam]

I largely agree with this comment, but this just means the underlying issues are more subtle. For example:

* I typically get thoughtful reviews, and in particular critical reviews are often thoughtful. Still, I know that some people assert influence on the outcome of the reviewing process of their own papers. Also, there's a fine line: for example, many journals allow you to recommend reviewers for your papers. If you're well-connected, you can recommend your best friends and your editor friend follows your advice. Is this misconduct? I'd say it depends. After some decades in a math-heavy community, most people who are experienced and competent in the context of a highly specialized sub-field will be your "friends".

* Journals that are outright predatory are, of course, a no-go. Still, there are edge cases like some MDPI journals; sometimes there are special issues from credible, relatively junior people in my community. Should we support them or not? Also, my funding provider wants me to publish in high-profile, "big publisher" journals and pays the open access fee; these journals typically rank much higher than community-owned/not-for-profit venues. On the long run, it's better to fully move to community-owned venues only, but as a junior researcher I feel I cannot afford it at the moment.

Personally, for me the problem does not come from outright and obvious ethics violations; sure, I know of some individuals who I consider somewhat problematic, but I don't think these people control the community. For me the problem is to navigate all the grey areas, knowing that the system is largely ethics-agnostic.

[8note]

I find it intriguing that reproducing the experiment isn't a part of the review, or visiting the lab to look for potential flaws in their set up.

These reviews seem far from what you hear about for disproving N rays and the like

[swiley]

Do you checkout your co-worker's branches and run the unit tests for every PR they make? In an ideal world that might be nice but in the real world with normal sized projects that's to time consuming so everyone just reasons through patches.

[andi999]

If reproducing were a requirement for reviewing, publication cost might be exorbitant (like when CERN tries to publish a paper you have to rebuild the particle accelerator?)

[bjornsing]

So how do you explain the information in the OP?

[dash2]

I don't know enough about the specific situation of computer science at the University of Florida to add anything useful. It sounds like a serious scandal.

[Vhano]

yes, this is localized and the system is not corrupt as a whole.

[CobrastanJorji]

Oh sure, the system is bad; it has perverse incentives and insufficient checks. But that doesn't also mean that these specific actors aren't more specifically bad in very concrete, condemnable ways.

[izacus]

> - Many reviewers care more about you quoting their own paper than giving any relevant feedback.

My supervisor once told me that I should quote a paper from a pair of reviewers (he knew exactly who was reviewing the papers) before he'd allow publishing. It was a CS paper and the reviewers were in the field of geology.

A few of those fun cases were the reason why I left the academia - corporate bs doesn't come close to academic bs.

[ntsplnkv2]

Academic BS is just as bad, if not worse. You have far less options. It's way easier to just "find another job" than it is to "find another phd program" and colleges/professors abuse the shit out of it.

[eptcyka]

I take it you meant to say 'competing lab' instead of 'concurrent lab' in your 2nd to last comment.

[Tistron]

Curious. In Swedish we say "konkurrerande" for "competing", I guess adev speaks a language that does something similar. In Danish they say konkurrence for competition.

[lou1306]

French and Italian also use similar terms (concurrence, concorrenza) for business competition, so it's not a far-fetched hypothesis.

[rjsw]

And for sports competition.

[gambiting]

Konkurencja in Polish, which sounds very very similar to concurrence

[eptcyka]

And in Latvian, probably Slavic labguages too.

[adev_]

Indeed. Fixed.

[dehrmann]

This sounds like something like the tragedy of the commons or Gresham's Law for academia. It's in everyone's best interest to do take advantage of the system, at least as long as there's public support for academia.

I don't see the situation getting much better. Republicans like free-market solutions, but not facing market constraints is part of the value of academia. Meanwhile, especially with consternation around student debt, Democrats will blindly support academia, failing to put any pressure on it.

[jgeada]

The reason you know about academic fraud is that lots of the process is public and has many unaffiliated people involved that are potential leaks. So, sure, fraud happens and clearly that’s bad. But you do hear about it.

By contrast the same events within corporate culture happen behind veils of secrecy and HR departments & heavy hitting lawyers to make life hell if you don’t toe the line. And the cult of personality and oversized egos is very well represented in boards and upper management.

[pca006132]

But the problem is, apart from peer review process, what measures can be used to ensure the quality of the paper being published? And what should be used as the metric for academics?

[Gatsky]

Peer review is an ok idea. Academia is the problem. I saw two wasps fighting on the carcass of a dead bird the other day, it was a very apt analogy.