How do peer reviews happen, then, if nobody gets paid to do them? Does research money (that goes to the production of the paper) not also cover peer-reviews?
I wouldn't see any incentive for any work to be peer-reviewed, ever, then.
We tend to think of journals as being run by their editorial boards -- i.e., by scientists in our field. The publishers are in the background, not taking a very active role, simply skimming off truckloads of money in the process. In particular, peer-review requests tend to come from scientists whom we know at least by reputation, and often personally.
When I spend serious time peer-reviewing a paper, it will be on a quality paper on a subject that I enjoy reading about anyway. (If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, I'll either decline the request or write a quick report recommending that the paper not be accepted.)
It's interesting work, it's important for the health of the field, it helps me support other researchers in my field (if I like their paper and give it a positive review), and it helps my own reputation (the journal editors are often bigshots). From this point of view, it's a win-win-win-win.
Finally, paying for peer review doesn't make much sense because the time required is highly variable. I might take between 20 minutes and 30+ hours to review a paper, and I wouldn't necessarily expect the editor to be able to predict how much (even within an order of magnitude) in advance.
However, I see no reason why high-profit publishers should be able to insert themselves as middlemen and engage in rent-seeking behavior. I hope we figure out a way to give them the boot, and then continue approximately as we have been doing.
> If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, ... or write a quick report recommending that the paper not be accepted.
Often (but definitely not always), requests to peer-review are accompanied by statements like "We consider our Journal X to be on par with Journals Y and Z, and aim to be extremely selective. Please only recommend publication if blah-blah-blah...."
I'll then read the introduction. If I don't have sufficient expertise to referee the paper, I'll decline the request. If I do, then I'll see what the authors have accomplished. If, in my judgment, this doesn't rise to the level that the editors asked for, then it doesn't take me a long time to decide this and say so.
Conversely, if the introduction does impress me, then I will want to check the proofs in very close detail. In this case I will commit to writing a detailed report in the future.
It's quite common for papers to be declined from individual journals; it's happened to me plenty. There are tons of other journals out there; you can submit somewhere else. And when I decide that a negative report is called for, I write it right away, so as to not keep the authors waiting forever.
I think the "harsh" comment was because you suggested you might recommend the journal reject a paper because it was outside your interests and expertise, not because of any lack of merit. But that implication only arose from some selective editing.
Not really. If work is flawed, it will need to be fixed before recommending it to readers (which is now the only marginal value provided by editors). If it appears to be unfixable, the correct choice may be to recommend that it not be endorsed by a journal.
You can always put work up on arXiv if the goal is priority or distribution. Peer review is, in principle, designed to ensure that work is new and true (and that what is new is true). It’s not meant to fix flawed work, but to make good work better.
The problem is the preceding if clause. You say that you do this if you find it uninteresting, or if you have no expertise in the field of the paper. Both of these are qualifiers of you, not of the paper, so indeed it seems harsh to punish the authors for the bad referee selection of the editor.
He'll decline the request or say it should not be accepted. Declining the request is just fine, and a reasonable thing to do. There are many mathematical results that are in fact uninteresting (to just about anyone) and it's fine to suggest those be rejected from a good journal.
I've had many paper rejections, none because of false or uninteresting claims but almost all because "it's not good enough for this prestige journal" or "it's too specialized for this journal". I expect a certain number of those, because I submit to reach journals now and then and hope I get a reviewer and editor that like the work :) If I wanted certain publication immediately I'd submit to lower-tier journals first, which I have also done when I just needed something out.
Economics and linguistics also publish their own journals which costs some money but significantly less than Elsevier's subscriptions (I believe linguistics' is 300 USD per year). There's probably enough momentum behind middlemen-run journals that prevents this from happening more en masse.
I've only worked with professors in economics and political science but I'm under the impression that this is true more generally of academia. Some people do it to help advance the state of the field. Career incentives and standing in the field also come into play. As the poster above mentioned, a professor would be able to say more.
To respond succinctly to your comments: money isn't the only incentive in life, so it seems weird to not see "any incentive" as soon as money is taken out of the picture.
I'm a professor (in mathematics), and there are some reasons why I personally might say "yes" to review a paper. (1) If the paper is so interesting that when I see it I think "I really, really want to read this paper!", then I am likely to peer review it because as a reviewer when I read it, I get to ask any question and the author will take my questions and concerns very seriously. (2) I owe the editor who is asking me to peer review the paper (or maybe even the author of the paper), e.g., if the editor put a lot of work into ensuring a paper I once wrote got properly reviewed. (3) Even if I say "no" to fully reviewing a paper, as an expert in the area I can often provide quick valuable feedback about the relative value of the claimed contribution, and who is most likely to be suited to review it; I'll do a quick informal review like that in some cases.
From the outside, it is easy to forgot the extreme extent to which academics are motivated by (or view themselves as motivated by) principles of "good citizenship". This is selection -- the people who aren't this way, often do not get hired or promoted. Also, many academics (at least in pure math) view motivation for money as a "lower order term". It's a good thing for them, because for much of your long career in academia there is little you can do to impact your salary, besides applying for a job somewhere else. For me it's always been: each year you get some maximum possible merit raise of between 0% and 4%, depending on external economics that the department has no control over. Academic book royalties might also raise your yearly salary by 2%. Being highly money-motivated in some parts of academia would end up being very frustrating indeed. E.g., even when I've got a big NFS grant, that doesn't change my salary one bit; instead, it changes how many students and/or postdocs I could support.
Well, yeah, but without students & postdocs it is very difficult to make a material impact on your field of endeavor.
There’s also an ulterior motive for some lines of work, where sending a paper to be critically reviewed by rivals is a mechanism to torture test the work & conclusions.
This can be taken too far (especially in biology, the glam fluffing and $million additional irrelevant experiments are legendary), but in principle, if even your most motivated critics can’t find a fatal flaw in your work, it’s a reasonable bet that the work is sound. At least, that’s the principle. Editorial overrides sometimes break this safeguard, though (lord knows I’ve seen a few).
Well, yeah, but if you submit a paper to "torture test" and it gets rejected, that cuts off a journal to be able to submit to.
Well, yeah, but I think what he's saying is it's probably the case that a large number if not majority of academic make the decision to go through the grueling PhD process and give up years of earnings in their prime for things other than monetary gain. But of course people can become disillusioned later down the road.
I claim that it’s better to request major revisions than recommend rejection, but that’s mostly due to my lack of faith in authors.
Too many examples of “oh well, let’s try the next journal” and not enough of “gee maybe we should fix these glaring flaws”. It’s not because I’m a nice person; it’s because I would prefer the literature not to be a toxic waste dump. Nobody is entitled to be published anywhere. In an ideal world, doubly blinded review would become a part of the published record. At some journals it already does.
The reviews offer valuable context, which is often sorely lacking in high profile venues. (The canonical examples of STAP and arseniclife come to mind, but also much more subtle details where an overall sound paper somehow only gets cited for the one shaky assertion in the results)
My primary motivation in trying to do a good job as a referee is that if I write a paper, I also want a referee who tries to do a good job. This can improve the quality of a paper tremendously.
> To respond succinctly to your comments: money isn't the only incentive in life, so it seems weird to not see "any incentive" as soon as money is taken out of the picture.
Especially for a group of people where many of them have taken a pass on more monetarily lucrative careers.
In my experience, academia is largely driven by money. Professors at my university were recognized for the money they brought to the university first and foremost it seemed, rather than for any discoveries/innovations in a particular field.
Ah, I know a former engineering professor who left a large flagship university in the Midwest for exactly that reason. Based on what he told me that's more a phenomenon specific to institutions without a lot of money than a general truth about academia.
This resonates with me, as my alma mater definitely doesn't have much (compared to the top endowments out there). It's not amount to sneeze at, but still, there are others ahead of us.
The incentives are 1) professional responsibility; 2) the opportunity to participate in the direction of new research, stay abreast of the field, and maintain a high level of integrity; 3) young researchers joining the field benefit from the process, both as reviewers and authors.
I can't remember the last time I've read a headline about the peer-reviewer that confirmed some groundbreaking finding.
Maybe it gives you the opportunity to network with higher-profile contributors to your field, if you offer to peer review their work (though to be honest, I don't know how the peer review process works, and if you can even "offer" to perform it for a specific paper).
At least in the social sciences, journal editors delegate papers to reviewers. Your first sentence also suggests that you don't know what peer review is.
Peer review is not the same thing as replication, which is replicating the results of a paper after it's been published (e.g., confirming some groundbreaking finding). Peer review happens at the stage before publication of the original paper. Researcher(s) submit the paper to the journal. The journal editor sends the paper out to some reviewers, who review the paper (this is the "peer review" stage). Pending reviewer feedback and editor approval, the paper is published.
Edit: also the "benefit" that rsa4046 refers to probably doesn't mean networking. AFAIK, reviewers are always anonymous to the authors (which can generate its own problems e.g., if the reviewer gets a paper authored by someone he/she doesn't get along with). The benefit being referred to, I believe, is that of learning to write better reviews, and having reviewed other's work, learning how to improve your own.
Well, yeah, but learning to review is less difficult than learning how to communicate clearly, crisply, and forcefully. So in a well administered review, most benefits accrue to the authors even if the reviews are poorly done, so long as they are ethical. (Editors can step in when reviewers are being unreasonable or ignoring explicit instructions; unfortunately professional editors at glam journals can also step in to push exciting or politically expedient work into press before there is time to adequately vet it, which sucks and gives everyone a bad name).
Well, yeah, there are other irrelevant things about reviewing that I also have the ability to bring up but my point is that to say that the "benefit" of reviewing is for the purpose of networking is misinformed.
I sometimes agree to review out of a sense of obligation, where I know damned well that the authors recommended me, but you’re right, that’s not networking per se. There’s no reasonable expectation of benefit and it would be unethical as hell to request any.
The authors most likely won’t (can’t) ever know that I agreed to review their paper; it’s more of a good citizen affair. Sometimes, afterwards, it will become apparent that a particular referee was someone familiar. I would like to think that the original poster was imagining something along those lines, but your take is probably closer to the truth. Oh well.
We tend to think of journals as being run by their editorial boards -- i.e., by scientists in our field. The publishers are in the background, not taking a very active role, simply skimming off truckloads of money in the process. In particular, peer-review requests tend to come from scientists whom we know at least by reputation, and often personally.
When I spend serious time peer-reviewing a paper, it will be on a quality paper on a subject that I enjoy reading about anyway. (If I get asked to peer-review a paper that looks uninteresting to me, or something outside my interests and expertise, I'll either decline the request or write a quick report recommending that the paper not be accepted.)
It's interesting work, it's important for the health of the field, it helps me support other researchers in my field (if I like their paper and give it a positive review), and it helps my own reputation (the journal editors are often bigshots). From this point of view, it's a win-win-win-win.
Finally, paying for peer review doesn't make much sense because the time required is highly variable. I might take between 20 minutes and 30+ hours to review a paper, and I wouldn't necessarily expect the editor to be able to predict how much (even within an order of magnitude) in advance.
However, I see no reason why high-profit publishers should be able to insert themselves as middlemen and engage in rent-seeking behavior. I hope we figure out a way to give them the boot, and then continue approximately as we have been doing.