[blueMist]

Agree with a lot of what is being said here. Have seen vicious anonymous reviews in CS - as we all have - both due to someone misunderstanding the work but also due to people from "a different camp" simply disliking a given approach. This must be common in other fields as well. Someone said on this thread that anonymity is not a guarantee of quality - completely agree. The move towards "open reviewing" for conferences in CS is very encouraging, however we will see how far it gets.

[javert]

I'm going to say something controversial.

Minor reform of the referee system is treating the symptoms, not the disease.

The disease is that there is no honest, mutual, voluntary exchange between a party that values a given research project and the party that produces it.

Rather, we have federal bureaucrats handing out money taken from taxpayers willy-nilly via an old-boy network as described above. (The difference between taxpayers and slaves is that taxpayers only must sacrifice a fraction of their productivity to this absurd system, not all of it.)

In CS, all the research we do either goes to benefit shareholders of companies that ultimately profit from it, or (the majority) is just ignored because it's part of the paper mill competition. Those companies should be the ones paying for the research, not the taxpayers. Those companies are free riding, and "we the people" should put an end to it.

[wmf]

I'm not sure what the one has to do with the other. My research is corporate-funded and my papers still get reviews from people who didn't read the paper carefully. Unless you're proposing to eliminate the concept of publishing research at all...

[javert]

What one has to do with the other is that the entire system is broken, here's my explanation for why the system is fundamentally messed up, and my explanation for how it should be. Simply trying to change refeering by itself is treating the symptom, not the disease.

> Unless you're proposing to eliminate the concept of publishing research at all...

I think putting my research on my website would be just as good as publishing it in a conference proceeding. I'm not against sharing research and having it be public, but I think conference proceeidngs and journals in CS are of very little value. If people want my research, they can find it on my website. And I think this generalizes... if I want your research, I can (presumably) find it on your website.

Ultimately, it may be useful to have some kind of aggregation of what research is coming out in various subfields. Internet fora and the like would serve the purpose fine. But sure, having a group of referees and some way to screen research and highlight what is good could certainly be useful. All that stuff should develop organically as it is needed, though.

The model we have now, with peer review, journals made out of dead trees, etc. is a hold-over from a pre-Internet time when a few people in every sphere of life controlled the information that was disseminated, because it wasn't possible for things to be open, since we lacked the technology. That is no longer an issue. Of course, academics will hold onto their little racket as long as they can (probably indefinitely).