[ysapir]

Quality control is essential. I have an interest in linguistics. But on the internet, some of the linguistics arguments are ridiculous. There are websites that claim that English (and all world languages) are descendant from Turkish. It is rubbish. In all types of subjects you find people with these pet theories, and they can be very prolific, putting their opinion on wikipedia or wikia or wherever, on websites, in discussions, and there can even be a following. In software, you can tell rubbish. It compiles or it does not. It has a lot of bug requests or it does not. This does not map to science. You can't run a scientific article through a compiler to tell if it is good or bad. You can't tell if a new physics theory is reputable or some science fiction. A theory may be 10 years old, and the professor who wrote it unable to respond to all the queries, "bug requests," but it is still valuable. And a prolific pseudo-science author may have little "bug requests" because no one reputable who knows something about the subject has any time to deal with his nonsense. Without effective quality control, there will be no science. The article had no real solution to the problem.

[fchollet]

Actually, you can (run a scientific claim through a "compiler"): it's the principle of reproducibility. You should be able to repeat the steps of the researcher (whether it's an algorithm or a biology experiment...) and get the same results.

And if we're dealing with a field where there is no objective way to verify a claim, then any claim should be viewed as mere opinion (a more or less valid opinion depending on how mainstream it is). As for fields where all claims are in the realm of opinion... they're not actually part of the scientific family.

[ysapir]

Even in natural sciences, there may be experiments that are not easily reproducible. Finding the Higgs can only be done in a long while (decade or more) with great financial investment.

Or take the 4th grade test about dinosaurs[1]. Objectively, we can't verify if the world if thousands or millions or billions of years old, and we can't verify if dinosaurs lived concurrent with humans or not. We weren't there. There is evidence, and how we interpret the evidence, and yet the test features a rather forced interpretation of the evidence. Now, numerically there may be a lot of people all over the world who prefer the fundamentalist interpretation, even if they are accredited scientists in universities. In a completely open environment, this opinion would get more weight than it deserves, a weight that does not represent its true standing among scientists who understand all the different implications of the evidence.

[1] http://www.snopes.com/photos/signs/sciencetest.asp

[saraid216]

The OA is not proposing that we do away with peer review.

[ysapir]

That isn't clear to me, but in any case, they seem to be talking about a new type of peer review process, which is more immediate -- something akin to forking on github or editing a wiki maybe. I'm not sure.

From the article - "They argue that the current journal system slows down the publication of science research. Peer review rarely takes less than a month, and journals often ask for papers to be rewritten or new analysis undertaken, which stretches out publication for half a year or more. While quality control is necessary, thanks to the Internet, articles don’t need to be in a final form before they appear. ... “We want to go after peer review,” CEO Toni Gemayel told us."

And I am saying, quality control is essential, and yet an open internet-based process would mean a lot of people with pet theories they want to drive could game the system. I agree with the poster who wrote that you need to take into consideration the author and his caliber even in scientific journals -- the quality control problem is a problem already today really -- but the effort necessary to get an article to publication raises the quality somewhat (and in turn prestige of relevant journals).

[saraid216]

> I agree with the poster who wrote that you need to take into consideration the author and his caliber even in scientific journals -- the quality control problem is a problem already today really -- but the effort necessary to get an article to publication raises the quality somewhat (and in turn prestige of relevant journals).

You realize that your argument is essentially "the blogosphere is not real journalism", right?