Hacker News new | ask | show | jobs
by godelski 745 days ago
Peer review is great in theory, but fails because it makes too many assumptions and because the incentive structure. And it certainly doesn't scale well. It's important to remember that the point of publishing is communication. Anything more or less is ill founded.

It is obtuse to think that a few people can read a paper and know it's validity. It's falsifiable but even that's fuzzy. The problem then comes down to the incentive structures. Why do people cheat? Because we're lazy evaluators. It's odd to me that we won't read the works of peers in a department, lab, whatever. But doing that would be a much stronger form of evaluation than anything that could be inferred from citations, h-index, conference ranking, etc. Plus, the structure is to push novelty fast and frequently. That's not only not possible but ignores a fundamental aspect of science: reproducibility.

But this also doesn't mean there aren't scam publishers and publishers scammers prefer. But I'd say that those are a result of the former issue. Because metrics are not being treated as guides. It's just Goodhart's Law in action.

1 comments

entirely agreed

i don't think mdpi is a scam publisher, but then, i don't read mdpi papers from following some kind of latest-mdpi-papers feed; i read them because other papers cite them, so i couldn't tell you if the utter-bullshit-paper percentage on mdpi is 1% or 99%

i just know i heave a sigh of relief when the paper i'm looking for turns out to be on mdpi, because i know that not only will i be able to read it without hassle, it will have a clearly marked creative-commons license that permits me to archive and redistribute the paper. same with hindawi actually, though i'm mmaybe a bit prejudiced against hindawi papers

Yeah I can't say anything about MDPI, and this isn't in my domain. My domain is in ML and all I can say there is that the signal to noise ratio over conference publications and arxiv papers is within error. But research continues for the same reasons it always has, because people are communicating and niches know their niches. But I think it doesn't bode well for Academia or even industry, who are letting the metrics dictate how they evaluate people. Especially for industry, where things have to end up working. I think Gemini is doing a great job at showing how being great on benchmarks doesn't mean you're a great tool. It's because benchmarks are only guides. When they aren't, they will be hacked (if they already aren't). And that's a dangerous situation to be in.
here are some mdpi papers i have in my bookmarks file; all of these seemed pretty decent, if i recall correctly

http://www.mdpi.com/1996-1073/9/8/622/htm a 2016 #paper on #solar #energy payback times and EROEI (“EROI”) and life-cycle analysis (LCA) and net energy analysis (NEA). In particular fixed ground-mounted “multicrystalline” silicon #photovoltaic cells have EPBT of 0.9 to 2.1 years depending on irradiation (insolation), and an EROEI of 15 to 35. This is surprising because that’s pretty much what it was when NREL published an EPBT analysis around 2001 or so.

https://www.mdpi.com/1996-1073/2/1/1/htm “#Energy Inputs in #Food Crop Production in Developing and Developed Nations,” Pimentel’s #EROEI #paper where he found about four joules returned per joule spent on industrialized #agriculture, #CC-BY, Energies 2009, 2(1), 1-24; https://doi.org/10.3390/en20100001

https://www.mdpi.com/1424-8220/21/2/438 Elfring, Torta, and van de Molengraft’s more comprehensive #particle-filters #tutorial #paper (with #PDF) which supposedly has example code that I can’t find. This is recent (02021) and open-access (#CC BY, I think), and many pages long, and has an overview of a lot of the motivation, but in some sense not very approachable.

https://www.mdpi.com/2075-163X/12/2/220 #CC #paper describes various excavation methods (plasma, foam injection, etc.) including #expanding-grout, which it says works by hydrating lime under confinement and also producing ettringite, but lists four different types: type K (4CaO·3Al₂O₃·SO₃ + 8CaSO₄⋅H₂O + 6Ca(OH)₂ + 74H₂0 → 3(3CaO⋅Al₂O₃⋅3 CaSO₄⋅32H₂O, can be enhanced with silica fume and plasticizer), type M (CaO⋅Al₂O₃ + 3CaSO₄⋅H₂O + 2Ca(OH)₂ + 24H₂O → 3CaO⋅Al₂O₃⋅3CaSO₄⋅32H₂O), type S (3CaO⋅Al₂O₃ + 3CaSO₄⋅H₂O + 26H₂O → 3CaO⋅Al₂O₃⋅3CaSO₄⋅32H₂O), and class [sic] G (CaO + H₂O → Ca(OH)₂). Calls it “SCDA” and mentions ASTM C 845 “Standard Specification for Expansive Hydraulic Cement”. It says they work too slowly (12h) to replace explosives in the usual mining cycle, but I suspect that’s not true of class G, where the main problem is keeping it from reacting to completion before you can pour it into the hole.