[epolanski]

Former solar researcher here, had the same experience.

I'll summarize it like this:

- join one of the most prestigious laboratories for my master's thesis in the world

- be assigned work based on a paper published in the same lab by a previous researcher

- can't replicate the results for s*t for months, put in insane overtime hours getting ridiculously good at all the processes, still nothing

- randomly talk about my issues with a random phd in the lab (great scientist with tens of thousands of citations) which quickly scans the data and notes that the voltage obtained by the system in the publication is literally impossible, but by raising the voltage you can easily fake out the amount of electricity generated by the system. Nobody really caught it before because you need some very intimate experience with those systems, and it's just one random (albeit important) point.

- ask why this happens

- she explains that only high impact numbers get citations, only citations get you a chance to progress in the academia pyramid

- she explains that only professors that run labs with a huge number of citations can find good funding

- only good funding can allow you to get the material, equipment and countless number of bodies (phds) to run as many experiments as possible and thus grow your position in the scientific world

Essentially there's way too many incentives to cheat and ignore the cheating for all the people involved.

And due to the fact that as soon as you enter a niche (and literally everything is a niche in science) everybody knows each other toxic things happen all around.

I wanted to be a researcher, but having wasted ultimately 7 months of my life trying to get numbers that were impossible to get, and having understood it was ALL about money (no funds -> no researchers/equipment -> papers -> citations -> funds) and politics I called it quits.

I don't know how to fix it other than several governments and their education ministries making a joined effort to have scientific papers where each result has to be thoroughly reviewed by multiple other labs. It's expensive, but I don't see other ways.

[alisonatwork]

My mother worked in academia on the teaching and administrative side and said it was pretty much the same there too. In her experience even at public universities, it was all about the money. In order for the department or faculty to justify itself, it needs to bring in revenue, and the number one way to do that (along with those research grants) is international students. But the most reliable source of international students who can afford the fees are not necessarily the world's best and brightest, they're the kids of wealthy elites who see education as a business transaction - we pay, you pass our kids. So the syllabus is adjusted to suit, and the teaching methods are adjusted to suit, and in the end everybody suffers because the system becomes structured around keeping the money train coming in. The education still happens, just like the research still happens, but it's happening in a suboptimal fashion, or as a side-effect rather than as the primary focus.

Sometimes I think about tapping out of private industry and getting into academia because in my imagination at least the work would be more pure, but then I think back to the stories my mother told me and realize most likely it isn't.

I agree that the only answer seems to be serious change at the highest levels of government, i.e. revolution. Aside from advocating for that, it seems the best we can do is try exist within these systems and find niches where we can create value for society without feeling too much like our morals are being compromised in the process. It's not easy.

[foxglacier]

Great story and it shows what everybody knows but won't say - normal academics are the frauds. Not just mysterious strangers in foreign countries, paper mills, etc. but normal medium or high status academics in prestigious universities in western countries doing the fraud themselves.

You can say it's not their fault - they're forced into it because it's the only way to succeed. But it's still their fault. I remember a case of a politician in China being arrested for corruption and he said that the only way to reach his position was with corruption. Yea he was probably right but he was still corrupt.

[epolanski]

For the curious, the laboratory I'm talking about is the Laboratory of Photonics Interfaces[1] at the EPFL in Switzerland ran by Michael Gratzel[2].

I want to stress out that the lab is great, the people in there are extremely hardworking, Gratzel is a great scientist, but at the end of the day research is what it is and stuff like this can slip under both your lab managers and reviewers. I have never ever seen the slightiest indication that lab staff ever encouraged nor tolerated such stuff, but it's easy for it to happen and there's not enough incentives (nor possibility) to review every single experiment.

But the reproducibility problem does exist and the number of scientists tweaking numbers by tiny percentages here and there to make sure they publish is relevant.

[1] https://www.epfl.ch/labs/lpi/

[2] https://en.wikipedia.org/wiki/Michael_Gr%C3%A4tzel

[foxglacier]

I can understand mistakes happening and going unnoticed. But from what that lady told you, it sounds like they have a culture of sweeping mistakes under the rug. Was the affected paper retracted or at least a correction published? If not, they're still benefiting from the fake success which makes it unethical even if it's not technically academic fraud.

[TheBigSalad]

And this encourages the people with integrity to quit.

[epolanski]

I wouldn't say they all quit, but they ultimately have to settle in less prestigious and less funded labs/universities.

I've met countless great scientists in Italy which were ultimately wasted as professors and achieved little as scientists.

I'm not saying that teaching isn't important, but it's a skill completely unrelated to being a good scientist, there's no overlap at all.

[mountainb]

There are so many other ways to make money that don't involve crime. And there are even many crimes that make more money that are far less harmful to society.

[thaumasiotes]

> I don't know how to fix it other than several governments and their education ministries making a joined effort to have scientific papers where each result has to be thoroughly reviewed by multiple other labs. It's expensive, but I don't see other ways.

This is just aggravating the problem. Science is mostly fraud because it's mostly done on behalf of a funder who doesn't want it. It's easier to write a paper describing the results you'd like to get than to actually get those results. And the funding agency is indifferent between those two things. So mostly you just get the papers.

[Ashkee]

Feels like the incentives are backward and it pushes people to publish fast instead of verify. You could try HifiveStar to track and surface trustworthy feedback signals across sources, it helped me sift noise from solid input. The result is fewer wild claims and more time spent on work that holds up.

[epolanski]

How would that be aggravating the problem?

The current system has essentially no requirement of reproducibility.

Having a paper that only allows reproducible experiments (where there's funding for random labs to reproduce results) may be difficult, or an utopia, or whatever, but not aggravating for sure.

[thaumasiotes]

It's aggravating the problem because you're proposing to put that indifferent funder in charge of ensuring quality. There are two problems here:

(1) This is not a recipe for actually getting any quality.

(2) By virtue of providing the funding, they already are in charge. They're not going to get better results by wishing harder. But they can easily waste more money than they already do.

Your mental model seems to be that the government received a mandate to cause research to happen, and they did that as faithfully as they could, with the only problem being that we forgot to specify that we didn't want fake research. So if we change the mandate to "cause non-fake research", the kind of research we get will change.

But that makes no sense. "Non-fake" was always a requirement. It was an unenforced requirement because it didn't matter to anyone, but you aren't proposing to change that.

[fuzzfactor]

>requirement of reproducibility.

Check my other comments.

Reproducibility can be a working requirement before publication when the progress is expected to be serious.

At ASTM the publishing company is non-profit and more non-academic industries pay for the (not cheap) publications every year. The employees are well-paid journalists and efficient bureaucrats specializing in continuous quality improvement themselves, highly skilled at organizing the scientists. The scientists are all volunteers.

>indifferent funder in charge of ensuring quality.

Nice not to have. Publication requires complete consensus of the volunteer scientists, and the institution is crafted to progress toward valid consensus.

It's all about quality from day zero.

In more ways than one, more than you can count actually.

So it didn't take 125 years to get that way, it had a better start than most, and has only gotten more strict over the recent decades as computer statistics became mainstream.

Edit: Forgot to mention, there's no eminence. Nobody's name appears at the top of the document, and almost nobody (still living) ever appears within the text.

Further edit: I guess you could say that ASTM is a product of the Industrial Revolution, and there hasn't been an equivalent Academic Revolution yet.

[fuzzfactor]

Seeking eminence costs more than making breakthroughs.

How else are you going to pay for lack of breakthroughs?

Some people aren't going to be capable of breakthroughs anyway, lots of them even know it from the beginning, so they naturally or intentionally seek different things using the same institutions and resources that could otherwise yield breakthroughs instead.

Edit: As a footnote suggested by my own question, you pay for lack of breakthroughs by building eminence out of thin air, if it wasn't obvious.

[julienb_sea]

This is likely a generalized problem with basic science. In applied science you need to be very careful about fraud because ultimately the application of research findings will end up in customers hands who can and will pursue legal action if the original claims turn out to be false.

[chrisBob]

One thing that helps to counter this somewhat is that if your paper is proven to be wrong, the journal can force a retraction. A retraction isn't exactly career ending, but it is a huge deal and will have an impact on future jobs and funding.

[nothercastle]

For the expendables yes for the lab not career ending at all

[geokon]

> one of the most prestigious laboratories

my gut feeling is that the more famous a group/lab, the more likely there is some funny stuff going on. Smaller groups/labs are less cutthroat. But it also depends on the discipline...

[msteffen]

> randomly talk about my issues with a random phd in the lab (great scientist with tens of thousands of citations) which quickly scans the data and notes that the voltage obtained by the system in the publication is literally impossible

I'm interested in the apparent contradiction between the "tens of thousands of citations" credential for your evidently competent labmate who caught the fraud, and the "only high impact numbers get citations, only citations get you a chance to progress in the academia pyramid" indictment of the evidently mediocre fraudster.

How much of the science that happens do you think is due to people like your labmate, who seems to have earned her citations organically with talent, and how much is due to the fraud? Are most citations still going to talented people?

My parents were both academics who built their careers in the 70s and 80s. I don't know that they had a perspective on fraud per se, but they did say that science used to be totally different—lots of money going into a smaller research community. But because each PI trains many PhDs, the research community has grown unsustainably and now funding is highly competitive. For the system to be sustainable, the vast majority of PhDs need to leave science, and there need to better exits for those people (or else there need to be many fewer PhD candidates and researchers need to stop relying on PhD candidates for labor). I wonder if the fraud is a consequence of this problem.

[epolanski]

> I'm interested in the apparent contradiction between the "tens of thousands of citations" credential for your evidently competent labmate who caught the fraud, and the "only high impact numbers get citations, only citations get you a chance to progress in the academia pyramid" indictment of the evidently mediocre fraudster.

Not every paper out there is fake and Yella Aswani [1], was an excellent PhD in Switzerland before becoming a full time professor in India.

[1] https://scholar.google.pl/citations?user=PHS1UAcAAAAJ&hl=en&...

That being said, some of her colleagues might have felt desperate to publish something meaningful before ending their PhD and cooked the numbers by that 8/10% that makes it impressive. Either that or they took an outlier result that overperformed for some reason (poor instrument calibration e.g.) and never investigated and just published.

In any case, the numbers didn't match up.