[asdf123qweasd]

At some point, the good scientists leave and the fraudsters start to filter for more fraudsters. If that goes on, its over- the academia has gone. Entirely. It can not grow back. Its just a building with conman in labcoats.

My suggestion stands: Give true scientists the ability to hunt fraudsters for budgets. If you hunt and nail down a fraudster, you get his funding for your research.

[bluecheese452]

All the fraudsters will nail the honest ones before they know what hit them.

[Balgair]

I mean, the replication crisis had come and gone, about 5 years now. The fraudsters are running the place and have been for at least the last half decade, full stop.

[searine]

That is a ridiculous exaggeration. Yes, like in every walk of life, fraud happens. However, the extreme success of academic science shows that most of it is real, honest, work.

[godelski]

It is field dependent but I'm not entirely against what the parent said. I work in ML and I am positive that all this is going on[0]. There's lots of true believers though and that's what makes things extra hard. Sometimes the fraudsters take over by making the system become incompetent and everyone is in good company. In this was fraud isn't committed with intent, weirdly enough.

Just look at all the ML reasoning papers. Wither you believe LLMs reason or not, an important factor you have to disentangle when trying to prove this is what data the models were trained on. To distinguish memorization from reasoning. You won't find this analysis because it's almost impossible given that the data is a trade secret, even by Meta.

This year at ACL a paper (mission impossible language models) won best poste paper despite their results running contrary to their claim, and very obviously so too.

Or there is the HumanEval paper which proposed that they created a data set which was not spoiled because they "hand wrote" over a hundred "Leetcode style problems". 60 authors and they didn't bother to check... But why would you check when the questions are things like "calculate the mean". What fucking programmer thinks there isn't python code on GitHub pre 2021 that: calculates the mean, takes the floor, checks if a string is a palindrome, calculates greatest common devisor, or any similar question. How did this become an influential dataset‽

[0] the big reason I'm upset is because I love the field. I'm not in it for money. I'm in it because I grew up on Asimov books and because I want our community to work towards AGI. But now every person that can do print("hello world") feels that they can lecture me, a published researcher about what these machines do while they talk about the Turing test (lol, what is this, the 60's?) and how they're black boxes (opaque, but certainly not black). I'm fine with armchair experts, but not when they come in swinging with a baseball bat

[lupire]

How so? The fraction of academic science that is applied to anything, anywhere, with clearly identifiable impact is both a tiny fraction of academic science, and also often detrimental quality of impact.

[searine]

Pretty much every industry functions on a foundation of basic scientific knowledge discovered in academic labs, run by honest people trying to understand the natural world.

Fraud happens. Bad theories happen. The slow turn of scientific wheels takes centuries to crush them but it will always win. Profit doesn't turn those wheels. Our entire modern lifestyle is the impact.

[mistercheph]

Is it possible that society is the thing with turning wheels and knowledge is the thing getting crushed underneath it?

[searine]

That how it was, and it can easily go back to that.

[asdf123qweasd]

Thats not true entirely. There is research, with huge impacts and the money leveraging- keeps it brutally honest. Nothing makes it out of a lab and into a fab, without certainty of the method working at least in lab conditions reproduceable. There are many billions, but not that many billions.

[fluidcruft]

> nothing makes it out of a lab and into a fab, without certainty of the method working at least in lab conditions reproducible.

The article describes multiple full clinical drug trials both completed (inconclusive==can't prove harm, effect is so small that benefit cannot be excluded) and ongoing that are fundamentally built on the fabricated results which literally do represent many billions of private investment.

Ultimately research falsification is a con game, and you seem to have faith in something magical about "money leveraging" that smokes out cons. I do think shit eventually hits the fan in the market since reality affects the market, but it's not because the market is more rigorous than "science". Ultimately, investors are not experts and the are listening to the same people who do "peer review" and didn't notice (or ignored) the fraud in the first place.

[asdf123qweasd]

Fair point, thank you for your time and well crafted argument.

[azan_]

It's really easy to verify that this claim is pure nonsense - we still have very real, astonishing scientific progress.

[asdf123qweasd]

But is it proportional to the investment of time and people participating in it. If you take the amount of money/people invested into the sciences at a point in time , lets take: https://en.wikipedia.org/wiki/Solvay_Conference

And then, put the number of contributors and the rate of progress against one another, my guess is that you would see a massive slowdown of progress, so massive actually, that explenations about the slowdown abound. There is the "all easy apples have been picked" theory, the "only life&death systemic competition forces contributors to produce good science" theory and a ton of others. All basically trying to explain the same phenomena- which could also be explained by: "hackers, hacking hackers, hacking processes, leave no financial substance behind to have people who actually do the scientific leg-work."