Hacker News new | ask | show | jobs
by JumpCrisscross 2 days ago
> We are very rapidly automating humans out of the academic publication loop

We are rendering it irrelevant. If this is the norm for academia, I’m sympathetic to the folks looking to cut its funding.

3 comments

This is the truth. As a researcher, I say good riddance. It was always kinda stupid, AI is just accelerating the demise of something that hasn’t worked correctly for a few decades now. The time is way overdue for us to figure out some other system.
I feel this is a case of "perfect the enemy of good". Looking at the output (scientific progress in biology, medicine, material physics, etc.) the "something" seems to have worked well. Could it be optimized? Probably. Should we completely destroy it and hope a new system will be better? My read of history is that in many cases the new ideas were worse of what they were replacing and it took a long time for a fix.

So, if you have ideas of a new, better system let's talk about those, before getting happy something gets destroyed and hope someone else will come with a better solution.

Look into the replication crisis. It has seriously impacted some fields very negatively. Psychology and Alzheimer’s research in particular have been set back decades.
Look into cancer survival rates : https://www.cancer.org/research/acs-research-news/people-are... , to quote "Decades of cancer research have provided health care professionals with the tools to treat cancer more effectively, so that cancer in general is becoming less of a death sentence and more of a treatable chronic disease,"

So with the current system, some things work, some things don't work (ex: psychology/Alzheimer). Yes, the system should be improved. No, I am not convinced that "destroying" the current system will result very easily into something better.

We need to discuss actual solutions for the replication crisis. There are even steps towards improving that, like requiring open data for papers, which makes it harder for people to do some of the manipulations that resulted in the replication crisis. I personally would go even further: you should provide complete documentation (tools, notes, data, raw files, etc.), but then there are some people opposing that due to "privacy" (for medical) or "patents" (for industrial stuff).

What you are implicitly assuming is that the publication and review system matters for that cancer research. Cancer progress, as I understand, is mostly driven by NIH priorities, which are set by governmental review committees. Published work plays a part in that, but it is not the decentralized-review that is typical of other areas of research (e.g. psychology and Alzheimer's).

The peer review system as we know it today has only really existed for less than a century. It is no how science was traditionally done. It was adopted due to some real problems with the old system, so I'm not saying we should go back. But it's not clear either that unpaid peer-review is the ultimate end-state either.

100% agreement on your last paragraph though.

This has always been true for science. We used to think the atom was akin to plum pudding. Models and ideas change though, in light of new evidence. When you stop generating new evidence and postulating new theories given that evidence, that is the real loss for science. Not the idea that we might not have a perfect model of the universe at this moment, but the idea that nothing will improve going forward.
This is the norm for the industry. When there is a lot of money to be made, people will chase it by any means they can think of.

AI is a special case, as academia isn't usually that lucrative. In the rest of CS, many major conferences still have ~300 people, and interesting stuff often happens in specialized meetings with fewer than 100 participants.

This is the norm for the economy. When there is money to be made, at least one person will chase it by any means they can think of.

Academia is a specific case. It happens everywhere.

And make even less knowledge and progress in science public and ever more in the hands of capital and private interests?
> make even less knowledge and progress in science public and ever more

If a discipline is spending public dollars at OpenAI and Anthropic, we're funding them with extra steps. (And losing nothing somebody else couldn't do.)

You mean token holders