[PlasmonOwl]

Bullshit. Journals select for novelty, this is true. But this is novelty in a trifling fashion, like teaching a dog to roll over. The entire system, at least in the UK is set up to actively discourage risky research entirely. Doing something truly new is discouraged unless you are truly a senior or leader. By then, you've probably lost most of your imagination anyway.

[Frost1x]

My anecdata from the US agrees. The best part is when funding agencies require proposal reviewers to actually send their critiques back on research proposals. I've read through many reviewers replies to proposals designed for specifically set risky/novel research funding and the responses are sometimes so conservatively comical without a truly valid argument you can't help but laugh.

I've also worked at research organizations specifically touted as being highly independent with freedom to explore novel paths and the story is basically the same. Ultimately, someone somewhere holds the purse strings and conservatism kicks in. You at minimum need to spin your research to fit popular trend keyword language or make it apply to these areas.

Financial structures ultimately dictate the lack of support for novelty. Basically, novel research has freedom only when it's independently funded, which limits the scale of most novel research. No one wants to take risk, they want to market taking high risk while taking low risk and selling mediocrity. I think the fundamental underlying issue we have across multiple societies is that we no longer proportionately reward risk anymore and that's why people seek low risk everywhere. Taking high risk often has limited opportunity for reward, even if you are successful, so why bother? You'll net more success taking low risks and only fools take high risk.

[api]

I've thought for a long time that this is why you see so much innovation during wartime. War overrides conservatism and gets risky things funded.

The same can apply to war-like scenarios. A lot of novel biological research got funded during COVID. We should expect to see a burst in advancement in biomedical technology as it bears fruit over the next 5-10 years.

The core problem is probably that conservatism makes the most sense at the individual level for most people. Even if you had a casino whose net payout was positive, it still may not make sense individually to gamble. If individual rewards are sporadic and concentrated on a power law distribution then statistically most players will lose.

An obvious solution is a less power-law type distribution, which tends toward some kind of socialism, but that has its own problems. Nobody's figured out yet how to provide a productive channel in a socialist system for humans' totally normal drives to compete and maximize individual outcome. Trying to suppress it doesn't work; it's like trying to totally suppress sexuality. Extreme socialist societies (e.g. Soviet Communism) often evolve into totalitarian mafia states because if there is no positive outlet for those urges they drive people into crime. Eventually you get a meritocracy concentrated in crime and the mob takes over everything.

[anonymouskimmer]

"Nobody's figured out yet how to provide a productive channel in a socialist system for humans' totally normal drives to compete and maximize individual outcome."

Nobody has figured out how to maximize individual outcome, or provide a channel for such pursuit. Period. Socialism has nothing to do with it. People are crap at managing more than a few dozen other people, even for the best managers who have ever lived. And the systems we have in place since agriculture took over are trying to manage many, many more people.

Most people in more developed countries live in partially socialist systems. This doesn't prevent them from getting private sector jobs, creating companies, and otherwise trying to pursue the maximization of their individual outcomes.

[ngcc_hk]

Or fake crime. Solving real crime is too hard. Why not fake it.

[elcritch]

I have a hunch that this conservatism problem is why ADHD genes / traits remain so prevalent. The percentage of populations with it is around 5% and remarkably consistent across drastically different cultures.

The reason is that those with ADHD are often physiologically incapable of not pursuing an interest. Often those interest lie in novel areas. It’s unfortunate for those with the traits, but a net benefit for society.

So there’s a game theory equilibrium wherein successful societies will retain a certain percentage of those genes or become too conservative.

[Brigand]

Traits that are not beneficial to the individual will die out, the kind of group selection this process would require is implausible at best.

I would assume that the kind of personality your are referring to is adaptive for the individual as well, especially the mild variants.

[elcritch]

> Traits that are not beneficial to the individual will die out

Not necessarily, examples from several species where males get eaten after mating would seem to counter that. In some the males try to escape if possible to mate again.

> the kind of group selection this process would require is implausible at best.

There can certainly be group theory optimums that are less beneficial for individuals but beneficial enough to the whole group that traits bad for an individual will persist. I’d say bees dying after stinging an intruder would be an example. Well or even the hive principle where only the queen reproduces.

Still I would agree that the mild variations of adhd traits can be beneficial to some individuals. At least enough to largely eliminate the traits despite being largely maladaptive in non-nomadic societies given that we’re not hive creatures. One example I’ve read was a paper that showed increased offspring in individuals with adhd traits in nomadic tribes in Africa, with decreased reproductive rates in adjacent non-nomadic tribes.

Our modern society is odd in that it’s not like either traditional society. ADHD traits can be very beneficial for individuals in technology careers requiring creativity while simultaneously being painful due to society being largely optimized for non-adhd neurotypes.

[ThereIsNoWorry]

That was my conclusion after publishing a paper during my BSc. Academia is full of petty politics. Professors that gatekeep whole fields on a "like"-basis, reviewers arguing with "not enough experiments done" and questioning uniqueness and novelty in ways that just don't make any sense (to me). It is a very bureaucratic process that cannot possibly be good for innovation and progress. Then you get incentives of fields that don't make any progress to keep the hype train going so their jobs don't become meaningless over night. It just keeps going. So, I obviously declined to do a PhD and went into industry. At the bare minimum, playing politics in industry leads to vastly better (pay) outcomes than in academia.

[godelski]

Dog whistles in academic reviewing; phrases that can be valid but are typically being used in an invalid way:

- Not novel: Novelty is ill-defined. Most journals and conferences have guidelines on this that come down to "will someone find it useful" which should make "not novel" a rare review, not the most common. Additionally, what's obvious post hoc is not obvious a priori (I actually saw an AC override reviewers because of this!). If something is well written it also comes across as easy and simple. Lack of novelty could be a true lack of novelty or just an indication of a well written paper.

- Incremental: All research is incremental. Research can be too incremental (not enough work done) but this is never expanded upon in those types of reviews. This kind of statement is meaningless in isolation. Incrementalism will always exist with a publish or perish paradigm. One cannot publish breakthroughs year over year. Those take significant periods of time or a lucky break. Neither is a common occurrence, by definition.

- Not enough experiments: "Money is all you need" has become a popular phrase. There may not be enough experiments and this can be true but perfection is the enemy of good. Especially in research, where the experiment space is non-exhaustive. Can a reviewer be satisfied in a finite amount of time and with finite resources?

I've started to change my mind about where blame falls for all this. It starts with the reviewers, but if this becomes the norm then the system is at fault. It has not removed the rotten apples and thus let the barrel spoil. All while we've had decades of discussion about this happening. I think a lot falls on the Area Chairs and Metareviewers, as they should be preventing these types of reviews to pass. But the whole incentive structure of publishing is wrong. There is high pressure to reject and zero pressure to let papers through. This especially hurts our junior researchers (grad students) as it becomes a lottery process for determining if they can graduate. After all, we can't have any wizards without any noobs.

[ThereIsNoWorry]

It is absolutely a systemic problem. A problem that will not be changed or solved within our lifetimes, that's for sure.

[godelski]

> A problem that will not be changed or solved within our lifetimes

Honestly, I don't like opinions like this. That's just passing off the work that needs to be done to fix these problems. It wasn't even a hundred years ago when people published more in the open and the publish or perish paradigm didn't exist. The latter is a relatively tool, and collapsed because Goodhart's Law.

We can make the changes, but not with defeatist attitudes.

[ThereIsNoWorry]

Fair point. I was just thinking that one sensible change could be to publish everything without significant constraints on open platforms. Then, in some way, the community and some moderate moderation would basically decide which papers and researchers get "famous". In the best case, that would result in more true "breakthroughs", but reflecting about how social media works (incentives to constantly push narratives, hype and simple "truths" that explain complex world issues) at the moment, that seems a bit too optimistic. That's why I peddled back to "not solvable within our lifetimes". Who knows. I'd like to do research, but if I go into academia I'd be immensely constrained and would have no freedom what to work on. The only viable option is to become sufficiently wealthy to fund myself and my ideas. Would be nice, even if a tiny bit unrealistic.

[godelski]

There's a double edged sword here. Currently CVPR has a social media ban that is extremely strict. I've definitely been seeing Twitter act differently than it normally does around this so we'll see if it works. But why this matters here is that there's an incredibly strong correlation between the popularity of the lab, the amount of eyes that can read the preprint (with authors names), and acceptance at top conferences (i.e. CVPR). I often say that double blind only exists for small fries. But even still, big labs will get more likes from twitter accounts that post arxiv papers. There are plenty of means to bypass this social media ban.

But even from this experiment we can see that there are people actively trying the fix the situation. Likely naive, but something is better than nothing. I wish there would be more push for ACs and Meta Reviewers having the highest standards, but this is where we are.

I am totally for us abandoning journals and conferences and just publishing to arxiv and open review. This is exactly what I would do if I was independently wealthy and could research without the constraints that I am in. But we also need to recognize the issues with this and that popularity and outreach ability greatly affect perceptions of quality of the paper. That this is not meritocratic.

[rscho]

As a clinical researcher, I wholeheartedly agree. Even worse, meta analysis is comparatively so well cited (even shitty ones) that doing anything else, such as original research, will be suboptimal since the reduced time investment allows you to publish nonstop. Truly, meta analysis has become a blood-ducking parasite of medical science.

I know many young researchers who've specialized in meta analysis for that reason.

[nerdponx]

Isn't there tremendous real value in meta-analysis? The problem seems to lie in how credit is allocated (via citations).

[n4r9]

I'd argue the problem is a combination of citations and short-termism. We don't give researchers the space to do something novel, because institutions demand results in the short term. If you give someone a tangible, quantifiable demand A and a nebulous, low-priority demand B, they will spend nearly 100% of their time chasing A. Truly novel results can take months or years of hard focus; every grant proposal, teaching duty, and routine publication is a distraction.

[einpoklum]

> Isn't there tremendous real value in meta-analysis?

There's also tremendous value in doing the dishes and mopping the floor. Without it we'd get diseases and live in filth. And when we do this, we should be respected and appreciated. But - not as brilliant trail-blazers.

[a_bonobo]

>By then, you've probably lost most of your imagination anyway.

I've recently left academia after ~10 years to work on more blue-sky-stuff in industry, and this rings so true. It took me a few months to unlearn academia's focus on the minimal project we can do to get the next paper out (still unlearning!).

It's hard to get in, it's harder to get out.

[anonymouskimmer]

I worked as a permatemp for almost 7 years in industry and didn't have the ability to order my own reagents. It took 3 or 4 years of working in a DOE lab to break that habit and start ordering my own reagents to innovate with.

Ingrained work habits are hard to break.

[asdff]

Certain fields are a lot easier to do novel work in. For a grad student in computer science/computational biology, you practically don't need any grant funding at all. You can download public data, work on the university hpc for basically free, publish whatever the hell you want, and pay your stipend with a TAship. Others, where you have to regularly buy reagents or maintain an animal model, you pretty much need grant support to buy these things.

[teekert]

I heard established profs talk about how they ask for funding for something already done or almost successfully done. Then they do something new with the money.

[isaacfrond]

[citation needed]

[bildung]

My background is in humanities but the dynamics are the same: Chasing the goal of objectivity, easily measurable datapoints like number of publications and number of citations are used for quantifying impact of a researcher. This incentivizes researching/publishing low hanging fruit, because actually working on deep or unusual approaches means sinking more time into the endevour while risking less or no publishable results. For people trying to earn a living in academic institutions this would be stupid.

[CrazyStat]

I can confirm the GP matches my experience in academia (PhD, published a paper based on my dissertation, left academia shortly after).

There is immense pressure for early career researchers to work on low-hanging fruit that will give easy guaranteed publications: make a minor change to previous work and write a paper about it; write a paper about what other people wrote papers about (either a review or a meta-analysis); etc. etc. This is how you rack up publications for your tenure review. It's also how you rack up publications to get funding.

Once you have tenure you have a bit more freedom, but you still need to worry about funding (generally). And by that time you've been in the habit of grabbing the lowest-hanging fruit for 1-1.5 decades, assuming you were ~well trained~ in graduate school. After so long reinforcing that habit it becomes hard and scary to build a ladder and go after the better, higher fruit.

[rscho]

Habit is not the problem. Once you get tenure, a good part of your job is to ensure that the younger generation in your lab/uni publishes well, i.e encourage them to pursue the next low-hanging fruit... So no, you do not in fact have more freedom.

[CrazyStat]

That's a fair point. I jumped ship before I got that far, obviously. From my perspective as a student it looked a lot like habit on the part of the senior/tenured people in the department, but I should not have been so quick to assert that.

[ackbar03]

I suggest the author cite the following works which I believe are related to this topic:

list of own publications