[light_hue_1]

I think you're romanticizing a past era and idea of science.

Approaches aren't old fashioned for arbitrary reasons! They're old fashioned because they don't work. Science is pretty cutthroat. No one wants to leave anything on the table. If those methods worked to discover big ideas people would be pushing them.

There are plenty of people out there who are what I'd call crash and burn scientists. The kinds of people who bet their entire careers on doing one big splashy thing. If it works out, great! If it doesn't, they're out of a job. If the old timey methods worked for finding such ideas people would be pumping them hard. But don't.

And there's a reason why we hardly made any progress until about 100 years ago when we stopped doing science this way.

Splitting subgroups isn't inherently bad! Heck, something like stratifying autism is one of those big problems people burn their entire careers on. But after trying to do this based on case studies for decades, we've learned it basically doesn't work as a method. Splits come out from mechanistic hypotheses and/or in the data at scale when you have controls.

I do agree that the old-timey way of doing this is much more pleasant. But it's just not effective or what we would call science today.

Even if you look at the case of Omegaven. It was the animal studies that saved them. Boring science with controls, etc. The main problem they had is that funding agencies are too conservative. Which is true! The vast majority of scientists would agree with that statement. We need to take more risks and funding agencies need to be ok with more failures. Sadly, funding agencies don't agree.

I'm not writing this to be mean! I'm writing it because I see the impulse to follow this approach to science all the time in new grad students.

I think you could be far more effective if you took a different much more scientific approach. Citizen science has a real chance of completely revolutionizing nutritional science because our studies are so incredibly bad. But the solution isn't to throw the science part out, it's to fix the part that we know is bad: small studies with bad controls and really poor reporting. We're a good app + a large community that's willing to play ball away from changing the world.

Happy to chat more :)

[Pannoniae]

While it's true that conservatively performed science does lead to more consistent results (so not every correlation would be claimed as a brand new discovery), nowadays we have two basic problems.

1. Science has become so abstract that it's very easy to cheat with statistics and methodology. This is both due to the enormous complexity which makes it hard to verify, but also because there are few low-hanging fruits, and people are pressured to show a result. I even had that pressure in high school in some classes to show results from nothing!

2. There are many areas which are so advanced that the overwhelming majority of us, and even experts barely comprehend any of it, but simultaneously there's many things which we barely know anything about. Sadly or not sadly, many of these areas are simply neglected because universities and companies aren't interested in them. That doesn't mean that these areas are useless or impossible to research, they are just either niche or non-existent. Now, people in those areas obviously won't use the "new" scientific methods since they don't even know themselves what will happen or how it works, so it's not possible to theoretise it in a similar way, unless if they randomly make them up.

Also, many (I would even say most) publications can safely be cut down by 50% without losing anything of value. I am saying this because you mentioned science is cutthroat - but it isn't cutthroat in coming up with new experiments, tools and making observations, it's mostly cutthroat in academia politics, falsifying/p-hacking data and writing eloquent bullshit in the form of shibboleths designed to be read by other peers in academia.

[light_hue_1]

> 1. Science has become so abstract that it's very easy to cheat with statistics and methodology. This is both due to the enormous complexity which makes it hard to verify, but also because there are few low-hanging fruits, and people are pressured to show a result. I even had that pressure in high school in some classes to show results from nothing!

I agree with the diagnosis, I don't agree with the cause. It's not because science has become abstract. It's because we haven't adopted modern computing yet.

Back in the day it was impossible to share data. You collected data and you reported summary results. And we had to trust that you did it all right, and since the space of methods wasn't that big, we were mostly ok with one another's methods. That's not the case anymore. Much of science is using woefully outdated statistics.

The reality today is that pretty much every single study in the universe can and should share all of their data and methods. Raw, unfiltered, data. And we should have a standardized format for describing it, its properties, and the properties of the experiment in a machine understandable way. And every scientist will have their own toolkit for analysis that runs automatically when they open the paper. So we'll each get to see our own analysis, as well as whatever opinions the authors might have. We'll get there in the next 100 years.

> 2. There are many areas which are so advanced that the overwhelming majority of us, and even experts barely comprehend any of it,

This I would disagree with. For every field you'll find people who are so dedicated they really get it. In so far as humanity gets that problem at all.

> but simultaneously there's many things which we barely know anything about. Sadly or not sadly, many of these areas are simply neglected because universities and companies aren't interested in them. That doesn't mean that these areas are useless or impossible to research, they are just either niche or non-existent. Now, people in those areas obviously won't use the "new" scientific methods since they don't even know themselves what will happen or how it works, so it's not possible to theoretise it in a similar way, unless if they randomly make them up.

Universities by their nature attract people who are interested in a lot of crazy things. The reason why those areas aren't explored isn't because people don't want to. Or don't know how. Or anything lke that. It's because science budgets are obscenely low. We spend worldwide $5B on cancer research. That's like 1/20th of the ad revenue of Facebook. If we want to go level up as a civilization we need to spend more on more crazy science.

> Also, many (I would even say most) publications can safely be cut down by 50% without losing anything of value. I am saying this because you mentioned science is cutthroat - but it isn't cutthroat in coming up with new experiments, tools and making observations, it's mostly cutthroat in academia politics, falsifying/p-hacking data and writing eloquent bullshit in the form of shibboleths designed to be read by other peers in academia.

I don't agree at all. Yeah, some people p-hack or distort facts or whatever. Some because they just want to get ahead and don't care. Most simply because they don't know better. The reason why say, a lot of the scientific output of some countries is basically junk, isn't because those people are evil or stupid. It's because they don't know better.

Your opinion is that 50% of the content of papers can be deleted. But how many papers have you tried to replicate in your life? It's easy to skim through and say you get it and move on. But, when you're trying to reproduce a paper all of those details you want to take out matter.

Most scientists just want to figure out how things work. They're smart people who make a conscious decision to give up millions of dollars of income over their lifetimes in exchange for working far harder to find new ideas.

> writing eloquent bullshit in the form of shibboleths designed to be read by other peers in academia.

The fact that you can't understand a paper simply means you need to level up your skills in that area. Often we need to be extremely precise with our language, and that results in papers that are hard to read. But if we weren't that precise, we would never make any progress because no one would know what anyone else is saying.

At the end of the day science works. Your phone gets better all the time. Cancer survival rates are getting better. ChatGPT gets better. etc.

It's just become a fun activity for a lot of people to dump on a group of people who are doing their best to make everyone's life better at their own expense and that of their families. I can hardly think of a better description of the direction our society has been moving in for the past decade.

[jbotz]

I think your attitude is very condescending and really misguided and counter-productive. Yes, of course the modern scientific method is superior to what came before, but it was what came before that got us to the modern age and kick-started the industrial revolution that makes modern science possible. And modern science has its own set of problems, one of which is that it's incredibly resource intensive (or, as you yourself said, "hard work")... in fact it's impossibly resource intensive, as shown by the fact that even though we have millions of scientists working today (90% of all scientists who have ever lived) we still have gigantic gaps in our knowledge about basic live-or-death issues that affect millions of people. Especially in the area of human health we still take as dogma things that later turn out to be almost certainly completely wrong and in the meantime have cost millions of lives. And political and economic factors still distort the scientific process at least as much as what you call "crash and burn science".

The hard work of modern science has also produced fantastic results, but I don't think it can ever fill all the gaps in our understanding of the world by itself. We need creative approaches, wild speculation and experimentation, phenomenology, and philosophical and spiritual exploration just as much as we need those millions of scientists doing the painstaking work of endless iterations of controlled experiments.

[light_hue_1]

Or maybe, my attitude is that of someone with decades of experience actually moving the needle on problems. And with decades of experience training grad students out of exactly this kind of naive view of science so that they can go on to be extremely productive.

> And modern science has its own set of problems, one of which is that it's incredibly resource intensive

No way. Science is obscenely resource efficient and low resource. People don't realize how tight science budgets are. Like, much of the CS research in the US is supported by a single organization whose budget is $1B. Cancer research is only funded to the tune of $5B per year worldwide.

The US alone spent $15B on a worthless border wall under Trump. To put useless pieces of steel in a desert. That someone can walk around or cut in a few minutes. We spent 3x the yearly cancer research budget. Yet despite all of that, cancer survival is going up rapidly.

> Especially in the area of human health we still take as dogma things that later turn out to be almost certainly completely wrong and in the meantime have cost millions of lives. And political and economic factors still distort the scientific process at least as much as what you call "crash and burn science".

I swear, listening to people here you would think all scientists go through a brainwashing course and then attend a yearly conspiracy meeting to compare notes and never discover anything. Seriously, you have no idea how hard people work to prove everything they possibly can wrong.

I do agree that there are massive gaps and millions die because of it. I would actually characterize it as billions of dead people.

The solution is simple. Fund science and be ok with failure. The gatekeepers are not individual scientists. The gatekeepers are funding agencies. That are massively starved for funds so they need to be extremely selective. And that means being extremely conservative. They cannot bet on high-risk high-reward science, because if all their bets don't pay off, then what do they tell Congress?

> The hard work of modern science has also produced fantastic results, but I don't think it can ever fill all the gaps in our understanding of the world by itself. We need creative approaches, wild speculation and experimentation, phenomenology, and philosophical and spiritual exploration just as much as we need those millions of scientists doing the painstaking work of endless iterations of controlled experiments.

Yeah, so what can I tell you? I takes me 5-6 years to teach a new PhD student that this is totally wrong. You can't make progress by random wild stabs in the dark, philosophy, spiritual whatever, wild experiments. It doesn't work. You can't build up a coherent research program that way. You can't go from result to result in a systematic way breaking down a problem. At the end of all of that "exploration" your grandma still dies of cancer because you spent your time on wild experiments instead of real work. Seriously, those are the stakes here. Work is personal for a lot of people.

You can build intuition that way, and that's cool! I run tiny experiments all the time looking for fun things. But then you need to roll up your sleeves and do the hard part. Have a hypothesis, break it down systematically, think about mechanisms and outcomes, run controlled experiments, etc. That has to be 99% of the work, because that's how you convince people, build up a body of evidence, find new ideas, etc.

[randysalami]

> You can't make progress by random wild stabs in the dark, philosophy, spiritual whatever, wild experiments. It doesn't work.

It must be hard for a young grad student to swallow the pill of human mortality and reconcile the disconnect that a lifetime of hard and rigorous science will merely move the needle. Fruits of their labor which might only be tasted centuries after their death. Of course, only through this sacrifice has the needle been moved so much that we have the things we have now. Something something shoulders of giants. Understanding this is a key part of accepting reality and becoming a mature adult, something it seems you've helped many people do, and something many people never do.

> The solution is simple. Fund science and be ok with failure

We also live in a society dictated by political and economic forces. These forces have no qualms rejecting reality and believing in eternal life. Will science ,which has an infinite number of directions to grow and is limited in speed by the rigor of its very nature, ever be fast enough to protect itself? Or is this science just for the sake of science until societal collapse occurs? To really save and protect the billions of lives, I think a new kind of societal advancement is needed outside of science. A vanguard force of sorts. Not political or economic in nature but also not scientific because it can not be bound by the speed of the field.

I think software as envisioned by some people long ago could have been this place. Of course, software has been co-opted by all three groups but I still think there is a chance. Restructure society, reign in political and economic forces, and allow science to do its thing for thousands of years to come.

[light_hue_1]

> It must be hard for a young grad student to swallow the pill of human mortality and reconcile the disconnect that a lifetime of hard and rigorous science will merely move the needle. Fruits of their labor which might only be tasted centuries after their death. Of course, only through this sacrifice has the needle been moved so much that we have the things we have now. Something something shoulders of giants. Understanding this is a key part of accepting reality and becoming a mature adult, something it seems you've helped many people do, and something many people never do.

Totally. It's so much harder in some fields too.

A friend who is a primate researcher describes it as having 20 experiments that he can run in his entire career. I still think about that sometimes.