[huijzer]

> 1) it takes a undergrad to make and measure a superconductor by a Nobel laureate to explain it.

Another example of Taleb’s idea that experimentation comes first and theories second. Universities pretend that it’s the other way around but apart from an Einstein it’s usually not.

[anonymous_sorry]

It's a feedback loop. Experimental results suggest new theories, which suggest new experiments to test those theories.

[visarga]

That's what I am thinking when commenters compare LLMs with humans. Humans got the experimental loop going through the real world, pure LLMs are brains in vats. They need experimental confirmation to become truly creative.

But on the other hand are the commenters who think AGI to ASI is going to take hours. When is there time to experiment anything? Even ASI needs real world confirmation to do research.

[mgfist]

The claim is that an AGI can instruct humans to do the experimentation (knowingly or not).

[WitCanStain]

AGI to ASI might take hours if all that's needed is more compute. Such an intelligence might exponentially spread across the internet. Though I don't think it's the most likely scenario.

[johndunne]

Well put. I'd add that theories can produce predictions, and if the predictions can be proved experimentally, all the better for the theory.

[anonymous_sorry]

Or perhaps even stronger - theories _should_ produce predictions.

[mrguyorama]

Any theory that cannot make a prediction isn't a theory but rather a data compression method.

[gmadsen]

not really?, Dirac, Von Neumann, etc. taking a look at the history of the nobel prize, its typically the case that something is predicted by theory, then once an experiment is able to verify, then prizes are given out.

[BlueTemplar]

What universities ? They don't have labs ?

[psychoslave]

Einstein did experimented a lot too, even if it’s generally not as highlighted as his thought experiments.

https://www.amazon.com/Practical-Einstein-Experiments-Patent...

[tdehnel]

Not possible. Theory comes first. Scientists need a theory first to know which experiments to do.

Any interesting experimental results are interesting precisely because they do not conform to the existing theories.

Edit: here's a decent writeup to show what I mean https://en.wikipedia.org/wiki/Theory-ladenness

[sdenton4]

Hope the air's good in the tower...

Good science has often proceeded from 'weird' observations, leading to experiments trying to isolate the weirdness. The weird is necessarily outside the zone of existing theory, and requires experimentation to recreate the effect reliably and quantifiably. Once that's done, you can iterate on conjectures and experiments to try to get to the bottom of what's going on.

In reality, I suspect that the divide between theorists and experimentalists is really only a fundamental physics thing. And physics has been more-or-less at an impasse for the last thirty years, so that there are relatively few experiments worth running, and the bulk of the theorists are just (making stuff/rebranding as mathematicians) up because they don't know what else to do.

In other areas, there's so much weird still untouched that you don't get the same division of labor. Take a look at CRISPR - there was a lot of bench work and curiosity-driven exploration involved, simply because there was no theory describing what they were discovering... Or machine learning - the theory is still quite tenuous and mainly follows the experimental results.

[tdehnel]

To consider an observation “weird” you need a theory about what a normal observation looks like.

Additionally, all observations require baked in theories about the observation being accurate.

Btw, I didn’t appreciate your insulting first sentence.

Edit: here's a decent writeup to show what I mean https://en.wikipedia.org/wiki/Theory-ladenness

[sdenton4]

If we are sufficiently expansive in our definition of what theory is, then sure, there is no experiment without theory, and experimentalists only discover things by being secretly theorists who occasionally stop thinking long enough to get their hands dirty. But I don't think this is typically what is meant by 'theory.'

CRISPR is again a good example - the initial 'weirdness' was an observation of lots of long-ish palindromic subsequences in bacterial DNA. To my knowledge, there was no pre-existing theory on the preponderance of palindromic subsequences in DNA.

Now, we could say that these researchers were proceeding from a theory that there's no easily discernable combinatorial macroscopic structure in DNA sequences, but I think that this would stretch the idea of 'theory' beyond common usage or even usefulness: There's no theorem or axioms of DNA sequencing being violated here.

In fact, I would expect that weirdness is a good sign of missing theory - a repeatable observation which is unsupported by existing theory. For perturbations in the orbit of Mercury, we see an explicit violation of Newtonian mechanics, but in many other cases (like CRISPR and palindromic subsequences) we have observations of structure in areas where theory simply does not yet exist.

(Apologies for the harsh first sentence; FWIW, your comment seemed belittling of the work of a large fraction of important scientists.)

[tdehnel]

i see what you mean, i suppose my concept of “explanation” could be closer to what you mean by theory

[blueprint]

how exactly has physics been at an impasse? this trope is so common now. did you hear it from Weinstein? we've had a metric ton of mindblowing theory emerge in the past 30 years. do you even study physics?

[sdenton4]

I was careful to say fundamental physics - there's great stuff happening in materials science.

We are in a world where exponentially growing amounts of energy are needed to confirm/deny an increasingly small slice of the standard model. We still don't know what dark matter or energy is (almost literal holes in theory), we haven't figured out how to scale up quantum computers, and we don't have scalable fusion reactors. The cost of progress is growing and the rewards are diminishing; I call that an impasse.

(Arguably we could call the ongoing advances in materials science cases of applied quantum mechanics, with some blurry line with fundamental physics.)

You are right that I'm not a physicist - I'm trained as a mathematician, and these days work in the intersection of machine learning, acoustics, and ecology. Having looked around a lot with impact-colored glasses, I don't see the argument for fundamental physics, but am happy to be wrong.

[blueprint]

none of the things you listed have anything to do with fundamental or theoretical physics. It also seems that you don't know about some of the recent advances. we don't need to deny the standard model. Nature has already shown us proof that it's not sufficient. but the standard model is not actually fundamental physics. It's basically just a mash up of a bunch of aspects of particle systems. There's, again, a ton of work on alternatives. and I would have to disagree, it would seem that we do know how to scale, quantum computers, but we're still doing the material science for it. Physics is probably far more advanced than almost anyone really realizes... and that is the bigger issue. We are still working out what advances to pay attention to. Yet what we do know about , if the engineering work were done, is enough to get us "free" energy, for example.

[mrguyorama]

Call me when someone finally schedules a test to possibly disprove string theory. It's been stuck in "The math works but we haven't solidified it" for like decades. It makes testable predictions right?

[blueprint]

i'm not talking about string theory anyway

[ethbr0]

Comment would have stood on its own, without that first sentence.

[ajuc]

We discovered first "crystal diodes" in 1870s. There was a whole idustry doing "magic needle+crystal diodes" without understanding why they work. They just knew you had to touch a special spot on the crystal with a needle. Some spots worked better than others. It was art not science.

We had the first theory on how they operate in 1920s. Eventually we got transistors from that theory in 1950s.

[klabb3]

> Not possible. Theory comes first.

I used to think this way but it’s significantly more liberating and true to see theory as plausible explanations. Theories are very often afterthoughts, where the utility is prediction of new events, or narrowing the search space of experiments. A theory explaining past events only is not really a theory at all. But you can narrow search space in many ways without elaborate theories. For instance, you may assume that since birds can fly, if we make a machine that looks similar, we might be able to fly as well. Terribly simplistic theory, but nevertheless a good starting point.

[tdehnel]

Calling theories useful if they predict future events is called Instrumentalism and it’s a dead end if your goal is to understand reality.

“The magician will pull a rabbit from the hat” might reliably predict what happens, and be testable, but it is not a good scientific explanation because it doesn’t help anyone understand how the trick works.

[davidivadavid]

While correct you're being downvoted because you're equivocating on the term "theory." People are assuming you're talking about a fully baked, documented theory couched in precise language and equations. What you're actually talking about is tacit assumptions / models that are built in observations.

"Theory comes first" is particularly unfortunate wording because it seems to imply the naive point of view that people first come up with a thorough mathematical model of some phenomenon before testing and validating it with experiments. That's obviously not how science has worked throughout history.

Saying "all observations/measurements imply an underlying theory" would have been maybe more accurate.

[imtringued]

You are assuming that we have thought of all theories that an experiment can confirm, before conducting the experiment. Considering the large search space, that would be quite surprising.

[tdehnel]

Can you tell me why a scientist would decide to do a certain experiment without a theory about what to test and why? Why that experiment and not any of infinite others?

[robwwilliams]

We do this much of the time in genetics, genomics, and neuroscience.

New technology; Large set of samples; Quantify x<1> … x<n> mRNAs, proteins, lipids, metabolites; Estimate how units stick together (statistically or literally); Develop a “theory” of what units and groups of units interact to account for and predict higher order phenotypes (risk of neurodegeneration; lifespan).

Data take precedence. Mini-theories of molecular and cellular causality are assembled with some basic brain power and yes—-a dollop of theory and priors—-on the back of a massive pool of well structured data.

Exploratory biology of this type is/was insulted using the terms “fishing” or “mere description” but with current high throughput and high content technologies should be considered research “trawling” and factory-level science; not a cottage industry of small labs. This new style of science can be highly effective in biology and in astronomy as we are learning from Webb.

But it bugs the hell out of some classically trained reductionists who demand that clear hypotheses should drive science forward.

Much of the progress in modern biology falls into this alternative almost hypothesis-free style. I would say “story-free” style of science. Too damn many story-tellers.

[tdehnel]

I wouldn't disparage that method of discovery at all. But some of the theories required to do it are:

- Many brute-force combinations of these specific organic molecules will yield interesting results (this is the main hypothesis of the experiment)

- We don't need to include the molecules that we haven't included

- We have good criteria for determining what results are interesting

- Our instruments/methods of phenotype prediction are well-understood and working as expected

Once the experiment is run, the observations are meaningless unless interpreted in the context of whatever prevailing theories the scientists have in mind.

[ajuc]

Because he already has a sample and it doesn't hurt to check other properties of it.

[tdehnel]

Which properties to check? Why that sample and not countless others? Why not spend the rest of his life checking different properties of the one sample?

[ajuc]

Whatever suits your fancy.

> Why not spend the rest of his life checking different properties of the one sample?

The material in question is called LK-99 because it was first produced in 1999. Why not, indeed :) ?

[jack_riminton]

The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka!” (I found it!) but “That’s funny …” — Isaac Asimov

[tdehnel]

“That’s funny…” implies you had a pre existing theory about how it was supposed to work.

[SanderNL]

Ah, I see. You mean we need some (implicit) theory to even detect what is interesting about a result.

Let’s say you measure a particle going c*2. Without theory you wouldn’t even notice this.

Is that your point?

[jack_riminton]

Not really. I don’t have my own theories on how most things work but I can recognise patterns that fall out of the norm

[tdehnel]

so your theory is “things have predictable patterns” or “the future will be like the past”

[overboard2]

You don't need a preexisting theory though, you just need to lack one that explains the results of an experiment

[tdehnel]

Why would someone do an experiment without a theory to test? How would they know that a particular result was interesting without a theory about the expected result?

[foodjinn]

Because rubbing your chin doesn't accomplish anything, messing around with stuff does.

[__sy__]

For the wetter sciences, the answer is somewhere in between. We usually have somewhat of an understanding of the prior theory. We then throw a lot of stuff at the wall…. Oh, this sticks? Eh, why? And then usually some more clever person will come up with reasons for why X works beyond what we already knew. A few years later, the field reaches some sort of consensus around one of the hypothesis.

[tdehnel]

The “oh this sticks” part is only ever interesting precisely because it doesn’t conform to the pre existing theory.

You cannot even make an observation without theories baked in:

- Normally these things do X

- My senses or instruments are detecting reality accurately

- This thing I’m measuring will result in something interesting etc

[s1artibartfast]

This is only true from the perspective of developing new theories for why are things happen. Of course new theories arise from other theories.

It's not true for Discovery in general. New phenomenon can be created and observed without any theory for why they occur, either before or after observation

[tdehnel]

You can't even observe something without theories. For example:

- I need to observe here and not anywhere else

- I can reliably interpret my senses/the instrumentation is working correctly

- Objects of this type normally behave in X way, because of Y

- etc

Think of it a different way. You can come up with a theory without any observation whatsoever. Black holes, for example, were conjectured well before they were observed.

[s1artibartfast]

Well sure, that's the point that you're trying to make. I thought anything most people felt you were claiming you need a theory related to the novel Discovery or convention. That is to say how it might work and what the expected outcomes are. I can't go to the lab without a theory that my car can get me there. That doesn't mean I have a hypothesis for what will happen when I mix two substances in the lab.

Thinking that something will either happen or not happen if I mix those two substances is not a theory.

[mrguyorama]

Consider the story of an artificial sweetener being discovered by a chemist who didn't wash their hands properly and the bread they ate that night was sweet.

Are you suggesting this was only possible because they had a "theory" that bread is not sweet?

[robinduckett]

So before Isaac Newton, apples would float up off the tree when they fell?

[tdehnel]

The pre-existing theory Newton needed was “things move when other things act on them.” Without that, the observation would be meaningless.

Seeing the apple fall told Newton “some unseen force is acting on the apple.”

Incidentally, this was wrong. There is no “force”, only a warping of space time.