[oersted]

I'm reticent of falling into this endless conversation again, but it's a bit addictive.

I believe the point is that the Standard Model is still built on Quantum Field Theory, but looking into what's really going on under quantum mechanics is a bit taboo and very few people are spending time on it.

Frankly, I acknowledge that I am an outsider and that surely my perspective is far too naive and simplistic, I'm sure the reality in the field is far more nuanced. But still, I'm in good company when I question what's going on, like Einstein and many other prominent physicists have.

It is true that the end of the conversation was just "shut up and calculate", which is wise on some level, but it also implies that the mathematical model is the end-truth and you should unquestionably just build on it. There is a consensus that the mathematical model of quantum theory is really what is mechanistically happening, and that the wave function is a real physical thing.

But one can also argue that it is merely a probabilistic theory that, yes, describes the probability distributions of what will happen extremely accurately, but still is a partial answer and it doesn't actually predict what will happen. In every other context, we use probability to describe processes that we don't have a complete model for. It is a bit presumptuous to think that in quantum mechanics probabilities are somehow fundamental and not emerging from more deterministic underlying phenomena.

You can have a theory that says that there's an equal likelyhood of getting heads or tails on a coin, and that theory can reflect the statistics extremely accurately. But that is obviously not the end of the story, there are deeper mechanics determining if you get heads or tails at each given time, it is not fundamentally random, you just don't fully understand what's going on yet.

[HappMacDonald]

But the real litmus test is and always has been "can a theory predict the results of empirical measurements (such as controlled experiments) more accurately than its competitors".

The Danish-interpretation quantum physicists might urge you to shut up and calculate, but what we lack is any model capable of both describing why and how the wave function collapses and simultaneously offering more accurate predictions than QM does. The most well known low hanging fruit being: describing events in noticeably curved spacetime (or put in other words, in situations where the effects of gravity are non-trivial) because we don't even need a better model to know that QM breaks there, QM itself will happily tell us that much by piling on infinities and singularities that can no longer be canceled via renormalization.

So the take home is less "shut up and calculate" and more "don't waste their time with speculation about underlying mechanics until it can also offer more accurate predictions, and until then redirect your energies back to calculating: making use out of the tool we already have at least".

If nothing else, familiarity with the tool that does work up to a certain standard is more likely to lead someone to the next big step than hanging back in the wings of Newton and Aristotle with layperson intuitions about macroscopic objects in terrestrial gravity.

[oersted]

Of course, but such a theory doesn't pop out of nowhere. Like in any technical field, new paradigms are always inferior to the incumbent on key metrics, and need room to be nurtured for long periods.

My perception is that there's a cultural issue suppressing the investigation of the causes of quantum mechanics, making it much harder for a better theory to flourish.

It's also really-really hard. If you take my earlier example, the probabilistic theory of the coin-flip is worlds apart in terms of complexity from a real understanding of the forces acting on the coin when it's flipped. It's a massive gap to jump over, you have to come at it from another easier and longer path, like it was done with classical physics.

[exmadscientist]

Yes, this, exactly.

The moment someone has a theory that's experimentally testable somehow (somehow possible, anyway), there will be plenty of interest. There has never been such a theory to date.

Until that arrives, it's all just hot air.

[inigyou]

A new theory could also explain known observations using fewer assumptions. The standard model has a lot of odd things baked into it - why these specific quarks and leptons? Why no right-handed neutrinos? Most simply, why does each particle have the rest mass it does? And why is there a Higgs field that's very similar to the field of condensation of Cooper pairs in a normal superconductor, but with more dimensions?

Solving just one "why" question might earn you a Nobel prize.