[edmundsauto]

Looking for “why” is often a path to becoming more fragile and/or frustrated.

I do a lot of split tests (a/b testing). People always want to know why a treatment worked. I believe this is human nature.

However, the answer to any specific situation is generally unknowable. I may have only changed one color and gotten way different results, but there could be a billion or a trillion situation dependent factors that cause the situation to happen. It is pure chaos.

But our brains latch onto our cognitive biases to scaffold a reason, such as “because people find blue more reassuring than red.”

This is not generalizable, but more importantly: it probably doesn’t matter in order to resolve the decision that was the reason for the test.

Since then, I have accepted that seeking the “why” is generally a fools errand, unless you are doing pure science in a controlled, closed-input system.

That’s the beauty of not asking why. It’s a competitive advantage.

[mindslight]

What you're finding with A/B testing is local maximums. Which might make sense when your sole goal is squeezing a few more pennies out of a mundane business. But generalizing this to all of existence is sorely mistaken.

Asking "why" is the act of cognition. Discerning structure, compression, building a model (science). Then applying that model to new territory (ie engineering), to achieve better gains than undirected walk. Eschewing this is basically nihilism, and it's far too common these days.

[edmundsauto]

Asking why can lead down two paths. In a system where it's possible to understand the "why" (often overlaps with "hard" sciences), it is crucial.

In fields where the inputs are unknown, obscured, non-linear, and sometimes non-deterministic: asking "why" is going to create cognitive biases that are very difficult to break.

- Why did the Soviet Union fall?

- Why did Trump lose in 2020?

- Why did I eat a cheeseburger when I'm trying to lose weight?

Those all turn into narratives that may or may not be correct, but are usually self-reinforcing. Confirmation bias also means we evaluate the "why" question as follows: "given what I know about this situation, and how I think things generally work, can I see this singular narrative as being true?" That leads to blindness of other possible interpretations, as well as other cognitive dysfunction.

To be a little cheeky - that is why commentators on politics, economics, social sciences, psychology... are full of shit.

Asking why may be the act of cognition, as you say - but my take is that the act of cognition itself will not lead to understanding the truth, or better decisions. Making up fairy tails is also cognition but does not help us build mental models to better understand the world around us.