[pizzathyme]

I agree, I have worked at 5 separate tech companies and have conducted hundreds of statistically significant experiments, changing what people select by default. These methods have been effective at helping hundreds of millions of people improve choices. It’s common practice, think: pricing on Amazon, default tips on Uber, default purchase price in a video game, etc.

I don’t see how this would make me reinterpret all those successful results. Maybe I don’t understand what this is saying.

[efitz]

How much of this is “nudging” vs. “clearly explaining trade-offs”?

I’ve not done any rigorous research, but I’ve participated in projects that resulted in dramatic shifts towards customers choosing what the dev team thought was the “best” outcome, just by altering wording, or making “dangerous” choices harder (such as by requiring more clicks to enable).

[ad8e]

One interpretation is that it is very hard to extract value from the nudge literature. When reading research articles, one must estimate and adjust for biases. This adjustment shifts the p-values by unclear amounts. So a positive result may just be a fluke.

As for your own AB tests, you have seen the processes that go into them and do not need to adjust for unknown biases. So when they demonstrate a nudge effect, you can believe it.

[salawat]

All you've proven is that controlling the default is a viable means of making a decision for someone.

I assure you, Opt-outs are much less likely to happen when combined with not explaining there is a choice to be made in the first place.