[Aaargh20318]

A problem with this approach could be that you're now only testing the feature with the kind of people who would sign up for an A/B test. This group may not be representative of your whole user-base.

[jfarmer]

So they'd need more robust experimental designs and statistical methods. They exist.

And unlike the university context, there’s a glut of data.

A basic technique: https://en.wikipedia.org/wiki/Inverse_probability_weighting

Or https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4384809