[chambers]

What's particularly interesting is Statsig essentially separated the concept of "A/B test" from experiment. Compare https://statsig.com/features/feature-gates and https://statsig.com/features/experiments

I think this isn't just marketing; I think this reflects a deeper understanding that you can do A/B testing on top of a feature flag and not consider it an experiment. It's an innovation that reduces inter-organizational strife on the decision to launch a feature to all users, i.e., ship.

If everything with automated metric analysis is considered an experiment, then the shipping decision only be made after analysis is performed. That leads to tension when Product Managers have already decided to ship and will only rollback if something hurts their metric. Data Scientists will then question why their "experiment" is only running for a day, and then question the decision to ship entirely. You can imagine how many meetings a misdefinition spawns.

The sooner you address these definitions, I think the happier you'll find your enterprise users.