Hacker News new | ask | show | jobs
by jrdorn 1572 days ago
Yeah, Statsig is building a really cool product. There's definitely a gap in GrowthBook right now, like you mentioned, between pure feature flags and experiments. We're working on some light-weight event tracking features to try and bridge that gap. So you'll be able to see how many people are using the features, which values they are getting, and easily integrate with tools like DataDog and NewRelic for performance monitoring.
1 comments

What's particularly interesting is Statsig essentially separated the concept of "A/B test" from experiment. Compare https://statsig.com/features/feature-gates and https://statsig.com/features/experiments

I think this isn't just marketing; I think this reflects a deeper understanding that you can do A/B testing on top of a feature flag and not consider it an experiment. It's an innovation that reduces inter-organizational strife on the decision to launch a feature to all users, i.e., ship.

If everything with automated metric analysis is considered an experiment, then the shipping decision only be made after analysis is performed. That leads to tension when Product Managers have already decided to ship and will only rollback if something hurts their metric. Data Scientists will then question why their "experiment" is only running for a day, and then question the decision to ship entirely. You can imagine how many meetings a misdefinition spawns.

The sooner you address these definitions, I think the happier you'll find your enterprise users.