[Eduard]

> That first set now has both features, and 56-FF have none. It turns out you can't draw meaningful conclusions when some users get every new feature and some get none at all.

00—1A: have feature flags A and B

1B-55: have feature flag A only

56-FF: have no feature flag

So the actual gotcha here is that there is no cohort for "feature flag B only", right?

This setup can actually be desirable if feature B depends on feature A.

[duskwuff]

> So the actual gotcha here is that there is no cohort for "feature flag B only", right?

And that, as you add more tests, user 00 will always get the test treatment for every test. If you're running a lot of tests which introduce experimental features or changes to workflows, user 00 is probably going to find the site a lot more chaotic and hard to understand than user FF, and that will skew your test results.

[dylanpyle]

> So the actual gotcha here is that there is no cohort for "feature flag B only", right?

Yep, exactly - by "that first set" I meant the 00-1A group, could have been clearer. Whatever the smallest rollout bucket is, that group is guaranteed to have every single feature.

This was quite a while ago, but I think the actual case we noticed this with was several features released to 50% of the userbase - so every single user either had all or none at once (unintentionally)