[tmoertel]

One easy way to avoid this problem is to give each feature its own independent space of “dice rolls” by hashing the user ids with feature-specific constants before interpreting them as dice rolls:

    feature1_enabled = hash(user_id + "-feature1") / HASH_MAX < 0.3;  # 30% of users.
    feature2_enabled = hash(user_id + "-feature2") / HASH_MAX < 0.1;  # 10% of users.

[phreeza]

If you suspect some flag effects interact with each other (e.g. one flag increases button size by 10%, and the other decreases it by 10%) you can go one step further and define feature groups and hash by user-id + group-id and then assign non overlapping ranges to the flags.

[marcosdumay]

Or you can cut and redistribute the non-performing changes, increase the distribution of the well-performing ones, and let evolution take care of disentangle correlated behavior.

But if need that kind of analysis for usability A/B testing, you most be doing something very wrong on a previous step.

[groestl]

For reference: https://en.wikipedia.org/wiki/Consistent_hashing

[singron]

How is that related?

[groestl]

Consistent hashing (or more generally weighted rendezvous hashing) provides pseudorandom allocations with minimal reallocations when parameters change. This is exactly the properties desired for rolling out feature flags gradually over an input space like user IDs. It is a special case of a consistent hash function, where the possible assignments are just two (weighted) buckets, corresponding to the feature flag's value of true/false.