This gives you a distribution unrelated to active use, puts users in the same bucket (with the same number you’re going to have the same users in the first 10%) and links combinations together.
Often problems are more complex than they seem at first sight and I have found it’s a good approach to think “what am I missing” rather than “lots of people must be making very obviously bad decisions” and reach the latter conclusion only after more work. Usually I’ve missed something.
> This gives you a distribution unrelated to active use
Is that a problem?
> puts users in the same bucket (with the same number you’re going to have the same users in the first 10%) and links combinations together.
Are you trying to do a bunch of separate staged rollouts at the same time?
> Often problems are more complex than they seem at first sight and I have found it’s a good approach to think “what am I missing” rather than “lots of people must be making very obviously bad decisions” and reach the latter conclusion only after more work. Usually I’ve missed something.
They said it gets you 80% of the way there, and that seems fitting with the replies they got.
Yes! Are you totally missing some countries, did you set it to try and hit 20% but it’s actually 60% of traffic, does it happen to include some VIP type users, how does it impact the thing you’re trying to measure while you do this?
Do you give qa user ids in the right ranges to get applied and non applied views?
Even then you’re not able to get specific combinations.
> Are you trying to do a bunch of separate staged rollouts at the same time?
That would be very common yes, otherwise a staged rollout can be a big blocker.
But more than that it means there’s one user who always gets the beta testing of everything for example.
Assuming you want a random distribution and don't want to take any other attributes into account.
We're a small company but new feature release for big features is typically targeted at low risk users/customers first. That usually means a few attributes are taken into account (age, customer value, customer sentiment, which features they use)