[arkades]

Sample error.

When you had 100 users, it was feasible to get detailed feedback on 90 out of 100 users. When you have 1.5m users, you cannot get detailed feedback on 1.35m users. Dealing with much smaller fractions of your user base, sampling error grows into larger and larger an issue. That's not to say there aren't wonderful statisticians working on sampling issues, but fundamentally it's not the same as actually interacting with the vast majority of your user base.