Every single thing for the past 10 years has had (opt-out, which most people didn't) telemetry and that correlates with a decline in quality, not improvement.
- Use of analytics tends to replace user trials/interviews entirely, trading away rich signals for weaker ones
- Analytics can be used to justify otherwise unpopular or ill-advised changes
- When combined with certain changes (e.g. making features harder to access), the numbers can be “steered” in a particular direction to favor a particular outcome and better enable the last point (“Looks like nobody’s using that thing we hid behind an obscure feature flag! Guess we’re safe to remove it entirely now!”).
In theory telemetry/analytics have strong potential for improving software quality, but more often than not they’re just massaged and misused by product managers bent on pushing the software a particular direction.
- Use of analytics tends to replace user trials/interviews entirely, trading away rich signals for weaker ones
- Analytics can be used to justify otherwise unpopular or ill-advised changes
- When combined with certain changes (e.g. making features harder to access), the numbers can be “steered” in a particular direction to favor a particular outcome and better enable the last point (“Looks like nobody’s using that thing we hid behind an obscure feature flag! Guess we’re safe to remove it entirely now!”).
In theory telemetry/analytics have strong potential for improving software quality, but more often than not they’re just massaged and misused by product managers bent on pushing the software a particular direction.