[michaelt]

Here's a different example: A new driver improves game performance for 99.9% of customers, and disables graphics output for 0.1% of customers. The bug could have been caught with a few days of automated testing in a test lab with lots of different cards and system configurations, but that's slow and costs a lot of money to maintain - chucking it over the wall to consumers is quicker and cheaper.

Seems to me getting the poorly tested drivers earlier isn't much of a deal for users. After all, just because I'm in the 99.9% this week, doesn't mean I will be next week.

[Klathmon]

But in my experience there are a few more 9's on your "works" number for a specific issue.

Hardware-related problems are so fucking hard to debug in a lab. Many of the issues i've dealt with in the past only show up with pretty esoteric hardware mixtures, or are so specific that i'd be surprised if more than one person has that exact config on the planet.

A test lab might be able to get maybe a hundred different configurations. But in the wild, something like 60% of our users are on unique hardware configurations (as in nobody else has that exact set of hardware among our users).

Telemetry is pretty much the only real way you are going to be able to support a wide range of hardware when you start interacting with it on the lower levels, because buying and assembling all the different configurations is all but impossible, even if you prioritised it.