[jmull]

There's just no way for Apple to maintain and run comprehensive test suites for all the different software platforms out there, even "popular" ones.

That's why they release betas early -- that gives each project an opportunity to run their own test suites, however comprehensive they may be.

It's a little hard to hold Apple responsible when there are a lot of app teams in a better position to catch this than Apple, and apparently none of them did.

(Maybe it was a late change in Tahoe? Still, no one found it in the RC either it seems.)

[hu3]

What's stopping Apple from running automated performance regression tests for popular applications? They certainly have the resources.

[philistine]

Here’s the kicker: does it matter if culturally they’re never going to let the data influence them?

[eviks]

Yes, because known information has a chance you change that culture. It is not a static system

[hu3]

You have a point.

Why measure if it's going to be ignored.

[eviks]

> all the different software platforms out there

Yes, but this is a straw man

> even "popular" ones.

No, this is possible, but again, just drain the bathwater of "all, everything, comprehensive 100%" while blocking the baby of glaringly obvious system-wide visible bugs any good 9x% testing would've caught.

[jmull]

> glaringly obvious system-wide visible bugs

Yet the electron team didn't catch it, nor any of the many apps that use electron, despite beta access. Hm...

[eviks]

Why would you expect smaller teams do proper testing instead of sharing your approach of rationalizing not doing it? You're not alone!