[reaperducer]

if I were in charge of a major OS, and I having one of the major app frameworks used on my OS tested on for my annual upgrade

Each program contains its own version of Electron. How is Apple going to know the version of Electron your particular version of your particular app that you installed on one particular date perhaps years in the past works?

n apps in the world × n versions of Electron

The problem isn't Apple. It's that the developer of your program is using an outdated version of Electron, or it's put out an update and you haven't updated.

[buildfocus]

They only needed to test with the latest Electron at the time of release (or indeed, any chance version - they're all affected - but latest is a reasonable baseline). If they had, they would've seen this.

There are patches out now, but only after Apple released the OS to the entire team world and people reported the issue to the Electron team.

Imo, Electron is sufficiently popular that somebody should test at least one Electron app on a major new OS version sometime before releasing it as done! Any app would've worked, and there's plenty of popular ones, as this post shows.

[jmull]

There's just no way for Apple to maintain and run comprehensive test suites for all the different software platforms out there, even "popular" ones.

That's why they release betas early -- that gives each project an opportunity to run their own test suites, however comprehensive they may be.

It's a little hard to hold Apple responsible when there are a lot of app teams in a better position to catch this than Apple, and apparently none of them did.

(Maybe it was a late change in Tahoe? Still, no one found it in the RC either it seems.)

[hu3]

What's stopping Apple from running automated performance regression tests for popular applications? They certainly have the resources.

[philistine]

Here’s the kicker: does it matter if culturally they’re never going to let the data influence them?

[eviks]

Yes, because known information has a chance you change that culture. It is not a static system

[hu3]

You have a point.

Why measure if it's going to be ignored.

[eviks]

> all the different software platforms out there

Yes, but this is a straw man

> even "popular" ones.

No, this is possible, but again, just drain the bathwater of "all, everything, comprehensive 100%" while blocking the baby of glaringly obvious system-wide visible bugs any good 9x% testing would've caught.

[jmull]

> glaringly obvious system-wide visible bugs

Yet the electron team didn't catch it, nor any of the many apps that use electron, despite beta access. Hm...

[eviks]

Why would you expect smaller teams do proper testing instead of sharing your approach of rationalizing not doing it? You're not alone!