[adamckay]

But the next largest legitimate repo is going to be something like 20M commits shy of that, so it seems like it's excellent that GH engineers only just started to care.

It's entirely possible that such a load test has been considered, but deemed non-realistic so not prioritised for some time. If I were running the QA team I'd be annoyed if time were spent on abusive destructive testing than realistic testing that real-world users may experience, especially because load testing like this would have to be on an identical environment to PROD so rather expensive.

It reminds me of that old QA joke:

A QA engineer walks into a bar and orders a beer. She orders 2 beers.

She orders 0 beers.

She orders -1 beers.

She orders a lizard.

She orders a NULLPTR.

She tries to leave without paying.

Satisfied, she declares the bar ready for business. The first customer comes in an orders a beer. They finish their drink, and then ask where the bathroom is.

The bar explodes.

[blowski]

In your joke, QA was only doing exploratory testing. Somebody - perhaps the builders, bar staff, or QA - should have also been doing integration testing for key user stories, and the system has failed because nobody ensured that was happening.

GitHub hasn't failed here - it continued to perform at normal levels for other users, so far as I can see, and they had an upstream process which caught the issue without the system failing. Maybe some exploratory testing had previously identified where that process should kick in, but without having an automated process since it was so unlikely to happen.